commit 2bec25728933077e04b1576495ce087a0c163a5e Author: MiFaZhan Date: Thu Oct 22 16:37:56 2026 +0800 chore: 初始开源发布 iotlcm-worker-svr：异步执行中枢（RabbitMQ 消费者 + 全量同步编排）。 - Go module：iotlcm-worker-svr → git.5iot.com/iotlcm-public/iotlcm-worker-svr（214 文件） - 凭据与内部标识脱敏：MySQL/Redis 凭据、私网与生产 IP、内网域名、 库名 xiujie_admin → iotlcm、哨兵名占位 - 159 处真实手机号 → 合法 11 位编造号（保留号段与后 4 位，维持互异， 不破坏「校验 MSISDN 格式」等测试断言） - 移除内部资产：CI 触发器、Jenkins 触发测试文档、预演门禁与镜像 digest 记录 - Dockerfile 改用公共基础镜像（golang:1.25-alpine → alpine:3.23）， 移除写死的 git 凭据 - README 重写（架构/全量同步模型/配置/已知失败），补 Apache-2.0 LICENSE，LF 行尾 + .gitattributes 已知：3 个测试包在源仓同一提交上同样失败（client/upstream、 service/allocate、service/inbound），属既有缺陷，本仓未引入亦未修复。 diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..5837541 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,13 @@ +# 统一使用 LF，避免跨平台协作时产生整文件 diff +* text=auto eol=lf + +# 二进制文件不做行尾转换 +*.png binary +*.jpg binary +*.gif binary +*.ico binary +*.woff binary +*.woff2 binary +*.ttf binary +*.pdf binary +*.exe binary diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..457a43b --- /dev/null +++ b/.gitignore @@ -0,0 +1,23 @@ +.buildpath +.hgignore.swp +.project +.orig +.swp +.idea/ +.settings/ +.vscode/ +bin/ +**/.DS_Store +gf +main +*.exe +output/ +manifest/output/ +temp/ +temp.yaml +*.log + +# 本地配置文件（各人的数据库/上游地址不同，不纳入版本控制） +config/config.yaml +.keep-if-necessary +.gitkeep diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..261eeb9 --- /dev/null +++ b/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..cd0800f --- /dev/null +++ b/Makefile @@ -0,0 +1,7 @@ +ROOT_DIR = $(shell pwd) +NAMESPACE = "default" +DEPLOY_NAME = "iotlcm-worker-svr" +DOCKER_NAME = "iotlcm-worker-svr" + +include ./hack/hack-cli.mk +include ./hack/hack.mk diff --git a/README.md b/README.md new file mode 100644 index 0000000..c89af50 --- /dev/null +++ b/README.md @@ -0,0 +1,120 @@ +# iotlcm-worker-svr + +iotLCM 物联网卡全生命周期管理平台的**异步执行中枢** —— RabbitMQ 消费者 + 全量同步编排器。 + +平台的大部分耗时操作（批量开停复机、续费变更、流量/状态/IMEI 全量同步、达量断网、工单自动关闭）都在本服务里异步执行。 + +## 特性 + +- **多队列消费者**：注册业务消费者处理重试队列、批次执行、工单等 +- **全量同步编排**：卡状态 / 本月流量 / 日流量 / 月流量历史 / IMEI 五条全量同步链路，统一的 Producer（扫描 + 供料）→ Consumer（领取 + 执行）模型 +- **通道级并发与限速**：按通道模板配置 QPS，按通道隔离并发度 +- **多轮执行**：失败卡进入下一轮重试，单轮上限与轮次上限可配 +- **运行归档**：每轮执行产出终态 JSON 归档，供通知与排障使用 +- **可观测**：阶段耗时、库存水位、缺料观测等遥测写入归档 + +## 技术栈 + +- Go 1.25 + GoFrame v2 +- RabbitMQ（`github.com/rabbitmq/amqp091-go`） +- MySQL（业务库）、Redis（任务队列与锁）、TDengine（用量时序） +- 支付宝（余额充值）、excelize（Excel 导入导出） + +## 快速开始 + +```bash +go mod tidy +go run . +``` + +## 配置 + +配置文件在 `config/config_default.yaml`（默认配置）与 `config/config.yaml`（部署时覆盖）。**公开版里的凭据与地址全部是占位符**： + +| 配置项 | 占位 | 说明 | +| --- | --- | --- | +| `database.default.link` | `${DB_USER}:${DB_PASSWORD}@tcp(${DB_HOST}:3306)` | 业务库 | +| `redis.default.address` | `${REDIS_ADDR}` | 任务队列 | +| `redis.default.pass` | `${REDIS_PASSWORD}` | 缓存口令 | + +上游凭据（AppKey / AppSecret）**不写在本仓**，存放在通道表 `t_sim_channels` 与通道模板表里，由平台写入。 + +## 目录结构 + +```text +iotlcm-worker-svr/ +├── internal/ +│ ├── cmd/ # 启动装配 +│ ├── consumer/ +│ │ ├── rabbitmq/ # RabbitMQ 消费者（批次、重试、工单等） +│ │ └── zset/ # 全量同步消费者（ZSet 任务队列） +│ ├── producer/ +│ │ ├── simstatus/ # 卡状态全量同步 +│ │ ├── monthlyusage/ # 本月流量全量同步 +│ │ ├── dailyusage/ # 日流量全量同步 +│ │ ├── monthlyusagehistory/ # 月流量历史同步 +│ │ └── imei/ # IMEI 全量同步 +│ ├── fullsync/ # 全量同步运行时、归档、契约、HTTP 诊断 +│ ├── service/ # 各业务动作执行器 +│ ├── repository/ # 数据访问 +│ ├── queue/zset/ # 任务队列与锁 +│ └── library/rabbitmq/ # MQ 封装 +├── config/ # 配置 +├── build/ # Dockerfile 与镜像内配置 +├── manifest/deploy/kustomize/ # K8s 部署清单 +└── manifest/docker/ # compose 与启动脚本 +``` + +## 全量同步模型 + +```text +Producer（扫描卡 → 按通道分片入队） + │ + ▼ + ZSet 任务队列（Redis） + │ + ▼ +Consumer（按通道 QPS 领取 → 调上游 → 写库 → 结算） + │ + ▼ + 运行归档（终态 JSON，供 Bot 通知与排障） +``` + +关键设计： + +- **通道隔离**：每个通道独立并发与 QPS，一个慢通道不拖垮其他通道 +- **供料与消费解耦**：Producer 只负责入队，Consumer 按节奏领取，避免上游被瞬时打满 +- **多轮重试**：单卡失败进入下一轮，轮次上限与单轮上限均可配 +- **归档去重**：按 `runId` 标识一次执行，Bot 据此去重推送 + +## 容器化构建 + +```bash +docker build -f build/Dockerfile -t iotlcm-worker-svr:latest . +``` + +镜像内不含任何凭据；`config/config.yaml` 以只读方式挂载覆盖。 + +## 测试 + +```bash +go test ./... +``` + +测试使用本地替身，不访问真实运营商接口。 + +> **已知失败（与上游主分支一致，非本仓引入）**： +> `TestValidateUpstreamBasicInfoAcceptsMatchingIdentifiers`、 +> `TestConfiguredInboundBasicInfoClientUsesDedicatedEndpoint`、 +> `TestAllocateProcessingRetryAfterHasLargeBatchDefault`。 +> 这三项在源仓库同一提交上同样失败，属既有缺陷。 + +## 相关项目 + +- [iotlcm-bot](https://git.5iot.com/iotlcm-public/iotlcm-bot) —— 读取归档 JSON 并推送飞书卡片 +- [iotlcm-upstream-chan](https://git.5iot.com/iotlcm-public/iotlcm-upstream-chan) —— 运营商上游适配 +- [iotlcm-admin-svr](https://git.5iot.com/iotlcm-public/iotlcm-admin-svr) —— 管理后台（发起批次与定时任务） + +## License + +[Apache License 2.0](./LICENSE) diff --git a/SIM_STATUS_RUN_ACCOUNTING.md b/SIM_STATUS_RUN_ACCOUNTING.md new file mode 100644 index 0000000..2e5d7b8 --- /dev/null +++ b/SIM_STATUS_RUN_ACCOUNTING.md @@ -0,0 +1,34 @@ +# 卡状态同步 Run 计数与收尾 + +状态同步此前创建了 Run 元数据，但使用渠道级 `EnqueueBatch/Enqueue` 和 `AckBatch`，这些操作不会维护 Run 计数，也不会调用 Run 完成判定。任务实际处理结束后，元数据因此可能长期停在 `running`，成功、失败计数均为零。 + +## 新任务行为 + +全量和单卡状态同步均改用 Run 队列。`RunQueueEnabled` 控制 Run 消费，`RoundRetryEnabled` 独立控制多轮重试；状态同步固定单轮，流量同步继续保留原来的多轮策略。 + +- 入队按稳定 Task ID 去重；`activeCount` 包含排队及租约中的任务，背压不能再次加上 `inflightCount`。 +- 成功结果先写 MySQL，再凭租约令牌原子确认、累加成功数。永久业务失败记入失败数及失败记录；数据库失败和可恢复网络失败保留任务，等待租约恢复。 +- 电信渠道受限时，先确认同批已落库的成功结果，再凭触发任务的有效租约封禁该 Run 的渠道。其余已入队任务记为 `channel_blocked` 终态失败，保留逐卡记录；尚未入队而被拒绝的任务不计入已处理数量。此计数规则仅用于状态同步。 +- 只有 `producerDone=1`，且 `activeCount/inflightCount/nextRoundCount` 均为零，才写入 `completedAt`。无失败为 `completed`，存在失败为 `completed_with_failures`。周期性检查覆盖空 Run 和消费先结束的情形。 +- 元数据、任务状态和失败记录在收尾后保留 30 天。`sim_status_queue_round_batch` 是批次尝试日志，不能直接按其 `failed` 汇总终态失败；最终 Run 计数以元数据及 `sim_status_queue_run` 事件为准。 + +## 配置和后续上线步骤 + +使用 `simStatusQueueSync.redisKeyPrefix`，默认 `sim-status:`。生产者和消费者锁仍使用原键。 + +本次代码交付不部署、不修改旧 Run。后续上线应遵循以下顺序： + +1. 暂停状态同步的新触发入口（定时、手动及单卡），让旧 Worker 完成现有工作。核对两个状态 MQ 队列的 ready/unacked 为零、无在途 producer，以及旧命名空间所有渠道 ZSet 与租约为零。历史 `status=running` 本身不能作为仍有业务任务的判据。 +2. 使用新代码和 `redisKeyPrefix: "sim-status:"` 启动，保持锁键配置不变；恢复触发入口。 +3. 先运行小规模/单卡同步，再核验一次全量同步：MySQL 状态及同步时间符合成功结果，Run 计数持续变化，最终唯一入队数量（任务状态 Hash 的 HLEN）等于成功数加失败数，活动计数归零、终态与完成时间正确。查询 `sim-status:{}:meta`。 +4. 如需回滚，先暂停新触发并等待 v2 队列处理结束，再停新 Worker、恢复旧版本和旧配置；有未完成 v2 任务时应保留可消费 v2 的版本继续处理，不直接切回旧版遗留任务。 + +## 自动验证 + +```text +go test ./... +``` + +Redis 集成测试要求 PATH 存在 `redis-server`，或设置 `REDIS_SERVER_BINARY` 为可执行文件路径。每个测试启动独立的回环地址 Redis，使用临时目录且关闭持久化，退出后清理进程；不会读取应用 Redis 配置。缺少二进制时会明确 skip，验收时应确认这些测试实际执行。 + +覆盖成功/失败混合、空 Run、生产消费交错、重复入队/确认、租约失效及恢复、数据库和网络失败、渠道封禁及逐卡失败记录、多 Run 并行、单卡入口和背压；另回归流量五轮重试与原渠道封禁语义。 diff --git a/build/Dockerfile b/build/Dockerfile new file mode 100644 index 0000000..694305e --- /dev/null +++ b/build/Dockerfile @@ -0,0 +1,33 @@ +FROM golang:1.25-alpine AS builder + +LABEL stage=gobuilder + +ENV GOPROXY=https://goproxy.cn,direct +ENV TZ=Asia/Shanghai + +WORKDIR /build + +COPY go.mod go.sum ./ + +RUN go mod download + +COPY . . + +ARG BuildVersion +ARG BuildTime +ARG CommitID + +RUN go build -v -o /build/iotlcm-worker-svr -ldflags "-X main.BuildVersion=${BuildVersion} -X main.CommitID=${CommitID} -X main.BuildTime=${BuildTime}" + +FROM alpine:3.23 + +RUN apk add --no-cache ca-certificates tzdata + +COPY --from=builder /usr/share/zoneinfo/Asia/Shanghai /usr/share/zoneinfo/Asia/Shanghai +ENV TZ=Asia/Shanghai + +WORKDIR /app +COPY --from=builder /build/iotlcm-worker-svr /app/iotlcm-worker-svr +COPY --from=builder /build/build/build_config.yaml /app/config/config.yaml + +CMD ["./iotlcm-worker-svr"] diff --git a/build/build_config.yaml b/build/build_config.yaml new file mode 100644 index 0000000..35a2688 --- /dev/null +++ b/build/build_config.yaml @@ -0,0 +1,346 @@ +# config.yaml 配置文件 +# @Link https://github.com/xiujiecn/xiu-admin +# @Copyright Copyright (c) 2025 LiXiujie +# @Author Lxj +# @License https://github.com/xiujiecn/xiu-admin/blob/master/LICENSE + +# 系统配置 +system: + name: "IotLCM" + version: "1.0" + description: "IotLCM Server" + gfmode: "dev" + upload: + local: + path: "resource/upload" + +# 日志配置 +logger: + path: "/app/logs" # 日志文件路径。默认为空，表示关闭，仅输出到终端 + file: "{Y-m-d-H}.log" # 日志文件格式。默认为"{Y-m-d}.log" + prefix: "" # 日志内容输出前缀。默认为空 + level: "dev" # 日志输出级别 + timeFormat: "2006-01-02T15:04:05" # 自定义日志输出的时间格式，使用Golang标准的时间格式配置 + ctxKeys: [] # 自定义Context上下文变量名称，自动打印Context的变量到日志中。默认为空 + header: true # 是否打印日志的头信息。默认true + stdout: true # 日志是否同时输出到终端。默认true + rotateSize: 0 # 按照日志文件大小对文件进行滚动切分。默认为0，表示关闭滚动切分特性 + rotateExpire: 0 # 按照日志文件时间间隔对文件滚动切分。默认为0，表示关闭滚动切分特性 + rotateBackupLimit: 0 # 按照切分的文件数量清理切分文件，当滚动切分特性开启时有效。默认为0，表示不备份，切分则删除 + rotateBackupExpire: 0 # 按照切分的文件有效期清理切分文件，当滚动切分特性开启时有效。默认为0，表示不备份，切分则删除 + rotateBackupCompress: 0 # 滚动切分文件的压缩比（0-9）。默认为0，表示不压缩 + rotateCheckInterval: "1h" # 滚动切分的时间检测间隔，一般不需要设置。默认为1小时 + stdoutColorDisabled: false # 关闭终端的颜色打印。默认开启 + writerColorEnable: false # 日志文件是否带上颜色。默认false，表示不带颜色 + +# 数据库配置 +database: + logger: + path: "/app/logs/sql" + level: "all" + stdout: true + default: # 分组名称，可自定义，默认为default + link: "mysql:${DB_USER}:${DB_PASSWORD}@tcp(${DB_HOST}:3306)/db_iotlcm?charset=utf8mb4&parseTime=True&loc=Asia%2FShanghai" + debug: true #开启调试模式 + charset: "utf8mb4" # (可选)数据库编码(如: utf8mb4/utf8/gbk/gb2312)，一般设置为utf8mb4。默认为utf8。 + dryRun: false #ORM空跑(只读不写) + maxIdle: 30 #连接池最大闲置的连接数 + maxOpen: 100 #连接池最大打开的连接数 + maxLifetime: 120 #(单位秒)连接对象可重复使用的时间长度 + memorydb: + # link: "sqlite::@file(jieiotmem.db)?journal_mode=WAL" + link: "sqlite::@file(:memory:)" + # link: "" + debug: true + maxIdle: 1 + maxOpen: 1 + maxLifetime: 30 + +# 缓存配置 +cache: + adapter: "redis" # 缓存适配器，支持：file|redis + fileDir: "resource/cache" # 缓存文件目录 + +# Redis 配置 +redis: + # 单实例配置 + default: + mode: "single" #redis模式，支持：single|cluster|sentinel + sentinelMasterName: "iotlcm-master" #哨兵模式主节点名称 + address: 192.0.2.29:6379 + db: 0 + pass: "${REDIS_PASSWORD}" + minIdle: 5 #最小空闲连接数 + maxIdle: 30 #允许闲置的最大连接数 + maxActive: 5000 #最大活跃连接数 + idleTimeout: "60s" #连接最大空闲时间，使用时间字符串例如30s/2m/2d + ClientName: "SystemCache" + + +alipay: + appId: "${MOBILE_APP_ID}" + privateKey: "MIIEowIBAAKCAQEAg+4BpLO2Fw5GMbrihnugyQqiLBp2RhSbSsvjNyzaG10u900N1DcVlh8HNdcus5AGjfbiUwMQwv7Os+NnFrEZA3CtWLXH+uczHIpcMqJJhoVgA//4gf2TX9K4OSQ1pgJeAhTS9WWMEB7DQLb7A6xGjvBdGNbPAzucGhUmtG1tgZSVOcpWqN2h4dQxkZhfR+6m3AqWAPahpQXsNINHewetQ4mIG+b3+z/OknEll0RDRmw7OihOJ24B51Phy7lewl160LgLpLffyaQUUXcFdcNo+HvWqtIT0mH8WWHAgqfN/WVkMz/WVjE0rzbIVgmpQc7+hMbrcafTmeq0wdnVXrKS6QIDAQABAoIBAEagy1ZuMyRiqueCojTWyHv4altsCzd3Xq+LOWfXZihSGYuhP68FklPc5pJHUaQljazAp+dC891AZk5dhlnak4aI8vRT1OyYi39S/wrIYM5YZs+dYF/O7e/VsnQ800+9YrCAj3N6WqJiparmirkrqFZrjZt3nDmh1u8Ltb3P+1jr90MBEdS+BXaEQP/KGxFiKZyoWFn9B4l0aPFkPExF+hgtRpP3gxFcyljQKGVmSUIkT6V08F1oRm72NQOhcizCh9lX+5HYLTkUVQVg6FHwdZs/KeJcZu6Y829pNSNeJmPbR3VPUpTbvwb0qeEoc42T3H/C0xXUeTYizUpHmhJJqnUCgYEA3Y7Xz+MUk98zRJnlr2BDUc+Qnhv3wmSA5LCcRbxTNNBp4TXjnZknY6Ms9D+nJupL4nw6QSYkO6O+tAEmU6gaSDEHKI1uKqU0zOdbGIHNkMqka3rinQcMJNNRZ8T1FLCPFna7S1FNlxulZgWxjqxP/KvWOEpCWLCLzwZNk8fEd+MCgYEAmHBNbaMmT4WvLevY/OMv/5e5csnDFIcsNJKvFLZZMBLuKT/26YN2ryVUivY+TTyHL5zk4VybeauTdQHIvX6sNJoh2Ii6vthKfSufL4rkMob8PHiG33iHBx3VxAwiVXfu3JHVnbCPcXTOzsdRzT17X3rODK9t8319qfVjd0Gxi8MCgYEAqgNc0+ea+MXD35aAzA0AW5z9inZHruHUjqh/onubFeH//ZOOsHa/5RNoAZ1+tLwBszBEu417LFDAm/xt3rAWSuNkbNfqe8r+M2YtXybsCXmWpetMTdcEmJWX2BspPdEmW8SMgIVJFvVd/GXOFH9hebCa1+MT+l6vShO6H/ahaqUCgYAzon6nr4bPkvHWtTfH2VCt2//mIO+iaXMQz8rcUjP9s7vWXyNMz8oI47UyaNyLnwAPoa5po8vKN9L35EXR4jGEk9z5Er9m267HnVoLaZmIf+66sPzGf3YFdBavlIiguMJBFmjsRB0iijVvF1bfFX9PxTI8W8P7oTtKsaMihy8TrwKBgCgjA8DL8HJbRwZ+V/KWOh8rzO5cBUUH3ThIj6u7Wn4tMlscfi+KFKTXghjSORfIVx8v9v2aNZKwGZ547AOPRXhDF09cLAcCfl7uOrwsftziIeBQnrqtpIqcVTgRwGMIMBJEtQEks5N/E/d+dSL1YQKC76mVC8/RMf52WYKH1QCa" + alipayPublicKey: "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA8Df1EPFkA2xy8NgF0UdRYxsshslUi2XSGTG01oQKsrYNB+rhAw9MJdMlunVQxSrBZsw5N7htXGdiTceaLo/n8Wuibp0cRHdErJSp9GhP4pW2IR3FoCCPYlJlyiCUR/NInI3XMai0UdjgiPX03bvYLRAKE8lLiOSgxNagQmOFLmQYRYfFhGbxDe98XMg83YdSWvZKrlM+vzgk8/aPgq7BFLpbRKqLvd8OXXBHDCaWMSSGwCN0whhdEiwbIhSVgwTgkhWXUuNDmT741/+7D46djp8wX/84SDts0fWb63/in9ATTGtNivJX7HPDyqrmcN2iUi5xFHS9XwYBdT4yu/xFQwIDAQAB" + notifyUrl: "https://evolution-starless-municipal.ngrok-free.dev/api/v1/pay/alipay/notify" + returnUrl: "http://localhost:5666/enterpriseManagement/tBalanceDetail/tAgentBalance" + isProduction: false + +# RabbitMQ 配置（worker 核心依赖） +rabbitmq: + enabled: true + # 示例：amqp://guest:guest@127.0.0.1:5672/ + # dsn: "amqp://admin:admin@127.0.0.1:5672/" + dsn: "amqp://root:sdyz2026@192.0.2.29:5672/" + balanceRecharge: + retryDelay: "1m" # Alipay 查询失败后的延时重试时间 + maxRetry: 100 # 最大重试次数，超过后投递到 iotlcm.balance.recharge.dlq + batchExecute: + executionTimeout: "0s" + finalizationTimeout: "10s" + retryDelay: "30s" + maxRetries: 3 + +batch: + inbound: + upstreamBatchSize: 1 + upstreamConcurrency: 8 + upstreamRateLimit: + enabled: true + mode: "redis" + redisGroup: "default" + keyPrefix: "iotlcm:worker:inbound:upstream_basic_info" + redisFailureFallback: "error" + default: + qps: 20 + burst: 1 + requestCost: 1 + channels: {} + +# 本月流量全量同步。开发环境常驻监听，Admin 未配置定时任务时不会自动启动全量扫描。 +trafficUsageFullSync: + enabled: true + redisKeyPrefix: "flow:" + retry: + enabled: true + maxRounds: 5 + failureRetention: "720h" + producer: + # channel requires idx_card_chan_key; never creates or rebuilds an index. + scanConcurrency: 4 # SQL burst protection, not a measured optimum + pageSize: 100 + globalHighWaterMark: 10000 + channelHighWaterMark: 2000 + pollInterval: "2s" + runLockKey: "flow:producer:active" + runLockTTL: "5m" + consumer: + enabled: true + chanClientName: "192.0.2.29:8110" + chanClientTimeout: "90s" + lockKey: "flow:consumer:active" + lockTTL: "1m" + pollInterval: "1s" + leaseTTL: "3m" + taskTimeout: "2m" + retryDelay: "30s" + maxRetryDelay: "30m" + claimBatch: 1 + mobileConcurrency: 30 + telecomConcurrency: 20 + genericConcurrency: 1 + maxChannels: 8 + channelConcurrency: 1 + maxInFlight: 8 + maxAttempts: 5 + # 失败池告警阀值（条目数）：单条 FailureRecord 约 1KB 且保留整个 + # failureRetention，超阀值只打 WARN 日志，绝不截断 Run。 + failurePoolAlertThreshold: 50000 + +# 独立的日流量同步链路，与月流量/状态同步同构。 +# 发布配置默认仍关闭：先用手动 API + 单通道 scope 预演，确认产出与旧链路逐字段一致后再打开。 +trafficUsageDailySync: + enabled: false + redisKeyPrefix: "daily:" + retry: + enabled: true + maxRounds: 3 + failureRetention: "720h" + producer: + pageSize: 100 + globalHighWaterMark: 0 + channelHighWaterMark: 0 + pollInterval: "2s" + runLockKey: "daily:producer:active" + runLockTTL: "5m" + consumer: + enabled: false + lockKey: "daily:consumer:active" + lockTTL: "1m" + pollInterval: "1s" + leaseTTL: "3m" + taskTimeout: "2m" + claimBatch: 100 + mobileConcurrency: 30 + telecomConcurrency: 8 + genericConcurrency: 1 + genericQPS: 1.5 + templateQPS: + sim_yd_v5: 30 + sim_yd_mock_v5: 30 + sim_onelink_tengyu_v1: 30 + sim_onelink_beixin_v1: 30 + sim_onelink_xuzhou_mobile_v1: 30 + sim_onelink_lelian_v1: 30 + sim_onelink_qianxun_v1: 30 + sim_onelink_henan_xinjing_v1: 30 + sim_onelink_daihe_v1: 30 + sim_dx_v1: 25 + +# 独立的月流量历史同步链路（写 TDengine 月表），与日流量/状态同步同构。 +# 注意：这与 trafficUsageFullSync 不是一回事——后者填的是 MySQL 当月快照。 +# 默认关闭：日流量验证通过后再单独打开。 +trafficUsageMonthlyHistorySync: + enabled: false + redisKeyPrefix: "monthly-history:" + retry: + enabled: true + maxRounds: 3 + failureRetention: "720h" + producer: + pageSize: 100 + globalHighWaterMark: 0 + channelHighWaterMark: 0 + pollInterval: "2s" + runLockKey: "monthly-history:producer:active" + runLockTTL: "5m" + consumer: + enabled: false + lockKey: "monthly-history:consumer:active" + lockTTL: "1m" + pollInterval: "1s" + leaseTTL: "3m" + taskTimeout: "2m" + claimBatch: 100 + mobileConcurrency: 30 + telecomConcurrency: 8 + genericConcurrency: 1 + genericQPS: 1.5 + # 三条链路共用出口 IP，各自独立限速，避免互相挤占运营商配额。 + templateQPS: + sim_yd_v5: 30 + sim_yd_mock_v5: 30 + sim_onelink_tengyu_v1: 30 + sim_onelink_beixin_v1: 30 + sim_onelink_xuzhou_mobile_v1: 30 + sim_onelink_lelian_v1: 30 + sim_onelink_qianxun_v1: 30 + sim_onelink_henan_xinjing_v1: 30 + sim_onelink_daihe_v1: 30 + sim_dx_v1: 25 + +# 独立的状态同步队列链路，默认关闭以保留旧逻辑作为切换期兜底。 +simStatusQueueSync: + enabled: true + redisKeyPrefix: "sim-status:" + retry: + enabled: true + maxRounds: 3 + failureRetention: "720h" + producer: + pageSize: 1000 + globalHighWaterMark: 0 + pollInterval: "1s" + runLockKey: "sim-status:producer:active" + runLockTTL: "5m" + consumer: + enabled: true + lockKey: "sim-status:consumer:active" + lockTTL: "1m" + pollInterval: "1s" + leaseTTL: "3m" + taskTimeout: "2m" + claimBatch: 100 + mobileConcurrency: 100 + telecomConcurrency: 8 + genericConcurrency: 1 + genericQPS: 1.5 + networkRetries: 2 + # 失败池告警阀值（条目数）：单条 FailureRecord 约 1KB 且保留整个 + # failureRetention，超阀值只打 WARN 日志，绝不截断 Run。 + failurePoolAlertThreshold: 50000 + templateQPS: + sim_yd_v5: 100 + sim_yd_mock_v5: 100 + sim_onelink_tengyu_v1: 100 + sim_onelink_xuzhou_mobile_v1: 100 + sim_onelink_lelian_v1: 100 + sim_dx_v1: 5 + +# 独立的 IMEI 全量同步链路。 +# 默认关闭：先完成预演验证（含 lanshang_8000/8003 是否支持 IMEI 的实测） +# 再按环境开启，否则未验证的通道会直接进失败池。 +imeiFullSync: + enabled: false + redisKeyPrefix: "imei:" + retry: + enabled: true + maxRounds: 5 + failureRetention: "720h" + producer: + pageSize: 1000 + globalHighWaterMark: 0 + pollInterval: "1s" + runLockKey: "imei:producer:active" + runLockTTL: "5m" + consumer: + enabled: false + chanClientName: "iotlcm.example.com:8110" + chanClientTimeout: "30s" + lockKey: "imei:consumer:active" + lockTTL: "1m" + pollInterval: "1s" + leaseTTL: "3m" + taskTimeout: "2m" + claimBatch: 100 + mobileConcurrency: 100 + telecomConcurrency: 8 + genericConcurrency: 1 + genericQPS: 1.5 + networkRetries: 2 + # 失败池告警阀值（条目数）：单条 FailureRecord 约 1KB 且保留整个 + # failureRetention，超阀值只打 WARN 日志，绝不截断 Run。 + failurePoolAlertThreshold: 50000 + templateQPS: + sim_yd_v5: 100 + sim_yd_mock_v5: 100 + sim_onelink_tengyu_v1: 100 + sim_onelink_xuzhou_mobile_v1: 100 + sim_onelink_lelian_v1: 100 + sim_dx_v1: 5 + +# 诊断消费者开关（仅本地调试开） +workerDiagnostic: + enabled: true + +# 时序数据库配置 +tsdb: + database: "TdEngine" #可选择 TdEngine、Influxdb + tdengine: + # type: "taosRestful" #http连接方式，端口是6041 + # dsn: "root:taosdata@http(127.0.0.1:6041)/" + type: "taosWS" #websocket连接方式，端口是6041 + # dsn: "root:taosdata@ws(127.0.0.1:6041)/" + dsn: "root:sdyz2026__@ws(192.0.2.29:6041)/" + dbName: "iotlcm" + maxOpenConns: 100 + maxIdleConns: 10 + +# 上游服务调用 +upstream: + proxy: + address: "iotlcm.example.com:8510" + # address: "localhost:8510" + timeout: 10s + gateway: + address: "iotlcm.example.com:8110" + timeout: 10s diff --git a/config/config_default.yaml b/config/config_default.yaml new file mode 100644 index 0000000..e8c51d0 --- /dev/null +++ b/config/config_default.yaml @@ -0,0 +1,376 @@ +# config_default.yaml 配置文件 +# @Link https://github.com/xiujiecn/xiu-admin +# @Copyright Copyright (c) 2025 LiXiujie +# @Author Lxj +# @License https://github.com/xiujiecn/xiu-admin/blob/master/LICENSE + +# 系统配置 +workerDiagnostic: + enabled: false + +fullSyncOps: + control: + # Worker control is reachable only on the private Docker network. + listenAddress: ":18081" + readTimeout: "5s" + writeTimeout: "15s" + shutdownTimeout: "10s" + archive: + enabled: false + rootDir: "/app/logs/full-sync" + pollInterval: "10s" + retryMin: "5s" + retryMax: "5m" + replay: + # Retry-round budget for a replayed Run when the request omits maxRounds. + # Requests may override it up to 100; larger requests are rejected. + maxRounds: 3 + +# Alipay credentials are injected by the deployment configuration. +alipay: + appId: "" + privateKey: "" + alipayPublicKey: "" + isProduction: false + +rabbitmq: + balanceRecharge: + retryDelay: "1m" + # 充值超时消息最大重试次数，超过后投递到 iotlcm.balance.recharge.dlq + maxRetry: 100 + batchExecute: + executionTimeout: "0s" + finalizationTimeout: "60s" + retryDelay: "30s" + maxRetries: 3 + +batch: + allocate: + processingRetryAfter: "30m" + deadlockMaxRetries: 3 + deadlockRetryDelay: "200ms" + inbound: + upstreamBatchSize: 1 + upstreamConcurrency: 8 + upstreamRateLimit: + enabled: true + mode: "redis" + redisGroup: "default" + keyPrefix: "iotlcm:worker:inbound:upstream_basic_info" + redisFailureFallback: "error" + default: + qps: 20 + burst: 1 + requestCost: 1 + channels: {} + +# 本月流量全量同步（首版默认关闭，完成 PoC 后按环境开启） +trafficUsageFullSync: + enabled: false + supplyObservation: + enabled: false # 1s sampling, 10s snapshots; Run first-round channel scanning only + redisKeyPrefix: "flow:" + retry: + enabled: true + maxRounds: 5 + failureRetention: "720h" + producer: + # channel requires idx_card_chan_key; never creates or rebuilds an index. + scanConcurrency: 4 # SQL burst protection, not a measured optimum + pageSize: 100 + globalHighWaterMark: 10000 + channelHighWaterMark: 2000 + pollInterval: "2s" + runLockKey: "flow:producer:active" + runLockTTL: "5m" + consumer: + enabled: false + # Full-sync consumer calls Chan directly; leave empty only for local tests. + chanClientName: "localhost:8110" + # A batch of 100 cards may queue behind the gateway's per-IP Tibiot + # limiter (about 650ms per request) before its request is served. + chanClientTimeout: "90s" + lockKey: "flow:consumer:active" + lockTTL: "1m" + pollInterval: "1s" + leaseTTL: "3m" + taskTimeout: "2m" + claimBatch: 100 + mobileConcurrency: 30 + telecomConcurrency: 20 + genericConcurrency: 1 + genericQPS: 1.5 + # 失败池告警阀值（条目数）：单条 FailureRecord 约 1KB 且保留整个 + # failureRetention，超阀值只打 WARN 日志，绝不截断 Run。 + failurePoolAlertThreshold: 50000 + templateQPS: + sim_yd_v5: 30 + sim_yd_mock_v5: 30 + sim_onelink_tengyu_v1: 30 + sim_onelink_beixin_v1: 30 + lanshang_8000: 30 + lanshang_8003: 30 + sim_onelink_xuzhou_mobile_v1: 30 + sim_onelink_lelian_v1: 30 + sim_onelink_qianxun_v1: 30 + sim_onelink_henan_xinjing_v1: 30 + sim_onelink_daihe_v1: 30 + sim_dx_v1: 25 + +# 独立的日流量同步链路，与月流量/状态同步同构。 +# 默认关闭：打开后 worker 才装配日流量 producer 与 consumer。 +# 与 trafficUsageFullSync 使用独立前缀与 templateQPS，避免三条链路互相抢运营商配额； +# 定时窗口建议与全量流量同步错峰 2-3 小时（避开 0-4 点出账窗口）。 +trafficUsageDailySync: + enabled: false + redisKeyPrefix: "daily:" + retry: + enabled: true + # 默认 3 轮，对齐状态同步；失败卡最终进失败池，可 replay。 + maxRounds: 3 + failureRetention: "720h" + producer: + pageSize: 100 + globalHighWaterMark: 0 + channelHighWaterMark: 0 + pollInterval: "2s" + runLockKey: "daily:producer:active" + runLockTTL: "5m" + consumer: + enabled: false + lockKey: "daily:consumer:active" + lockTTL: "1m" + pollInterval: "1s" + leaseTTL: "3m" + taskTimeout: "2m" + claimBatch: 100 + mobileConcurrency: 30 + telecomConcurrency: 8 + genericConcurrency: 1 + genericQPS: 1.5 + # 与 trafficUsageFullSync 分开配置：日流量与月流量走同一出口 IP， + # 共用一份 QPS 会让两轮同步互相挤占运营商配额。 + templateQPS: + sim_yd_v5: 30 + sim_yd_mock_v5: 30 + sim_onelink_tengyu_v1: 30 + sim_onelink_beixin_v1: 30 + lanshang_8000: 30 + lanshang_8003: 30 + sim_onelink_xuzhou_mobile_v1: 30 + sim_onelink_lelian_v1: 30 + sim_onelink_qianxun_v1: 30 + sim_onelink_henan_xinjing_v1: 30 + sim_onelink_daihe_v1: 30 + sim_dx_v1: 25 + +# 独立的月流量历史同步链路（写 TDengine 月表），与日流量/状态同步同构。 +# 注意：这与 trafficUsageFullSync 不是一回事——后者填的是 MySQL 当月快照。 +# 默认关闭：日流量验证通过后再单独打开。 +trafficUsageMonthlyHistorySync: + enabled: false + redisKeyPrefix: "monthly-history:" + retry: + enabled: true + maxRounds: 3 + failureRetention: "720h" + producer: + pageSize: 100 + globalHighWaterMark: 0 + channelHighWaterMark: 0 + pollInterval: "2s" + runLockKey: "monthly-history:producer:active" + runLockTTL: "5m" + consumer: + enabled: false + lockKey: "monthly-history:consumer:active" + lockTTL: "1m" + pollInterval: "1s" + leaseTTL: "3m" + taskTimeout: "2m" + claimBatch: 100 + mobileConcurrency: 30 + telecomConcurrency: 8 + genericConcurrency: 1 + genericQPS: 1.5 + # 三条链路共用出口 IP，各自独立限速，避免互相挤占运营商配额。 + templateQPS: + sim_yd_v5: 30 + sim_yd_mock_v5: 30 + sim_onelink_tengyu_v1: 30 + sim_onelink_beixin_v1: 30 + lanshang_8000: 30 + lanshang_8003: 30 + sim_onelink_xuzhou_mobile_v1: 30 + sim_onelink_lelian_v1: 30 + sim_onelink_qianxun_v1: 30 + sim_onelink_henan_xinjing_v1: 30 + sim_onelink_daihe_v1: 30 + sim_dx_v1: 25 + +# 独立的状态同步队列链路，作为消费 worker 常驻启用。 +simStatusQueueSync: + enabled: true + redisKeyPrefix: "sim-status:" + retry: + enabled: true + maxRounds: 3 + failureRetention: "720h" + producer: + pageSize: 1000 + globalHighWaterMark: 0 + pollInterval: "1s" + runLockKey: "sim-status:producer:active" + runLockTTL: "5m" + consumer: + enabled: true + lockKey: "sim-status:consumer:active" + lockTTL: "1m" + pollInterval: "1s" + leaseTTL: "3m" + taskTimeout: "2m" + claimBatch: 100 + mobileConcurrency: 100 + telecomConcurrency: 8 + genericConcurrency: 1 + genericQPS: 1.5 + networkRetries: 2 + # 失败池告警阀值（条目数）：单条 FailureRecord 约 1KB 且保留整个 + # failureRetention，超阀值只打 WARN 日志，绝不截断 Run。 + # 50000 条 ≈ 50MB，约当 255,976 张卡的 20% 失败率。 + failurePoolAlertThreshold: 50000 + templateQPS: + sim_yd_v5: 100 + sim_yd_mock_v5: 100 + sim_onelink_tengyu_v1: 100 + sim_onelink_beixin_v1: 100 + lanshang_8000: 100 + lanshang_8003: 100 + sim_onelink_xuzhou_mobile_v1: 100 + sim_onelink_lelian_v1: 100 + sim_dx_v1: 5 + +# 独立的 IMEI 全量同步链路（首版默认关闭，PoC 后按环境开启）。 +# +# 该链路是"只增不减"的：只写非空 IMEI，上游返回空值只记心跳不写值， +# 所以一次上游抖动不会清空全库 IMEI。 +# +# 移动 IMEI 没有批量接口，chan 内部逐卡请求，吞吐 = 卡数 × 单请求耗时。 +# 电信 18 QPS 额度与状态/流量链路共享，故 templateQPS 压到 5。 +imeiFullSync: + enabled: false + redisKeyPrefix: "imei:" + retry: + enabled: true + # 移动 13000 系统忙是常态（1 QPS 下仍有约 70%），轮数给足 + maxRounds: 5 + failureRetention: "720h" + producer: + pageSize: 1000 + globalHighWaterMark: 0 + pollInterval: "1s" + runLockKey: "imei:producer:active" + runLockTTL: "5m" + consumer: + enabled: false + # 全量同步直连 chan；留空仅供本地测试 + chanClientName: "" + chanClientTimeout: "30s" + lockKey: "imei:consumer:active" + lockTTL: "1m" + pollInterval: "1s" + leaseTTL: "3m" + taskTimeout: "2m" + claimBatch: 100 + mobileConcurrency: 100 + telecomConcurrency: 8 + genericConcurrency: 1 + genericQPS: 1.5 + networkRetries: 2 + # 失败池告警阀值（条目数）：单条 FailureRecord 约 1KB 且保留整个 + # failureRetention（30 天），超阀值只打 WARN 日志，绝不截断 Run。 + # 50000 条 ≈ 50MB，约当 255,976 张卡的 20% 失败率。 + failurePoolAlertThreshold: 50000 + templateQPS: + sim_yd_v5: 100 + sim_yd_mock_v5: 100 + sim_onelink_tengyu_v1: 100 + sim_onelink_beixin_v1: 100 + lanshang_8000: 100 + lanshang_8003: 100 + sim_onelink_xuzhou_mobile_v1: 100 + sim_onelink_lelian_v1: 100 + # 电信额度与状态/流量链路共享，压到最低 + sim_dx_v1: 5 + +upstream: + proxy: + address: "localhost:8510" + timeout: 10s + gateway: + address: "localhost:8110" + timeout: 10s + # 入库前卡还未进入本地卡表，三码校验直连 upstream-chan，避免走 8510 proxy 的本地卡拦截。 + +system: + name: "IotLCM" + version: "1.0" + description: "IotLCM Server" + gfmode: "dev" + upload: + local: + path: "resource/upload" + +# 日志配置 +logger: + path: "/app/logs" # 日志文件路径。默认为空，表示关闭，仅输出到终端 + file: "{Y-m-d-H}.log" # 日志文件格式。默认为"{Y-m-d}.log" + prefix: "" # 日志内容输出前缀。默认为空 + level: "dev" # 日志输出级别 + timeFormat: "2006-01-02T15:04:05" # 自定义日志输出的时间格式，使用Golang标准的时间格式配置 + ctxKeys: [] # 自定义Context上下文变量名称，自动打印Context的变量到日志中。默认为空 + header: true # 是否打印日志的头信息。默认true + stdout: true # 日志是否同时输出到终端。默认true + rotateSize: 0 # 按照日志文件大小对文件进行滚动切分。默认为0，表示关闭滚动切分特性 + rotateExpire: 0 # 按照日志文件时间间隔对文件滚动切分。默认为0，表示关闭滚动切分特性 + rotateBackupLimit: 0 # 按照切分的文件数量清理切分文件，当滚动切分特性开启时有效。默认为0，表示不备份，切分则删除 + rotateBackupExpire: 0 # 按照切分的文件有效期清理切分文件，当滚动切分特性开启时有效。默认为0，表示不备份，切分则删除 + rotateBackupCompress: 0 # 滚动切分文件的压缩比（0-9）。默认为0，表示不压缩 + rotateCheckInterval: "1h" # 滚动切分的时间检测间隔，一般不需要设置。默认为1小时 + stdoutColorDisabled: false # 关闭终端的颜色打印。默认开启 + writerColorEnable: false # 日志文件是否带上颜色。默认false，表示不带颜色 + +# 数据库配置 +database: + logger: + path: "/app/logs/sql" + level: "all" + stdout: true + default: # 分组名称，可自定义，默认为default + link: "mysql:${DB_USER}:${DB_PASSWORD}@tcp(${DB_HOST}:3306)/iotlcm?charset=utf8mb4&parseTime=True&loc=Asia%2FShanghai" + debug: true #开启调试模式 + charset: "utf8mb4" # (可选)数据库编码(如: utf8mb4/utf8/gbk/gb2312)，一般设置为utf8mb4。默认为utf8。 + dryRun: false #ORM空跑(只读不写) + maxIdle: 30 #连接池最大闲置的连接数 + maxOpen: 100 #连接池最大打开的连接数 + maxLifetime: 120 #(单位秒)连接对象可重复使用的时间长度 + +# 缓存配置 +cache: + adapter: "redis" # 缓存适配器，支持：file|redis + fileDir: "resource/cache" # 缓存文件目录 + +# Redis 配置 +redis: + # 单实例配置 + default: + mode: "single" #redis模式，支持：single|cluster|sentinel + sentinelMasterName: "${REDIS_SENTINEL_MASTER}" #哨兵模式主节点名称 + address: 192.0.2.200:6379 + db: 0 + pass: "${REDIS_PASSWORD}" + minIdle: 5 #最小空闲连接数 + maxIdle: 30 #允许闲置的最大连接数 + maxActive: 5000 #最大活跃连接数 + idleTimeout: "60s" #连接最大空闲时间，使用时间字符串例如30s/2m/2d + ClientName: "SystemCache" diff --git a/go.mod b/go.mod new file mode 100644 index 0000000..1ee195b --- /dev/null +++ b/go.mod @@ -0,0 +1,70 @@ +module git.5iot.com/iotlcm-public/iotlcm-worker-svr + +go 1.25.5 + +require ( + github.com/gogf/gf/contrib/drivers/mysql/v2 v2.10.0 + github.com/gogf/gf/contrib/drivers/sqlite/v2 v2.10.0 + github.com/gogf/gf/contrib/nosql/redis/v2 v2.10.0 + github.com/gogf/gf/v2 v2.10.0 + github.com/rabbitmq/amqp091-go v1.10.0 + github.com/smartwalle/alipay/v3 v3.2.29 + github.com/taosdata/driver-go/v3 v3.8.0 + github.com/xuri/excelize/v2 v2.9.0 +) + +require ( + github.com/BurntSushi/toml v1.5.0 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect + github.com/clbanning/mxj/v2 v2.7.0 // indirect + github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect + github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect + github.com/dustin/go-humanize v1.0.1 // indirect + github.com/emirpasic/gods/v2 v2.0.0-alpha // indirect + github.com/fatih/color v1.18.0 // indirect + github.com/fsnotify/fsnotify v1.9.0 // indirect + github.com/glebarez/go-sqlite v1.21.2 // indirect + github.com/go-logr/logr v1.4.3 // indirect + github.com/go-logr/stdr v1.2.2 // indirect + github.com/go-sql-driver/mysql v1.7.1 // indirect + github.com/google/uuid v1.6.0 // indirect + github.com/gorilla/websocket v1.5.3 // indirect + github.com/grokify/html-strip-tags-go v0.1.0 // indirect + github.com/json-iterator/go v1.1.12 // indirect + github.com/magiconair/properties v1.8.10 // indirect + github.com/mattn/go-colorable v0.1.13 // indirect + github.com/mattn/go-isatty v0.0.20 // indirect + github.com/mattn/go-runewidth v0.0.16 // indirect + github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect + github.com/modern-go/reflect2 v1.0.2 // indirect + github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 // indirect + github.com/olekukonko/errors v1.1.0 // indirect + github.com/olekukonko/ll v0.0.9 // indirect + github.com/olekukonko/tablewriter v1.1.0 // indirect + github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect + github.com/redis/go-redis/v9 v9.18.0 // indirect + github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect + github.com/richardlehane/mscfb v1.0.4 // indirect + github.com/richardlehane/msoleps v1.0.4 // indirect + github.com/rivo/uniseg v0.2.0 // indirect + github.com/smartwalle/ncrypto v1.0.4 // indirect + github.com/smartwalle/ngx v1.1.0 // indirect + github.com/smartwalle/nsign v1.0.9 // indirect + github.com/xuri/efp v0.0.0-20240408161823-9ad904a10d6d // indirect + github.com/xuri/nfp v0.0.0-20240318013403-ab9948c2c4a7 // indirect + go.opentelemetry.io/auto/sdk v1.2.1 // indirect + go.opentelemetry.io/otel v1.39.0 // indirect + go.opentelemetry.io/otel/metric v1.39.0 // indirect + go.opentelemetry.io/otel/sdk v1.39.0 // indirect + go.opentelemetry.io/otel/trace v1.39.0 // indirect + go.uber.org/atomic v1.11.0 // indirect + golang.org/x/crypto v0.46.0 // indirect + golang.org/x/net v0.48.0 // indirect + golang.org/x/sys v0.39.0 // indirect + golang.org/x/text v0.32.0 // indirect + gopkg.in/yaml.v3 v3.0.1 // indirect + modernc.org/libc v1.22.5 // indirect + modernc.org/mathutil v1.5.0 // indirect + modernc.org/memory v1.5.0 // indirect + modernc.org/sqlite v1.23.1 // indirect +) diff --git a/go.sum b/go.sum new file mode 100644 index 0000000..2b28f14 --- /dev/null +++ b/go.sum @@ -0,0 +1,172 @@ +github.com/BurntSushi/toml v1.5.0 h1:W5quZX/G/csjUnuI8SUYlsHs9M38FC7znL0lIO+DvMg= +github.com/BurntSushi/toml v1.5.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho= +github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs= +github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c= +github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA= +github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/clbanning/mxj/v2 v2.7.0 h1:WA/La7UGCanFe5NpHF0Q3DNtnCsVoxbPKuyBNHWRyME= +github.com/clbanning/mxj/v2 v2.7.0/go.mod h1:hNiWqW14h+kc+MdF9C6/YoRfjEJoR3ou6tn/Qo+ve2s= +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM= +github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78= +github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc= +github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= +github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= +github.com/emirpasic/gods/v2 v2.0.0-alpha h1:dwFlh8pBg1VMOXWGipNMRt8v96dKAIvBehtCt6OtunU= +github.com/emirpasic/gods/v2 v2.0.0-alpha/go.mod h1:W0y4M2dtBB9U5z3YlghmpuUhiaZT2h6yoeE+C1sCp6A= +github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM= +github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU= +github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k= +github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0= +github.com/glebarez/go-sqlite v1.21.2 h1:3a6LFC4sKahUunAmynQKLZceZCOzUthkRkEAl9gAXWo= +github.com/glebarez/go-sqlite v1.21.2/go.mod h1:sfxdZyhQjTM2Wry3gVYWaW072Ri1WMdWJi0k6+3382k= +github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= +github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI= +github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= +github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= +github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= +github.com/go-sql-driver/mysql v1.7.1 h1:lUIinVbN1DY0xBg0eMOzmmtGoHwWBbvnWubQUrtU8EI= +github.com/go-sql-driver/mysql v1.7.1/go.mod h1:OXbVy3sEdcQ2Doequ6Z5BW6fXNQTmx+9S1MCJN5yJMI= +github.com/gogf/gf/contrib/drivers/mysql/v2 v2.10.0 h1:UvqxwinkelKxwdwnKUfdy51/ls4RL7MCeJqAZOVAy0I= +github.com/gogf/gf/contrib/drivers/mysql/v2 v2.10.0/go.mod h1:6v7oGBF9wv59WERJIOJxXmLhkUcxwON3tPYW3AZ7wbY= +github.com/gogf/gf/contrib/drivers/sqlite/v2 v2.10.0 h1:OyAH7Ls2c9Un7CJiAq7G6eY1jWIICRkN8C5SyM94rnY= +github.com/gogf/gf/contrib/drivers/sqlite/v2 v2.10.0/go.mod h1:fwhAMG0qZpeHbbP2JE78rJRfV7eBbu9jXkxTMM1lwyo= +github.com/gogf/gf/contrib/nosql/redis/v2 v2.10.0 h1:N/F9CuDdUZLoM1nVRqrDE/33pDZuhVxpNY4wYdeIaBs= +github.com/gogf/gf/contrib/nosql/redis/v2 v2.10.0/go.mod h1:x6uoJGfZOtirIRQls8xUlYzC6f7T/eULPUa9er368X0= +github.com/gogf/gf/v2 v2.10.0 h1:rzDROlyqGMe/eM6dCalSR8dZOuMIdLhmxKSH1DGhbFs= +github.com/gogf/gf/v2 v2.10.0/go.mod h1:Svl1N+E8G/QshU2DUbh/3J/AJauqCgUnxHurXWR4Qx0= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26 h1:Xim43kblpZXfIBQsbuBVKCudVG457BR2GZFIz3uw3hQ= +github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26/go.mod h1:dDKJzRmX4S37WGHujM7tX//fmj1uioxKzKxz3lo4HJo= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/gorilla/websocket v1.5.0/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= +github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg= +github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= +github.com/grokify/html-strip-tags-go v0.1.0 h1:03UrQLjAny8xci+R+qjCce/MYnpNXCtgzltlQbOBae4= +github.com/grokify/html-strip-tags-go v0.1.0/go.mod h1:ZdzgfHEzAfz9X6Xe5eBLVblWIxXfYSQ40S/VKrAOGpc= +github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= +github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= +github.com/klauspost/cpuid/v2 v2.2.3 h1:sxCkb+qR91z4vsqw4vGGZlDgPz3G7gjaLyK3V8y70BU= +github.com/klauspost/cpuid/v2 v2.2.3/go.mod h1:RVVoqg1df56z8g3pUjL/3lE5UfnlrJX8tyFgg4nqhuY= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8SYxI99mE= +github.com/magiconair/properties v1.8.10/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0= +github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA= +github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg= +github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM= +github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= +github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= +github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc= +github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w= +github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M= +github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= +github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 h1:RWengNIwukTxcDr9M+97sNutRR1RKhG96O6jWumTTnw= +github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826/go.mod h1:TaXosZuwdSHYgviHp1DAtfrULt5eUgsSMsZf+YrPgl8= +github.com/olekukonko/errors v1.1.0 h1:RNuGIh15QdDenh+hNvKrJkmxxjV4hcS50Db478Ou5sM= +github.com/olekukonko/errors v1.1.0/go.mod h1:ppzxA5jBKcO1vIpCXQ9ZqgDh8iwODz6OXIGKU8r5m4Y= +github.com/olekukonko/ll v0.0.9 h1:Y+1YqDfVkqMWuEQMclsF9HUR5+a82+dxJuL1HHSRpxI= +github.com/olekukonko/ll v0.0.9/go.mod h1:En+sEW0JNETl26+K8eZ6/W4UQ7CYSrrgg/EdIYT2H8g= +github.com/olekukonko/tablewriter v1.1.0 h1:N0LHrshF4T39KvI96fn6GT8HEjXRXYNDrDjKFDB7RIY= +github.com/olekukonko/tablewriter v1.1.0/go.mod h1:5c+EBPeSqvXnLLgkm9isDdzR3wjfBkHR9Nhfp3NWrzo= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U= +github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/rabbitmq/amqp091-go v1.10.0 h1:STpn5XsHlHGcecLmMFCtg7mqq0RnD+zFr4uzukfVhBw= +github.com/rabbitmq/amqp091-go v1.10.0/go.mod h1:Hy4jKW5kQART1u+JkDTF9YYOQUHXqMuhrgxOEeS7G4o= +github.com/redis/go-redis/v9 v9.18.0 h1:pMkxYPkEbMPwRdenAzUNyFNrDgHx9U+DrBabWNfSRQs= +github.com/redis/go-redis/v9 v9.18.0/go.mod h1:k3ufPphLU5YXwNTUcCRXGxUoF1fqxnhFQmscfkCoDA0= +github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= +github.com/richardlehane/mscfb v1.0.4 h1:WULscsljNPConisD5hR0+OyZjwK46Pfyr6mPu5ZawpM= +github.com/richardlehane/mscfb v1.0.4/go.mod h1:YzVpcZg9czvAuhk9T+a3avCpcFPMUWm7gK3DypaEsUk= +github.com/richardlehane/msoleps v1.0.1/go.mod h1:BWev5JBpU9Ko2WAgmZEuiz4/u3ZYTKbjLycmwiWUfWg= +github.com/richardlehane/msoleps v1.0.4 h1:WuESlvhX3gH2IHcd8UqyCuFY5yiq/GR/yqaSM/9/g00= +github.com/richardlehane/msoleps v1.0.4/go.mod h1:BWev5JBpU9Ko2WAgmZEuiz4/u3ZYTKbjLycmwiWUfWg= +github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY= +github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc= +github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ= +github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc= +github.com/smartwalle/alipay/v3 v3.2.29 h1:roGFqlml8hDa//0TpFmlyxZhndTYs7rbYLu/HlNFNJo= +github.com/smartwalle/alipay/v3 v3.2.29/go.mod h1:XarBLuAkwK3ah7mYjVtghRu+ysxzlex9sRkgqNMzMRU= +github.com/smartwalle/ncrypto v1.0.4 h1:P2rqQxDepJwgeO5ShoC+wGcK2wNJDmcdBOWAksuIgx8= +github.com/smartwalle/ncrypto v1.0.4/go.mod h1:Dwlp6sfeNaPMnOxMNayMTacvC5JGEVln3CVdiVDgbBk= +github.com/smartwalle/ngx v1.1.0 h1:q8nANgWSPRGeI/u+ixBoA4mf68DrUq6vZ+n9L5UKv9I= +github.com/smartwalle/ngx v1.1.0/go.mod h1:mx/nz2Pk5j+RBs7t6u6k22MPiBG/8CtOMpCnALIG8Y0= +github.com/smartwalle/nsign v1.0.9 h1:8poAgG7zBd8HkZy9RQDwasC6XZvJpDGQWSjzL2FZL6E= +github.com/smartwalle/nsign v1.0.9/go.mod h1:eY6I4CJlyNdVMP+t6z1H6Jpd4m5/V+8xi44ufSTxXgc= +github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= +github.com/stretchr/objx v0.5.0 h1:1zr/of2m5FGMsad5YfcqgdqdWrIhu+EBEJRhR1U7z/c= +github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= +github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= +github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= +github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +github.com/taosdata/driver-go/v3 v3.8.0 h1:qY1xFidspISuuZc0TAbeJWMtAdKJOSYQOXfepkUK6fI= +github.com/taosdata/driver-go/v3 v3.8.0/go.mod h1:S6OGOinfR0xxxaMGsvBi9cLkYxEIW1p6qqr8QJATTlg= +github.com/xuri/efp v0.0.0-20240408161823-9ad904a10d6d h1:llb0neMWDQe87IzJLS4Ci7psK/lVsjIS2otl+1WyRyY= +github.com/xuri/efp v0.0.0-20240408161823-9ad904a10d6d/go.mod h1:ybY/Jr0T0GTCnYjKqmdwxyxn2BQf2RcQIIvex5QldPI= +github.com/xuri/excelize/v2 v2.9.0 h1:1tgOaEq92IOEumR1/JfYS/eR0KHOCsRv/rYXXh6YJQE= +github.com/xuri/excelize/v2 v2.9.0/go.mod h1:uqey4QBZ9gdMeWApPLdhm9x+9o2lq4iVmjiLfBS5hdE= +github.com/xuri/nfp v0.0.0-20240318013403-ab9948c2c4a7 h1:hPVCafDV85blFTabnqKgNhDCkJX25eik94Si9cTER4A= +github.com/xuri/nfp v0.0.0-20240318013403-ab9948c2c4a7/go.mod h1:WwHg+CVyzlv/TX9xqBFXEZAuxOPxn2k1GNHwG41IIUQ= +github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0= +github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA= +go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64= +go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y= +go.opentelemetry.io/otel v1.39.0 h1:8yPrr/S0ND9QEfTfdP9V+SiwT4E0G7Y5MO7p85nis48= +go.opentelemetry.io/otel v1.39.0/go.mod h1:kLlFTywNWrFyEdH0oj2xK0bFYZtHRYUdv1NklR/tgc8= +go.opentelemetry.io/otel/metric v1.39.0 h1:d1UzonvEZriVfpNKEVmHXbdf909uGTOQjA0HF0Ls5Q0= +go.opentelemetry.io/otel/metric v1.39.0/go.mod h1:jrZSWL33sD7bBxg1xjrqyDjnuzTUB0x1nBERXd7Ftcs= +go.opentelemetry.io/otel/sdk v1.39.0 h1:nMLYcjVsvdui1B/4FRkwjzoRVsMK8uL/cj0OyhKzt18= +go.opentelemetry.io/otel/sdk v1.39.0/go.mod h1:vDojkC4/jsTJsE+kh+LXYQlbL8CgrEcwmt1ENZszdJE= +go.opentelemetry.io/otel/sdk/metric v1.39.0 h1:cXMVVFVgsIf2YL6QkRF4Urbr/aMInf+2WKg+sEJTtB8= +go.opentelemetry.io/otel/sdk/metric v1.39.0/go.mod h1:xq9HEVH7qeX69/JnwEfp6fVq5wosJsY1mt4lLfYdVew= +go.opentelemetry.io/otel/trace v1.39.0 h1:2d2vfpEDmCJ5zVYz7ijaJdOF59xLomrvj7bjt6/qCJI= +go.opentelemetry.io/otel/trace v1.39.0/go.mod h1:88w4/PnZSazkGzz/w84VHpQafiU4EtqqlVdxWy+rNOA= +go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE= +go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= +golang.org/x/crypto v0.46.0 h1:cKRW/pmt1pKAfetfu+RCEvjvZkA9RimPbh7bhFjGVBU= +golang.org/x/crypto v0.46.0/go.mod h1:Evb/oLKmMraqjZ2iQTwDwvCtJkczlDuTmdJXoZVzqU0= +golang.org/x/image v0.18.0 h1:jGzIakQa/ZXI1I0Fxvaa9W7yP25TqT6cHIHn+6CqvSQ= +golang.org/x/image v0.18.0/go.mod h1:4yyo5vMFQjVjUcVk4jEQcU9MGy/rulF5WvUILseCM2E= +golang.org/x/net v0.48.0 h1:zyQRTTrjc33Lhh0fBgT/H3oZq9WuvRR5gPC70xpDiQU= +golang.org/x/net v0.48.0/go.mod h1:+ndRgGjkh8FGtu1w1FGbEC31if4VrNVMuKTgcAAnQRY= +golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.39.0 h1:CvCKL8MeisomCi6qNZ+wbb0DN9E5AATixKsvNtMoMFk= +golang.org/x/sys v0.39.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/text v0.32.0 h1:ZD01bjUt1FQ9WJ0ClOL5vxgxOI/sVCNgX1YtKwcY0mU= +golang.org/x/text v0.32.0/go.mod h1:o/rUWzghvpD5TXrTIBuJU77MTaN0ljMWE47kxGJQ7jY= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +modernc.org/libc v1.22.5 h1:91BNch/e5B0uPbJFgqbxXuOnxBQjlS//icfQEGmvyjE= +modernc.org/libc v1.22.5/go.mod h1:jj+Z7dTNX8fBScMVNRAYZ/jF91K8fdT2hYMThc3YjBY= +modernc.org/mathutil v1.5.0 h1:rV0Ko/6SfM+8G+yKiyI830l3Wuz1zRutdslNoQ0kfiQ= +modernc.org/mathutil v1.5.0/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E= +modernc.org/memory v1.5.0 h1:N+/8c5rE6EqugZwHii4IFsaJ7MUhoWX07J5tC/iI5Ds= +modernc.org/memory v1.5.0/go.mod h1:PkUhL0Mugw21sHPeskwZW4D6VscE/GQJOnIpCnW6pSU= +modernc.org/sqlite v1.23.1 h1:nrSBg4aRQQwq59JpvGEQ15tNxoO5pX/kUjcRNwSAGQM= +modernc.org/sqlite v1.23.1/go.mod h1:OrDj17Mggn6MhE+iPbBNf7RGKODDE9NFT0f3EwDzJqk= diff --git a/hack/config.yaml b/hack/config.yaml new file mode 100644 index 0000000..997d6b3 --- /dev/null +++ b/hack/config.yaml @@ -0,0 +1,17 @@ +# CLI tool, only in development environment. +# https://goframe.org/docs/cli +gfcli: + gen: + dao: + - link: "mysql:${DB_USER}:${DB_PASSWORD}@tcp(${DB_HOST}:3306)/db_iotlcm?charset=utf8mb4&parseTime=True&loc=Asia%2FShanghai" + tables: "batch_main,batch_detail,sys_dept,sys_oss,t_resource_batch_download,t_sim_card_cancellation,t_sim_card_info,t_sim_flow_pool,t_warn_config,t_warn_log,t_wo_main,t_wo_record,t_message_log,t_user_message" + descriptionTag: true + noModelComment: true + jsonCase: "CamelLower" + gJsonSupport: true + group: "default" + + docker: + build: "-a amd64 -s linux -p temp -ew" + tagPrefixes: + - iotlcm-worker-svr diff --git a/hack/hack-cli.mk b/hack/hack-cli.mk new file mode 100644 index 0000000..526124e --- /dev/null +++ b/hack/hack-cli.mk @@ -0,0 +1,18 @@ + +# Install/Update to the latest CLI tool. +.PHONY: cli +cli: + @set -e; \ + echo "go install github.com/gogf/gf/cmd/gf/v2@latest"; \ + go install github.com/gogf/gf/cmd/gf/v2@latest; \ + echo "GoFame CLI installed successfully!" + + +# Check and install CLI tool. +.PHONY: cli.install +cli.install: + @set -e; \ + gf -v > /dev/null 2>&1 || if [[ "$?" -ne "0" ]]; then \ + echo "GoFame CLI is not installed, start proceeding auto installation..."; \ + make cli; \ + fi; \ No newline at end of file diff --git a/hack/hack.mk b/hack/hack.mk new file mode 100644 index 0000000..2f68179 --- /dev/null +++ b/hack/hack.mk @@ -0,0 +1,75 @@ +.DEFAULT_GOAL := build + +# Update GoFrame and its CLI to latest stable version. +.PHONY: up +up: cli.install + @gf up -a + +# Build binary using configuration from hack/config.yaml. +.PHONY: build +build: cli.install + @gf build -ew + +# Parse api and generate controller/sdk. +.PHONY: ctrl +ctrl: cli.install + @gf gen ctrl + +# Generate Go files for DAO/DO/Entity. +.PHONY: dao +dao: cli.install + @gf gen dao + +# Parse current project go files and generate enums go file. +.PHONY: enums +enums: cli.install + @gf gen enums + +# Generate Go files for Service. +.PHONY: service +service: cli.install + @gf gen service + + +# Build docker image. +.PHONY: image +image: cli.install + $(eval _TAG = $(shell git rev-parse --short HEAD)) +ifneq (, $(shell git status --porcelain 2>/dev/null)) + $(eval _TAG = $(_TAG).dirty) +endif + $(eval _TAG = $(if ${TAG}, ${TAG}, $(_TAG))) + $(eval _PUSH = $(if ${PUSH}, ${PUSH}, )) + @gf docker ${_PUSH} -tn $(DOCKER_NAME):${_TAG}; + + +# Build docker image and automatically push to docker repo. +.PHONY: image.push +image.push: cli.install + @make image PUSH=-p; + + +# Deploy image and yaml to current kubectl environment. +.PHONY: deploy +deploy: cli.install + $(eval _TAG = $(if ${TAG}, ${TAG}, develop)) + + @set -e; \ + mkdir -p $(ROOT_DIR)/temp/kustomize;\ + cd $(ROOT_DIR)/manifest/deploy/kustomize/overlays/${_ENV};\ + kustomize build > $(ROOT_DIR)/temp/kustomize.yaml;\ + kubectl apply -f $(ROOT_DIR)/temp/kustomize.yaml; \ + if [ $(DEPLOY_NAME) != "" ]; then \ + kubectl patch -n $(NAMESPACE) deployment/$(DEPLOY_NAME) -p "{\"spec\":{\"template\":{\"metadata\":{\"labels\":{\"date\":\"$(shell date +%s)\"}}}}}"; \ + fi; + + +# Parsing protobuf files and generating go files. +.PHONY: pb +pb: cli.install + @gf gen pb + +# Generate protobuf files for database tables. +.PHONY: pbentity +pbentity: cli.install + @gf gen pbentity \ No newline at end of file diff --git a/internal/batch/batch.go b/internal/batch/batch.go new file mode 100644 index 0000000..1e647f2 --- /dev/null +++ b/internal/batch/batch.go @@ -0,0 +1,152 @@ +package batch + +import ( + "context" + "errors" + + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +const ( + Type3 = 3 + Type4 = 4 + Type11 = 11 + Type12 = 12 + Type40 = 40 + Type41 = 41 + Type101 = 101 + Type102 = 102 + Type103 = 103 + Type104 = 104 + Type105 = 105 + Type106 = 106 + Type107 = 107 + Type108 = 108 + Type109 = 109 + Type110 = 110 + + AsyncStatusPending = 1 + AsyncStatusProcessing = 2 + AsyncStatusSuccess = 3 + AsyncStatusFailed = 4 + RunResultRunning = 1 + RunResultSuccess = 2 + RunResultFailed = 3 + + DetailStatusPending = 0 + DetailStatusRunning = 1 + DetailStatusSuccess = 2 + DetailStatusFailed = 3 +) + +type Message struct { + BatchNo string `json:"batchNo"` + BatchType int `json:"batchType"` + FailureReason string `json:"failureReason,omitempty"` + Retryable bool `json:"retryable,omitempty"` +} + +// RetryableError marks an executor failure that may succeed after delayed redelivery. +// The batch remains in processing until retries are exhausted and finalized. +type RetryableError struct { + Err error +} + +func (e *RetryableError) Error() string { + if e == nil || e.Err == nil { + return "retryable batch execution error" + } + return e.Err.Error() +} + +func (e *RetryableError) Unwrap() error { + if e == nil { + return nil + } + return e.Err +} + +func Retryable(err error) error { + if err == nil { + return nil + } + if IsRetryable(err) { + return err + } + return &RetryableError{Err: err} +} + +func IsRetryable(err error) bool { + var retryableErr *RetryableError + return errors.As(err, &retryableErr) +} + +// FailureMarkError 表示执行已结束，但失败终态尚未成功写入数据库。 +// 消费者重投这类消息时只能补写失败终态，不能再次执行具体业务。 +type FailureMarkError struct { + Reason string + Err error +} + +func (e *FailureMarkError) Error() string { + return "mark batch failed after executor error: " + e.Reason + ": " + e.Err.Error() +} + +func (e *FailureMarkError) Unwrap() error { + return e.Err +} + +type Executor interface { + Execute(ctx context.Context, batchNo string) error +} + +type ProcessingExecutor interface { + CanExecuteProcessing(ctx context.Context, main *entity.BatchMain) (bool, error) +} + +// SelfClaimingProcessingExecutor marks executors that perform their own +// processing-state CAS before executing a re-entry phase. +type SelfClaimingProcessingExecutor interface { + ClaimsProcessingReentry() bool +} + +type ExecutionFailureHandler interface { + HandleExecutionFailure(ctx context.Context, batchNo string, batchType int, reason string) error +} + +type ProcessingClaimer interface { + ReclaimProcessing(ctx context.Context, batchNo string, batchType int, startedAt *gtime.Time) (bool, error) +} + +type Dispatcher struct { + executors map[int]Executor +} + +func NewDispatcher(executor Executor) *Dispatcher { + return &Dispatcher{executors: map[int]Executor{ + Type103: executor, + Type104: executor, + }} +} + +func (d *Dispatcher) Register(batchType int, executor Executor) { + if d == nil || executor == nil { + return + } + d.executors[batchType] = executor +} + +func (d *Dispatcher) Executor(batchType int) (Executor, bool) { + if d == nil { + return nil, false + } + executor, ok := d.executors[batchType] + return executor, ok && executor != nil +} + +func (d *Dispatcher) Has(batchType int) bool { + _, ok := d.Executor(batchType) + return ok +} diff --git a/internal/batch/batch_test.go b/internal/batch/batch_test.go new file mode 100644 index 0000000..b3a6c0a --- /dev/null +++ b/internal/batch/batch_test.go @@ -0,0 +1,516 @@ +package batch + +import ( + "context" + "encoding/json" + "errors" + "io" + "reflect" + "strings" + "testing" + "time" + + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +type executorStub struct { + calls int + batchNo string + err error +} + +func (s *executorStub) Execute(_ context.Context, batchNo string) error { + s.calls++ + s.batchNo = batchNo + return s.err +} + +type processingExecutorStub struct { + executorStub + canCalls int + main *entity.BatchMain + can bool + canErr error + selfClaim bool +} + +type blockingExecutorStub struct { + calls int +} + +func (s *blockingExecutorStub) Execute(ctx context.Context, _ string) error { + s.calls++ + <-ctx.Done() + return ctx.Err() +} + +func (s *processingExecutorStub) CanExecuteProcessing(_ context.Context, main *entity.BatchMain) (bool, error) { + s.canCalls++ + s.main = main + return s.can, s.canErr +} + +func (s *processingExecutorStub) ClaimsProcessingReentry() bool { + return s.selfClaim +} + +type repositoryStub struct { + mains []*entity.BatchMain + findErr error + markErr error + claimed bool + findCalls int + markCalls int + markBatchNo string + markType int + failErr error + failCalls int + failBatchNo string + failType int + failReason string + successErr error + successCalls int + successBatchNo string + successType int + successCount int + successFails int + successReason string + reclaimErr error + reclaimSet bool + reclaimed bool + reclaimCalls int + reclaimBatchNo string + reclaimType int +} + +func (s *repositoryStub) FindMain(context.Context, string) (*entity.BatchMain, error) { + s.findCalls++ + if s.findErr != nil { + return nil, s.findErr + } + if len(s.mains) == 0 { + return nil, nil + } + index := s.findCalls - 1 + if index >= len(s.mains) { + index = len(s.mains) - 1 + } + return s.mains[index], nil +} + +func (s *repositoryStub) MarkProcessing(_ context.Context, batchNo string, batchType int) (bool, error) { + s.markCalls++ + s.markBatchNo = batchNo + s.markType = batchType + return s.claimed, s.markErr +} + +func (s *repositoryStub) MarkFailed(_ context.Context, batchNo string, batchType int, reason string) (bool, error) { + s.failCalls++ + s.failBatchNo = batchNo + s.failType = batchType + s.failReason = reason + return s.failErr == nil, s.failErr +} + +func (s *repositoryStub) MarkSuccess(_ context.Context, batchNo string, batchType, success, fail int, reason string) (bool, error) { + s.successCalls++ + s.successBatchNo = batchNo + s.successType = batchType + s.successCount = success + s.successFails = fail + s.successReason = reason + return s.successErr == nil, s.successErr +} + +func (s *repositoryStub) ReclaimProcessing(_ context.Context, batchNo string, batchType int, _ *gtime.Time) (bool, error) { + s.reclaimCalls++ + s.reclaimBatchNo = batchNo + s.reclaimType = batchType + if !s.reclaimSet { + return s.reclaimErr == nil, s.reclaimErr + } + return s.reclaimed, s.reclaimErr +} + +func TestMessageJSONContract(t *testing.T) { + body, err := json.Marshal(Message{BatchNo: "BATCH-1", BatchType: Type103}) + if err != nil { + t.Fatalf("marshal message: %v", err) + } + const expected = `{"batchNo":"BATCH-1","batchType":103}` + if string(body) != expected { + t.Fatalf("message JSON = %s, want %s", body, expected) + } +} + +func TestDispatcherSupportsLifecycleTypesAndExplicitRegistration(t *testing.T) { + executor := &executorStub{} + dispatcher := NewDispatcher(executor) + for _, batchType := range []int{Type103, Type104} { + actual, ok := dispatcher.Executor(batchType) + if !ok || actual != executor { + t.Fatalf("batch type %d was not registered with shared executor", batchType) + } + } + if _, ok := dispatcher.Executor(102); ok { + t.Fatal("unexpected executor for unsupported batch type") + } + + inbound := &executorStub{} + dispatcher.Register(1, inbound) + actual, ok := dispatcher.Executor(1) + if !ok || actual != inbound { + t.Fatal("registered inbound executor was not returned") + } + + dataExecutor := &executorStub{} + dispatcher.Register(Type107, dataExecutor) + dispatcher.Register(Type109, dataExecutor) + dispatcher.Register(Type110, dataExecutor) + for _, batchType := range []int{Type107, Type109, Type110} { + actual, ok = dispatcher.Executor(batchType) + if !ok || actual != dataExecutor { + t.Fatalf("batch type %d was not registered with the data service executor", batchType) + } + } +} + +func TestRuntimeAcknowledgesInvalidAndUnclaimableMessages(t *testing.T) { + g.Log().SetWriter(io.Discard) + tests := []struct { + name string + message Message + mains []*entity.BatchMain + claimed bool + wantFind int + wantMarked int + }{ + {name: "empty batch number", message: Message{BatchType: Type103}}, + {name: "blank batch number", message: Message{BatchNo: " ", BatchType: Type103}}, + {name: "unsupported type", message: Message{BatchNo: "B-1", BatchType: 102}}, + {name: "not found", message: Message{BatchNo: "B-1", BatchType: Type103}, wantFind: 1}, + {name: "type mismatch", message: Message{BatchNo: "B-1", BatchType: Type103}, mains: []*entity.BatchMain{{BatchNo: "B-1", BatchType: Type104, BatchAsyncStatus: AsyncStatusPending}}, wantFind: 1}, + {name: "processing", message: Message{BatchNo: "B-1", BatchType: Type103}, mains: []*entity.BatchMain{{BatchNo: "B-1", BatchType: Type103, BatchAsyncStatus: AsyncStatusProcessing}}, wantFind: 1}, + {name: "terminal", message: Message{BatchNo: "B-1", BatchType: Type103}, mains: []*entity.BatchMain{{BatchNo: "B-1", BatchType: Type103, BatchAsyncStatus: 3}}, wantFind: 1}, + {name: "CAS lost", message: Message{BatchNo: "B-1", BatchType: Type103}, mains: []*entity.BatchMain{{BatchNo: "B-1", BatchType: Type103, BatchAsyncStatus: AsyncStatusPending}, {BatchNo: "B-1", BatchType: Type103, BatchAsyncStatus: AsyncStatusProcessing}}, wantFind: 2, wantMarked: 1}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + repository := &repositoryStub{mains: tt.mains, claimed: tt.claimed} + executor := &executorStub{} + runtime := NewRuntime(repository, NewDispatcher(executor)) + if err := runtime.Handle(context.Background(), tt.message); err != nil { + t.Fatalf("Handle() error = %v", err) + } + if repository.findCalls != tt.wantFind || repository.markCalls != tt.wantMarked { + t.Fatalf("repository calls: find=%d mark=%d, want find=%d mark=%d", repository.findCalls, repository.markCalls, tt.wantFind, tt.wantMarked) + } + if executor.calls != 0 { + t.Fatalf("executor calls = %d, want 0", executor.calls) + } + }) + } +} + +func TestRuntimeExecutesRegisteredDataServiceBatch(t *testing.T) { + repository := &repositoryStub{ + mains: []*entity.BatchMain{{BatchNo: "B-109", BatchType: Type109, BatchAsyncStatus: AsyncStatusPending}}, + claimed: true, + } + dataExecutor := &executorStub{} + dispatcher := NewDispatcher(&executorStub{}) + dispatcher.Register(Type109, dataExecutor) + dispatcher.Register(Type110, dataExecutor) + runtime := NewRuntime(repository, dispatcher) + + if err := runtime.Handle(context.Background(), Message{BatchNo: "B-109", BatchType: Type109}); err != nil { + t.Fatalf("Handle() error = %v", err) + } + if dataExecutor.calls != 1 || dataExecutor.batchNo != "B-109" { + t.Fatalf("data executor was not invoked: %+v", dataExecutor) + } +} + +func TestRuntimeExecutesTrafficCutoffConfigWithDedicatedExecutor(t *testing.T) { + repository := &repositoryStub{ + mains: []*entity.BatchMain{{BatchNo: "B-41", BatchType: Type41, BatchAsyncStatus: AsyncStatusPending}}, + claimed: true, + } + dataExecutor := &executorStub{err: errors.New("data executor must not be called")} + trafficCutoffConfigExecutor := &executorStub{} + dispatcher := NewDispatcher(&executorStub{}) + dispatcher.Register(Type107, dataExecutor) + dispatcher.Register(Type41, trafficCutoffConfigExecutor) + runtime := NewRuntime(repository, dispatcher) + + if err := runtime.Handle(context.Background(), Message{BatchNo: "B-41", BatchType: Type41}); err != nil { + t.Fatalf("Handle() error = %v", err) + } + if trafficCutoffConfigExecutor.calls != 1 || trafficCutoffConfigExecutor.batchNo != "B-41" { + t.Fatalf("traffic cutoff config executor was not invoked: %+v", trafficCutoffConfigExecutor) + } + if dataExecutor.calls != 0 { + t.Fatalf("data executor calls = %d, want 0", dataExecutor.calls) + } +} + +func TestRuntimeIgnoresFinalizedTrafficCutoffConfigDuplicateMessage(t *testing.T) { + repository := &repositoryStub{ + mains: []*entity.BatchMain{{BatchNo: "B-41-DONE", BatchType: Type41, BatchAsyncStatus: AsyncStatusSuccess}}, + } + trafficCutoffConfigExecutor := &executorStub{err: errors.New("traffic cutoff config executor must not be called")} + dispatcher := NewDispatcher(&executorStub{}) + dispatcher.Register(Type41, trafficCutoffConfigExecutor) + runtime := NewRuntime(repository, dispatcher) + + if err := runtime.Handle(context.Background(), Message{BatchNo: "B-41-DONE", BatchType: Type41}); err != nil { + t.Fatalf("Handle() error = %v", err) + } + if trafficCutoffConfigExecutor.calls != 0 { + t.Fatalf("traffic cutoff config executor calls = %d, want 0", trafficCutoffConfigExecutor.calls) + } +} + +func TestRuntimeMarksExecutorFailureAsFailed(t *testing.T) { + expectedErr := errors.New("execute failed") + repository := &repositoryStub{ + mains: []*entity.BatchMain{{BatchNo: "B-104", BatchType: Type104, BatchAsyncStatus: AsyncStatusPending}}, + claimed: true, + } + executor := &executorStub{err: expectedErr} + runtime := NewRuntime(repository, NewDispatcher(executor)) + + if err := runtime.Handle(context.Background(), Message{BatchNo: "B-104", BatchType: Type104}); err != nil { + t.Fatalf("Handle() error = %v, want nil", err) + } + if repository.markCalls != 1 || repository.markBatchNo != "B-104" || repository.markType != Type104 { + t.Fatalf("unexpected claim call: %+v", repository) + } + if executor.calls != 1 || executor.batchNo != "B-104" { + t.Fatalf("unexpected executor call: %+v", executor) + } + if repository.failCalls != 1 || repository.failBatchNo != "B-104" || repository.failType != Type104 || repository.failReason != expectedErr.Error() { + t.Fatalf("unexpected failure mark call: %+v", repository) + } +} + +func TestRuntimeReturnsRetryableExecutorFailureWithoutFinalizing(t *testing.T) { + sourceErr := errors.New("source file is temporarily unavailable") + repository := &repositoryStub{ + mains: []*entity.BatchMain{{BatchNo: "INB-RETRY", BatchType: 1, BatchAsyncStatus: AsyncStatusPending}}, + claimed: true, + } + executor := &executorStub{err: Retryable(sourceErr)} + dispatcher := NewDispatcher(&executorStub{}) + dispatcher.Register(1, executor) + runtime := NewRuntime(repository, dispatcher) + + err := runtime.Handle(context.Background(), Message{BatchNo: "INB-RETRY", BatchType: 1}) + if !errors.Is(err, sourceErr) || !IsRetryable(err) { + t.Fatalf("Handle() error = %v, want retryable wrapper of %v", err, sourceErr) + } + if repository.failCalls != 0 { + t.Fatalf("MarkFailed calls = %d, want 0", repository.failCalls) + } +} + +func TestRuntimeReclaimsRetryableProcessingBatch(t *testing.T) { + repository := &repositoryStub{ + mains: []*entity.BatchMain{{BatchNo: "INB-RETRY", BatchType: 1, BatchAsyncStatus: AsyncStatusProcessing}}, + reclaimSet: true, + reclaimed: true, + } + executor := &executorStub{} + dispatcher := NewDispatcher(&executorStub{}) + dispatcher.Register(1, executor) + runtime := NewRuntime(repository, dispatcher) + + if err := runtime.Handle(context.Background(), Message{BatchNo: "INB-RETRY", BatchType: 1, Retryable: true}); err != nil { + t.Fatalf("Handle() error = %v", err) + } + if repository.reclaimCalls != 1 || repository.reclaimBatchNo != "INB-RETRY" || repository.reclaimType != 1 { + t.Fatalf("unexpected reclaim call: %+v", repository) + } + if executor.calls != 1 { + t.Fatalf("executor calls = %d, want 1", executor.calls) + } +} + +func TestRuntimeMarksTimeoutAsFailed(t *testing.T) { + repository := &repositoryStub{ + mains: []*entity.BatchMain{{BatchNo: "B-TIMEOUT", BatchType: Type104, BatchAsyncStatus: AsyncStatusPending}}, + claimed: true, + } + executor := &blockingExecutorStub{} + runtime := NewRuntime( + repository, + NewDispatcher(executor), + WithExecutionTimeout(time.Millisecond), + WithFinalizationTimeout(time.Second), + ) + + if err := runtime.Handle(context.Background(), Message{BatchNo: "B-TIMEOUT", BatchType: Type104}); err != nil { + t.Fatalf("Handle() error = %v, want nil", err) + } + if executor.calls != 1 { + t.Fatalf("executor calls = %d, want 1", executor.calls) + } + if repository.failCalls != 1 || !strings.Contains(repository.failReason, "timed out after 1ms") { + t.Fatalf("unexpected timeout failure mark: %+v", repository) + } +} + +func TestRuntimeReturnsFailureMarkErrors(t *testing.T) { + executeErr := errors.New("execute failed") + markErr := errors.New("mark failed") + repository := &repositoryStub{ + mains: []*entity.BatchMain{{BatchNo: "B-104", BatchType: Type104, BatchAsyncStatus: AsyncStatusPending}}, + claimed: true, + failErr: markErr, + } + runtime := NewRuntime(repository, NewDispatcher(&executorStub{err: executeErr})) + if err := runtime.Handle(context.Background(), Message{BatchNo: "B-104", BatchType: Type104}); !errors.Is(err, markErr) { + t.Fatalf("Handle() error = %v, want %v", err, markErr) + } +} + +func TestRuntimeCompensatesFailureMarkWithoutReexecutingBatch(t *testing.T) { + markErr := errors.New("mark failed") + repository := &repositoryStub{ + mains: []*entity.BatchMain{{BatchNo: "B-RETRY", BatchType: Type104, BatchAsyncStatus: AsyncStatusProcessing}}, + failErr: markErr, + } + executor := &executorStub{err: errors.New("must not execute again")} + runtime := NewRuntime(repository, NewDispatcher(executor)) + + err := runtime.Handle(context.Background(), Message{ + BatchNo: "B-RETRY", + BatchType: Type104, + FailureReason: "original executor failure", + }) + if !errors.Is(err, markErr) { + t.Fatalf("Handle() error = %v, want %v", err, markErr) + } + if executor.calls != 0 { + t.Fatalf("executor calls = %d, want 0 for failure finalization retry", executor.calls) + } + if repository.markCalls != 0 { + t.Fatalf("MarkProcessing calls = %d, want 0 for failure finalization retry", repository.markCalls) + } + if repository.failCalls != 1 || repository.failReason != "original executor failure" { + t.Fatalf("unexpected failure finalization call: %+v", repository) + } +} + +func TestRuntimeAllowsSelfClaimingProcessingReEntryWithoutRefreshingStartTime(t *testing.T) { + repository := &repositoryStub{ + mains: []*entity.BatchMain{{BatchNo: "RST-1", BatchType: Type105, BatchAsyncStatus: AsyncStatusProcessing}}, + } + executor := &processingExecutorStub{can: true, selfClaim: true} + dispatcher := NewDispatcher(&executorStub{}) + dispatcher.Register(Type105, executor) + runtime := NewRuntime(repository, dispatcher) + + if err := runtime.Handle(context.Background(), Message{BatchNo: "RST-1", BatchType: Type105}); err != nil { + t.Fatalf("Handle() error = %v", err) + } + if executor.canCalls != 1 || executor.main == nil || executor.main.BatchNo != "RST-1" { + t.Fatalf("processing check not called with main: %+v", executor) + } + if repository.markCalls != 0 { + t.Fatalf("MarkProcessing calls = %d, want 0", repository.markCalls) + } + if repository.reclaimCalls != 0 { + t.Fatalf("ReclaimProcessing calls = %d, want 0 for self-claiming executor", repository.reclaimCalls) + } + if executor.calls != 1 || executor.batchNo != "RST-1" { + t.Fatalf("executor calls = %+v", executor.executorStub) + } +} + +func TestRuntimeMarksProcessingReEntryFailureAsFailed(t *testing.T) { + expectedErr := errors.New("resume failed") + repository := &repositoryStub{ + mains: []*entity.BatchMain{{BatchNo: "RST-1", BatchType: Type105, BatchAsyncStatus: AsyncStatusProcessing}}, + } + executor := &processingExecutorStub{can: true} + executor.err = expectedErr + dispatcher := NewDispatcher(&executorStub{}) + dispatcher.Register(Type105, executor) + runtime := NewRuntime(repository, dispatcher) + + if err := runtime.Handle(context.Background(), Message{BatchNo: "RST-1", BatchType: Type105}); err != nil { + t.Fatalf("Handle() error = %v, want nil", err) + } + if repository.markCalls != 0 { + t.Fatalf("MarkProcessing calls = %d, want 0", repository.markCalls) + } + if repository.failCalls != 1 || repository.failBatchNo != "RST-1" || repository.failType != Type105 || repository.failReason != expectedErr.Error() { + t.Fatalf("unexpected failure mark call: %+v", repository) + } +} + +func TestRuntimeIgnoresProcessingWhenOptInExecutorDeclines(t *testing.T) { + repository := &repositoryStub{ + mains: []*entity.BatchMain{{BatchNo: "RST-2", BatchType: Type105, BatchAsyncStatus: AsyncStatusProcessing}}, + } + executor := &processingExecutorStub{can: false} + dispatcher := NewDispatcher(&executorStub{}) + dispatcher.Register(Type105, executor) + runtime := NewRuntime(repository, dispatcher) + + if err := runtime.Handle(context.Background(), Message{BatchNo: "RST-2", BatchType: Type105}); err != nil { + t.Fatalf("Handle() error = %v", err) + } + if executor.canCalls != 1 { + t.Fatalf("processing check calls = %d, want 1", executor.canCalls) + } + if executor.calls != 0 { + t.Fatalf("executor calls = %d, want 0", executor.calls) + } +} + +func TestRuntimeReturnsDatabaseErrors(t *testing.T) { + findErr := errors.New("find failed") + runtime := NewRuntime(&repositoryStub{findErr: findErr}, NewDispatcher(&executorStub{})) + if err := runtime.Handle(context.Background(), Message{BatchNo: "B-1", BatchType: Type103}); !errors.Is(err, findErr) { + t.Fatalf("find error = %v, want %v", err, findErr) + } + + markErr := errors.New("mark failed") + runtime = NewRuntime(&repositoryStub{ + mains: []*entity.BatchMain{{BatchNo: "B-1", BatchType: Type103, BatchAsyncStatus: AsyncStatusPending}}, + markErr: markErr, + }, NewDispatcher(&executorStub{})) + if err := runtime.Handle(context.Background(), Message{BatchNo: "B-1", BatchType: Type103}); !errors.Is(err, markErr) { + t.Fatalf("mark error = %v, want %v", err, markErr) + } + + checkErr := errors.New("check failed") + processing := &processingExecutorStub{canErr: checkErr} + dispatcher := NewDispatcher(&executorStub{}) + dispatcher.Register(Type105, processing) + runtime = NewRuntime(&repositoryStub{ + mains: []*entity.BatchMain{{BatchNo: "RST-3", BatchType: Type105, BatchAsyncStatus: AsyncStatusProcessing}}, + }, dispatcher) + if err := runtime.Handle(context.Background(), Message{BatchNo: "RST-3", BatchType: Type105}); !errors.Is(err, checkErr) { + t.Fatalf("processing check error = %v, want %v", err, checkErr) + } +} + +func TestMessageHasOnlyContractFields(t *testing.T) { + typeOfMessage := reflect.TypeOf(Message{}) + if typeOfMessage.NumField() != 4 { + t.Fatalf("Message has %d fields, want 4", typeOfMessage.NumField()) + } + if field, ok := typeOfMessage.FieldByName("FailureReason"); !ok || field.Tag.Get("json") != "failureReason,omitempty" { + t.Fatalf("FailureReason JSON contract is missing or incorrect") + } + if field, ok := typeOfMessage.FieldByName("Retryable"); !ok || field.Tag.Get("json") != "retryable,omitempty" { + t.Fatalf("Retryable JSON contract is missing or incorrect") + } +} diff --git a/internal/batch/runtime.go b/internal/batch/runtime.go new file mode 100644 index 0000000..cd1707c --- /dev/null +++ b/internal/batch/runtime.go @@ -0,0 +1,197 @@ +package batch + +import ( + "context" + "strings" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +type Repository interface { + FindMain(ctx context.Context, batchNo string) (*entity.BatchMain, error) + MarkProcessing(ctx context.Context, batchNo string, batchType int) (bool, error) + MarkFailed(ctx context.Context, batchNo string, batchType int, reason string) (bool, error) + MarkSuccess(ctx context.Context, batchNo string, batchType int, success, fail int, reason string) (bool, error) +} + +type Runtime struct { + repository Repository + dispatcher *Dispatcher + executionTimeout time.Duration + finalizationTimeout time.Duration +} + +func NewRuntime(repository Repository, dispatcher *Dispatcher, options ...RuntimeOption) *Runtime { + runtime := &Runtime{ + repository: repository, + dispatcher: dispatcher, + executionTimeout: 0, + finalizationTimeout: 10 * time.Second, + } + for _, option := range options { + option(runtime) + } + return runtime +} + +type RuntimeOption func(*Runtime) + +func WithExecutionTimeout(timeout time.Duration) RuntimeOption { + return func(runtime *Runtime) { + if timeout >= 0 { + runtime.executionTimeout = timeout + } + } +} + +func WithFinalizationTimeout(timeout time.Duration) RuntimeOption { + return func(runtime *Runtime) { + if timeout > 0 { + runtime.finalizationTimeout = timeout + } + } +} + +func (r *Runtime) Handle(ctx context.Context, message Message) error { + if strings.TrimSpace(message.BatchNo) == "" { + g.Log().Warning(ctx, "batch message dropped, empty batchNo") + return nil + } + executor, ok := r.dispatcher.Executor(message.BatchType) + if !ok { + g.Log().Warningf(ctx, "batch message dropped, unsupported batchType:%d, batchNo:%s", message.BatchType, message.BatchNo) + return nil + } + + main, err := r.repository.FindMain(ctx, message.BatchNo) + if err != nil { + return gerror.Wrap(err, "load batch main failed") + } + if main == nil { + g.Log().Warningf(ctx, "batch message ignored, batch not found, batchNo:%s", message.BatchNo) + return nil + } + if main.BatchType != message.BatchType { + g.Log().Warningf(ctx, "batch message ignored, type mismatch, batchNo:%s, messageType:%d, storedType:%d", message.BatchNo, message.BatchType, main.BatchType) + return nil + } + if strings.TrimSpace(message.FailureReason) != "" { + return r.markExecutionFailed(ctx, message, executor, message.FailureReason) + } + if main.BatchAsyncStatus != AsyncStatusPending { + if main.BatchAsyncStatus == AsyncStatusProcessing { + if message.Retryable { + return r.reclaimAndExecute(ctx, message, executor, main) + } + if processingExecutor, ok := executor.(ProcessingExecutor); ok { + canExecute, checkErr := processingExecutor.CanExecuteProcessing(ctx, main) + if checkErr != nil { + return gerror.Wrap(checkErr, "check processing batch re-entry failed") + } + if canExecute { + if selfClaimer, selfClaims := executor.(SelfClaimingProcessingExecutor); selfClaims && selfClaimer.ClaimsProcessingReentry() { + return r.execute(ctx, message, executor) + } + return r.reclaimAndExecute(ctx, message, executor, main) + } + } + } + logUnclaimable(ctx, main) + return nil + } + + claimed, err := r.repository.MarkProcessing(ctx, message.BatchNo, message.BatchType) + if err != nil { + return gerror.Wrap(err, "mark batch processing failed") + } + if !claimed { + current, findErr := r.repository.FindMain(ctx, message.BatchNo) + if findErr != nil { + return gerror.Wrap(findErr, "reload batch main after CAS failed") + } + if current == nil { + g.Log().Warningf(ctx, "batch CAS lost, batch not found, batchNo:%s", message.BatchNo) + return nil + } + if current.BatchType != message.BatchType { + g.Log().Warningf(ctx, "batch CAS lost, type mismatch, batchNo:%s, messageType:%d, storedType:%d", message.BatchNo, message.BatchType, current.BatchType) + return nil + } + logUnclaimable(ctx, current) + return nil + } + + g.Log().Infof(ctx, "batch claimed, batchNo:%s, batchType:%d", message.BatchNo, message.BatchType) + return r.execute(ctx, message, executor) +} + +func (r *Runtime) reclaimAndExecute(ctx context.Context, message Message, executor Executor, main *entity.BatchMain) error { + if claimer, claimSupported := r.repository.(ProcessingClaimer); claimSupported { + reclaimed, claimErr := claimer.ReclaimProcessing(ctx, message.BatchNo, message.BatchType, main.StartTime) + if claimErr != nil { + return gerror.Wrap(claimErr, "claim stale processing batch failed") + } + if !reclaimed { + g.Log().Infof(ctx, "batch re-entry claim lost, batchNo:%s", message.BatchNo) + return nil + } + } + return r.execute(ctx, message, executor) +} + +func (r *Runtime) execute(ctx context.Context, message Message, executor Executor) error { + executionCtx := ctx + var cancel context.CancelFunc = func() {} + if r.executionTimeout > 0 { + executionCtx, cancel = context.WithTimeout(ctx, r.executionTimeout) + } + defer cancel() + + if err := executor.Execute(executionCtx, message.BatchNo); err != nil { + if IsRetryable(err) { + return err + } + reason := err.Error() + if r.executionTimeout > 0 && executionCtx.Err() == context.DeadlineExceeded { + reason = gerror.Newf("batch execution timed out after %s", r.executionTimeout).Error() + } + return r.markExecutionFailed(ctx, message, executor, reason) + } + if r.executionTimeout > 0 && executionCtx.Err() == context.DeadlineExceeded { + return r.markExecutionFailed(ctx, message, executor, gerror.Newf("batch execution timed out after %s", r.executionTimeout).Error()) + } + return nil +} + +func (r *Runtime) markExecutionFailed(ctx context.Context, message Message, executor Executor, reason string) error { + finalizationCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), r.finalizationTimeout) + defer cancel() + + failed, markErr := r.repository.MarkFailed(finalizationCtx, message.BatchNo, message.BatchType, reason) + if markErr != nil { + return &FailureMarkError{Reason: reason, Err: markErr} + } + if handler, ok := executor.(ExecutionFailureHandler); ok { + if err := handler.HandleExecutionFailure(finalizationCtx, message.BatchNo, message.BatchType, reason); err != nil { + return &FailureMarkError{Reason: reason, Err: err} + } + } + if !failed { + g.Log().Warningf(ctx, "batch execution failed but status was already finalized, batchNo:%s, reason:%s", message.BatchNo, reason) + return nil + } + g.Log().Errorf(ctx, "batch execution failed and was marked failed, batchNo:%s, reason:%s", message.BatchNo, reason) + return nil +} + +func logUnclaimable(ctx context.Context, main *entity.BatchMain) { + if main.BatchAsyncStatus == AsyncStatusProcessing { + g.Log().Infof(ctx, "batch message ignored, batch already processing, batchNo:%s", main.BatchNo) + return + } + g.Log().Infof(ctx, "batch message ignored, batch is terminal or not pending, batchNo:%s, status:%d", main.BatchNo, main.BatchAsyncStatus) +} diff --git a/internal/client/upstream/client.go b/internal/client/upstream/client.go new file mode 100644 index 0000000..81fbc9f --- /dev/null +++ b/internal/client/upstream/client.go @@ -0,0 +1,241 @@ +package upstream + +import ( + "context" + "encoding/json" + "fmt" + "net/http" + "os" + "strings" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/net/gclient" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +const inboundBasicInfoClientNameEnv = "UPSTREAM_INBOUND_BASIC_INFO_CLIENT_NAME" +const upstreamProxyMaxTimeoutConfigKey = "upstream.proxy.maxTimeout" + +type Client struct { + currentUsageClientName string + currentUsageTimeout time.Duration + inboundBasicInfoClientName string + imeiClientName string + imeiTimeout time.Duration +} + +type response[T any] struct { + Code int `json:"code"` + Message string `json:"message"` + Data []model.SimBaseRes[T] `json:"data"` +} + +func New() *Client { + return &Client{} +} + +func (c *Client) QuerySimBasicInfo(ctx context.Context, input *model.SimBasicInfoInput) ([]model.SimBaseRes[model.SimBasicInfoItem], error) { + address, err := c.inboundBasicInfoAddress(ctx) + if err != nil { + return nil, err + } + return requestAllWithAddress[model.SimBasicInfoItem](ctx, "/sim/query/sim-basic-info", input, len(input.SimKeys), address) +} + +// NewWithCurrentUsageClient keeps the full-sync current-usage call independent +// from the configured upstream.proxy endpoint. +func NewWithCurrentUsageClient(address string, timeout ...time.Duration) *Client { + var currentUsageTimeout time.Duration + if len(timeout) > 0 { + currentUsageTimeout = timeout[0] + } + return &Client{currentUsageClientName: address, currentUsageTimeout: currentUsageTimeout} +} + +func NewWithInboundBasicInfoClient(address string) *Client { + return &Client{inboundBasicInfoClientName: address} +} + +// NewWithImeiClient pins the IMEI full-sync path to an explicit chan endpoint +// and timeout. An empty address falls back to the shared upstream.proxy +// configuration, which is only intended for local runs. +func NewWithImeiClient(address string, timeout ...time.Duration) *Client { + var imeiTimeout time.Duration + if len(timeout) > 0 { + imTimeout := timeout[0] + if imTimeout > 0 { + imeiTimeout = imTimeout + } + } + return &Client{imeiClientName: address, imeiTimeout: imeiTimeout} +} + +func (c *Client) inboundBasicInfoAddress(ctx context.Context) (string, error) { + if strings.TrimSpace(c.inboundBasicInfoClientName) != "" { + return strings.TrimSpace(c.inboundBasicInfoClientName), nil + } + if address := strings.TrimSpace(os.Getenv(inboundBasicInfoClientNameEnv)); address != "" { + return address, nil + } + config, ok, err := optionalUpstreamConfig(ctx, "upstream.gateway") + if err != nil { + return "", err + } + if !ok || strings.TrimSpace(config.Address) == "" { + return "", gerror.Newf("upstream gateway address is empty, configure upstream.gateway.address or %s", inboundBasicInfoClientNameEnv) + } + return strings.TrimSpace(config.Address), nil +} + +// QuerySimDataUsage calls Chan's current-month real-time usage endpoint. The +// response keeps one item per requested card so the caller can classify +// per-card errors without reissuing successful cards. +func (c *Client) QuerySimDataUsage(ctx context.Context, input *model.SimDataUsageInput) ([]model.SimBaseRes[model.SimDataUsageItem], error) { + return requestAllWithAddressTimeout[model.SimDataUsageItem](ctx, "/sim/query/sim-datausage", input, len(input.SimKeys), c.currentUsageClientName, c.currentUsageTimeout) +} + +// QuerySimCardStatus queries the carrier status for the supplied cards. The +// status queue currently sends one card per request, while retaining the +// response shape needed for future batch support. +func (c *Client) QuerySimCardStatus(ctx context.Context, input *model.SimCardStatusInput) ([]model.SimBaseRes[model.SimCardStatusItem], error) { + return requestAll[model.SimCardStatusItem](ctx, "/sim/query/sim-status", input, len(input.SimKeys)) +} + +// QuerySimImei queries the terminal IMEI for the supplied cards. The IMEI queue +// sends one card per request. +// +// It deliberately uses requestAll rather than request: a per-item carrier +// failure (unsupported template, missing value) must reach the handler as an +// item with a non-zero Code, not be silently dropped from the slice. +func (c *Client) QuerySimImei(ctx context.Context, input *model.SimImeiInput) ([]model.SimBaseRes[model.SimImeiItem], error) { + return requestAllWithAddressTimeout[model.SimImeiItem](ctx, "/sim/query/sim-imei", input, len(input.SimKeys), c.imeiClientName, c.imeiTimeout) +} + +func (c *Client) QuerySimDatausageDaily(ctx context.Context, input *model.SimDataUsageDailyInput) ([]model.SimBaseRes[model.SimDataUsageDailyOutputItem], error) { + return requestAll[model.SimDataUsageDailyOutputItem](ctx, "/sim/query/sim-datausage-daily", input, len(input.SimKeys)) +} + +func (c *Client) QuerySimDatausageMonthly(ctx context.Context, input *model.SimDataUsageMonthlyInput) ([]model.SimBaseRes[model.SimDataUsageMonthlyOutputItem], error) { + return requestAll[model.SimDataUsageMonthlyOutputItem](ctx, "/sim/query/sim-datausage-monthly", input, len(input.SimKeys)) +} + +// ChangeSimStatus returns one result for every upstream item, including failed items. +// Lifecycle batch execution needs those failures to be persisted to the matching detail. +func (c *Client) ChangeSimStatus(ctx context.Context, input *model.SimCardStatusChangeInput) ([]model.SimBaseRes[model.SimCardStatusChangeOutputItem], error) { + return requestAll[model.SimCardStatusChangeOutputItem](ctx, "/sim/change/sim-status", input, len(input.SimKeys)) +} + +// ChangeSimApnFunction returns one result for every upstream item, including failed items. +// Data service batch execution needs those failures to be persisted to the matching detail. +func (c *Client) ChangeSimApnFunction(ctx context.Context, input *model.SimCardApnChangeInput) ([]model.SimBaseRes[model.SimCardApnChangeOutputItem], error) { + return requestAll[model.SimCardApnChangeOutputItem](ctx, "/sim/change/sim-apn", input, len(input.SimKeys)) +} + +func request[T any](ctx context.Context, path string, input any, itemCount int) ([]model.SimBaseRes[T], error) { + items, err := requestAll[T](ctx, path, input, itemCount) + if err != nil { + return nil, err + } + + successItems := make([]model.SimBaseRes[T], 0, len(items)) + for _, item := range items { + if item.Code == 0 { + successItems = append(successItems, item) + } + } + return successItems, nil +} + +func requestAll[T any](ctx context.Context, path string, input any, itemCount int) ([]model.SimBaseRes[T], error) { + return requestAllWithAddress[T](ctx, path, input, itemCount, "") +} + +func requestAllWithAddress[T any](ctx context.Context, path string, input any, itemCount int, address string) ([]model.SimBaseRes[T], error) { + return requestAllWithAddressTimeout[T](ctx, path, input, itemCount, address, 0) +} + +func requestAllWithAddressTimeout[T any](ctx context.Context, path string, input any, itemCount int, address string, timeoutOverride time.Duration) ([]model.SimBaseRes[T], error) { + config, err := upstreamConfig(ctx, "upstream.proxy") + if err != nil { + return nil, err + } + if address == "" { + address = config.Address + } + if address == "" { + return nil, gerror.New("upstream proxy address is empty") + } + if itemCount < 1 { + itemCount = 1 + } + + client := g.Client() + timeout := timeoutOverride + if timeout <= 0 { + timeout = config.Timeout + } + if timeout <= 0 { + timeout = 10 * time.Second + } + if itemCount > 1 { + timeout *= time.Duration(itemCount) + } + if maxTimeout := g.Cfg().MustGet(ctx, upstreamProxyMaxTimeoutConfigKey, 2*time.Minute).Duration(); maxTimeout > 0 && timeout > maxTimeout { + timeout = maxTimeout + } + setClientTimeout(client, timeout) + clientResponse, err := client.Post(ctx, fmt.Sprintf("http://%s%s", address, path), input) + if err != nil { + return nil, gerror.Wrap(err, "request upstream failed") + } + defer clientResponse.Close() + + return decodeResponse[T](clientResponse.ReadAll()) +} + +func optionalUpstreamConfig(ctx context.Context, key string) (model.UpstreamChanConfig, bool, error) { + configValue, err := g.Cfg().Get(ctx, key) + if err != nil { + return model.UpstreamChanConfig{}, false, gerror.Wrapf(err, "read upstream configuration failed: %s", key) + } + if configValue == nil || configValue.IsNil() { + return model.UpstreamChanConfig{}, false, nil + } + config := model.UpstreamChanConfig{} + if err = configValue.Scan(&config); err != nil { + return model.UpstreamChanConfig{}, false, gerror.Wrapf(err, "parse upstream configuration failed: %s", key) + } + return config, true, nil +} + +func upstreamConfig(ctx context.Context, key string) (model.UpstreamChanConfig, error) { + config, ok, err := optionalUpstreamConfig(ctx, key) + if err != nil { + return model.UpstreamChanConfig{}, err + } + if !ok { + return model.UpstreamChanConfig{}, gerror.Newf("upstream configuration is missing: %s", key) + } + return config, nil +} + +func decodeResponse[T any](body []byte) ([]model.SimBaseRes[T], error) { + output := response[T]{} + if err := json.Unmarshal(body, &output); err != nil { + return nil, gerror.Wrap(err, "parse upstream response failed") + } + if output.Code != 0 { + return nil, gerror.New(output.Message) + } + return output.Data, nil +} + +func setClientTimeout(client *gclient.Client, timeout time.Duration) { + client.SetTimeout(timeout) + if transport, ok := client.Transport.(*http.Transport); ok { + transport.ResponseHeaderTimeout = timeout + } +} diff --git a/internal/client/upstream/client_test.go b/internal/client/upstream/client_test.go new file mode 100644 index 0000000..a3fa4a0 --- /dev/null +++ b/internal/client/upstream/client_test.go @@ -0,0 +1,133 @@ +package upstream + +import ( + "context" + "encoding/json" + "os" + "strings" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +func TestSimCardStatusChangeProtocol(t *testing.T) { + input := model.SimCardStatusChangeInput{ + SimKeys: []*model.SimKey{{ChanKey: "channel-a", ICCID: "89860001"}}, + ReqData: model.SimCardStatusChangeInputItem{OperType: "1"}, + } + body, err := json.Marshal(input) + if err != nil { + t.Fatal(err) + } + const want = `{"simKeys":[{"chanKey":"channel-a","iccid":"89860001","imsi":"","msisdn":""}],"reqData":{"operType":"1"}}` + if got := string(body); got != want { + t.Fatalf("request body = %s, want %s", got, want) + } +} + +func TestCurrentUsageConsumerProtocol(t *testing.T) { + input := model.SimDataUsageInput{ + SimKeys: []*model.SimKey{{ChanKey: "channel-a", ICCID: "89860001", IMSI: "460001", MSISDN: "13800000000"}}, + } + body, err := json.Marshal(input) + if err != nil { + t.Fatal(err) + } + const want = `{"simKeys":[{"chanKey":"channel-a","iccid":"89860001","imsi":"460001","msisdn":"13800000000"}],"reqData":{}}` + if got := string(body); got != want { + t.Fatalf("request body = %s, want %s", got, want) + } + client := NewWithCurrentUsageClient("iotlcmupstreamchan-http-svc:8110") + if client.currentUsageClientName != "iotlcmupstreamchan-http-svc:8110" { + t.Fatalf("current usage client = %q", client.currentUsageClientName) + } + if client.currentUsageTimeout != 0 { + t.Fatalf("default current usage timeout = %s, want unset", client.currentUsageTimeout) + } + configured := NewWithCurrentUsageClient("iotlcmupstreamchan-http-svc:8110", 90*time.Second) + if configured.currentUsageTimeout != 90*time.Second { + t.Fatalf("configured current usage timeout = %s, want 90s", configured.currentUsageTimeout) + } +} + +func TestInboundBasicInfoClientCanUseDedicatedEndpoint(t *testing.T) { + client := NewWithInboundBasicInfoClient("iotlcm.example.com:8110") + clientName, err := client.inboundBasicInfoAddress(context.Background()) + if err != nil { + t.Fatal(err) + } + if clientName != "iotlcm.example.com:8110" { + t.Fatalf("inbound basic info client = %q, want dedicated chan endpoint", clientName) + } +} + +func TestInboundBasicInfoClientCanUseEnvEndpoint(t *testing.T) { + const endpoint = "iotlcm.example.com:8110" + t.Setenv(inboundBasicInfoClientNameEnv, endpoint) + client := New() + clientName, err := client.inboundBasicInfoAddress(context.Background()) + if err != nil { + t.Fatal(err) + } + if clientName != endpoint { + t.Fatalf("inbound basic info client = %q, want env endpoint", clientName) + } +} + +func TestConfiguredInboundBasicInfoClientUsesDedicatedEndpoint(t *testing.T) { + if value := os.Getenv(inboundBasicInfoClientNameEnv); value != "" { + t.Setenv(inboundBasicInfoClientNameEnv, "") + } + client := New() + clientName, err := client.inboundBasicInfoAddress(context.Background()) + if err != nil { + t.Fatal(err) + } + if !strings.Contains(clientName, ":8110") { + t.Fatalf("inbound basic info client = %q, want dedicated chan endpoint", clientName) + } +} + +func TestSimCardApnChangeProtocol(t *testing.T) { + input := model.SimCardApnChangeInput{ + SimKeys: []*model.SimKey{{ChanKey: "channel-a", ICCID: "89860001"}}, + ReqData: model.SimCardApnChangeInputItem{OperType: "0", ApnName: "CMIOT"}, + } + body, err := json.Marshal(input) + if err != nil { + t.Fatal(err) + } + const want = `{"simKeys":[{"chanKey":"channel-a","iccid":"89860001","imsi":"","msisdn":""}],"reqData":{"operType":"0","apnName":"CMIOT"}}` + if got := string(body); got != want { + t.Fatalf("request body = %s, want %s", got, want) + } +} + +func TestDecodeSimCardApnChangeResponseKeepsFailedItems(t *testing.T) { + body := []byte(`{"code":0,"message":"ok","data":[{"code":0,"message":"success","simKey":{"iccid":"89860001"},"resData":{"orderNum":"order-1"}},{"code":4001,"message":"denied","simKey":{"iccid":"89860002"},"resData":{"orderNum":"order-2"}}]}`) + items, err := decodeResponse[model.SimCardApnChangeOutputItem](body) + if err != nil { + t.Fatal(err) + } + if len(items) != 2 { + t.Fatalf("item count = %d, want 2", len(items)) + } + if items[1].Code != 4001 || items[1].ResData.OrderNum != "order-2" { + t.Fatalf("failed item was not preserved: %+v", items[1]) + } +} + +func TestDecodeSimCardStatusChangeResponseKeepsFailedItems(t *testing.T) { + body := []byte(`{"code":0,"message":"ok","data":[{"code":0,"message":"success","simKey":{"iccid":"89860001"},"resData":{"orderNum":"order-1"}},{"code":4001,"message":"denied","simKey":{"iccid":"89860002"},"resData":{"orderNum":"order-2"}}]}`) + items, err := decodeResponse[model.SimCardStatusChangeOutputItem](body) + if err != nil { + t.Fatal(err) + } + if len(items) != 2 { + t.Fatalf("item count = %d, want 2", len(items)) + } + if items[1].Code != 4001 || items[1].ResData.OrderNum != "order-2" { + t.Fatalf("failed item was not preserved: %+v", items[1]) + } +} diff --git a/internal/cmd/cmd.go b/internal/cmd/cmd.go new file mode 100644 index 0000000..d07d1b9 --- /dev/null +++ b/internal/cmd/cmd.go @@ -0,0 +1,18 @@ +package cmd + +import ( + "context" + + "github.com/gogf/gf/v2/os/gcmd" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/worker" +) + +var Main = gcmd.Command{ + Name: "main", + Usage: "main", + Brief: "start RabbitMQ worker", + Func: func(ctx context.Context, _ *gcmd.Parser) error { + return worker.Run(ctx) + }, +} diff --git a/internal/consts/batch_submit.go b/internal/consts/batch_submit.go new file mode 100644 index 0000000..a9b989b --- /dev/null +++ b/internal/consts/batch_submit.go @@ -0,0 +1,11 @@ +package consts + +// BatchSubBatchSize 是批量开停数据 / 停复机执行时，单个上游通道每次请求携带的卡数量上限。 +// 原来一个通道整组一次性发给 upstream-chan，大批量会触发 worker 侧 HTTP 超时（默认 2 分钟上限）导致整批失败。 +// 按此值拆分为多个小请求后，每个请求都能在超时上限内返回。 +const BatchSubBatchSize = 100 + +// BatchSubBatchConcurrency 是同一通道内子批次的并发度，受信号量限制： +// 即最多同时向上游发起 BatchSubBatchConcurrency 个请求，等价于最多同时发起同样数量的运营商调用， +// 既避免串行过慢，也避免无节制并发打爆运营商限流。如需更保守可调小该值。 +const BatchSubBatchConcurrency = 5 diff --git a/internal/consts/lifecycle.go b/internal/consts/lifecycle.go new file mode 100644 index 0000000..03efb56 --- /dev/null +++ b/internal/consts/lifecycle.go @@ -0,0 +1,154 @@ +package consts + +func FormalPackageTypes() []int { return []int{PackageTypeCard, PackageTypePool} } + +func IsAddPackageType(packageType int) bool { + return packageType == PackageTypeCardAddPackage || packageType == PackageTypePoolAddPackage +} + +const ( + BatchTypeInbound = 1 + BatchTypeAllocate = 3 + BatchTypeCardAddPackage = 11 + BatchTypePoolAddPackage = 12 + BatchTypeSimCardStart = 103 + BatchTypeSimCardStop = 104 + BatchTypeSimCardRestart = 105 + BatchTypeSimCardCancel = 106 + BatchTypeSimCardUnsub = 108 + BatchTypeDataCut = 107 + BatchTypeDataStart = 109 + BatchTypeDataStop = 110 + BatchTypeSimWarningConfig = 40 + BatchTypeTrafficCutoffConfig = 41 + BatchTypeTrafficCutoff = BatchTypeTrafficCutoffConfig // Deprecated: use BatchTypeTrafficCutoffConfig. + + // 异步续费/变更扩展 + BatchTypePrepare = 4 // 预处理（renew/change 共用 prepare） + BatchTypeRenew = 101 // 卡续费 + BatchTypeChange = 102 // 卡变更套餐 + + // t_sim_card_bind_pkg.package_type 中，只有正式卡套餐和池套餐参与续费/变更判断。 + PackageTypeCard = 101 + PackageTypePool = 102 + PackageTypeCardAddPackage = 103 + PackageTypePoolAddPackage = 104 + + BatchSourceTypeBatchSelect = 0 + BatchSourceTypeRange = 1 + BatchSourceTypeBatchInput = 2 + BatchSourceTypeFile = 3 + BatchSourceTypeScheduled = 4 + + BatchSourceTypeExcel = BatchSourceTypeFile + BatchSourceTypeInput = BatchSourceTypeBatchInput + + BatchNoPrefixCardAddPackage = "ADP" + BatchNoPrefixPoolAddPackage = "PAP" + + SimCardStatusOperTypeStop = "0" + SimCardStatusOperTypeStart = "1" + + RestartPhaseCreated = "created" + RestartPhaseWaitingResume = "waiting_resume" + RestartPhaseResuming = "resuming" + RestartPhaseFinished = "finished" + + SimBossCardStatusActivated = 2 + SimBossCardStatusStopped = 4 + SimBossCardStatusCanceled = 9 + + SimApnOperTypeOpen = "0" + SimApnOperTypeStop = "1" + + SimApnNameDefault = "CMIOT" + + SimDataServiceStatusEnabled = 1 + SimDataServiceStatusDisabled = 0 + + BatchAsyncStatusPending = 1 + BatchAsyncStatusProcessing = 2 + BatchAsyncStatusSuccess = 3 + BatchAsyncStatusFailed = 4 + BatchAsyncStatusPartialSuccess = 5 + + BatchRunResultPending = 0 + BatchRunResultRunning = 1 + BatchRunResultSuccess = 2 + BatchRunResultFailed = 3 + BatchRunResultPartialSuccess = 4 + + BatchDetailStatusPending = 0 + BatchDetailStatusRunning = 1 + BatchDetailStatusSuccess = 2 + BatchDetailStatusFailed = 3 + + ResourceTypeExcel = 1 + DownloadStatusGenerated = 1 + + ResourceExportSceneCardAddPackage = 11 + ResourceExportScenePoolAddPackage = 12 + + // 卡套餐有效状态 + PackageStatusNone = 0 + PackageStatusPending = 1 + PackageStatusEffective = 2 + PackageStatusExpired = 3 + + // 订购记录有效状态（t_sim_card_bind_pkg.effective_status） + EffectiveStatusPending = 0 + EffectiveStatusEffective = 1 + EffectiveStatusExpired = 2 + EffectiveStatusTerminated = 3 + EffectiveStatusVoided = 4 + + // 退订状态（t_order.return_status） + OrderReturnStatusNone = 0 + OrderReturnStatusSuccess = 4 + OrderReturnStatusPartial = 3 + + // Renew/change order contract. These values mirror the Admin cardpackage + // domain and the production t_order/t_order_item enum contract. + OrderTypeCardSubscribe = 1 + OrderTypeCardRenew = 2 + OrderTypeCardChange = 3 + OrderStatusCompleted = 5 + CustomerTypeAgent = 1 + PayTypeOffline = 3 + PayTypeBalance = 4 + PayStatusPaid = 2 + ProductTypeCard = 1 + PayModeYearMonth = 1 + ShippingMethodNone = 3 + + PackageBusinessTypeSubscribe = 0 + PackageBusinessTypeChange = 1 + PackageBusinessTypeRenew = 2 + + ResourceBatchDownloadTaskSimCardAddPackage = "单卡加油包结果" + ResourceBatchDownloadTaskSimPoolAddPackage = "流量池加油包结果" + + SimCardAddPackageMaxFileSize = 50 << 20 + SimPoolAddPackageMaxFileSize = 50 << 20 + + SimCardAddPackageSubDirImportExcel = "sim-card-add-package/import" + SimCardAddPackageSubDirResultExcel = "sim-card-add-package/result" + SimPoolAddPackageSubDirImportExcel = "sim-pool-add-package/import" + SimPoolAddPackageSubDirResultExcel = "sim-pool-add-package/result" + + // 续费/变更结果目录 + SimCardRenewResultSubDir = "sim-card-renew/result" + SimCardChangeResultSubDir = "sim-card-change/result" + + // 续费/变更结果任务名（沿用 Admin 约定） + ResultTaskSimCardRenew = "卡续费结果" + ResultTaskSimCardChange = "卡变更套餐结果" + ResultTaskSimCardPrepare = "卡续费/变更预处理结果" +) + +// RenewPriceSource 续费/变更价格来源 +const ( + RenewPriceSourceSystem = "system" + RenewPriceSourceAgentDefault = "agentDefault" + RenewPriceSourceTarget = "target" +) diff --git a/internal/consts/package_duration.go b/internal/consts/package_duration.go new file mode 100644 index 0000000..5ddbb11 --- /dev/null +++ b/internal/consts/package_duration.go @@ -0,0 +1,42 @@ +package consts + +import "strings" + +const ( + BillingMethodMonth = 1 + BillingMethodYear = 2 + + RenewDurationUnitMonth = "month" + RenewDurationUnitYear = "year" +) + +func BillingMethodDurationUnit(billingMethod int) string { + if billingMethod == BillingMethodMonth { + return RenewDurationUnitMonth + } + return RenewDurationUnitYear +} + +func NormalizeRenewDurationUnit(unit string) string { + if strings.EqualFold(strings.TrimSpace(unit), RenewDurationUnitMonth) { + return RenewDurationUnitMonth + } + return RenewDurationUnitYear +} + +func RenewDurationUnitText(unit string) string { + if NormalizeRenewDurationUnit(unit) == RenewDurationUnitMonth { + return "个月" + } + return "年" +} + +func DurationValueToMonths(unit string, value int) int { + if value <= 0 { + return 0 + } + if NormalizeRenewDurationUnit(unit) == RenewDurationUnitMonth { + return value + } + return value * 12 +} diff --git a/internal/consts/queue.go b/internal/consts/queue.go new file mode 100644 index 0000000..79cd451 --- /dev/null +++ b/internal/consts/queue.go @@ -0,0 +1,31 @@ +package consts + +const ( + QueueTrafficUsageDailyRetry = "iotlcm.traffic_usage.daily.retry" + QueueTrafficUsageMonthlyRetry = "iotlcm.traffic_usage.monthly.retry" + QueueTrafficUsageMonthlyStart = "iotlcm.traffic_usage.monthly.start" + // QueueTrafficUsageDailyStart carries one command per full daily usage run. + // Admin publishes it; Worker owns the card scan and Redis production. + QueueTrafficUsageDailyStart = "iotlcm.traffic_usage.daily.start" + // QueueTrafficUsageMonthlyHistoryStart carries one command per full monthly + // usage history run, which writes the TDengine monthly table. It is distinct + // from QueueTrafficUsageMonthlyStart, which fills the MySQL current-month + // snapshot. + QueueTrafficUsageMonthlyHistoryStart = "iotlcm.traffic_usage.monthly_history.start" + QueueSimStatusQueueStart = "iotlcm.sim_status.queue.start" + QueueSimStatusQueueTask = "iotlcm.sim_status.queue.task" + QueueImeiQueueStart = "iotlcm.imei.queue.start" + QueueWorkOrderAutoClose = "iotlcm.work_order.auto_close" + QueueBalanceRechargeTimeout = "iotlcm.balance.recharge.timeout" + QueueBalanceRechargeDelay = "iotlcm.balance.recharge.delay" + QueueWorkerDiagnostic = "iotlcm.worker.diagnostic" + QueueBatchExecute = "iotlcm.batch.execute" + QueueBatchExecuteRetry = "iotlcm.batch.execute.retry" + QueueBatchExecuteDeadLetter = "iotlcm.batch.execute.dlq" + QueueBatchRestartResumeDelay = "iotlcm.batch.restart.resume.delay" + QueuePackageCustomerSync = "iotlcm.package_customer.sync" + QueueChannelRefresh = "iotlcm.channel.refresh" + // QueueBalanceRechargeDeadLetter 保存重试次数耗尽后仍失败的充值超时消息， + // 供人工排查（例如支付宝凭据未配置导致的永久失败），不再自动重投。 + QueueBalanceRechargeDeadLetter = "iotlcm.balance.recharge.dlq" +) diff --git a/internal/consumer/rabbitmq/balance_recharge.go b/internal/consumer/rabbitmq/balance_recharge.go new file mode 100644 index 0000000..014025f --- /dev/null +++ b/internal/consumer/rabbitmq/balance_recharge.go @@ -0,0 +1,103 @@ +package rabbitmq + +import ( + "context" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +// defaultBalanceRechargeMaxRetry 是充值超时消息的默认最大重试次数。 +// 支付宝凭据未配置等永久性失败会让 CheckRechargeTimeout 一直返回错误， +// 必须设置上限，否则消息会无限重投并刷屏日志。 +const defaultBalanceRechargeMaxRetry = 100 + +// 包级变量便于测试注入发布行为，与 trafficUsagePublish 保持一致。 +var balanceRechargePublish = mq.Publish +var balanceRechargePublishDelayed = mq.PublishDelayed + +func (c *Consumers) initBalanceRechargeTimeout(ctx context.Context) error { + if !mq.Enabled(ctx) { + g.Log().Info(ctx, "balance recharge timeout consumer skipped, RabbitMQ disabled") + return nil + } + if c.balance == nil { + return gerror.New("balance recharge handler is not configured") + } + retryDelay, err := getBalanceRechargeRetryDelay(ctx) + if err != nil { + return err + } + maxRetry, err := getBalanceRechargeMaxRetry(ctx) + if err != nil { + return err + } + c.balanceRechargeRetryDelay = retryDelay + c.balanceRechargeMaxRetry = maxRetry + if err := mq.DeclareQueue(ctx, consts.QueueBalanceRechargeDeadLetter); err != nil { + return gerror.Wrap(err, "declare balance recharge dead-letter queue failed") + } + if err := mq.ConsumeJSON[model.BalanceRechargeTimeoutMessage]( + ctx, + consts.QueueBalanceRechargeTimeout, + "iotlcm-balance-recharge-timeout", + c.handleBalanceRechargeTimeout, + ); err != nil { + return err + } + g.Log().Infof(ctx, "balance recharge timeout consumer started, queue:%s, retryDelay:%s, maxRetry:%d", consts.QueueBalanceRechargeTimeout, retryDelay, maxRetry) + return nil +} + +func (c *Consumers) handleBalanceRechargeTimeout(ctx context.Context, message model.BalanceRechargeTimeoutMessage) error { + if message.OutTradeNo == "" { + g.Log().Errorf(ctx, "balance recharge timeout message dropped, invalid message:%+v", message) + return nil + } + if c.balance == nil { + return gerror.New("balance recharge handler is not configured") + } + if err := c.balance.CheckRechargeTimeout(ctx, message.OutTradeNo); err != nil { + message.RetryCount++ + if shouldDeadLetterBalanceRecharge(message.RetryCount, c.balanceRechargeMaxRetry) { + if publishErr := balanceRechargePublish(ctx, consts.QueueBalanceRechargeDeadLetter, message); publishErr != nil { + return gerror.Wrap(publishErr, "publish balance recharge dead-letter message failed") + } + g.Log().Errorf(ctx, "balance recharge timeout retries exhausted and message sent to dead-letter queue, outTradeNo:%s, retryCount:%d, maxRetry:%d, err:%+v", message.OutTradeNo, message.RetryCount, c.balanceRechargeMaxRetry, err) + return nil + } + if retryErr := balanceRechargePublishDelayed(ctx, consts.QueueBalanceRechargeDelay, consts.QueueBalanceRechargeTimeout, c.balanceRechargeRetryDelay, message); retryErr != nil { + return gerror.Wrap(retryErr, "republish balance recharge timeout message failed") + } + g.Log().Warningf(ctx, "balance recharge timeout query failed and will retry, outTradeNo:%s, retryCount:%d, err:%+v", message.OutTradeNo, message.RetryCount, err) + } + return nil +} + +// shouldDeadLetterBalanceRecharge 判断本次失败后是否放弃重试。 +// retryCount 是自增后的重试次数，因此 maxRetry 表示最多重投多少次。 +func shouldDeadLetterBalanceRecharge(retryCount, maxRetry int) bool { + return retryCount > maxRetry +} + +func getBalanceRechargeRetryDelay(ctx context.Context) (time.Duration, error) { + value := g.Cfg().MustGet(ctx, "rabbitmq.balanceRecharge.retryDelay", "1m").String() + delay, err := time.ParseDuration(value) + if err != nil || delay <= 0 { + return 0, gerror.Newf("balance recharge retry delay is invalid: %q", value) + } + return delay, nil +} + +func getBalanceRechargeMaxRetry(ctx context.Context) (int, error) { + maxRetry := g.Cfg().MustGet(ctx, "rabbitmq.balanceRecharge.maxRetry", defaultBalanceRechargeMaxRetry).Int() + if maxRetry <= 0 { + return 0, gerror.Newf("balance recharge max retry is invalid: %d", maxRetry) + } + return maxRetry, nil +} diff --git a/internal/consumer/rabbitmq/balance_recharge_test.go b/internal/consumer/rabbitmq/balance_recharge_test.go new file mode 100644 index 0000000..e1569c2 --- /dev/null +++ b/internal/consumer/rabbitmq/balance_recharge_test.go @@ -0,0 +1,153 @@ +package rabbitmq + +import ( + "context" + "errors" + "testing" + "time" + + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gcfg" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +func stubBalanceRechargePublish(t *testing.T) { + t.Helper() + originalPublish, originalDelayed := balanceRechargePublish, balanceRechargePublishDelayed + t.Cleanup(func() { + balanceRechargePublish, balanceRechargePublishDelayed = originalPublish, originalDelayed + }) + balanceRechargePublish = func(context.Context, string, any) error { return nil } + balanceRechargePublishDelayed = func(context.Context, string, string, time.Duration, any) error { return nil } +} + +func setBalanceRechargeTestConfig(t *testing.T, content string) { + t.Helper() + oldAdapter := g.Cfg().GetAdapter() + adapter, err := gcfg.NewAdapterContent(content) + if err != nil { + t.Fatal(err) + } + g.Cfg().SetAdapter(adapter) + t.Cleanup(func() { g.Cfg().SetAdapter(oldAdapter) }) +} + +func TestHandleBalanceRechargeTimeoutRepublishesDelayedBelowMaxRetry(t *testing.T) { + stubBalanceRechargePublish(t) + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{err: errors.New("Alipay credentials are not configured")}) + consumers.balanceRechargeRetryDelay = time.Minute + consumers.balanceRechargeMaxRetry = 3 + + delayedCalls := 0 + balanceRechargePublishDelayed = func(_ context.Context, delayQueue, targetQueue string, delay time.Duration, data any) error { + delayedCalls++ + if delayQueue != consts.QueueBalanceRechargeDelay || targetQueue != consts.QueueBalanceRechargeTimeout || delay != time.Minute { + t.Fatalf("unexpected delayed publish: %s -> %s, delay:%s", delayQueue, targetQueue, delay) + } + message, ok := data.(model.BalanceRechargeTimeoutMessage) + if !ok || message.OutTradeNo != "ALI-1" || message.RetryCount != 3 { + t.Fatalf("unexpected retry payload: %+v", data) + } + return nil + } + deadLetterCalls := 0 + balanceRechargePublish = func(_ context.Context, queue string, _ any) error { + deadLetterCalls++ + return nil + } + + // 自增后达到上限仍未超过，应继续重投延迟队列。 + if err := consumers.handleBalanceRechargeTimeout(context.Background(), model.BalanceRechargeTimeoutMessage{OutTradeNo: "ALI-1", RetryCount: 2}); err != nil { + t.Fatalf("retry below cap returned error: %v", err) + } + if delayedCalls != 1 || deadLetterCalls != 0 { + t.Fatalf("delayedCalls=%d deadLetterCalls=%d, want 1 and 0", delayedCalls, deadLetterCalls) + } +} + +func TestHandleBalanceRechargeTimeoutDeadLettersAboveMaxRetry(t *testing.T) { + stubBalanceRechargePublish(t) + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{err: errors.New("Alipay credentials are not configured")}) + consumers.balanceRechargeRetryDelay = time.Minute + consumers.balanceRechargeMaxRetry = 3 + + delayedCalls := 0 + balanceRechargePublishDelayed = func(context.Context, string, string, time.Duration, any) error { + delayedCalls++ + return nil + } + deadLetterCalls := 0 + balanceRechargePublish = func(_ context.Context, queue string, data any) error { + deadLetterCalls++ + if queue != consts.QueueBalanceRechargeDeadLetter { + t.Fatalf("dead letter queue = %s, want %s", queue, consts.QueueBalanceRechargeDeadLetter) + } + message, ok := data.(model.BalanceRechargeTimeoutMessage) + if !ok || message.OutTradeNo != "ALI-1" || message.RetryCount != 4 { + t.Fatalf("unexpected dead-letter payload: %+v", data) + } + return nil + } + + if err := consumers.handleBalanceRechargeTimeout(context.Background(), model.BalanceRechargeTimeoutMessage{OutTradeNo: "ALI-1", RetryCount: 3}); err != nil { + t.Fatalf("dead-letter path returned error: %v", err) + } + if delayedCalls != 0 || deadLetterCalls != 1 { + t.Fatalf("delayedCalls=%d deadLetterCalls=%d, want 0 and 1", delayedCalls, deadLetterCalls) + } +} + +func TestHandleBalanceRechargeTimeoutReturnsPublishErrors(t *testing.T) { + stubBalanceRechargePublish(t) + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{err: errors.New("Alipay credentials are not configured")}) + consumers.balanceRechargeRetryDelay = time.Minute + consumers.balanceRechargeMaxRetry = 3 + + expected := errors.New("broker unavailable") + balanceRechargePublish = func(context.Context, string, any) error { return expected } + balanceRechargePublishDelayed = func(context.Context, string, string, time.Duration, any) error { return expected } + + // 重投失败必须向上返回错误，让消费框架 Nack 重新入队，不能静默丢失消息。 + if err := consumers.handleBalanceRechargeTimeout(context.Background(), model.BalanceRechargeTimeoutMessage{OutTradeNo: "ALI-1", RetryCount: 0}); !errors.Is(err, expected) { + t.Fatalf("delayed publish failure = %v, want %v", err, expected) + } + if err := consumers.handleBalanceRechargeTimeout(context.Background(), model.BalanceRechargeTimeoutMessage{OutTradeNo: "ALI-1", RetryCount: 3}); !errors.Is(err, expected) { + t.Fatalf("dead-letter publish failure = %v, want %v", err, expected) + } +} + +func TestGetBalanceRechargeMaxRetry(t *testing.T) { + t.Run("defaults when unconfigured", func(t *testing.T) { + setBalanceRechargeTestConfig(t, "rabbitmq:\n balanceRecharge:\n retryDelay: \"1m\"\n") + maxRetry, err := getBalanceRechargeMaxRetry(context.Background()) + if err != nil || maxRetry != defaultBalanceRechargeMaxRetry { + t.Fatalf("maxRetry = %d (err=%v), want %d", maxRetry, err, defaultBalanceRechargeMaxRetry) + } + }) + + t.Run("reads configured value", func(t *testing.T) { + setBalanceRechargeTestConfig(t, "rabbitmq:\n balanceRecharge:\n maxRetry: 7\n") + maxRetry, err := getBalanceRechargeMaxRetry(context.Background()) + if err != nil || maxRetry != 7 { + t.Fatalf("maxRetry = %d (err=%v), want 7", maxRetry, err) + } + }) + + for _, test := range []struct { + name string + value string + }{ + {name: "zero", value: "0"}, + {name: "negative", value: "-1"}, + {name: "non-numeric", value: "\"abc\""}, + } { + t.Run("rejects "+test.name, func(t *testing.T) { + setBalanceRechargeTestConfig(t, "rabbitmq:\n balanceRecharge:\n maxRetry: "+test.value+"\n") + if _, err := getBalanceRechargeMaxRetry(context.Background()); err == nil { + t.Fatalf("maxRetry=%s was accepted, want error", test.value) + } + }) + } +} diff --git a/internal/consumer/rabbitmq/batch.go b/internal/consumer/rabbitmq/batch.go new file mode 100644 index 0000000..ead05d8 --- /dev/null +++ b/internal/consumer/rabbitmq/batch.go @@ -0,0 +1,178 @@ +package rabbitmq + +import ( + "context" + "encoding/json" + "errors" + "strconv" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + amqp "github.com/rabbitmq/amqp091-go" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" +) + +const batchRetryCountHeader = "x-iotlcm-retry-count" + +func (c *Consumers) initBatchExecute(ctx context.Context) error { + if !mq.Enabled(ctx) { + g.Log().Info(ctx, "batch execute consumer skipped, RabbitMQ disabled") + return nil + } + if c.batch == nil { + return gerror.New("batch execute handler is not configured") + } + retryDelay, maxRetries, err := getBatchExecuteRetryConfig(ctx) + if err != nil { + return err + } + c.batchExecuteRetryDelay = retryDelay + c.batchExecuteMaxRetries = maxRetries + if err = mq.DeclareQueue(ctx, consts.QueueBatchExecuteDeadLetter); err != nil { + return gerror.Wrap(err, "declare batch execute dead-letter queue failed") + } + if err = mq.ConsumeJSON[batch.Message]( + ctx, + consts.QueueBatchExecute, + "iotlcm-batch-execute", + c.handleBatchExecute, + ); err != nil { + return err + } + g.Log().Infof(ctx, "batch execute consumer started, queue:%s, retryDelay:%s, maxRetries:%d", consts.QueueBatchExecute, retryDelay, maxRetries) + return nil +} + +func (c *Consumers) handleBatchExecute(ctx context.Context, message batch.Message) error { + if c.batch == nil { + return gerror.New("batch execute handler is not configured") + } + g.Log().Infof(ctx, "batch execute message received, batchNo:%s, batchType:%d", message.BatchNo, message.BatchType) + if err := c.batch.Handle(ctx, message); err != nil { + if _, delivered := mq.DeliveryMetadataFromContext(ctx); !delivered { + return err + } + if batch.IsRetryable(err) { + message.Retryable = true + } + var failureMarkErr *batch.FailureMarkError + if errors.As(err, &failureMarkErr) { + message.FailureReason = failureMarkErr.Reason + } + return c.retryOrDeadLetterBatchMessage(ctx, message, err) + } + return nil +} + +func (c *Consumers) retryOrDeadLetterBatchMessage(ctx context.Context, message batch.Message, handlerErr error) error { + metadata, _ := mq.DeliveryMetadataFromContext(ctx) + retryCount := batchRetryCount(metadata.Headers) + headers := amqp.Table{batchRetryCountHeader: retryCount + 1} + if shouldDeadLetterBatch(retryCount, c.batchExecuteMaxRetries) { + if message.Retryable { + finalizationMessage := message + finalizationMessage.Retryable = false + finalizationMessage.FailureReason = handlerErr.Error() + if err := c.batch.Handle(ctx, finalizationMessage); err != nil { + return gerror.Wrap(err, "finalize retryable batch failure before dead-lettering") + } + message.FailureReason = handlerErr.Error() + } + deadLetterBody, marshalErr := json.Marshal(batchDeadLetterMessage{ + Message: message, + RetryCount: retryCount, + Error: handlerErr.Error(), + FailedAt: time.Now().Format(time.RFC3339), + }) + if marshalErr != nil { + return gerror.Wrap(marshalErr, "marshal batch execute dead-letter message failed") + } + if err := mq.PublishBytesWithHeaders(ctx, consts.QueueBatchExecuteDeadLetter, deadLetterBody, nil); err != nil { + return gerror.Wrap(err, "publish batch execute dead-letter message failed") + } + g.Log().Errorf(ctx, "batch execute message sent to dead-letter queue, batchNo:%s, batchType:%d, retryCount:%d, err:%+v", message.BatchNo, message.BatchType, retryCount, handlerErr) + return nil + } + body, marshalErr := json.Marshal(message) + if marshalErr != nil { + return gerror.Wrap(marshalErr, "marshal batch execute retry message failed") + } + if err := mq.PublishDelayedBytesWithHeaders( + ctx, + consts.QueueBatchExecuteRetry, + consts.QueueBatchExecute, + c.batchExecuteRetryDelay, + body, + headers, + ); err != nil { + return gerror.Wrap(err, "republish batch execute retry message failed") + } + g.Log().Warningf(ctx, "batch execute message scheduled for retry, batchNo:%s, batchType:%d, retryCount:%d, delay:%s, err:%+v", message.BatchNo, message.BatchType, retryCount+1, c.batchExecuteRetryDelay, handlerErr) + return nil +} + +type batchDeadLetterMessage struct { + Message batch.Message `json:"message"` + RetryCount int `json:"retryCount"` + Error string `json:"error"` + FailedAt string `json:"failedAt"` +} + +func shouldDeadLetterBatch(retryCount, maxRetries int) bool { + return retryCount >= maxRetries +} + +func batchRetryCount(headers map[string]any) int { + if headers == nil { + return 0 + } + value, ok := headers[batchRetryCountHeader] + if !ok { + return 0 + } + switch count := value.(type) { + case int: + return max(count, 0) + case int8: + return max(int(count), 0) + case int16: + return max(int(count), 0) + case int32: + return max(int(count), 0) + case int64: + return max(int(count), 0) + case uint8: + return int(count) + case uint16: + return int(count) + case uint32: + return int(count) + case uint64: + if count > uint64(^uint(0)>>1) { + return int(^uint(0) >> 1) + } + return int(count) + case string: + parsed, err := strconv.Atoi(count) + if err == nil { + return max(parsed, 0) + } + } + return 0 +} + +func getBatchExecuteRetryConfig(ctx context.Context) (time.Duration, int, error) { + delay := g.Cfg().MustGet(ctx, "rabbitmq.batchExecute.retryDelay", "30s").Duration() + if delay <= 0 { + return 0, 0, gerror.New("batch execute retry delay must be positive") + } + maxRetries := g.Cfg().MustGet(ctx, "rabbitmq.batchExecute.maxRetries", 3).Int() + if maxRetries < 0 { + return 0, 0, gerror.New("batch execute max retries cannot be negative") + } + return delay, maxRetries, nil +} diff --git a/internal/consumer/rabbitmq/batch_test.go b/internal/consumer/rabbitmq/batch_test.go new file mode 100644 index 0000000..1ee79eb --- /dev/null +++ b/internal/consumer/rabbitmq/batch_test.go @@ -0,0 +1,48 @@ +package rabbitmq + +import "testing" + +func TestShouldDeadLetterBatch(t *testing.T) { + tests := []struct { + name string + retryCount int + maxRetries int + want bool + }{ + {name: "first attempt retries", retryCount: 0, maxRetries: 3, want: false}, + {name: "second attempt retries", retryCount: 1, maxRetries: 3, want: false}, + {name: "last configured retry retries", retryCount: 2, maxRetries: 3, want: false}, + {name: "next failure dead letters", retryCount: 3, maxRetries: 3, want: true}, + {name: "zero retries dead letters immediately", retryCount: 0, maxRetries: 0, want: true}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if actual := shouldDeadLetterBatch(tt.retryCount, tt.maxRetries); actual != tt.want { + t.Fatalf("shouldDeadLetterBatch(%d, %d) = %t, want %t", tt.retryCount, tt.maxRetries, actual, tt.want) + } + }) + } +} + +func TestBatchRetryCount(t *testing.T) { + tests := []struct { + name string + headers map[string]any + want int + }{ + {name: "missing", want: 0}, + {name: "int", headers: map[string]any{batchRetryCountHeader: 2}, want: 2}, + {name: "int64", headers: map[string]any{batchRetryCountHeader: int64(3)}, want: 3}, + {name: "uint", headers: map[string]any{batchRetryCountHeader: uint16(4)}, want: 4}, + {name: "string", headers: map[string]any{batchRetryCountHeader: "5"}, want: 5}, + {name: "negative", headers: map[string]any{batchRetryCountHeader: -1}, want: 0}, + {name: "invalid", headers: map[string]any{batchRetryCountHeader: "not-a-number"}, want: 0}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if actual := batchRetryCount(tt.headers); actual != tt.want { + t.Fatalf("batchRetryCount() = %d, want %d", actual, tt.want) + } + }) + } +} diff --git a/internal/consumer/rabbitmq/channel_refresh.go b/internal/consumer/rabbitmq/channel_refresh.go new file mode 100644 index 0000000..9448048 --- /dev/null +++ b/internal/consumer/rabbitmq/channel_refresh.go @@ -0,0 +1,243 @@ +package rabbitmq + +import ( + "context" + "encoding/json" + "fmt" + "net/url" + "strings" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +const channelRefreshConsumer = "iotlcm-worker-channel-refresh" + +type channelRefreshConfig struct { + Address string `json:"address"` + Timeout time.Duration `json:"timeout"` +} + +type channelRefreshResponse struct { + Code int `json:"code"` + Message string `json:"message"` +} + +func (c *Consumers) initChannelRefresh(ctx context.Context) error { + if !rabbitmq.Enabled(ctx) { + g.Log().Info(ctx, "channel refresh consumer skipped, RabbitMQ disabled") + return nil + } + if err := rabbitmq.ConsumeJSON[model.ChannelRefreshMessage]( + ctx, + consts.QueueChannelRefresh, + channelRefreshConsumer, + c.handleChannelRefresh, + ); err != nil { + return err + } + g.Log().Infof(ctx, "channel refresh consumer started, queue:%s", consts.QueueChannelRefresh) + return nil +} + +func (c *Consumers) handleChannelRefresh(ctx context.Context, message model.ChannelRefreshMessage) error { + switch strings.ToLower(strings.TrimSpace(message.Type)) { + case "channel": + if strings.TrimSpace(message.ChanKey) == "" { + g.Log().Errorf(ctx, "channel refresh message dropped, chanKey is empty") + return nil + } + if err := refreshChannelWithRetry(ctx, message.ChanKey); err != nil { + g.Log().Errorf(ctx, "channel refresh failed, chanKey=%s, err=%+v", message.ChanKey, err) + return err + } + return nil + case "template": + if strings.TrimSpace(message.TempKey) == "" { + g.Log().Errorf(ctx, "channel refresh message dropped, tempKey is empty") + return nil + } + return refreshTemplateChannels(ctx, message.TempKey) + case "channel_rename": + oldChanKey := strings.TrimSpace(message.OldChanKey) + newChanKey := strings.TrimSpace(message.NewChanKey) + if oldChanKey == "" || newChanKey == "" || oldChanKey == newChanKey { + g.Log().Errorf(ctx, "channel rename message dropped, oldChanKey=%q newChanKey=%q", oldChanKey, newChanKey) + return nil + } + if err := renameChannelWithRetry(ctx, oldChanKey, newChanKey); err != nil { + // 返回 error 让消费框架 Nack 并重新入队：rename 各步骤幂等， + // 重复投递安全；静默 Ack 会导致旧通道实例永久残留，只能重启恢复。 + g.Log().Errorf(ctx, "channel rename failed after retries, will requeue, oldChanKey=%s newChanKey=%s err=%+v", oldChanKey, newChanKey, err) + return err + } + return nil + default: + g.Log().Errorf(ctx, "channel refresh message dropped, unsupported type=%s", message.Type) + return nil + } +} + +func renameChannelWithRetry(ctx context.Context, oldChanKey, newChanKey string) error { + var err error + for attempt := 1; attempt <= 3; attempt++ { + err = renameChannel(ctx, oldChanKey, newChanKey) + if err == nil { + return nil + } + if attempt == 3 { + break + } + delay := time.Second + if attempt == 2 { + delay = 3 * time.Second + } + select { + case <-ctx.Done(): + return ctx.Err() + case <-time.After(delay): + } + } + return gerror.Wrapf(err, "channel rename failed after %d attempts, oldChanKey=%s newChanKey=%s", 3, oldChanKey, newChanKey) +} + +// renameChannel is ordered so that the old runtime remains available until +// both upstreams have built the new instance. Card data and ICCID routes are +// intentionally left untouched for operations staff to migrate manually. +func renameChannel(ctx context.Context, oldChanKey, newChanKey string) error { + if err := refreshChannel(ctx, newChanKey); err != nil { + return gerror.Wrap(err, "refresh new channel failed") + } + if err := removeChannelTarget(ctx, "upstream.gateway", oldChanKey); err != nil { + return gerror.Wrap(err, "remove old gateway channel failed") + } + if err := removeChannelTarget(ctx, "upstream.proxy", oldChanKey); err != nil { + return gerror.Wrap(err, "remove old proxy channel failed") + } + g.Log().Infof(ctx, "channel instance rebuilt after rename, oldChanKey=%s newChanKey=%s", oldChanKey, newChanKey) + return nil +} + +func refreshTemplateChannels(ctx context.Context, tempKey string) error { + var rows []struct { + ChanKey string `json:"chanKey"` + } + if err := g.DB().Model("t_sim_channels").Ctx(ctx). + Fields("chan_key"). + Where("temp_key", strings.TrimSpace(tempKey)). + Scan(&rows); err != nil { + return gerror.Wrap(err, "query template channels failed") + } + + var firstErr error + for _, row := range rows { + if err := refreshChannelWithRetry(ctx, row.ChanKey); err != nil { + g.Log().Errorf(ctx, "template channel refresh failed, tempKey=%s, chanKey=%s, err=%+v", tempKey, row.ChanKey, err) + if firstErr == nil { + firstErr = err + } + } + } + return firstErr +} + +func refreshChannelWithRetry(ctx context.Context, chanKey string) error { + var err error + for attempt := 1; attempt <= 3; attempt++ { + err = refreshChannel(ctx, chanKey) + if err == nil { + return nil + } + if attempt == 3 { + break + } + delay := time.Second + if attempt == 2 { + delay = 3 * time.Second + } + select { + case <-ctx.Done(): + return gerror.Wrapf(err, "channel refresh canceled after %d attempts, chanKey=%s", attempt, chanKey) + case <-time.After(delay): + } + } + return gerror.Wrapf(err, "channel refresh failed after %d attempts, chanKey=%s", 3, chanKey) +} + +func refreshChannel(ctx context.Context, chanKey string) error { + chanKey = strings.TrimSpace(chanKey) + if chanKey == "" { + return gerror.New("chanKey is required") + } + if err := refreshChannelTarget(ctx, "upstream.gateway", chanKey); err != nil { + return gerror.Wrap(err, "refresh gateway channel failed") + } + if err := refreshChannelTarget(ctx, "upstream.proxy", chanKey); err != nil { + return gerror.Wrap(err, "refresh proxy channel failed") + } + return nil +} + +func refreshChannelTarget(ctx context.Context, configKey, chanKey string) error { + return channelControlTarget(ctx, configKey, "/internal/channel/refresh", chanKey) +} + +func removeChannelTarget(ctx context.Context, configKey, chanKey string) error { + return channelControlTarget(ctx, configKey, "/internal/channel/remove", chanKey) +} + +func channelControlTarget(ctx context.Context, configKey, path, chanKey string) error { + configValue, err := g.Cfg().Get(ctx, configKey) + if err != nil { + return gerror.Wrapf(err, "read upstream configuration failed: %s", configKey) + } + if configValue == nil || configValue.IsNil() { + return gerror.Newf("upstream configuration is missing: %s", configKey) + } + config := channelRefreshConfig{} + if err = configValue.Scan(&config); err != nil { + return gerror.Wrapf(err, "parse upstream configuration failed: %s", configKey) + } + address := strings.TrimSpace(config.Address) + if address == "" { + return gerror.Newf("upstream address is empty: %s", configKey) + } + if config.Timeout <= 0 { + config.Timeout = 10 * time.Second + } + requestCtx, cancel := context.WithTimeout(ctx, config.Timeout) + defer cancel() + + client := g.Client() + client.SetTimeout(config.Timeout) + controlURL := fmt.Sprintf("http://%s%s?chanKey=%s", address, path, url.QueryEscape(chanKey)) + response, err := client.Post(requestCtx, controlURL, nil) + if err != nil { + return gerror.Wrapf(err, "request upstream channel control failed: target=%s path=%s", configKey, path) + } + defer response.Close() + + output := channelRefreshResponse{} + statusCode := 0 + if response.Response != nil { + statusCode = response.StatusCode + } + if err = json.Unmarshal(response.ReadAll(), &output); err != nil { + return gerror.Wrapf(err, "parse upstream channel control response failed: target=%s path=%s status=%d", configKey, path, statusCode) + } + if output.Code != 0 { + if output.Message == "" { + output.Message = "upstream channel control failed" + } + return gerror.Newf("upstream channel control failed: target=%s path=%s status=%d message=%s", configKey, path, statusCode, output.Message) + } + if statusCode < 200 || statusCode >= 300 { + return gerror.Newf("upstream channel control failed: target=%s path=%s status=%d", configKey, path, statusCode) + } + return nil +} diff --git a/internal/consumer/rabbitmq/channel_refresh_test.go b/internal/consumer/rabbitmq/channel_refresh_test.go new file mode 100644 index 0000000..dd3e905 --- /dev/null +++ b/internal/consumer/rabbitmq/channel_refresh_test.go @@ -0,0 +1,134 @@ +package rabbitmq + +import ( + "context" + "fmt" + "net/http" + "net/http/httptest" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gcfg" +) + +func setChannelRefreshTestConfig(t *testing.T, gateway, proxy string) { + t.Helper() + oldAdapter := g.Cfg().GetAdapter() + adapter, err := gcfg.NewAdapterContent(fmt.Sprintf(` +upstream: + gateway: + address: %q + timeout: 1s + proxy: + address: %q + timeout: 1s +`, gateway, proxy)) + if err != nil { + t.Fatal(err) + } + g.Cfg().SetAdapter(adapter) + t.Cleanup(func() { g.Cfg().SetAdapter(oldAdapter) }) +} + +func testServerAddress(server *httptest.Server) string { + return strings.TrimPrefix(server.URL, "http://") +} + +func TestRefreshChannelCallsGatewayAndProxy(t *testing.T) { + var gatewayCalls atomic.Int32 + var proxyCalls atomic.Int32 + newServer := func(counter *atomic.Int32) *httptest.Server { + return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, request *http.Request) { + counter.Add(1) + if request.URL.Path != "/internal/channel/refresh" || request.URL.Query().Get("chanKey") != "channel-a" { + t.Fatalf("unexpected request: %s", request.URL.String()) + } + _, _ = w.Write([]byte(`{"code":0,"message":"success"}`)) + })) + } + gateway := newServer(&gatewayCalls) + defer gateway.Close() + proxy := newServer(&proxyCalls) + defer proxy.Close() + setChannelRefreshTestConfig(t, testServerAddress(gateway), testServerAddress(proxy)) + + if err := refreshChannel(context.Background(), "channel-a"); err != nil { + t.Fatal(err) + } + if gatewayCalls.Load() != 1 || proxyCalls.Load() != 1 { + t.Fatalf("gateway calls=%d proxy calls=%d", gatewayCalls.Load(), proxyCalls.Load()) + } +} + +func TestRenameChannelOnlyRebuildsRuntimeInstances(t *testing.T) { + var ( + mu sync.Mutex + events []string + ) + newServer := func(target string) *httptest.Server { + return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, request *http.Request) { + mu.Lock() + events = append(events, target+":"+request.URL.Path+":"+request.URL.Query().Get("chanKey")) + mu.Unlock() + _, _ = w.Write([]byte(`{"code":0,"message":"success"}`)) + })) + } + gateway := newServer("gateway") + defer gateway.Close() + proxy := newServer("proxy") + defer proxy.Close() + setChannelRefreshTestConfig(t, testServerAddress(gateway), testServerAddress(proxy)) + + if err := renameChannel(context.Background(), "channel-old", "channel-new"); err != nil { + t.Fatal(err) + } + want := []string{ + "gateway:/internal/channel/refresh:channel-new", + "proxy:/internal/channel/refresh:channel-new", + "gateway:/internal/channel/remove:channel-old", + "proxy:/internal/channel/remove:channel-old", + } + if strings.Join(events, "\n") != strings.Join(want, "\n") { + t.Fatalf("rename events = %q, want %q", events, want) + } +} + +func TestChannelControlErrorIncludesTargetStatusAndMessage(t *testing.T) { + gateway := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusBadGateway) + _, _ = w.Write([]byte(`{"code":50,"message":"carrier unavailable"}`)) + })) + defer gateway.Close() + setChannelRefreshTestConfig(t, testServerAddress(gateway), "127.0.0.1:1") + + err := refreshChannelTarget(context.Background(), "upstream.gateway", "channel-a") + if err == nil { + t.Fatal("expected target error") + } + message := err.Error() + for _, part := range []string{"target=upstream.gateway", "status=502", "carrier unavailable"} { + if !strings.Contains(message, part) { + t.Fatalf("error %q does not contain %q", message, part) + } + } +} + +func TestHandleChannelRefreshReturnsError(t *testing.T) { + gateway := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + _, _ = w.Write([]byte(`{"code":50,"message":"refresh rejected"}`)) + })) + defer gateway.Close() + setChannelRefreshTestConfig(t, testServerAddress(gateway), "127.0.0.1:1") + + ctx, cancel := context.WithTimeout(context.Background(), 20*time.Millisecond) + defer cancel() + consumer := &Consumers{} + if err := consumer.handleChannelRefresh(ctx, model.ChannelRefreshMessage{Type: "channel", ChanKey: "channel-a"}); err == nil { + t.Fatal("refresh failure was acknowledged as success") + } +} diff --git a/internal/consumer/rabbitmq/consumer.go b/internal/consumer/rabbitmq/consumer.go new file mode 100644 index 0000000..4612193 --- /dev/null +++ b/internal/consumer/rabbitmq/consumer.go @@ -0,0 +1,175 @@ +package rabbitmq + +import ( + "context" + "time" + + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +type TrafficUsageHandler interface { + RetryDaily(context.Context, model.TrafficUsageDailyRetryMessage) error + RetryMonthly(context.Context, model.TrafficUsageMonthlyRetryMessage) error +} + +type WorkOrderHandler interface { + AutoCloseResolved(context.Context, model.WorkOrderAutoCloseMessage) error +} + +type BalanceRechargeHandler interface { + CheckRechargeTimeout(context.Context, string) error +} + +type BatchHandler interface { + Handle(context.Context, batch.Message) error +} + +type PackageCustomerSyncHandler interface { + SyncDeptPackages(context.Context, model.PackageCustomerSyncMessage) error +} + +type TrafficUsageMonthlyStartHandler interface { + Produce(context.Context, model.TrafficUsageMonthlyStartMessage) error +} + +type TrafficUsageDailyStartHandler interface { + Produce(context.Context, model.TrafficUsageDailyStartMessage) error +} + +type TrafficUsageMonthlyHistoryStartHandler interface { + Produce(context.Context, model.TrafficUsageMonthlyHistoryStartMessage) error +} + +type SimStatusQueueStartHandler interface { + Produce(context.Context, model.SimStatusQueueStartMessage) error +} + +type SimStatusQueueTaskHandler interface { + EnqueueCard(context.Context, model.SimStatusQueueTaskMessage) error +} + +type ImeiQueueStartHandler interface { + Produce(context.Context, model.ImeiQueueStartMessage) error +} + +type Consumers struct { + trafficUsage TrafficUsageHandler + workOrder WorkOrderHandler + balance BalanceRechargeHandler + balanceRechargeRetryDelay time.Duration + balanceRechargeMaxRetry int + batch BatchHandler + batchExecuteRetryDelay time.Duration + batchExecuteMaxRetries int + packageCustomerSync PackageCustomerSyncHandler + trafficUsageMonthlyStart TrafficUsageMonthlyStartHandler + trafficUsageDailyStart TrafficUsageDailyStartHandler + trafficUsageMonthlyHistoryStart TrafficUsageMonthlyHistoryStartHandler + simStatusQueueStart SimStatusQueueStartHandler + simStatusQueueTask SimStatusQueueTaskHandler + imeiQueueStart ImeiQueueStartHandler +} + +// WithDailyUsageStart adds the previous-day usage producer without changing any +// existing RabbitMQ consumer wiring. +func (c *Consumers) WithDailyUsageStart(start TrafficUsageDailyStartHandler) *Consumers { + c.trafficUsageDailyStart = start + return c +} + +// WithMonthlyHistoryStart adds the previous-month usage history producer without +// changing any existing RabbitMQ consumer wiring. +func (c *Consumers) WithMonthlyHistoryStart(start TrafficUsageMonthlyHistoryStartHandler) *Consumers { + c.trafficUsageMonthlyHistoryStart = start + return c +} + +// WithSimStatusQueue adds the independent status queue handlers without +// changing any existing RabbitMQ consumer wiring. +func (c *Consumers) WithSimStatusQueue(start SimStatusQueueStartHandler, task SimStatusQueueTaskHandler) *Consumers { + c.simStatusQueueStart = start + c.simStatusQueueTask = task + return c +} + +// WithImeiQueue adds the independent IMEI queue handler without changing any +// existing RabbitMQ consumer wiring. +func (c *Consumers) WithImeiQueue(start ImeiQueueStartHandler) *Consumers { + c.imeiQueueStart = start + return c +} + +func New(trafficUsage TrafficUsageHandler, workOrder WorkOrderHandler, balance BalanceRechargeHandler, batchHandlers ...BatchHandler) *Consumers { + consumers := &Consumers{ + trafficUsage: trafficUsage, + workOrder: workOrder, + balance: balance, + } + if len(batchHandlers) > 0 { + consumers.batch = batchHandlers[0] + } + return consumers +} + +func NewWithPackageCustomerSync(trafficUsage TrafficUsageHandler, workOrder WorkOrderHandler, balance BalanceRechargeHandler, packageCustomerSync PackageCustomerSyncHandler, batchHandlers ...BatchHandler) *Consumers { + consumers := New(trafficUsage, workOrder, balance, batchHandlers...) + consumers.packageCustomerSync = packageCustomerSync + return consumers +} + +func NewWithPackageCustomerSyncAndMonthlyUsage(trafficUsage TrafficUsageHandler, workOrder WorkOrderHandler, balance BalanceRechargeHandler, packageCustomerSync PackageCustomerSyncHandler, monthlyStart TrafficUsageMonthlyStartHandler, batchHandlers ...BatchHandler) *Consumers { + consumers := NewWithPackageCustomerSync(trafficUsage, workOrder, balance, packageCustomerSync, batchHandlers...) + consumers.trafficUsageMonthlyStart = monthlyStart + return consumers +} + +func (c *Consumers) Start(ctx context.Context) error { + initializers := c.initializers(g.Cfg().MustGet(ctx, "workerDiagnostic.enabled", false).Bool()) + for _, initialize := range initializers { + if err := initialize(ctx); err != nil { + return err + } + } + return nil +} + +func (c *Consumers) initializers(workerDiagnosticEnabled bool) []func(context.Context) error { + initializers := []func(context.Context) error{ + c.initTrafficUsageDailyRetry, + c.initTrafficUsageMonthlyRetry, + c.initWorkOrderAutoClose, + c.initBalanceRechargeTimeout, + c.initBatchExecute, + c.initChannelRefresh, + } + if c.packageCustomerSync != nil { + initializers = append(initializers, c.initPackageCustomerSync) + } + if c.trafficUsageMonthlyStart != nil { + initializers = append(initializers, c.initTrafficUsageMonthlyStart) + } + if c.trafficUsageDailyStart != nil { + initializers = append(initializers, c.initTrafficUsageDailyStart) + } + if c.trafficUsageMonthlyHistoryStart != nil { + initializers = append(initializers, c.initTrafficUsageMonthlyHistoryStart) + } + if c.simStatusQueueStart != nil && c.simStatusQueueTask != nil { + initializers = append(initializers, c.initSimStatusQueueStart, c.initSimStatusQueueTask) + } + if c.imeiQueueStart != nil { + initializers = append(initializers, c.initImeiQueueStart) + } + if workerDiagnosticEnabled { + initializers = append(initializers, c.initWorkerDiagnostic) + } + return initializers +} + +func (c *Consumers) Stop(ctx context.Context) error { + return mq.Close(ctx) +} diff --git a/internal/consumer/rabbitmq/consumer_test.go b/internal/consumer/rabbitmq/consumer_test.go new file mode 100644 index 0000000..58b25bd --- /dev/null +++ b/internal/consumer/rabbitmq/consumer_test.go @@ -0,0 +1,526 @@ +package rabbitmq + +import ( + "context" + "encoding/json" + "errors" + "io" + "os" + "reflect" + "testing" + "time" + + "github.com/gogf/gf/v2/frame/g" + amqp "github.com/rabbitmq/amqp091-go" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + dailyusageproducer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/producer/dailyusage" + monthlyhistoryproducer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/producer/monthlyusagehistory" + packagecustomerservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/packagecustomer" +) + +func TestMain(m *testing.M) { + g.Log().SetWriter(io.Discard) + os.Exit(m.Run()) +} + +type trafficUsageHandlerStub struct { + dailyCalls int + monthlyCalls int + dailyMessage model.TrafficUsageDailyRetryMessage + monthMessage model.TrafficUsageMonthlyRetryMessage + dailyErr error + monthlyErr error +} + +func (s *trafficUsageHandlerStub) RetryDaily(_ context.Context, message model.TrafficUsageDailyRetryMessage) error { + s.dailyCalls++ + s.dailyMessage = message + return s.dailyErr +} + +func (s *trafficUsageHandlerStub) RetryMonthly(_ context.Context, message model.TrafficUsageMonthlyRetryMessage) error { + s.monthlyCalls++ + s.monthMessage = message + return s.monthlyErr +} + +type workOrderHandlerStub struct { + calls int + message model.WorkOrderAutoCloseMessage + err error +} + +type balanceRechargeHandlerStub struct { + calls int + outTradeNo string + err error +} + +func (s *balanceRechargeHandlerStub) CheckRechargeTimeout(_ context.Context, outTradeNo string) error { + s.calls++ + s.outTradeNo = outTradeNo + return s.err +} + +type batchHandlerStub struct { + calls int + message batch.Message + err error +} + +func (s *batchHandlerStub) Handle(_ context.Context, message batch.Message) error { + s.calls++ + s.message = message + return s.err +} + +type packageCustomerSyncHandlerStub struct { + calls int + message model.PackageCustomerSyncMessage + err error +} + +type dailyStartHandlerStub struct { + calls int + message model.TrafficUsageDailyStartMessage + err error +} + +func (s *dailyStartHandlerStub) Produce(_ context.Context, message model.TrafficUsageDailyStartMessage) error { + s.calls++ + s.message = message + return s.err +} + +type monthlyHistoryStartHandlerStub struct { + calls int + message model.TrafficUsageMonthlyHistoryStartMessage + err error +} + +func (s *monthlyHistoryStartHandlerStub) Produce(_ context.Context, message model.TrafficUsageMonthlyHistoryStartMessage) error { + s.calls++ + s.message = message + return s.err +} + +func (s *packageCustomerSyncHandlerStub) SyncDeptPackages(_ context.Context, message model.PackageCustomerSyncMessage) error { + s.calls++ + s.message = message + return s.err +} + +func (s *workOrderHandlerStub) AutoCloseResolved(_ context.Context, message model.WorkOrderAutoCloseMessage) error { + s.calls++ + s.message = message + return s.err +} + +func TestHandleTrafficUsageDailyRetry(t *testing.T) { + stubTrafficPublish(t) + t.Run("invalid messages are dead lettered", func(t *testing.T) { + messages := []model.TrafficUsageDailyRetryMessage{ + {}, + {SimKey: &model.SimKey{}, Card: model.TrafficUsageSimCard{Iccid: "card"}}, + {SimKey: &model.SimKey{ICCID: "sim"}}, + } + for _, message := range messages { + handler := &trafficUsageHandlerStub{} + consumers := New(handler, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}) + if err := consumers.handleTrafficUsageDailyRetry(context.Background(), message); err != nil { + t.Fatalf("invalid message returned error: %v", err) + } + if handler.dailyCalls != 0 { + t.Fatalf("invalid message called handler %d times", handler.dailyCalls) + } + } + }) + + t.Run("valid message is delegated", func(t *testing.T) { + expectedErr := errors.New("daily failed") + expected := model.TrafficUsageDailyRetryMessage{ + SimKey: &model.SimKey{ICCID: "sim"}, + Card: model.TrafficUsageSimCard{Iccid: "sim", PackageCode: "package"}, + QueryDate: "20260714", + } + handler := &trafficUsageHandlerStub{dailyErr: expectedErr} + consumers := New(handler, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}) + if err := consumers.handleTrafficUsageDailyRetry(context.Background(), expected); err != nil { + t.Fatalf("expected delegated error, got %v", err) + } + if handler.dailyCalls != 1 || !reflect.DeepEqual(handler.dailyMessage, expected) { + t.Fatalf("daily message was not delegated exactly once: %+v", handler.dailyMessage) + } + }) +} + +func TestHandleTrafficUsageMonthlyRetry(t *testing.T) { + stubTrafficPublish(t) + t.Run("invalid messages are dead lettered", func(t *testing.T) { + messages := []model.TrafficUsageMonthlyRetryMessage{ + {}, + {SimKey: &model.SimKey{}, Card: model.TrafficUsageSimCard{Iccid: "card"}}, + {SimKey: &model.SimKey{ICCID: "sim"}}, + } + for _, message := range messages { + handler := &trafficUsageHandlerStub{} + consumers := New(handler, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}) + if err := consumers.handleTrafficUsageMonthlyRetry(context.Background(), message); err != nil { + t.Fatalf("invalid message returned error: %v", err) + } + if handler.monthlyCalls != 0 { + t.Fatalf("invalid message called handler %d times", handler.monthlyCalls) + } + } + }) + + t.Run("valid message is delegated", func(t *testing.T) { + expectedErr := errors.New("monthly failed") + expected := model.TrafficUsageMonthlyRetryMessage{ + SimKey: &model.SimKey{ICCID: "sim"}, + Card: model.TrafficUsageSimCard{Iccid: "sim", PackageCode: "package"}, + QueryDate: "202607", + } + handler := &trafficUsageHandlerStub{monthlyErr: expectedErr} + consumers := New(handler, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}) + if err := consumers.handleTrafficUsageMonthlyRetry(context.Background(), expected); err != nil { + t.Fatalf("expected delegated error, got %v", err) + } + if handler.monthlyCalls != 1 || !reflect.DeepEqual(handler.monthMessage, expected) { + t.Fatalf("monthly message was not delegated exactly once: %+v", handler.monthMessage) + } + }) +} + +func stubTrafficPublish(t *testing.T) { + t.Helper() + original, delayed := trafficUsagePublish, trafficUsagePublishDelayed + t.Cleanup(func() { trafficUsagePublish, trafficUsagePublishDelayed = original, delayed }) + trafficUsagePublish = func(context.Context, string, []byte, amqp.Table) error { return nil } + trafficUsagePublishDelayed = func(context.Context, string, string, time.Duration, []byte, amqp.Table) error { return nil } +} + +func TestHandleWorkOrderAutoClose(t *testing.T) { + handler := &workOrderHandlerStub{} + consumers := New(&trafficUsageHandlerStub{}, handler, &balanceRechargeHandlerStub{}) + if err := consumers.handleWorkOrderAutoClose(context.Background(), model.WorkOrderAutoCloseMessage{}); err != nil { + t.Fatalf("invalid message returned error: %v", err) + } + if handler.calls != 0 { + t.Fatalf("invalid message called handler %d times", handler.calls) + } + + expectedErr := errors.New("work order failed") + expected := model.WorkOrderAutoCloseMessage{WoNo: "WO-1", ResolveTimeTs: 1} + handler.err = expectedErr + if err := consumers.handleWorkOrderAutoClose(context.Background(), expected); !errors.Is(err, expectedErr) { + t.Fatalf("expected delegated error, got %v", err) + } + if handler.calls != 1 || !reflect.DeepEqual(handler.message, expected) { + t.Fatalf("work order message was not delegated exactly once: %+v", handler.message) + } +} + +func TestHandleBalanceRechargeTimeout(t *testing.T) { + handler := &balanceRechargeHandlerStub{} + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, handler) + if err := consumers.handleBalanceRechargeTimeout(context.Background(), model.BalanceRechargeTimeoutMessage{}); err != nil { + t.Fatalf("invalid message returned error: %v", err) + } + if handler.calls != 0 { + t.Fatalf("invalid message called handler %d times", handler.calls) + } + + message := model.BalanceRechargeTimeoutMessage{OutTradeNo: "BAL-1"} + if err := consumers.handleBalanceRechargeTimeout(context.Background(), message); err != nil { + t.Fatalf("valid message returned error: %v", err) + } + if handler.calls != 1 || handler.outTradeNo != message.OutTradeNo { + t.Fatalf("balance recharge message was not delegated exactly once: %+v", handler) + } +} + +func TestHandleBatchExecute(t *testing.T) { + expectedErr := errors.New("batch failed") + expected := batch.Message{BatchNo: "BATCH-1", BatchType: batch.Type103} + handler := &batchHandlerStub{err: expectedErr} + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}, handler) + + if err := consumers.handleBatchExecute(context.Background(), expected); !errors.Is(err, expectedErr) { + t.Fatalf("expected delegated error, got %v", err) + } + if handler.calls != 1 || !reflect.DeepEqual(handler.message, expected) { + t.Fatalf("batch message was not delegated exactly once: %+v", handler.message) + } +} + +func TestHandlePackageCustomerSync(t *testing.T) { + t.Run("invalid message is dropped", func(t *testing.T) { + handler := &packageCustomerSyncHandlerStub{} + consumers := NewWithPackageCustomerSync(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}, handler) + if err := consumers.handlePackageCustomerSync(context.Background(), model.PackageCustomerSyncMessage{}); err != nil { + t.Fatalf("invalid message returned error: %v", err) + } + if handler.calls != 0 { + t.Fatalf("invalid message called handler %d times", handler.calls) + } + }) + + t.Run("missing dept is dropped", func(t *testing.T) { + expected := model.PackageCustomerSyncMessage{TenantId: "000000", DeptId: 164, Trigger: "package_lifecycle", SourceBatchNo: "898604017"} + handler := &packageCustomerSyncHandlerStub{err: packagecustomerservice.ErrDeptNotFound} + consumers := NewWithPackageCustomerSync(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}, handler) + if err := consumers.handlePackageCustomerSync(context.Background(), expected); err != nil { + t.Fatalf("missing dept returned error: %v", err) + } + if handler.calls != 1 || !reflect.DeepEqual(handler.message, expected) { + t.Fatalf("package customer sync message was not delegated exactly once: %+v", handler.message) + } + }) + + t.Run("other errors are retried", func(t *testing.T) { + expectedErr := errors.New("database failed") + expected := model.PackageCustomerSyncMessage{TenantId: "000000", DeptId: 117} + handler := &packageCustomerSyncHandlerStub{err: expectedErr} + consumers := NewWithPackageCustomerSync(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}, handler) + if err := consumers.handlePackageCustomerSync(context.Background(), expected); !errors.Is(err, expectedErr) { + t.Fatalf("expected delegated error, got %v", err) + } + }) +} + +func TestHandleWorkerDiagnostic(t *testing.T) { + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}) + + if err := consumers.handleWorkerDiagnostic(context.Background(), model.WorkerDiagnosticMessage{}); err != nil { + t.Fatalf("invalid message returned error: %v", err) + } + + message := model.WorkerDiagnosticMessage{ + TraceID: "worker-test-1", + SentAt: 1780000000000, + Message: "worker ping", + } + if err := consumers.handleWorkerDiagnostic(context.Background(), message); err != nil { + t.Fatalf("valid message returned error: %v", err) + } +} + +func TestWorkerDiagnosticMessageJSONContract(t *testing.T) { + message := model.WorkerDiagnosticMessage{ + TraceID: "worker-test-1", + SentAt: 1780000000000, + Message: "worker ping", + } + body, err := json.Marshal(message) + if err != nil { + t.Fatalf("marshal diagnostic message: %v", err) + } + const expected = `{"traceId":"worker-test-1","sentAt":1780000000000,"message":"worker ping"}` + if string(body) != expected { + t.Fatalf("unexpected diagnostic JSON: %s", body) + } +} + +func TestChannelRefreshMessageJSONContract(t *testing.T) { + message := model.ChannelRefreshMessage{ + Type: "channel_rename", + OldChanKey: "old_channel", + NewChanKey: "new_channel", + } + body, err := json.Marshal(message) + if err != nil { + t.Fatalf("marshal channel rename message: %v", err) + } + const expected = `{"type":"channel_rename","oldChanKey":"old_channel","newChanKey":"new_channel"}` + if string(body) != expected { + t.Fatalf("unexpected channel rename JSON: %s", body) + } +} + +func TestHandleChannelRefreshDropsInvalidRename(t *testing.T) { + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}) + for _, message := range []model.ChannelRefreshMessage{ + {Type: "channel_rename"}, + {Type: "channel_rename", OldChanKey: "old_channel"}, + {Type: "channel_rename", NewChanKey: "new_channel"}, + {Type: "channel_rename", OldChanKey: "same", NewChanKey: "same"}, + } { + if err := consumers.handleChannelRefresh(context.Background(), message); err != nil { + t.Fatalf("invalid channel rename returned error: %v", err) + } + } +} + +func TestHandleChannelRefreshRenameFailureReturnsError(t *testing.T) { + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}) + ctx, cancel := context.WithCancel(context.Background()) + cancel() + message := model.ChannelRefreshMessage{ + Type: "channel_rename", + OldChanKey: "old_channel", + NewChanKey: "new_channel", + } + // ctx 已取消时 renameChannelWithRetry 会在首次重试等待处返回 ctx.Err()， + // 消费者必须把错误向上抛出，让框架 Nack 重新入队而不是 Ack 丢弃。 + if err := consumers.handleChannelRefresh(ctx, message); err == nil { + t.Fatalf("expected failed channel rename to return error for Nack+requeue") + } +} + +func TestHandleTrafficUsageDailyStart(t *testing.T) { + t.Run("invalid message is dropped", func(t *testing.T) { + for _, message := range []model.TrafficUsageDailyStartMessage{ + {}, + {RunID: "run-1"}, + {QueryDate: "20261009"}, + } { + handler := &dailyStartHandlerStub{} + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}).WithDailyUsageStart(handler) + if err := consumers.handleTrafficUsageDailyStart(context.Background(), message); err != nil { + t.Fatalf("invalid message returned error: %v", err) + } + if handler.calls != 0 { + t.Fatalf("invalid message called producer %d times", handler.calls) + } + } + }) + + t.Run("valid message is delegated", func(t *testing.T) { + expected := model.TrafficUsageDailyStartMessage{ + RunID: "daily-usage-20261009-abc", + Operation: model.OperationPreviousDayUsageFullSync, + QueryDate: "20261009", + TriggeredAt: "2026-10-10T06:00:00+08:00", + Scope: &model.SyncScope{Channels: []string{"217"}}, + } + handler := &dailyStartHandlerStub{} + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}).WithDailyUsageStart(handler) + if err := consumers.handleTrafficUsageDailyStart(context.Background(), expected); err != nil { + t.Fatalf("valid message returned error: %v", err) + } + if handler.calls != 1 || !reflect.DeepEqual(handler.message, expected) { + t.Fatalf("daily start message was not delegated exactly once: %+v", handler.message) + } + }) + + t.Run("an already running producer is acked", func(t *testing.T) { + handler := &dailyStartHandlerStub{err: dailyusageproducer.ErrAlreadyRunning} + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}).WithDailyUsageStart(handler) + if err := consumers.handleTrafficUsageDailyStart(context.Background(), model.TrafficUsageDailyStartMessage{RunID: "r", QueryDate: "20261009"}); err != nil { + t.Fatalf("ErrAlreadyRunning must be acked, got %v", err) + } + }) + + t.Run("a finished run is acked instead of being resurrected", func(t *testing.T) { + handler := &dailyStartHandlerStub{err: dailyusageproducer.ErrRunFinished} + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}).WithDailyUsageStart(handler) + if err := consumers.handleTrafficUsageDailyStart(context.Background(), model.TrafficUsageDailyStartMessage{RunID: "r", QueryDate: "20261009"}); err != nil { + t.Fatalf("ErrRunFinished must be acked, got %v", err) + } + }) + + t.Run("other errors are surfaced for requeue", func(t *testing.T) { + expectedErr := errors.New("redis unavailable") + handler := &dailyStartHandlerStub{err: expectedErr} + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}).WithDailyUsageStart(handler) + if err := consumers.handleTrafficUsageDailyStart(context.Background(), model.TrafficUsageDailyStartMessage{RunID: "r", QueryDate: "20261009"}); !errors.Is(err, expectedErr) { + t.Fatalf("expected the producer error to propagate, got %v", err) + } + }) +} + +// TestDailyUsageStartConsumerIsOptIn pins the assembly contract: without an +// explicit WithDailyUsageStart the initializer list must be unchanged, so a +// disabled daily chain cannot alter existing deployments. +func TestDailyUsageStartConsumerIsOptIn(t *testing.T) { + base := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}) + withDaily := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}).WithDailyUsageStart(&dailyStartHandlerStub{}) + if got, want := len(withDaily.initializers(false)), len(base.initializers(false))+1; got != want { + t.Fatalf("initializers = %d, want %d", got, want) + } +} + +func TestHandleTrafficUsageMonthlyHistoryStart(t *testing.T) { + t.Run("invalid message is dropped", func(t *testing.T) { + for _, message := range []model.TrafficUsageMonthlyHistoryStartMessage{ + {}, + {RunID: "run-1"}, + {QueryMonth: "202609"}, + } { + handler := &monthlyHistoryStartHandlerStub{} + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}).WithMonthlyHistoryStart(handler) + if err := consumers.handleTrafficUsageMonthlyHistoryStart(context.Background(), message); err != nil { + t.Fatalf("invalid message returned error: %v", err) + } + if handler.calls != 0 { + t.Fatalf("invalid message called producer %d times", handler.calls) + } + } + }) + + t.Run("valid message is delegated", func(t *testing.T) { + expected := model.TrafficUsageMonthlyHistoryStartMessage{ + RunID: "monthly-history-202609-abc", + Operation: model.OperationPreviousMonthUsageFullSync, + QueryMonth: "202609", + TriggeredAt: "2026-10-01T06:00:00+08:00", + Scope: &model.SyncScope{Channels: []string{"217"}}, + } + handler := &monthlyHistoryStartHandlerStub{} + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}).WithMonthlyHistoryStart(handler) + if err := consumers.handleTrafficUsageMonthlyHistoryStart(context.Background(), expected); err != nil { + t.Fatalf("valid message returned error: %v", err) + } + if handler.calls != 1 || !reflect.DeepEqual(handler.message, expected) { + t.Fatalf("monthly history start message was not delegated exactly once: %+v", handler.message) + } + }) + + t.Run("an already running producer is acked", func(t *testing.T) { + handler := &monthlyHistoryStartHandlerStub{err: monthlyhistoryproducer.ErrAlreadyRunning} + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}).WithMonthlyHistoryStart(handler) + if err := consumers.handleTrafficUsageMonthlyHistoryStart(context.Background(), model.TrafficUsageMonthlyHistoryStartMessage{RunID: "r", QueryMonth: "202609"}); err != nil { + t.Fatalf("ErrAlreadyRunning must be acked, got %v", err) + } + }) + + t.Run("a finished run is acked instead of being resurrected", func(t *testing.T) { + handler := &monthlyHistoryStartHandlerStub{err: monthlyhistoryproducer.ErrRunFinished} + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}).WithMonthlyHistoryStart(handler) + if err := consumers.handleTrafficUsageMonthlyHistoryStart(context.Background(), model.TrafficUsageMonthlyHistoryStartMessage{RunID: "r", QueryMonth: "202609"}); err != nil { + t.Fatalf("ErrRunFinished must be acked, got %v", err) + } + }) + + t.Run("other errors are surfaced for requeue", func(t *testing.T) { + expectedErr := errors.New("redis unavailable") + handler := &monthlyHistoryStartHandlerStub{err: expectedErr} + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}).WithMonthlyHistoryStart(handler) + if err := consumers.handleTrafficUsageMonthlyHistoryStart(context.Background(), model.TrafficUsageMonthlyHistoryStartMessage{RunID: "r", QueryMonth: "202609"}); !errors.Is(err, expectedErr) { + t.Fatalf("expected the producer error to propagate, got %v", err) + } + }) +} + +// TestMonthlyHistoryStartConsumerIsOptIn mirrors the daily chain: a disabled +// monthly-history chain must not add an initializer. +func TestMonthlyHistoryStartConsumerIsOptIn(t *testing.T) { + base := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}) + withHistory := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}).WithMonthlyHistoryStart(&monthlyHistoryStartHandlerStub{}) + if got, want := len(withHistory.initializers(false)), len(base.initializers(false))+1; got != want { + t.Fatalf("initializers = %d, want %d", got, want) + } +} + +func TestConsumerInitializersRespectWorkerDiagnosticSwitch(t *testing.T) { + consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{}) + if got := len(consumers.initializers(false)); got != 6 { + t.Fatalf("diagnostic disabled: expected 6 business consumers, got %d", got) + } + if got := len(consumers.initializers(true)); got != 7 { + t.Fatalf("diagnostic enabled: expected 7 consumers, got %d", got) + } +} diff --git a/internal/consumer/rabbitmq/diagnostic.go b/internal/consumer/rabbitmq/diagnostic.go new file mode 100644 index 0000000..50cec74 --- /dev/null +++ b/internal/consumer/rabbitmq/diagnostic.go @@ -0,0 +1,43 @@ +package rabbitmq + +import ( + "context" + + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +func (c *Consumers) initWorkerDiagnostic(ctx context.Context) error { + if !mq.Enabled(ctx) { + g.Log().Info(ctx, "worker diagnostic consumer skipped, RabbitMQ disabled") + return nil + } + if err := mq.ConsumeJSON[model.WorkerDiagnosticMessage]( + ctx, + consts.QueueWorkerDiagnostic, + "iotlcm-worker-diagnostic", + c.handleWorkerDiagnostic, + ); err != nil { + return err + } + g.Log().Infof(ctx, "worker diagnostic consumer started, queue:%s", consts.QueueWorkerDiagnostic) + return nil +} + +func (c *Consumers) handleWorkerDiagnostic(ctx context.Context, message model.WorkerDiagnosticMessage) error { + if message.TraceID == "" { + g.Log().Errorf(ctx, "worker diagnostic message dropped, invalid traceId, message:%+v", message) + return nil + } + g.Log().Infof( + ctx, + "worker diagnostic message consumed, traceId:%s, sentAt:%d, message:%s", + message.TraceID, + message.SentAt, + message.Message, + ) + return nil +} diff --git a/internal/consumer/rabbitmq/imei_queue_start.go b/internal/consumer/rabbitmq/imei_queue_start.go new file mode 100644 index 0000000..aa7e2c9 --- /dev/null +++ b/internal/consumer/rabbitmq/imei_queue_start.go @@ -0,0 +1,44 @@ +package rabbitmq + +import ( + "context" + "errors" + + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + imeiproducer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/producer/imei" +) + +func (c *Consumers) initImeiQueueStart(ctx context.Context) error { + if !mq.Enabled(ctx) { + g.Log().Info(ctx, "imei queue start consumer skipped, RabbitMQ disabled") + return nil + } + if err := mq.ConsumeJSON[model.ImeiQueueStartMessage](ctx, consts.QueueImeiQueueStart, "iotlcm-imei-queue-start", c.handleImeiQueueStart); err != nil { + return err + } + g.Log().Infof(ctx, "imei queue start consumer started, queue:%s", consts.QueueImeiQueueStart) + return nil +} + +func (c *Consumers) handleImeiQueueStart(ctx context.Context, message model.ImeiQueueStartMessage) error { + if message.RunID == "" { + g.Log().Errorf(ctx, "imei queue start message dropped, invalid message:%+v", message) + return nil + } + err := c.imeiQueueStart.Produce(ctx, message) + if errors.Is(err, imeiproducer.ErrAlreadyRunning) { + g.Log().Warningf(ctx, "imei queue start skipped because a run is active, runId:%s", message.RunID) + return nil + } + // A run that already finished must not be resurrected by a redelivered + // message. Ack it instead of retrying forever. + if errors.Is(err, imeiproducer.ErrRunFinished) { + g.Log().Warningf(ctx, "imei queue start skipped because run already finished, runId:%s", message.RunID) + return nil + } + return err +} diff --git a/internal/consumer/rabbitmq/package_customer_sync.go b/internal/consumer/rabbitmq/package_customer_sync.go new file mode 100644 index 0000000..3248769 --- /dev/null +++ b/internal/consumer/rabbitmq/package_customer_sync.go @@ -0,0 +1,53 @@ +package rabbitmq + +import ( + "context" + "errors" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + packagecustomerservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/packagecustomer" +) + +func (c *Consumers) initPackageCustomerSync(ctx context.Context) error { + if !mq.Enabled(ctx) { + g.Log().Info(ctx, "package customer sync consumer skipped, RabbitMQ disabled") + return nil + } + if c.packageCustomerSync == nil { + return gerror.New("package customer sync handler is not configured") + } + if err := mq.ConsumeJSON[model.PackageCustomerSyncMessage]( + ctx, + consts.QueuePackageCustomerSync, + "iotlcm-package-customer-sync", + c.handlePackageCustomerSync, + ); err != nil { + return err + } + g.Log().Infof(ctx, "package customer sync consumer started, queue:%s", consts.QueuePackageCustomerSync) + return nil +} + +func (c *Consumers) handlePackageCustomerSync(ctx context.Context, message model.PackageCustomerSyncMessage) error { + if c.packageCustomerSync == nil { + return gerror.New("package customer sync handler is not configured") + } + if message.TenantId == "" || message.DeptId <= 0 { + g.Log().Errorf(ctx, "package customer sync message dropped, invalid message:%+v", message) + return nil + } + if err := c.packageCustomerSync.SyncDeptPackages(ctx, message); err != nil { + if errors.Is(err, packagecustomerservice.ErrDeptNotFound) { + g.Log().Warningf(ctx, "package customer sync message dropped, dept not found, tenantId:%s, deptId:%d, trigger:%s, sourceBatchNo:%s", + message.TenantId, message.DeptId, message.Trigger, message.SourceBatchNo) + return nil + } + return err + } + return nil +} diff --git a/internal/consumer/rabbitmq/sim_status_queue_start.go b/internal/consumer/rabbitmq/sim_status_queue_start.go new file mode 100644 index 0000000..cc44cda --- /dev/null +++ b/internal/consumer/rabbitmq/sim_status_queue_start.go @@ -0,0 +1,44 @@ +package rabbitmq + +import ( + "context" + "errors" + + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + simstatusproducer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/producer/simstatus" +) + +func (c *Consumers) initSimStatusQueueStart(ctx context.Context) error { + if !mq.Enabled(ctx) { + g.Log().Info(ctx, "sim status queue start consumer skipped, RabbitMQ disabled") + return nil + } + if err := mq.ConsumeJSON[model.SimStatusQueueStartMessage](ctx, consts.QueueSimStatusQueueStart, "iotlcm-sim-status-queue-start", c.handleSimStatusQueueStart); err != nil { + return err + } + g.Log().Infof(ctx, "sim status queue start consumer started, queue:%s", consts.QueueSimStatusQueueStart) + return nil +} + +func (c *Consumers) handleSimStatusQueueStart(ctx context.Context, message model.SimStatusQueueStartMessage) error { + if message.RunID == "" { + g.Log().Errorf(ctx, "sim status queue start message dropped, invalid message:%+v", message) + return nil + } + err := c.simStatusQueueStart.Produce(ctx, message) + if errors.Is(err, simstatusproducer.ErrAlreadyRunning) { + g.Log().Warningf(ctx, "sim status queue start skipped because a run is active, runId:%s", message.RunID) + return nil + } + // A run that already finished must not be resurrected by a redelivered + // message. Ack it instead of retrying forever. + if errors.Is(err, simstatusproducer.ErrRunFinished) { + g.Log().Warningf(ctx, "sim status queue start skipped because run already finished, runId:%s", message.RunID) + return nil + } + return err +} diff --git a/internal/consumer/rabbitmq/sim_status_queue_task.go b/internal/consumer/rabbitmq/sim_status_queue_task.go new file mode 100644 index 0000000..04d57cf --- /dev/null +++ b/internal/consumer/rabbitmq/sim_status_queue_task.go @@ -0,0 +1,31 @@ +package rabbitmq + +import ( + "context" + + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +func (c *Consumers) initSimStatusQueueTask(ctx context.Context) error { + if !mq.Enabled(ctx) { + g.Log().Info(ctx, "sim status queue task consumer skipped, RabbitMQ disabled") + return nil + } + if err := mq.ConsumeJSON[model.SimStatusQueueTaskMessage](ctx, consts.QueueSimStatusQueueTask, "iotlcm-sim-status-queue-task", c.handleSimStatusQueueTask); err != nil { + return err + } + g.Log().Infof(ctx, "sim status queue task consumer started, queue:%s", consts.QueueSimStatusQueueTask) + return nil +} + +func (c *Consumers) handleSimStatusQueueTask(ctx context.Context, message model.SimStatusQueueTaskMessage) error { + if message.CardID <= 0 || message.ICCID == "" || message.ChanKey == "" || message.TempKey == "" { + g.Log().Errorf(ctx, "sim status queue task dropped, invalid message:%+v", message) + return nil + } + return c.simStatusQueueTask.EnqueueCard(ctx, message) +} diff --git a/internal/consumer/rabbitmq/traffic_usage_daily.go b/internal/consumer/rabbitmq/traffic_usage_daily.go new file mode 100644 index 0000000..fb5e7a4 --- /dev/null +++ b/internal/consumer/rabbitmq/traffic_usage_daily.go @@ -0,0 +1,42 @@ +package rabbitmq + +import ( + "context" + + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +func (c *Consumers) initTrafficUsageDailyRetry(ctx context.Context) error { + if !mq.Enabled(ctx) { + g.Log().Info(ctx, "traffic usage daily retry consumer skipped, RabbitMQ disabled") + return nil + } + if err := mq.ConsumeJSON[model.TrafficUsageDailyRetryMessage]( + ctx, + consts.QueueTrafficUsageDailyRetry, + "iotlcm-traffic-usage-daily", + c.handleTrafficUsageDailyRetry, + ); err != nil { + return err + } + g.Log().Infof(ctx, "traffic usage daily retry consumer started, queue:%s", consts.QueueTrafficUsageDailyRetry) + return nil +} + +func (c *Consumers) handleTrafficUsageDailyRetry(ctx context.Context, message model.TrafficUsageDailyRetryMessage) error { + err := validateTrafficUsageIdentity(message.SimKey, message.Card) + if err == nil { + err = c.trafficUsage.RetryDaily(ctx, message) + } + if err == nil { + return nil + } + message.Attempt++ + message.LastError = err.Error() + permanent := trafficUsageFailureSnapshot(err, &message.UsageMb) + return publishTrafficUsageFailure(ctx, consts.QueueTrafficUsageDailyRetry, message, message.Attempt, permanent) +} diff --git a/internal/consumer/rabbitmq/traffic_usage_daily_start.go b/internal/consumer/rabbitmq/traffic_usage_daily_start.go new file mode 100644 index 0000000..ee92ac2 --- /dev/null +++ b/internal/consumer/rabbitmq/traffic_usage_daily_start.go @@ -0,0 +1,49 @@ +package rabbitmq + +import ( + "context" + "errors" + + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + dailyusageproducer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/producer/dailyusage" +) + +func (c *Consumers) initTrafficUsageDailyStart(ctx context.Context) error { + if !mq.Enabled(ctx) { + g.Log().Info(ctx, "traffic usage daily start consumer skipped, RabbitMQ disabled") + return nil + } + if err := mq.ConsumeJSON[model.TrafficUsageDailyStartMessage]( + ctx, + consts.QueueTrafficUsageDailyStart, + "iotlcm-traffic-usage-daily-start", + c.handleTrafficUsageDailyStart, + ); err != nil { + return err + } + g.Log().Infof(ctx, "traffic usage daily start consumer started, queue:%s", consts.QueueTrafficUsageDailyStart) + return nil +} + +func (c *Consumers) handleTrafficUsageDailyStart(ctx context.Context, message model.TrafficUsageDailyStartMessage) error { + if message.RunID == "" || message.QueryDate == "" { + g.Log().Errorf(ctx, "traffic usage daily start message dropped, invalid message:%+v", message) + return nil + } + err := c.trafficUsageDailyStart.Produce(ctx, message) + if errors.Is(err, dailyusageproducer.ErrAlreadyRunning) { + g.Log().Warningf(ctx, "traffic usage daily start skipped because previous run is active, runId:%s", message.RunID) + return nil + } + // A run that already finished must not be resurrected by a redelivered + // message. Ack it instead of retrying forever. + if errors.Is(err, dailyusageproducer.ErrRunFinished) { + g.Log().Warningf(ctx, "traffic usage daily start skipped because run already finished, runId:%s", message.RunID) + return nil + } + return err +} diff --git a/internal/consumer/rabbitmq/traffic_usage_monthly.go b/internal/consumer/rabbitmq/traffic_usage_monthly.go new file mode 100644 index 0000000..3ba5061 --- /dev/null +++ b/internal/consumer/rabbitmq/traffic_usage_monthly.go @@ -0,0 +1,42 @@ +package rabbitmq + +import ( + "context" + + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +func (c *Consumers) initTrafficUsageMonthlyRetry(ctx context.Context) error { + if !mq.Enabled(ctx) { + g.Log().Info(ctx, "traffic usage monthly retry consumer skipped, RabbitMQ disabled") + return nil + } + if err := mq.ConsumeJSON[model.TrafficUsageMonthlyRetryMessage]( + ctx, + consts.QueueTrafficUsageMonthlyRetry, + "iotlcm-traffic-usage-monthly", + c.handleTrafficUsageMonthlyRetry, + ); err != nil { + return err + } + g.Log().Infof(ctx, "traffic usage monthly retry consumer started, queue:%s", consts.QueueTrafficUsageMonthlyRetry) + return nil +} + +func (c *Consumers) handleTrafficUsageMonthlyRetry(ctx context.Context, message model.TrafficUsageMonthlyRetryMessage) error { + err := validateTrafficUsageIdentity(message.SimKey, message.Card) + if err == nil { + err = c.trafficUsage.RetryMonthly(ctx, message) + } + if err == nil { + return nil + } + message.Attempt++ + message.LastError = err.Error() + permanent := trafficUsageFailureSnapshot(err, &message.UsageMb) + return publishTrafficUsageFailure(ctx, consts.QueueTrafficUsageMonthlyRetry, message, message.Attempt, permanent) +} diff --git a/internal/consumer/rabbitmq/traffic_usage_monthly_history_start.go b/internal/consumer/rabbitmq/traffic_usage_monthly_history_start.go new file mode 100644 index 0000000..89fcb3b --- /dev/null +++ b/internal/consumer/rabbitmq/traffic_usage_monthly_history_start.go @@ -0,0 +1,49 @@ +package rabbitmq + +import ( + "context" + "errors" + + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + monthlyhistoryproducer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/producer/monthlyusagehistory" +) + +func (c *Consumers) initTrafficUsageMonthlyHistoryStart(ctx context.Context) error { + if !mq.Enabled(ctx) { + g.Log().Info(ctx, "traffic usage monthly history start consumer skipped, RabbitMQ disabled") + return nil + } + if err := mq.ConsumeJSON[model.TrafficUsageMonthlyHistoryStartMessage]( + ctx, + consts.QueueTrafficUsageMonthlyHistoryStart, + "iotlcm-traffic-usage-monthly-history-start", + c.handleTrafficUsageMonthlyHistoryStart, + ); err != nil { + return err + } + g.Log().Infof(ctx, "traffic usage monthly history start consumer started, queue:%s", consts.QueueTrafficUsageMonthlyHistoryStart) + return nil +} + +func (c *Consumers) handleTrafficUsageMonthlyHistoryStart(ctx context.Context, message model.TrafficUsageMonthlyHistoryStartMessage) error { + if message.RunID == "" || message.QueryMonth == "" { + g.Log().Errorf(ctx, "traffic usage monthly history start message dropped, invalid message:%+v", message) + return nil + } + err := c.trafficUsageMonthlyHistoryStart.Produce(ctx, message) + if errors.Is(err, monthlyhistoryproducer.ErrAlreadyRunning) { + g.Log().Warningf(ctx, "traffic usage monthly history start skipped because previous run is active, runId:%s", message.RunID) + return nil + } + // A run that already finished must not be resurrected by a redelivered + // message. Ack it instead of retrying forever. + if errors.Is(err, monthlyhistoryproducer.ErrRunFinished) { + g.Log().Warningf(ctx, "traffic usage monthly history start skipped because run already finished, runId:%s", message.RunID) + return nil + } + return err +} diff --git a/internal/consumer/rabbitmq/traffic_usage_monthly_start.go b/internal/consumer/rabbitmq/traffic_usage_monthly_start.go new file mode 100644 index 0000000..2cc000e --- /dev/null +++ b/internal/consumer/rabbitmq/traffic_usage_monthly_start.go @@ -0,0 +1,49 @@ +package rabbitmq + +import ( + "context" + "errors" + + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + monthlyusageproducer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/producer/monthlyusage" +) + +func (c *Consumers) initTrafficUsageMonthlyStart(ctx context.Context) error { + if !mq.Enabled(ctx) { + g.Log().Info(ctx, "traffic usage monthly start consumer skipped, RabbitMQ disabled") + return nil + } + if err := mq.ConsumeJSON[model.TrafficUsageMonthlyStartMessage]( + ctx, + consts.QueueTrafficUsageMonthlyStart, + "iotlcm-traffic-usage-monthly-start", + c.handleTrafficUsageMonthlyStart, + ); err != nil { + return err + } + g.Log().Infof(ctx, "traffic usage monthly start consumer started, queue:%s", consts.QueueTrafficUsageMonthlyStart) + return nil +} + +func (c *Consumers) handleTrafficUsageMonthlyStart(ctx context.Context, message model.TrafficUsageMonthlyStartMessage) error { + if message.RunID == "" || message.QueryMonth == "" { + g.Log().Errorf(ctx, "traffic usage monthly start message dropped, invalid message:%+v", message) + return nil + } + err := c.trafficUsageMonthlyStart.Produce(ctx, message) + if errors.Is(err, monthlyusageproducer.ErrAlreadyRunning) { + g.Log().Warningf(ctx, "traffic usage monthly start skipped because previous run is active, runId:%s", message.RunID) + return nil + } + // A run that already finished must not be resurrected by a redelivered + // message. Ack it instead of retrying forever. + if errors.Is(err, monthlyusageproducer.ErrRunFinished) { + g.Log().Warningf(ctx, "traffic usage monthly start skipped because run already finished, runId:%s", message.RunID) + return nil + } + return err +} diff --git a/internal/consumer/rabbitmq/traffic_usage_retry.go b/internal/consumer/rabbitmq/traffic_usage_retry.go new file mode 100644 index 0000000..e34cce4 --- /dev/null +++ b/internal/consumer/rabbitmq/traffic_usage_retry.go @@ -0,0 +1,68 @@ +package rabbitmq + +import ( + "context" + "encoding/json" + "errors" + "time" + + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/trafficusage" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" +) + +var trafficUsagePublish = mq.PublishBytesWithHeaders +var trafficUsagePublishDelayed = mq.PublishDelayedBytesWithHeaders + +func validateTrafficUsageIdentity(simKey *model.SimKey, card model.TrafficUsageSimCard) error { + if simKey == nil || card.Iccid == "" || simKey.ICCID != card.Iccid { + return &trafficusage.Failure{Cause: gerror.New("invalid traffic usage card identity"), Permanent: true} + } + return nil +} + +func trafficUsageFailureSnapshot(err error, usage **float64) bool { + var failure *trafficusage.Failure + if errors.As(err, &failure) { + if failure.UsageMb != nil { + *usage = failure.UsageMb + } + return failure.Permanent + } + return false +} + +func publishTrafficUsageFailure(ctx context.Context, queue string, message any, attempt int, permanent bool) error { + ctx, cancel := context.WithTimeout(ctx, 15*time.Second) + defer cancel() + body, err := json.Marshal(message) + if err != nil { + return err + } + if permanent || attempt >= 10 { + g.Log().Errorf(ctx, "traffic usage retry exhausted or invalid, queue:%s, attempt:%d, payload:%s", queue, attempt, body) + err = trafficUsagePublish(ctx, queue+".dlq", body, nil) + return waitTrafficUsagePublishFailure(ctx, err) + } + delay := 30 * time.Second + if attempt >= 3 { + delay = 2 * time.Minute + } + if attempt >= 6 { + delay = 5 * time.Minute + } + err = trafficUsagePublishDelayed(ctx, queue+".delay."+delay.String(), queue, delay, body, nil) + return waitTrafficUsagePublishFailure(ctx, err) +} + +func waitTrafficUsagePublishFailure(ctx context.Context, err error) error { + if err != nil { + select { + case <-ctx.Done(): + case <-time.After(time.Second): + } + } + return err +} diff --git a/internal/consumer/rabbitmq/traffic_usage_retry_test.go b/internal/consumer/rabbitmq/traffic_usage_retry_test.go new file mode 100644 index 0000000..ed109db --- /dev/null +++ b/internal/consumer/rabbitmq/traffic_usage_retry_test.go @@ -0,0 +1,60 @@ +package rabbitmq + +import ( + "context" + "encoding/json" + "errors" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/trafficusage" + amqp "github.com/rabbitmq/amqp091-go" + "testing" + "time" +) + +func TestTrafficUsageRetryRetainsPayload(t *testing.T) { + stubTrafficPublish(t) + usage := 12.5 + handler := &trafficUsageHandlerStub{dailyErr: &trafficusage.Failure{Cause: errors.New("database down"), UsageMb: &usage}} + consumer := New(handler, nil, nil) + message := model.TrafficUsageDailyRetryMessage{SimKey: &model.SimKey{ICCID: "card"}, Card: model.TrafficUsageSimCard{Iccid: "card"}, QueryDate: "20260831"} + calls := 0 + trafficUsagePublishDelayed = func(_ context.Context, queue, target string, delay time.Duration, body []byte, _ amqp.Table) error { + calls++ + var retry model.TrafficUsageDailyRetryMessage + if err := json.Unmarshal(body, &retry); err != nil { + t.Fatal(err) + } + if retry.Attempt != 1 || retry.UsageMb == nil || *retry.UsageMb != usage || retry.QueryDate != message.QueryDate || retry.LastError == "" || delay != 30*time.Second || target != consts.QueueTrafficUsageDailyRetry || queue == target { + t.Fatalf("invalid retry: %+v", retry) + } + return nil + } + if err := consumer.handleTrafficUsageDailyRetry(context.Background(), message); err != nil || calls != 1 { + t.Fatalf("calls=%d err=%v", calls, err) + } + message.Attempt = 9 + dlqCalls := 0 + trafficUsagePublish = func(_ context.Context, queue string, body []byte, _ amqp.Table) error { + dlqCalls++ + if queue != consts.QueueTrafficUsageDailyRetry+".dlq" { + t.Fatal(queue) + } + var retry model.TrafficUsageDailyRetryMessage + if err := json.Unmarshal(body, &retry); err != nil { + t.Fatal(err) + } + if retry.Attempt != 10 || retry.UsageMb == nil { + t.Fatal("DLQ lost snapshot") + } + return nil + } + if err := consumer.handleTrafficUsageDailyRetry(context.Background(), message); err != nil || dlqCalls != 1 || calls != 1 { + t.Fatalf("DLQ not published: %v", err) + } + expected := errors.New("broker unavailable") + trafficUsagePublish = func(context.Context, string, []byte, amqp.Table) error { return expected } + if err := consumer.handleTrafficUsageDailyRetry(context.Background(), message); !errors.Is(err, expected) { + t.Fatal("failed publication must not ack original") + } +} diff --git a/internal/consumer/rabbitmq/work_order.go b/internal/consumer/rabbitmq/work_order.go new file mode 100644 index 0000000..6d4a75a --- /dev/null +++ b/internal/consumer/rabbitmq/work_order.go @@ -0,0 +1,53 @@ +package rabbitmq + +import ( + "context" + "time" + + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +func (c *Consumers) initWorkOrderAutoClose(ctx context.Context) error { + if scanner, ok := c.workOrder.(interface{ CloseOverdue(context.Context) error }); ok { + go func() { + ticker := time.NewTicker(5 * time.Minute) + defer ticker.Stop() + for { + if err := scanner.CloseOverdue(ctx); err != nil { + g.Log().Errorf(ctx, "工单到期补偿失败: %v", err) + } + select { + case <-ctx.Done(): + return + case <-ticker.C: + } + } + }() + } + if !mq.Enabled(ctx) { + g.Log().Info(ctx, "work order auto close consumer skipped, RabbitMQ disabled") + return nil + } + if err := mq.ConsumeJSON[model.WorkOrderAutoCloseMessage]( + ctx, + consts.QueueWorkOrderAutoClose, + "iotlcm-work-order-auto-close", + c.handleWorkOrderAutoClose, + ); err != nil { + return err + } + g.Log().Infof(ctx, "work order auto close consumer started, queue:%s", consts.QueueWorkOrderAutoClose) + return nil +} + +func (c *Consumers) handleWorkOrderAutoClose(ctx context.Context, message model.WorkOrderAutoCloseMessage) error { + if message.WoNo == "" { + g.Log().Errorf(ctx, "work order auto close message dropped, invalid message:%+v", message) + return nil + } + return c.workOrder.AutoCloseResolved(ctx, message) +} diff --git a/internal/consumer/zset/consumer.go b/internal/consumer/zset/consumer.go new file mode 100644 index 0000000..c5d4a5e --- /dev/null +++ b/internal/consumer/zset/consumer.go @@ -0,0 +1,1118 @@ +// Package zset contains the long-running Redis ZSet task consumer. It is an +// independent runtime role; wiring it into the process is intentionally left +// to the application bootstrap. +package zset + +import ( + "context" + "crypto/rand" + "encoding/hex" + "errors" + "fmt" + "strings" + "sync" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +var ( + ErrAlreadyRunning = errors.New("zset consumer is already running") + ErrLockLost = errors.New("zset consumer lock was lost") + ErrLeaseLost = errors.New("zset task lease was lost") +) + +// Handler is the boundary to the independent Chan gateway and result store. +type Handler interface { + Query(context.Context, queue.Task) Result + Apply(context.Context, []Result) error +} + +type Result struct { + Task queue.Task + UsageMB float64 + // CardStatus carries status-sync results without overloading the traffic + // usage field. It is additive so the existing monthly usage handler keeps + // its original contract. + CardStatus *int + // Imei carries the IMEI-sync value. It is separate from CardStatus because + // an IMEI is a string and because "synced with an empty value" is a + // meaningful state that Success alone cannot express. + Imei string + // ImeiSynced reports that the carrier answered successfully, so the card + // was covered by this polling pass. It is independent of Success: a + // successful-but-empty answer sets ImeiSynced without Success, which is how + // the write path records coverage while leaving the stored value untouched. + ImeiSynced bool + Code int + ErrCode string + Message string + ErrorClass string + Success bool + Retryable bool + BlockChannel bool + // Covered marks a terminal answer that says something definitive about the + // card but yields no value, such as "the carrier holds no record of this + // card" (code 14000) or "the carrier returned an empty IMEI". The run was + // covered for that card, so it is neither retried nor counted as a failure; + // Success stays false because nothing may be written to the stored value. + Covered bool +} + +// Config controls the bounded scheduling window. Values are deliberately +// explicit so production limits can be selected after the capacity PoC. +type Config struct { + SupplyObservation *supply.Tracker + LockKey string + LockTTL time.Duration + PollInterval time.Duration + LeaseTTL time.Duration + TaskTimeout time.Duration + ClaimBatch int + MobileConcurrency int + TelecomConcurrency int + GenericConcurrency int + // GenericQPS is the per-channel pacing rate used for unknown templates. + GenericQPS float64 + // TemplateQPS overrides the per-channel pacing rate for channel templates. + // A nil map uses the built-in template defaults. + TemplateQPS map[string]float64 + RunQueueEnabled bool + RoundRetryEnabled bool + MaxRounds int + FailureRetention time.Duration + // FailurePoolAlertThreshold emits a warning when one run's terminal failure + // pool exceeds this many entries. It is an observability guard, not a + // limiter: the run is never aborted, because a genuinely broken upstream + // must still leave a complete, reviewable failure list. Zero disables it. + FailurePoolAlertThreshold int64 + // ShouldRetryNextRound selects failures that are deferred into the next + // Run round. A nil predicate preserves the monthly-usage mobile_busy rule. + ShouldRetryNextRound func(Result) bool + OnFailure func(context.Context, Result) + // BlockRun optionally provides a run-scoped channel blocker. The legacy + // monthly consumer continues to use Store.BlockRun by default. + BlockRun func(context.Context, string, string) (int64, error) + // Status synchronization counts blocked queued tasks as terminal failures. + BlockedTasksAsFailures bool + EventPrefix string +} + +const ( + telecomTempKey = "sim_dx_v1" + mobileTempKey = "sim_yd_v5" + mobileMockKey = "sim_yd_mock_v5" + sharedTengyuTempKey = "sim_onelink_tengyu_v1" + sharedXuzhouMobileTempKey = "sim_onelink_xuzhou_mobile_v1" + sharedLelianTempKey = "sim_onelink_lelian_v1" + sharedQianxunTempKey = "sim_onelink_qianxun_v1" + sharedHenanTempKey = "sim_onelink_henan_xinjing_v1" + sharedDaiheTempKey = "sim_onelink_daihe_v1" + sharedBeixinTempKey = "sim_onelink_beixin_v1" + // lanshang_8000 / lanshang_8003 是蓝尚的 OneLink 共享 Token 模板。它们 + // 虽然名字不带 sim_onelink_ 前缀，但走的是同一套移动 OneLink 协议， + // 必须归入 carrierMobile：否则会落到 carrierGeneric 的单通道 1.5 QPS， + // 全量同步一张卡要 0.67 秒，6060 张卡要跑 1 小时以上。 + sharedLanshang8000TempKey = "lanshang_8000" + sharedLanshang8003TempKey = "lanshang_8003" + telecomQPS = 25.0 + mobileQPS = 30.0 + defaultGenericQPS = 1.5 +) + +type carrier string + +const ( + carrierMobile carrier = "mobile" + carrierTelecom carrier = "telecom" + carrierGeneric carrier = "generic" +) + +func (c *Config) setDefaults() { + if c.LockKey == "" { + c.LockKey = "flow:consumer:active" + } + if c.LockTTL <= 0 { + c.LockTTL = time.Minute + } + if c.PollInterval <= 0 { + c.PollInterval = time.Second + } + if c.LeaseTTL <= 0 { + c.LeaseTTL = time.Minute + } + if c.TaskTimeout <= 0 { + c.TaskTimeout = 30 * time.Second + } + if c.ClaimBatch <= 0 { + c.ClaimBatch = 100 + } + if c.MobileConcurrency <= 0 { + c.MobileConcurrency = 30 + } + if c.TelecomConcurrency <= 0 { + c.TelecomConcurrency = 10 + } + if c.GenericConcurrency <= 0 { + c.GenericConcurrency = 1 + } + if c.GenericQPS <= 0 { + c.GenericQPS = defaultGenericQPS + } + if len(c.TemplateQPS) > 0 { + normalized := make(map[string]float64, len(c.TemplateQPS)) + for key, qps := range c.TemplateQPS { + key = strings.ToLower(strings.TrimSpace(key)) + if key != "" && qps > 0 { + normalized[key] = qps + } + } + c.TemplateQPS = normalized + } + if c.RoundRetryEnabled { + c.RunQueueEnabled = true + } + if c.RunQueueEnabled && !c.RoundRetryEnabled { + c.MaxRounds = 1 + } else if c.MaxRounds <= 0 { + c.MaxRounds = 5 + } + if c.EventPrefix == "" { + c.EventPrefix = "monthly_usage" + } + if c.FailureRetention <= 0 { + c.FailureRetention = 30 * 24 * time.Hour + } +} + +// Consumer periodically enumerates registered channels and handles a bounded +// claim window for each one. It does not scan the database or implement any +// carrier protocol. +type Consumer struct { + store *queue.Store + handler Handler + config Config + owner string + pools *channelPools + pacer *channelPacer + runningMu sync.Mutex + runningChannels map[string]struct{} + channelGroup sync.WaitGroup + summaryMu sync.Mutex + roundSummaries map[string]*roundSummary +} + +type roundSummary struct { + runID string + round int + inputCount int + successCount int + failedCount int + deferredCount int + terminalFailedCount int + startedAt time.Time + lastBatchAt time.Time + batchCount int +} + +func New(store *queue.Store, handler Handler, config Config) (*Consumer, error) { + if store == nil { + return nil, fmt.Errorf("zset store is nil") + } + if handler == nil { + return nil, fmt.Errorf("zset handler is nil") + } + config.setDefaults() + if config.BlockRun == nil { + config.BlockRun = store.BlockRun + } + if config.LeaseTTL <= config.TaskTimeout { + return nil, fmt.Errorf("lease ttl must exceed task timeout") + } + return &Consumer{ + store: store, + handler: handler, + config: config, + owner: newOwner(), + pools: newChannelPools(config.MobileConcurrency, config.TelecomConcurrency, config.GenericConcurrency), + pacer: newChannelPacer(), + runningChannels: make(map[string]struct{}), + roundSummaries: make(map[string]*roundSummary), + }, nil +} + +// Run obtains a single active-consumer lease and processes until ctx is done. +// A second active instance receives ErrAlreadyRunning and does not claim work. +func (c *Consumer) Run(ctx context.Context) (err error) { + if ctx == nil { + ctx = context.Background() + } + startedAt := time.Now() + acquired, err := c.store.AcquireLock(ctx, c.config.LockKey, c.owner, c.config.LockTTL) + if err != nil { + g.Log().Errorf(ctx, "event=%s_consumer status=failed stage=lock durationMs=%d error=%q", c.config.EventPrefix, time.Since(startedAt).Milliseconds(), err.Error()) + return err + } + if !acquired { + g.Log().Warningf(ctx, "event=%s_consumer status=skipped reason=already_running durationMs=%d", c.config.EventPrefix, time.Since(startedAt).Milliseconds()) + return ErrAlreadyRunning + } + if observer := c.config.SupplyObservation; observer != nil { + observer.ConsumerActive.Store(true) + defer observer.ConsumerActive.Store(false) + } + g.Log().Infof(ctx, "event=%s_consumer status=started lockKey=%s", c.config.EventPrefix, c.config.LockKey) + + runCtx, cancel := context.WithCancel(ctx) + defer func() { + cancel() + c.channelGroup.Wait() + releaseCtx, releaseCancel := context.WithTimeout(context.Background(), time.Second) + defer releaseCancel() + _, _ = c.store.ReleaseLock(releaseCtx, c.config.LockKey, c.owner) + status := "stopped" + if err != nil { + status = "failed" + } + message := fmt.Sprintf("event=%s_consumer status=%s durationMs=%d error=%q", c.config.EventPrefix, status, time.Since(startedAt).Milliseconds(), errorMessage(err)) + if err != nil { + g.Log().Error(ctx, message) + return + } + g.Log().Info(ctx, message) + }() + + renewErr := make(chan error, 1) + go c.renewLock(runCtx, cancel, renewErr) + + for { + _, err := c.processOnce(runCtx) + if err != nil { + if ctx.Err() != nil { + return nil + } + select { + case lockErr := <-renewErr: + return lockErr + default: + } + return err + } + timer := time.NewTimer(c.config.PollInterval) + select { + case <-runCtx.Done(): + if ctx.Err() != nil { + return nil + } + select { + case err := <-renewErr: + return err + default: + return runCtx.Err() + } + case err := <-renewErr: + return err + case <-timer.C: + } + } +} + +func (c *Consumer) renewLock(ctx context.Context, cancel context.CancelFunc, errors chan<- error) { + interval := c.config.LockTTL / 3 + if interval <= 0 { + interval = time.Millisecond + } + ticker := time.NewTicker(interval) + defer ticker.Stop() + for { + select { + case <-ctx.Done(): + return + case <-ticker.C: + ok, err := c.store.RenewLock(ctx, c.config.LockKey, c.owner, c.config.LockTTL) + if err != nil { + select { + case errors <- err: + default: + } + cancel() + return + } + if !ok { + select { + case errors <- ErrLockLost: + default: + } + cancel() + return + } + } + } +} + +func (c *Consumer) processOnce(ctx context.Context) (bool, error) { + started := false + if c.config.RunQueueEnabled { + runs, err := c.store.ActiveRuns(ctx) + if err != nil { + return false, err + } + for _, runID := range runs { + // The transition is guarded by the Run metadata counters, so it is + // safe for every poller to attempt it. + meta, err := c.store.GetRunMeta(ctx, runID) + if err != nil { + return false, err + } + if c.config.RoundRetryEnabled { + maxRounds := c.config.MaxRounds + if meta.MaxRound > 0 { + maxRounds = meta.MaxRound + } + advanced, err := c.store.AdvanceRunRound(ctx, runID, maxRounds) + if err != nil { + return false, err + } + if advanced { + c.flushRoundSummary(ctx, runID, meta.CurrentRound) + } + } + channels, err := c.store.RunChannels(ctx, runID) + if err != nil { + return false, err + } + for _, chanKey := range channels { + started = c.startRunChannel(ctx, runID, chanKey) || started + } + if err := c.completeRun(ctx, runID); err != nil { + return false, err + } + } + } + channels, err := c.store.ActiveChannels(ctx) + if err != nil { + return false, err + } + if len(channels) == 0 { + return started, nil + } + + for _, chanKey := range channels { + started = c.startChannel(ctx, chanKey) || started + } + return started, nil +} + +func (c *Consumer) startRunChannel(ctx context.Context, runID, chanKey string) bool { + key := runID + "\x00" + chanKey + c.runningMu.Lock() + if _, exists := c.runningChannels[key]; exists { + c.runningMu.Unlock() + return false + } + c.runningChannels[key] = struct{}{} + c.channelGroup.Add(1) + c.runningMu.Unlock() + + go func() { + defer c.channelGroup.Done() + defer func() { + c.runningMu.Lock() + delete(c.runningChannels, key) + c.runningMu.Unlock() + }() + startedAt := time.Now() + for ctx.Err() == nil { + processed, err := c.processRunChannel(ctx, runID, chanKey) + if err != nil { + g.Log().Errorf(ctx, "event=%s_batch status=failed stage=claim_or_process runId=%s chanKey=%s durationMs=%d error=%q", c.config.EventPrefix, runID, chanKey, time.Since(startedAt).Milliseconds(), err.Error()) + return + } + if !processed { + return + } + } + }() + return true +} + +func (c *Consumer) processRunChannel(ctx context.Context, runID, chanKey string) (bool, error) { + now := time.Now() + claimed, err := c.store.ClaimRun(ctx, runID, chanKey, now, now.Add(c.config.LeaseTTL), c.config.ClaimBatch) + if err != nil { + return false, err + } + if len(claimed) == 0 { + if observer := c.config.SupplyObservation; observer != nil { + observer.Empty(runID, chanKey) + } + return false, nil + } + if observer := c.config.SupplyObservation; observer != nil { + observer.Processing(runID, chanKey, 1) + defer observer.Processing(runID, chanKey, -1) + } + return true, c.handleRunClaims(ctx, claimed) +} + +func (c *Consumer) handleRunClaims(ctx context.Context, claimed []queue.ClaimedTask) error { + if len(claimed) == 0 { + return nil + } + startedAt := time.Now() + channelCarrier, qps := policyForTemplate(claimed[0].Task.TempKey, c.config) + batchCtx, cancelBatch := context.WithCancel(ctx) + queryCtx, cancelQueries := context.WithCancel(batchCtx) + defer cancelQueries() + renewErr := make(chan error, 1) + renewDone := make(chan struct{}) + go c.renewBatchLeases(batchCtx, cancelBatch, claimed, renewErr, renewDone) + defer func() { cancelBatch(); <-renewDone }() + + var group sync.WaitGroup + resultsCh := make(chan Result, len(claimed)) + for _, task := range claimed { + task := task + group.Add(1) + go func() { + defer group.Done() + result := c.queryClaim(queryCtx, task, channelCarrier, qps) + if result.BlockChannel { + cancelQueries() + } + resultsCh <- result + }() + } + group.Wait() + close(resultsCh) + results := make([]Result, 0, len(claimed)) + blockChannel := false + for result := range resultsCh { + results = append(results, result) + blockChannel = blockChannel || result.BlockChannel + } + if err := currentBatchError(batchCtx, renewErr); err != nil { + return err + } + for _, result := range results { + if !result.Success && !result.Covered && c.config.OnFailure != nil { + c.config.OnFailure(ctx, result) + } + } + if err := c.handler.Apply(batchCtx, results); err != nil { + return gerror.Wrap(err, "apply run result batch") + } + if err := currentBatchError(batchCtx, renewErr); err != nil { + return err + } + + meta, err := c.store.GetRunMeta(batchCtx, claimed[0].Task.RunID) + if err != nil { + return err + } + maxRounds := c.config.MaxRounds + if meta.MaxRound > 0 { + maxRounds = meta.MaxRound + } + finalizedClaims := make([]queue.ClaimedTask, 0, len(results)) + outcomes := make([]queue.TaskOutcome, 0, len(results)) + deferred := 0 + for _, result := range results { + // A telecom channel-limit response is handled by BlockRunScoped below. + // Leave the triggering lease in place so the same atomic block operation + // marks it (and all other queued/leased channel tasks) channel_blocked; + // it must not become a mobile round retry or a card-level final failure. + if result.BlockChannel { + continue + } + roundRetry := result.Retryable && c.shouldRetryNextRound(result) + if result.Retryable && !roundRetry && !result.Success && !result.BlockChannel { + // Infrastructure and lease-recoverable failures intentionally stay + // leased; the existing lease expiry path will recover them. + continue + } + claimedTask := claimedTaskForResult(claimed, result) + if claimedTask.Task.TaskID == "" { + continue + } + outcome := queue.TaskOutcome{ + Success: result.Success, + Retryable: false, + ErrorClass: result.ErrorClass, + FailureType: result.ErrorClass, + Code: result.Code, + ErrCode: result.ErrCode, + Message: result.Message, + BlockChannel: result.BlockChannel, + Covered: result.Covered, + } + if c.config.RoundRetryEnabled && roundRetry && meta.CurrentRound < maxRounds { + outcome.Retry = true + deferred++ + } + finalizedClaims = append(finalizedClaims, claimedTask) + outcomes = append(outcomes, outcome) + } + finalized := 0 + if len(finalizedClaims) > 0 { + count, finalizeErr := c.store.FinalizeRunTasks(batchCtx, finalizedClaims, outcomes, maxRounds) + if finalizeErr != nil { + return finalizeErr + } + if count != len(finalizedClaims) { + return ErrLeaseLost + } + finalized = count + } + if blockChannel { + if c.config.BlockedTasksAsFailures { + for _, result := range results { + if !result.BlockChannel { + continue + } + removed, err := c.store.BlockRunScopedAsFailed(batchCtx, claimedTaskForResult(claimed, result), queue.TaskOutcome{Code: result.Code, ErrCode: result.ErrCode, Message: result.Message}) + if err != nil { + return err + } + if removed < 0 { + return ErrLeaseLost + } + c.recordRoundCounts(claimed[0].Task.RunID, meta.CurrentRound, int(removed), 0, 0, int(removed), startedAt) + break + } + } else if _, err := c.store.BlockRunScoped(batchCtx, claimed[0].Task.RunID, claimed[0].Task.ChanKey); err != nil { + return err + } + } + c.recordRoundBatch(claimed[0].Task.RunID, meta.CurrentRound, finalizedClaims, outcomes, startedAt) + if err := c.completeRun(batchCtx, claimed[0].Task.RunID); err != nil { + return err + } + g.Log().Infof(ctx, "event=%s_round_batch status=completed runId=%s queryMonth=%s chanKey=%s tempKey=%s round=%d claimed=%d success=%d covered=%d failed=%d deferred=%d finalized=%d durationMs=%d", c.config.EventPrefix, claimed[0].Task.RunID, claimed[0].Task.QueryMonth, claimed[0].Task.ChanKey, claimed[0].Task.TempKey, meta.CurrentRound, len(claimed), countSuccessful(results), countCovered(results), len(results)-countSuccessful(results)-countCovered(results), deferred, finalized, time.Since(startedAt).Milliseconds()) + return nil +} + +func (c *Consumer) recordRoundBatch(runID string, round int, claimed []queue.ClaimedTask, outcomes []queue.TaskOutcome, startedAt time.Time) { + if len(claimed) == 0 || len(claimed) != len(outcomes) { + return + } + success, failed, deferred := 0, 0, 0 + for _, outcome := range outcomes { + if outcome.Success || outcome.Covered { + // A covered task answered definitively and produced no failure, so + // it closes the round like a success. Keeping it out of both buckets + // would break the enqueued = success + nextRound + failed identity + // that the run archive validates. + success++ + continue + } + failed++ + if outcome.Retry { + deferred++ + } + } + c.recordRoundCounts(runID, round, len(claimed), success, deferred, failed-deferred, startedAt) +} + +func (c *Consumer) recordRoundCounts(runID string, round, input, success, deferred, terminalFailed int, startedAt time.Time) { + if input <= 0 { + return + } + key := runID + "\x00" + fmt.Sprint(round) + now := time.Now() + c.summaryMu.Lock() + summary := c.roundSummaries[key] + if summary == nil { + summary = &roundSummary{runID: runID, round: round, startedAt: startedAt} + c.roundSummaries[key] = summary + } + if startedAt.Before(summary.startedAt) { + summary.startedAt = startedAt + } + summary.inputCount += input + summary.successCount += success + summary.failedCount += deferred + terminalFailed + summary.deferredCount += deferred + summary.terminalFailedCount += terminalFailed + summary.batchCount++ + summary.lastBatchAt = now + c.summaryMu.Unlock() +} + +func (c *Consumer) flushRoundSummary(ctx context.Context, runID string, round int) { + key := runID + "\x00" + fmt.Sprint(round) + c.summaryMu.Lock() + summary := c.roundSummaries[key] + c.summaryMu.Unlock() + if summary == nil { + return + } + if summary.lastBatchAt.IsZero() { + summary.lastBatchAt = time.Now() + } + completedAt := summary.lastBatchAt + stats, err := c.store.GetRoundStats(ctx, runID) + if err != nil { + g.Log().Warningf(ctx, "event=%s_round_summary status=failed runId=%s round=%d stage=read_existing error=%q", c.config.EventPrefix, runID, round, err.Error()) + return + } + startedAt := summary.startedAt + entry := queue.RoundStats{ + RunID: runID, + Round: round, + EnqueuedCount: int64(summary.inputCount), + NextRoundCount: int64(summary.deferredCount), + SuccessCount: int64(summary.successCount), + FailedCount: int64(summary.terminalFailedCount), + BatchCount: int64(summary.batchCount), + StartedAt: &startedAt, + CompletedAt: &completedAt, + } + found := false + for i := range stats { + if stats[i].Round == round { + stats[i] = entry + found = true + break + } + } + if !found { + stats = append(stats, entry) + } + if err := c.store.SaveRoundStats(ctx, runID, stats); err != nil { + g.Log().Warningf(ctx, "event=%s_round_summary status=failed runId=%s round=%d stage=persist error=%q", c.config.EventPrefix, runID, round, err.Error()) + return + } + c.summaryMu.Lock() + delete(c.roundSummaries, key) + c.summaryMu.Unlock() + g.Log().Infof(ctx, "event=%s_round_summary status=completed runId=%s round=%d inputCount=%d successCount=%d failedCount=%d deferredCount=%d terminalFailedCount=%d startedAt=%s completedAt=%s durationMs=%d", c.config.EventPrefix, runID, round, summary.inputCount, summary.successCount, summary.failedCount, summary.deferredCount, summary.terminalFailedCount, summary.startedAt.Format(time.RFC3339), summary.lastBatchAt.Format(time.RFC3339), summary.lastBatchAt.Sub(summary.startedAt).Milliseconds()) +} + +func (c *Consumer) completeRun(ctx context.Context, runID string) error { + completed, err := c.store.CompleteRunIfIdle(ctx, runID, c.config.FailureRetention) + if err != nil || !completed { + return err + } + meta, err := c.store.GetRunMeta(ctx, runID) + if err != nil { + return gerror.Wrap(err, "read completed run metadata") + } + c.flushRoundSummary(ctx, runID, meta.CurrentRound) + g.Log().Infof(ctx, "event=%s_run status=%s runId=%s successCount=%d failedCount=%d completedAt=%s", c.config.EventPrefix, meta.Status, runID, meta.SuccessCount, meta.FailedCount, meta.CompletedAt.Format(time.RFC3339)) + c.reportFailurePoolSize(ctx, runID, meta) + return nil +} + +// reportFailurePoolSize warns when one run left behind an unusually large +// terminal failure pool. +// +// The pool is the only per-card structure that grows with the card count +// (each FailureRecord is roughly 1KB of JSON), and it is retained for +// FailureRetention — so an upstream-wide outage on a large run is the one +// scenario that can push Redis usage up by hundreds of megabytes. The alert +// exists so that condition is visible in logs instead of being discovered as a +// memory graph later; the run itself is deliberately never truncated. +func (c *Consumer) reportFailurePoolSize(ctx context.Context, runID string, meta queue.RunMeta) { + threshold := c.config.FailurePoolAlertThreshold + // Strictly greater than: the documented contract is "exceeds this many + // entries", so a run sitting exactly on the threshold is not yet an + // anomaly. + if threshold <= 0 || meta.FailedCount <= threshold { + return + } + failureKey, err := c.store.FailedKey(runID) + if err != nil { + return + } + size, err := c.store.RedisClient().Do(ctx, "HLEN", failureKey) + if err != nil { + return + } + g.Log().Warningf(ctx, "event=%s_failure_pool status=large runId=%s failedCount=%d poolEntries=%d threshold=%d retention=%s action=none_observability_only", + c.config.EventPrefix, runID, meta.FailedCount, size.Int(), threshold, c.config.FailureRetention) +} + +func (c *Consumer) shouldRetryNextRound(result Result) bool { + if c.config.ShouldRetryNextRound != nil { + return c.config.ShouldRetryNextRound(result) + } + return result.ErrorClass == "mobile_busy" +} + +func (c *Consumer) startChannel(ctx context.Context, chanKey string) bool { + c.runningMu.Lock() + if _, exists := c.runningChannels[chanKey]; exists { + c.runningMu.Unlock() + return false + } + c.runningChannels[chanKey] = struct{}{} + c.channelGroup.Add(1) + c.runningMu.Unlock() + + go func() { + defer c.channelGroup.Done() + defer func() { + c.runningMu.Lock() + delete(c.runningChannels, chanKey) + c.runningMu.Unlock() + }() + startedAt := time.Now() + for ctx.Err() == nil { + processed, err := c.processChannel(ctx, chanKey) + if err != nil { + g.Log().Errorf(ctx, "event=monthly_usage_batch status=failed stage=claim_or_process chanKey=%s durationMs=%d error=%q", chanKey, time.Since(startedAt).Milliseconds(), err.Error()) + return + } + if !processed { + return + } + } + }() + return true +} + +func (c *Consumer) processChannel(ctx context.Context, chanKey string) (bool, error) { + now := time.Now() + claimed, err := c.store.Claim(ctx, chanKey, now, now.Add(c.config.LeaseTTL), c.config.ClaimBatch) + if err != nil { + return false, err + } + if len(claimed) == 0 { + return false, nil + } + return true, c.handleClaims(ctx, claimed) +} + +func (c *Consumer) handleClaims(ctx context.Context, claimed []queue.ClaimedTask) error { + if len(claimed) == 0 { + return nil + } + startedAt := time.Now() + logBatchFailure := func(stage string, err error) { + g.Log().Errorf(ctx, "event=monthly_usage_batch status=failed stage=%s runId=%s queryMonth=%s chanKey=%s tempKey=%s claimed=%d durationMs=%d error=%q", stage, claimed[0].Task.RunID, claimed[0].Task.QueryMonth, claimed[0].Task.ChanKey, claimed[0].Task.TempKey, len(claimed), time.Since(startedAt).Milliseconds(), errorMessage(err)) + } + channelCarrier, qps := policyForTemplate(claimed[0].Task.TempKey, c.config) + batchCtx, cancelBatch := context.WithCancel(ctx) + queryCtx, cancelQueries := context.WithCancel(batchCtx) + defer cancelQueries() + renewErr := make(chan error, 1) + renewDone := make(chan struct{}) + go c.renewBatchLeases(batchCtx, cancelBatch, claimed, renewErr, renewDone) + defer func() { + cancelBatch() + <-renewDone + }() + + var group sync.WaitGroup + resultsCh := make(chan Result, len(claimed)) + for _, task := range claimed { + task := task + group.Add(1) + go func() { + defer group.Done() + result := c.queryClaim(queryCtx, task, channelCarrier, qps) + if result.BlockChannel { + cancelQueries() + } + resultsCh <- result + }() + } + group.Wait() + close(resultsCh) + results := make([]Result, 0, len(claimed)) + blockChannel := false + for result := range resultsCh { + results = append(results, result) + blockChannel = blockChannel || result.BlockChannel + } + if err := currentBatchError(batchCtx, renewErr); err != nil { + logBatchFailure("query_or_lease", err) + return err + } + for _, result := range results { + if !result.Success && c.config.OnFailure != nil { + c.config.OnFailure(ctx, result) + } + } + + if applyErr := c.handler.Apply(batchCtx, results); applyErr != nil { + logBatchFailure("apply", applyErr) + return fmt.Errorf("apply monthly usage result batch: %w", applyErr) + } + if err := currentBatchError(batchCtx, renewErr); err != nil { + logBatchFailure("lease", err) + return err + } + if blockChannel { + removed, err := c.config.BlockRun(batchCtx, claimed[0].Task.ChanKey, claimed[0].Task.RunID) + if err != nil { + logBatchFailure("block", err) + return err + } + g.Log().Errorf(ctx, "event=monthly_usage_channel status=blocked runId=%s queryMonth=%s chanKey=%s tempKey=%s errCode=S00004 claimed=%d removed=%d durationMs=%d", claimed[0].Task.RunID, claimed[0].Task.QueryMonth, claimed[0].Task.ChanKey, claimed[0].Task.TempKey, len(claimed), removed, time.Since(startedAt).Milliseconds()) + return nil + } + acknowledged := make([]queue.ClaimedTask, 0, len(claimed)) + for _, result := range results { + // The legacy channel queue has no round state. A mobile busy response + // therefore remains a terminal result there; only Run-scoped consumers + // can defer it into the next round. Infrastructure failures retain the + // historical lease-recovery behaviour. + if result.Success || !result.Retryable || (result.ErrorClass == "mobile_busy" && !c.config.RoundRetryEnabled) { + acknowledged = append(acknowledged, claimedTaskForResult(claimed, result)) + } + } + if err := c.ackBatch(batchCtx, acknowledged); err != nil { + logBatchFailure("ack", err) + return err + } + success := countSuccessful(results) + retryCount := 0 + for _, result := range results { + if !result.Success && result.Retryable && !result.BlockChannel && !(result.ErrorClass == "mobile_busy" && !c.config.RoundRetryEnabled) { + retryCount++ + } + } + g.Log().Infof(ctx, "event=monthly_usage_batch status=completed runId=%s queryMonth=%s chanKey=%s tempKey=%s claimed=%d success=%d failed=%d retryPending=%d dbUpdated=%d acked=%d durationMs=%d", claimed[0].Task.RunID, claimed[0].Task.QueryMonth, claimed[0].Task.ChanKey, claimed[0].Task.TempKey, len(claimed), success, len(results)-success, retryCount, success, len(acknowledged), time.Since(startedAt).Milliseconds()) + return nil +} + +func (c *Consumer) renewBatchLeases(ctx context.Context, cancel context.CancelFunc, claimed []queue.ClaimedTask, errors chan<- error, done chan<- struct{}) { + defer close(done) + interval := c.config.LeaseTTL / 3 + if interval <= 0 { + interval = time.Millisecond + } + ticker := time.NewTicker(interval) + defer ticker.Stop() + for { + select { + case <-ctx.Done(): + return + case <-ticker.C: + var count int + var err error + if claimed[0].Round > 0 { + count, err = c.store.RenewRunBatch(ctx, claimed, time.Now().Add(c.config.LeaseTTL)) + } else { + count, err = c.store.RenewBatch(ctx, claimed, time.Now().Add(c.config.LeaseTTL)) + } + if err == nil && count != len(claimed) { + err = ErrLeaseLost + } + if err != nil { + select { + case errors <- err: + default: + } + cancel() + return + } + } + } +} + +func currentBatchError(ctx context.Context, renewErr <-chan error) error { + select { + case err := <-renewErr: + return err + default: + } + return ctx.Err() +} + +func (c *Consumer) queryClaim(ctx context.Context, claimed queue.ClaimedTask, channelCarrier carrier, qps float64) Result { + if err := c.pacer.Wait(ctx, claimed.Task.ChanKey, qps); err != nil { + return Result{Task: claimed.Task, ErrorClass: "canceled", Message: err.Error(), Retryable: true} + } + semaphore := c.pools.get(claimed.Task.ChanKey, channelCarrier) + select { + case semaphore <- struct{}{}: + case <-ctx.Done(): + return Result{Task: claimed.Task, ErrorClass: "canceled", Message: ctx.Err().Error(), Retryable: true} + } + defer func() { <-semaphore }() + taskCtx := ctx + var cancel context.CancelFunc + if c.config.TaskTimeout > 0 { + taskCtx, cancel = context.WithTimeout(ctx, c.config.TaskTimeout) + defer cancel() + } + return c.handler.Query(taskCtx, claimed.Task) +} + +func claimedTaskForResult(claimed []queue.ClaimedTask, result Result) queue.ClaimedTask { + for _, item := range claimed { + if item.Task.TaskID == result.Task.TaskID { + return item + } + } + return queue.ClaimedTask{} +} + +func (c *Consumer) ackBatch(ctx context.Context, claimed []queue.ClaimedTask) error { + if len(claimed) == 0 { + return nil + } + count, err := c.store.AckBatch(ctx, claimed) + if err != nil { + return err + } + if count != len(claimed) { + return ErrLeaseLost + } + return nil +} + +func countSuccessful(results []Result) int { + count := 0 + for _, result := range results { + if result.Success { + count++ + } + } + return count +} + +// countCovered counts results that were answered definitively but carry no +// value. They are logged separately from failures because they are neither a +// success (nothing was written) nor a failure (nothing went wrong). +func countCovered(results []Result) int { + count := 0 + for _, result := range results { + if !result.Success && result.Covered { + count++ + } + } + return count +} + +func errorMessage(err error) string { + if err == nil { + return "" + } + return err.Error() +} + +// channelPools hands each chanKey its own in-flight semaphore. One channel is +// one carrier account, so concurrency must never be shared across channels. +type channelPools struct { + mu sync.Mutex + sems map[string]chan struct{} + mobile int + telecom int + generic int +} + +func newChannelPools(mobile, telecom, generic int) *channelPools { + return &channelPools{sems: make(map[string]chan struct{}), mobile: mobile, telecom: telecom, generic: generic} +} + +func (p *channelPools) get(chanKey string, channelCarrier carrier) chan struct{} { + p.mu.Lock() + defer p.mu.Unlock() + if sem, ok := p.sems[chanKey]; ok { + return sem + } + size := p.mobile + if channelCarrier == carrierTelecom { + size = p.telecom + } else if channelCarrier == carrierGeneric { + size = p.generic + } + sem := make(chan struct{}, size) + p.sems[chanKey] = sem + return sem +} + +type channelPacer struct { + mu sync.Mutex + next map[string]time.Time +} + +func newChannelPacer() *channelPacer { + return &channelPacer{next: make(map[string]time.Time)} +} + +func (p *channelPacer) Wait(ctx context.Context, chanKey string, qps float64) error { + delay := p.reserve(chanKey, qps, time.Now()) + if delay <= 0 { + return nil + } + timer := time.NewTimer(delay) + defer timer.Stop() + select { + case <-timer.C: + return nil + case <-ctx.Done(): + return ctx.Err() + } +} + +func (p *channelPacer) reserve(chanKey string, qps float64, now time.Time) time.Duration { + if qps <= 0 { + return 0 + } + p.mu.Lock() + defer p.mu.Unlock() + interval := time.Duration(float64(time.Second) / qps) + if interval < time.Nanosecond { + interval = time.Nanosecond + } + next := p.next[chanKey] + if next.Before(now) { + next = now + } + p.next[chanKey] = next.Add(interval) + return next.Sub(now) +} + +func policyForTemplate(tempKey string, config Config) (carrier, float64) { + tempKey = strings.ToLower(strings.TrimSpace(tempKey)) + qps := func(defaultRate float64) float64 { + if configured, ok := config.TemplateQPS[tempKey]; ok && configured > 0 { + return configured + } + return defaultRate + } + if model.IsTelecomTemplate(tempKey) { + return carrierTelecom, qps(telecomQPS) + } + switch tempKey { + case mobileTempKey, mobileMockKey, sharedTengyuTempKey, sharedXuzhouMobileTempKey, sharedLelianTempKey, sharedQianxunTempKey, sharedHenanTempKey, sharedDaiheTempKey, sharedBeixinTempKey, sharedLanshang8000TempKey, sharedLanshang8003TempKey: + return carrierMobile, qps(mobileQPS) + default: + genericRate := config.GenericQPS + if genericRate <= 0 { + genericRate = defaultGenericQPS + } + return carrierGeneric, qps(genericRate) + } +} + +// carrierForOperator is retained as a small compatibility helper for status +// policy tests and callers that only have the channel's operator dictionary. +func carrierForOperator(operator int) (carrier, float64) { + switch operator { + case 1: + return carrierMobile, mobileQPS + case 2: + return carrierTelecom, telecomQPS + default: + return carrierGeneric, 0 + } +} + +func newOwner() string { + var bytes [12]byte + if _, err := rand.Read(bytes[:]); err == nil { + return "consumer-" + hex.EncodeToString(bytes[:]) + } + return fmt.Sprintf("consumer-%d", time.Now().UnixNano()) +} diff --git a/internal/consumer/zset/consumer_test.go b/internal/consumer/zset/consumer_test.go new file mode 100644 index 0000000..57761e5 --- /dev/null +++ b/internal/consumer/zset/consumer_test.go @@ -0,0 +1,706 @@ +package zset + +import ( + "context" + "encoding/json" + "errors" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/gogf/gf/v2/container/gvar" + + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +type redisCall struct { + command string + args []any +} + +type fakeRedis struct { + mu sync.Mutex + calls []redisCall + responses []*gvar.Var +} + +func (f *fakeRedis) Do(_ context.Context, command string, args ...any) (*gvar.Var, error) { + f.mu.Lock() + defer f.mu.Unlock() + f.calls = append(f.calls, redisCall{command: command, args: args}) + if len(f.responses) == 0 { + return gvar.New(int64(1)), nil + } + response := f.responses[0] + f.responses = f.responses[1:] + return response, nil +} + +func (f *fakeRedis) Eval(_ context.Context, script string, numKeys int64, keys []string, args []any) (*gvar.Var, error) { + allArgs := []any{script, numKeys} + for _, key := range keys { + allArgs = append(allArgs, key) + } + allArgs = append(allArgs, args...) + return f.Do(context.Background(), "EVAL", allArgs...) +} + +type testHandler struct { + query func(context.Context, queue.Task) Result + apply func(context.Context, []Result) error +} + +func (h *testHandler) Query(ctx context.Context, task queue.Task) Result { + if h.query != nil { + return h.query(ctx, task) + } + return Result{Task: task, Success: true, UsageMB: float64(task.CardID)} +} + +func (h *testHandler) Apply(ctx context.Context, results []Result) error { + if h.apply != nil { + return h.apply(ctx, results) + } + return nil +} + +func taskMember(task queue.Task) string { + value, _ := json.Marshal(task) + return string(value) +} + +func claimedTask(task queue.Task) queue.ClaimedTask { + return queue.ClaimedTask{Task: task, Member: taskMember(task), LeaseToken: "token-" + task.TaskID} +} + +func newTestConsumer(t *testing.T, fake *fakeRedis, handler Handler, config Config) *Consumer { + t.Helper() + store, err := queue.New(fake, "flow:") + if err != nil { + t.Fatal(err) + } + consumer, err := New(store, handler, config) + if err != nil { + t.Fatal(err) + } + return consumer +} + +func TestRoundSummaryAccumulatesAndFlushes(t *testing.T) { + fake := &fakeRedis{responses: []*gvar.Var{gvar.New([]any{"runId", "run-1"})}} + consumer := newTestConsumer(t, fake, &testHandler{}, Config{RoundRetryEnabled: true}) + started := time.Date(2026, 9, 22, 3, 0, 0, 0, time.UTC) + consumer.recordRoundBatch("run-1", 1, []queue.ClaimedTask{ + claimedTask(queue.Task{RunID: "run-1", TaskID: "a"}), + claimedTask(queue.Task{RunID: "run-1", TaskID: "b"}), + }, []queue.TaskOutcome{ + {Success: true}, + {Retry: true}, + }, started) + consumer.recordRoundBatch("run-1", 1, []queue.ClaimedTask{ + claimedTask(queue.Task{RunID: "run-1", TaskID: "c"}), + }, []queue.TaskOutcome{{Success: false}}, started.Add(2*time.Second)) + + key := "run-1\x001" + consumer.summaryMu.Lock() + summary := *consumer.roundSummaries[key] + consumer.summaryMu.Unlock() + if summary.inputCount != 3 || summary.successCount != 1 || summary.failedCount != 2 || summary.deferredCount != 1 || summary.terminalFailedCount != 1 { + t.Fatalf("unexpected summary: %+v", summary) + } + if !summary.startedAt.Equal(started) { + t.Fatalf("startedAt = %v, want %v", summary.startedAt, started) + } + + consumer.flushRoundSummary(context.Background(), "run-1", 1) + consumer.summaryMu.Lock() + _, exists := consumer.roundSummaries[key] + consumer.summaryMu.Unlock() + if exists { + t.Fatal("round summary was not removed after flush") + } +} + +func TestHandleClaimsAppliesSuccessOnceAndAcknowledgesBatch(t *testing.T) { + fake := &fakeRedis{responses: []*gvar.Var{gvar.New(int64(2))}} + applyCalls := 0 + var applied []Result + handler := &testHandler{apply: func(_ context.Context, results []Result) error { + applyCalls++ + applied = append(applied, results...) + return nil + }} + consumer := newTestConsumer(t, fake, handler, Config{ClaimBatch: 2, MobileConcurrency: 2, TelecomConcurrency: 1}) + claimed := []queue.ClaimedTask{ + claimedTask(queue.Task{TaskID: "1", RunID: "run", ChanKey: "mobile-a", TempKey: mobileTempKey, Operator: 1, CardID: 1}), + claimedTask(queue.Task{TaskID: "2", RunID: "run", ChanKey: "mobile-a", TempKey: mobileTempKey, Operator: 1, CardID: 2}), + } + if err := consumer.handleClaims(context.Background(), claimed); err != nil { + t.Fatal(err) + } + if applyCalls != 1 || len(applied) != 2 { + t.Fatalf("apply calls = %d, results = %d", applyCalls, len(applied)) + } + if len(fake.calls) != 1 || fake.calls[0].command != "EVAL" { + t.Fatalf("Redis calls = %+v, want one batch ack", fake.calls) + } +} + +func TestDatabaseFailureLeavesEntireBatchLeased(t *testing.T) { + fake := &fakeRedis{} + handler := &testHandler{ + query: func(_ context.Context, task queue.Task) Result { + if task.CardID == 2 { + return Result{Task: task, ErrorClass: "business", Code: 11011, Message: "invalid card"} + } + return Result{Task: task, Success: true, UsageMB: 1.5} + }, + apply: func(context.Context, []Result) error { return errors.New("database unavailable") }, + } + consumer := newTestConsumer(t, fake, handler, Config{ClaimBatch: 2, MobileConcurrency: 2, TelecomConcurrency: 1}) + claimed := []queue.ClaimedTask{ + claimedTask(queue.Task{TaskID: "success", RunID: "run", ChanKey: "mobile-a", TempKey: mobileTempKey, Operator: 1, CardID: 1}), + claimedTask(queue.Task{TaskID: "failed", RunID: "run", ChanKey: "mobile-a", TempKey: mobileTempKey, Operator: 1, CardID: 2}), + } + err := consumer.handleClaims(context.Background(), claimed) + if err == nil || err.Error() != "apply monthly usage result batch: database unavailable" { + t.Fatalf("handleClaims() error = %v", err) + } + if len(fake.calls) != 0 { + t.Fatalf("Redis calls = %+v, want no ACK when database update fails", fake.calls) + } +} + +func TestBusinessFailuresAreTerminalWithoutRetry(t *testing.T) { + fake := &fakeRedis{responses: []*gvar.Var{gvar.New(int64(2))}} + failureCount := 0 + handler := &testHandler{query: func(_ context.Context, task queue.Task) Result { + return Result{Task: task, ErrorClass: "business", Code: 13000, Message: "busy"} + }} + consumer := newTestConsumer(t, fake, handler, Config{ + MobileConcurrency: 2, + TelecomConcurrency: 1, + OnFailure: func(context.Context, Result) { + failureCount++ + }, + }) + claimed := []queue.ClaimedTask{ + claimedTask(queue.Task{TaskID: "1", RunID: "run", ChanKey: "mobile-a", TempKey: mobileTempKey, Operator: 1, CardID: 1}), + claimedTask(queue.Task{TaskID: "2", RunID: "run", ChanKey: "mobile-a", TempKey: mobileTempKey, Operator: 1, CardID: 2}), + } + if err := consumer.handleClaims(context.Background(), claimed); err != nil { + t.Fatal(err) + } + if failureCount != 2 { + t.Fatalf("failure count = %d, want 2", failureCount) + } + if len(fake.calls) != 1 || fake.calls[0].command != "EVAL" { + t.Fatalf("Redis calls = %+v; business failures must only be acknowledged", fake.calls) + } +} + +func TestRetryableFailuresRemainLeasedWhileSuccessesAreAcknowledged(t *testing.T) { + fake := &fakeRedis{responses: []*gvar.Var{gvar.New(int64(1))}} + var applied []Result + handler := &testHandler{ + query: func(_ context.Context, task queue.Task) Result { + if task.CardID == 1 { + return Result{Task: task, ErrorClass: "network", Retryable: true, Message: "temporary failure"} + } + return Result{Task: task, Success: true, UsageMB: 2} + }, + apply: func(_ context.Context, results []Result) error { + applied = append(applied, results...) + return nil + }, + } + consumer := newTestConsumer(t, fake, handler, Config{MobileConcurrency: 2, TelecomConcurrency: 1}) + claimed := []queue.ClaimedTask{ + claimedTask(queue.Task{TaskID: "retry", RunID: "run", ChanKey: "mobile-a", Operator: 1, CardID: 1}), + claimedTask(queue.Task{TaskID: "success", RunID: "run", ChanKey: "mobile-a", Operator: 1, CardID: 2}), + } + if err := consumer.handleClaims(context.Background(), claimed); err != nil { + t.Fatal(err) + } + if len(applied) != 2 { + t.Fatalf("applied results = %d, want both result rows for handler", len(applied)) + } + if len(fake.calls) != 1 || fake.calls[0].command != "EVAL" { + t.Fatalf("Redis calls = %+v, want one ACK for successful/terminal result", fake.calls) + } + if len(fake.calls[0].args) == 0 { + t.Fatal("ACK call did not include arguments") + } +} + +func TestRunRejectsAnotherActiveConsumer(t *testing.T) { + fake := &fakeRedis{responses: []*gvar.Var{gvar.New(nil)}} + consumer := newTestConsumer(t, fake, &testHandler{}, Config{}) + if err := consumer.Run(context.Background()); !errors.Is(err, ErrAlreadyRunning) { + t.Fatalf("Run error = %v, want ErrAlreadyRunning", err) + } +} + +func TestProcessOnceStartsEveryRegisteredChannel(t *testing.T) { + fake := &fakeRedis{responses: []*gvar.Var{ + gvar.New([]any{"mobile-a", "telecom-a", "mobile-b"}), + gvar.New([]any{}), + gvar.New([]any{}), + gvar.New([]any{}), + }} + consumer := newTestConsumer(t, fake, &testHandler{}, Config{}) + processed, err := consumer.processOnce(context.Background()) + if err != nil { + t.Fatal(err) + } + if !processed { + t.Fatal("processOnce() did not start registered channels") + } + deadline := time.Now().Add(time.Second) + for { + fake.mu.Lock() + callCount := len(fake.calls) + fake.mu.Unlock() + if callCount == 4 { + break + } + if time.Now().After(deadline) { + t.Fatalf("Redis calls = %d, want channels + 3 claims", callCount) + } + time.Sleep(time.Millisecond) + } + consumer.channelGroup.Wait() + if len(fake.calls) != 4 { + t.Fatalf("Redis calls = %d, want channels + 3 claims", len(fake.calls)) + } +} + +type channelRedis struct { + mu sync.Mutex + claims map[string][]queue.Task + claimCounts map[string]int +} + +func (r *channelRedis) Do(_ context.Context, command string, _ ...any) (*gvar.Var, error) { + return nil, errors.New("unexpected command " + command) +} + +func (r *channelRedis) Eval(_ context.Context, script string, _ int64, keys []string, args []any) (*gvar.Var, error) { + r.mu.Lock() + defer r.mu.Unlock() + if strings.Contains(script, "ZRANGEBYSCORE") { + chanKey := strings.TrimPrefix(keys[0], "flow:") + r.claimCounts[chanKey]++ + tasks := r.claims[chanKey] + if len(tasks) == 0 { + return gvar.New([]any{}), nil + } + task := tasks[0] + r.claims[chanKey] = tasks[1:] + return gvar.New([]any{taskMember(task), "token-" + task.TaskID}), nil + } + if len(keys) == 2 && len(args) == 2 { + return gvar.New(int64(1)), nil + } + return nil, errors.New("unexpected eval") +} + +func TestSlowChannelDoesNotBlockAnotherChannelFromClaimingAgain(t *testing.T) { + redis := &channelRedis{ + claims: map[string][]queue.Task{ + "telecom-a": {{TaskID: "t1", RunID: "run", ChanKey: "telecom-a", TempKey: telecomTempKey, Operator: 2}}, + "mobile-a": { + {TaskID: "m1", RunID: "run", ChanKey: "mobile-a", TempKey: mobileTempKey, Operator: 1}, + {TaskID: "m2", RunID: "run", ChanKey: "mobile-a", TempKey: mobileTempKey, Operator: 1}, + }, + }, + claimCounts: make(map[string]int), + } + telecomStarted := make(chan struct{}) + releaseTelecom := make(chan struct{}) + handler := &testHandler{query: func(_ context.Context, task queue.Task) Result { + if task.ChanKey == "telecom-a" { + close(telecomStarted) + <-releaseTelecom + } + return Result{Task: task, Success: true} + }} + store, err := queue.New(redis, "flow:") + if err != nil { + t.Fatal(err) + } + consumer, err := New(store, handler, Config{ClaimBatch: 1, MobileConcurrency: 1, TelecomConcurrency: 1, LeaseTTL: time.Minute}) + if err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + consumer.startChannel(ctx, "telecom-a") + consumer.startChannel(ctx, "mobile-a") + select { + case <-telecomStarted: + case <-time.After(time.Second): + t.Fatal("telecom channel did not start") + } + if consumer.startChannel(ctx, "telecom-a") { + t.Fatal("channel executor started twice while first executor is running") + } + deadline := time.Now().Add(time.Second) + for { + redis.mu.Lock() + mobileClaims := redis.claimCounts["mobile-a"] + redis.mu.Unlock() + if mobileClaims >= 2 { + break + } + if time.Now().After(deadline) { + t.Fatalf("mobile claim count = %d while telecom is blocked", mobileClaims) + } + time.Sleep(time.Millisecond) + } + close(releaseTelecom) + consumer.channelGroup.Wait() +} + +func TestClaimedBatchRenewsLeaseUntilAcknowledged(t *testing.T) { + fake := &fakeRedis{} + handler := &testHandler{query: func(_ context.Context, task queue.Task) Result { + time.Sleep(45 * time.Millisecond) + return Result{Task: task, Success: true} + }} + consumer := newTestConsumer(t, fake, handler, Config{LeaseTTL: 30 * time.Millisecond, TaskTimeout: 20 * time.Millisecond, MobileConcurrency: 1, TelecomConcurrency: 1}) + claimed := []queue.ClaimedTask{claimedTask(queue.Task{TaskID: "1", RunID: "run", ChanKey: "mobile-a", TempKey: mobileTempKey, Operator: 1})} + if err := consumer.handleClaims(context.Background(), claimed); err != nil { + t.Fatal(err) + } + fake.mu.Lock() + defer fake.mu.Unlock() + if len(fake.calls) < 2 { + t.Fatalf("Redis calls = %d, want lease renewal and ACK", len(fake.calls)) + } +} + +func TestLostBatchLeasePreventsApplyAndAck(t *testing.T) { + fake := &fakeRedis{responses: []*gvar.Var{gvar.New(int64(0))}} + applyCalls := 0 + handler := &testHandler{ + query: func(ctx context.Context, task queue.Task) Result { + <-ctx.Done() + return Result{Task: task, ErrorClass: "canceled"} + }, + apply: func(context.Context, []Result) error { + applyCalls++ + return nil + }, + } + consumer := newTestConsumer(t, fake, handler, Config{LeaseTTL: 30 * time.Millisecond, TaskTimeout: 20 * time.Millisecond, MobileConcurrency: 1, TelecomConcurrency: 1}) + claimed := []queue.ClaimedTask{claimedTask(queue.Task{TaskID: "1", RunID: "run", ChanKey: "mobile-a", TempKey: mobileTempKey, Operator: 1})} + err := consumer.handleClaims(context.Background(), claimed) + if !errors.Is(err, ErrLeaseLost) { + t.Fatalf("handleClaims() error = %v, want ErrLeaseLost", err) + } + if applyCalls != 0 { + t.Fatalf("apply calls = %d, want 0 after lease loss", applyCalls) + } + fake.mu.Lock() + defer fake.mu.Unlock() + if len(fake.calls) != 1 { + t.Fatalf("Redis calls = %d, want only failed lease renewal", len(fake.calls)) + } +} + +func newTrackingHandler(release chan struct{}, current, maximum *atomic.Int32) *testHandler { + return &testHandler{query: func(_ context.Context, task queue.Task) Result { + active := current.Add(1) + for { + observed := maximum.Load() + if active <= observed || maximum.CompareAndSwap(observed, active) { + break + } + } + <-release + current.Add(-1) + return Result{Task: task, Success: true} + }} +} + +func TestMobileChannelsHaveIndependentConcurrencyPools(t *testing.T) { + var current atomic.Int32 + var maximum atomic.Int32 + release := make(chan struct{}) + consumer := newTestConsumer(t, &fakeRedis{}, newTrackingHandler(release, ¤t, &maximum), Config{MobileConcurrency: 2, TelecomConcurrency: 1}) + var group sync.WaitGroup + for index := 0; index < 3; index++ { + group.Add(1) + go func(index int) { + defer group.Done() + task := queue.Task{TaskID: string(rune('1' + index)), ChanKey: "mobile-" + string(rune('a'+index)), TempKey: mobileTempKey, CardID: int64(index + 1)} + _ = consumer.queryClaim(context.Background(), claimedTask(task), carrierMobile, mobileQPS) + }(index) + } + time.Sleep(100 * time.Millisecond) + if maximum.Load() != 3 { + t.Fatalf("maximum in-flight across channels = %d, want 3 (one pool per channel)", maximum.Load()) + } + close(release) + group.Wait() +} + +func TestChannelConcurrencyPoolCapsWithinChannel(t *testing.T) { + var current atomic.Int32 + var maximum atomic.Int32 + release := make(chan struct{}) + consumer := newTestConsumer(t, &fakeRedis{}, newTrackingHandler(release, ¤t, &maximum), Config{MobileConcurrency: 2, TelecomConcurrency: 1}) + var group sync.WaitGroup + for index := 0; index < 3; index++ { + group.Add(1) + go func(index int) { + defer group.Done() + task := queue.Task{TaskID: string(rune('1' + index)), ChanKey: "mobile-a", TempKey: mobileTempKey, CardID: int64(index + 1)} + _ = consumer.queryClaim(context.Background(), claimedTask(task), carrierMobile, mobileQPS) + }(index) + } + time.Sleep(100 * time.Millisecond) + if maximum.Load() != 2 { + t.Fatalf("maximum in-flight within one channel = %d, want 2", maximum.Load()) + } + close(release) + group.Wait() +} + +func TestMobileAndTelecomUseIndependentChannelPools(t *testing.T) { + started := make(chan carrier, 2) + release := make(chan struct{}) + handler := &testHandler{query: func(_ context.Context, task queue.Task) Result { + channelCarrier, _ := policyForTemplate(task.TempKey, Config{}) + started <- channelCarrier + <-release + return Result{Task: task, Success: true} + }} + consumer := newTestConsumer(t, &fakeRedis{}, handler, Config{MobileConcurrency: 1, TelecomConcurrency: 1}) + go func() { + task := queue.Task{TaskID: "mobile", ChanKey: "mobile-a", TempKey: mobileTempKey, Operator: 1} + _ = consumer.queryClaim(context.Background(), claimedTask(task), carrierMobile, mobileQPS) + }() + go func() { + task := queue.Task{TaskID: "telecom", ChanKey: "telecom-a", TempKey: telecomTempKey, Operator: 2} + _ = consumer.queryClaim(context.Background(), claimedTask(task), carrierTelecom, telecomQPS) + }() + + seen := map[carrier]bool{} + for len(seen) < 2 { + select { + case value := <-started: + seen[value] = true + case <-time.After(time.Second): + t.Fatalf("carrier pools did not progress independently: %+v", seen) + } + } + close(release) +} + +func TestUnknownTemplateUsesGenericStrategy(t *testing.T) { + fake := &fakeRedis{responses: []*gvar.Var{gvar.New(int64(2))}} + queryCalls := 0 + failures := 0 + applyCalls := 0 + handler := &testHandler{ + query: func(_ context.Context, task queue.Task) Result { + queryCalls++ + return Result{Task: task, Success: true, UsageMB: 1} + }, + apply: func(_ context.Context, results []Result) error { + applyCalls++ + for _, result := range results { + if !result.Success || result.BlockChannel { + t.Fatalf("unexpected result = %+v", result) + } + } + return nil + }, + } + consumer := newTestConsumer(t, fake, handler, Config{OnFailure: func(context.Context, Result) { failures++ }}) + claimed := []queue.ClaimedTask{ + claimedTask(queue.Task{TaskID: "1", RunID: "run", ChanKey: "unknown-a", TempKey: "unknown-template", Operator: 1, CardID: 1}), + claimedTask(queue.Task{TaskID: "2", RunID: "run", ChanKey: "unknown-a", TempKey: "unknown-template", Operator: 2, CardID: 2}), + } + if err := consumer.handleClaims(context.Background(), claimed); err != nil { + t.Fatal(err) + } + if queryCalls != 2 || failures != 0 || applyCalls != 1 { + t.Fatalf("query calls = %d, failures = %d, apply calls = %d", queryCalls, failures, applyCalls) + } + if len(fake.calls) != 1 || fake.calls[0].command != "EVAL" { + t.Fatalf("Redis calls = %+v, want one ack script", fake.calls) + } + if len(fake.responses) != 0 { + t.Fatalf("unexpected leftover scripted responses: %+v", fake.responses) + } +} + +func TestChannelPacerLimitsEachChannelIndependently(t *testing.T) { + pacer := newChannelPacer() + now := time.Unix(100, 0) + pacedQPS := 30.0 + if delay := pacer.reserve("mobile-a", pacedQPS, now); delay != 0 { + t.Fatalf("first reservation delay = %s", delay) + } + if delay := pacer.reserve("mobile-a", pacedQPS, now); delay < 33*time.Millisecond || delay > 34*time.Millisecond { + t.Fatalf("second reservation delay = %s", delay) + } + if delay := pacer.reserve("mobile-b", pacedQPS, now); delay != 0 { + t.Fatalf("independent channel delay = %s", delay) + } +} + +func TestChannelPacerSkipsWhenQPSIsDisabled(t *testing.T) { + pacer := newChannelPacer() + now := time.Unix(100, 0) + if delay := pacer.reserve("mobile-a", 0, now); delay != 0 { + t.Fatalf("disabled qps delay = %s", delay) + } + if len(pacer.next) != 0 { + t.Fatalf("disabled qps should not reserve channel slots, got %+v", pacer.next) + } +} + +func TestNewDefaultsTaskTimeout(t *testing.T) { + consumer := newTestConsumer(t, &fakeRedis{}, &testHandler{}, Config{}) + if consumer.config.TaskTimeout != 30*time.Second { + t.Fatalf("TaskTimeout = %s, want 30s default", consumer.config.TaskTimeout) + } + if consumer.config.GenericQPS != defaultGenericQPS { + t.Fatalf("GenericQPS = %v, want %v default", consumer.config.GenericQPS, defaultGenericQPS) + } + if _, err := New(consumer.store, &testHandler{}, Config{LeaseTTL: 10 * time.Second}); err == nil { + t.Fatal("New() with LeaseTTL below default TaskTimeout should fail") + } +} + +func TestPolicyForTemplateUsesTemplateValues(t *testing.T) { + tests := []struct { + name string + tempKey string + carrier carrier + qps float64 + }{ + {name: "mobile", tempKey: mobileTempKey, carrier: carrierMobile, qps: mobileQPS}, + {name: "mobile mock", tempKey: mobileMockKey, carrier: carrierMobile, qps: mobileQPS}, + {name: "shared Tengyu", tempKey: sharedTengyuTempKey, carrier: carrierMobile, qps: mobileQPS}, + {name: "shared Xuzhou mobile", tempKey: sharedXuzhouMobileTempKey, carrier: carrierMobile, qps: mobileQPS}, + {name: "shared Lelian", tempKey: sharedLelianTempKey, carrier: carrierMobile, qps: mobileQPS}, + {name: "shared Qianxun", tempKey: sharedQianxunTempKey, carrier: carrierMobile, qps: mobileQPS}, + {name: "shared Henan", tempKey: sharedHenanTempKey, carrier: carrierMobile, qps: mobileQPS}, + {name: "shared Daihe", tempKey: sharedDaiheTempKey, carrier: carrierMobile, qps: mobileQPS}, + {name: "shared Beixin", tempKey: sharedBeixinTempKey, carrier: carrierMobile, qps: mobileQPS}, + // lanshang_8000/8003 are OneLink shared-token templates whose names + // omit the sim_onelink_ prefix. Before this case was added they fell + // through to carrierGeneric and were throttled to 1.5 QPS, which made a + // 6k-card status/traffic/IMEI run take over an hour per channel. + {name: "shared Lanshang 8000", tempKey: sharedLanshang8000TempKey, carrier: carrierMobile, qps: mobileQPS}, + {name: "shared Lanshang 8003", tempKey: sharedLanshang8003TempKey, carrier: carrierMobile, qps: mobileQPS}, + {name: "shared Lanshang case and padding", tempKey: " LANSHANG_8000 ", carrier: carrierMobile, qps: mobileQPS}, + {name: "telecom", tempKey: telecomTempKey, carrier: carrierTelecom, qps: telecomQPS}, + {name: "telecom v3", tempKey: " SIM_DX_V3 ", carrier: carrierTelecom, qps: telecomQPS}, + {name: "unknown telecom version", tempKey: "sim_dx_v2_future", carrier: carrierGeneric, qps: defaultGenericQPS}, + {name: "unknown template", tempKey: "unknown-template", carrier: carrierGeneric, qps: defaultGenericQPS}, + {name: "missing template", tempKey: "", carrier: carrierGeneric, qps: defaultGenericQPS}, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + gotCarrier, gotQPS := policyForTemplate(test.tempKey, Config{}) + if gotCarrier != test.carrier || gotQPS != test.qps { + t.Fatalf("policyForTemplate(%q) = %q,%v", test.tempKey, gotCarrier, gotQPS) + } + }) + } +} + +func TestPolicyForTemplateConfigurationOverridesDefaults(t *testing.T) { + config := Config{ + GenericQPS: 2.5, + TemplateQPS: map[string]float64{ + " SIM_YD_V5 ": 12, + "unknown-template": 3, + "invalid": 0, + }, + } + config.setDefaults() + + if gotCarrier, gotQPS := policyForTemplate("sim_yd_v5", config); gotCarrier != carrierMobile || gotQPS != 12 { + t.Fatalf("mobile override = %q,%v, want mobile,12", gotCarrier, gotQPS) + } + if gotCarrier, gotQPS := policyForTemplate("unknown-template", config); gotCarrier != carrierGeneric || gotQPS != 3 { + t.Fatalf("generic override = %q,%v, want generic,3", gotCarrier, gotQPS) + } + if gotCarrier, gotQPS := policyForTemplate("other-template", config); gotCarrier != carrierGeneric || gotQPS != 2.5 { + t.Fatalf("generic default = %q,%v, want generic,2.5", gotCarrier, gotQPS) + } +} + +// TestFailurePoolAlertThresholdIsObservabilityOnly 覆盖失败池告警： +// - 未配置阀值（0）时不发 HLEN 查询 +// - 失败数低于阀值时不告警 +// - 超过阀值时发一次 HLEN 并打 WARN，但绝不截断 Run +// +// 告警本身是纯观测行为，所以断言的是「有没有多打 Redis 命令」， +// 而不是「Run 有没有被改坏」——后者必须永远为「没被改坏」。 +func TestFailurePoolAlertThresholdIsObservabilityOnly(t *testing.T) { + newMeta := func(failed int64) queue.RunMeta { + return queue.RunMeta{RunID: "run-1", Status: "completed_with_failures", FailedCount: failed, CurrentRound: 1} + } + + cases := []struct { + name string + threshold int64 + failed int64 + wantHLenCalls int + }{ + {"未配置阀值不查询", 0, 1_000_000, 0}, + {"低于阀值不告警", 50000, 49999, 0}, + {"等于阀值不告警", 50000, 50000, 0}, + {"超过阀值告警一次", 50000, 50001, 1}, + {"远超阀值也只告警一次", 50000, 500_000, 1}, + } + for _, tc := range cases { + fake := &fakeRedis{} + consumer := newTestConsumer(t, fake, &testHandler{}, Config{ + FailurePoolAlertThreshold: tc.threshold, + }) + consumer.reportFailurePoolSize(context.Background(), "run-1", newMeta(tc.failed)) + + hlen := 0 + for _, call := range fake.calls { + if call.command == "HLEN" { + hlen++ + } + } + if hlen != tc.wantHLenCalls { + t.Fatalf("%s: HLEN 调用 %d 次, want %d", tc.name, hlen, tc.wantHLenCalls) + } + } +} + +// TestFailurePoolAlertReadsCorrectKey 告警必须查该 Run 自己的失败池键。 +func TestFailurePoolAlertReadsCorrectKey(t *testing.T) { + fake := &fakeRedis{} + consumer := newTestConsumer(t, fake, &testHandler{}, Config{FailurePoolAlertThreshold: 10}) + consumer.reportFailurePoolSize(context.Background(), "run-abc", queue.RunMeta{ + RunID: "run-abc", Status: "completed_with_failures", FailedCount: 11, CurrentRound: 1, + }) + + var key string + for _, call := range fake.calls { + if call.command == "HLEN" { + if len(call.args) > 0 { + key, _ = call.args[0].(string) + } + } + } + want, _ := consumer.store.FailedKey("run-abc") + if key != want { + t.Fatalf("HLEN key = %q, want %q", key, want) + } +} diff --git a/internal/consumer/zset/run_integration_test.go b/internal/consumer/zset/run_integration_test.go new file mode 100644 index 0000000..fc1a485 --- /dev/null +++ b/internal/consumer/zset/run_integration_test.go @@ -0,0 +1,480 @@ +package zset + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "sync" + "testing" + "time" + + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +type runHandler struct { + query func(queue.Task) Result + apply func([]Result) error +} + +func (h runHandler) Query(_ context.Context, task queue.Task) Result { return h.query(task) } +func (h runHandler) Apply(_ context.Context, results []Result) error { + if h.apply != nil { + return h.apply(results) + } + return nil +} + +func runTasks(t *testing.T, store *queue.Store, run string, count int) []queue.Task { + t.Helper() + ctx := context.Background() + if err := store.CreateRun(ctx, queue.RunMeta{RunID: run}); err != nil { + t.Fatal(err) + } + tasks := make([]queue.Task, count) + for i := range tasks { + tasks[i] = queue.Task{RunID: run, TaskID: fmt.Sprintf("%s:%d", run, i), CardID: int64(i + 1), ChanKey: "channel", TempKey: mobileTempKey, ICCID: fmt.Sprintf("card-%d", i)} + } + if n, err := store.EnqueueRunBatch(ctx, run, "channel", tasks, time.Now()); err != nil || n != count { + t.Fatalf("enqueue %d: %v", n, err) + } + return tasks +} + +func runMeta(t *testing.T, store *queue.Store, run string) queue.RunMeta { + t.Helper() + meta, err := store.GetRunMeta(context.Background(), run) + if err != nil { + t.Fatal(err) + } + return meta +} + +func runConsumer(t *testing.T, store *queue.Store, handler Handler, rounds bool) *Consumer { + t.Helper() + c, err := New(store, handler, Config{RunQueueEnabled: true, RoundRetryEnabled: rounds, BlockedTasksAsFailures: !rounds, EventPrefix: "sim_status_queue", TemplateQPS: map[string]float64{mobileTempKey: 1000000}}) + if err != nil { + t.Fatal(err) + } + return c +} + +func claimRun(t *testing.T, store *queue.Store, run string, now time.Time, count int) []queue.ClaimedTask { + t.Helper() + claims, err := store.ClaimRun(context.Background(), run, "channel", now, now.Add(time.Minute), count) + if err != nil || len(claims) != count { + t.Fatalf("claim got %d want %d: %v", len(claims), count, err) + } + return claims +} + +func finishProducer(t *testing.T, store *queue.Store, run string) { + t.Helper() + if err := store.MarkProducerDone(context.Background(), run); err != nil { + t.Fatal(err) + } +} + +func checkTerminal(t *testing.T, store *queue.Store, run string, success, failed int64) { + t.Helper() + meta := runMeta(t, store, run) + want := "completed" + if failed > 0 { + want = "completed_with_failures" + } + if meta.Status != want || meta.CompletedAt.IsZero() || meta.SuccessCount != success || meta.FailedCount != failed || meta.ActiveCount != 0 || meta.InflightCount != 0 || meta.NextRoundCount != 0 { + t.Fatalf("unexpected terminal metadata: %+v", meta) + } +} + +func TestRunConsumerCountsAndCompletes(t *testing.T) { + redis := testutil.Redis(t) + ctx := context.Background() + for _, failure := range []bool{false, true} { + t.Run(fmt.Sprint(failure), func(t *testing.T) { + store, _ := queue.New(redis, "sim-status:") + run := fmt.Sprint("counts-", failure) + tasks := runTasks(t, store, run, 3) + if n, err := store.EnqueueRunBatch(ctx, run, "channel", tasks, time.Now()); err != nil || n != 0 { + t.Fatalf("duplicate enqueue: %d %v", n, err) + } + finishProducer(t, store, run) + handler := runHandler{query: func(task queue.Task) Result { + return Result{Task: task, Success: !failure || task.CardID != 2, ErrorClass: "business"} + }, apply: func(_ []Result) error { + if meta := runMeta(t, store, run); meta.SuccessCount != 0 || meta.FailedCount != 0 { + return errors.New("counted before DB apply") + } + return nil + }} + c := runConsumer(t, store, handler, false) + claims := claimRun(t, store, run, time.Now(), 3) + if err := c.handleRunClaims(ctx, claims); err != nil { + t.Fatal(err) + } + var failures int64 + if failure { + failures = 1 + } + checkTerminal(t, store, run, 3-failures, failures) + outcomes := []queue.TaskOutcome{{Success: true}, {Success: true}, {Success: true}} + if n, err := store.FinalizeRunTasks(ctx, claims, outcomes); err != nil || n != 0 { + t.Fatalf("duplicate finalize: %d %v", n, err) + } + checkTerminal(t, store, run, 3-failures, failures) + key, _ := store.RunMetaKey(run) + ttl, err := redis.Do(ctx, "TTL", key) + if err != nil || ttl.Int64() < 29*24*3600 { + t.Fatalf("retention=%v err=%v", ttl, err) + } + }) + } +} + +func TestRunConsumerLateProducerEmptyAndHistoricalRunCompletion(t *testing.T) { + redis := testutil.Redis(t) + store, _ := queue.New(redis, "sim-status:") + ctx := context.Background() + if err := store.CreateRun(ctx, queue.RunMeta{RunID: "historical", Status: "running", ProducerDone: true}); err != nil { + t.Fatal(err) + } + runTasks(t, store, "empty", 0) + runTasks(t, store, "late", 1) + c := runConsumer(t, store, runHandler{query: func(task queue.Task) Result { return Result{Task: task, Success: true} }}, false) + claims := claimRun(t, store, "late", time.Now(), 1) + if err := c.handleRunClaims(ctx, claims); err != nil { + t.Fatal(err) + } + if meta := runMeta(t, store, "late"); !meta.CompletedAt.IsZero() || meta.SuccessCount != 1 { + t.Fatalf("premature completion: %+v", meta) + } + finishProducer(t, store, "late") + finishProducer(t, store, "empty") + if _, err := c.processOnce(ctx); err != nil { + t.Fatal(err) + } + c.channelGroup.Wait() + checkTerminal(t, store, "late", 1, 0) + checkTerminal(t, store, "empty", 0, 0) + checkTerminal(t, store, "historical", 0, 0) +} + +func TestRunConsumerRecoveryAndFencing(t *testing.T) { + redis := testutil.Redis(t) + store, _ := queue.New(redis, "sim-status:") + ctx := context.Background() + for _, mode := range []string{"database", "network", "stale"} { + t.Run(mode, func(t *testing.T) { + runTasks(t, store, mode, 1) + finishProducer(t, store, mode) + now := time.Now() + claims := claimRun(t, store, mode, now, 1) + good := runHandler{query: func(task queue.Task) Result { return Result{Task: task, Success: true} }} + handler := good + if mode == "database" { + handler.apply = func([]Result) error { return errors.New("database unavailable") } + } + if mode == "network" { + handler.query = func(task queue.Task) Result { return Result{Task: task, Retryable: true, ErrorClass: "network"} } + } + c := runConsumer(t, store, handler, false) + if mode != "stale" { + err := c.handleRunClaims(ctx, claims) + if (err != nil) != (mode == "database") { + t.Fatalf("handle: %v", err) + } + if meta := runMeta(t, store, mode); meta.SuccessCount != 0 || meta.FailedCount != 0 || meta.ActiveCount != 1 || meta.InflightCount != 1 || meta.Status != "running" { + t.Fatalf("lost pending task: %+v", meta) + } + } + fresh := claimRun(t, store, mode, now.Add(2*time.Minute), 1) + if n, err := store.RenewRunBatch(ctx, claims, now.Add(3*time.Minute)); err != nil || n != 0 { + t.Fatalf("stale renewal: %d %v", n, err) + } + if n, err := store.FinalizeRunTasks(ctx, claims, []queue.TaskOutcome{{Success: true}}); err != nil || n != 0 { + t.Fatalf("stale ack: %d %v", n, err) + } + if n, err := store.BlockRunScopedAsFailed(ctx, claims[0], queue.TaskOutcome{}); err != nil || n != -1 { + t.Fatalf("stale block: %d %v", n, err) + } + c.handler = good + if err := c.handleRunClaims(ctx, fresh); err != nil { + t.Fatal(err) + } + checkTerminal(t, store, mode, 1, 0) + }) + } +} + +func TestRunConsumerBlockedChannelCountsOnlyUnfinishedTasks(t *testing.T) { + redis := testutil.Redis(t) + store, _ := queue.New(redis, "sim-status:") + ctx := context.Background() + runTasks(t, store, "blocked", 5) + runTasks(t, store, "other", 1) + // Production continues while the channel is blocked. + claims := claimRun(t, store, "blocked", time.Now(), 2) + var applied int + successQueried := make(chan struct{}) + handler := runHandler{query: func(task queue.Task) Result { + if task.CardID == 1 { + close(successQueried) + return Result{Task: task, Success: true} + } + <-successQueried + return Result{Task: task, BlockChannel: true, ErrorClass: "channel_limited", Code: 797, ErrCode: "S00004", Message: "limited"} + }, apply: func(results []Result) error { applied = countSuccessful(results); return nil }} + c := runConsumer(t, store, handler, false) + // Ensure the success response arrives before the channel-limit response. + if err := c.handleRunClaims(ctx, claims); err != nil { + t.Fatal(err) + } + meta := runMeta(t, store, "blocked") + if applied != 1 || meta.SuccessCount != 1 || meta.FailedCount != 4 || meta.ActiveCount != 0 || meta.InflightCount != 0 { + t.Fatalf("applied=%d meta=%+v", applied, meta) + } + if n, err := store.BlockRunScopedAsFailed(ctx, claims[1], queue.TaskOutcome{}); err != nil || n != -1 { + t.Fatalf("duplicate block: %d %v", n, err) + } + late := queue.Task{TaskID: "late", RunID: "blocked", ChanKey: "channel", CardID: 99} + if n, err := store.EnqueueRunBatch(ctx, "blocked", "channel", []queue.Task{late}, time.Now()); err != nil || n != 0 { + t.Fatalf("blocked enqueue: %d %v", n, err) + } + key, _ := store.FailedKey("blocked") + failures, err := redis.Do(ctx, "HLEN", key) + if err != nil || failures.Int() != 4 { + t.Fatalf("failure records=%v err=%v", failures, err) + } + stored, err := redis.Do(ctx, "HGET", key, "blocked:4") + if err != nil { + t.Fatal(err) + } + var record queue.FailureRecord + if err := json.Unmarshal([]byte(stored.String()), &record); err != nil { + t.Fatal(err) + } + if record.TaskID != "blocked:4" || record.CardID != 5 || record.ICCID != "card-4" || record.Attempts != 0 || record.FailureType != "channel_blocked" || record.ErrCode != "S00004" { + t.Fatalf("failure identity/reason: %+v", record) + } + stateKey, _ := store.TaskStateKey("blocked") + state, err := redis.Do(ctx, "HGET", stateKey, "blocked:4") + if err != nil || state.MapStrAny()["status"] != "channel_blocked" { + t.Fatalf("blocked state=%v err=%v", state, err) + } + if meta := runMeta(t, store, "other"); meta.ActiveCount != 1 || meta.FailedCount != 0 { + t.Fatalf("other run affected: %+v", meta) + } + finishProducer(t, store, "blocked") + if err := c.completeRun(ctx, "blocked"); err != nil { + t.Fatal(err) + } + checkTerminal(t, store, "blocked", 1, 4) +} + +func TestRunConsumerMonthlyRoundsRemainEnabled(t *testing.T) { + redis := testutil.Redis(t) + store, _ := queue.New(redis, "flow:") + ctx := context.Background() + runTasks(t, store, "monthly", 1) + finishProducer(t, store, "monthly") + c := runConsumer(t, store, runHandler{query: func(task queue.Task) Result { return Result{Task: task, ErrorClass: "mobile_busy", Retryable: true} }}, true) + for round := 1; round <= 5; round++ { + if err := c.handleRunClaims(ctx, claimRun(t, store, "monthly", time.Now(), 1)); err != nil { + t.Fatal(err) + } + if round < 5 { + if meta := runMeta(t, store, "monthly"); meta.NextRoundCount != 1 || meta.FailedCount != 0 || meta.Status != "running" { + t.Fatalf("round %d: %+v", round, meta) + } + if ok, err := store.AdvanceRunRound(ctx, "monthly", 5); err != nil || !ok { + t.Fatalf("advance %v %v", ok, err) + } + } + } + checkTerminal(t, store, "monthly", 0, 1) +} + +func TestRunConsumerStatusRetriesRetryableResultsForThreeRounds(t *testing.T) { + redis := testutil.Redis(t) + store, _ := queue.New(redis, "sim-status:") + ctx := context.Background() + runTasks(t, store, "status", 1) + finishProducer(t, store, "status") + c, err := New(store, runHandler{query: func(task queue.Task) Result { + return Result{Task: task, ErrorClass: "carrier_retry", Retryable: true, Code: 9999, ErrCode: "S00004"} + }}, Config{ + RunQueueEnabled: true, + RoundRetryEnabled: true, + MaxRounds: 3, + EventPrefix: "sim_status_queue", + ShouldRetryNextRound: func(result Result) bool { + return result.Retryable + }, + TemplateQPS: map[string]float64{mobileTempKey: 1000000}, + }) + if err != nil { + t.Fatal(err) + } + for round := 1; round <= 3; round++ { + if err := c.handleRunClaims(ctx, claimRun(t, store, "status", time.Now(), 1)); err != nil { + t.Fatal(err) + } + if round < 3 { + if meta := runMeta(t, store, "status"); meta.NextRoundCount != 1 || meta.FailedCount != 0 || meta.Status != "running" { + t.Fatalf("round %d: %+v", round, meta) + } + if ok, err := store.AdvanceRunRound(ctx, "status", 3); err != nil || !ok { + t.Fatalf("advance %v %v", ok, err) + } + } + } + checkTerminal(t, store, "status", 0, 1) +} + +func TestReplayMaxRoundOverridesConsumerDefault(t *testing.T) { + redis := testutil.Redis(t) + store, _ := queue.New(redis, "sim-status:") + ctx := context.Background() + if err := store.CreateRun(ctx, queue.RunMeta{RunID: "replay-one", Operation: "status", SyncType: "replay", MaxRound: 1}); err != nil { + t.Fatal(err) + } + task := queue.Task{RunID: "replay-one", TaskID: "task", CardID: 1, ICCID: "card", ChanKey: "channel", TempKey: mobileTempKey} + if _, err := store.EnqueueRunBatch(ctx, task.RunID, task.ChanKey, []queue.Task{task}, time.Now()); err != nil { + t.Fatal(err) + } + finishProducer(t, store, "replay-one") + c, err := New(store, runHandler{query: func(task queue.Task) Result { return Result{Task: task, ErrorClass: "mobile_busy", Retryable: true} }}, Config{RunQueueEnabled: true, RoundRetryEnabled: true, MaxRounds: 3, TemplateQPS: map[string]float64{mobileTempKey: 1000000}}) + if err != nil { + t.Fatal(err) + } + if err := c.handleRunClaims(ctx, claimRun(t, store, "replay-one", time.Now(), 1)); err != nil { + t.Fatal(err) + } + checkTerminal(t, store, "replay-one", 0, 1) + meta := runMeta(t, store, "replay-one") + if meta.CurrentRound != 1 || meta.NextRoundCount != 0 { + t.Fatalf("replay advanced rounds: %+v", meta) + } +} + +func TestRunQueueModeDoesNotEnableRoundRetry(t *testing.T) { + redis := testutil.Redis(t) + store, _ := queue.New(redis, "sim-status:") + runTasks(t, store, "single-round", 1) + finishProducer(t, store, "single-round") + c := runConsumer(t, store, runHandler{query: func(task queue.Task) Result { return Result{Task: task, ErrorClass: "mobile_busy", Retryable: true} }}, false) + if err := c.handleRunClaims(context.Background(), claimRun(t, store, "single-round", time.Now(), 1)); err != nil { + t.Fatal(err) + } + checkTerminal(t, store, "single-round", 0, 1) + if meta := runMeta(t, store, "single-round"); meta.CurrentRound != 1 { + t.Fatalf("unexpected retry: %+v", meta) + } +} + +func TestMonthlyChannelBlockKeepsExistingCountingPolicy(t *testing.T) { + redis := testutil.Redis(t) + store, _ := queue.New(redis, "flow:") + ctx := context.Background() + runTasks(t, store, "monthly-block", 2) + claimRun(t, store, "monthly-block", time.Now(), 1) + for i := 0; i < 2; i++ { + want := int64(2) + if i == 1 { + want = 0 + } + if n, err := store.BlockRunScoped(ctx, "monthly-block", "channel"); err != nil || n != want { + t.Fatalf("monthly block=%d err=%v", n, err) + } + } + if meta := runMeta(t, store, "monthly-block"); meta.FailedCount != 0 || meta.ActiveCount != 0 || meta.InflightCount != 0 { + t.Fatalf("monthly counting changed: %+v", meta) + } +} + +func TestRunConsumerConcurrentRuns(t *testing.T) { + redis := testutil.Redis(t) + store, _ := queue.New(redis, "sim-status:") + ctx := context.Background() + var mu sync.Mutex + applied := map[string]int{} + c := runConsumer(t, store, runHandler{query: func(task queue.Task) Result { return Result{Task: task, Success: true} }, apply: func(results []Result) error { + mu.Lock() + defer mu.Unlock() + for _, result := range results { + applied[result.Task.RunID]++ + } + return nil + }}, false) + for _, run := range []string{"a", "b"} { + runTasks(t, store, run, 3) + finishProducer(t, store, run) + } + if started, err := c.processOnce(ctx); err != nil || !started { + t.Fatalf("process: %v %v", started, err) + } + c.channelGroup.Wait() + for _, run := range []string{"a", "b"} { + checkTerminal(t, store, run, 3, 0) + if applied[run] != 3 { + t.Fatalf("applied %s: %d", run, applied[run]) + } + } +} + +// TestRunConsumerCoveredResultsCompleteWithoutFailures drives the full +// consumer path for a covered result: the round must close, the run must end +// as "completed" rather than "completed_with_failures", and the failure pool +// must stay empty. This is the IMEI 14000 "carrier has no record" case — on +// 2026-10-20 it accounted for every failure recorded in production while the +// run success rate read 21%. +func TestRunConsumerCoveredResultsCompleteWithoutFailures(t *testing.T) { + redis := testutil.Redis(t) + store, _ := queue.New(redis, "imei:") + ctx := context.Background() + runTasks(t, store, "covered", 4) + finishProducer(t, store, "covered") + queried := 0 + c := runConsumer(t, store, runHandler{query: func(task queue.Task) Result { + queried++ + switch task.CardID { + case 1: + return Result{Task: task, Success: true, Imei: "865466070817587", ImeiSynced: true} + case 2: + return Result{Task: task, ErrorClass: "no_record", Code: 14000, ImeiSynced: true, Covered: true, Message: "系统无记录，请检查输入参数是否正确"} + case 3: + return Result{Task: task, ErrorClass: "data_invalid", ImeiSynced: true, Covered: true, Message: "carrier returned no imei value"} + default: + return Result{Task: task, ErrorClass: "business", Code: 11011, Message: "invalid card"} + } + }}, false) + if err := c.handleRunClaims(ctx, claimRun(t, store, "covered", time.Now(), 4)); err != nil { + t.Fatal(err) + } + if queried != 4 { + t.Fatalf("queried = %d, want 4", queried) + } + meta := runMeta(t, store, "covered") + // Three cards produced a usable or definitive answer (success + 2 covered), + // and only the business failure is counted. Without the covered channel the + // two 14000-style answers would have made failedCount=3. + if meta.Status != "completed_with_failures" || meta.SuccessCount != 3 || meta.FailedCount != 1 || meta.NextRoundCount != 0 { + t.Fatalf("unexpected run accounting: %+v", meta) + } + failureKey, _ := store.FailedKey("covered") + size, err := redis.Do(ctx, "HLEN", failureKey) + if err != nil || size.Int64() != 1 { + t.Fatalf("failure pool entries = %d err=%v, want 1 (only the business failure)", size.Int64(), err) + } + for _, taskID := range []string{"covered:1", "covered:2"} { + exists, err := redis.Do(ctx, "HEXISTS", failureKey, taskID) + if err != nil || exists.Int() != 0 { + t.Fatalf("covered card %s leaked into the failure pool: %v %v", taskID, exists, err) + } + } + exists, err := redis.Do(ctx, "HEXISTS", failureKey, "covered:3") + if err != nil || exists.Int() != 1 { + t.Fatalf("the business failure is missing from the pool: %v %v", exists, err) + } +} diff --git a/internal/dao/batch_detail.go b/internal/dao/batch_detail.go new file mode 100644 index 0000000..68fee84 --- /dev/null +++ b/internal/dao/batch_detail.go @@ -0,0 +1,22 @@ +// ================================================================================= +// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed. +// ================================================================================= + +package dao + +import ( + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal" +) + +// batchDetailDao is the data access object for the table batch_detail. +// You can define custom methods on it to extend its functionality as needed. +type batchDetailDao struct { + *internal.BatchDetailDao +} + +var ( + // BatchDetail is a globally accessible object for table batch_detail operations. + BatchDetail = batchDetailDao{internal.NewBatchDetailDao()} +) + +// Add your custom methods and functionality below. diff --git a/internal/dao/batch_main.go b/internal/dao/batch_main.go new file mode 100644 index 0000000..4561891 --- /dev/null +++ b/internal/dao/batch_main.go @@ -0,0 +1,22 @@ +// ================================================================================= +// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed. +// ================================================================================= + +package dao + +import ( + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal" +) + +// batchMainDao is the data access object for the table batch_main. +// You can define custom methods on it to extend its functionality as needed. +type batchMainDao struct { + *internal.BatchMainDao +} + +var ( + // BatchMain is a globally accessible object for table batch_main operations. + BatchMain = batchMainDao{internal.NewBatchMainDao()} +) + +// Add your custom methods and functionality below. diff --git a/internal/dao/internal/batch_detail.go b/internal/dao/internal/batch_detail.go new file mode 100644 index 0000000..2facffc --- /dev/null +++ b/internal/dao/internal/batch_detail.go @@ -0,0 +1,109 @@ +// ========================================================================== +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ========================================================================== + +package internal + +import ( + "context" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +// BatchDetailDao is the data access object for the table batch_detail. +type BatchDetailDao struct { + table string // table is the underlying table name of the DAO. + group string // group is the database configuration group name of the current DAO. + columns BatchDetailColumns // columns contains all the column names of Table for convenient usage. + handlers []gdb.ModelHandler // handlers for customized model modification. +} + +// BatchDetailColumns defines and stores column names for the table batch_detail. +type BatchDetailColumns struct { + Id string // 主键ID + BatchNo string // 批次编号 + BatchType string // 批次类型 枚举（与主表一致） + DetailStatus string // 明细异步处理状态 0-待处理 1-处理中 2-成功 3-失败 + BlankOutStatus string // 作废状态 0-是 1-否 + RowNo string // 行号 + ItemId string // 项ID：卡ICCID/设备编码 + ErrorCode string // 异常编码 + ErrorReason string // 异常原因 + Snapshot string // 信息快照 + TenantId string // 租户ID + CreatedDept string // 创建部门ID + CreatedBy string // 创建者ID + CreatedAt string // 创建时间 + UpdatedBy string // 更新者ID + UpdatedAt string // 更新时间 +} + +// batchDetailColumns holds the columns for the table batch_detail. +var batchDetailColumns = BatchDetailColumns{ + Id: "id", + BatchNo: "batch_no", + BatchType: "batch_type", + DetailStatus: "detail_status", + BlankOutStatus: "blank_out_status", + RowNo: "row_no", + ItemId: "item_id", + ErrorCode: "error_code", + ErrorReason: "error_reason", + Snapshot: "snapshot", + TenantId: "tenant_id", + CreatedDept: "created_dept", + CreatedBy: "created_by", + CreatedAt: "created_at", + UpdatedBy: "updated_by", + UpdatedAt: "updated_at", +} + +// NewBatchDetailDao creates and returns a new DAO object for table data access. +func NewBatchDetailDao(handlers ...gdb.ModelHandler) *BatchDetailDao { + return &BatchDetailDao{ + group: "default", + table: "batch_detail", + columns: batchDetailColumns, + handlers: handlers, + } +} + +// DB retrieves and returns the underlying raw database management object of the current DAO. +func (dao *BatchDetailDao) DB() gdb.DB { + return g.DB(dao.group) +} + +// Table returns the table name of the current DAO. +func (dao *BatchDetailDao) Table() string { + return dao.table +} + +// Columns returns all column names of the current DAO. +func (dao *BatchDetailDao) Columns() BatchDetailColumns { + return dao.columns +} + +// Group returns the database configuration group name of the current DAO. +func (dao *BatchDetailDao) Group() string { + return dao.group +} + +// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation. +func (dao *BatchDetailDao) Ctx(ctx context.Context) *gdb.Model { + model := dao.DB().Model(dao.table) + for _, handler := range dao.handlers { + model = handler(model) + } + return model.Safe().Ctx(ctx) +} + +// Transaction wraps the transaction logic using function f. +// It rolls back the transaction and returns the error if function f returns a non-nil error. +// It commits the transaction and returns nil if function f returns nil. +// +// Note: Do not commit or roll back the transaction in function f, +// as it is automatically handled by this function. +func (dao *BatchDetailDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) { + return dao.Ctx(ctx).Transaction(ctx, f) +} diff --git a/internal/dao/internal/batch_main.go b/internal/dao/internal/batch_main.go new file mode 100644 index 0000000..3ffc6a3 --- /dev/null +++ b/internal/dao/internal/batch_main.go @@ -0,0 +1,139 @@ +// ========================================================================== +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ========================================================================== + +package internal + +import ( + "context" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +// BatchMainDao is the data access object for the table batch_main. +type BatchMainDao struct { + table string // table is the underlying table name of the DAO. + group string // group is the database configuration group name of the current DAO. + columns BatchMainColumns // columns contains all the column names of Table for convenient usage. + handlers []gdb.ModelHandler // handlers for customized model modification. +} + +// BatchMainColumns defines and stores column names for the table batch_main. +type BatchMainColumns struct { + Id string // 主键ID + BatchNo string // 批次编号 + OrderSn string // 订单编号 + BatchType string // 批次类型 枚举 + BatchStatus string // 批次状态 枚举 + BatchAsyncStatus string // 异步处理状态 + SourceType string // 来源方式 枚举 + SourceBatchNo string // 来源批次编号 + SourceOssId string // 源文件OSS ID + ResultOssId string // 结果文件OSS ID + IsCrossDept string // 是否跨级操作机构 + TargetDeptId string // 目标机构编号 + DeptId string // 机构编号 + TotalCount string // 总数量 + ValidCount string // 有效数量 + SuccessCount string // 成功数量 + FailCount string // 失败数量 + RunResult string // 运行结果 + FailReason string // 失败原因 + BatchInfo string // 批次信息：入库/出库展示字段 + Remark string // 备注扩展字段 + StartTime string // 开始时间 + EndTime string // 结束时间 + TenantId string // 租户ID + CreatedDept string // 创建部门ID + CreatedBy string // 创建者ID + CreatedAt string // 创建时间 + UpdatedBy string // 更新者ID + UpdatedAt string // 更新时间 + SourceChannel string // 发起渠道:console/mini_program/api/system/legacy + ExecutionMode string // 操作规模:0未知1单卡2批量 +} + +// batchMainColumns holds the columns for the table batch_main. +var batchMainColumns = BatchMainColumns{ + Id: "id", + BatchNo: "batch_no", + OrderSn: "order_sn", + BatchType: "batch_type", + BatchStatus: "batch_status", + BatchAsyncStatus: "batch_async_status", + SourceType: "source_type", + SourceBatchNo: "source_batch_no", + SourceOssId: "source_oss_id", + ResultOssId: "result_oss_id", + IsCrossDept: "is_cross_dept", + TargetDeptId: "target_dept_id", + DeptId: "dept_id", + TotalCount: "total_count", + ValidCount: "valid_count", + SuccessCount: "success_count", + FailCount: "fail_count", + RunResult: "run_result", + FailReason: "fail_reason", + BatchInfo: "batch_info", + Remark: "remark", + StartTime: "start_time", + EndTime: "end_time", + TenantId: "tenant_id", + CreatedDept: "created_dept", + CreatedBy: "created_by", + CreatedAt: "created_at", + UpdatedBy: "updated_by", + UpdatedAt: "updated_at", + SourceChannel: "source_channel", + ExecutionMode: "execution_mode", +} + +// NewBatchMainDao creates and returns a new DAO object for table data access. +func NewBatchMainDao(handlers ...gdb.ModelHandler) *BatchMainDao { + return &BatchMainDao{ + group: "default", + table: "batch_main", + columns: batchMainColumns, + handlers: handlers, + } +} + +// DB retrieves and returns the underlying raw database management object of the current DAO. +func (dao *BatchMainDao) DB() gdb.DB { + return g.DB(dao.group) +} + +// Table returns the table name of the current DAO. +func (dao *BatchMainDao) Table() string { + return dao.table +} + +// Columns returns all column names of the current DAO. +func (dao *BatchMainDao) Columns() BatchMainColumns { + return dao.columns +} + +// Group returns the database configuration group name of the current DAO. +func (dao *BatchMainDao) Group() string { + return dao.group +} + +// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation. +func (dao *BatchMainDao) Ctx(ctx context.Context) *gdb.Model { + model := dao.DB().Model(dao.table) + for _, handler := range dao.handlers { + model = handler(model) + } + return model.Safe().Ctx(ctx) +} + +// Transaction wraps the transaction logic using function f. +// It rolls back the transaction and returns the error if function f returns a non-nil error. +// It commits the transaction and returns nil if function f returns nil. +// +// Note: Do not commit or roll back the transaction in function f, +// as it is automatically handled by this function. +func (dao *BatchMainDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) { + return dao.Ctx(ctx).Transaction(ctx, f) +} diff --git a/internal/dao/internal/sys_dept.go b/internal/dao/internal/sys_dept.go new file mode 100644 index 0000000..724901e --- /dev/null +++ b/internal/dao/internal/sys_dept.go @@ -0,0 +1,125 @@ +// ========================================================================== +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ========================================================================== + +package internal + +import ( + "context" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +// SysDeptDao is the data access object for the table sys_dept. +type SysDeptDao struct { + table string // table is the underlying table name of the DAO. + group string // group is the database configuration group name of the current DAO. + columns SysDeptColumns // columns contains all the column names of Table for convenient usage. + handlers []gdb.ModelHandler // handlers for customized model modification. +} + +// SysDeptColumns defines and stores column names for the table sys_dept. +type SysDeptColumns struct { + DeptId string // 部门id + TenantId string // 租户编号 + ParentId string // 父部门id + Ancestors string // 祖级列表 + DeptKey string // 机构编号 + DeptName string // 部门名称 + DeptType string // 部门类型(0:部门 1:公司) + DeptCategory string // 部门类别编码 + OrderNum string // 显示顺序 + Leader string // 负责人 + Phone string // 联系电话 + Email string // 邮箱 + Status string // 部门状态（0正常 1停用） + CreditCode string // 统一社会信用代码 + AreaCode string // 行政区域代码 + Address string // 地址 + DeptRole string // 机构默认角色ID + CreatedDept string // 创建部门 + CreatedBy string // 创建者 + CreatedAt string // 创建时间 + UpdatedBy string // 更新者 + UpdatedAt string // 更新时间 + DeletedBy string // 删除人 + DeletedAt string // 删除时间 +} + +// sysDeptColumns holds the columns for the table sys_dept. +var sysDeptColumns = SysDeptColumns{ + DeptId: "dept_id", + TenantId: "tenant_id", + ParentId: "parent_id", + Ancestors: "ancestors", + DeptKey: "dept_key", + DeptName: "dept_name", + DeptType: "dept_type", + DeptCategory: "dept_category", + OrderNum: "order_num", + Leader: "leader", + Phone: "phone", + Email: "email", + Status: "status", + CreditCode: "credit_code", + AreaCode: "area_code", + Address: "address", + DeptRole: "dept_role", + CreatedDept: "created_dept", + CreatedBy: "created_by", + CreatedAt: "created_at", + UpdatedBy: "updated_by", + UpdatedAt: "updated_at", + DeletedBy: "deleted_by", + DeletedAt: "deleted_at", +} + +// NewSysDeptDao creates and returns a new DAO object for table data access. +func NewSysDeptDao(handlers ...gdb.ModelHandler) *SysDeptDao { + return &SysDeptDao{ + group: "default", + table: "sys_dept", + columns: sysDeptColumns, + handlers: handlers, + } +} + +// DB retrieves and returns the underlying raw database management object of the current DAO. +func (dao *SysDeptDao) DB() gdb.DB { + return g.DB(dao.group) +} + +// Table returns the table name of the current DAO. +func (dao *SysDeptDao) Table() string { + return dao.table +} + +// Columns returns all column names of the current DAO. +func (dao *SysDeptDao) Columns() SysDeptColumns { + return dao.columns +} + +// Group returns the database configuration group name of the current DAO. +func (dao *SysDeptDao) Group() string { + return dao.group +} + +// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation. +func (dao *SysDeptDao) Ctx(ctx context.Context) *gdb.Model { + model := dao.DB().Model(dao.table) + for _, handler := range dao.handlers { + model = handler(model) + } + return model.Safe().Ctx(ctx) +} + +// Transaction wraps the transaction logic using function f. +// It rolls back the transaction and returns the error if function f returns a non-nil error. +// It commits the transaction and returns nil if function f returns nil. +// +// Note: Do not commit or roll back the transaction in function f, +// as it is automatically handled by this function. +func (dao *SysDeptDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) { + return dao.Ctx(ctx).Transaction(ctx, f) +} diff --git a/internal/dao/internal/sys_oss.go b/internal/dao/internal/sys_oss.go new file mode 100644 index 0000000..c0ce038 --- /dev/null +++ b/internal/dao/internal/sys_oss.go @@ -0,0 +1,111 @@ +// ========================================================================== +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ========================================================================== + +package internal + +import ( + "context" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +// SysOssDao is the data access object for the table sys_oss. +type SysOssDao struct { + table string // table is the underlying table name of the DAO. + group string // group is the database configuration group name of the current DAO. + columns SysOssColumns // columns contains all the column names of Table for convenient usage. + handlers []gdb.ModelHandler // handlers for customized model modification. +} + +// SysOssColumns defines and stores column names for the table sys_oss. +type SysOssColumns struct { + OssId string // 对象存储主键 + TenantId string // 租户编号 + FileName string // 文件名 + OriginalName string // 原名 + FileSuffix string // 文件后缀名 + Path string // 存储路径 + Url string // URL地址 + CreatedDept string // 创建部门 + CreatedAt string // 创建时间 + CreatedBy string // 创建者 + UpdatedAt string // 更新时间 + UpdatedBy string // 更新者 + Service string // 服务商 + Md5 string // 文件MD5 + FileSize string // 文件大小 + FileCrc16 string // 文件Crc16 + FileSum string // 文件校验和 +} + +// sysOssColumns holds the columns for the table sys_oss. +var sysOssColumns = SysOssColumns{ + OssId: "oss_id", + TenantId: "tenant_id", + FileName: "file_name", + OriginalName: "original_name", + FileSuffix: "file_suffix", + Path: "path", + Url: "url", + CreatedDept: "created_dept", + CreatedAt: "created_at", + CreatedBy: "created_by", + UpdatedAt: "updated_at", + UpdatedBy: "updated_by", + Service: "service", + Md5: "md5", + FileSize: "file_size", + FileCrc16: "file_crc16", + FileSum: "file_sum", +} + +// NewSysOssDao creates and returns a new DAO object for table data access. +func NewSysOssDao(handlers ...gdb.ModelHandler) *SysOssDao { + return &SysOssDao{ + group: "default", + table: "sys_oss", + columns: sysOssColumns, + handlers: handlers, + } +} + +// DB retrieves and returns the underlying raw database management object of the current DAO. +func (dao *SysOssDao) DB() gdb.DB { + return g.DB(dao.group) +} + +// Table returns the table name of the current DAO. +func (dao *SysOssDao) Table() string { + return dao.table +} + +// Columns returns all column names of the current DAO. +func (dao *SysOssDao) Columns() SysOssColumns { + return dao.columns +} + +// Group returns the database configuration group name of the current DAO. +func (dao *SysOssDao) Group() string { + return dao.group +} + +// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation. +func (dao *SysOssDao) Ctx(ctx context.Context) *gdb.Model { + model := dao.DB().Model(dao.table) + for _, handler := range dao.handlers { + model = handler(model) + } + return model.Safe().Ctx(ctx) +} + +// Transaction wraps the transaction logic using function f. +// It rolls back the transaction and returns the error if function f returns a non-nil error. +// It commits the transaction and returns nil if function f returns nil. +// +// Note: Do not commit or roll back the transaction in function f, +// as it is automatically handled by this function. +func (dao *SysOssDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) { + return dao.Ctx(ctx).Transaction(ctx, f) +} diff --git a/internal/dao/internal/t_message_log.go b/internal/dao/internal/t_message_log.go new file mode 100644 index 0000000..a99cdbc --- /dev/null +++ b/internal/dao/internal/t_message_log.go @@ -0,0 +1,107 @@ +// ========================================================================== +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ========================================================================== + +package internal + +import ( + "context" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +// TMessageLogDao is the data access object for the table t_message_log. +type TMessageLogDao struct { + table string // table is the underlying table name of the DAO. + group string // group is the database configuration group name of the current DAO. + columns TMessageLogColumns // columns contains all the column names of Table for convenient usage. + handlers []gdb.ModelHandler // handlers for customized model modification. +} + +// TMessageLogColumns defines and stores column names for the table t_message_log. +type TMessageLogColumns struct { + MessageLogId string // 主键ID + MessageType string // 消息类型 + SendUserType string // 发送用户类型 + UserIds string // 接收用户ID集合(JSON数组) + MessageTitle string // 消息标题 + MessageContent string // 消息内容 + MessageLink string // 跳转链接地址 + SendTime string // 发送时间 + IsRecall string // 是否撤回 + TenantId string // 租户ID + CreatedDept string // 创建部门 + CreatedBy string // 创建者 + CreatedAt string // 创建时间 + UpdatedBy string // 更新者 + UpdatedAt string // 更新时间 +} + +// tMessageLogColumns holds the columns for the table t_message_log. +var tMessageLogColumns = TMessageLogColumns{ + MessageLogId: "message_log_id", + MessageType: "message_type", + SendUserType: "send_user_type", + UserIds: "user_ids", + MessageTitle: "message_title", + MessageContent: "message_content", + MessageLink: "message_link", + SendTime: "send_time", + IsRecall: "is_recall", + TenantId: "tenant_id", + CreatedDept: "created_dept", + CreatedBy: "created_by", + CreatedAt: "created_at", + UpdatedBy: "updated_by", + UpdatedAt: "updated_at", +} + +// NewTMessageLogDao creates and returns a new DAO object for table data access. +func NewTMessageLogDao(handlers ...gdb.ModelHandler) *TMessageLogDao { + return &TMessageLogDao{ + group: "default", + table: "t_message_log", + columns: tMessageLogColumns, + handlers: handlers, + } +} + +// DB retrieves and returns the underlying raw database management object of the current DAO. +func (dao *TMessageLogDao) DB() gdb.DB { + return g.DB(dao.group) +} + +// Table returns the table name of the current DAO. +func (dao *TMessageLogDao) Table() string { + return dao.table +} + +// Columns returns all column names of the current DAO. +func (dao *TMessageLogDao) Columns() TMessageLogColumns { + return dao.columns +} + +// Group returns the database configuration group name of the current DAO. +func (dao *TMessageLogDao) Group() string { + return dao.group +} + +// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation. +func (dao *TMessageLogDao) Ctx(ctx context.Context) *gdb.Model { + model := dao.DB().Model(dao.table) + for _, handler := range dao.handlers { + model = handler(model) + } + return model.Safe().Ctx(ctx) +} + +// Transaction wraps the transaction logic using function f. +// It rolls back the transaction and returns the error if function f returns a non-nil error. +// It commits the transaction and returns nil if function f returns nil. +// +// Note: Do not commit or roll back the transaction in function f, +// as it is automatically handled by this function. +func (dao *TMessageLogDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) { + return dao.Ctx(ctx).Transaction(ctx, f) +} diff --git a/internal/dao/internal/t_resource_batch_download.go b/internal/dao/internal/t_resource_batch_download.go new file mode 100644 index 0000000..c7a8b4d --- /dev/null +++ b/internal/dao/internal/t_resource_batch_download.go @@ -0,0 +1,141 @@ +// ========================================================================== +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ========================================================================== + +package internal + +import ( + "context" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +// TResourceBatchDownloadDao is the data access object for the table t_resource_batch_download. +type TResourceBatchDownloadDao struct { + table string // table is the underlying table name of the DAO. + group string // group is the database configuration group name of the current DAO. + columns TResourceBatchDownloadColumns // columns contains all the column names of Table for convenient usage. + handlers []gdb.ModelHandler // handlers for customized model modification. +} + +// TResourceBatchDownloadColumns defines and stores column names for the table t_resource_batch_download. +type TResourceBatchDownloadColumns struct { + Id string // 主键ID + BatchNo string // 批次编号（全局唯一） + TaskName string // 任务名称 + ResourceType string // 资源类型 枚举 + ExportScene string // 导出场景 枚举 + QueryParams string // 导出查询条件快照 + TotalCount string // 预计导出总条数 + SuccessCount string // 已成功导出条数 + FailCount string // 失败条数 + DownloadStatus string // 下载状态 枚举 + OssId string // OSS文件ID + FileName string // 导出文件名 + FileExt string // 文件扩展名 + FileUrl string // 文件访问地址 + FilePath string // 服务器本地路径 + FileSize string // 文件大小（字节） + FileMd5 string // 文件MD5 + DownloadCount string // 下载次数 + LastDownloadTime string // 最后下载时间 + MaxDownloadCount string // 最大下载次数限制 + ErrorMessage string // 失败原因 + DeptId string // 所属机构ID + StartTime string // 任务开始时间 + FinishTime string // 任务完成时间 + ExpireTime string // 文件过期时间 + Remark string // 备注 + TenantId string // 租户ID + CreatedDept string // 创建部门ID + CreatedBy string // 创建者ID + CreatedAt string // 创建时间 + UpdatedBy string // 更新者ID + UpdatedAt string // 更新时间 +} + +// tResourceBatchDownloadColumns holds the columns for the table t_resource_batch_download. +var tResourceBatchDownloadColumns = TResourceBatchDownloadColumns{ + Id: "id", + BatchNo: "batch_no", + TaskName: "task_name", + ResourceType: "resource_type", + ExportScene: "export_scene", + QueryParams: "query_params", + TotalCount: "total_count", + SuccessCount: "success_count", + FailCount: "fail_count", + DownloadStatus: "download_status", + OssId: "oss_id", + FileName: "file_name", + FileExt: "file_ext", + FileUrl: "file_url", + FilePath: "file_path", + FileSize: "file_size", + FileMd5: "file_md5", + DownloadCount: "download_count", + LastDownloadTime: "last_download_time", + MaxDownloadCount: "max_download_count", + ErrorMessage: "error_message", + DeptId: "dept_id", + StartTime: "start_time", + FinishTime: "finish_time", + ExpireTime: "expire_time", + Remark: "remark", + TenantId: "tenant_id", + CreatedDept: "created_dept", + CreatedBy: "created_by", + CreatedAt: "created_at", + UpdatedBy: "updated_by", + UpdatedAt: "updated_at", +} + +// NewTResourceBatchDownloadDao creates and returns a new DAO object for table data access. +func NewTResourceBatchDownloadDao(handlers ...gdb.ModelHandler) *TResourceBatchDownloadDao { + return &TResourceBatchDownloadDao{ + group: "default", + table: "t_resource_batch_download", + columns: tResourceBatchDownloadColumns, + handlers: handlers, + } +} + +// DB retrieves and returns the underlying raw database management object of the current DAO. +func (dao *TResourceBatchDownloadDao) DB() gdb.DB { + return g.DB(dao.group) +} + +// Table returns the table name of the current DAO. +func (dao *TResourceBatchDownloadDao) Table() string { + return dao.table +} + +// Columns returns all column names of the current DAO. +func (dao *TResourceBatchDownloadDao) Columns() TResourceBatchDownloadColumns { + return dao.columns +} + +// Group returns the database configuration group name of the current DAO. +func (dao *TResourceBatchDownloadDao) Group() string { + return dao.group +} + +// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation. +func (dao *TResourceBatchDownloadDao) Ctx(ctx context.Context) *gdb.Model { + model := dao.DB().Model(dao.table) + for _, handler := range dao.handlers { + model = handler(model) + } + return model.Safe().Ctx(ctx) +} + +// Transaction wraps the transaction logic using function f. +// It rolls back the transaction and returns the error if function f returns a non-nil error. +// It commits the transaction and returns nil if function f returns nil. +// +// Note: Do not commit or roll back the transaction in function f, +// as it is automatically handled by this function. +func (dao *TResourceBatchDownloadDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) { + return dao.Ctx(ctx).Transaction(ctx, f) +} diff --git a/internal/dao/internal/t_sim_card_cancellation.go b/internal/dao/internal/t_sim_card_cancellation.go new file mode 100644 index 0000000..0c2c7bc --- /dev/null +++ b/internal/dao/internal/t_sim_card_cancellation.go @@ -0,0 +1,191 @@ +// ========================================================================== +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ========================================================================== + +package internal + +import ( + "context" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +// TSimCardCancellationDao is the data access object for the table t_sim_card_cancellation. +type TSimCardCancellationDao struct { + table string // table is the underlying table name of the DAO. + group string // group is the database configuration group name of the current DAO. + columns TSimCardCancellationColumns // columns contains all the column names of Table for convenient usage. + handlers []gdb.ModelHandler // handlers for customized model modification. +} + +// TSimCardCancellationColumns defines and stores column names for the table t_sim_card_cancellation. +type TSimCardCancellationColumns struct { + Id string // 主键ID + Iccid string // ICCID + Msisdn string // MSISDN/卡号 + Imsi string // IMSI + Imei string // IMEI + RegionCode string // 归属地 + DeptId string // 渠道客户ID + CurrentPackageCode string // 当前套餐编号 + PackageStatus string // 套餐状态：0 无套餐，1待生效，2生效中，3已过期 + AccessType string // 接入方式：ipv4、ipv6、ipv4_ipv6 + BossCardStatus string // 运营商卡状态 枚举 + CancelBatchNo string // 系统销户批次号 + CancelledAt string // 系统销户完成时间 + CardType string // 卡类型 枚举 + BossOpenAt string // 运营商开户日期 + BossActivateAt string // 运营商激活日期 + IsBossActivateEnabled string // 是否按运营商激活减去赠送时长方：1是，0否 + InboundBatchNo string // 入库批次 + OutboundBatchNo string // 出库批次 + AllocationBatchNo string // 划拨批次 + IsFlowPool string // 是否为池卡：1是，0否 + FlowPoolCode string // 流量池编号 + FlowPkgBeginAt string // 当前套餐生效时间 + FlowPkgPresentAt string // 套餐赠送结束时间 + FlowPkgRealPresentAt string // 实际赠送结束时间 + FlowPkgEndAt string // 当前套餐预计到期时间 + DataServiceStatus string // 数据服务状态：0启用,1停用 + PackageTotalFlowMb string // 套餐总流量MB + UsedFlowMb string // 已用流量MB + TrafficCutoffStatus string // 达量断网配置状态：0未配置，1已开启，2已关闭 + TrafficCutoffThreshold string // 达量断网阈值 + SmsServiceStatus string // 短信服务状态：0启用,1停用 + PackageTotalSms string // 套餐短信总条数 + UsedSms string // 已用短信条数 + VoiceServiceStatus string // 语音服务状态：0启用,1停用 + PackageTotalVoice string // 套餐语音总分钟数 + UsedVoice string // 已用语音分钟数 + RoamServiceStatus string // 国漫服务状态：0启用,1停用 + PackageTotalRoamFlowMb string // 套餐国际流量MB + UsedRoamFlowMb string // 已用国际流量MB + StatusSyncTime string // 状态最后一次同步时间 + FlowSyncTime string // 流量最后一次同步时间 + SmsSyncTime string // 短信最后一次同步时间 + VoiceSyncTime string // 语音最后一次同步时间 + RoamSyncTime string // 国漫最后一次同步时间 + ExtraFeeEnabled string // 是否收取附加费：0是，1否 + ExtraFeeMultiple string // 附加费用倍数 + ExtraFee string // 附加金额 + ChanKey string // 通道编号 + NetModel string // 网络类型 枚举 + Remark string // 备注 + TenantId string // 租户ID + CreatedDept string // 创建部门 + CreatedBy string // 创建者 + CreatedAt string // 创建时间 + UpdatedBy string // 更新者 + UpdatedAt string // 更新时间 +} + +// tSimCardCancellationColumns holds the columns for the table t_sim_card_cancellation. +var tSimCardCancellationColumns = TSimCardCancellationColumns{ + Id: "id", + Iccid: "iccid", + Msisdn: "msisdn", + Imsi: "imsi", + Imei: "imei", + RegionCode: "region_code", + DeptId: "dept_id", + CurrentPackageCode: "current_package_code", + PackageStatus: "package_status", + AccessType: "access_type", + BossCardStatus: "boss_card_status", + CancelBatchNo: "cancel_batch_no", + CancelledAt: "cancelled_at", + CardType: "card_type", + BossOpenAt: "boss_open_at", + BossActivateAt: "boss_activate_at", + IsBossActivateEnabled: "is_boss_activate_enabled", + InboundBatchNo: "inbound_batch_no", + OutboundBatchNo: "outbound_batch_no", + AllocationBatchNo: "allocation_batch_no", + IsFlowPool: "is_flow_pool", + FlowPoolCode: "flow_pool_code", + FlowPkgBeginAt: "flow_pkg_begin_at", + FlowPkgPresentAt: "flow_pkg_present_at", + FlowPkgRealPresentAt: "flow_pkg_real_present_at", + FlowPkgEndAt: "flow_pkg_end_at", + DataServiceStatus: "data_service_status", + PackageTotalFlowMb: "package_total_flow_mb", + UsedFlowMb: "used_flow_mb", + TrafficCutoffStatus: "traffic_cutoff_status", + TrafficCutoffThreshold: "traffic_cutoff_threshold", + SmsServiceStatus: "sms_service_status", + PackageTotalSms: "package_total_sms", + UsedSms: "used_sms", + VoiceServiceStatus: "voice_service_status", + PackageTotalVoice: "package_total_voice", + UsedVoice: "used_voice", + RoamServiceStatus: "roam_service_status", + PackageTotalRoamFlowMb: "package_total_roam_flow_mb", + UsedRoamFlowMb: "used_roam_flow_mb", + StatusSyncTime: "status_sync_time", + FlowSyncTime: "flow_sync_time", + SmsSyncTime: "sms_sync_time", + VoiceSyncTime: "voice_sync_time", + RoamSyncTime: "roam_sync_time", + ExtraFeeEnabled: "extra_fee_enabled", + ExtraFeeMultiple: "extra_fee_multiple", + ExtraFee: "extra_fee", + ChanKey: "chan_key", + NetModel: "net_model", + Remark: "remark", + TenantId: "tenant_id", + CreatedDept: "created_dept", + CreatedBy: "created_by", + CreatedAt: "created_at", + UpdatedBy: "updated_by", + UpdatedAt: "updated_at", +} + +// NewTSimCardCancellationDao creates and returns a new DAO object for table data access. +func NewTSimCardCancellationDao(handlers ...gdb.ModelHandler) *TSimCardCancellationDao { + return &TSimCardCancellationDao{ + group: "default", + table: "t_sim_card_cancellation", + columns: tSimCardCancellationColumns, + handlers: handlers, + } +} + +// DB retrieves and returns the underlying raw database management object of the current DAO. +func (dao *TSimCardCancellationDao) DB() gdb.DB { + return g.DB(dao.group) +} + +// Table returns the table name of the current DAO. +func (dao *TSimCardCancellationDao) Table() string { + return dao.table +} + +// Columns returns all column names of the current DAO. +func (dao *TSimCardCancellationDao) Columns() TSimCardCancellationColumns { + return dao.columns +} + +// Group returns the database configuration group name of the current DAO. +func (dao *TSimCardCancellationDao) Group() string { + return dao.group +} + +// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation. +func (dao *TSimCardCancellationDao) Ctx(ctx context.Context) *gdb.Model { + model := dao.DB().Model(dao.table) + for _, handler := range dao.handlers { + model = handler(model) + } + return model.Safe().Ctx(ctx) +} + +// Transaction wraps the transaction logic using function f. +// It rolls back the transaction and returns the error if function f returns a non-nil error. +// It commits the transaction and returns nil if function f returns nil. +// +// Note: Do not commit or roll back the transaction in function f, +// as it is automatically handled by this function. +func (dao *TSimCardCancellationDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) { + return dao.Ctx(ctx).Transaction(ctx, f) +} diff --git a/internal/dao/internal/t_sim_card_info.go b/internal/dao/internal/t_sim_card_info.go new file mode 100644 index 0000000..7882c42 --- /dev/null +++ b/internal/dao/internal/t_sim_card_info.go @@ -0,0 +1,193 @@ +// ========================================================================== +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ========================================================================== + +package internal + +import ( + "context" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +// TSimCardInfoDao is the data access object for the table t_sim_card_info. +type TSimCardInfoDao struct { + table string // table is the underlying table name of the DAO. + group string // group is the database configuration group name of the current DAO. + columns TSimCardInfoColumns // columns contains all the column names of Table for convenient usage. + handlers []gdb.ModelHandler // handlers for customized model modification. +} + +// TSimCardInfoColumns defines and stores column names for the table t_sim_card_info. +type TSimCardInfoColumns struct { + Id string // 主键ID + Iccid string // ICCID + Msisdn string // MSISDN/卡号 + Imsi string // IMSI + Imei string // IMEI + RegionCode string // 归属地 + DeptId string // 渠道客户ID + CurrentPackageCode string // 当前套餐编号 + PackageStatus string // 套餐状态：0 无套餐，1待生效，2生效中，3已过期 + AccessType string // 接入方式：ipv4、ipv6、ipv4_ipv6 + BossCardStatus string // 运营商卡状态 枚举 + CardType string // 卡类型 枚举 + BossOpenAt string // 运营商开户日期 + BossActivateAt string // 运营商激活日期 + IsBossActivateEnabled string // 是否按运营商激活减去赠送时长方：1是，0否 + InboundBatchNo string // 入库批次 + OutboundBatchNo string // 出库批次 + AllocationBatchNo string // 划拨批次 + IsFlowPool string // 是否为池卡：1是，0否 + FlowPoolCode string // 流量池编号 + OutStockTime string // 出库时间 + FlowPkgBeginAt string // 当前套餐生效时间 + FlowPkgPresentAt string // 套餐赠送结束时间 + FlowPkgRealPresentAt string // 实际赠送结束时间 + FlowPkgEndAt string // 当前套餐预计到期时间 + DataServiceStatus string // 数据服务状态：1启用、0禁用 + PackageTotalFlowMb string // 套餐总流量MB + UsedFlowMb string // 已用流量MB + TrafficCutoffStatus string // 达量断网配置状态：0未配置，1已开启，2已关闭 + TrafficCutoffThreshold string // 达量断网阈值 + SmsServiceStatus string // 短信服务状态：1启用、0禁用 + PackageTotalSms string // 套餐短信总条数 + UsedSms string // 已用短信条数 + VoiceServiceStatus string // 语音服务状态：1启用、0禁用 + PackageTotalVoice string // 套餐语音总分钟数 + UsedVoice string // 已用语音分钟数 + RoamServiceStatus string // 国漫服务状态：1启用、0禁用 + PackageTotalRoamFlowMb string // 套餐国际流量MB + UsedRoamFlowMb string // 已用国际流量MB + StatusSyncTime string // 状态最后一次同步时间 + FlowSyncTime string // 流量最后一次同步时间 + SmsSyncTime string // 短信最后一次同步时间 + VoiceSyncTime string // 语音最后一次同步时间 + RoamSyncTime string // 国漫最后一次同步时间 + ImeiSyncTime string // IMEI 最后一次轮询同步时间 + ImeiUpdateTime string // IMEI 值最后一次实际变更时间 + ExtraFeeEnabled string // 是否收取附加费：1是，0否 + ExtraFeeMultiple string // 附加费用倍数 + ExtraFee string // 附加金额 + ChanKey string // 通道编号 + NetModel string // 网络类型：4G、5G、NB-IoT + Remark string // 备注 + TenantId string // 租户ID + CreatedDept string // 创建部门 + CreatedBy string // 创建者 + CreatedAt string // 创建时间 + UpdatedBy string // 更新者 + UpdatedAt string // 更新时间 +} + +// tSimCardInfoColumns holds the columns for the table t_sim_card_info. +var tSimCardInfoColumns = TSimCardInfoColumns{ + Id: "id", + Iccid: "iccid", + Msisdn: "msisdn", + Imsi: "imsi", + Imei: "imei", + RegionCode: "region_code", + DeptId: "dept_id", + CurrentPackageCode: "current_package_code", + PackageStatus: "package_status", + AccessType: "access_type", + BossCardStatus: "boss_card_status", + CardType: "card_type", + BossOpenAt: "boss_open_at", + BossActivateAt: "boss_activate_at", + IsBossActivateEnabled: "is_boss_activate_enabled", + InboundBatchNo: "inbound_batch_no", + OutboundBatchNo: "outbound_batch_no", + AllocationBatchNo: "allocation_batch_no", + IsFlowPool: "is_flow_pool", + FlowPoolCode: "flow_pool_code", + OutStockTime: "out_stock_time", + FlowPkgBeginAt: "flow_pkg_begin_at", + FlowPkgPresentAt: "flow_pkg_present_at", + FlowPkgRealPresentAt: "flow_pkg_real_present_at", + FlowPkgEndAt: "flow_pkg_end_at", + DataServiceStatus: "data_service_status", + PackageTotalFlowMb: "package_total_flow_mb", + UsedFlowMb: "used_flow_mb", + TrafficCutoffStatus: "traffic_cutoff_status", + TrafficCutoffThreshold: "traffic_cutoff_threshold", + SmsServiceStatus: "sms_service_status", + PackageTotalSms: "package_total_sms", + UsedSms: "used_sms", + VoiceServiceStatus: "voice_service_status", + PackageTotalVoice: "package_total_voice", + UsedVoice: "used_voice", + RoamServiceStatus: "roam_service_status", + PackageTotalRoamFlowMb: "package_total_roam_flow_mb", + UsedRoamFlowMb: "used_roam_flow_mb", + StatusSyncTime: "status_sync_time", + FlowSyncTime: "flow_sync_time", + SmsSyncTime: "sms_sync_time", + VoiceSyncTime: "voice_sync_time", + RoamSyncTime: "roam_sync_time", + ImeiSyncTime: "imei_sync_time", + ImeiUpdateTime: "imei_update_time", + ExtraFeeEnabled: "extra_fee_enabled", + ExtraFeeMultiple: "extra_fee_multiple", + ExtraFee: "extra_fee", + ChanKey: "chan_key", + NetModel: "net_model", + Remark: "remark", + TenantId: "tenant_id", + CreatedDept: "created_dept", + CreatedBy: "created_by", + CreatedAt: "created_at", + UpdatedBy: "updated_by", + UpdatedAt: "updated_at", +} + +// NewTSimCardInfoDao creates and returns a new DAO object for table data access. +func NewTSimCardInfoDao(handlers ...gdb.ModelHandler) *TSimCardInfoDao { + return &TSimCardInfoDao{ + group: "default", + table: "t_sim_card_info", + columns: tSimCardInfoColumns, + handlers: handlers, + } +} + +// DB retrieves and returns the underlying raw database management object of the current DAO. +func (dao *TSimCardInfoDao) DB() gdb.DB { + return g.DB(dao.group) +} + +// Table returns the table name of the current DAO. +func (dao *TSimCardInfoDao) Table() string { + return dao.table +} + +// Columns returns all column names of the current DAO. +func (dao *TSimCardInfoDao) Columns() TSimCardInfoColumns { + return dao.columns +} + +// Group returns the database configuration group name of the current DAO. +func (dao *TSimCardInfoDao) Group() string { + return dao.group +} + +// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation. +func (dao *TSimCardInfoDao) Ctx(ctx context.Context) *gdb.Model { + model := dao.DB().Model(dao.table) + for _, handler := range dao.handlers { + model = handler(model) + } + return model.Safe().Ctx(ctx) +} + +// Transaction wraps the transaction logic using function f. +// It rolls back the transaction and returns the error if function f returns a non-nil error. +// It commits the transaction and returns nil if function f returns nil. +// +// Note: Do not commit or roll back the transaction in function f, +// as it is automatically handled by this function. +func (dao *TSimCardInfoDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) { + return dao.Ctx(ctx).Transaction(ctx, f) +} diff --git a/internal/dao/internal/t_sim_flow_pool.go b/internal/dao/internal/t_sim_flow_pool.go new file mode 100644 index 0000000..72b890a --- /dev/null +++ b/internal/dao/internal/t_sim_flow_pool.go @@ -0,0 +1,119 @@ +// ========================================================================== +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ========================================================================== + +package internal + +import ( + "context" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +// TSimFlowPoolDao is the data access object for the table t_sim_flow_pool. +type TSimFlowPoolDao struct { + table string // table is the underlying table name of the DAO. + group string // group is the database configuration group name of the current DAO. + columns TSimFlowPoolColumns // columns contains all the column names of Table for convenient usage. + handlers []gdb.ModelHandler // handlers for customized model modification. +} + +// TSimFlowPoolColumns defines and stores column names for the table t_sim_flow_pool. +type TSimFlowPoolColumns struct { + Id string // 主键ID + FlowPoolCode string // 流量池编号 + DeptId string // 客户编号 + PackageCodes string // 套餐编号列表（冗余） + PoolPackageCode string // 池套餐编码 + CardCount string // 关联卡数量 + CardFlowMb string // 套餐卡流量MB + PoolPackageFlowMb string // 套内总流量MB + AddOnFlowMb string // 加油包总流量MB + TotalFlowMb string // 总流量MB(套内+加油包) + UsedFlowMb string // 已使用流量MB + RemainFlowMb string // 剩余流量MB + LastStatTime string // 最后流量统计时间 + PoolStatus string // 状态：0 正常、1 停用、2 锁定 + Remark string // 备注 + TenantId string // 租户ID + CreatedDept string // 创建部门 + CreatedBy string // 创建者 + CreatedAt string // 创建时间 + UpdatedBy string // 更新者 + UpdatedAt string // 更新时间 +} + +// tSimFlowPoolColumns holds the columns for the table t_sim_flow_pool. +var tSimFlowPoolColumns = TSimFlowPoolColumns{ + Id: "id", + FlowPoolCode: "flow_pool_code", + DeptId: "dept_id", + PackageCodes: "package_codes", + PoolPackageCode: "pool_package_code", + CardCount: "card_count", + CardFlowMb: "card_flow_mb", + PoolPackageFlowMb: "pool_package_flow_mb", + AddOnFlowMb: "add_on_flow_mb", + TotalFlowMb: "total_flow_mb", + UsedFlowMb: "used_flow_mb", + RemainFlowMb: "remain_flow_mb", + LastStatTime: "last_stat_time", + PoolStatus: "pool_status", + Remark: "remark", + TenantId: "tenant_id", + CreatedDept: "created_dept", + CreatedBy: "created_by", + CreatedAt: "created_at", + UpdatedBy: "updated_by", + UpdatedAt: "updated_at", +} + +// NewTSimFlowPoolDao creates and returns a new DAO object for table data access. +func NewTSimFlowPoolDao(handlers ...gdb.ModelHandler) *TSimFlowPoolDao { + return &TSimFlowPoolDao{ + group: "default", + table: "t_sim_flow_pool", + columns: tSimFlowPoolColumns, + handlers: handlers, + } +} + +// DB retrieves and returns the underlying raw database management object of the current DAO. +func (dao *TSimFlowPoolDao) DB() gdb.DB { + return g.DB(dao.group) +} + +// Table returns the table name of the current DAO. +func (dao *TSimFlowPoolDao) Table() string { + return dao.table +} + +// Columns returns all column names of the current DAO. +func (dao *TSimFlowPoolDao) Columns() TSimFlowPoolColumns { + return dao.columns +} + +// Group returns the database configuration group name of the current DAO. +func (dao *TSimFlowPoolDao) Group() string { + return dao.group +} + +// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation. +func (dao *TSimFlowPoolDao) Ctx(ctx context.Context) *gdb.Model { + model := dao.DB().Model(dao.table) + for _, handler := range dao.handlers { + model = handler(model) + } + return model.Safe().Ctx(ctx) +} + +// Transaction wraps the transaction logic using function f. +// It rolls back the transaction and returns the error if function f returns a non-nil error. +// It commits the transaction and returns nil if function f returns nil. +// +// Note: Do not commit or roll back the transaction in function f, +// as it is automatically handled by this function. +func (dao *TSimFlowPoolDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) { + return dao.Ctx(ctx).Transaction(ctx, f) +} diff --git a/internal/dao/internal/t_user_message.go b/internal/dao/internal/t_user_message.go new file mode 100644 index 0000000..55910e7 --- /dev/null +++ b/internal/dao/internal/t_user_message.go @@ -0,0 +1,97 @@ +// ========================================================================== +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ========================================================================== + +package internal + +import ( + "context" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +// TUserMessageDao is the data access object for the table t_user_message. +type TUserMessageDao struct { + table string // table is the underlying table name of the DAO. + group string // group is the database configuration group name of the current DAO. + columns TUserMessageColumns // columns contains all the column names of Table for convenient usage. + handlers []gdb.ModelHandler // handlers for customized model modification. +} + +// TUserMessageColumns defines and stores column names for the table t_user_message. +type TUserMessageColumns struct { + MessageId string // 主键ID + MessageLogId string // 消息日志ID + UserId string // 用户ID + IsRead string // 是否已读 + TenantId string // 租户ID + CreatedDept string // 创建部门 + CreatedBy string // 创建者 + CreatedAt string // 创建时间 + UpdatedBy string // 更新者 + UpdatedAt string // 更新时间 +} + +// tUserMessageColumns holds the columns for the table t_user_message. +var tUserMessageColumns = TUserMessageColumns{ + MessageId: "message_id", + MessageLogId: "message_log_id", + UserId: "user_id", + IsRead: "is_read", + TenantId: "tenant_id", + CreatedDept: "created_dept", + CreatedBy: "created_by", + CreatedAt: "created_at", + UpdatedBy: "updated_by", + UpdatedAt: "updated_at", +} + +// NewTUserMessageDao creates and returns a new DAO object for table data access. +func NewTUserMessageDao(handlers ...gdb.ModelHandler) *TUserMessageDao { + return &TUserMessageDao{ + group: "default", + table: "t_user_message", + columns: tUserMessageColumns, + handlers: handlers, + } +} + +// DB retrieves and returns the underlying raw database management object of the current DAO. +func (dao *TUserMessageDao) DB() gdb.DB { + return g.DB(dao.group) +} + +// Table returns the table name of the current DAO. +func (dao *TUserMessageDao) Table() string { + return dao.table +} + +// Columns returns all column names of the current DAO. +func (dao *TUserMessageDao) Columns() TUserMessageColumns { + return dao.columns +} + +// Group returns the database configuration group name of the current DAO. +func (dao *TUserMessageDao) Group() string { + return dao.group +} + +// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation. +func (dao *TUserMessageDao) Ctx(ctx context.Context) *gdb.Model { + model := dao.DB().Model(dao.table) + for _, handler := range dao.handlers { + model = handler(model) + } + return model.Safe().Ctx(ctx) +} + +// Transaction wraps the transaction logic using function f. +// It rolls back the transaction and returns the error if function f returns a non-nil error. +// It commits the transaction and returns nil if function f returns nil. +// +// Note: Do not commit or roll back the transaction in function f, +// as it is automatically handled by this function. +func (dao *TUserMessageDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) { + return dao.Ctx(ctx).Transaction(ctx, f) +} diff --git a/internal/dao/internal/t_warn_config.go b/internal/dao/internal/t_warn_config.go new file mode 100644 index 0000000..1747b93 --- /dev/null +++ b/internal/dao/internal/t_warn_config.go @@ -0,0 +1,123 @@ +// ========================================================================== +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ========================================================================== + +package internal + +import ( + "context" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +// TWarnConfigDao is the data access object for the table t_warn_config. +type TWarnConfigDao struct { + table string // table is the underlying table name of the DAO. + group string // group is the database configuration group name of the current DAO. + columns TWarnConfigColumns // columns contains all the column names of Table for convenient usage. + handlers []gdb.ModelHandler // handlers for customized model modification. +} + +// TWarnConfigColumns defines and stores column names for the table t_warn_config. +type TWarnConfigColumns struct { + Id string // 主键ID + TenantId string // 租户ID + BatchNo string // 批次编号 + ItemType string // 对象类型:1卡,2流量池 + ItemId string // 对象ID:单卡为t_sim_card_info.id,流量池为t_sim_flow_pool.id + ItemCode string // 对象编码快照:ICCID/MSISDN/流量池编码 + WarnUnit string // 预警单位:1MB,2百分比,3天数,4月数 + FirstWarn string // 第一档预警阈值,单位由warn_unit决定 + SecondWarn string // 第二档预警阈值,单位由warn_unit决定 + ThirdWarn string // 第三档预警阈值,单位由warn_unit决定 + WarnMethods string // 预警方式:0短信,1邮箱,2电话,3平台消息;多选逗号分隔 + WarnPhones string // 预警手机号,多个逗号分隔 + WarnEmails string // 预警邮箱,多个逗号分隔 + AutoAction string // 自动动作:0无,2断网;第一期只保存配置不自动执行 + IsEnabled string // 是否启用:1启用,0停用 + WxTemplateId string // 微信订阅消息模板ID + Remark string // 备注 + CreatedDept string // 创建部门 + CreatedBy string // 创建者 + CreatedAt string // 创建时间 + UpdatedBy string // 更新者 + UpdatedAt string // 更新时间 + AppNotifyEnabled string // 是否开启小程序预警通知:0关闭,1开启 +} + +// tWarnConfigColumns holds the columns for the table t_warn_config. +var tWarnConfigColumns = TWarnConfigColumns{ + Id: "id", + TenantId: "tenant_id", + BatchNo: "batch_no", + ItemType: "item_type", + ItemId: "item_id", + ItemCode: "item_code", + WarnUnit: "warn_unit", + FirstWarn: "first_warn", + SecondWarn: "second_warn", + ThirdWarn: "third_warn", + WarnMethods: "warn_methods", + WarnPhones: "warn_phones", + WarnEmails: "warn_emails", + AutoAction: "auto_action", + IsEnabled: "is_enabled", + WxTemplateId: "wx_template_id", + Remark: "remark", + CreatedDept: "created_dept", + CreatedBy: "created_by", + CreatedAt: "created_at", + UpdatedBy: "updated_by", + UpdatedAt: "updated_at", + AppNotifyEnabled: "app_notify_enabled", +} + +// NewTWarnConfigDao creates and returns a new DAO object for table data access. +func NewTWarnConfigDao(handlers ...gdb.ModelHandler) *TWarnConfigDao { + return &TWarnConfigDao{ + group: "default", + table: "t_warn_config", + columns: tWarnConfigColumns, + handlers: handlers, + } +} + +// DB retrieves and returns the underlying raw database management object of the current DAO. +func (dao *TWarnConfigDao) DB() gdb.DB { + return g.DB(dao.group) +} + +// Table returns the table name of the current DAO. +func (dao *TWarnConfigDao) Table() string { + return dao.table +} + +// Columns returns all column names of the current DAO. +func (dao *TWarnConfigDao) Columns() TWarnConfigColumns { + return dao.columns +} + +// Group returns the database configuration group name of the current DAO. +func (dao *TWarnConfigDao) Group() string { + return dao.group +} + +// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation. +func (dao *TWarnConfigDao) Ctx(ctx context.Context) *gdb.Model { + model := dao.DB().Model(dao.table) + for _, handler := range dao.handlers { + model = handler(model) + } + return model.Safe().Ctx(ctx) +} + +// Transaction wraps the transaction logic using function f. +// It rolls back the transaction and returns the error if function f returns a non-nil error. +// It commits the transaction and returns nil if function f returns nil. +// +// Note: Do not commit or roll back the transaction in function f, +// as it is automatically handled by this function. +func (dao *TWarnConfigDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) { + return dao.Ctx(ctx).Transaction(ctx, f) +} diff --git a/internal/dao/internal/t_warn_log.go b/internal/dao/internal/t_warn_log.go new file mode 100644 index 0000000..97cba18 --- /dev/null +++ b/internal/dao/internal/t_warn_log.go @@ -0,0 +1,125 @@ +// ========================================================================== +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ========================================================================== + +package internal + +import ( + "context" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +// TWarnLogDao is the data access object for the table t_warn_log. +type TWarnLogDao struct { + table string // table is the underlying table name of the DAO. + group string // group is the database configuration group name of the current DAO. + columns TWarnLogColumns // columns contains all the column names of Table for convenient usage. + handlers []gdb.ModelHandler // handlers for customized model modification. +} + +// TWarnLogColumns defines and stores column names for the table t_warn_log. +type TWarnLogColumns struct { + Id string // 主键ID + TenantId string // 租户ID + ConfigId string // 预警配置ID + ItemType string // 对象类型:1单卡,2流量池 + ItemId string // 对象ID:单卡为t_sim_card_info.id,流量池为t_sim_flow_pool.id + ItemCode string // 对象编码快照:ICCID/MSISDN/流量池编码 + WarnStage string // 触发档位:1第一档,2第二档,3第三档 + WarnLevel string // 预警级别:1提示,2注意,3警告,4危险,5紧急 + WarnUnit string // 预警单位/维度:1MB,2百分比,3天数,4月数 + ThresholdValue string // 触发阈值,单位由warn_unit决定 + ActualValue string // 触发时实际值,单位由warn_unit决定 + TotalFlowMb string // 触发时总流量MB;流量类预警使用 + RemainFlowMb string // 触发时剩余流量MB;流量类预警使用 + WarnMethods string // 实际预警方式快照 + WarnPhones string // 实际预警手机号快照 + WarnEmails string // 实际预警邮箱快照 + MessageLogId string // 平台消息ID;聚合通知时可能为空 + WarnTime string // 预警时间 + Remark string // 备注 + CreatedDept string // 创建部门 + CreatedBy string // 创建者 + CreatedAt string // 创建时间 + UpdatedBy string // 更新者 + UpdatedAt string // 更新时间 +} + +// tWarnLogColumns holds the columns for the table t_warn_log. +var tWarnLogColumns = TWarnLogColumns{ + Id: "id", + TenantId: "tenant_id", + ConfigId: "config_id", + ItemType: "item_type", + ItemId: "item_id", + ItemCode: "item_code", + WarnStage: "warn_stage", + WarnLevel: "warn_level", + WarnUnit: "warn_unit", + ThresholdValue: "threshold_value", + ActualValue: "actual_value", + TotalFlowMb: "total_flow_mb", + RemainFlowMb: "remain_flow_mb", + WarnMethods: "warn_methods", + WarnPhones: "warn_phones", + WarnEmails: "warn_emails", + MessageLogId: "message_log_id", + WarnTime: "warn_time", + Remark: "remark", + CreatedDept: "created_dept", + CreatedBy: "created_by", + CreatedAt: "created_at", + UpdatedBy: "updated_by", + UpdatedAt: "updated_at", +} + +// NewTWarnLogDao creates and returns a new DAO object for table data access. +func NewTWarnLogDao(handlers ...gdb.ModelHandler) *TWarnLogDao { + return &TWarnLogDao{ + group: "default", + table: "t_warn_log", + columns: tWarnLogColumns, + handlers: handlers, + } +} + +// DB retrieves and returns the underlying raw database management object of the current DAO. +func (dao *TWarnLogDao) DB() gdb.DB { + return g.DB(dao.group) +} + +// Table returns the table name of the current DAO. +func (dao *TWarnLogDao) Table() string { + return dao.table +} + +// Columns returns all column names of the current DAO. +func (dao *TWarnLogDao) Columns() TWarnLogColumns { + return dao.columns +} + +// Group returns the database configuration group name of the current DAO. +func (dao *TWarnLogDao) Group() string { + return dao.group +} + +// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation. +func (dao *TWarnLogDao) Ctx(ctx context.Context) *gdb.Model { + model := dao.DB().Model(dao.table) + for _, handler := range dao.handlers { + model = handler(model) + } + return model.Safe().Ctx(ctx) +} + +// Transaction wraps the transaction logic using function f. +// It rolls back the transaction and returns the error if function f returns a non-nil error. +// It commits the transaction and returns nil if function f returns nil. +// +// Note: Do not commit or roll back the transaction in function f, +// as it is automatically handled by this function. +func (dao *TWarnLogDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) { + return dao.Ctx(ctx).Transaction(ctx, f) +} diff --git a/internal/dao/internal/t_wo_main.go b/internal/dao/internal/t_wo_main.go new file mode 100644 index 0000000..06a6dea --- /dev/null +++ b/internal/dao/internal/t_wo_main.go @@ -0,0 +1,137 @@ +// ========================================================================== +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ========================================================================== + +package internal + +import ( + "context" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +// TWoMainDao is the data access object for the table t_wo_main. +type TWoMainDao struct { + table string // table is the underlying table name of the DAO. + group string // group is the database configuration group name of the current DAO. + columns TWoMainColumns // columns contains all the column names of Table for convenient usage. + handlers []gdb.ModelHandler // handlers for customized model modification. +} + +// TWoMainColumns defines and stores column names for the table t_wo_main. +type TWoMainColumns struct { + Id string // 主键ID + WoNo string // 工单编号 + WoType string // 工单类型 0卡,1模组,2设备,3订单,4余额,5提现 + RelatedId string // 关联业务ID + Title string // 工单标题 + Content string // 问题描述 + Attachment string // 附件 + CreatorType string // 创建人类型：1终端用户，2代理商 + CreatorDeptId string // 创建机构ID + CreatorUserId string // 创建用户ID + CreatorName string // 创建人姓名 + CreatorPhone string // 创建人手机号 + AssignDeptId string // 分派方机构ID + AssignUserId string // 分派人用户ID + CurrentStatus string // 当前处理状态：1已分派，2处理中，3已解决，4已完结_关单，5已取消 + HandlerDeptId string // 处理方机构ID + HandlerUserId string // 处理人用户ID + FirstResponseTime string // 首次响应时间 + ResolveTime string // 确认时间 + CloseTime string // 关闭时间 + SatisfactionLevel string // 满意度：1-5分 + EvalContent string // 评价内容 + EvalTime string // 评价时间 + Remark string // 备注 + TenantId string // 租户ID + CreatedDept string // 创建部门 + CreatedBy string // 创建者 + CreatedAt string // 创建时间 + UpdatedBy string // 更新者 + UpdatedAt string // 更新时间 +} + +// tWoMainColumns holds the columns for the table t_wo_main. +var tWoMainColumns = TWoMainColumns{ + Id: "id", + WoNo: "wo_no", + WoType: "wo_type", + RelatedId: "related_id", + Title: "title", + Content: "content", + Attachment: "attachment", + CreatorType: "creator_type", + CreatorDeptId: "creator_dept_id", + CreatorUserId: "creator_user_id", + CreatorName: "creator_name", + CreatorPhone: "creator_phone", + AssignDeptId: "assign_dept_id", + AssignUserId: "assign_user_id", + CurrentStatus: "current_status", + HandlerDeptId: "handler_dept_id", + HandlerUserId: "handler_user_id", + FirstResponseTime: "first_response_time", + ResolveTime: "resolve_time", + CloseTime: "close_time", + SatisfactionLevel: "satisfaction_level", + EvalContent: "eval_content", + EvalTime: "eval_time", + Remark: "remark", + TenantId: "tenant_id", + CreatedDept: "created_dept", + CreatedBy: "created_by", + CreatedAt: "created_at", + UpdatedBy: "updated_by", + UpdatedAt: "updated_at", +} + +// NewTWoMainDao creates and returns a new DAO object for table data access. +func NewTWoMainDao(handlers ...gdb.ModelHandler) *TWoMainDao { + return &TWoMainDao{ + group: "default", + table: "t_wo_main", + columns: tWoMainColumns, + handlers: handlers, + } +} + +// DB retrieves and returns the underlying raw database management object of the current DAO. +func (dao *TWoMainDao) DB() gdb.DB { + return g.DB(dao.group) +} + +// Table returns the table name of the current DAO. +func (dao *TWoMainDao) Table() string { + return dao.table +} + +// Columns returns all column names of the current DAO. +func (dao *TWoMainDao) Columns() TWoMainColumns { + return dao.columns +} + +// Group returns the database configuration group name of the current DAO. +func (dao *TWoMainDao) Group() string { + return dao.group +} + +// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation. +func (dao *TWoMainDao) Ctx(ctx context.Context) *gdb.Model { + model := dao.DB().Model(dao.table) + for _, handler := range dao.handlers { + model = handler(model) + } + return model.Safe().Ctx(ctx) +} + +// Transaction wraps the transaction logic using function f. +// It rolls back the transaction and returns the error if function f returns a non-nil error. +// It commits the transaction and returns nil if function f returns nil. +// +// Note: Do not commit or roll back the transaction in function f, +// as it is automatically handled by this function. +func (dao *TWoMainDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) { + return dao.Ctx(ctx).Transaction(ctx, f) +} diff --git a/internal/dao/internal/t_wo_record.go b/internal/dao/internal/t_wo_record.go new file mode 100644 index 0000000..365bf7c --- /dev/null +++ b/internal/dao/internal/t_wo_record.go @@ -0,0 +1,107 @@ +// ========================================================================== +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ========================================================================== + +package internal + +import ( + "context" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +// TWoRecordDao is the data access object for the table t_wo_record. +type TWoRecordDao struct { + table string // table is the underlying table name of the DAO. + group string // group is the database configuration group name of the current DAO. + columns TWoRecordColumns // columns contains all the column names of Table for convenient usage. + handlers []gdb.ModelHandler // handlers for customized model modification. +} + +// TWoRecordColumns defines and stores column names for the table t_wo_record. +type TWoRecordColumns struct { + Id string // 主键ID + WoNo string // 工单编号 + OperationType string // 操作类型：0创建，1分派，2处理，3完成，4关单，5取消 + OperationDeptId string // 操作机构ID + OperationUserId string // 操作用户ID + TargetDeptId string // 目标部门ID + TargetUserId string // 目标用户ID + BeforeStatus string // 变更前状态 + AfterStatus string // 变更后状态 + TenantId string // 租户ID + CreatedDept string // 创建部门 + CreatedBy string // 创建者 + CreatedAt string // 创建时间 + UpdatedBy string // 更新者 + UpdatedAt string // 更新时间 +} + +// tWoRecordColumns holds the columns for the table t_wo_record. +var tWoRecordColumns = TWoRecordColumns{ + Id: "id", + WoNo: "wo_no", + OperationType: "operation_type", + OperationDeptId: "operation_dept_id", + OperationUserId: "operation_user_id", + TargetDeptId: "target_dept_id", + TargetUserId: "target_user_id", + BeforeStatus: "before_status", + AfterStatus: "after_status", + TenantId: "tenant_id", + CreatedDept: "created_dept", + CreatedBy: "created_by", + CreatedAt: "created_at", + UpdatedBy: "updated_by", + UpdatedAt: "updated_at", +} + +// NewTWoRecordDao creates and returns a new DAO object for table data access. +func NewTWoRecordDao(handlers ...gdb.ModelHandler) *TWoRecordDao { + return &TWoRecordDao{ + group: "default", + table: "t_wo_record", + columns: tWoRecordColumns, + handlers: handlers, + } +} + +// DB retrieves and returns the underlying raw database management object of the current DAO. +func (dao *TWoRecordDao) DB() gdb.DB { + return g.DB(dao.group) +} + +// Table returns the table name of the current DAO. +func (dao *TWoRecordDao) Table() string { + return dao.table +} + +// Columns returns all column names of the current DAO. +func (dao *TWoRecordDao) Columns() TWoRecordColumns { + return dao.columns +} + +// Group returns the database configuration group name of the current DAO. +func (dao *TWoRecordDao) Group() string { + return dao.group +} + +// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation. +func (dao *TWoRecordDao) Ctx(ctx context.Context) *gdb.Model { + model := dao.DB().Model(dao.table) + for _, handler := range dao.handlers { + model = handler(model) + } + return model.Safe().Ctx(ctx) +} + +// Transaction wraps the transaction logic using function f. +// It rolls back the transaction and returns the error if function f returns a non-nil error. +// It commits the transaction and returns nil if function f returns nil. +// +// Note: Do not commit or roll back the transaction in function f, +// as it is automatically handled by this function. +func (dao *TWoRecordDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) { + return dao.Ctx(ctx).Transaction(ctx, f) +} diff --git a/internal/dao/sys_dept.go b/internal/dao/sys_dept.go new file mode 100644 index 0000000..2fa0cb8 --- /dev/null +++ b/internal/dao/sys_dept.go @@ -0,0 +1,22 @@ +// ================================================================================= +// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed. +// ================================================================================= + +package dao + +import ( + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal" +) + +// sysDeptDao is the data access object for the table sys_dept. +// You can define custom methods on it to extend its functionality as needed. +type sysDeptDao struct { + *internal.SysDeptDao +} + +var ( + // SysDept is a globally accessible object for table sys_dept operations. + SysDept = sysDeptDao{internal.NewSysDeptDao()} +) + +// Add your custom methods and functionality below. diff --git a/internal/dao/sys_oss.go b/internal/dao/sys_oss.go new file mode 100644 index 0000000..db17375 --- /dev/null +++ b/internal/dao/sys_oss.go @@ -0,0 +1,22 @@ +// ================================================================================= +// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed. +// ================================================================================= + +package dao + +import ( + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal" +) + +// sysOssDao is the data access object for the table sys_oss. +// You can define custom methods on it to extend its functionality as needed. +type sysOssDao struct { + *internal.SysOssDao +} + +var ( + // SysOss is a globally accessible object for table sys_oss operations. + SysOss = sysOssDao{internal.NewSysOssDao()} +) + +// Add your custom methods and functionality below. diff --git a/internal/dao/t_message_log.go b/internal/dao/t_message_log.go new file mode 100644 index 0000000..4b80428 --- /dev/null +++ b/internal/dao/t_message_log.go @@ -0,0 +1,22 @@ +// ================================================================================= +// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed. +// ================================================================================= + +package dao + +import ( + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal" +) + +// tMessageLogDao is the data access object for the table t_message_log. +// You can define custom methods on it to extend its functionality as needed. +type tMessageLogDao struct { + *internal.TMessageLogDao +} + +var ( + // TMessageLog is a globally accessible object for table t_message_log operations. + TMessageLog = tMessageLogDao{internal.NewTMessageLogDao()} +) + +// Add your custom methods and functionality below. diff --git a/internal/dao/t_resource_batch_download.go b/internal/dao/t_resource_batch_download.go new file mode 100644 index 0000000..cb99255 --- /dev/null +++ b/internal/dao/t_resource_batch_download.go @@ -0,0 +1,22 @@ +// ================================================================================= +// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed. +// ================================================================================= + +package dao + +import ( + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal" +) + +// tResourceBatchDownloadDao is the data access object for the table t_resource_batch_download. +// You can define custom methods on it to extend its functionality as needed. +type tResourceBatchDownloadDao struct { + *internal.TResourceBatchDownloadDao +} + +var ( + // TResourceBatchDownload is a globally accessible object for table t_resource_batch_download operations. + TResourceBatchDownload = tResourceBatchDownloadDao{internal.NewTResourceBatchDownloadDao()} +) + +// Add your custom methods and functionality below. diff --git a/internal/dao/t_sim_card_cancellation.go b/internal/dao/t_sim_card_cancellation.go new file mode 100644 index 0000000..2dfcd94 --- /dev/null +++ b/internal/dao/t_sim_card_cancellation.go @@ -0,0 +1,22 @@ +// ================================================================================= +// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed. +// ================================================================================= + +package dao + +import ( + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal" +) + +// tSimCardCancellationDao is the data access object for the table t_sim_card_cancellation. +// You can define custom methods on it to extend its functionality as needed. +type tSimCardCancellationDao struct { + *internal.TSimCardCancellationDao +} + +var ( + // TSimCardCancellation is a globally accessible object for table t_sim_card_cancellation operations. + TSimCardCancellation = tSimCardCancellationDao{internal.NewTSimCardCancellationDao()} +) + +// Add your custom methods and functionality below. diff --git a/internal/dao/t_sim_card_info.go b/internal/dao/t_sim_card_info.go new file mode 100644 index 0000000..fbd31dd --- /dev/null +++ b/internal/dao/t_sim_card_info.go @@ -0,0 +1,22 @@ +// ================================================================================= +// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed. +// ================================================================================= + +package dao + +import ( + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal" +) + +// tSimCardInfoDao is the data access object for the table t_sim_card_info. +// You can define custom methods on it to extend its functionality as needed. +type tSimCardInfoDao struct { + *internal.TSimCardInfoDao +} + +var ( + // TSimCardInfo is a globally accessible object for table t_sim_card_info operations. + TSimCardInfo = tSimCardInfoDao{internal.NewTSimCardInfoDao()} +) + +// Add your custom methods and functionality below. diff --git a/internal/dao/t_sim_flow_pool.go b/internal/dao/t_sim_flow_pool.go new file mode 100644 index 0000000..81e312c --- /dev/null +++ b/internal/dao/t_sim_flow_pool.go @@ -0,0 +1,22 @@ +// ================================================================================= +// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed. +// ================================================================================= + +package dao + +import ( + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal" +) + +// tSimFlowPoolDao is the data access object for the table t_sim_flow_pool. +// You can define custom methods on it to extend its functionality as needed. +type tSimFlowPoolDao struct { + *internal.TSimFlowPoolDao +} + +var ( + // TSimFlowPool is a globally accessible object for table t_sim_flow_pool operations. + TSimFlowPool = tSimFlowPoolDao{internal.NewTSimFlowPoolDao()} +) + +// Add your custom methods and functionality below. diff --git a/internal/dao/t_user_message.go b/internal/dao/t_user_message.go new file mode 100644 index 0000000..a6cecd9 --- /dev/null +++ b/internal/dao/t_user_message.go @@ -0,0 +1,22 @@ +// ================================================================================= +// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed. +// ================================================================================= + +package dao + +import ( + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal" +) + +// tUserMessageDao is the data access object for the table t_user_message. +// You can define custom methods on it to extend its functionality as needed. +type tUserMessageDao struct { + *internal.TUserMessageDao +} + +var ( + // TUserMessage is a globally accessible object for table t_user_message operations. + TUserMessage = tUserMessageDao{internal.NewTUserMessageDao()} +) + +// Add your custom methods and functionality below. diff --git a/internal/dao/t_warn_config.go b/internal/dao/t_warn_config.go new file mode 100644 index 0000000..13bba62 --- /dev/null +++ b/internal/dao/t_warn_config.go @@ -0,0 +1,22 @@ +// ================================================================================= +// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed. +// ================================================================================= + +package dao + +import ( + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal" +) + +// tWarnConfigDao is the data access object for the table t_warn_config. +// You can define custom methods on it to extend its functionality as needed. +type tWarnConfigDao struct { + *internal.TWarnConfigDao +} + +var ( + // TWarnConfig is a globally accessible object for table t_warn_config operations. + TWarnConfig = tWarnConfigDao{internal.NewTWarnConfigDao()} +) + +// Add your custom methods and functionality below. diff --git a/internal/dao/t_warn_log.go b/internal/dao/t_warn_log.go new file mode 100644 index 0000000..d810293 --- /dev/null +++ b/internal/dao/t_warn_log.go @@ -0,0 +1,22 @@ +// ================================================================================= +// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed. +// ================================================================================= + +package dao + +import ( + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal" +) + +// tWarnLogDao is the data access object for the table t_warn_log. +// You can define custom methods on it to extend its functionality as needed. +type tWarnLogDao struct { + *internal.TWarnLogDao +} + +var ( + // TWarnLog is a globally accessible object for table t_warn_log operations. + TWarnLog = tWarnLogDao{internal.NewTWarnLogDao()} +) + +// Add your custom methods and functionality below. diff --git a/internal/dao/t_wo_main.go b/internal/dao/t_wo_main.go new file mode 100644 index 0000000..6e22f12 --- /dev/null +++ b/internal/dao/t_wo_main.go @@ -0,0 +1,22 @@ +// ================================================================================= +// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed. +// ================================================================================= + +package dao + +import ( + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal" +) + +// tWoMainDao is the data access object for the table t_wo_main. +// You can define custom methods on it to extend its functionality as needed. +type tWoMainDao struct { + *internal.TWoMainDao +} + +var ( + // TWoMain is a globally accessible object for table t_wo_main operations. + TWoMain = tWoMainDao{internal.NewTWoMainDao()} +) + +// Add your custom methods and functionality below. diff --git a/internal/dao/t_wo_record.go b/internal/dao/t_wo_record.go new file mode 100644 index 0000000..819b6fa --- /dev/null +++ b/internal/dao/t_wo_record.go @@ -0,0 +1,22 @@ +// ================================================================================= +// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed. +// ================================================================================= + +package dao + +import ( + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal" +) + +// tWoRecordDao is the data access object for the table t_wo_record. +// You can define custom methods on it to extend its functionality as needed. +type tWoRecordDao struct { + *internal.TWoRecordDao +} + +var ( + // TWoRecord is a globally accessible object for table t_wo_record operations. + TWoRecord = tWoRecordDao{internal.NewTWoRecordDao()} +) + +// Add your custom methods and functionality below. diff --git a/internal/fullsync/archive/archive.go b/internal/fullsync/archive/archive.go new file mode 100644 index 0000000..b368cd5 --- /dev/null +++ b/internal/fullsync/archive/archive.go @@ -0,0 +1,802 @@ +// Package archive durably stores summary-only receipts for completed full-sync +// runs and replay actions. It depends only on the shared full-sync contracts; +// repository implementations and service wiring live elsewhere. +package archive + +import ( + "bufio" + "context" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "io" + "os" + "path/filepath" + "sort" + "strings" + "sync" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" +) + +const ( + defaultInterval = time.Minute + defaultBackoff = time.Second + defaultMaxBackoff = time.Minute + defaultMaxAttempts = 4 + defaultReplayLeaseTTL = time.Minute + defaultReplayLeaseName = "fullsync-archive" + defaultReleaseTimeout = 5 * time.Second + maxRounds = 10000 +) + +var errChecksumMismatch = gerror.New("archive checksum verification failed") + +// Config controls the recovery scanner. Root is the archive directory. Zero +// timing values select conservative defaults. +type Config struct { + Root string + Interval time.Duration + InitialBackoff time.Duration + MaxBackoff time.Duration + MaxAttempts int + ReplayLeaseOwner string + ReplayLeaseTTL time.Duration + Now func() time.Time +} + +// ReplayRepositories are the contract boundaries used to recover replay +// actions left in receipt_pending after a process restart. +type ReplayRepositories struct { + Discovery contract.ReplayWorkDiscovery + Leases contract.ReplayLeaseStore + Progress contract.ReplayProgressStore +} + +// Service publishes archive files and scans durable repository indexes at +// startup and periodically. A Service may be configured for runs, actions, or +// both. +type Service struct { + root string + runDiscovery contract.ArchiveWorkDiscovery + runLoader contract.ArchiveWorkLoader + runRegistrar contract.ArchiveRegistrar + replay ReplayRepositories + interval time.Duration + initialBackoff time.Duration + maxBackoff time.Duration + maxAttempts int + leaseOwner string + leaseTTL time.Duration + now func() time.Time + + // Publications are serialized so repeat writes in this process observe the + // same final file timestamp and therefore produce the same receipt. + publishMu sync.Mutex + + mu sync.Mutex + running bool + cancel context.CancelFunc + done chan struct{} + + // These hooks keep failure-path tests deterministic without weakening the + // production filesystem path. + replaceFile func(string, string) error + afterReplace func(string) error +} + +// New constructs a run-archive service with default scanner settings. +func New(root string, discovery contract.ArchiveWorkDiscovery, loader contract.ArchiveWorkLoader, registrar contract.ArchiveRegistrar) *Service { + return NewService(Config{Root: root}, discovery, loader, registrar) +} + +// NewService constructs a run-archive service. Use NewServiceWithReplay when +// replay receipt recovery is also required. +func NewService(cfg Config, discovery contract.ArchiveWorkDiscovery, loader contract.ArchiveWorkLoader, registrar contract.ArchiveRegistrar) *Service { + return NewServiceWithReplay(cfg, discovery, loader, registrar, ReplayRepositories{}) +} + +// NewServiceWithReplay constructs a lifecycle-enabled run and replay archive +// service using only shared contract interfaces. +func NewServiceWithReplay( + cfg Config, + discovery contract.ArchiveWorkDiscovery, + loader contract.ArchiveWorkLoader, + registrar contract.ArchiveRegistrar, + replay ReplayRepositories, +) *Service { + if cfg.Interval <= 0 { + cfg.Interval = defaultInterval + } + if cfg.InitialBackoff <= 0 { + cfg.InitialBackoff = defaultBackoff + } + if cfg.MaxBackoff <= 0 { + cfg.MaxBackoff = defaultMaxBackoff + } + if cfg.MaxBackoff < cfg.InitialBackoff { + cfg.MaxBackoff = cfg.InitialBackoff + } + if cfg.MaxAttempts <= 0 { + cfg.MaxAttempts = defaultMaxAttempts + } + if strings.TrimSpace(cfg.ReplayLeaseOwner) == "" { + cfg.ReplayLeaseOwner = defaultReplayLeaseName + } + if cfg.ReplayLeaseTTL <= 0 { + cfg.ReplayLeaseTTL = defaultReplayLeaseTTL + } + if cfg.Now == nil { + cfg.Now = time.Now + } + + return &Service{ + root: cfg.Root, + runDiscovery: discovery, + runLoader: loader, + runRegistrar: registrar, + replay: replay, + interval: cfg.Interval, + initialBackoff: cfg.InitialBackoff, + maxBackoff: cfg.MaxBackoff, + maxAttempts: cfg.MaxAttempts, + leaseOwner: cfg.ReplayLeaseOwner, + leaseTTL: cfg.ReplayLeaseTTL, + now: cfg.Now, + replaceFile: atomicReplace, + afterReplace: func(string) error { return nil }, + } +} + +// Start launches an immediate recovery scan followed by periodic scans. It is +// idempotent while the service is running. +func (s *Service) Start(parent context.Context) error { + if parent == nil { + return gerror.New("archive parent context is nil") + } + if err := s.validateConfiguration(); err != nil { + return err + } + + s.mu.Lock() + if s.running { + s.mu.Unlock() + return nil + } + ctx, cancel := context.WithCancel(parent) + done := make(chan struct{}) + s.cancel, s.done, s.running = cancel, done, true + s.mu.Unlock() + + go func() { + defer func() { + s.mu.Lock() + if s.done == done { + s.running = false + s.cancel = nil + } + s.mu.Unlock() + close(done) + }() + s.scanLoop(ctx) + }() + return nil +} + +// Stop cancels the scanner and waits for it to finish. It is safe to call more +// than once. +func (s *Service) Stop() error { + s.mu.Lock() + if !s.running { + s.mu.Unlock() + return nil + } + cancel, done := s.cancel, s.done + s.mu.Unlock() + + cancel() + <-done + return nil +} + +// Close implements the common service lifecycle convention. +func (s *Service) Close() error { return s.Stop() } + +// Done is closed when the current scanner invocation exits. It returns nil +// before the service has been started. +func (s *Service) Done() <-chan struct{} { + s.mu.Lock() + defer s.mu.Unlock() + return s.done +} + +// RunOnce performs one recovery pass. It is useful for explicit lifecycle +// wiring and deterministic integration tests. +func (s *Service) RunOnce(ctx context.Context) error { + if ctx == nil { + return gerror.New("archive scan context is nil") + } + if err := s.validateConfiguration(); err != nil { + return err + } + + var scanErrors []error + if s.runDiscovery != nil { + if err := s.scanRuns(ctx); err != nil { + scanErrors = append(scanErrors, gerror.Wrap(err, "scan run archives")) + } + } + if s.replay.Discovery != nil && ctx.Err() == nil { + if err := s.scanActions(ctx); err != nil { + scanErrors = append(scanErrors, gerror.Wrap(err, "scan replay action archives")) + } + } + return errors.Join(scanErrors...) +} + +// PublishRun atomically writes a run summary and then conditionally registers +// the durable receipt against the work version. +func (s *Service) PublishRun(ctx context.Context, work contract.ArchiveWork, snapshot contract.RunArchive) (contract.ArchiveReceipt, error) { + if err := validateWork(work); err != nil { + return contract.ArchiveReceipt{}, err + } + normalized, err := normalizeRunArchive(work, snapshot) + if err != nil { + return contract.ArchiveReceipt{}, err + } + + pathTime := normalized.Run.CompletedAt + if pathTime.IsZero() { + pathTime = normalized.Run.CreatedAt + } + receipt, err := s.publishJSON(ctx, "runs", work.RunID, pathTime, normalized) + if err != nil { + return contract.ArchiveReceipt{}, err + } + if s.runRegistrar != nil { + if err := s.runRegistrar.RegisterArchive(ctx, work, receipt); err != nil { + return contract.ArchiveReceipt{}, gerror.Wrap(err, "register run archive") + } + } + return receipt, nil +} + +// PublishAction atomically writes a receipt_pending replay action summary. The +// repository update is performed by ArchiveAction because it requires a fenced +// replay lease. +func (s *Service) PublishAction(ctx context.Context, action contract.ReplayAction) (contract.ArchiveReceipt, error) { + normalized, err := normalizeAction(action) + if err != nil { + return contract.ArchiveReceipt{}, err + } + return s.publishJSON(ctx, "actions", normalized.ActionID, normalized.CreatedAt, normalized) +} + +// ArchiveRun loads, publishes, and registers one durable run work item. +func (s *Service) ArchiveRun(ctx context.Context, work contract.ArchiveWork) error { + if s.runLoader == nil { + return gerror.New("archive run loader is nil") + } + if s.runRegistrar == nil { + return gerror.New("archive run registrar is nil") + } + + snapshot, err := s.runLoader.LoadArchiveWork(ctx, work) + if err != nil { + s.recordRunFailure(ctx, work, err) + return gerror.Wrap(err, "load run archive work") + } + if _, err = s.PublishRun(ctx, work, snapshot); err != nil { + s.recordRunFailure(ctx, work, err) + return err + } + return nil +} + +// ArchiveAction acquires a replay lease, atomically publishes the action +// summary, and fences the completed receipt update through ReplayProgressStore. +func (s *Service) ArchiveAction(ctx context.Context, action contract.ReplayAction) error { + if s.replay.Leases == nil || s.replay.Progress == nil { + return gerror.New("replay archive repositories are incomplete") + } + if action.Status != contract.ActionReceiptPending { + return gerror.Wrap(contract.ErrInvalidArgument, "replay action is not receipt_pending") + } + if _, err := normalizeAction(action); err != nil { + return err + } + + lease, err := s.replay.Leases.AcquireReplayLease(ctx, action.ActionID, s.leaseOwner, s.leaseTTL) + if err != nil { + return gerror.Wrap(err, "acquire replay archive lease") + } + defer func() { + releaseCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), defaultReleaseTimeout) + defer cancel() + _ = s.replay.Leases.ReleaseReplayLease(releaseCtx, lease) + }() + + receipt, err := s.PublishAction(ctx, action) + if err != nil { + return err + } + lease, err = s.replay.Leases.RenewReplayLease(ctx, lease, s.leaseTTL) + if err != nil { + return gerror.Wrap(err, "renew replay archive lease") + } + completedAt := receipt.ArchivedAt + _, err = s.replay.Progress.UpdateReplayAction(ctx, contract.ReplayActionUpdate{ + Lease: lease, + ExpectedRevision: action.Revision, + Status: contract.ActionCompleted, + Progress: action.Progress, + Receipt: &receipt, + CompletedAt: &completedAt, + }) + if err != nil { + return gerror.Wrap(err, "register replay action archive") + } + return nil +} + +func (s *Service) scanLoop(ctx context.Context) { + _ = s.RunOnce(ctx) + ticker := time.NewTicker(s.interval) + defer ticker.Stop() + for { + select { + case <-ctx.Done(): + return + case <-ticker.C: + _ = s.RunOnce(ctx) + } + } +} + +func (s *Service) scanRuns(ctx context.Context) error { + page := contract.PageRequest{Limit: contract.DefaultPageLimit} + seenCursors := map[contract.Cursor]struct{}{page.Cursor: {}} + var itemErrors []error + for { + if err := ctx.Err(); err != nil { + return err + } + result, err := s.runDiscovery.ListArchiveWork(ctx, page) + if err != nil { + return err + } + for _, work := range result.Items { + err := s.retry(ctx, func() error { return s.ArchiveRun(ctx, work) }) + if err != nil { + if ctx.Err() != nil { + return ctx.Err() + } + itemErrors = append(itemErrors, gerror.Wrapf(err, "archive run %q", work.RunID)) + } + } + if result.NextCursor == "" { + return errors.Join(itemErrors...) + } + if _, repeated := seenCursors[result.NextCursor]; repeated { + return errors.Join(append(itemErrors, gerror.New("run archive pagination cursor did not advance"))...) + } + seenCursors[result.NextCursor] = struct{}{} + page.Cursor = result.NextCursor + } +} + +func (s *Service) scanActions(ctx context.Context) error { + page := contract.PageRequest{Limit: contract.DefaultPageLimit} + seenCursors := map[contract.Cursor]struct{}{page.Cursor: {}} + var itemErrors []error + for { + if err := ctx.Err(); err != nil { + return err + } + result, err := s.replay.Discovery.ListReplayWork(ctx, page) + if err != nil { + return err + } + for _, action := range result.Items { + if action.Status != contract.ActionReceiptPending { + continue + } + err := s.retry(ctx, func() error { return s.ArchiveAction(ctx, action) }) + if errors.Is(err, contract.ErrBusy) { + continue + } + if err != nil { + if ctx.Err() != nil { + return ctx.Err() + } + itemErrors = append(itemErrors, gerror.Wrapf(err, "archive replay action %q", action.ActionID)) + } + } + if result.NextCursor == "" { + return errors.Join(itemErrors...) + } + if _, repeated := seenCursors[result.NextCursor]; repeated { + return errors.Join(append(itemErrors, gerror.New("replay archive pagination cursor did not advance"))...) + } + seenCursors[result.NextCursor] = struct{}{} + page.Cursor = result.NextCursor + } +} + +func (s *Service) retry(ctx context.Context, fn func() error) error { + delay := s.initialBackoff + var lastErr error + for attempt := 0; attempt < s.maxAttempts; attempt++ { + if err := ctx.Err(); err != nil { + return err + } + lastErr = fn() + if lastErr == nil || errors.Is(lastErr, contract.ErrBusy) { + return lastErr + } + if attempt+1 == s.maxAttempts { + break + } + timer := time.NewTimer(delay) + select { + case <-ctx.Done(): + if !timer.Stop() { + <-timer.C + } + return ctx.Err() + case <-timer.C: + } + if delay <= s.maxBackoff/2 { + delay *= 2 + } else { + delay = s.maxBackoff + } + } + return lastErr +} + +func (s *Service) recordRunFailure(ctx context.Context, work contract.ArchiveWork, archiveErr error) { + if s.runRegistrar == nil || ctx.Err() != nil { + return + } + _ = s.runRegistrar.RegisterArchiveFailure(ctx, work, contract.BoundedErrorMessage(archiveErr.Error())) +} + +func (s *Service) publishJSON(ctx context.Context, kind, id string, pathTime time.Time, value any) (contract.ArchiveReceipt, error) { + if ctx == nil { + return contract.ArchiveReceipt{}, gerror.New("archive publication context is nil") + } + if err := ctx.Err(); err != nil { + return contract.ArchiveReceipt{}, err + } + dir, final, relative, err := s.storagePath(kind, id, pathTime) + if err != nil { + return contract.ArchiveReceipt{}, err + } + + s.publishMu.Lock() + defer s.publishMu.Unlock() + + if err := os.MkdirAll(dir, 0o755); err != nil { + return contract.ArchiveReceipt{}, gerror.Wrap(err, "create archive directory") + } + if err := ctx.Err(); err != nil { + return contract.ArchiveReceipt{}, err + } + + temp, err := os.CreateTemp(dir, "."+id+"-*.tmp") + if err != nil { + return contract.ArchiveReceipt{}, gerror.Wrap(err, "create archive temporary file") + } + tempName := temp.Name() + defer func() { _ = os.Remove(tempName) }() + + hasher := sha256.New() + buffer := bufio.NewWriter(io.MultiWriter(contextWriter{ctx: ctx, writer: temp}, hasher)) + encoder := json.NewEncoder(buffer) + if err = encoder.Encode(value); err == nil { + err = buffer.Flush() + } + if err == nil { + err = temp.Chmod(0o644) + } + if err == nil { + err = temp.Sync() + } + closeErr := temp.Close() + if err == nil { + err = closeErr + } + if err != nil { + _ = temp.Close() + return contract.ArchiveReceipt{}, gerror.Wrap(err, "encode archive temporary file") + } + expectedChecksum := hex.EncodeToString(hasher.Sum(nil)) + + if receipt, ok, existingErr := s.existingReceipt(final, relative, kind, id, expectedChecksum); existingErr != nil { + return contract.ArchiveReceipt{}, existingErr + } else if ok { + if err := syncDirectory(dir); err != nil { + return contract.ArchiveReceipt{}, gerror.Wrap(err, "sync archive directory") + } + return receipt, nil + } + + if err := ctx.Err(); err != nil { + return contract.ArchiveReceipt{}, err + } + if err := s.replaceFile(tempName, final); err != nil { + return contract.ArchiveReceipt{}, gerror.Wrap(err, "publish archive file") + } + if err := syncDirectory(dir); err != nil { + return contract.ArchiveReceipt{}, gerror.Wrap(err, "sync archive directory") + } + if err := s.afterReplace(final); err != nil { + return contract.ArchiveReceipt{}, gerror.Wrap(err, "archive post-publish verification hook") + } + + stored, err := os.ReadFile(final) + if err != nil { + return contract.ArchiveReceipt{}, gerror.Wrap(err, "read published archive file") + } + storedSum := sha256.Sum256(stored) + if hex.EncodeToString(storedSum[:]) != expectedChecksum { + return contract.ArchiveReceipt{}, errChecksumMismatch + } + info, err := os.Stat(final) + if err != nil { + return contract.ArchiveReceipt{}, gerror.Wrap(err, "stat published archive file") + } + receipt := contract.ArchiveReceipt{ + Version: contract.ArchiveSchemaVersion, + Path: relative, + Checksum: expectedChecksum, + ArchivedAt: archiveModTime(info, s.now), + } + if err := receipt.Validate(); err != nil { + return contract.ArchiveReceipt{}, err + } + return receipt, nil +} + +func (s *Service) existingReceipt(final, relative, kind, id, expectedChecksum string) (contract.ArchiveReceipt, bool, error) { + info, err := os.Lstat(final) + if errors.Is(err, os.ErrNotExist) { + return contract.ArchiveReceipt{}, false, nil + } + if err != nil { + return contract.ArchiveReceipt{}, false, gerror.Wrap(err, "inspect existing archive file") + } + if !info.Mode().IsRegular() { + return contract.ArchiveReceipt{}, false, nil + } + stored, err := os.ReadFile(final) + if err != nil { + return contract.ArchiveReceipt{}, false, gerror.Wrap(err, "read existing archive file") + } + storedSum := sha256.Sum256(stored) + if hex.EncodeToString(storedSum[:]) != expectedChecksum { + if validExistingArchive(kind, id, stored) { + return contract.ArchiveReceipt{}, false, gerror.Wrap(contract.ErrConflict, "archive path already contains different valid content") + } + return contract.ArchiveReceipt{}, false, nil + } + receipt := contract.ArchiveReceipt{ + Version: contract.ArchiveSchemaVersion, + Path: relative, + Checksum: expectedChecksum, + ArchivedAt: archiveModTime(info, s.now), + } + if err := receipt.Validate(); err != nil { + return contract.ArchiveReceipt{}, false, err + } + return receipt, true, nil +} + +func validExistingArchive(kind, id string, data []byte) bool { + switch kind { + case "runs": + var snapshot contract.RunArchive + if json.Unmarshal(data, &snapshot) != nil { + return false + } + _, err := normalizeRunArchive(contract.ArchiveWork{RunID: id, Version: "existing"}, snapshot) + return err == nil + case "actions": + var action contract.ReplayAction + if json.Unmarshal(data, &action) != nil || action.ActionID != id { + return false + } + _, err := normalizeAction(action) + return err == nil + default: + return false + } +} + +func archiveModTime(info os.FileInfo, now func() time.Time) time.Time { + archivedAt := info.ModTime().UTC() + if archivedAt.IsZero() { + archivedAt = now().UTC() + } + return archivedAt +} + +func (s *Service) storagePath(kind, id string, pathTime time.Time) (string, string, string, error) { + if err := validateStorageID(id); err != nil { + return "", "", "", err + } + if kind != "runs" && kind != "actions" { + return "", "", "", gerror.Wrap(contract.ErrInvalidArgument, "unknown archive kind") + } + if strings.TrimSpace(s.root) == "" { + return "", "", "", gerror.New("archive root is empty") + } + if pathTime.IsZero() { + pathTime = s.now().UTC() + } else { + pathTime = pathTime.UTC() + } + if pathTime.Year() < 1 || pathTime.Year() > 9999 { + return "", "", "", gerror.Wrap(contract.ErrInvalidArgument, "archive timestamp is outside the supported year range") + } + + root := filepath.Clean(s.root) + prefix := "run-" + if kind == "actions" { + prefix = "action-" + } + relativeOS := prefix + id + ".json" + final := filepath.Join(root, relativeOS) + contained, err := filepath.Rel(root, final) + if err != nil || filepath.IsAbs(contained) || contained == ".." || strings.HasPrefix(contained, ".."+string(filepath.Separator)) { + return "", "", "", gerror.Wrap(contract.ErrInvalidArgument, "archive path escapes root") + } + return filepath.Dir(final), final, filepath.ToSlash(relativeOS), nil +} + +func (s *Service) validateConfiguration() error { + if strings.TrimSpace(s.root) == "" { + return gerror.New("archive root is empty") + } + if s.runDiscovery != nil && (s.runLoader == nil || s.runRegistrar == nil) { + return gerror.New("run archive repositories are incomplete") + } + if s.replay.Discovery != nil && (s.replay.Leases == nil || s.replay.Progress == nil) { + return gerror.New("replay archive repositories are incomplete") + } + return nil +} + +func validateWork(work contract.ArchiveWork) error { + if err := validateStorageID(work.RunID); err != nil { + return err + } + if strings.TrimSpace(work.Version) == "" { + return gerror.Wrap(contract.ErrInvalidArgument, "archive work version is required") + } + return nil +} + +func validateStorageID(id string) error { + if err := contract.ValidateID(id); err != nil { + return err + } + if id == "." || id == ".." || filepath.VolumeName(id) != "" || strings.ContainsRune(id, ':') || filepath.Base(id) != id { + return gerror.Wrap(contract.ErrInvalidArgument, "resource identifier is not safe for archive storage") + } + name := strings.ToUpper(strings.TrimSuffix(id, filepath.Ext(id))) + switch name { + case "CON", "PRN", "AUX", "NUL", "COM1", "COM2", "COM3", "COM4", "COM5", "COM6", "COM7", "COM8", "COM9", "LPT1", "LPT2", "LPT3", "LPT4", "LPT5", "LPT6", "LPT7", "LPT8", "LPT9": + return gerror.Wrap(contract.ErrInvalidArgument, "resource identifier is reserved by the filesystem") + } + return nil +} + +func normalizeRunArchive(work contract.ArchiveWork, snapshot contract.RunArchive) (contract.RunArchive, error) { + if snapshot.Version == 0 { + snapshot.Version = contract.ArchiveSchemaVersion + } + if snapshot.Version != contract.ArchiveSchemaVersion { + return contract.RunArchive{}, gerror.Wrap(contract.ErrInvalidArgument, "invalid archive schema version") + } + if snapshot.Run.RunID != work.RunID { + return contract.RunArchive{}, gerror.Wrap(contract.ErrInvalidArgument, "archive run ID does not match work") + } + if !snapshot.Run.Status.Terminal() { + return contract.RunArchive{}, gerror.Wrap(contract.ErrInvalidArgument, "only terminal runs can be archived") + } + if err := snapshot.Run.Validate(); err != nil { + return contract.RunArchive{}, err + } + if len(snapshot.Rounds) > maxRounds { + return contract.RunArchive{}, gerror.Wrap(contract.ErrInvalidArgument, "too many archived rounds") + } + + snapshot.Run = normalizeRunMeta(snapshot.Run) + snapshot.Rounds = append([]contract.RoundStats(nil), snapshot.Rounds...) + if snapshot.Rounds == nil { + snapshot.Rounds = make([]contract.RoundStats, 0) + } + seen := make(map[int]struct{}, len(snapshot.Rounds)) + for i := range snapshot.Rounds { + round := &snapshot.Rounds[i] + if round.RunID != work.RunID { + return contract.RunArchive{}, gerror.Wrap(contract.ErrInvalidArgument, "archived round belongs to another run") + } + if _, duplicate := seen[round.Round]; duplicate { + return contract.RunArchive{}, gerror.Wrap(contract.ErrInvalidArgument, "duplicate archived round") + } + seen[round.Round] = struct{}{} + if err := round.Validate(); err != nil { + return contract.RunArchive{}, err + } + normalizedCounts, err := round.ErrorCounts.Normalized() + if err != nil { + return contract.RunArchive{}, err + } + round.ErrorCounts = normalizedCounts + round.StartedAt = utcTimePtr(round.StartedAt) + round.CompletedAt = utcTimePtr(round.CompletedAt) + } + sort.Slice(snapshot.Rounds, func(i, j int) bool { return snapshot.Rounds[i].Round < snapshot.Rounds[j].Round }) + return snapshot, nil +} + +func normalizeRunMeta(meta contract.RunMeta) contract.RunMeta { + meta.CreatedAt = meta.CreatedAt.UTC() + meta.ProducerCompletedAt = meta.ProducerCompletedAt.UTC() + meta.CompletedAt = meta.CompletedAt.UTC() + meta.StartedAt = utcTimePtr(meta.StartedAt) + meta.ScanCompletedAt = utcTimePtr(meta.ScanCompletedAt) + // Archive state describes the repository registration and is intentionally + // excluded from the file whose receipt populates that state. + meta.ArchiveStatus = "" + meta.ArchiveVersion = 0 + meta.ArchivePath = "" + meta.ArchiveChecksum = "" + meta.ArchivedAt = nil + return meta +} + +func normalizeAction(action contract.ReplayAction) (contract.ReplayAction, error) { + if err := validateStorageID(action.ActionID); err != nil { + return contract.ReplayAction{}, err + } + if action.Status != contract.ActionReceiptPending { + return contract.ReplayAction{}, gerror.Wrap(contract.ErrInvalidArgument, "only receipt_pending replay actions can be archived") + } + action.Error = contract.BoundedErrorMessage(action.Error) + action.CreatedAt = action.CreatedAt.UTC() + action.UpdatedAt = action.UpdatedAt.UTC() + action.CompletedAt = nil + action.Receipt = nil + if err := action.Validate(); err != nil { + return contract.ReplayAction{}, err + } + return action, nil +} + +func utcTimePtr(value *time.Time) *time.Time { + if value == nil { + return nil + } + utc := value.UTC() + return &utc +} + +type contextWriter struct { + ctx context.Context + writer io.Writer +} + +func (w contextWriter) Write(p []byte) (int, error) { + if err := w.ctx.Err(); err != nil { + return 0, err + } + return w.writer.Write(p) +} diff --git a/internal/fullsync/archive/archive_test.go b/internal/fullsync/archive/archive_test.go new file mode 100644 index 0000000..fc79c3e --- /dev/null +++ b/internal/fullsync/archive/archive_test.go @@ -0,0 +1,588 @@ +package archive + +import ( + "bytes" + "context" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "os" + "path/filepath" + "reflect" + "runtime" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" +) + +func TestPublishRunIsDurableDeterministicAndSummaryOnly(t *testing.T) { + root := t.TempDir() + repository := newFakeRunRepository(testRunArchive("run-1")) + service := New(root, nil, nil, repository) + work := contract.ArchiveWork{RunID: "run-1", Version: "version-1"} + + staleDir := root + if err := os.MkdirAll(staleDir, 0o755); err != nil { + t.Fatal(err) + } + stalePath := filepath.Join(staleDir, ".run-1-stale.tmp") + if err := os.WriteFile(stalePath, []byte("incomplete"), 0o600); err != nil { + t.Fatal(err) + } + + receipt, err := service.PublishRun(context.Background(), work, repository.snapshot) + if err != nil { + t.Fatalf("PublishRun() error = %v", err) + } + if receipt.Path != "run-run-1.json" { + t.Fatalf("receipt path = %q", receipt.Path) + } + if err := receipt.Validate(); err != nil { + t.Fatalf("receipt validation failed: %v", err) + } + + archivePath := filepath.Join(root, filepath.FromSlash(receipt.Path)) + info, err := os.Stat(archivePath) + if err != nil { + t.Fatal(err) + } + if runtime.GOOS != "windows" { + if got := info.Mode().Perm(); got != 0o644 { + t.Fatalf("archive mode = %o, want 644", got) + } + } + data, err := os.ReadFile(archivePath) + if err != nil { + t.Fatal(err) + } + sum := sha256.Sum256(data) + if got := hex.EncodeToString(sum[:]); got != receipt.Checksum { + t.Fatalf("checksum = %q, want %q", receipt.Checksum, got) + } + for _, forbidden := range [][]byte{[]byte(`"archiveStatus"`), []byte(`"archivePath"`), []byte(`"taskId"`), []byte(`"card"`), []byte(`"message"`)} { + if bytes.Contains(data, forbidden) { + t.Fatalf("archive contains forbidden detail %s: %s", forbidden, data) + } + } + var decoded contract.RunArchive + if err := json.Unmarshal(data, &decoded); err != nil { + t.Fatal(err) + } + if len(decoded.Rounds) != 2 || decoded.Rounds[0].Round != 1 || decoded.Rounds[1].Round != 2 { + t.Fatalf("rounds were not deterministically sorted: %+v", decoded.Rounds) + } + if decoded.Rounds[1].ErrorCounts[contract.ErrorRateLimited] != 1 { + t.Fatalf("error categories were not normalized: %+v", decoded.Rounds[1].ErrorCounts) + } + + repeated, err := service.PublishRun(context.Background(), work, repository.snapshot) + if err != nil { + t.Fatalf("repeated PublishRun() error = %v", err) + } + if !reflect.DeepEqual(repeated, receipt) { + t.Fatalf("repeated receipt changed:\nfirst: %+v\nsecond: %+v", receipt, repeated) + } + if _, err := os.Stat(stalePath); err != nil { + t.Fatalf("stale temp residue should not block publication: %v", err) + } + repository.mu.Lock() + defer repository.mu.Unlock() + if len(repository.receipts) != 2 || !reflect.DeepEqual(repository.receipts[0], repository.receipts[1]) { + t.Fatalf("registered receipts = %+v", repository.receipts) + } +} + +func TestArchiveActionPublishesAndRegistersFencedReceipt(t *testing.T) { + root := t.TempDir() + action := testReplayAction("action-1") + action.Error = strings.Repeat("x", contract.MaxErrorMessageBytes+50) + repository := newFakeReplayRepository(action) + service := NewServiceWithReplay( + Config{Root: root}, + nil, + nil, + nil, + ReplayRepositories{Discovery: repository, Leases: repository, Progress: repository}, + ) + + if err := service.ArchiveAction(context.Background(), action); err != nil { + t.Fatalf("ArchiveAction() error = %v", err) + } + repository.mu.Lock() + update := repository.update + released := repository.released + repository.mu.Unlock() + if update == nil || update.Status != contract.ActionCompleted || update.Receipt == nil || update.CompletedAt == nil { + t.Fatalf("registered update = %+v", update) + } + if update.ExpectedRevision != action.Revision || update.Receipt.ArchivedAt != *update.CompletedAt { + t.Fatalf("receipt registration lost fencing/time: %+v", update) + } + if update.Lease.Token != "lease-token-2" { + t.Fatalf("receipt registration used the unrenewed lease: %+v", update.Lease) + } + if released.Token != "lease-token-2" { + t.Fatalf("archive released the stale lease: %+v", released) + } + if update.Receipt.Path != "action-action-1.json" { + t.Fatalf("action receipt path = %q", update.Receipt.Path) + } + + data, err := os.ReadFile(filepath.Join(root, filepath.FromSlash(update.Receipt.Path))) + if err != nil { + t.Fatal(err) + } + for _, forbidden := range [][]byte{[]byte("idempotencyKey"), []byte("fingerprint"), []byte("leaseOwner"), []byte("leaseToken"), []byte("revision"), []byte(`"receipt"`), []byte("completedAt")} { + if bytes.Contains(data, forbidden) { + t.Fatalf("action archive contains internal field %q: %s", forbidden, data) + } + } + var decoded contract.ReplayAction + if err := json.Unmarshal(data, &decoded); err != nil { + t.Fatal(err) + } + if decoded.Status != contract.ActionReceiptPending || len(decoded.Error) != contract.MaxErrorMessageBytes { + t.Fatalf("archived action was not bounded receipt_pending summary: %+v", decoded) + } + repeated, err := service.PublishAction(context.Background(), action) + if err != nil { + t.Fatalf("repeated PublishAction() error = %v", err) + } + if !reflect.DeepEqual(repeated, *update.Receipt) { + t.Fatalf("repeated action receipt changed: %+v != %+v", repeated, *update.Receipt) + } +} + +func TestPublishRunCleansTempAfterRenameFailure(t *testing.T) { + root := t.TempDir() + repository := newFakeRunRepository(testRunArchive("run-rename")) + service := New(root, nil, nil, repository) + renameErr := gerror.New("injected rename failure") + service.replaceFile = func(string, string) error { return renameErr } + + _, err := service.PublishRun(context.Background(), contract.ArchiveWork{RunID: "run-rename", Version: "v1"}, repository.snapshot) + if !errors.Is(err, renameErr) { + t.Fatalf("PublishRun() error = %v, want rename failure", err) + } + if matches, globErr := filepath.Glob(filepath.Join(root, ".run-rename-*.tmp")); globErr != nil || len(matches) != 0 { + t.Fatalf("temporary files after failure = %v, error = %v", matches, globErr) + } + if _, statErr := os.Stat(filepath.Join(root, "run-run-rename.json")); !errors.Is(statErr, os.ErrNotExist) { + t.Fatalf("final file exists after failed rename: %v", statErr) + } + if len(repository.receipts) != 0 { + t.Fatalf("registered receipt after failed rename: %+v", repository.receipts) + } +} + +func TestChecksumFailureIsRecoveredByExactPathReplacement(t *testing.T) { + root := t.TempDir() + repository := newFakeRunRepository(testRunArchive("run-checksum")) + service := New(root, nil, nil, repository) + service.afterReplace = func(path string) error { + return os.WriteFile(path, []byte("corrupt"), 0o600) + } + work := contract.ArchiveWork{RunID: "run-checksum", Version: "v1"} + + if _, err := service.PublishRun(context.Background(), work, repository.snapshot); !errors.Is(err, errChecksumMismatch) { + t.Fatalf("PublishRun() error = %v, want checksum mismatch", err) + } + if len(repository.receipts) != 0 { + t.Fatalf("registered receipt after checksum failure: %+v", repository.receipts) + } + service.afterReplace = func(string) error { return nil } + receipt, err := service.PublishRun(context.Background(), work, repository.snapshot) + if err != nil { + t.Fatalf("recovery PublishRun() error = %v", err) + } + if receipt.Path != "run-run-checksum.json" { + t.Fatalf("replacement used different path: %q", receipt.Path) + } + data, err := os.ReadFile(filepath.Join(root, filepath.FromSlash(receipt.Path))) + if err != nil { + t.Fatal(err) + } + if string(data) == "corrupt" { + t.Fatal("corrupt final file was not replaced") + } +} + +func TestRegistrationRetryReusesIdenticalReceipt(t *testing.T) { + root := t.TempDir() + repository := newFakeRunRepository(testRunArchive("run-register")) + repository.registerErr = gerror.New("registration result was uncertain") + service := New(root, nil, nil, repository) + work := contract.ArchiveWork{RunID: "run-register", Version: "v1"} + + if _, err := service.PublishRun(context.Background(), work, repository.snapshot); !errors.Is(err, repository.registerErr) { + t.Fatalf("first PublishRun() error = %v", err) + } + repository.mu.Lock() + firstReceipt := repository.receipts[0] + repository.registerErr = nil + repository.mu.Unlock() + + receipt, err := service.PublishRun(context.Background(), work, repository.snapshot) + if err != nil { + t.Fatalf("retry PublishRun() error = %v", err) + } + if !reflect.DeepEqual(receipt, firstReceipt) { + t.Fatalf("retry changed uncertain receipt:\nfirst: %+v\nretry: %+v", firstReceipt, receipt) + } +} + +func TestPublishRunRejectsDifferentValidContentAtSamePath(t *testing.T) { + root := t.TempDir() + repository := newFakeRunRepository(testRunArchive("run-conflict")) + service := New(root, nil, nil, repository) + work := contract.ArchiveWork{RunID: "run-conflict", Version: "v1"} + + first, err := service.PublishRun(context.Background(), work, repository.snapshot) + if err != nil { + t.Fatal(err) + } + changed := repository.snapshot + changed.Run.TriggerSource = "different-valid-summary" + if _, err = service.PublishRun(context.Background(), work, changed); !errors.Is(err, contract.ErrConflict) { + t.Fatalf("PublishRun() error = %v, want conflict", err) + } + data, err := os.ReadFile(filepath.Join(root, filepath.FromSlash(first.Path))) + if err != nil { + t.Fatal(err) + } + sum := sha256.Sum256(data) + if got := hex.EncodeToString(sum[:]); got != first.Checksum { + t.Fatalf("conflicting publication replaced the original file: %q != %q", got, first.Checksum) + } +} + +func TestScannerRecoversFailedRunAfterRestart(t *testing.T) { + root := t.TempDir() + repository := newFakeRunRepository(testRunArchive("run-restart")) + repository.load = func(context.Context, contract.ArchiveWork) (contract.RunArchive, error) { + return contract.RunArchive{}, gerror.New("source temporarily unavailable") + } + first := NewService(Config{Root: root, Interval: time.Hour, MaxAttempts: 1}, repository, repository, repository) + if err := first.Start(context.Background()); err != nil { + t.Fatal(err) + } + waitSignal(t, repository.failed, "initial archive failure") + if err := first.Stop(); err != nil { + t.Fatal(err) + } + + repository.mu.Lock() + repository.load = nil + repository.mu.Unlock() + second := NewService(Config{Root: root, Interval: time.Hour, MaxAttempts: 1}, repository, repository, repository) + if err := second.Start(context.Background()); err != nil { + t.Fatal(err) + } + waitSignal(t, repository.succeeded, "restart archive success") + if err := second.Stop(); err != nil { + t.Fatal(err) + } + if _, err := os.Stat(filepath.Join(root, "run-run-restart.json")); err != nil { + t.Fatalf("recovered archive file missing: %v", err) + } +} + +func TestPeriodicScannerRetriesPendingRun(t *testing.T) { + root := t.TempDir() + repository := newFakeRunRepository(testRunArchive("run-periodic")) + var calls atomic.Int32 + repository.load = func(context.Context, contract.ArchiveWork) (contract.RunArchive, error) { + if calls.Add(1) == 1 { + return contract.RunArchive{}, gerror.New("first scan fails") + } + return repository.snapshot, nil + } + service := NewService(Config{Root: root, Interval: 10 * time.Millisecond, MaxAttempts: 1}, repository, repository, repository) + if err := service.Start(context.Background()); err != nil { + t.Fatal(err) + } + waitSignal(t, repository.succeeded, "periodic archive success") + if err := service.Stop(); err != nil { + t.Fatal(err) + } + if calls.Load() < 2 { + t.Fatalf("loader calls = %d, want at least 2", calls.Load()) + } +} + +func TestStartupScannerArchivesReceiptPendingAction(t *testing.T) { + action := testReplayAction("action-startup") + repository := newFakeReplayRepository(action) + service := NewServiceWithReplay( + Config{Root: t.TempDir(), Interval: time.Hour, MaxAttempts: 1}, + nil, + nil, + nil, + ReplayRepositories{Discovery: repository, Leases: repository, Progress: repository}, + ) + if err := service.Start(context.Background()); err != nil { + t.Fatal(err) + } + waitSignal(t, repository.completed, "startup replay archive success") + if err := service.Stop(); err != nil { + t.Fatal(err) + } +} + +func TestCancellationStopsBlockedScannerAndPublication(t *testing.T) { + root := t.TempDir() + repository := newFakeRunRepository(testRunArchive("run-cancel")) + started := make(chan struct{}) + repository.load = func(ctx context.Context, _ contract.ArchiveWork) (contract.RunArchive, error) { + close(started) + <-ctx.Done() + return contract.RunArchive{}, ctx.Err() + } + service := NewService(Config{Root: root, Interval: time.Hour, InitialBackoff: time.Hour}, repository, repository, repository) + if err := service.Start(context.Background()); err != nil { + t.Fatal(err) + } + waitSignal(t, started, "blocked loader start") + stopped := make(chan struct{}) + go func() { + _ = service.Stop() + close(stopped) + }() + waitSignal(t, stopped, "scanner cancellation") + + canceled, cancel := context.WithCancel(context.Background()) + cancel() + _, err := service.PublishRun(canceled, contract.ArchiveWork{RunID: "run-cancel", Version: "v1"}, repository.snapshot) + if !errors.Is(err, context.Canceled) { + t.Fatalf("canceled PublishRun() error = %v", err) + } + if _, err := os.Stat(filepath.Join(root, "run-run-cancel.json")); !errors.Is(err, os.ErrNotExist) { + t.Fatalf("canceled publication wrote final file: %v", err) + } +} + +func TestCancellationAfterRenameStillVerifiesPublicationAndReleasesLease(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + action := testReplayAction("action-cancel-after-rename") + repository := newFakeReplayRepository(action) + service := NewServiceWithReplay( + Config{Root: t.TempDir()}, + nil, + nil, + nil, + ReplayRepositories{Discovery: repository, Leases: repository, Progress: repository}, + ) + service.afterReplace = func(string) error { + cancel() + return nil + } + + if err := service.ArchiveAction(ctx, action); err != nil { + t.Fatalf("ArchiveAction() error = %v", err) + } + repository.mu.Lock() + releaseCtxErr := repository.releaseCtxErr + repository.mu.Unlock() + if releaseCtxErr != nil { + t.Fatalf("lease release inherited canceled context: %v", releaseCtxErr) + } +} + +func TestArchiveRejectsTraversalAndUnsafeFilesystemIDs(t *testing.T) { + root := t.TempDir() + service := New(root, nil, nil, newFakeRunRepository(contract.RunArchive{})) + for _, id := range []string{"../escape", `..\escape`, "/absolute", `C:\escape`, "drive:name", ".", "..", "{hash}", "CON"} { + t.Run(strings.ReplaceAll(id, "/", "_"), func(t *testing.T) { + snapshot := testRunArchive(id) + _, err := service.PublishRun(context.Background(), contract.ArchiveWork{RunID: id, Version: "v1"}, snapshot) + if !errors.Is(err, contract.ErrInvalidArgument) { + t.Fatalf("PublishRun(%q) error = %v, want invalid argument", id, err) + } + }) + } + entries, err := os.ReadDir(root) + if err != nil { + t.Fatal(err) + } + if len(entries) != 0 { + t.Fatalf("unsafe IDs created archive entries: %+v", entries) + } +} + +func testRunArchive(runID string) contract.RunArchive { + created := time.Date(2026, time.August, 31, 23, 0, 0, 0, time.FixedZone("test", 8*60*60)) + completed := time.Date(2026, time.September, 20, 16, 0, 0, 0, time.FixedZone("test", 8*60*60)) + archived := completed.Add(time.Minute) + return contract.RunArchive{ + Version: contract.ArchiveSchemaVersion, + Run: contract.RunMeta{ + RunID: runID, Status: contract.RunCompletedWithFailures, ProducerDone: true, + CurrentRound: 2, MaxRound: 2, EnqueuedCount: 3, SuccessCount: 2, FailedCount: 1, PendingFailureCount: 1, + CreatedAt: created, ProducerCompletedAt: completed, CompletedAt: completed, + ArchiveStatus: contract.ArchiveFailed, ArchiveVersion: 99, ArchivePath: "old.json", + ArchiveChecksum: strings.Repeat("a", 64), ArchivedAt: &archived, + }, + Rounds: []contract.RoundStats{ + {RunID: runID, Round: 2, EnqueuedCount: 1, FailedCount: 1, ErrorCounts: contract.ErrorCounts{"mobile_busy": 1}}, + {RunID: runID, Round: 1, EnqueuedCount: 2, SuccessCount: 2, ErrorCounts: nil}, + }, + } +} + +func testReplayAction(actionID string) contract.ReplayAction { + created := time.Date(2026, time.September, 20, 12, 0, 0, 0, time.FixedZone("test", 8*60*60)) + return contract.ReplayAction{ + ActionID: actionID, SourceRunID: "source-1", TargetRunID: "target-1", RequestedBy: "operator", + IdempotencyKey: "secret-key", Fingerprint: "secret-fingerprint", Status: contract.ActionReceiptPending, + ReplayCycle: 1, Progress: contract.ReplayProgress{DiscoveredCount: 2, EnqueuedCount: 2, CleanedCount: 2}, + LeaseOwner: "engine", LeaseToken: "secret-token", LeaseUntil: created.Add(time.Hour), Revision: 7, + CreatedAt: created, UpdatedAt: created.Add(time.Minute), + } +} + +type fakeRunRepository struct { + mu sync.Mutex + work contract.ArchiveWork + snapshot contract.RunArchive + load func(context.Context, contract.ArchiveWork) (contract.RunArchive, error) + registered bool + receipts []contract.ArchiveReceipt + failures []string + registerErr error + succeeded chan struct{} + failed chan struct{} +} + +func newFakeRunRepository(snapshot contract.RunArchive) *fakeRunRepository { + return &fakeRunRepository{ + work: contract.ArchiveWork{RunID: snapshot.Run.RunID, Version: "repository-version"}, + snapshot: snapshot, + succeeded: make(chan struct{}, 8), + failed: make(chan struct{}, 8), + } +} + +func (r *fakeRunRepository) ListArchiveWork(ctx context.Context, _ contract.PageRequest) (contract.ArchiveWorkPage, error) { + if err := ctx.Err(); err != nil { + return contract.ArchiveWorkPage{}, err + } + r.mu.Lock() + defer r.mu.Unlock() + if r.registered { + return contract.ArchiveWorkPage{Items: []contract.ArchiveWork{}}, nil + } + return contract.ArchiveWorkPage{Items: []contract.ArchiveWork{r.work}}, nil +} + +func (r *fakeRunRepository) LoadArchiveWork(ctx context.Context, work contract.ArchiveWork) (contract.RunArchive, error) { + r.mu.Lock() + load := r.load + snapshot := r.snapshot + r.mu.Unlock() + if load != nil { + return load(ctx, work) + } + return snapshot, nil +} + +func (r *fakeRunRepository) RegisterArchive(_ context.Context, _ contract.ArchiveWork, receipt contract.ArchiveReceipt) error { + r.mu.Lock() + r.receipts = append(r.receipts, receipt) + err := r.registerErr + if err == nil { + r.registered = true + } + r.mu.Unlock() + if err != nil { + return err + } + notify(r.succeeded) + return nil +} + +func (r *fakeRunRepository) RegisterArchiveFailure(_ context.Context, _ contract.ArchiveWork, message string) error { + r.mu.Lock() + r.failures = append(r.failures, message) + r.mu.Unlock() + notify(r.failed) + return nil +} + +type fakeReplayRepository struct { + mu sync.Mutex + action contract.ReplayAction + update *contract.ReplayActionUpdate + released contract.ReplayLease + releaseCtxErr error + completed chan struct{} +} + +func newFakeReplayRepository(action contract.ReplayAction) *fakeReplayRepository { + return &fakeReplayRepository{action: action, completed: make(chan struct{}, 4)} +} + +func (r *fakeReplayRepository) ListReplayWork(ctx context.Context, _ contract.PageRequest) (contract.ReplayActionPage, error) { + if err := ctx.Err(); err != nil { + return contract.ReplayActionPage{}, err + } + r.mu.Lock() + defer r.mu.Unlock() + if r.update != nil { + return contract.ReplayActionPage{Items: []contract.ReplayAction{}}, nil + } + return contract.ReplayActionPage{Items: []contract.ReplayAction{r.action}}, nil +} + +func (r *fakeReplayRepository) AcquireReplayLease(_ context.Context, actionID, owner string, ttl time.Duration) (contract.ReplayLease, error) { + return contract.ReplayLease{ActionID: actionID, Owner: owner, Token: "lease-token", Until: time.Now().Add(ttl)}, nil +} + +func (r *fakeReplayRepository) RenewReplayLease(_ context.Context, lease contract.ReplayLease, ttl time.Duration) (contract.ReplayLease, error) { + lease.Token = "lease-token-2" + lease.Until = time.Now().Add(ttl) + return lease, nil +} + +func (r *fakeReplayRepository) ReleaseReplayLease(ctx context.Context, lease contract.ReplayLease) error { + r.mu.Lock() + r.released = lease + r.releaseCtxErr = ctx.Err() + r.mu.Unlock() + return nil +} + +func (r *fakeReplayRepository) UpdateReplayAction(_ context.Context, update contract.ReplayActionUpdate) (contract.ReplayAction, error) { + r.mu.Lock() + copyOfUpdate := update + r.update = ©OfUpdate + action := r.action + action.Status = update.Status + action.Receipt = update.Receipt + action.CompletedAt = update.CompletedAt + r.mu.Unlock() + notify(r.completed) + return action, nil +} + +func notify(ch chan struct{}) { + select { + case ch <- struct{}{}: + default: + } +} + +func waitSignal(t *testing.T, ch <-chan struct{}, operation string) { + t.Helper() + select { + case <-ch: + case <-time.After(3 * time.Second): + t.Fatalf("timed out waiting for %s", operation) + } +} diff --git a/internal/fullsync/archive/replace_other.go b/internal/fullsync/archive/replace_other.go new file mode 100644 index 0000000..6b0849b --- /dev/null +++ b/internal/fullsync/archive/replace_other.go @@ -0,0 +1,21 @@ +//go:build !windows + +package archive + +import "os" + +func atomicReplace(from, to string) error { + return os.Rename(from, to) +} + +func syncDirectory(path string) error { + dir, err := os.Open(path) + if err != nil { + return err + } + if err := dir.Sync(); err != nil { + _ = dir.Close() + return err + } + return dir.Close() +} diff --git a/internal/fullsync/archive/replace_windows.go b/internal/fullsync/archive/replace_windows.go new file mode 100644 index 0000000..f3b966d --- /dev/null +++ b/internal/fullsync/archive/replace_windows.go @@ -0,0 +1,21 @@ +//go:build windows + +package archive + +import "golang.org/x/sys/windows" + +func atomicReplace(from, to string) error { + fromUTF16, err := windows.UTF16PtrFromString(from) + if err != nil { + return err + } + toUTF16, err := windows.UTF16PtrFromString(to) + if err != nil { + return err + } + return windows.MoveFileEx(fromUTF16, toUTF16, windows.MOVEFILE_REPLACE_EXISTING|windows.MOVEFILE_WRITE_THROUGH) +} + +// Windows has no portable directory fsync operation. MoveFileEx with +// MOVEFILE_WRITE_THROUGH provides the strongest supported replacement flush. +func syncDirectory(string) error { return nil } diff --git a/internal/fullsync/contract/README.md b/internal/fullsync/contract/README.md new file mode 100644 index 0000000..fb30fca --- /dev/null +++ b/internal/fullsync/contract/README.md @@ -0,0 +1,124 @@ +# Full-sync shared contracts + +This package establishes the boundary for later queue accounting, archive, +replay, and HTTP work. It does not change running service behavior. Production +code in this package does not import the queue package; compatibility tests +compare its models and keys with the existing queue implementation. + +## Compatibility and counters + +- `RunMeta` retains all existing JSON fields and adds optional fields. Existing + `zset.RunMeta`, `zset.Task`, and `zset.FailureRecord` remain unchanged; adapters + are a later integration step. The new `RunStatus` is a named string type. +- `Validate` accepts missing extension fields on old records. + `ValidateAccounting` is opt-in for runs with known extended counters and + enforces `enqueuedCount = successCount + failedCount + activeCount + nextRoundCount`. + `inflightCount` is a subset of `activeCount`. +- Source cleanup changes `pendingFailureCount`, never historical `failedCount`. +- Round statistics have no task/card/channel dimensions. `ErrorCounts` + normalizes both JSON input and output to eight canonical categories, merging + unknown names into `other` and rejecting negative/overflowing totals. +- Existing failure payloads retain their free-form classification and message. + The HTTP `Failure` projection normalizes categories and bounds messages to + 2,048 UTF-8 bytes. Historical blocked tasks may have round zero. + +## Status domains + +| Domain | Wire values | +| --- | --- | +| Run | `producing`, `running`, `completed`, `completed_with_failures` | +| Sync type | `normal`, `replay` | +| Archive | `disabled`, `pending`, `processing`, `succeeded`, `failed` | +| Source replay disposition | `none`, `pending`, `processing`, `completed`, `needs_attention` | +| Replay action | `accepted`, `processing`, `cleanup_pending`, `receipt_pending`, `completed`, `needs_attention` | +| Error category | `timeout`, `rate_limited`, `authentication`, `validation`, `upstream`, `transport`, `internal`, `other` | + +The source field is `replayDisposition`, distinct from the replay action's +`status`. Lease tokens, record versions, action revision, idempotency key, and +request fingerprint are internal and excluded from HTTP JSON. + +## Interfaces and persistence obligations + +HTTP consumers depend on `RunReader`, `FailurePager`, `ReplayRequester`, and +`ActionReader`. Separate live/archive readers support fallback only after a live +`ErrNotFound`; storage failures must remain visible. An archived summary cannot +provide per-card details: use `ErrFailureDetailsUnavailable` when live details +are gone. Repositories and HTTP both apply the default page limit 100 and +maximum 500. Cursors remain opaque and are bound to the original query. An empty +scan page with a next cursor does not signal completion. + +Archive discovery, scheduling, snapshot loading, and result registration use +separate narrow interfaces. Registration is conditional on an opaque work +version. Archives contain run and round aggregates only. Archive receipts use +a schema version, storage-owned path, lowercase SHA-256 checksum of stored +bytes, and publication time. Archive schema version starts at 1. + +Replay acceptance atomically reserves `(sourceRunId, idempotencyKey)` with a +SHA-256 fingerprint of the source and exact requestedBy body value. Repeating +the request returns its original action/target; changing the request returns +`ErrConflict`. The action fixes its target ID before target creation/enqueue. + +Replay work discovery, lease management, progress updates, source pagination, +target creation/enqueue, and source cleanup are separate interfaces. Progress +updates are fenced and use a revision check. Target enqueue distinguishes +durably present duplicates from rejected writes; a zero insert count by itself +does not authorize source cleanup. Cleanup checks exact source record versions +and target acceptance evidence. Implementations must persist enough evidence +to recover partial batches; a scan cursor by itself is insufficient. Source +pagination must remain correct while already-enqueued entries are deleted. + +`ErrNotFound`, `ErrConflict`, `ErrInvalidArgument`, +`ErrFailureDetailsUnavailable`, `ErrLeaseLost`, and `ErrBusy` support +`errors.Is` through wrapping. HTTP error envelopes and status mappings remain +the HTTP implementation's responsibility. + +## Redis naming + +All names are prefixed with the configured namespace (`flow:` by default; +`sim-status:` for status sync). Existing run/round/task-state/failure names are +preserved. New keys include: + +| Suffix after namespace | Scope | +| --- | --- | +| `{runId}:round:N:stats` | Run slot | +| `{runId}:archive` | Run slot | +| `runs:all` | Global index | +| `archive:pending` | Global index | +| `{replay}:action:actionId:meta` | Replay slot | +| `{replay}:action:actionId:lease` | Replay slot | +| `{replay}:idempotency:SHA256(sourceRunId,key)` | Replay slot; tuple encoded as a JSON array before hashing | +| `{replay}:pending` | Replay slot | + +Replay action creation, idempotency, and discovery can be atomic within their +shared slot. Source/target runs and global indexes can occupy different slots; +their coordination requires recoverable registrations/receipts rather than a +single cross-slot Lua transaction. New identifier validation rejects hash-tag +or path delimiters without rewriting legacy JSON on decode. + +## HTTP surface + +| Method | Route | Response | +| --- | --- | --- | +| GET | `/internal/v1/full-sync-runs?syncKind=&limit=` | `RunList` merged from active Redis Runs and `run-.json` archives | +| GET | `/internal/v1/full-sync-runs/{runId}` | `RunView` with `source`, `run`, optional `rounds` | +| GET | `/internal/v1/full-sync-runs/{runId}/failures?cursor=&limit=` | `FailurePage` with `items`, `nextCursor` | +| POST | `/internal/v1/full-sync-runs/{runId}/replay` | HTTP 202 `ReplayAccepted` | +| GET | `/internal/v1/full-sync-actions/{actionId}` | `ReplayAction` | + +POST requires `Idempotency-Key` and JSON `{"requestedBy":"..."}`. Authentication, +HTTP handlers, storage, replay execution, archive files, and service wiring are +intentionally left to subsequent implementations. + +## Verification + +```text +gofmt -l internal/fullsync/contract +go test ./internal/fullsync/contract +go test ./internal/queue/zset ./internal/consumer/zset ./internal/producer/simstatus ./internal/producer/monthlyusage ./internal/service/simstatus ./internal/service/monthlyusage +``` + +Tests cover legacy schema and JSON compatibility, existing key compatibility, +cluster key boundaries, bounded categories/messages, overflow, run accounting, +pagination limits, status domains, replay identity, internal field exclusion, +and receipt/progress validation. Regression tests use the repository's isolated +Redis helper and require `redis-server` or `REDIS_SERVER_BINARY`. diff --git a/internal/fullsync/contract/archive.go b/internal/fullsync/contract/archive.go new file mode 100644 index 0000000..7ce0647 --- /dev/null +++ b/internal/fullsync/contract/archive.go @@ -0,0 +1,46 @@ +package contract + +import "time" + +const ArchiveSchemaVersion = 1 + +// ArchiveReceipt identifies a durably published archive object. Path is a +// storage-owned reference, never a client-supplied destination. Checksum is +// lowercase hex SHA-256 of the final stored bytes; version is the schema. +type ArchiveReceipt struct { + Version int `json:"version"` + Path string `json:"path"` + Checksum string `json:"checksum"` + ArchivedAt time.Time `json:"archivedAt"` +} + +func (r ArchiveReceipt) Validate() error { + if r.Version < 1 || r.Path == "" || r.ArchivedAt.IsZero() || len(r.Checksum) != 64 { + return invalid("incomplete archive receipt") + } + for _, c := range r.Checksum { + if !((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f')) { + return invalid("archive checksum must be lowercase SHA-256 hex") + } + } + return nil +} + +// RunArchive is summary-only: never embed per-card task/failure records. +type RunArchive struct { + Version int `json:"version"` + Run RunMeta `json:"run"` + Rounds []RoundStats `json:"rounds"` +} + +// ArchiveWork.Version is an opaque concurrency token for registration; it is +// distinct from the archive schema version. Work discovery must survive restarts. +type ArchiveWork struct { + RunID string `json:"runId"` + Version string `json:"-"` +} + +type ArchiveWorkPage struct { + Items []ArchiveWork `json:"items"` + NextCursor Cursor `json:"nextCursor"` +} diff --git a/internal/fullsync/contract/compatibility_test.go b/internal/fullsync/contract/compatibility_test.go new file mode 100644 index 0000000..c4ac758 --- /dev/null +++ b/internal/fullsync/contract/compatibility_test.go @@ -0,0 +1,107 @@ +package contract_test + +import ( + "encoding/json" + "reflect" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +// Check every old field, including zero/omitempty behavior, so future adapters +// can adopt the contract without silently changing the persisted wire format. +func TestLegacyFieldSchemas(t *testing.T) { + for _, pair := range []struct{ old, next any }{ + {zset.RunMeta{}, contract.RunMeta{}}, + {zset.FailureRecord{}, contract.FailureRecord{}}, + {zset.Task{}, contract.Task{}}, + } { + old, next := reflect.TypeOf(pair.old), reflect.TypeOf(pair.next) + for i := 0; i < old.NumField(); i++ { + field := old.Field(i) + replacement, ok := next.FieldByName(field.Name) + if !ok || replacement.Tag.Get("json") != field.Tag.Get("json") || replacement.Type.Kind() != field.Type.Kind() { + t.Errorf("%s.%s lost its legacy schema: old=%+v new=%+v", old.Name(), field.Name, field, replacement) + } + } + } +} + +func TestLegacyRunJSONRoundTrip(t *testing.T) { + now := time.Date(2026, 9, 20, 1, 2, 3, 456, time.UTC) + legacy := zset.RunMeta{ + RunID: "legacy", Operation: "current_month_usage_full_sync", QueryMonth: "202609", + Status: "completed_with_failures", ProducerDone: true, CurrentRound: 5, + SuccessCount: 9, FailedCount: 1, CreatedAt: now, ProducerCompletedAt: now, CompletedAt: now, + } + data := mustJSON(t, legacy) + var decoded contract.RunMeta + if err := json.Unmarshal(data, &decoded); err != nil { + t.Fatal(err) + } + if err := decoded.Validate(); err != nil { + t.Fatalf("legacy run rejected: %v", err) + } + if got := string(mustJSON(t, decoded)); got != string(data) { + t.Fatalf("legacy wire format changed:\n%s\n%s", data, got) + } + // New fields do not prevent old readers from consuming the same metadata. + decoded.SyncType = contract.SyncReplay + decoded.SourceRunID = "source" + decoded.ReplayCycle = 1 + decoded.EnqueuedCount = 10 + decoded.PendingFailureCount = 1 + var oldReader zset.RunMeta + if err := json.Unmarshal(mustJSON(t, decoded), &oldReader); err != nil { + t.Fatal(err) + } + oldReader.SyncType = "" + oldReader.SourceRunID = "" + oldReader.ReplayCycle = 0 + oldReader.EnqueuedCount = 0 + oldReader.PendingFailureCount = 0 + if !reflect.DeepEqual(oldReader, legacy) { + t.Fatalf("old reader lost fields: %+v", oldReader) + } +} + +func TestLegacyFailureAndTaskJSON(t *testing.T) { + legacy := zset.FailureRecord{ + Version: 1, TaskID: "source:op:42", OriginalRunID: "source", LatestRunID: "latest", + QueryMonth: "202609", Operation: "op", CardID: 42, ICCID: "iccid", MSISDN: "msisdn", IMSI: "imsi", + ChanKey: "chan", TempKey: "template", Carrier: "mobile", FailureType: "mobile_busy_exhausted", + Round: 5, Attempts: 7, Code: 13000, ErrCode: "BUSY", Message: "try again", + ReviewStatus: "pending", ReviewRemark: "remark", ReviewedBy: "operator", ReviewedAt: "2026-09-20", + RetryCycle: 2, ReplayRunID: "replayed", FirstFailedAt: time.Unix(1, 0).UTC(), LastFailedAt: time.Unix(2, 0).UTC(), + } + var record contract.FailureRecord + if err := json.Unmarshal(mustJSON(t, legacy), &record); err != nil { + t.Fatal(err) + } + if string(mustJSON(t, record)) != string(mustJSON(t, legacy)) { + t.Fatal("failure record wire format changed") + } + if summary := record.Summary(); summary.ErrorCategory != contract.ErrorRateLimited || summary.TaskID != legacy.TaskID || summary.FailedAt != legacy.LastFailedAt { + t.Fatalf("incorrect failure projection: %+v", summary) + } + task := zset.Task{TaskID: "r:op:42", RunID: "r", Operation: "op", QueryMonth: "202609", ChanKey: "c", TempKey: "t", Operator: 2, CardID: 42, ICCID: "i", MSISDN: "m", IMSI: "s", NextExecuteAt: 1234} + var newTask contract.Task + if err := json.Unmarshal(mustJSON(t, task), &newTask); err != nil { + t.Fatal(err) + } + newTask.NextExecuteAt = 9999 + if string(mustJSON(t, newTask)) != string(mustJSON(t, task)) { + t.Fatal("task wire format or scheduling exclusion changed") + } +} + +func mustJSON(t *testing.T, value any) []byte { + t.Helper() + data, err := json.Marshal(value) + if err != nil { + t.Fatal(err) + } + return data +} diff --git a/internal/fullsync/contract/doc.go b/internal/fullsync/contract/doc.go new file mode 100644 index 0000000..546fd31 --- /dev/null +++ b/internal/fullsync/contract/doc.go @@ -0,0 +1,14 @@ +// Package contract defines the shared full-sync domain and service boundaries. +// It contains no Redis, filesystem, replay execution, or HTTP handler code. +// +// Legacy queue models remain in internal/queue/zset until adapters adopt this +// package. Their existing JSON fields are retained here; new run fields are +// optional so historical records still decode. Missing extended counters mean +// unknown, not a reconstructed total. FailedCount is the historical terminal +// count for a run; cleaning source failures changes PendingFailureCount only. +// +// Implementations must preserve these boundaries: run-local Redis keys share +// one hash tag, global indexes do not; archive summaries never contain card +// details; cursors are opaque and scoped to the original query; replay writes +// are idempotent and fenced by a lease before source cleanup is allowed. +package contract diff --git a/internal/fullsync/contract/errors.go b/internal/fullsync/contract/errors.go new file mode 100644 index 0000000..3f1c23a --- /dev/null +++ b/internal/fullsync/contract/errors.go @@ -0,0 +1,112 @@ +package contract + +import ( + "encoding/json" + "math" + "strings" + "unicode/utf8" +) + +type ErrorCategory string + +const ( + ErrorTimeout ErrorCategory = "timeout" + ErrorRateLimited ErrorCategory = "rate_limited" + ErrorAuthentication ErrorCategory = "authentication" + ErrorValidation ErrorCategory = "validation" + ErrorUpstream ErrorCategory = "upstream" + ErrorTransport ErrorCategory = "transport" + ErrorInternal ErrorCategory = "internal" + ErrorOther ErrorCategory = "other" + MaxErrorCategories = 8 + MaxErrorMessageBytes = 2048 +) + +// NormalizeErrorCategory maps the existing consumer classifications into the +// approved, bounded vocabulary. Never use an upstream message/code as a key. +func NormalizeErrorCategory(value string) ErrorCategory { + switch strings.ToLower(strings.TrimSpace(value)) { + case "timeout": + return ErrorTimeout + case "rate_limited", "mobile_busy", "mobile_busy_exhausted", "channel_limited", "channel_blocked": + return ErrorRateLimited + case "authentication": + return ErrorAuthentication + case "validation": + return ErrorValidation + case "upstream", "business", "carrier_retry": + return ErrorUpstream + case "transport", "network": + return ErrorTransport + case "internal", "canceled": + return ErrorInternal + default: + return ErrorOther + } +} + +func (c ErrorCategory) MarshalJSON() ([]byte, error) { + return json.Marshal(string(NormalizeErrorCategory(string(c)))) +} + +func (c *ErrorCategory) UnmarshalJSON(data []byte) error { + var value string + if err := json.Unmarshal(data, &value); err != nil { + return invalid("error category must be a string: %v", err) + } + *c = NormalizeErrorCategory(value) + return nil +} + +// ErrorCounts is normalized on both JSON input and output, including map keys +// (encoding/json does not invoke ErrorCategory.MarshalJSON for map keys). +// Counts are per category, not card IDs or individual error messages. +type ErrorCounts map[ErrorCategory]int64 + +func (counts ErrorCounts) Normalized() (ErrorCounts, error) { + normalized := make(ErrorCounts) + for category, count := range counts { + key := NormalizeErrorCategory(string(category)) + if count < 0 || count > math.MaxInt64-normalized[key] { + return nil, invalid("error counts must be nonnegative and must not overflow") + } + normalized[key] += count + } + return normalized, nil +} + +func (counts ErrorCounts) MarshalJSON() ([]byte, error) { + normalized, err := counts.Normalized() + if err != nil { + return nil, err + } + type wireCounts map[ErrorCategory]int64 + return json.Marshal(wireCounts(normalized)) +} + +func (counts *ErrorCounts) UnmarshalJSON(data []byte) error { + var decoded map[ErrorCategory]int64 + if err := json.Unmarshal(data, &decoded); err != nil { + return invalid("invalid error counts: %v", err) + } + normalized, err := ErrorCounts(decoded).Normalized() + if err != nil { + return err + } + *counts = normalized + return nil +} + +// BoundedErrorMessage makes new summary/action error messages valid UTF-8 and +// caps their byte length. It does not modify legacy FailureRecord.Message. +func BoundedErrorMessage(message string) string { + message = strings.ToValidUTF8(message, "\uFFFD") + if len(message) <= MaxErrorMessageBytes { + return message + } + end := MaxErrorMessageBytes + for !utf8.RuneStart(message[end]) { + end-- + } + return message[:end] +} diff --git a/internal/fullsync/contract/errors_test.go b/internal/fullsync/contract/errors_test.go new file mode 100644 index 0000000..c79bafc --- /dev/null +++ b/internal/fullsync/contract/errors_test.go @@ -0,0 +1,91 @@ +package contract_test + +import ( + "encoding/json" + "errors" + "fmt" + "math" + "strings" + "testing" + "unicode/utf8" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" +) + +func TestBoundedErrorCategories(t *testing.T) { + for raw, want := range map[string]contract.ErrorCategory{ + "timeout": contract.ErrorTimeout, " RATE_LIMITED ": contract.ErrorRateLimited, + "authentication": contract.ErrorAuthentication, "validation": contract.ErrorValidation, + "upstream": contract.ErrorUpstream, "transport": contract.ErrorTransport, + "internal": contract.ErrorInternal, "other": contract.ErrorOther, + "network": contract.ErrorTransport, "mobile_busy": contract.ErrorRateLimited, + "mobile_busy_exhausted": contract.ErrorRateLimited, "channel_blocked": contract.ErrorRateLimited, + "channel_limited": contract.ErrorRateLimited, "business": contract.ErrorUpstream, + "carrier_retry": contract.ErrorUpstream, "canceled": contract.ErrorInternal, + "": contract.ErrorOther, "card-898600-unique-error": contract.ErrorOther, + } { + if got := contract.NormalizeErrorCategory(raw); got != want { + t.Errorf("normalize %q = %q, want %q", raw, got, want) + } + var decoded contract.ErrorCategory + if err := json.Unmarshal(mustJSON(t, raw), &decoded); err != nil || decoded != want { + t.Errorf("decode %q = %q, err %v", raw, decoded, err) + } + if got := string(mustJSON(t, contract.ErrorCategory(raw))); got != string(mustJSON(t, string(want))) { + t.Errorf("encode %q = %s", raw, got) + } + } +} + +func TestErrorCountsCannotGrowWithCardinality(t *testing.T) { + counts := contract.ErrorCounts{contract.ErrorTimeout: 2, contract.ErrorOther: 3, "network": 4} + for i := 0; i < 1000; i++ { + counts[contract.ErrorCategory(fmt.Sprintf("card-error-%d", i))] = 1 + } + var decoded contract.ErrorCounts + if err := json.Unmarshal(mustJSON(t, counts), &decoded); err != nil { + t.Fatal(err) + } + if len(decoded) != 3 || len(decoded) > contract.MaxErrorCategories || decoded[contract.ErrorOther] != 1003 || decoded[contract.ErrorTransport] != 4 { + t.Fatalf("unbounded or lost counts: %+v", decoded) + } + if err := json.Unmarshal([]byte(`{"unknown":2,"different":3,"other":4,"network":5}`), &decoded); err != nil { + t.Fatal(err) + } + if len(decoded) != 2 || decoded[contract.ErrorOther] != 9 || decoded[contract.ErrorTransport] != 5 { + t.Fatalf("input normalization failed: %+v", decoded) + } +} + +func TestErrorCountsRejectInvalidNumbers(t *testing.T) { + for _, raw := range []string{`{"timeout":-1}`, `{"one":9223372036854775807,"two":1}`, `{"other":1.5}`, `{"other":"1"}`} { + var counts contract.ErrorCounts + if err := json.Unmarshal([]byte(raw), &counts); !errors.Is(err, contract.ErrInvalidArgument) { + t.Errorf("invalid input %s returned %v", raw, err) + } + } + for _, counts := range []contract.ErrorCounts{{"timeout": -1}, {"one": math.MaxInt64, "two": 1}} { + if _, err := json.Marshal(counts); !errors.Is(err, contract.ErrInvalidArgument) { + t.Errorf("invalid output counts returned %v", err) + } + } +} + +func TestBoundedFailureMessage(t *testing.T) { + for _, message := range []string{"short", strings.Repeat("x", 3000), strings.Repeat("卡", 1000), strings.Repeat("x", 2047) + "卡", "invalid\xfftext"} { + got := contract.BoundedErrorMessage(message) + if len(got) > contract.MaxErrorMessageBytes || !utf8.ValidString(got) { + t.Fatalf("invalid bounded output: bytes=%d", len(got)) + } + if utf8.ValidString(message) && len(message) <= contract.MaxErrorMessageBytes && got != message { + t.Fatal("short message changed") + } + } + record := contract.FailureRecord{TaskID: "task", Message: strings.Repeat("卡", 1000)} + if err := record.Summary().Validate(); err != nil { + t.Fatal(err) + } + if len(record.Message) != 3000 { + t.Fatal("summary mutated legacy failure message") + } +} diff --git a/internal/fullsync/contract/failure.go b/internal/fullsync/contract/failure.go new file mode 100644 index 0000000..1626879 --- /dev/null +++ b/internal/fullsync/contract/failure.go @@ -0,0 +1,129 @@ +package contract + +import ( + "time" + "unicode/utf8" +) + +// Cursor is an opaque repository-issued continuation token. Clients must not +// parse, increment, or reuse it for another run/filter. Empty starts a scan; +// an empty nextCursor ends it. An empty page with a cursor is not end-of-scan. +// Repositories validate token contents and query binding with ErrInvalidArgument. +type Cursor string + +const ( + DefaultPageLimit = 100 + MaxPageLimit = 500 +) + +type PageRequest struct { + Cursor Cursor `json:"cursor,omitempty"` + Limit int `json:"limit,omitempty"` +} + +// Normalized must be called at both HTTP and repository boundaries. Limits +// outside the bounds are rejected, not silently truncated. +func (p PageRequest) Normalized() (PageRequest, error) { + if p.Limit == 0 { + p.Limit = DefaultPageLimit + } + if p.Limit < 1 || p.Limit > MaxPageLimit { + return PageRequest{}, invalid("limit must be between 1 and %d", MaxPageLimit) + } + return p, nil +} + +// Failure is the bounded public projection, not the replay payload. Legacy +// free-form failureType/message fields remain available in FailureRecord. +type Failure struct { + TaskID string `json:"taskId"` + CardID int64 `json:"cardId"` + ErrorCategory ErrorCategory `json:"errorCategory"` + ErrorMessage string `json:"errorMessage"` + FailedAt time.Time `json:"failedAt"` + Round int `json:"round"` +} + +func (f Failure) Validate() error { + if f.TaskID == "" || f.Round < 0 || len(f.ErrorMessage) > MaxErrorMessageBytes || !utf8.ValidString(f.ErrorMessage) { + return invalid("invalid failure summary") + } + return nil +} + +type FailurePage struct { + Items []Failure `json:"items"` + NextCursor Cursor `json:"nextCursor"` +} + +// FailureRecord preserves the existing persisted payload without changing +// review semantics or discarding fields needed to reconstruct replay tasks. +type FailureRecord struct { + Version int `json:"version"` + TaskID string `json:"taskId"` + OriginalRunID string `json:"originalRunId"` + LatestRunID string `json:"latestRunId"` + QueryMonth string `json:"queryMonth"` + Operation string `json:"operation"` + CardID int64 `json:"cardId"` + ICCID string `json:"iccid"` + MSISDN string `json:"msisdn"` + IMSI string `json:"imsi"` + ChanKey string `json:"chanKey"` + TempKey string `json:"tempKey"` + // PackageCode is additive; older failure records decode it as empty. + PackageCode string `json:"packageCode,omitempty"` + Carrier string `json:"carrier"` + FailureType string `json:"failureType"` + Round int `json:"round"` + Attempts int `json:"attempts"` + Code int `json:"code"` + ErrCode string `json:"errCode"` + Message string `json:"message"` + ReviewStatus string `json:"reviewStatus"` + ReviewRemark string `json:"reviewRemark"` + ReviewedBy string `json:"reviewedBy"` + ReviewedAt string `json:"reviewedAt"` + RetryCycle int `json:"retryCycle"` + ReplayRunID string `json:"replayRunId"` + FirstFailedAt time.Time `json:"firstFailedAt"` + LastFailedAt time.Time `json:"lastFailedAt"` +} + +func (f FailureRecord) Summary() Failure { + return Failure{ + TaskID: f.TaskID, CardID: f.CardID, ErrorCategory: NormalizeErrorCategory(f.FailureType), + ErrorMessage: BoundedErrorMessage(f.Message), FailedAt: f.LastFailedAt, Round: f.Round, + } +} + +// FailureEntry carries an opaque version for conditional source cleanup. It +// must identify the precise record read, not just its reusable task ID. +type FailureEntry struct { + Record FailureRecord `json:"record"` + Version string `json:"-"` +} + +type FailureRecordPage struct { + Items []FailureEntry `json:"items"` + NextCursor Cursor `json:"nextCursor"` +} + +// Task keeps the existing queue member serialization. The scheduling score +// must never be part of the JSON member or producer retries lose idempotency. +type Task struct { + TaskID string `json:"taskId"` + RunID string `json:"runId"` + Operation string `json:"operation"` + QueryMonth string `json:"queryMonth"` + ChanKey string `json:"chanKey"` + TempKey string `json:"tempKey"` + Operator int `json:"operator,omitempty"` + CardID int64 `json:"cardId"` + ICCID string `json:"iccid"` + MSISDN string `json:"msisdn"` + IMSI string `json:"imsi"` + // PackageCode is additive; older members decode it as empty. + PackageCode string `json:"packageCode,omitempty"` + NextExecuteAt int64 `json:"-"` +} diff --git a/internal/fullsync/contract/http.go b/internal/fullsync/contract/http.go new file mode 100644 index 0000000..7fd337c --- /dev/null +++ b/internal/fullsync/contract/http.go @@ -0,0 +1,28 @@ +package contract + +import "net/url" + +const ( + ListRunsRoute = "/internal/v1/full-sync-runs" + GetRunRoute = "/internal/v1/full-sync-runs/{runId}" + GetFailuresRoute = "/internal/v1/full-sync-runs/{runId}/failures" + PostReplayRoute = "/internal/v1/full-sync-runs/{runId}/replay" + GetActionRoute = "/internal/v1/full-sync-actions/{actionId}" + IdempotencyHeader = "Idempotency-Key" +) + +// ReplayAccepted is the 202 response from a successful ReplayRequester call. +// A repeated request returns the original action and its current status. +type ReplayAccepted struct { + ActionID string `json:"actionId"` + TargetRunID string `json:"targetRunId"` + Status ActionStatus `json:"status"` + StatusURL string `json:"statusUrl"` +} + +func ActionStatusURL(actionID string) (string, error) { + if err := ValidateID(actionID); err != nil { + return "", err + } + return "/internal/v1/full-sync-actions/" + url.PathEscape(actionID), nil +} diff --git a/internal/fullsync/contract/interfaces.go b/internal/fullsync/contract/interfaces.go new file mode 100644 index 0000000..a637607 --- /dev/null +++ b/internal/fullsync/contract/interfaces.go @@ -0,0 +1,145 @@ +package contract + +import ( + "context" + "time" +) + +// RunReader is the HTTP lookup boundary. Fall back to archive only when the +// live reader returns ErrNotFound, not on transient live storage failures. +type RunReader interface { + GetRun(ctx context.Context, runID string) (RunView, error) +} + +type RunListFilter struct { + SyncKind SyncKind + Limit int +} + +type RunList struct { + Items []RunView `json:"items"` + Total int `json:"total"` +} + +type RunLister interface { + ListRuns(ctx context.Context, filter RunListFilter) (RunList, error) +} + +type LiveRunReader interface { + GetRunMeta(ctx context.Context, runID string) (RunMeta, error) +} + +type ArchivedRunReader interface { + GetArchivedRun(ctx context.Context, runID string) (RunArchive, error) +} + +type RoundStatsReader interface { + GetRoundStats(ctx context.Context, runID string) ([]RoundStats, error) +} + +// FailurePager returns live details only. An archived summary without live +// details returns ErrFailureDetailsUnavailable, never a misleading empty page. +// Items must serialize as [] on empty pages; NextCursor alone signals EOF. +type FailurePager interface { + ListFailures(ctx context.Context, runID string, page PageRequest) (FailurePage, error) +} + +type ReplayRequester interface { + RequestReplay(ctx context.Context, request ReplayRequest) (ReplayAccepted, error) +} + +type ActionReader interface { + GetReplayAction(ctx context.Context, actionID string) (ReplayAction, error) +} + +// ArchiveWorkDiscovery returns pending/retryable work from a durable index. +// Terminal run registration must not depend on an in-memory notification. +type ArchiveWorkDiscovery interface { + ListArchiveWork(ctx context.Context, page PageRequest) (ArchiveWorkPage, error) +} + +// RegisterArchiveWork idempotently schedules a terminal run/version. Since the +// index can occupy another Redis slot, callers must reconcile registration +// after a crash between completing the run and updating the index. +type ArchiveWorkRegistrar interface { + RegisterArchiveWork(ctx context.Context, work ArchiveWork) error +} + +type ArchiveWorkLoader interface { + LoadArchiveWork(ctx context.Context, work ArchiveWork) (RunArchive, error) +} + +// ArchiveRegistrar conditionally records a durable publication against the +// work version. Repeating the same receipt is a no-op; a different version or +// conflicting receipt returns ErrConflict. Failure retains discoverable work. +type ArchiveRegistrar interface { + RegisterArchive(ctx context.Context, work ArchiveWork, receipt ArchiveReceipt) error + RegisterArchiveFailure(ctx context.Context, work ArchiveWork, message string) error +} + +// ReplayActionCreator atomically reserves (sourceRunId, idempotencyKey) with +// its fingerprint and action. Same key/request returns the original action +// regardless of candidate IDs; different request returns ErrConflict. The +// candidate starts accepted and fixes the target ID before any target writes. +type ReplayActionCreator interface { + GetOrCreateReplayAction(ctx context.Context, request ReplayRequest, candidate ReplayAction) (ReplayActionCreation, error) +} + +// Discovery includes accepted, processing, cleanup_pending and receipt_pending +// actions after crashes. Completed/needs_attention actions require no automatic +// work. Lease acquisition, not listing, chooses an exclusive owner. +type ReplayWorkDiscovery interface { + ListReplayWork(ctx context.Context, page PageRequest) (ReplayActionPage, error) +} + +type ReplayLeaseStore interface { + AcquireReplayLease(ctx context.Context, actionID, owner string, ttl time.Duration) (ReplayLease, error) + RenewReplayLease(ctx context.Context, lease ReplayLease, ttl time.Duration) (ReplayLease, error) + ReleaseReplayLease(ctx context.Context, lease ReplayLease) error +} + +// All updates are fenced and compare ExpectedRevision. Expired/replaced leases +// return ErrLeaseLost; revision conflicts return ErrConflict. Implementations +// reject progress regression and preserve immutable action identity. +type ReplayProgressStore interface { + UpdateReplayAction(ctx context.Context, update ReplayActionUpdate) (ReplayAction, error) +} + +// The implementation must preserve scan continuity while source entries are +// cleaned. This can use a durable selection/snapshot; deleting records must not +// make a later cursor skip an unprocessed failure. Tokens are bound to actionID. +type ReplayFailurePager interface { + ListReplayFailures(ctx context.Context, actionID string, page PageRequest) (FailureRecordPage, error) +} + +// TargetRunCreator is idempotent: retrying the same target preserves counters; +// incompatible source/operation/month/cycle returns ErrConflict. Replay targets +// are distinct runs and must not mutate the source's historical FailedCount. +type TargetRunCreator interface { + EnsureTargetRun(ctx context.Context, meta RunMeta) (RunMeta, error) +} + +// TargetEnqueueResult distinguishes a durable duplicate from a rejected write. +// A zero NewlyEnqueued count alone is never sufficient evidence for cleanup. +// Every input TaskID must be durably present on nil error; otherwise return an +// error and let the caller safely retry the identical batch. Both counts are +// unique task counts, and their sum equals the number of distinct input IDs. +type TargetEnqueueResult struct { + NewlyEnqueued int64 + AlreadyPresent int64 +} + +type TargetRunEnqueuer interface { + // Tasks must match the target run and channel and use stable logical IDs. + EnqueueTargetBatch(ctx context.Context, lease ReplayLease, runID, chanKey string, tasks []Task, nextExecuteAt time.Time) (TargetEnqueueResult, error) + MarkTargetProducerDone(ctx context.Context, lease ReplayLease, runID string) error +} + +// Cleanup is idempotent and conditional on the lease, exact source versions, +// and durable evidence of target acceptance. It changes pending failures only, +// never historical failedCount. Conflicts must remain visible for attention. +// Source keys and action/target keys may occupy different Redis Cluster slots; +// implementations need recoverable receipts, not cross-slot Lua assumptions. +type ReplayFailureCleaner interface { + CleanupReplayFailures(ctx context.Context, cleanup ReplayCleanup) (ReplayCleanupResult, error) +} diff --git a/internal/fullsync/contract/keys.go b/internal/fullsync/contract/keys.go new file mode 100644 index 0000000..aa9bb78 --- /dev/null +++ b/internal/fullsync/contract/keys.go @@ -0,0 +1,116 @@ +package contract + +import ( + "crypto/sha256" + "encoding/hex" + "encoding/json" + "fmt" + "strings" +) + +const ( + DefaultFlowPrefix = "flow:" + DefaultSimStatusPrefix = "sim-status:" +) + +// Keys is a naming contract only. It does not create indexes or storage. +// Existing run names are byte-for-byte compatible with zset.Store. Global +// run/archive indexes are outside run hash slots and cannot join run-local Lua. +type Keys struct{ prefix string } + +func NewKeys(prefix string) (Keys, error) { + if strings.TrimSpace(prefix) == "" { + prefix = DefaultFlowPrefix + } + if strings.ContainsAny(prefix, "{}") { + return Keys{}, invalid("prefix must not contain Redis hash tags") + } + return Keys{prefix: prefix}, nil +} + +// The zero value uses the legacy flow prefix, just like NewKeys(""). +func (k Keys) Prefix() string { + if k.prefix == "" { + return DefaultFlowPrefix + } + return k.prefix +} + +func (k Keys) runKey(runID, suffix string) (string, error) { + if err := ValidateID(runID); err != nil { + return "", err + } + return k.Prefix() + "{" + runID + "}:" + suffix, nil +} + +func (k Keys) RunMetaKey(runID string) (string, error) { return k.runKey(runID, "meta") } +func (k Keys) RunChannelsKey(runID string) (string, error) { return k.runKey(runID, "channels") } +func (k Keys) TaskStateKey(runID string) (string, error) { return k.runKey(runID, "task-state") } +func (k Keys) FailedKey(runID string) (string, error) { return k.runKey(runID, "failed") } +func (k Keys) ArchiveStateKey(runID string) (string, error) { return k.runKey(runID, "archive") } + +func (k Keys) roundKey(runID string, round int, suffix string) (string, error) { + if round < 1 { + return "", invalid("round must be positive") + } + return k.runKey(runID, fmt.Sprintf("round:%d:%s", round, suffix)) +} + +func (k Keys) RoundStatsKey(runID string, round int) (string, error) { + return k.roundKey(runID, round, "stats") +} + +func (k Keys) roundChannelKey(runID string, round int, kind, chanKey string) (string, error) { + if strings.TrimSpace(chanKey) == "" { + return "", invalid("chanKey is empty") + } + return k.roundKey(runID, round, kind+":"+chanKey) +} + +func (k Keys) RoundActiveKey(runID string, round int, chanKey string) (string, error) { + return k.roundChannelKey(runID, round, "active", chanKey) +} + +func (k Keys) RoundWaitingKey(runID string, round int, chanKey string) (string, error) { + return k.roundChannelKey(runID, round, "waiting", chanKey) +} + +func (k Keys) RoundLeaseKey(runID string, round int, chanKey string) (string, error) { + return k.roundChannelKey(runID, round, "lease", chanKey) +} + +func (k Keys) FailedRunsKey() string { return k.Prefix() + "failed:runs" } +func (k Keys) ActiveRunsKey() string { return k.Prefix() + "runs:active" } +func (k Keys) AllRunsKey() string { return k.Prefix() + "runs:all" } +func (k Keys) PendingArchivesKey() string { return k.Prefix() + "archive:pending" } + +// Replay action state, leases, idempotency reservations and pending index share +// a dedicated slot, allowing atomic acceptance/discovery. Source/target run +// writes remain in their own slots and require a recoverable replay protocol. +func (k Keys) replayKey(actionID, suffix string) (string, error) { + if err := ValidateID(actionID); err != nil { + return "", err + } + return k.Prefix() + "{replay}:action:" + actionID + ":" + suffix, nil +} + +func (k Keys) ReplayActionKey(actionID string) (string, error) { return k.replayKey(actionID, "meta") } +func (k Keys) ReplayLeaseKey(actionID string) (string, error) { return k.replayKey(actionID, "lease") } +func (k Keys) PendingReplayActionsKey() string { return k.Prefix() + "{replay}:pending" } + +// Hash the complete tuple so neither delimiter collisions nor arbitrary +// Idempotency-Key characters can change the Redis key or hash slot. +func (k Keys) ReplayIdempotencyKey(sourceRunID, idempotencyKey string) (string, error) { + if err := ValidateID(sourceRunID); err != nil { + return "", err + } + if err := ValidateIdempotencyKey(idempotencyKey); err != nil { + return "", err + } + data, err := json.Marshal([2]string{sourceRunID, idempotencyKey}) + if err != nil { + return "", invalid("cannot encode idempotency scope: %v", err) + } + digest := sha256.Sum256(data) + return k.Prefix() + "{replay}:idempotency:" + hex.EncodeToString(digest[:]), nil +} diff --git a/internal/fullsync/contract/keys_test.go b/internal/fullsync/contract/keys_test.go new file mode 100644 index 0000000..0ebca9f --- /dev/null +++ b/internal/fullsync/contract/keys_test.go @@ -0,0 +1,116 @@ +package contract_test + +import ( + "context" + "errors" + "strings" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + + "github.com/gogf/gf/v2/container/gvar" +) + +type unusedRedis struct{} + +func (unusedRedis) Do(context.Context, string, ...any) (*gvar.Var, error) { + panic("naming contracts must not access Redis") +} + +func (unusedRedis) Eval(context.Context, string, int64, []string, []any) (*gvar.Var, error) { + panic("naming contracts must not access Redis") +} + +func TestKeysMatchExistingQueue(t *testing.T) { + for _, prefix := range []string{"", "flow:", "sim-status:", "custom-prefix"} { + keys, err := contract.NewKeys(prefix) + if err != nil { + t.Fatal(err) + } + store, err := zset.New(unusedRedis{}, prefix) + if err != nil { + t.Fatal(err) + } + for _, pair := range []struct { + old, next func(string) (string, error) + }{ + {store.RunMetaKey, keys.RunMetaKey}, {store.RunChannelsKey, keys.RunChannelsKey}, + {store.TaskStateKey, keys.TaskStateKey}, {store.FailedKey, keys.FailedKey}, + } { + old, oldErr := pair.old("run:1") + next, nextErr := pair.next("run:1") + if oldErr != nil || nextErr != nil || old != next || !strings.Contains(next, "{run:1}") { + t.Errorf("run naming changed: old=%q new=%q errors=%v/%v", old, next, oldErr, nextErr) + } + } + for _, pair := range []struct { + old, next func(string, int, string) (string, error) + }{ + {store.RoundActiveKey, keys.RoundActiveKey}, {store.RoundWaitingKey, keys.RoundWaitingKey}, + {store.RoundLeaseKey, keys.RoundLeaseKey}, + } { + old, oldErr := pair.old("run:1", 3, "channel") + next, nextErr := pair.next("run:1", 3, "channel") + if oldErr != nil || nextErr != nil || old != next { + t.Errorf("round naming changed: old=%q new=%q errors=%v/%v", old, next, oldErr, nextErr) + } + } + if keys.ActiveRunsKey() != store.ActiveRunsKey() || keys.FailedRunsKey() != store.FailedRunsKey() { + t.Fatal("existing global index names changed") + } + } +} + +func TestNewKeysAndClusterBoundaries(t *testing.T) { + keys, _ := contract.NewKeys("flow:") + stats, _ := keys.RoundStatsKey("run", 2) + archive, _ := keys.ArchiveStateKey("run") + if stats != "flow:{run}:round:2:stats" || archive != "flow:{run}:archive" { + t.Fatalf("new run keys changed: %s, %s", stats, archive) + } + action, _ := keys.ReplayActionKey("a") + lease, _ := keys.ReplayLeaseKey("a") + idempotency, _ := keys.ReplayIdempotencyKey("source", "a{different-slot}:key") + for _, key := range []string{action, lease, idempotency, keys.PendingReplayActionsKey()} { + if !strings.HasPrefix(key, "flow:{replay}:") || strings.Count(key, "{") != 1 { + t.Fatalf("replay key escaped action slot: %s", key) + } + } + if action != "flow:{replay}:action:a:meta" || lease != "flow:{replay}:action:a:lease" { + t.Fatal("action naming changed") + } + for _, key := range []string{keys.ActiveRunsKey(), keys.FailedRunsKey(), keys.AllRunsKey(), keys.PendingArchivesKey()} { + if strings.ContainsAny(key, "{}") { + t.Fatalf("global index unexpectedly shares run slot: %s", key) + } + } + first, _ := keys.ReplayIdempotencyKey("a:b", "c") + second, _ := keys.ReplayIdempotencyKey("a", "b:c") + repeat, _ := keys.ReplayIdempotencyKey("a:b", "c") + otherSource, _ := keys.ReplayIdempotencyKey("other", "c") + if first == second || first == otherSource || first != repeat { + t.Fatal("idempotency scope is ambiguous or unstable") + } +} + +func TestInvalidKeyParts(t *testing.T) { + keys := contract.Keys{} + for _, id := range []string{"", " ", "trim ", "{run}", "run}", "run/path", "run\\path", "run\n", "invalid\xff"} { + if _, err := keys.RunMetaKey(id); !errors.Is(err, contract.ErrInvalidArgument) { + t.Errorf("invalid ID %q returned %v", id, err) + } + } + if _, err := contract.NewKeys("prefix:{wrong}:"); !errors.Is(err, contract.ErrInvalidArgument) { + t.Fatalf("invalid prefix returned %v", err) + } + if _, err := keys.RoundStatsKey("run", 0); !errors.Is(err, contract.ErrInvalidArgument) { + t.Fatalf("invalid round returned %v", err) + } + if _, err := keys.RoundActiveKey("run", 1, " "); !errors.Is(err, contract.ErrInvalidArgument) { + t.Fatalf("invalid channel returned %v", err) + } + if key, err := keys.RunMetaKey("run"); err != nil || key != "flow:{run}:meta" { + t.Fatalf("zero-value keys differ from legacy default: %s %v", key, err) + } +} diff --git a/internal/fullsync/contract/replay.go b/internal/fullsync/contract/replay.go new file mode 100644 index 0000000..fd15bd4 --- /dev/null +++ b/internal/fullsync/contract/replay.go @@ -0,0 +1,172 @@ +package contract + +import ( + "crypto/sha256" + "encoding/hex" + "encoding/json" + "strings" + "time" + "unicode" + "unicode/utf8" +) + +const MaxIdempotencyKeyBytes = 256 + +// ReplayRequest combines the path, header and body. Only requestedBy is in the +// HTTP body. Idempotency keys are scoped to the source run; fingerprints cover +// the source and exact request body, not the key or a generated target run ID. +type ReplayRequest struct { + SourceRunID string `json:"-"` + IdempotencyKey string `json:"-"` + RequestedBy string `json:"requestedBy"` +} + +func ValidateIdempotencyKey(key string) error { + if strings.TrimSpace(key) == "" || len(key) > MaxIdempotencyKeyBytes || !utf8.ValidString(key) || strings.ContainsFunc(key, unicode.IsControl) { + return invalid("invalid Idempotency-Key") + } + return nil +} + +func (r ReplayRequest) Validate() error { + if err := ValidateID(r.SourceRunID); err != nil { + return err + } + if err := ValidateIdempotencyKey(r.IdempotencyKey); err != nil { + return err + } + if strings.TrimSpace(r.RequestedBy) == "" || !utf8.ValidString(r.RequestedBy) { + return invalid("requestedBy is required") + } + return nil +} + +func (r ReplayRequest) Fingerprint() (string, error) { + if err := r.Validate(); err != nil { + return "", err + } + // JSON avoids ambiguous delimiter concatenation and fixes field ordering. + data, err := json.Marshal(struct { + SourceRunID string `json:"sourceRunId"` + RequestedBy string `json:"requestedBy"` + }{r.SourceRunID, r.RequestedBy}) + if err != nil { + return "", invalid("cannot encode replay request: %v", err) + } + digest := sha256.Sum256(data) + return hex.EncodeToString(digest[:]), nil +} + +// Counters describe distinct source failures, not scan visits or retry calls. +// ScanCursor alone is not an enqueue/cleanup checkpoint: repositories must +// retain sufficient durable per-batch evidence to resume partial writes. +type ReplayProgress struct { + ScanCursor Cursor `json:"scanCursor"` + DiscoveredCount int64 `json:"discoveredCount"` + EnqueuedCount int64 `json:"enqueuedCount"` + CleanedCount int64 `json:"cleanedCount"` + FailureCount int64 `json:"failureCount"` +} + +func (p ReplayProgress) Validate() error { + if err := nonnegative(p.DiscoveredCount, p.EnqueuedCount, p.CleanedCount, p.FailureCount); err != nil { + return err + } + if p.EnqueuedCount > p.DiscoveredCount || p.CleanedCount > p.EnqueuedCount { + return invalid("replay progress must satisfy cleaned <= enqueued <= discovered") + } + return nil +} + +type ReplayAction struct { + ActionID string `json:"actionId"` + SourceRunID string `json:"sourceRunId"` + TargetRunID string `json:"targetRunId"` + RequestedBy string `json:"requestedBy"` + IdempotencyKey string `json:"-"` + Fingerprint string `json:"-"` + Status ActionStatus `json:"status"` + ReplayCycle int `json:"replayCycle"` + Progress ReplayProgress `json:"progress"` + LeaseOwner string `json:"-"` + LeaseToken string `json:"-"` + LeaseUntil time.Time `json:"-"` + Revision uint64 `json:"-"` + Error string `json:"error,omitempty"` + CreatedAt time.Time `json:"createdAt"` + UpdatedAt time.Time `json:"updatedAt"` + CompletedAt *time.Time `json:"completedAt,omitempty"` + Receipt *ArchiveReceipt `json:"receipt,omitempty"` +} + +func (a ReplayAction) Validate() error { + for _, id := range []string{a.ActionID, a.SourceRunID, a.TargetRunID} { + if err := ValidateID(id); err != nil { + return err + } + } + if a.SourceRunID == a.TargetRunID || a.ReplayCycle < 1 || !a.Status.Valid() || strings.TrimSpace(a.RequestedBy) == "" { + return invalid("invalid replay action identity or status") + } + if len(a.Error) > MaxErrorMessageBytes || !utf8.ValidString(a.Error) { + return invalid("action error exceeds message bounds") + } + if err := a.Progress.Validate(); err != nil { + return err + } + if a.Receipt != nil { + if err := a.Receipt.Validate(); err != nil { + return err + } + } + if a.Status == ActionCompleted && (a.Receipt == nil || a.CompletedAt == nil || a.CompletedAt.IsZero()) { + return invalid("completed action requires a durable receipt and completedAt") + } + return nil +} + +type ReplayActionPage struct { + Items []ReplayAction `json:"items"` + NextCursor Cursor `json:"nextCursor"` +} + +type ReplayActionCreation struct { + Action ReplayAction + Created bool +} + +// Lease token fences a particular acquisition, even if the same owner name is +// reused. Repository methods check expiry against their authoritative clock. +type ReplayLease struct { + ActionID string + Owner string + Token string + Until time.Time +} + +// ReplayActionUpdate is a compare-and-swap of mutable action state. Identity, +// provenance, request fingerprint, and target run must never be changed. +type ReplayActionUpdate struct { + Lease ReplayLease + ExpectedRevision uint64 + Status ActionStatus + Progress ReplayProgress + Error string + Receipt *ArchiveReceipt + CompletedAt *time.Time +} + +// Cleanup contains the exact source versions durably accepted by the target. +// The engine must not request cleanup until target enqueue is confirmed. +type ReplayCleanup struct { + Lease ReplayLease + SourceRunID string + TargetRunID string + Entries []FailureEntry +} + +type ReplayCleanupResult struct { + CleanedCount int64 + AlreadyCleanedCount int64 + ConflictCount int64 +} diff --git a/internal/fullsync/contract/replay_test.go b/internal/fullsync/contract/replay_test.go new file mode 100644 index 0000000..e7556c4 --- /dev/null +++ b/internal/fullsync/contract/replay_test.go @@ -0,0 +1,103 @@ +package contract_test + +import ( + "encoding/json" + "errors" + "reflect" + "strings" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" +) + +func TestReplayRequestIdentity(t *testing.T) { + request := contract.ReplayRequest{SourceRunID: "source", IdempotencyKey: "key", RequestedBy: "operator"} + fingerprint, err := request.Fingerprint() + if err != nil || len(fingerprint) != 64 { + t.Fatalf("fingerprint=%q, err=%v", fingerprint, err) + } + for name, mutate := range map[string]func(*contract.ReplayRequest){ + "same": func(*contract.ReplayRequest) {}, + "key": func(r *contract.ReplayRequest) { r.IdempotencyKey = "another-key" }, + "source": func(r *contract.ReplayRequest) { r.SourceRunID = "another-source" }, + "body": func(r *contract.ReplayRequest) { r.RequestedBy = "another-operator" }, + } { + copy := request + mutate(©) + got, err := copy.Fingerprint() + if err != nil || (got == fingerprint) != (name == "same" || name == "key") { + t.Errorf("%s fingerprint=%s err=%v", name, got, err) + } + } + if got := string(mustJSON(t, request)); got != `{"requestedBy":"operator"}` { + t.Fatalf("path/header leaked into body: %s", got) + } + for _, bad := range []contract.ReplayRequest{ + {SourceRunID: "source", RequestedBy: "operator"}, + {SourceRunID: "source", IdempotencyKey: "key", RequestedBy: " "}, + {SourceRunID: "{source}", IdempotencyKey: "key", RequestedBy: "operator"}, + {SourceRunID: "source", IdempotencyKey: "key\r\ninjected", RequestedBy: "operator"}, + {SourceRunID: "source", IdempotencyKey: strings.Repeat("x", 257), RequestedBy: "operator"}, + } { + if _, err := bad.Fingerprint(); !errors.Is(err, contract.ErrInvalidArgument) { + t.Errorf("bad request returned %v", err) + } + } +} + +func TestActionProgressAndReceiptValidation(t *testing.T) { + for _, progress := range []contract.ReplayProgress{ + {DiscoveredCount: -1}, {DiscoveredCount: 1, EnqueuedCount: 2}, + {DiscoveredCount: 2, EnqueuedCount: 1, CleanedCount: 2}, {FailureCount: -1}, + } { + if err := progress.Validate(); !errors.Is(err, contract.ErrInvalidArgument) { + t.Fatalf("bad progress %+v returned %v", progress, err) + } + } + now := time.Now().UTC() + action := contract.ReplayAction{ActionID: "action", SourceRunID: "source", TargetRunID: "target", RequestedBy: "operator", ReplayCycle: 1, Status: contract.ActionAccepted} + if err := action.Validate(); err != nil { + t.Fatal(err) + } + action.Status = contract.ActionCompleted + if err := action.Validate(); !errors.Is(err, contract.ErrInvalidArgument) { + t.Fatalf("completed action without receipt returned %v", err) + } + action.Receipt = &contract.ArchiveReceipt{Version: 1, Path: "receipt.json", Checksum: strings.Repeat("a", 64), ArchivedAt: now} + action.CompletedAt = &now + if err := action.Validate(); err != nil { + t.Fatal(err) + } + action.Receipt.Checksum = "not-a-checksum" + if err := action.Validate(); !errors.Is(err, contract.ErrInvalidArgument) { + t.Fatalf("bad receipt returned %v", err) + } +} + +func TestActionWireFormatHidesInternalState(t *testing.T) { + action := contract.ReplayAction{ + ActionID: "action", SourceRunID: "source", TargetRunID: "target", RequestedBy: "operator", ReplayCycle: 1, + Status: contract.ActionCleanupPending, IdempotencyKey: "secret", Fingerprint: "internal-fingerprint", + LeaseOwner: "worker", LeaseToken: "token", LeaseUntil: time.Now(), Revision: 9, + Progress: contract.ReplayProgress{ScanCursor: "opaque", DiscoveredCount: 10, EnqueuedCount: 8, CleanedCount: 7, FailureCount: 2}, + } + data := mustJSON(t, action) + for _, hidden := range []string{"idempotency", "fingerprint", "lease", "revision", "secret", "token"} { + if strings.Contains(strings.ToLower(string(data)), hidden) { + t.Errorf("internal %s leaked: %s", hidden, data) + } + } + var decoded contract.ReplayAction + if err := json.Unmarshal(data, &decoded); err != nil || !reflect.DeepEqual(decoded.Progress, action.Progress) || decoded.Status != contract.ActionCleanupPending { + t.Fatalf("action roundtrip lost state: %+v, %v", decoded, err) + } + statusURL, err := contract.ActionStatusURL("action%1") + if err != nil || statusURL != "/internal/v1/full-sync-actions/action%251" { + t.Fatalf("status URL=%s, %v", statusURL, err) + } + accepted := contract.ReplayAccepted{ActionID: "action", TargetRunID: "target", Status: contract.ActionAccepted, StatusURL: "/internal/v1/full-sync-actions/action"} + if got := string(mustJSON(t, accepted)); got != `{"actionId":"action","targetRunId":"target","status":"accepted","statusUrl":"/internal/v1/full-sync-actions/action"}` { + t.Fatalf("202 contract changed: %s", got) + } +} diff --git a/internal/fullsync/contract/run.go b/internal/fullsync/contract/run.go new file mode 100644 index 0000000..6365209 --- /dev/null +++ b/internal/fullsync/contract/run.go @@ -0,0 +1,146 @@ +package contract + +import ( + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply" + "time" +) + +// RunMeta retains every existing queue metadata field and JSON tag. The +// optional fields are additive; zero extension fields are legal on old runs. +type RunMeta struct { + SupplyObservation *supply.Snapshot `json:"supplyObservation,omitempty"` + RunID string `json:"runId"` + Operation string `json:"operation"` + QueryMonth string `json:"queryMonth"` + Status RunStatus `json:"status"` + ProducerDone bool `json:"producerDone"` + CurrentRound int `json:"currentRound"` + ActiveCount int64 `json:"activeCount"` + InflightCount int64 `json:"inflightCount"` + NextRoundCount int64 `json:"nextRoundCount"` + SuccessCount int64 `json:"successCount"` + FailedCount int64 `json:"failedCount"` + CreatedAt time.Time `json:"createdAt"` + ProducerCompletedAt time.Time `json:"producerCompletedAt"` + CompletedAt time.Time `json:"completedAt"` + + SyncType SyncType `json:"syncType,omitempty"` + SourceRunID string `json:"sourceRunId,omitempty"` + ReplayCycle int `json:"replayCycle,omitempty"` + TriggerSource string `json:"triggerSource,omitempty"` + ScannedCount int64 `json:"scannedCount,omitempty"` + EnqueuedCount int64 `json:"enqueuedCount,omitempty"` + UpstreamAttemptCount int64 `json:"upstreamAttemptCount,omitempty"` + BatchCount int64 `json:"batchCount,omitempty"` + MaxRound int `json:"maxRound,omitempty"` + PendingFailureCount int64 `json:"pendingFailureCount,omitempty"` + StartedAt *time.Time `json:"startedAt,omitempty"` + ScanCompletedAt *time.Time `json:"scanCompletedAt,omitempty"` + ArchiveStatus ArchiveStatus `json:"archiveStatus,omitempty"` + ArchiveVersion int `json:"archiveVersion,omitempty"` + ArchivePath string `json:"archivePath,omitempty"` + ArchiveChecksum string `json:"archiveChecksum,omitempty"` + ArchivedAt *time.Time `json:"archivedAt,omitempty"` + ReplayDisposition ReplayDisposition `json:"replayDisposition,omitempty"` + // Scope fields mirror zset.RunMeta. ScopeChannels == 0 means a full sync. + ScopeChannels int `json:"scopeChannels,omitempty"` + ScopeDigest string `json:"scopeDigest,omitempty"` + ScopeChannelsSample string `json:"scopeChannelsSample,omitempty"` + RoundStats []RoundStats `json:"rounds,omitempty"` +} + +func (m RunMeta) Validate() error { + if err := ValidateID(m.RunID); err != nil { + return err + } + if m.Status != "" && !m.Status.Valid() { + return invalid("unknown run status") + } + if m.SyncType != "" && !m.SyncType.Valid() { + return invalid("unknown sync type") + } + if m.ArchiveStatus != "" && !m.ArchiveStatus.Valid() { + return invalid("unknown archive status") + } + if m.ReplayDisposition != "" && !m.ReplayDisposition.Valid() { + return invalid("unknown replay disposition") + } + if m.SourceRunID != "" { + if err := ValidateID(m.SourceRunID); err != nil { + return err + } + if m.SourceRunID == m.RunID { + return invalid("a run cannot replay itself") + } + } + if m.SyncType == SyncReplay && (m.SourceRunID == "" || m.ReplayCycle < 1) { + return invalid("replay requires sourceRunId and a positive replayCycle") + } + if m.SyncType == SyncNormal && (m.SourceRunID != "" || m.ReplayCycle != 0) { + return invalid("normal runs cannot have replay provenance") + } + if m.MaxRound > 0 && m.CurrentRound > m.MaxRound { + return invalid("currentRound exceeds maxRound") + } + return nonnegative(int64(m.CurrentRound), int64(m.MaxRound), int64(m.ReplayCycle), int64(m.ArchiveVersion), m.ActiveCount, m.InflightCount, m.NextRoundCount, m.SuccessCount, m.FailedCount, m.ScannedCount, m.EnqueuedCount, m.UpstreamAttemptCount, m.BatchCount, m.PendingFailureCount, int64(m.ScopeChannels)) +} + +// ValidateAccounting is opt-in for runs whose extended counters are known. +// Do not apply it to legacy metadata with a missing enqueuedCount. Inflight is +// a subset of active and is never added to the completion equation. +func (m RunMeta) ValidateAccounting() error { + if err := m.Validate(); err != nil { + return err + } + remaining := m.EnqueuedCount + for _, count := range []int64{m.SuccessCount, m.FailedCount, m.ActiveCount, m.NextRoundCount} { + if count > remaining { + return invalid("enqueuedCount does not balance run outcomes") + } + remaining -= count + } + if remaining != 0 || m.InflightCount > m.ActiveCount || m.PendingFailureCount > m.FailedCount { + return invalid("inconsistent run counters") + } + if m.Status.Terminal() && (!m.ProducerDone || m.ActiveCount != 0 || m.NextRoundCount != 0) { + return invalid("terminal run still has producer or queue work") + } + return nil +} + +// RoundStats contains only bounded aggregate fields. It must never grow with +// the number of cards, channels, task IDs, upstream codes, or error messages. +type RoundStats struct { + RunID string `json:"runId"` + Round int `json:"round"` + EnqueuedCount int64 `json:"enqueuedCount"` + ActiveCount int64 `json:"activeCount"` + NextRoundCount int64 `json:"nextRoundCount"` + SuccessCount int64 `json:"successCount"` + FailedCount int64 `json:"failedCount"` + UpstreamAttemptCount int64 `json:"upstreamAttemptCount"` + BatchCount int64 `json:"batchCount"` + StartedAt *time.Time `json:"startedAt,omitempty"` + CompletedAt *time.Time `json:"completedAt,omitempty"` + ErrorCounts ErrorCounts `json:"errorCounts"` +} + +func (s RoundStats) Validate() error { + if err := ValidateID(s.RunID); err != nil { + return err + } + if s.Round < 1 { + return invalid("round must be positive") + } + if err := nonnegative(s.EnqueuedCount, s.ActiveCount, s.NextRoundCount, s.SuccessCount, s.FailedCount, s.UpstreamAttemptCount, s.BatchCount); err != nil { + return err + } + _, err := s.ErrorCounts.Normalized() + return err +} + +type RunView struct { + Source RunSource `json:"source"` + Run RunMeta `json:"run"` + Rounds []RoundStats `json:"rounds,omitempty"` +} diff --git a/internal/fullsync/contract/status.go b/internal/fullsync/contract/status.go new file mode 100644 index 0000000..2f46e26 --- /dev/null +++ b/internal/fullsync/contract/status.go @@ -0,0 +1,137 @@ +package contract + +type RunStatus string + +const ( + RunProducing RunStatus = "producing" + RunRunning RunStatus = "running" + RunCompleted RunStatus = "completed" + RunCompletedWithFailures RunStatus = "completed_with_failures" +) + +func (s RunStatus) Valid() bool { + return s == RunProducing || s == RunRunning || s.Terminal() +} + +func (s RunStatus) Terminal() bool { return s == RunCompleted || s == RunCompletedWithFailures } + +type SyncType string + +const ( + SyncNormal SyncType = "normal" + SyncReplay SyncType = "replay" +) + +func (s SyncType) Valid() bool { return s == SyncNormal || s == SyncReplay } + +type SyncKind string + +const ( + SyncKindTrafficUsage SyncKind = "traffic_usage" + SyncKindSimStatus SyncKind = "sim_status" + SyncKindTrafficUsageDaily SyncKind = "traffic_usage_daily" + SyncKindTrafficUsageMonthly SyncKind = "traffic_usage_monthly" + SyncKindImei SyncKind = "imei" +) + +func (s SyncKind) Valid() bool { + return s == SyncKindTrafficUsage || s == SyncKindSimStatus || s == SyncKindTrafficUsageDaily || s == SyncKindTrafficUsageMonthly || s == SyncKindImei +} + +func SyncKindForOperation(operation string) SyncKind { + switch operation { + case "current_month_usage_full_sync": + return SyncKindTrafficUsage + case "sim_status_queue_full_sync": + return SyncKindSimStatus + case "previous_day_usage_full_sync": + return SyncKindTrafficUsageDaily + case "previous_month_usage_full_sync": + return SyncKindTrafficUsageMonthly + case "imei_full_sync": + return SyncKindImei + default: + return "" + } +} + +type ArchiveStatus string + +const ( + ArchiveDisabled ArchiveStatus = "disabled" + ArchivePending ArchiveStatus = "pending" + ArchiveProcessing ArchiveStatus = "processing" + ArchiveSucceeded ArchiveStatus = "succeeded" + ArchiveFailed ArchiveStatus = "failed" +) + +func (s ArchiveStatus) Valid() bool { + switch s { + case ArchiveDisabled, ArchivePending, ArchiveProcessing, ArchiveSucceeded, ArchiveFailed: + return true + } + return false +} + +// ReplayDisposition describes a source run, not an individual replay action. +type ReplayDisposition string + +const ( + ReplayNone ReplayDisposition = "none" + ReplayPending ReplayDisposition = "pending" + ReplayProcessing ReplayDisposition = "processing" + ReplayCompleted ReplayDisposition = "completed" + ReplayNeedsAttention ReplayDisposition = "needs_attention" +) + +func (s ReplayDisposition) Valid() bool { + switch s { + case ReplayNone, ReplayPending, ReplayProcessing, ReplayCompleted, ReplayNeedsAttention: + return true + } + return false +} + +type ActionStatus string + +const ( + ActionAccepted ActionStatus = "accepted" + ActionProcessing ActionStatus = "processing" + ActionCleanupPending ActionStatus = "cleanup_pending" + ActionReceiptPending ActionStatus = "receipt_pending" + ActionCompleted ActionStatus = "completed" + ActionNeedsAttention ActionStatus = "needs_attention" +) + +func (s ActionStatus) Valid() bool { + switch s { + case ActionAccepted, ActionProcessing, ActionCleanupPending, ActionReceiptPending, ActionCompleted, ActionNeedsAttention: + return true + } + return false +} + +// Terminal means automatic work has stopped; needs_attention may be resumed +// only by an explicit recovery policy, never by assuming it succeeded. +func (s ActionStatus) Terminal() bool { return s == ActionCompleted || s == ActionNeedsAttention } + +type RunSource string + +const ( + SourceLive RunSource = "live" + SourceArchive RunSource = "archive" +) + +func (s RunSource) Valid() bool { return s == SourceLive || s == SourceArchive } + +// These values are already persisted by the queue protocol. +const ( + TaskQueued = "queued" + TaskClaimed = "claimed" + TaskWaitingNextRound = "waiting_next_round" + TaskSucceeded = "succeeded" + TaskCovered = "covered" + TaskFailedPendingReview = "failed_pending_review" + TaskChannelBlocked = "channel_blocked" + ReviewPending = "pending" +) diff --git a/internal/fullsync/contract/status_test.go b/internal/fullsync/contract/status_test.go new file mode 100644 index 0000000..7487abc --- /dev/null +++ b/internal/fullsync/contract/status_test.go @@ -0,0 +1,57 @@ +package contract_test + +import ( + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +// TestSyncKindForDailyUsageOperation guards the mapping that makes a daily run +// discoverable. Catalog.ListRuns drops any run whose operation maps to an empty +// kind, so losing this case would make daily runs invisible in the monitor +// without any error. +func TestSyncKindForDailyUsageOperation(t *testing.T) { + if got := contract.SyncKindForOperation(model.OperationPreviousDayUsageFullSync); got != contract.SyncKindTrafficUsageDaily { + t.Fatalf("SyncKindForOperation(%q) = %q, want %q", + model.OperationPreviousDayUsageFullSync, got, contract.SyncKindTrafficUsageDaily) + } + if !contract.SyncKindTrafficUsageDaily.Valid() { + t.Fatal("SyncKindTrafficUsageDaily must be a valid kind") + } + if contract.SyncKindForOperation("some_unknown_operation") != "" { + t.Fatal("an unknown operation must not be assigned a sync kind") + } +} + +// TestSyncKindForMonthlyHistoryOperation guards the mapping that makes a monthly +// history run discoverable. Catalog.ListRuns drops any run whose operation maps +// to an empty kind, so losing this case would make monthly history runs +// invisible in the monitor without any error. +func TestSyncKindForMonthlyHistoryOperation(t *testing.T) { + if got := contract.SyncKindForOperation(model.OperationPreviousMonthUsageFullSync); got != contract.SyncKindTrafficUsageMonthly { + t.Fatalf("SyncKindForOperation(%q) = %q, want %q", + model.OperationPreviousMonthUsageFullSync, got, contract.SyncKindTrafficUsageMonthly) + } +} + +// TestDailyAndMonthlyUsageHaveDistinctSyncKinds keeps the usage runs separately +// filterable. There are three distinct usage runs now: the MySQL current-month +// snapshot, the TDengine daily history, and the TDengine monthly history. +func TestDailyAndMonthlyUsageHaveDistinctSyncKinds(t *testing.T) { + monthly := contract.SyncKindForOperation("current_month_usage_full_sync") + daily := contract.SyncKindForOperation(model.OperationPreviousDayUsageFullSync) + monthlyHistory := contract.SyncKindForOperation(model.OperationPreviousMonthUsageFullSync) + if monthly == daily { + t.Fatalf("current-month snapshot and daily history share the sync kind %q", daily) + } + if monthly == monthlyHistory { + t.Fatalf("current-month snapshot and monthly history share the sync kind %q", monthlyHistory) + } + if daily == monthlyHistory { + t.Fatalf("daily and monthly history share the sync kind %q", daily) + } + if !monthlyHistory.Valid() { + t.Fatal("SyncKindTrafficUsageMonthly must be a valid kind") + } +} diff --git a/internal/fullsync/contract/sync_kind_test.go b/internal/fullsync/contract/sync_kind_test.go new file mode 100644 index 0000000..47fbef4 --- /dev/null +++ b/internal/fullsync/contract/sync_kind_test.go @@ -0,0 +1,85 @@ +package contract_test + +import ( + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" +) + +// TestSyncKindForOperationCoversEveryChain pins the operation→kind mapping for +// all five full-sync chains. A missing case makes the run invisible in the +// admin monitor, because the catalog filters runs by SyncKind. +func TestSyncKindForOperationCoversEveryChain(t *testing.T) { + cases := []struct { + operation string + want contract.SyncKind + }{ + {"current_month_usage_full_sync", contract.SyncKindTrafficUsage}, + {"sim_status_queue_full_sync", contract.SyncKindSimStatus}, + {"previous_day_usage_full_sync", contract.SyncKindTrafficUsageDaily}, + {"previous_month_usage_full_sync", contract.SyncKindTrafficUsageMonthly}, + {"imei_full_sync", contract.SyncKindImei}, + } + for _, tc := range cases { + got := contract.SyncKindForOperation(tc.operation) + if got != tc.want { + t.Fatalf("SyncKindForOperation(%q) = %q, want %q", tc.operation, got, tc.want) + } + if !got.Valid() { + t.Fatalf("%q 应通过 Valid()", got) + } + } +} + +// TestSyncKindForOperationRejectsUnknown 未知 operation 返回空 kind， +// 由调用方决定是跳过还是报错。 +func TestSyncKindForOperationRejectsUnknown(t *testing.T) { + for _, operation := range []string{"", "unknown", "IMEI_FULL_SYNC"} { + got := contract.SyncKindForOperation(operation) + if got != "" { + t.Fatalf("SyncKindForOperation(%q) = %q, want empty", operation, got) + } + if got.Valid() { + t.Fatalf("空 kind 不应通过 Valid()") + } + } +} + +// TestSyncKindValidRejectsUnknown 只有五个已注册的 kind 有效。 +func TestSyncKindValidRejectsUnknown(t *testing.T) { + valid := []contract.SyncKind{ + contract.SyncKindTrafficUsage, + contract.SyncKindSimStatus, + contract.SyncKindTrafficUsageDaily, + contract.SyncKindTrafficUsageMonthly, + contract.SyncKindImei, + } + for _, kind := range valid { + if !kind.Valid() { + t.Fatalf("%q 应有效", kind) + } + } + invalid := []contract.SyncKind{"", "imei_full_sync", "IMEI", "traffic", "sim-status"} + for _, kind := range invalid { + if kind.Valid() { + t.Fatalf("%q 不应有效", kind) + } + } +} + +// TestSyncKindValuesAreStable 枚举字符串是持久化契约（写入 Run 元数据、 +// 归档 JSON 与 admin 查询参数），不得随意改动。 +func TestSyncKindValuesAreStable(t *testing.T) { + cases := map[contract.SyncKind]string{ + contract.SyncKindTrafficUsage: "traffic_usage", + contract.SyncKindSimStatus: "sim_status", + contract.SyncKindTrafficUsageDaily: "traffic_usage_daily", + contract.SyncKindTrafficUsageMonthly: "traffic_usage_monthly", + contract.SyncKindImei: "imei", + } + for kind, want := range cases { + if string(kind) != want { + t.Fatalf("kind = %q, want %q（持久化契约不得改动）", kind, want) + } + } +} diff --git a/internal/fullsync/contract/validation.go b/internal/fullsync/contract/validation.go new file mode 100644 index 0000000..ddfbcb7 --- /dev/null +++ b/internal/fullsync/contract/validation.go @@ -0,0 +1,42 @@ +package contract + +import ( + "strings" + "unicode" + "unicode/utf8" + + "github.com/gogf/gf/v2/errors/gerror" +) + +// Implementations wrap these errors so callers can use errors.Is without +// inspecting error text. HTTP adapters choose the response envelope themselves. +var ( + ErrNotFound = gerror.New("full-sync resource not found") + ErrConflict = gerror.New("full-sync conflict") + ErrInvalidArgument = gerror.New("invalid full-sync argument") + ErrFailureDetailsUnavailable = gerror.New("live failure details unavailable") + ErrLeaseLost = gerror.New("full-sync lease lost") + ErrBusy = gerror.New("full-sync resource busy") +) + +func invalid(format string, args ...any) error { + return gerror.Wrapf(ErrInvalidArgument, format, args...) +} + +// ValidateID checks new resource identifiers used both as Redis hash tags and +// HTTP path segments. It does not rewrite historical identifiers during decode. +func ValidateID(id string) error { + if id == "" || strings.TrimSpace(id) != id || !utf8.ValidString(id) || strings.ContainsAny(id, "{}/\\") || strings.ContainsFunc(id, unicode.IsControl) { + return invalid("invalid resource identifier") + } + return nil +} + +func nonnegative(values ...int64) error { + for _, value := range values { + if value < 0 { + return invalid("counts must be nonnegative") + } + } + return nil +} diff --git a/internal/fullsync/contract/validation_test.go b/internal/fullsync/contract/validation_test.go new file mode 100644 index 0000000..1aa8b10 --- /dev/null +++ b/internal/fullsync/contract/validation_test.go @@ -0,0 +1,121 @@ +package contract_test + +import ( + "errors" + "math" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" +) + +func TestRunAccounting(t *testing.T) { + valid := contract.RunMeta{ + RunID: "run", Status: contract.RunRunning, CurrentRound: 1, MaxRound: 5, + EnqueuedCount: 10, ActiveCount: 3, InflightCount: 2, NextRoundCount: 2, + SuccessCount: 4, FailedCount: 1, PendingFailureCount: 1, + } + if err := valid.ValidateAccounting(); err != nil { + t.Fatal(err) + } + for name, mutate := range map[string]func(*contract.RunMeta){ + "unbalanced": func(m *contract.RunMeta) { m.EnqueuedCount++ }, + "negative": func(m *contract.RunMeta) { m.BatchCount = -1 }, + "overflow": func(m *contract.RunMeta) { + m.EnqueuedCount = math.MaxInt64 + m.SuccessCount = math.MaxInt64 + m.FailedCount = math.MaxInt64 + }, + "inflight double count": func(m *contract.RunMeta) { m.InflightCount = m.ActiveCount + 1 }, + "pending exceeds history": func(m *contract.RunMeta) { m.PendingFailureCount = 2 }, + "terminal with work": func(m *contract.RunMeta) { m.Status = contract.RunCompletedWithFailures; m.ProducerDone = true }, + "unknown status": func(m *contract.RunMeta) { m.Status = "succeeded" }, + "unknown sync type": func(m *contract.RunMeta) { m.SyncType = "manual" }, + "action status in archive": func(m *contract.RunMeta) { m.ArchiveStatus = "completed" }, + "action status in disposition": func(m *contract.RunMeta) { m.ReplayDisposition = "cleanup_pending" }, + "round out of range": func(m *contract.RunMeta) { m.CurrentRound = 6 }, + "replay without source": func(m *contract.RunMeta) { m.SyncType = contract.SyncReplay }, + "self replay": func(m *contract.RunMeta) { + m.SyncType = contract.SyncReplay + m.SourceRunID = m.RunID + m.ReplayCycle = 1 + }, + "normal with provenance": func(m *contract.RunMeta) { m.SyncType = contract.SyncNormal; m.SourceRunID = "source" }, + } { + t.Run(name, func(t *testing.T) { + meta := valid + mutate(&meta) + if err := meta.ValidateAccounting(); !errors.Is(err, contract.ErrInvalidArgument) { + t.Fatalf("invalid accounting returned %v", err) + } + }) + } + // Cleaned source failures never subtract from the historical failed total. + completed := contract.RunMeta{RunID: "source", Status: contract.RunCompletedWithFailures, ProducerDone: true, EnqueuedCount: 5, SuccessCount: 3, FailedCount: 2, PendingFailureCount: 0} + if err := completed.ValidateAccounting(); err != nil { + t.Fatal(err) + } + completed.ProducerDone = false + if err := completed.ValidateAccounting(); err == nil { + t.Fatal("accepted terminal run with unfinished producer") + } + if err := (contract.RunMeta{RunID: "old", Status: contract.RunRunning, SuccessCount: 42}).Validate(); err != nil { + t.Fatalf("historical run without new counters rejected: %v", err) + } +} + +func TestRoundStatsValidation(t *testing.T) { + valid := contract.RoundStats{RunID: "run", Round: 1, ErrorCounts: contract.ErrorCounts{"unrecognized": 1}} + if err := valid.Validate(); err != nil { + t.Fatal(err) + } + for _, stats := range []contract.RoundStats{ + {RunID: "run", Round: 0}, {RunID: "run", Round: 1, BatchCount: -1}, + {RunID: "run", Round: 1, ErrorCounts: contract.ErrorCounts{contract.ErrorOther: -1}}, + } { + if err := stats.Validate(); !errors.Is(err, contract.ErrInvalidArgument) { + t.Fatalf("invalid round stats returned %v", err) + } + } +} + +func TestPageLimitsAndOpaqueCursor(t *testing.T) { + for _, limit := range []int{0, 1, 100, 500} { + request, err := (contract.PageRequest{Limit: limit, Cursor: "opaque+/=:%?next"}).Normalized() + if err != nil || request.Cursor != "opaque+/=:%?next" || request.Limit < 1 || request.Limit > contract.MaxPageLimit { + t.Fatalf("unexpected page: %+v, %v", request, err) + } + if limit == 0 && request.Limit != 100 { + t.Fatal("default limit changed") + } + } + for _, limit := range []int{-1, 501, math.MaxInt} { + if _, err := (contract.PageRequest{Limit: limit}).Normalized(); !errors.Is(err, contract.ErrInvalidArgument) { + t.Fatalf("limit %d returned %v", limit, err) + } + } + page := contract.FailurePage{Items: []contract.Failure{}, NextCursor: "continue"} + if got := string(mustJSON(t, page)); got != `{"items":[],"nextCursor":"continue"}` { + t.Fatalf("empty scan page lost its continuation: %s", got) + } +} + +func TestStatusVocabulary(t *testing.T) { + for _, status := range []contract.ActionStatus{"accepted", "processing", "cleanup_pending", "receipt_pending", "completed", "needs_attention"} { + if !status.Valid() || status.Terminal() != (status == "completed" || status == "needs_attention") { + t.Errorf("invalid action status %q", status) + } + } + for _, status := range []contract.ArchiveStatus{"disabled", "pending", "processing", "succeeded", "failed"} { + if !status.Valid() { + t.Errorf("invalid archive status %q", status) + } + } + for _, status := range []contract.ReplayDisposition{"none", "pending", "processing", "completed", "needs_attention"} { + if !status.Valid() { + t.Errorf("invalid disposition %q", status) + } + } + if contract.ActionStatus("succeeded").Valid() || contract.ArchiveStatus("completed").Valid() || contract.ReplayDisposition("receipt_pending").Valid() { + t.Fatal("status domains were conflated") + } +} diff --git a/internal/fullsync/httpapi/README.md b/internal/fullsync/httpapi/README.md new file mode 100644 index 0000000..caa0069 --- /dev/null +++ b/internal/fullsync/httpapi/README.md @@ -0,0 +1,99 @@ +# Full-sync HTTP control API + +The handler imports no queue or storage implementations. Inject the five +contract interfaces in `Dependencies`: `RunLister`, `RunReader`, +`FailurePager`, `ReplayRequester`, and `ActionReader`. The replay service owns durable +idempotency and execution; the HTTP layer does not cache requests or launch work. + +Use `NewRunReader(live, archive, rounds)` to compose `LiveRunReader`, +`ArchivedRunReader`, and `RoundStatsReader` into a `RunReader`. Live metadata +wins and returns `source: "live"`; only `ErrNotFound` from live metadata permits +archive fallback, returning `source: "archive"`. Live storage errors and round +lookup errors propagate. Archive and round readers may be nil to disable those +lookups. A failure listing always calls `FailurePager` directly, never archives. + +## Routes and responses + +All routes require `Authorization: Bearer `. Missing, wrong, +or ambiguous credentials return 401 with a `WWW-Authenticate` challenge. An +empty configured token is a startup configuration error. Responses use JSON and +`Cache-Control: no-store`. + +| Method | Route | Success body | +| --- | --- | --- | +| GET | `/internal/v1/full-sync-runs?syncKind=&limit=` | 200 `contract.RunList` | +| GET | `/internal/v1/full-sync-runs/{runId}` | 200 `contract.RunView` | +| GET | `/internal/v1/full-sync-runs/{runId}/failures?cursor=&limit=` | 200 `contract.FailurePage` | +| POST | `/internal/v1/full-sync-runs/{runId}/replay` | 202 `contract.ReplayAccepted` | +| GET | `/internal/v1/full-sync-actions/{actionId}` | 200 `contract.ReplayAction` | + +Run list results merge active Redis metadata with flat `run-.json` +archives under the configured archive root. Active Redis metadata wins when an +ID exists in both sources. `syncKind` accepts `traffic_usage` or `sim_status`; +`limit` defaults to 50 and is capped at 100. + +Failure limits use the contract rules: absent, empty, or zero selects 100; +positive values up to 500 are accepted; other values are rejected. Repeated +cursor/limit parameters and malformed query encoding are rejected. Cursors are +opaque and passed unchanged after URL decoding. The pager validates cursor +contents and run binding. Empty pages serialize `items: []` and preserve +`nextCursor`; only an empty next cursor signals completion. + +Replay requires a single `Idempotency-Key` header and a JSON body such as +`{"requestedBy":"operator"}`. The body is limited to 64 KiB, must contain one +JSON object with no unknown fields, and must satisfy `ReplayRequest.Validate`. +If supplied, Content-Type must be `application/json` (parameters are allowed). +The exact `requestedBy` value is preserved for fingerprinting. Repeated accepted +requests return the service's existing action/target and current status with +202. The adapter constructs `statusUrl` and the `Location` header from actionId. + +All application errors use this envelope, without storage error details: + +```json +{"error":{"code":"invalid_argument","message":"Invalid request"}} +``` + +| Condition | HTTP status | Error code | +| --- | --- | --- | +| Invalid path, query, header, body, or `ErrInvalidArgument` | 400 | `invalid_argument` | +| Missing/wrong bearer credentials | 401 | `unauthorized` | +| Unknown route/resource or `ErrNotFound` | 404 | `not_found` | +| Unsupported route method | 405 | `method_not_allowed` | +| `ErrConflict` (including changed idempotent request), `ErrLeaseLost` | 409 | `conflict` | +| `ErrFailureDetailsUnavailable` | 410 | `failure_details_unavailable` | +| Oversized replay body | 413 | `body_too_large` | +| `ErrBusy` | 503 | `busy` | +| Other service errors or unencodable service output | 500 | `internal_error` | + +## Lifecycle + +`NewHandler(token, deps)` can be mounted by an existing HTTP server. For an +independent listener, construct `NewServer(Config{Addr: address, BearerToken: +token}, deps)` and call `Start(workerCtx)`. `Start` binds synchronously and +returns address-in-use and other startup errors directly. `Addr()` exposes the +bound address, including an OS-assigned port when configured with port zero. + +Call `Wait()` after Start, normally in the worker's supervision goroutine. It +returns serving or shutdown failures; it also returns startup failures if Start +failed. The server is single-use and rejects a second Start. Wait can have +multiple callers but must not be called before Start. + +Canceling `workerCtx` closes the listener and drains active HTTP requests. +Active request contexts stay usable during the drain. `ShutdownTimeout` defaults +to ten seconds; if it expires, active connections are closed and Wait returns +an error wrapping `context.DeadlineExceeded`. Normal cancellation with a +successful drain returns nil. No worker wiring or configuration-file changes +are part of this package. + +## Verification + +```text +gofmt -l internal/fullsync/httpapi +go test ./internal/fullsync/... +go test ./internal/fullsync/httpapi -cover +go vet ./internal/fullsync/... +``` + +Tests use contract mocks and local HTTP listeners, requiring no Redis or archive +files. `go test -race ./internal/fullsync/...` additionally requires a supported +C compiler and CGO enabled. diff --git a/internal/fullsync/httpapi/SIMPLE_REPLAY.md b/internal/fullsync/httpapi/SIMPLE_REPLAY.md new file mode 100644 index 0000000..095c096 --- /dev/null +++ b/internal/fullsync/httpapi/SIMPLE_REPLAY.md @@ -0,0 +1,10 @@ +# Simple full-sync replay + +The Worker control server listens on `:18081` inside the private Docker network. Do not publish port 18081 to the host or the internet. Admin already enforces user JWT authentication and replay permissions and calls `http://iotlcm-worker-svr:18081` directly; no additional Worker URL or bearer token environment variables are needed. + +- `POST /api/v1/iot/fullSyncRuns/replay` with `{ "sourceRunId": "..." }` returns `replayRunId` and `accepted`. Admin requires `cpm:iot:fullSyncRuns:replay`. +- `GET /api/v1/iot/fullSyncRuns/replay/{replayRunId}` reports admission (`accepted`, `processing`, `needs_attention`, `completed`), remaining source records, enqueued count, and the independent target Run status. Requires `cpm:iot:fullSyncRuns:replay:view`. + +All failures in a completed `flow:` or `sim-status:` Run are selected. No retryability filter; the target Run has exactly one business round. Source failure entries are deleted only after target task-state confirms enqueue; original outcome counts and historical JSON remain unchanged. Repeat requests to the same source return the same target. After repairing a `needs_attention` condition, repeat the request to resume it. To retry again after completion, use the new Replay Run as the source. + +No deployment or live Replay request is part of this change. diff --git a/internal/fullsync/httpapi/handler.go b/internal/fullsync/httpapi/handler.go new file mode 100644 index 0000000..4315fef --- /dev/null +++ b/internal/fullsync/httpapi/handler.go @@ -0,0 +1,321 @@ +// Package httpapi exposes the internal full-sync control API through contract +// interfaces. Persistence and replay execution belong to the injected services. +package httpapi + +import ( + "bytes" + "crypto/subtle" + "encoding/json" + "errors" + "io" + "mime" + "net/http" + "net/url" + "strconv" + "strings" + "unicode" + "unicode/utf8" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" + fullsyncruntime "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/runtime" + + "github.com/gogf/gf/v2/errors/gerror" +) + +const maxReplayBodyBytes = 64 << 10 + +// Dependencies are the only application services used by the HTTP handlers. +type Dependencies struct { + Runs contract.RunReader + List contract.RunLister + Failures contract.FailurePager + Replays contract.ReplayRequester + Actions contract.ActionReader + Simple *fullsyncruntime.SimpleReplay +} + +// ErrorResponse is shared by authentication, routing, validation and service +// failures. Messages deliberately exclude underlying storage error details. +type ErrorResponse struct { + Error APIError `json:"error"` +} + +type APIError struct { + Code string `json:"code"` + Message string `json:"message"` +} + +// NewHandler exposes the control routes on the private Docker network. Admin +// performs the user JWT and permission checks before calling this listener. +func NewHandler(bearerToken string, deps Dependencies) (http.Handler, error) { + if strings.ContainsFunc(bearerToken, func(r rune) bool { return unicode.IsSpace(r) || unicode.IsControl(r) }) { + return nil, gerror.New("invalid full-sync HTTP token") + } + if deps.Runs == nil || deps.List == nil || deps.Failures == nil || deps.Replays == nil || deps.Actions == nil { + return nil, gerror.New("full-sync HTTP requires run list, run, failure, replay and action services") + } + mux := http.NewServeMux() + mux.HandleFunc(contract.ListRunsRoute, method(http.MethodGet, func(w http.ResponseWriter, r *http.Request) { + query, err := url.ParseQuery(r.URL.RawQuery) + if err != nil || len(query["limit"]) > 1 || len(query["syncKind"]) > 1 { + writeError(w, contract.ErrInvalidArgument) + return + } + filter := contract.RunListFilter{SyncKind: contract.SyncKind(query.Get("syncKind"))} + if filter.SyncKind != "" && !filter.SyncKind.Valid() { + writeError(w, contract.ErrInvalidArgument) + return + } + if query.Get("limit") != "" { + filter.Limit, err = strconv.Atoi(query.Get("limit")) + if err != nil || filter.Limit < 1 || filter.Limit > 100 { + writeError(w, contract.ErrInvalidArgument) + return + } + } + result, err := deps.List.ListRuns(r.Context(), filter) + if err != nil { + writeError(w, err) + return + } + writeJSON(w, http.StatusOK, result) + })) + mux.HandleFunc(contract.GetRunRoute, method(http.MethodGet, func(w http.ResponseWriter, r *http.Request) { + id := r.PathValue("runId") + if err := contract.ValidateID(id); err != nil { + writeError(w, err) + return + } + view, err := deps.Runs.GetRun(r.Context(), id) + if err != nil { + writeError(w, err) + return + } + writeJSON(w, http.StatusOK, view) + })) + mux.HandleFunc(contract.GetFailuresRoute, method(http.MethodGet, func(w http.ResponseWriter, r *http.Request) { + id := r.PathValue("runId") + if err := contract.ValidateID(id); err != nil { + writeError(w, err) + return + } + page, err := parsePage(r.URL.RawQuery) + if err != nil { + writeError(w, err) + return + } + result, err := deps.Failures.ListFailures(r.Context(), id, page) + if err != nil { + writeError(w, err) + return + } + if result.Items == nil { + result.Items = []contract.Failure{} + } + writeJSON(w, http.StatusOK, result) + })) + mux.HandleFunc(contract.PostReplayRoute, method(http.MethodPost, func(w http.ResponseWriter, r *http.Request) { + if deps.Simple != nil { + id := r.PathValue("runId") + if err := contract.ValidateID(id); err != nil { + writeError(w, err) + return + } + var request struct { + RequestedBy string `json:"requestedBy"` + MaxRounds int `json:"maxRounds"` + } + decoder := json.NewDecoder(http.MaxBytesReader(w, r.Body, maxReplayBodyBytes)) + decoder.DisallowUnknownFields() + if err := decoder.Decode(&request); err != nil { + writeError(w, contract.ErrInvalidArgument) + return + } + if err := decoder.Decode(new(any)); err != io.EOF { + writeError(w, contract.ErrInvalidArgument) + return + } + accepted, err := deps.Simple.Request(r.Context(), id, request.RequestedBy, request.MaxRounds) + if errors.Is(err, fullsyncruntime.ErrNoFailures) { + writeAPIError(w, http.StatusUnprocessableEntity, "no_failures", "Run has no failures to replay") + return + } + if err != nil { + writeError(w, err) + return + } + writeJSON(w, http.StatusAccepted, accepted) + return + } + request, err := parseReplay(w, r) + if err != nil { + var tooLarge *http.MaxBytesError + if errors.As(err, &tooLarge) { + writeAPIError(w, http.StatusRequestEntityTooLarge, "body_too_large", "Replay request body is too large") + } else { + writeError(w, err) + } + return + } + accepted, err := deps.Replays.RequestReplay(r.Context(), request) + if err != nil { + writeError(w, err) + return + } + // Construct a local status URL from the returned identity, never an + // externally supplied redirect. Invalid service output is a server error. + accepted.StatusURL, err = contract.ActionStatusURL(accepted.ActionID) + if err != nil || contract.ValidateID(accepted.TargetRunID) != nil || !accepted.Status.Valid() { + writeAPIError(w, http.StatusInternalServerError, "internal_error", "Internal server error") + return + } + w.Header().Set("Location", accepted.StatusURL) + writeJSON(w, http.StatusAccepted, accepted) + })) + mux.HandleFunc("/internal/v1/full-sync-replays/{replayRunId}", method(http.MethodGet, func(w http.ResponseWriter, r *http.Request) { + if deps.Simple == nil { + writeError(w, contract.ErrNotFound) + return + } + status, err := deps.Simple.Status(r.Context(), r.PathValue("replayRunId")) + if err != nil { + writeError(w, err) + return + } + writeJSON(w, http.StatusOK, status) + })) + mux.HandleFunc(contract.GetActionRoute, method(http.MethodGet, func(w http.ResponseWriter, r *http.Request) { + if deps.Simple != nil { + writeError(w, contract.ErrNotFound) + return + } + id := r.PathValue("actionId") + if err := contract.ValidateID(id); err != nil { + writeError(w, err) + return + } + action, err := deps.Actions.GetReplayAction(r.Context(), id) + if err != nil { + writeError(w, err) + return + } + writeJSON(w, http.StatusOK, action) + })) + mux.HandleFunc("/", func(w http.ResponseWriter, _ *http.Request) { + writeError(w, contract.ErrNotFound) + }) + if bearerToken == "" { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Cache-Control", "no-store") + mux.ServeHTTP(w, r) + }), nil + } + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Cache-Control", "no-store") + values := r.Header.Values("Authorization") + if len(values) == 1 { + parts := strings.Fields(values[0]) + if len(parts) == 2 && strings.EqualFold(parts[0], "Bearer") && subtle.ConstantTimeCompare([]byte(parts[1]), []byte(bearerToken)) == 1 { + mux.ServeHTTP(w, r) + return + } + } + w.Header().Set("WWW-Authenticate", `Bearer realm="full-sync"`) + writeAPIError(w, http.StatusUnauthorized, "unauthorized", "Valid bearer authentication is required") + }), nil +} + +func method(expected string, next http.HandlerFunc) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + if r.Method != expected { + w.Header().Set("Allow", expected) + writeAPIError(w, http.StatusMethodNotAllowed, "method_not_allowed", "Method not allowed") + return + } + next(w, r) + } +} + +func parsePage(rawQuery string) (contract.PageRequest, error) { + query, err := url.ParseQuery(rawQuery) + if err != nil || len(query["cursor"]) > 1 || len(query["limit"]) > 1 { + return contract.PageRequest{}, gerror.Wrap(contract.ErrInvalidArgument, "invalid pagination query") + } + page := contract.PageRequest{Cursor: contract.Cursor(query.Get("cursor"))} + if value := query.Get("limit"); value != "" { + page.Limit, err = strconv.Atoi(value) + if err != nil { + return contract.PageRequest{}, gerror.Wrap(contract.ErrInvalidArgument, "limit must be an integer") + } + } + return page.Normalized() +} + +func parseReplay(w http.ResponseWriter, r *http.Request) (contract.ReplayRequest, error) { + var request contract.ReplayRequest + if value := r.Header.Get("Content-Type"); value != "" { + mediaType, _, err := mime.ParseMediaType(value) + if err != nil || mediaType != "application/json" { + return request, gerror.Wrap(contract.ErrInvalidArgument, "replay body must be JSON") + } + } + data, err := io.ReadAll(http.MaxBytesReader(w, r.Body, maxReplayBodyBytes)) + if err != nil { + var tooLarge *http.MaxBytesError + if errors.As(err, &tooLarge) { + return request, err + } + return request, gerror.Wrap(contract.ErrInvalidArgument, "cannot read replay body") + } + if !utf8.Valid(data) { + return request, gerror.Wrap(contract.ErrInvalidArgument, "replay body must be UTF-8") + } + decoder := json.NewDecoder(bytes.NewReader(data)) + decoder.DisallowUnknownFields() + if err := decoder.Decode(&request); err != nil { + return request, gerror.Wrap(contract.ErrInvalidArgument, "invalid replay JSON") + } + if err := decoder.Decode(new(any)); err != io.EOF { + return request, gerror.Wrap(contract.ErrInvalidArgument, "replay body must contain exactly one JSON object") + } + if len(r.Header.Values(contract.IdempotencyHeader)) != 1 { + return request, gerror.Wrap(contract.ErrInvalidArgument, "exactly one Idempotency-Key is required") + } + request.SourceRunID = r.PathValue("runId") + request.IdempotencyKey = r.Header.Get(contract.IdempotencyHeader) + return request, request.Validate() +} + +func writeError(w http.ResponseWriter, err error) { + switch { + case errors.Is(err, contract.ErrInvalidArgument): + writeAPIError(w, http.StatusBadRequest, "invalid_argument", "Invalid request") + case errors.Is(err, contract.ErrNotFound): + writeAPIError(w, http.StatusNotFound, "not_found", "Resource not found") + case errors.Is(err, contract.ErrFailureDetailsUnavailable): + writeAPIError(w, http.StatusGone, "failure_details_unavailable", "Live failure details are no longer available") + case errors.Is(err, contract.ErrConflict), errors.Is(err, contract.ErrLeaseLost): + writeAPIError(w, http.StatusConflict, "conflict", "Request conflicts with the current resource") + case errors.Is(err, contract.ErrBusy): + writeAPIError(w, http.StatusServiceUnavailable, "busy", "Resource is busy; retry later") + default: + writeAPIError(w, http.StatusInternalServerError, "internal_error", "Internal server error") + } +} + +func writeAPIError(w http.ResponseWriter, status int, code, message string) { + writeJSON(w, status, ErrorResponse{Error: APIError{Code: code, Message: message}}) +} + +func writeJSON(w http.ResponseWriter, status int, value any) { + // Marshal before writing headers so an invalid service payload cannot + // produce a successful status with truncated JSON. + data, err := json.Marshal(value) + if err != nil { + status = http.StatusInternalServerError + data = []byte(`{"error":{"code":"internal_error","message":"Internal server error"}}`) + } + w.Header().Set("Content-Type", "application/json; charset=utf-8") + w.WriteHeader(status) + _, _ = w.Write(append(data, '\n')) +} diff --git a/internal/fullsync/httpapi/handler_test.go b/internal/fullsync/httpapi/handler_test.go new file mode 100644 index 0000000..b74018c --- /dev/null +++ b/internal/fullsync/httpapi/handler_test.go @@ -0,0 +1,437 @@ +package httpapi + +import ( + "context" + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "net/url" + "reflect" + "strings" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" +) + +const ( + testToken = "test-secret" + runPath = "/internal/v1/full-sync-runs/run-1" + failurePath = runPath + "/failures" + replayPath = runPath + "/replay" + actionPath = "/internal/v1/full-sync-actions/action-1" +) + +type runFunc func(context.Context, string) (contract.RunView, error) + +func (f runFunc) GetRun(ctx context.Context, id string) (contract.RunView, error) { + return f(ctx, id) +} + +type runListFunc func(context.Context, contract.RunListFilter) (contract.RunList, error) + +func (f runListFunc) ListRuns(ctx context.Context, filter contract.RunListFilter) (contract.RunList, error) { + return f(ctx, filter) +} + +type failureFunc func(context.Context, string, contract.PageRequest) (contract.FailurePage, error) + +func (f failureFunc) ListFailures(ctx context.Context, id string, page contract.PageRequest) (contract.FailurePage, error) { + return f(ctx, id, page) +} + +type replayFunc func(context.Context, contract.ReplayRequest) (contract.ReplayAccepted, error) + +func (f replayFunc) RequestReplay(ctx context.Context, req contract.ReplayRequest) (contract.ReplayAccepted, error) { + return f(ctx, req) +} + +type actionFunc func(context.Context, string) (contract.ReplayAction, error) + +func (f actionFunc) GetReplayAction(ctx context.Context, id string) (contract.ReplayAction, error) { + return f(ctx, id) +} + +func testDependencies() Dependencies { + return Dependencies{ + Runs: runFunc(func(context.Context, string) (contract.RunView, error) { + return contract.RunView{}, contract.ErrNotFound + }), + List: runListFunc(func(context.Context, contract.RunListFilter) (contract.RunList, error) { + return contract.RunList{}, contract.ErrNotFound + }), + Failures: failureFunc(func(context.Context, string, contract.PageRequest) (contract.FailurePage, error) { + return contract.FailurePage{}, contract.ErrNotFound + }), + Replays: replayFunc(func(context.Context, contract.ReplayRequest) (contract.ReplayAccepted, error) { + return contract.ReplayAccepted{}, contract.ErrNotFound + }), + Actions: actionFunc(func(context.Context, string) (contract.ReplayAction, error) { + return contract.ReplayAction{}, contract.ErrNotFound + }), + } +} + +func testHandler(t *testing.T, deps Dependencies) http.Handler { + t.Helper() + handler, err := NewHandler(testToken, deps) + if err != nil { + t.Fatal(err) + } + return handler +} + +func request(handler http.Handler, method, path, body string, headers ...string) *httptest.ResponseRecorder { + req := httptest.NewRequest(method, path, strings.NewReader(body)) + req.Header.Set("Authorization", "Bearer "+testToken) + for i := 0; i < len(headers); i += 2 { + req.Header.Set(headers[i], headers[i+1]) + } + recorder := httptest.NewRecorder() + handler.ServeHTTP(recorder, req) + return recorder +} + +func assertError(t *testing.T, response *httptest.ResponseRecorder, status int, code string) { + t.Helper() + if response.Code != status { + t.Fatalf("status = %d, want %d: %s", response.Code, status, response.Body) + } + if !strings.HasPrefix(response.Header().Get("Content-Type"), "application/json") { + t.Fatalf("missing JSON content type: %v", response.Header()) + } + var body ErrorResponse + if err := json.Unmarshal(response.Body.Bytes(), &body); err != nil { + t.Fatal(err) + } + if body.Error.Code != code || body.Error.Message == "" { + t.Fatalf("error = %+v, want code %q and message", body, code) + } +} + +func TestAuthenticationProtectsEveryRoute(t *testing.T) { + deps := testDependencies() + deps.Runs = runFunc(func(context.Context, string) (contract.RunView, error) { + t.Fatal("unauthenticated request reached service") + return contract.RunView{}, nil + }) + handler := testHandler(t, deps) + for _, path := range []string{runPath, failurePath, replayPath, actionPath, "/unknown"} { + for _, auth := range []string{"", "Bearer", "Bearer wrong", "Basic " + testToken, "Bearer " + testToken + " extra"} { + t.Run(path+"/"+auth, func(t *testing.T) { + response := request(handler, http.MethodGet, path, "", "Authorization", auth) + assertError(t, response, 401, "unauthorized") + if response.Header().Get("WWW-Authenticate") == "" { + t.Fatal("missing authentication challenge") + } + }) + } + } + req := httptest.NewRequest(http.MethodGet, runPath, nil) + req.Header.Add("Authorization", "Bearer "+testToken) + req.Header.Add("Authorization", "Bearer "+testToken) + response := httptest.NewRecorder() + handler.ServeHTTP(response, req) + assertError(t, response, 401, "unauthorized") +} + +func TestRunListQuery(t *testing.T) { + deps := testDependencies() + deps.List = runListFunc(func(_ context.Context, filter contract.RunListFilter) (contract.RunList, error) { + if filter.SyncKind != contract.SyncKindTrafficUsage || filter.Limit != 7 { + t.Fatalf("filter = %+v", filter) + } + return contract.RunList{Items: []contract.RunView{{Source: contract.SourceLive, Run: contract.RunMeta{RunID: "run-1", Operation: "current_month_usage_full_sync"}}}, Total: 1}, nil + }) + response := request(testHandler(t, deps), http.MethodGet, "/internal/v1/full-sync-runs?syncKind=traffic_usage&limit=7", "") + if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "run-1") { + t.Fatalf("response = %d %s", response.Code, response.Body) + } + for _, query := range []string{"syncKind=invalid", "limit=101", "limit=abc", "limit=1&limit=2"} { + assertError(t, request(testHandler(t, deps), http.MethodGet, "/internal/v1/full-sync-runs?"+query, ""), 400, "invalid_argument") + } +} + +func TestRoutesAndValidation(t *testing.T) { + handler := testHandler(t, testDependencies()) + for _, route := range []struct{ method, path string }{ + {http.MethodGet, runPath}, {http.MethodGet, failurePath}, + {http.MethodPost, replayPath}, {http.MethodGet, actionPath}, + } { + t.Run(route.path, func(t *testing.T) { + response := request(handler, route.method, route.path, `{"requestedBy":"operator"}`, contract.IdempotencyHeader, "key") + assertError(t, response, 404, "not_found") + wrongMethod := http.MethodPost + if route.method == http.MethodPost { + wrongMethod = http.MethodGet + } + response = request(handler, wrongMethod, route.path, "") + assertError(t, response, 405, "method_not_allowed") + if response.Header().Get("Allow") != route.method { + t.Fatalf("Allow = %q", response.Header().Get("Allow")) + } + }) + } + for _, path := range []string{"/unknown", "/internal/v1/full-sync-runs", runPath + "/other", runPath + "/"} { + assertError(t, request(handler, http.MethodGet, path, ""), 404, "not_found") + } + for _, path := range []string{ + "/internal/v1/full-sync-runs/%20run", + "/internal/v1/full-sync-runs/a%2Fb/failures", + "/internal/v1/full-sync-actions/%7Baction%7D", + } { + assertError(t, request(handler, http.MethodGet, path, ""), 400, "invalid_argument") + } + assertError(t, request(handler, http.MethodPost, "/internal/v1/full-sync-runs/a%5Cb/replay", `{"requestedBy":"operator"}`, contract.IdempotencyHeader, "key"), 400, "invalid_argument") +} + +func TestRunAndActionResponsesPropagateContext(t *testing.T) { + type contextKey struct{} + deps := testDependencies() + deps.Runs = runFunc(func(ctx context.Context, id string) (contract.RunView, error) { + if id != "run-1" || ctx.Value(contextKey{}) != "request" { + t.Fatal("run identity or request context lost") + } + return contract.RunView{Source: contract.SourceLive, Run: contract.RunMeta{RunID: id}}, nil + }) + deps.Actions = actionFunc(func(ctx context.Context, id string) (contract.ReplayAction, error) { + if id != "action-1" || ctx.Value(contextKey{}) != "request" { + t.Fatal("action identity or request context lost") + } + return contract.ReplayAction{ActionID: id, Status: contract.ActionProcessing, IdempotencyKey: "private-key", Fingerprint: "private-fingerprint", LeaseToken: "private-lease", Revision: 99}, nil + }) + handler := testHandler(t, deps) + for _, path := range []string{runPath, actionPath} { + req := httptest.NewRequest(http.MethodGet, path, nil).WithContext(context.WithValue(context.Background(), contextKey{}, "request")) + req.Header.Set("Authorization", "bearer "+testToken) + response := httptest.NewRecorder() + handler.ServeHTTP(response, req) + if response.Code != 200 || response.Header().Get("Cache-Control") != "no-store" { + t.Fatalf("response = %d %s", response.Code, response.Body) + } + if strings.Contains(response.Body.String(), "private-") || strings.Contains(response.Body.String(), "revision") { + t.Fatalf("internal fields leaked: %s", response.Body) + } + } +} + +func TestFailurePagination(t *testing.T) { + for _, test := range []struct { + query string + limit int + cursor contract.Cursor + }{ + {"", 100, ""}, {"?limit=", 100, ""}, {"?limit=0", 100, ""}, + {"?limit=1", 1, ""}, {"?limit=500", 500, ""}, + {"?limit=17&cursor=" + url.QueryEscape("opaque/+ token==:雪"), 17, "opaque/+ token==:雪"}, + } { + t.Run(test.query, func(t *testing.T) { + deps := testDependencies() + calls := 0 + deps.Failures = failureFunc(func(_ context.Context, id string, page contract.PageRequest) (contract.FailurePage, error) { + calls++ + if id != "run-1" || page.Limit != test.limit || page.Cursor != test.cursor { + t.Fatalf("query = %q %+v", id, page) + } + return contract.FailurePage{NextCursor: "next:opaque/+="}, nil + }) + response := request(testHandler(t, deps), http.MethodGet, failurePath+test.query, "") + if response.Code != 200 || calls != 1 { + t.Fatalf("response = %d, calls = %d", response.Code, calls) + } + var page contract.FailurePage + if err := json.Unmarshal(response.Body.Bytes(), &page); err != nil { + t.Fatal(err) + } + if page.Items == nil || len(page.Items) != 0 || page.NextCursor != "next:opaque/+=" { + t.Fatalf("empty scan page lost array or continuation: %s", response.Body) + } + }) + } + deps := testDependencies() + deps.Failures = failureFunc(func(context.Context, string, contract.PageRequest) (contract.FailurePage, error) { + t.Fatal("invalid pagination reached service") + return contract.FailurePage{}, nil + }) + handler := testHandler(t, deps) + for _, query := range []string{"limit=-1", "limit=501", "limit=abc", "limit=1.5", "limit=999999999999999999999999", "limit=1&limit=2", "cursor=a&cursor=b", "cursor=%ZZ", "cursor=a;b"} { + assertError(t, request(handler, http.MethodGet, failurePath+"?"+query, ""), 400, "invalid_argument") + } +} + +func TestFailureCursorValidationAndLiveOnly(t *testing.T) { + deps := testDependencies() + deps.Runs = runFunc(func(context.Context, string) (contract.RunView, error) { + t.Fatal("failure route must not query run summaries or fall back to archives") + return contract.RunView{}, nil + }) + deps.Failures = failureFunc(func(_ context.Context, _ string, page contract.PageRequest) (contract.FailurePage, error) { + if page.Cursor == "another-run-token" { + return contract.FailurePage{}, fmt.Errorf("wrong query binding: %w", contract.ErrInvalidArgument) + } + return contract.FailurePage{}, fmt.Errorf("expired details: %w", contract.ErrFailureDetailsUnavailable) + }) + handler := testHandler(t, deps) + assertError(t, request(handler, http.MethodGet, failurePath+"?cursor=another-run-token", ""), 400, "invalid_argument") + assertError(t, request(handler, http.MethodGet, failurePath, ""), 410, "failure_details_unavailable") +} + +func TestReplayValidation(t *testing.T) { + deps := testDependencies() + deps.Replays = replayFunc(func(context.Context, contract.ReplayRequest) (contract.ReplayAccepted, error) { + t.Fatal("invalid replay reached service") + return contract.ReplayAccepted{}, nil + }) + handler := testHandler(t, deps) + for _, body := range []string{"", "{", "null", "[]", `{}`, `{"requestedBy":""}`, `{"requestedBy":" "}`, `{"requestedBy":12}`, `{"requestedBy":"a","targetRunId":"injected"}`, `{"requestedBy":"a"} {}`, `{"requestedBy":"a"} trailing`, "{\"requestedBy\":\"\xff\"}"} { + t.Run(body, func(t *testing.T) { + assertError(t, request(handler, http.MethodPost, replayPath, body, contract.IdempotencyHeader, "key"), 400, "invalid_argument") + }) + } + for _, key := range []string{"", " ", strings.Repeat("x", contract.MaxIdempotencyKeyBytes+1), "bad\tkey"} { + assertError(t, request(handler, http.MethodPost, replayPath, `{"requestedBy":"operator"}`, contract.IdempotencyHeader, key), 400, "invalid_argument") + } + assertError(t, request(handler, http.MethodPost, replayPath, `{"requestedBy":"operator"}`), 400, "invalid_argument") + assertError(t, request(handler, http.MethodPost, replayPath, `{"requestedBy":"operator"}`, contract.IdempotencyHeader, "key", "Content-Type", "text/plain"), 400, "invalid_argument") + assertError(t, request(handler, http.MethodPost, replayPath, `{"requestedBy":"`+strings.Repeat("x", maxReplayBodyBytes)+`"}`, contract.IdempotencyHeader, "key"), 413, "body_too_large") + req := httptest.NewRequest(http.MethodPost, replayPath, strings.NewReader(`{"requestedBy":"operator"}`)) + req.Header.Set("Authorization", "Bearer "+testToken) + req.Header.Add(contract.IdempotencyHeader, "key") + req.Header.Add(contract.IdempotencyHeader, "key") + response := httptest.NewRecorder() + handler.ServeHTTP(response, req) + assertError(t, response, 400, "invalid_argument") +} + +// The mock owns idempotency; the HTTP adapter must pass exact request identity +// and return the service's existing action/target even across handler instances. +func TestReplayAcceptedIdempotencyAndConflict(t *testing.T) { + deps := testDependencies() + var ( + stored contract.ReplayRequest + calls int + ) + deps.Replays = replayFunc(func(_ context.Context, req contract.ReplayRequest) (contract.ReplayAccepted, error) { + calls++ + if calls == 1 { + stored = req + } + if req != stored { + return contract.ReplayAccepted{}, fmt.Errorf("changed fingerprint: %w", contract.ErrConflict) + } + return contract.ReplayAccepted{ActionID: "action-1", TargetRunID: "replay-1", Status: contract.ActionAccepted, StatusURL: "https://untrusted.invalid"}, nil + }) + var first contract.ReplayAccepted + for i := 0; i < 2; i++ { + response := request(testHandler(t, deps), http.MethodPost, replayPath, "{\"requestedBy\":\" operator \"}\n", contract.IdempotencyHeader, "key", "Content-Type", "application/json; charset=utf-8") + if response.Code != 202 { + t.Fatalf("response = %d %s", response.Code, response.Body) + } + var accepted contract.ReplayAccepted + if err := json.Unmarshal(response.Body.Bytes(), &accepted); err != nil { + t.Fatal(err) + } + if accepted.ActionID != "action-1" || accepted.TargetRunID != "replay-1" || accepted.StatusURL != actionPath || response.Header().Get("Location") != actionPath { + t.Fatalf("acceptance = %+v, headers = %v", accepted, response.Header()) + } + if i == 0 { + first = accepted + } else if !reflect.DeepEqual(first, accepted) { + t.Fatalf("repeat changed identity: %+v vs %+v", first, accepted) + } + } + if stored.SourceRunID != "run-1" || stored.RequestedBy != " operator " || stored.IdempotencyKey != "key" { + t.Fatalf("request identity changed: %+v", stored) + } + assertError(t, request(testHandler(t, deps), http.MethodPost, replayPath, `{"requestedBy":"operator"}`, contract.IdempotencyHeader, "key"), 409, "conflict") + if calls != 3 { + t.Fatalf("replay calls = %d", calls) + } +} + +func TestWrappedServiceErrorsUseConsistentEnvelope(t *testing.T) { + for _, test := range []struct { + err error + status int + code string + }{ + {contract.ErrInvalidArgument, 400, "invalid_argument"}, + {contract.ErrNotFound, 404, "not_found"}, + {contract.ErrConflict, 409, "conflict"}, + {contract.ErrLeaseLost, 409, "conflict"}, + {contract.ErrFailureDetailsUnavailable, 410, "failure_details_unavailable"}, + {contract.ErrBusy, 503, "busy"}, + {fmt.Errorf("private database password"), 500, "internal_error"}, + } { + t.Run(test.code, func(t *testing.T) { + err := fmt.Errorf("private database detail: %w", test.err) + deps := Dependencies{ + Runs: runFunc(func(context.Context, string) (contract.RunView, error) { return contract.RunView{}, err }), + List: runListFunc(func(context.Context, contract.RunListFilter) (contract.RunList, error) { + return contract.RunList{}, err + }), + Failures: failureFunc(func(context.Context, string, contract.PageRequest) (contract.FailurePage, error) { + return contract.FailurePage{}, err + }), + Replays: replayFunc(func(context.Context, contract.ReplayRequest) (contract.ReplayAccepted, error) { + return contract.ReplayAccepted{}, err + }), + Actions: actionFunc(func(context.Context, string) (contract.ReplayAction, error) { return contract.ReplayAction{}, err }), + } + handler := testHandler(t, deps) + for _, path := range []string{runPath, failurePath, actionPath, replayPath} { + method := http.MethodGet + if path == replayPath { + method = http.MethodPost + } + response := request(handler, method, path, `{"requestedBy":"operator"}`, contract.IdempotencyHeader, "key") + assertError(t, response, test.status, test.code) + if strings.Contains(response.Body.String(), "private") { + t.Fatal("service details leaked") + } + } + }) + } +} + +func TestInvalidServiceOutputReturns500(t *testing.T) { + deps := testDependencies() + deps.Replays = replayFunc(func(context.Context, contract.ReplayRequest) (contract.ReplayAccepted, error) { + return contract.ReplayAccepted{ActionID: "bad/id"}, nil + }) + deps.Runs = runFunc(func(context.Context, string) (contract.RunView, error) { + return contract.RunView{Rounds: []contract.RoundStats{{ErrorCounts: contract.ErrorCounts{contract.ErrorOther: -1}}}}, nil + }) + handler := testHandler(t, deps) + assertError(t, request(handler, http.MethodPost, replayPath, `{"requestedBy":"operator"}`, contract.IdempotencyHeader, "key"), 500, "internal_error") + assertError(t, request(handler, http.MethodGet, runPath, ""), 500, "internal_error") +} + +func TestNewHandlerRejectsInvalidConfiguration(t *testing.T) { + if _, err := NewHandler("", testDependencies()); err != nil { + t.Fatalf("empty token should use Docker-network-only access: %v", err) + } + for _, token := range []string{" ", "a b", "token\n"} { + if _, err := NewHandler(token, testDependencies()); err == nil { + t.Errorf("accepted token %q", token) + } + } + for i := 0; i < 5; i++ { + deps := testDependencies() + switch i { + case 0: + deps.Runs = nil + case 1: + deps.Failures = nil + case 2: + deps.Replays = nil + case 3: + deps.Actions = nil + case 4: + deps.List = nil + } + if _, err := NewHandler(testToken, deps); err == nil { + t.Errorf("accepted missing dependency %d", i) + } + } +} diff --git a/internal/fullsync/httpapi/run_reader.go b/internal/fullsync/httpapi/run_reader.go new file mode 100644 index 0000000..40cb5a4 --- /dev/null +++ b/internal/fullsync/httpapi/run_reader.go @@ -0,0 +1,48 @@ +package httpapi + +import ( + "context" + "errors" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" + + "github.com/gogf/gf/v2/errors/gerror" +) + +type runReader struct { + live contract.LiveRunReader + archive contract.ArchivedRunReader + rounds contract.RoundStatsReader +} + +// NewRunReader composes live-first run queries without knowing either storage +// implementation. Archive and rounds are optional (nil disables that lookup). +// Only absent live metadata permits archive fallback; live round errors do not. +func NewRunReader(live contract.LiveRunReader, archive contract.ArchivedRunReader, rounds contract.RoundStatsReader) (contract.RunReader, error) { + if live == nil { + return nil, gerror.New("full-sync run reader requires a live reader") + } + return &runReader{live: live, archive: archive, rounds: rounds}, nil +} + +func (r *runReader) GetRun(ctx context.Context, runID string) (contract.RunView, error) { + if err := contract.ValidateID(runID); err != nil { + return contract.RunView{}, err + } + meta, err := r.live.GetRunMeta(ctx, runID) + if err == nil { + view := contract.RunView{Source: contract.SourceLive, Run: meta} + if r.rounds != nil { + view.Rounds, err = r.rounds.GetRoundStats(ctx, runID) + } + return view, err + } + if !errors.Is(err, contract.ErrNotFound) || r.archive == nil { + return contract.RunView{}, err + } + archive, err := r.archive.GetArchivedRun(ctx, runID) + if err != nil { + return contract.RunView{}, err + } + return contract.RunView{Source: contract.SourceArchive, Run: archive.Run, Rounds: archive.Rounds}, nil +} diff --git a/internal/fullsync/httpapi/run_reader_test.go b/internal/fullsync/httpapi/run_reader_test.go new file mode 100644 index 0000000..2e971cb --- /dev/null +++ b/internal/fullsync/httpapi/run_reader_test.go @@ -0,0 +1,125 @@ +package httpapi + +import ( + "context" + "encoding/json" + "fmt" + "net/http" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" +) + +type liveFunc func(context.Context, string) (contract.RunMeta, error) + +func (f liveFunc) GetRunMeta(ctx context.Context, id string) (contract.RunMeta, error) { + return f(ctx, id) +} + +type archiveFunc func(context.Context, string) (contract.RunArchive, error) + +func (f archiveFunc) GetArchivedRun(ctx context.Context, id string) (contract.RunArchive, error) { + return f(ctx, id) +} + +type roundsFunc func(context.Context, string) ([]contract.RoundStats, error) + +func (f roundsFunc) GetRoundStats(ctx context.Context, id string) ([]contract.RoundStats, error) { + return f(ctx, id) +} + +func TestRunLookupLiveFirstAndArchiveFallback(t *testing.T) { + for _, test := range []struct { + name string + liveErr error + roundsErr error + archiveErr error + wantStatus int + wantSource contract.RunSource + wantArchives int + wantRounds int + }{ + {name: "live", wantStatus: 200, wantSource: contract.SourceLive, wantRounds: 1}, + {name: "archive", liveErr: fmt.Errorf("expired: %w", contract.ErrNotFound), wantStatus: 200, wantSource: contract.SourceArchive, wantArchives: 1}, + {name: "unknown", liveErr: contract.ErrNotFound, archiveErr: contract.ErrNotFound, wantStatus: 404, wantArchives: 1}, + {name: "live failure never falls back", liveErr: fmt.Errorf("live offline"), wantStatus: 500}, + {name: "round error never falls back", roundsErr: fmt.Errorf("rounds offline"), wantStatus: 500, wantRounds: 1}, + {name: "missing rounds never fall back", roundsErr: contract.ErrNotFound, wantStatus: 404, wantRounds: 1}, + {name: "archive error", liveErr: contract.ErrNotFound, archiveErr: fmt.Errorf("archive offline"), wantStatus: 500, wantArchives: 1}, + } { + t.Run(test.name, func(t *testing.T) { + var liveCalls, archiveCalls, roundsCalls int + reader, err := NewRunReader( + liveFunc(func(_ context.Context, id string) (contract.RunMeta, error) { + liveCalls++ + if id != "run-1" { + t.Fatalf("live ID = %q", id) + } + return contract.RunMeta{RunID: id, CurrentRound: 2}, test.liveErr + }), + archiveFunc(func(_ context.Context, id string) (contract.RunArchive, error) { + archiveCalls++ + if id != "run-1" { + t.Fatalf("archive ID = %q", id) + } + return contract.RunArchive{Run: contract.RunMeta{RunID: id, CurrentRound: 3}, Rounds: []contract.RoundStats{{RunID: id, Round: 3}}}, test.archiveErr + }), + roundsFunc(func(_ context.Context, id string) ([]contract.RoundStats, error) { + roundsCalls++ + if id != "run-1" { + t.Fatalf("rounds ID = %q", id) + } + return []contract.RoundStats{{RunID: id, Round: 2}}, test.roundsErr + }), + ) + if err != nil { + t.Fatal(err) + } + deps := testDependencies() + deps.Runs = reader + response := request(testHandler(t, deps), http.MethodGet, runPath, "") + if response.Code != test.wantStatus { + t.Fatalf("response = %d %s", response.Code, response.Body) + } + if liveCalls != 1 || archiveCalls != test.wantArchives || roundsCalls != test.wantRounds { + t.Fatalf("calls: live=%d archive=%d rounds=%d", liveCalls, archiveCalls, roundsCalls) + } + if response.Code == 200 { + var view contract.RunView + if err := json.Unmarshal(response.Body.Bytes(), &view); err != nil { + t.Fatal(err) + } + if view.Source != test.wantSource || view.Run.RunID != "run-1" || len(view.Rounds) != 1 || view.Rounds[0].Round != view.Run.CurrentRound { + t.Fatalf("view = %+v", view) + } + } + }) + } +} + +func TestRunReaderOptionalServicesAndValidation(t *testing.T) { + if _, err := NewRunReader(nil, nil, nil); err == nil { + t.Fatal("accepted nil live reader") + } + calls := 0 + reader, err := NewRunReader(liveFunc(func(_ context.Context, id string) (contract.RunMeta, error) { + calls++ + if id == "missing" { + return contract.RunMeta{}, contract.ErrNotFound + } + return contract.RunMeta{RunID: id}, nil + }), nil, nil) + if err != nil { + t.Fatal(err) + } + if _, err := reader.GetRun(context.Background(), "bad/id"); err == nil || calls != 0 { + t.Fatal("invalid ID reached live reader") + } + view, err := reader.GetRun(context.Background(), "run-1") + if err != nil || view.Source != contract.SourceLive || view.Rounds != nil { + t.Fatalf("view = %+v, err = %v", view, err) + } + if _, err := reader.GetRun(context.Background(), "missing"); err != contract.ErrNotFound { + t.Fatalf("error = %v", err) + } +} diff --git a/internal/fullsync/httpapi/server.go b/internal/fullsync/httpapi/server.go new file mode 100644 index 0000000..a9abe0d --- /dev/null +++ b/internal/fullsync/httpapi/server.go @@ -0,0 +1,119 @@ +package httpapi + +import ( + "context" + "errors" + "net" + "net/http" + "sync" + "time" + + "github.com/gogf/gf/v2/errors/gerror" +) + +// Config is deliberately independent of worker/config-file wiring. Addr must +// be explicit; use 127.0.0.1:0 to let the OS allocate a local test port. +type Config struct { + Addr string + BearerToken string // Optional for standalone callers; production Worker leaves it empty. + ShutdownTimeout time.Duration +} + +// Server is single-use. Start reports bind errors synchronously; after a +// successful Start, Wait reports serve or shutdown errors. Canceling the Start +// context stops accepting requests and drains active requests up to the timeout. +type Server struct { + server *http.Server + shutdownTimeout time.Duration + mu sync.Mutex + started bool + addr net.Addr + done chan struct{} + err error +} + +func NewServer(config Config, deps Dependencies) (*Server, error) { + if config.Addr == "" || config.ShutdownTimeout < 0 { + return nil, gerror.New("full-sync HTTP requires an address and a nonnegative shutdown timeout") + } + handler, err := NewHandler(config.BearerToken, deps) + if err != nil { + return nil, err + } + if config.ShutdownTimeout == 0 { + config.ShutdownTimeout = 10 * time.Second + } + return &Server{ + server: &http.Server{ + Addr: config.Addr, Handler: handler, + ReadHeaderTimeout: 5 * time.Second, + ReadTimeout: 30 * time.Second, + WriteTimeout: 30 * time.Second, + IdleTimeout: 60 * time.Second, + }, + shutdownTimeout: config.ShutdownTimeout, + done: make(chan struct{}), + }, nil +} + +func (s *Server) Start(ctx context.Context) error { + s.mu.Lock() + defer s.mu.Unlock() + if s.started { + return gerror.New("full-sync HTTP server has already been started") + } + s.started = true + if err := ctx.Err(); err != nil { + s.err = err + close(s.done) + return err + } + listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", s.server.Addr) + if err != nil { + s.err = gerror.Wrap(err, "listen for full-sync HTTP") + close(s.done) + return s.err + } + s.addr = listener.Addr() + go s.run(ctx, listener) + return nil +} + +// Addr returns the bound address, or nil before a successful Start. +func (s *Server) Addr() net.Addr { + s.mu.Lock() + defer s.mu.Unlock() + return s.addr +} + +// Wait may be called concurrently after Start (including a failed Start). +func (s *Server) Wait() error { + <-s.done + return s.err +} + +func (s *Server) run(ctx context.Context, listener net.Listener) { + serveDone := make(chan error, 1) + go func() { serveDone <- s.server.Serve(listener) }() + var err error + select { + case err = <-serveDone: + // An unexpected listener failure also needs to drain active requests. + case <-ctx.Done(): + } + shutdownCtx, cancel := context.WithTimeout(context.Background(), s.shutdownTimeout) + shutdownErr := s.server.Shutdown(shutdownCtx) + cancel() + if shutdownErr != nil { + // Shutdown does not close active connections when its deadline expires. + _ = s.server.Close() + } + if err == nil { + err = <-serveDone + } + if errors.Is(err, http.ErrServerClosed) { + err = nil + } + s.err = errors.Join(err, shutdownErr) + close(s.done) +} diff --git a/internal/fullsync/httpapi/server_test.go b/internal/fullsync/httpapi/server_test.go new file mode 100644 index 0000000..6fb783c --- /dev/null +++ b/internal/fullsync/httpapi/server_test.go @@ -0,0 +1,226 @@ +package httpapi + +import ( + "context" + "errors" + "io" + "net" + "net/http" + "sync" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" +) + +func awaitServer(t *testing.T, done <-chan error) error { + t.Helper() + select { + case err := <-done: + return err + case <-time.After(5 * time.Second): + t.Fatal("server did not stop") + return nil + } +} + +func startTestServer(t *testing.T, deps Dependencies, timeout time.Duration) (*Server, context.CancelFunc, <-chan error) { + t.Helper() + server, err := NewServer(Config{Addr: "127.0.0.1:0", BearerToken: testToken, ShutdownTimeout: timeout}, deps) + if err != nil { + t.Fatal(err) + } + if server.Addr() != nil { + t.Fatal("address before Start") + } + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + if err := server.Start(ctx); err != nil { + t.Fatal(err) + } + done := make(chan error, 1) + go func() { done <- server.Wait() }() + return server, cancel, done +} + +func serverRequest(server *Server) (*http.Response, error) { + req, err := http.NewRequest(http.MethodGet, "http://"+server.Addr().String()+runPath, nil) + if err != nil { + return nil, err + } + req.Header.Set("Authorization", "Bearer "+testToken) + client := &http.Client{Timeout: 4 * time.Second} + return client.Do(req) +} + +func TestServerStartServeAndCancel(t *testing.T) { + server, cancel, done := startTestServer(t, testDependencies(), 0) + response, err := serverRequest(server) + if err != nil { + t.Fatal(err) + } + _ = response.Body.Close() + if response.StatusCode != 404 { + t.Fatalf("status = %d", response.StatusCode) + } + if err := server.Start(context.Background()); err == nil { + t.Fatal("duplicate Start succeeded") + } + cancel() + if err := awaitServer(t, done); err != nil { + t.Fatal(err) + } + if err := server.Wait(); err != nil { + t.Fatal(err) + } + connection, err := net.DialTimeout("tcp", server.Addr().String(), time.Second) + if err == nil { + _ = connection.Close() + t.Fatal("listener still accepts after shutdown") + } +} + +func TestServerGracefulShutdownDrainsActiveRequest(t *testing.T) { + var ( + entered = make(chan struct{}) + release = make(chan struct{}) + once sync.Once + ) + t.Cleanup(func() { once.Do(func() { close(release) }) }) + deps := testDependencies() + deps.Runs = runFunc(func(ctx context.Context, id string) (contract.RunView, error) { + close(entered) + select { + case <-release: + return contract.RunView{Source: contract.SourceLive, Run: contract.RunMeta{RunID: id}}, nil + case <-ctx.Done(): + return contract.RunView{}, ctx.Err() + } + }) + server, cancel, done := startTestServer(t, deps, 2*time.Second) + requestDone := make(chan error, 1) + go func() { + response, err := serverRequest(server) + if err == nil { + _, err = io.ReadAll(response.Body) + _ = response.Body.Close() + if response.StatusCode != 200 { + err = errors.New("active request did not complete successfully") + } + } + requestDone <- err + }() + select { + case <-entered: + case <-time.After(3 * time.Second): + t.Fatal("request did not enter handler") + } + cancel() + select { + case err := <-done: + t.Fatalf("shutdown returned before draining: %v", err) + case <-time.After(50 * time.Millisecond): + } + once.Do(func() { close(release) }) + if err := awaitServer(t, requestDone); err != nil { + t.Fatal(err) + } + if err := awaitServer(t, done); err != nil { + t.Fatal(err) + } +} + +func TestServerShutdownDeadlineClosesActiveConnections(t *testing.T) { + var ( + entered = make(chan struct{}) + exited = make(chan struct{}) + ) + deps := testDependencies() + deps.Runs = runFunc(func(ctx context.Context, _ string) (contract.RunView, error) { + close(entered) + <-ctx.Done() + close(exited) + return contract.RunView{}, ctx.Err() + }) + server, cancel, done := startTestServer(t, deps, 50*time.Millisecond) + requestDone := make(chan error, 1) + go func() { + response, err := serverRequest(server) + if response != nil { + _ = response.Body.Close() + } + requestDone <- err + }() + select { + case <-entered: + case <-time.After(3 * time.Second): + t.Fatal("request did not enter handler") + } + cancel() + if err := awaitServer(t, done); !errors.Is(err, context.DeadlineExceeded) { + t.Fatalf("shutdown error = %v", err) + } + select { + case <-exited: + case <-time.After(3 * time.Second): + t.Fatal("forced close did not cancel request context") + } + _ = awaitServer(t, requestDone) +} + +func TestServerStartupFailureIsSynchronous(t *testing.T) { + listener, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = listener.Close() }) + server, err := NewServer(Config{Addr: listener.Addr().String(), BearerToken: testToken}, testDependencies()) + if err != nil { + t.Fatal(err) + } + startErr := server.Start(context.Background()) + if startErr == nil { + t.Fatal("Start did not report occupied port") + } + if server.Wait() != startErr || server.Addr() != nil { + t.Fatal("failed startup state was lost") + } + var netErr *net.OpError + if !errors.As(startErr, &netErr) { + t.Fatalf("network error cause lost: %v", startErr) + } +} + +func TestServerCanceledBeforeStartAndImmediateCancellation(t *testing.T) { + server, err := NewServer(Config{Addr: "127.0.0.1:0", BearerToken: testToken}, testDependencies()) + if err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithCancel(context.Background()) + cancel() + if err := server.Start(ctx); !errors.Is(err, context.Canceled) { + t.Fatalf("Start = %v", err) + } + if err := server.Wait(); !errors.Is(err, context.Canceled) { + t.Fatalf("Wait = %v", err) + } + // Cancellation can win the race before Serve starts accepting requests. + for i := 0; i < 10; i++ { + _, cancel, done := startTestServer(t, testDependencies(), time.Second) + cancel() + if err := awaitServer(t, done); err != nil { + t.Fatal(err) + } + } +} + +func TestNewServerRejectsInvalidConfiguration(t *testing.T) { + for _, config := range []Config{ + {BearerToken: testToken}, + {Addr: "127.0.0.1:0", BearerToken: testToken, ShutdownTimeout: -time.Second}, + } { + if _, err := NewServer(config, testDependencies()); err == nil { + t.Fatalf("accepted config %+v", config) + } + } +} diff --git a/internal/fullsync/httpapi/simple_replay_test.go b/internal/fullsync/httpapi/simple_replay_test.go new file mode 100644 index 0000000..23d82c1 --- /dev/null +++ b/internal/fullsync/httpapi/simple_replay_test.go @@ -0,0 +1,59 @@ +package httpapi + +import ( + "bytes" + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/runtime" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +func TestSimpleReplayHTTPPrivateNetworkAcceptance(t *testing.T) { + redis := testutil.Redis(t) + store, _ := zset.New(redis, "sim-status:") + service, _ := runtime.NewSimpleReplay(store) + ctx := context.Background() + key, _ := store.RunMetaKey("source") + failed, _ := store.FailedKey("source") + _, _ = redis.Do(ctx, "HSET", key, "runId", "source", "status", "completed_with_failures", "producerDone", "1", "currentRound", "1") + record := zset.FailureRecord{TaskID: "task", CardID: 1, ICCID: "card", ChanKey: "ch", TempKey: "template"} + value, _ := json.Marshal(record) + _, _ = redis.Do(ctx, "HSET", failed, "task", string(value)) + deps := testDependencies() + deps.Simple = service + handler, err := NewHandler("", deps) + if err != nil { + t.Fatal(err) + } + req := httptest.NewRequest(http.MethodPost, "/internal/v1/full-sync-runs/source/replay", bytes.NewBufferString(`{"requestedBy":"42"}`)) + w := httptest.NewRecorder() + handler.ServeHTTP(w, req) + if w.Code != 202 { + t.Fatalf("accepted: %d %s", w.Code, w.Body.String()) + } + var receipt runtime.ReplayReceipt + if err := json.Unmarshal(w.Body.Bytes(), &receipt); err != nil || receipt.ReplayRunID == "" { + t.Fatalf("receipt: %+v %v", receipt, err) + } + deadline := time.Now().Add(3 * time.Second) + for time.Now().Before(deadline) { + w := httptest.NewRecorder() + req := httptest.NewRequest(http.MethodGet, "/internal/v1/full-sync-replays/"+receipt.ReplayRunID, nil) + handler.ServeHTTP(w, req) + if w.Code == 200 { + var status runtime.ReplayStatus + _ = json.Unmarshal(w.Body.Bytes(), &status) + if status.Status == "completed" && status.Enqueued == 1 { + return + } + } + time.Sleep(20 * time.Millisecond) + } + t.Fatal("replay HTTP never completed") +} diff --git a/internal/fullsync/runtime/catalog.go b/internal/fullsync/runtime/catalog.go new file mode 100644 index 0000000..52221c1 --- /dev/null +++ b/internal/fullsync/runtime/catalog.go @@ -0,0 +1,211 @@ +package runtime + +import ( + "context" + "encoding/json" + "errors" + "os" + "path/filepath" + "sort" + "strings" + "time" + + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" +) + +const ( + defaultRunListLimit = 50 + maxRunListLimit = 100 +) + +// Catalog aggregates the flow and sim-status Redis namespaces while keeping +// each run's storage protocol inside its existing Repository. +type Catalog struct { + repositories []*Repository +} + +func NewCatalog(repositories ...*Repository) (*Catalog, error) { + if len(repositories) == 0 { + return nil, errors.New("full-sync catalog requires repositories") + } + for _, repository := range repositories { + if repository == nil { + return nil, errors.New("full-sync catalog repository is nil") + } + } + return &Catalog{repositories: repositories}, nil +} + +func (c *Catalog) GetRun(ctx context.Context, runID string) (contract.RunView, error) { + if err := contract.ValidateID(runID); err != nil { + return contract.RunView{}, err + } + for _, repository := range c.repositories { + meta, err := repository.GetRunMeta(ctx, runID) + if err == nil { + rounds, roundErr := repository.GetRoundStats(ctx, runID) + if roundErr != nil { + return contract.RunView{}, roundErr + } + return contract.RunView{Source: contract.SourceLive, Run: meta, Rounds: rounds}, nil + } + if !errors.Is(err, contract.ErrNotFound) { + return contract.RunView{}, err + } + } + for _, repository := range c.repositories { + archive, err := repository.GetArchivedRun(ctx, runID) + if err == nil { + return contract.RunView{Source: contract.SourceArchive, Run: archive.Run, Rounds: archive.Rounds}, nil + } + if !errors.Is(err, contract.ErrNotFound) { + return contract.RunView{}, err + } + } + return contract.RunView{}, contract.ErrNotFound +} + +func (c *Catalog) ListRuns(ctx context.Context, filter contract.RunListFilter) (contract.RunList, error) { + if filter.Limit == 0 { + filter.Limit = defaultRunListLimit + } + if filter.Limit < 1 || filter.Limit > maxRunListLimit { + return contract.RunList{}, contract.ErrInvalidArgument + } + byID := make(map[string]contract.RunView) + for _, repository := range c.repositories { + active, err := repository.store.ActiveRuns(ctx) + if err != nil { + return contract.RunList{}, err + } + for _, runID := range active { + view, getErr := repository.liveRun(ctx, runID) + if errors.Is(getErr, contract.ErrNotFound) { + continue + } + if getErr != nil { + return contract.RunList{}, getErr + } + byID[runID] = view + } + } + if err := c.listArchives(ctx, byID); err != nil { + return contract.RunList{}, err + } + items := make([]contract.RunView, 0, len(byID)) + for _, item := range byID { + kind := contract.SyncKindForOperation(item.Run.Operation) + if kind == "" || (filter.SyncKind != "" && kind != filter.SyncKind) { + continue + } + items = append(items, item) + } + sort.SliceStable(items, func(i, j int) bool { + return runSortTime(items[i].Run).After(runSortTime(items[j].Run)) + }) + total := len(items) + if len(items) > filter.Limit { + items = items[:filter.Limit] + } + if items == nil { + items = []contract.RunView{} + } + return contract.RunList{Items: items, Total: total}, nil +} + +func (c *Catalog) ListFailures(ctx context.Context, runID string, page contract.PageRequest) (contract.FailurePage, error) { + if err := contract.ValidateID(runID); err != nil { + return contract.FailurePage{}, err + } + for _, repository := range c.repositories { + meta, err := repository.GetRunMeta(ctx, runID) + if err == nil { + result, listErr := repository.ListFailures(ctx, runID, page) + if listErr != nil { + return contract.FailurePage{}, listErr + } + if meta.FailedCount > 0 && len(result.Items) == 0 && result.NextCursor == "" { + failedKey, _ := repository.store.FailedKey(runID) + size, sizeErr := repository.redis.Do(ctx, "HLEN", failedKey) + if sizeErr != nil { + return contract.FailurePage{}, sizeErr + } + if size.Int64() == 0 { + return contract.FailurePage{}, contract.ErrFailureDetailsUnavailable + } + } + return result, nil + } else if !errors.Is(err, contract.ErrNotFound) { + return contract.FailurePage{}, err + } + } + view, err := c.GetRun(ctx, runID) + if err != nil { + return contract.FailurePage{}, err + } + if view.Run.FailedCount == 0 { + return contract.FailurePage{Items: []contract.Failure{}}, nil + } + for _, repository := range c.repositories { + failedKey, _ := repository.store.FailedKey(runID) + size, sizeErr := repository.redis.Do(ctx, "HLEN", failedKey) + if sizeErr != nil { + return contract.FailurePage{}, sizeErr + } + if size.Int64() > 0 { + return repository.ListFailures(ctx, runID, page) + } + } + return contract.FailurePage{}, contract.ErrFailureDetailsUnavailable +} + +func (c *Catalog) listArchives(ctx context.Context, byID map[string]contract.RunView) error { + root := c.repositories[0].root + if strings.TrimSpace(root) == "" { + return nil + } + matches, err := filepath.Glob(filepath.Join(root, "run-*.json")) + if err != nil { + return err + } + for _, path := range matches { + data, readErr := os.ReadFile(path) + if readErr != nil { + g.Log().Warningf(ctx, "event=full_sync_list_archive status=skip path=%s error=%q", path, readErr.Error()) + continue + } + var archive contract.RunArchive + if decodeErr := json.Unmarshal(data, &archive); decodeErr != nil || archive.Run.RunID == "" || filepath.Base(path) != "run-"+archive.Run.RunID+".json" { + g.Log().Warningf(ctx, "event=full_sync_list_archive status=skip path=%s error=%v", path, decodeErr) + continue + } + if _, exists := byID[archive.Run.RunID]; !exists { + byID[archive.Run.RunID] = contract.RunView{Source: contract.SourceArchive, Run: archive.Run, Rounds: archive.Rounds} + } + } + return nil +} + +func (r *Repository) liveRun(ctx context.Context, runID string) (contract.RunView, error) { + meta, err := r.GetRunMeta(ctx, runID) + if err != nil { + return contract.RunView{}, err + } + rounds, err := r.GetRoundStats(ctx, runID) + if err != nil { + return contract.RunView{}, err + } + return contract.RunView{Source: contract.SourceLive, Run: meta, Rounds: rounds}, nil +} + +func runSortTime(run contract.RunMeta) (value time.Time) { + if run.StartedAt != nil && !run.StartedAt.IsZero() { + return *run.StartedAt + } + if !run.CompletedAt.IsZero() { + return run.CompletedAt + } + return run.CreatedAt +} diff --git a/internal/fullsync/runtime/catalog_test.go b/internal/fullsync/runtime/catalog_test.go new file mode 100644 index 0000000..7a36e2f --- /dev/null +++ b/internal/fullsync/runtime/catalog_test.go @@ -0,0 +1,123 @@ +package runtime + +import ( + "context" + "encoding/json" + "os" + "path/filepath" + "testing" + "time" + + "github.com/gogf/gf/v2/container/gvar" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +type catalogRedisStub struct { + responses []*gvar.Var + calls []string +} + +func (s *catalogRedisStub) Do(_ context.Context, command string, _ ...any) (*gvar.Var, error) { + s.calls = append(s.calls, command) + if len(s.responses) == 0 { + return gvar.New(nil), nil + } + response := s.responses[0] + s.responses = s.responses[1:] + return response, nil +} + +func (s *catalogRedisStub) Eval(context.Context, string, int64, []string, []any) (*gvar.Var, error) { + return gvar.New(int64(1)), nil +} + +func TestCatalogListRunsMergesLiveAndArchive(t *testing.T) { + root := t.TempDir() + now := time.Date(2026, 9, 28, 1, 0, 0, 0, time.UTC) + archive := contract.RunArchive{Version: 1, Run: contract.RunMeta{ + RunID: "archived", Operation: "sim_status_queue_full_sync", Status: contract.RunCompleted, + CreatedAt: now.Add(-time.Hour), CompletedAt: now.Add(-time.Minute), + }} + data, err := json.Marshal(archive) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(root, "run-archived.json"), data, 0o644); err != nil { + t.Fatal(err) + } + + metaResponse := gvar.New([]any{"runId", "live", "operation", "current_month_usage_full_sync", "status", "running", "currentRound", "1", "createdAt", now.Format(time.RFC3339Nano), "roundStats", "[]"}) + redis := &catalogRedisStub{responses: []*gvar.Var{ + gvar.New([]any{"live"}), metaResponse, metaResponse, + gvar.New([]any{"live"}), metaResponse, metaResponse, + }} + store, err := zset.New(redis, "flow:") + if err != nil { + t.Fatal(err) + } + repository, err := New(store, root) + if err != nil { + t.Fatal(err) + } + catalog, err := NewCatalog(repository) + if err != nil { + t.Fatal(err) + } + + result, err := catalog.ListRuns(context.Background(), contract.RunListFilter{Limit: 10}) + if err != nil { + t.Fatal(err) + } + if result.Total != 2 || len(result.Items) != 2 { + t.Fatalf("result = %+v", result) + } + if result.Items[0].Run.RunID != "live" || result.Items[1].Run.RunID != "archived" { + t.Fatalf("items = %+v", result.Items) + } + filtered, err := catalog.ListRuns(context.Background(), contract.RunListFilter{SyncKind: contract.SyncKindTrafficUsage, Limit: 10}) + if err != nil { + t.Fatal(err) + } + if filtered.Total != 1 || filtered.Items[0].Run.RunID != "live" { + t.Fatalf("filtered = %+v", filtered) + } + view, err := catalog.GetRun(context.Background(), "archived") + if err != nil || view.Source != contract.SourceArchive || view.Run.RunID != "archived" { + t.Fatalf("archived view = %+v, err = %v", view, err) + } +} + +func TestCatalogPrefersLiveAndSkipsBrokenArchive(t *testing.T) { + root := t.TempDir() + for name, payload := range map[string]string{ + "run-live.json": `{"version":1,"run":{"runId":"live","operation":"sim_status_queue_full_sync","status":"completed"}}`, + "run-broken.json": `{not-json}`, + } { + if err := os.WriteFile(filepath.Join(root, name), []byte(payload), 0o644); err != nil { + t.Fatal(err) + } + } + redis := &catalogRedisStub{responses: []*gvar.Var{ + gvar.New([]any{"live"}), + gvar.New([]any{"runId", "live", "operation", "sim_status_queue_full_sync", "status", "running"}), + gvar.New([]any{"runId", "live", "operation", "sim_status_queue_full_sync", "status", "running"}), + }} + store, err := zset.New(redis, "sim-status:") + if err != nil { + t.Fatal(err) + } + repository, err := New(store, root) + if err != nil { + t.Fatal(err) + } + catalog, err := NewCatalog(repository) + if err != nil { + t.Fatal(err) + } + result, err := catalog.ListRuns(context.Background(), contract.RunListFilter{Limit: 10}) + if err != nil || result.Total != 1 || result.Items[0].Source != contract.SourceLive || result.Items[0].Run.Status != contract.RunRunning { + t.Fatalf("result = %+v, err = %v", result, err) + } +} diff --git a/internal/fullsync/runtime/runtime.go b/internal/fullsync/runtime/runtime.go new file mode 100644 index 0000000..92efb30 --- /dev/null +++ b/internal/fullsync/runtime/runtime.go @@ -0,0 +1,497 @@ +// Package runtime provides the small concrete storage adapters used by the +// full-sync HTTP and archive services. It intentionally keeps queue protocol +// transitions in queue/zset and only reads or records durable metadata here. +package runtime + +import ( + "context" + "crypto/sha256" + "encoding/base64" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "os" + "path/filepath" + "strconv" + "strings" + "time" + + "github.com/gogf/gf/v2/container/gvar" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +type Repository struct { + store *zset.Store + redis zset.RedisClient + keys contract.Keys + root string +} + +func (r *Repository) GetRoundStats(ctx context.Context, runID string) ([]contract.RoundStats, error) { + stats, err := r.store.GetRoundStats(ctx, runID) + if err != nil { + return nil, wrapNotFound(err) + } + out := make([]contract.RoundStats, 0, len(stats)) + for _, stat := range stats { + out = append(out, contract.RoundStats{ + RunID: stat.RunID, + Round: stat.Round, + EnqueuedCount: stat.EnqueuedCount, + ActiveCount: stat.ActiveCount, + NextRoundCount: stat.NextRoundCount, + SuccessCount: stat.SuccessCount, + FailedCount: stat.FailedCount, + UpstreamAttemptCount: stat.UpstreamAttemptCount, + BatchCount: stat.BatchCount, + StartedAt: stat.StartedAt, + CompletedAt: stat.CompletedAt, + }) + } + return out, nil +} + +func (r *Repository) RequestReplay(ctx context.Context, req contract.ReplayRequest) (contract.ReplayAccepted, error) { + if err := req.Validate(); err != nil { + return contract.ReplayAccepted{}, err + } + fp, err := req.Fingerprint() + if err != nil { + return contract.ReplayAccepted{}, err + } + key, err := r.keys.ReplayIdempotencyKey(req.SourceRunID, req.IdempotencyKey) + if err != nil { + return contract.ReplayAccepted{}, err + } + actionID := "replay-" + fp[:24] + actionKey, _ := r.keys.ReplayActionKey(actionID) + targetID := "replay-run-" + fp[:24] + now := time.Now().UTC() + action := contract.ReplayAction{ActionID: actionID, SourceRunID: req.SourceRunID, TargetRunID: targetID, RequestedBy: req.RequestedBy, IdempotencyKey: req.IdempotencyKey, Fingerprint: fp, Status: contract.ActionAccepted, ReplayCycle: 1, Progress: contract.ReplayProgress{}, CreatedAt: now, UpdatedAt: now} + b, _ := json.Marshal(action) + // Reserve idempotency first. A repeat reads the original action; a changed + // request under the same key is rejected by fingerprint comparison. + set, e := r.redis.Do(ctx, "SET", key, actionID, "NX", "EX", 604800) + if e != nil { + return contract.ReplayAccepted{}, e + } + if set.String() == "" || strings.EqualFold(set.String(), "nil") { + reserved, e := r.redis.Do(ctx, "GET", key) + if e != nil { + return contract.ReplayAccepted{}, e + } + reservedID := reserved.String() + old, e := r.GetReplayAction(ctx, reservedID) + if e != nil { + return contract.ReplayAccepted{}, e + } + if old.Fingerprint != fp { + return contract.ReplayAccepted{}, contract.ErrConflict + } + u, _ := contract.ActionStatusURL(old.ActionID) + return contract.ReplayAccepted{ActionID: old.ActionID, TargetRunID: old.TargetRunID, Status: old.Status, StatusURL: u}, nil + } + if _, e = r.redis.Do(ctx, "SET", actionKey, string(b), "EX", 604800); e != nil { + return contract.ReplayAccepted{}, e + } + _, _ = r.redis.Do(ctx, "SADD", r.keys.PendingReplayActionsKey(), actionID) + go r.executeReplay(actionID) + u, _ := contract.ActionStatusURL(actionID) + return contract.ReplayAccepted{ActionID: actionID, TargetRunID: targetID, Status: action.Status, StatusURL: u}, nil +} + +// executeReplay is deliberately small: it snapshots the source failure hash, +// enqueues equivalent tasks into one target run, then removes only entries +// that were accepted. Redis action JSON is the durable retry checkpoint. +func (r *Repository) executeReplay(actionID string) { + ctx := context.Background() + action, err := r.GetReplayAction(ctx, actionID) + if err != nil { + return + } + action.Status = contract.ActionProcessing + action.UpdatedAt = time.Now().UTC() + _ = r.saveAction(ctx, action) + source, err := r.GetRunMeta(ctx, action.SourceRunID) + if err != nil { + action.Error = err.Error() + action.Status = contract.ActionNeedsAttention + action.UpdatedAt = time.Now().UTC() + _ = r.saveAction(ctx, action) + return + } + target := zset.RunMeta{RunID: action.TargetRunID, Operation: source.Operation, QueryMonth: source.QueryMonth, Status: "producing", CurrentRound: 1, CreatedAt: time.Now().UTC(), SyncType: string(contract.SyncReplay), SourceRunID: action.SourceRunID, ReplayCycle: action.ReplayCycle, TriggerSource: "replay"} + if err := r.store.CreateRun(ctx, target); err != nil { + if _, getErr := r.store.GetRunMeta(ctx, target.RunID); getErr != nil { + action.Error = err.Error() + action.Status = contract.ActionNeedsAttention + action.UpdatedAt = time.Now().UTC() + _ = r.saveAction(ctx, action) + return + } + } + fkey, _ := r.store.FailedKey(action.SourceRunID) + var records []struct { + field string + rec zset.FailureRecord + } + var cursor uint64 + for { + v, e := r.redis.Do(ctx, "HSCAN", fkey, cursor, "COUNT", 500) + if e != nil { + action.Error = e.Error() + action.Status = contract.ActionNeedsAttention + action.UpdatedAt = time.Now().UTC() + _ = r.saveAction(ctx, action) + return + } + cursor, pairs, e := parseScanResult(v) + if e != nil { + action.Error = e.Error() + action.Status = contract.ActionNeedsAttention + action.UpdatedAt = time.Now().UTC() + _ = r.saveAction(ctx, action) + return + } + for i := 0; i < len(pairs); i += 2 { + var rec zset.FailureRecord + if json.Unmarshal([]byte(pairs[i+1].String()), &rec) == nil { + records = append(records, struct { + field string + rec zset.FailureRecord + }{pairs[i].String(), rec}) + } + } + if cursor == 0 { + break + } + } + groups := map[string][]zset.Task{} + for _, item := range records { + rec := item.rec + chanKey := rec.ChanKey + if chanKey == "" { + chanKey = "default" + } + groups[chanKey] = append(groups[chanKey], zset.Task{TaskID: action.TargetRunID + ":" + rec.TaskID, RunID: action.TargetRunID, Operation: rec.Operation, QueryMonth: rec.QueryMonth, ChanKey: chanKey, TempKey: rec.TempKey, CardID: rec.CardID, ICCID: rec.ICCID, MSISDN: rec.MSISDN, IMSI: rec.IMSI}) + } + for chanKey, tasks := range groups { + if _, err := r.store.EnqueueRunBatch(ctx, action.TargetRunID, chanKey, tasks, time.Now()); err != nil { + action.Error = err.Error() + action.Status = contract.ActionNeedsAttention + action.UpdatedAt = time.Now().UTC() + _ = r.saveAction(ctx, action) + return + } + } + for _, item := range records { + _, _ = r.redis.Do(ctx, "HDEL", fkey, item.field) + } + _ = r.store.MarkProducerDone(ctx, action.TargetRunID) + action.Progress = contract.ReplayProgress{DiscoveredCount: int64(len(records)), EnqueuedCount: int64(len(records)), CleanedCount: int64(len(records)), FailureCount: int64(len(records))} + action.Status = contract.ActionCompleted + action.UpdatedAt = time.Now().UTC() + completed := action.UpdatedAt + action.CompletedAt = &completed + digest := sha256.Sum256([]byte(action.ActionID + action.TargetRunID)) + action.Receipt = &contract.ArchiveReceipt{Version: 1, Path: "replay://" + action.ActionID, Checksum: hex.EncodeToString(digest[:]), ArchivedAt: completed} + _ = r.saveAction(ctx, action) +} + +func (r *Repository) saveAction(ctx context.Context, action contract.ReplayAction) error { + b, err := json.Marshal(action) + if err != nil { + return err + } + key, err := r.keys.ReplayActionKey(action.ActionID) + if err != nil { + return err + } + _, err = r.redis.Do(ctx, "SET", key, string(b), "EX", 604800) + return err +} + +func (r *Repository) GetReplayAction(ctx context.Context, actionID string) (contract.ReplayAction, error) { + if err := contract.ValidateID(actionID); err != nil { + return contract.ReplayAction{}, err + } + key, _ := r.keys.ReplayActionKey(actionID) + v, err := r.redis.Do(ctx, "GET", key) + if err != nil { + return contract.ReplayAction{}, err + } + if v == nil || v.IsNil() || strings.TrimSpace(v.String()) == "" { + return contract.ReplayAction{}, contract.ErrNotFound + } + var action contract.ReplayAction + if err := json.Unmarshal([]byte(v.String()), &action); err != nil { + return contract.ReplayAction{}, err + } + return action, nil +} + +func New(store *zset.Store, archiveRoot string) (*Repository, error) { + if store == nil { + return nil, errors.New("full-sync runtime store is nil") + } + keys, err := contract.NewKeys(store.Prefix()) + if err != nil { + return nil, err + } + return &Repository{store: store, redis: store.RedisClient(), keys: keys, root: archiveRoot}, nil +} + +func (r *Repository) GetRunMeta(ctx context.Context, runID string) (contract.RunMeta, error) { + m, err := r.store.GetRunMeta(ctx, runID) + if err != nil { + return contract.RunMeta{}, wrapNotFound(err) + } + return toContractMeta(m), nil +} + +func (r *Repository) GetArchivedRun(ctx context.Context, runID string) (contract.RunArchive, error) { + if err := contract.ValidateID(runID); err != nil { + return contract.RunArchive{}, err + } + if strings.TrimSpace(r.root) == "" { + return contract.RunArchive{}, contract.ErrNotFound + } + b, err := os.ReadFile(filepath.Join(r.root, "run-"+runID+".json")) + if err != nil { + if errors.Is(err, os.ErrNotExist) { + return contract.RunArchive{}, contract.ErrNotFound + } + return contract.RunArchive{}, err + } + var archive contract.RunArchive + if err := json.Unmarshal(b, &archive); err != nil { + return contract.RunArchive{}, fmt.Errorf("decode run archive: %w", err) + } + if archive.Run.RunID != runID { + return contract.RunArchive{}, fmt.Errorf("archive run ID does not match filename") + } + return archive, nil +} + +func (r *Repository) ListFailures(ctx context.Context, runID string, page contract.PageRequest) (contract.FailurePage, error) { + p, err := page.Normalized() + if err != nil { + return contract.FailurePage{}, err + } + key, err := r.store.FailedKey(runID) + if err != nil { + return contract.FailurePage{}, err + } + start := uint64(0) + if p.Cursor != "" { + start, err = decodeCursor(p.Cursor, runID) + if err != nil { + return contract.FailurePage{}, err + } + } + var items []contract.Failure + var next uint64 + for { + v, e := r.redis.Do(ctx, "HSCAN", key, start, "COUNT", p.Limit) + if e != nil { + return contract.FailurePage{}, e + } + next, pairs, e := parseScanResult(v) + if e != nil { + return contract.FailurePage{}, e + } + for i := 1; i < len(pairs); i += 2 { + var rec zset.FailureRecord + if json.Unmarshal([]byte(pairs[i].String()), &rec) == nil { + items = append(items, toFailure(rec)) + if len(items) >= p.Limit { + break + } + } + } + if len(items) >= p.Limit || next == 0 { + break + } + start = next + } + result := contract.FailurePage{Items: items} + if next != 0 { + result.NextCursor = encodeCursor(runID, next) + } + if result.Items == nil { + result.Items = []contract.Failure{} + } + return result, nil +} + +func (r *Repository) ListArchiveWork(ctx context.Context, page contract.PageRequest) (contract.ArchiveWorkPage, error) { + p, err := page.Normalized() + if err != nil { + return contract.ArchiveWorkPage{}, err + } + start := uint64(0) + if p.Cursor != "" { + start, err = decodeCursor(p.Cursor, "archive") + if err != nil { + return contract.ArchiveWorkPage{}, err + } + } + v, err := r.redis.Do(ctx, "ZSCAN", r.keys.PendingArchivesKey(), start, "COUNT", p.Limit) + if err != nil { + return contract.ArchiveWorkPage{}, err + } + out := contract.ArchiveWorkPage{Items: []contract.ArchiveWork{}} + next, pairs, err := parseScanResult(v) + if err != nil { + return contract.ArchiveWorkPage{}, err + } + for i := 0; i < len(pairs); i += 2 { + out.Items = append(out.Items, contract.ArchiveWork{RunID: pairs[i].String(), Version: pairs[i+1].String()}) + } + if next != 0 { + out.NextCursor = encodeCursor("archive", next) + } + return out, nil +} + +func parseScanResult(value *gvar.Var) (uint64, []*gvar.Var, error) { + if value == nil { + return 0, nil, errors.New("redis scan returned nil response") + } + values := value.Vars() + if len(values) != 2 { + return 0, nil, fmt.Errorf("unexpected Redis scan response length: %d", len(values)) + } + pairs := values[1].Vars() + if len(pairs)%2 != 0 { + return 0, nil, fmt.Errorf("redis scan returned odd number of entry values: %d", len(pairs)) + } + return uint64(values[0].Int64()), pairs, nil +} + +func (r *Repository) LoadArchiveWork(ctx context.Context, work contract.ArchiveWork) (contract.RunArchive, error) { + if archived, err := r.GetArchivedRun(ctx, work.RunID); err == nil { + return archived, nil + } + meta, err := r.GetRunMeta(ctx, work.RunID) + if err != nil { + return contract.RunArchive{}, err + } + rounds, err := r.GetRoundStats(ctx, work.RunID) + if err != nil { + g.Log().Warningf(ctx, "event=full_sync_archive_rounds status=failed runId=%s stage=read error=%q", work.RunID, err.Error()) + return contract.RunArchive{Version: contract.ArchiveSchemaVersion, Run: meta, Rounds: []contract.RoundStats{}}, nil + } + if err := validateRoundAccounting(meta, rounds); err != nil { + g.Log().Warningf(ctx, "event=full_sync_archive_rounds status=failed runId=%s stage=validate error=%q", work.RunID, err.Error()) + return contract.RunArchive{Version: contract.ArchiveSchemaVersion, Run: meta, Rounds: []contract.RoundStats{}}, nil + } + return contract.RunArchive{Version: contract.ArchiveSchemaVersion, Run: meta, Rounds: rounds}, nil +} + +func validateRoundAccounting(meta contract.RunMeta, rounds []contract.RoundStats) error { + if len(rounds) == 0 { + return nil + } + var success, failed int64 + seen := make(map[int]struct{}, len(rounds)) + for _, round := range rounds { + if round.RunID != meta.RunID { + return fmt.Errorf("round %d belongs to run %q", round.Round, round.RunID) + } + if _, ok := seen[round.Round]; ok { + return fmt.Errorf("duplicate round %d", round.Round) + } + seen[round.Round] = struct{}{} + if round.EnqueuedCount != round.SuccessCount+round.NextRoundCount+round.FailedCount { + return fmt.Errorf("round %d counters do not close", round.Round) + } + success += round.SuccessCount + failed += round.FailedCount + } + if success != meta.SuccessCount || failed != meta.FailedCount { + return fmt.Errorf("run counters do not match round totals: success=%d/%d failed=%d/%d", success, meta.SuccessCount, failed, meta.FailedCount) + } + return nil +} + +func (r *Repository) RegisterArchiveWork(ctx context.Context, work contract.ArchiveWork) error { + if err := contract.ValidateID(work.RunID); err != nil { + return err + } + if strings.TrimSpace(work.Version) == "" { + return contract.ErrInvalidArgument + } + _, err := r.redis.Do(ctx, "ZADD", r.keys.PendingArchivesKey(), work.Version, work.RunID) + return err +} + +func (r *Repository) RegisterArchive(ctx context.Context, work contract.ArchiveWork, receipt contract.ArchiveReceipt) error { + if err := receipt.Validate(); err != nil { + return err + } + key, err := r.keys.ArchiveStateKey(work.RunID) + if err != nil { + return err + } + _, err = r.redis.Do(ctx, "HSET", key, "version", receipt.Version, "path", receipt.Path, "checksum", receipt.Checksum, "archivedAt", receipt.ArchivedAt.UTC().Format(time.RFC3339Nano)) + if err != nil { + return err + } + _, err = r.redis.Do(ctx, "ZREM", r.keys.PendingArchivesKey(), work.RunID) + return err +} + +func (r *Repository) RegisterArchiveFailure(ctx context.Context, work contract.ArchiveWork, message string) error { + _, err := r.redis.Do(ctx, "ZADD", r.keys.PendingArchivesKey(), time.Now().UnixMilli(), work.RunID) + return err +} + +func toContractMeta(m zset.RunMeta) contract.RunMeta { + syncType := contract.SyncType(m.SyncType) + if syncType == "" { + syncType = contract.SyncNormal + } + return contract.RunMeta{SupplyObservation: m.SupplyObservation, RunID: m.RunID, Operation: m.Operation, QueryMonth: m.QueryMonth, Status: contract.RunStatus(m.Status), ProducerDone: m.ProducerDone, CurrentRound: m.CurrentRound, ActiveCount: m.ActiveCount, InflightCount: m.InflightCount, NextRoundCount: m.NextRoundCount, SuccessCount: m.SuccessCount, FailedCount: m.FailedCount, CreatedAt: m.CreatedAt, ProducerCompletedAt: m.ProducerCompletedAt, CompletedAt: m.CompletedAt, SyncType: syncType, SourceRunID: m.SourceRunID, ReplayCycle: m.ReplayCycle, TriggerSource: m.TriggerSource, ScannedCount: m.ScannedCount, EnqueuedCount: m.EnqueuedCount, UpstreamAttemptCount: m.UpstreamAttemptCount, BatchCount: m.BatchCount, MaxRound: m.MaxRound, PendingFailureCount: m.PendingFailureCount, ArchiveStatus: contract.ArchiveStatus(m.ArchiveStatus), ArchiveVersion: m.ArchiveVersion, ArchivePath: m.ArchivePath, ArchiveChecksum: m.ArchiveChecksum, ReplayDisposition: contract.ReplayDisposition(m.ReplayDisposition), ScopeChannels: m.ScopeChannels, ScopeDigest: m.ScopeDigest, ScopeChannelsSample: m.ScopeChannelsSample, StartedAt: m.StartedAt, ScanCompletedAt: m.ScanCompletedAt, ArchivedAt: m.ArchivedAt} +} +func toFailure(f zset.FailureRecord) contract.Failure { + return contract.Failure{TaskID: f.TaskID, CardID: f.CardID, ErrorCategory: contract.NormalizeErrorCategory(f.FailureType), ErrorMessage: contract.BoundedErrorMessage(f.Message), FailedAt: f.LastFailedAt, Round: f.Round} +} +func wrapNotFound(err error) error { + if strings.Contains(strings.ToLower(err.Error()), "not found") { + return contract.ErrNotFound + } + return err +} +func encodeCursor(scope string, n uint64) contract.Cursor { + return contract.Cursor(base64.RawURLEncoding.EncodeToString([]byte(scope + ":" + strconv.FormatUint(n, 10)))) +} +func decodeCursor(c contract.Cursor, scope string) (uint64, error) { + b, e := base64.RawURLEncoding.DecodeString(string(c)) + if e != nil { + return 0, contract.ErrInvalidArgument + } + p := strings.Split(string(b), ":") + if len(p) != 2 || p[0] != scope { + return 0, contract.ErrInvalidArgument + } + n, e := strconv.ParseUint(p[1], 10, 64) + if e != nil { + return 0, contract.ErrInvalidArgument + } + return n, nil +} + +var _ contract.LiveRunReader = (*Repository)(nil) +var _ contract.ArchivedRunReader = (*Repository)(nil) +var _ contract.FailurePager = (*Repository)(nil) +var _ contract.ArchiveWorkDiscovery = (*Repository)(nil) +var _ contract.ArchiveWorkLoader = (*Repository)(nil) +var _ contract.ArchiveWorkRegistrar = (*Repository)(nil) +var _ contract.ArchiveRegistrar = (*Repository)(nil) +var _ contract.ReplayRequester = (*Repository)(nil) +var _ contract.ActionReader = (*Repository)(nil) diff --git a/internal/fullsync/runtime/runtime_test.go b/internal/fullsync/runtime/runtime_test.go new file mode 100644 index 0000000..459bd41 --- /dev/null +++ b/internal/fullsync/runtime/runtime_test.go @@ -0,0 +1,143 @@ +package runtime + +import ( + "context" + "encoding/json" + "testing" + "time" + + "github.com/gogf/gf/v2/container/gvar" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +type runtimeRedisStub struct { + responses []*gvar.Var +} + +func (s *runtimeRedisStub) Do(context.Context, string, ...any) (*gvar.Var, error) { + if len(s.responses) == 0 { + return gvar.New(nil), nil + } + response := s.responses[0] + s.responses = s.responses[1:] + return response, nil +} + +func (s *runtimeRedisStub) Eval(context.Context, string, int64, []string, []any) (*gvar.Var, error) { + return gvar.New(int64(1)), nil +} + +func newRuntimeRepository(t *testing.T, responses ...*gvar.Var) *Repository { + t.Helper() + store, err := zset.New(&runtimeRedisStub{responses: responses}, "flow:") + if err != nil { + t.Fatal(err) + } + repository, err := New(store, t.TempDir()) + if err != nil { + t.Fatal(err) + } + return repository +} + +func TestListArchiveWorkParsesNestedRedisScanResult(t *testing.T) { + repository := newRuntimeRepository(t, gvar.New([]any{ + "0", + []any{"run-1", "17", "run-2", "18"}, + })) + + page, err := repository.ListArchiveWork(context.Background(), contract.PageRequest{Limit: 10}) + if err != nil { + t.Fatal(err) + } + if len(page.Items) != 2 { + t.Fatalf("archive work count = %d, want 2", len(page.Items)) + } + if page.Items[0].RunID != "run-1" || page.Items[0].Version != "17" { + t.Fatalf("first archive work = %+v", page.Items[0]) + } + if page.Items[1].RunID != "run-2" || page.Items[1].Version != "18" { + t.Fatalf("second archive work = %+v", page.Items[1]) + } +} + +func TestGetRoundStatsReadsRunHashJSON(t *testing.T) { + raw, err := json.Marshal([]zset.RoundStats{{RunID: "run-1", Round: 2, EnqueuedCount: 3, SuccessCount: 2, FailedCount: 1}}) + if err != nil { + t.Fatal(err) + } + repository := newRuntimeRepository(t, gvar.New([]any{ + "runId", "run-1", "status", "completed", "roundStats", string(raw), + })) + + got, err := repository.GetRoundStats(context.Background(), "run-1") + if err != nil { + t.Fatal(err) + } + if len(got) != 1 || got[0].Round != 2 || got[0].SuccessCount != 2 || got[0].FailedCount != 1 { + t.Fatalf("unexpected round stats: %+v", got) + } +} + +func TestValidateRoundAccountingRejectsUnbalancedRound(t *testing.T) { + err := validateRoundAccounting(contract.RunMeta{RunID: "run-1", SuccessCount: 1, FailedCount: 0}, []contract.RoundStats{{RunID: "run-1", Round: 1, EnqueuedCount: 2, SuccessCount: 1}}) + if err == nil { + t.Fatal("expected unbalanced round error") + } +} + +func TestListArchiveWorkHandlesEmptyRedisScanResult(t *testing.T) { + repository := newRuntimeRepository(t, gvar.New([]any{"0", []any{}})) + + page, err := repository.ListArchiveWork(context.Background(), contract.PageRequest{Limit: 10}) + if err != nil { + t.Fatal(err) + } + if len(page.Items) != 0 { + t.Fatalf("archive work = %+v, want empty", page.Items) + } +} + +func TestListFailuresParsesNestedRedisScanResult(t *testing.T) { + failedAt := time.Date(2026, 9, 21, 6, 0, 0, 0, time.UTC) + record := zset.FailureRecord{ + TaskID: "task-1", + CardID: 42, + FailureType: "business", + Message: "upstream rejected request", + Round: 3, + LastFailedAt: failedAt, + } + payload, err := json.Marshal(record) + if err != nil { + t.Fatal(err) + } + repository := newRuntimeRepository(t, gvar.New([]any{ + "0", + []any{"task-1", string(payload)}, + })) + + page, err := repository.ListFailures(context.Background(), "run-1", contract.PageRequest{Limit: 10}) + if err != nil { + t.Fatal(err) + } + if len(page.Items) != 1 { + t.Fatalf("failure count = %d, want 1", len(page.Items)) + } + if page.Items[0].TaskID != "task-1" || page.Items[0].CardID != 42 || page.Items[0].Round != 3 { + t.Fatalf("failure = %+v", page.Items[0]) + } +} + +func TestListArchiveWorkRejectsOddRedisScanEntries(t *testing.T) { + repository := newRuntimeRepository(t, gvar.New([]any{ + "0", + []any{"run-without-score"}, + })) + + if _, err := repository.ListArchiveWork(context.Background(), contract.PageRequest{Limit: 10}); err == nil { + t.Fatal("ListArchiveWork() error = nil, want malformed response error") + } +} diff --git a/internal/fullsync/runtime/simple_replay.go b/internal/fullsync/runtime/simple_replay.go new file mode 100644 index 0000000..ed08400 --- /dev/null +++ b/internal/fullsync/runtime/simple_replay.go @@ -0,0 +1,474 @@ +package runtime + +import ( + "context" + "crypto/rand" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "strings" + "time" + + "github.com/gogf/gf/v2/container/gvar" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +// SimpleReplay hands every source failure to a separate Run. The source hash +// is the checkpoint: a record is removed only after the target's task-state +// confirms that EnqueueRunBatch accepted it. No cursor is retained across +// mutations of the source hash. +// +// The target's retry-round budget is taken from the caller when supplied and +// falls back to the configured default otherwise. It is persisted on the +// source reservation because a worker restart re-enters through reconcile(), +// which no longer has the caller's arguments. +type SimpleReplay struct { + stores []*zset.Store + redis zset.RedisClient + // defaultMaxRounds applies when a replay request does not specify one. + defaultMaxRounds int +} + +// Replay round bounds. MaxReplayRounds is a server-side guard, not a capacity +// limit: the observed chain of 10-09 needed 74 rounds to reach 98% recovery, +// so a low ceiling would cut off a legitimate convergence path. +// +// DefaultReplayMaxRounds is only the fallback for a request that omits a +// budget; callers may ask for more, up to MaxReplayRounds. +const ( + DefaultReplayMaxRounds = 3 + MaxReplayRounds = 100 + legacyReplayMaxRounds = 1 +) + +type ReplayReceipt struct { + ReplayRunID string `json:"replayRunId"` + Status string `json:"status"` +} + +type ReplayStatus struct { + ReplayRunID string `json:"replayRunId"` + SourceRunID string `json:"sourceRunId"` + Status string `json:"status"` + Remaining int64 `json:"remaining"` + Enqueued int64 `json:"enqueued"` + RunStatus string `json:"runStatus,omitempty"` + Error string `json:"error,omitempty"` +} + +var ErrNoFailures = errors.New("run has no failures to replay") + +func NewSimpleReplay(stores ...*zset.Store) (*SimpleReplay, error) { + return NewSimpleReplayWithDefault(DefaultReplayMaxRounds, stores...) +} + +// NewSimpleReplayWithDefault builds a replay service whose unspecified replay +// requests inherit defaultMaxRounds. A non-positive value falls back to +// DefaultReplayMaxRounds so a missing config key cannot disable the budget. +func NewSimpleReplayWithDefault(defaultMaxRounds int, stores ...*zset.Store) (*SimpleReplay, error) { + if len(stores) == 0 || stores[0] == nil { + return nil, fmt.Errorf("replay requires Redis stores") + } + for _, store := range stores { + if store == nil { + return nil, fmt.Errorf("replay store is nil") + } + } + if defaultMaxRounds <= 0 { + defaultMaxRounds = DefaultReplayMaxRounds + } + if defaultMaxRounds > MaxReplayRounds { + defaultMaxRounds = MaxReplayRounds + } + return &SimpleReplay{stores: stores, redis: stores[0].RedisClient(), defaultMaxRounds: defaultMaxRounds}, nil +} + +func (r *SimpleReplay) find(ctx context.Context, id string) (*zset.Store, zset.RunMeta, error) { + if err := contract.ValidateID(id); err != nil || len(id) > 128 { + return nil, zset.RunMeta{}, contract.ErrInvalidArgument + } + var found *zset.Store + var meta zset.RunMeta + for _, store := range r.stores { + key, _ := store.RunMetaKey(id) + v, err := r.redis.Do(ctx, "EXISTS", key) + if err != nil { + return nil, meta, err + } + if v.Int() == 0 { + continue + } + if found != nil { + return nil, meta, contract.ErrConflict + } + found = store + meta, err = store.GetRunMeta(ctx, id) + if err != nil { + return nil, meta, err + } + } + if found == nil { + return nil, meta, contract.ErrNotFound + } + return found, meta, nil +} + +// Request accepts a replay of the source run's failures. maxRounds == 0 means +// "use the configured default"; requests above MaxReplayRounds are rejected. +func (r *SimpleReplay) Request(ctx context.Context, sourceID, requestedBy string, maxRounds int) (ReplayReceipt, error) { + if len(requestedBy) == 0 || len(requestedBy) > 64 { + return ReplayReceipt{}, contract.ErrInvalidArgument + } + if maxRounds < 0 || maxRounds > MaxReplayRounds { + return ReplayReceipt{}, contract.ErrInvalidArgument + } + if maxRounds == 0 { + maxRounds = r.defaultMaxRounds + } + store, _, err := r.find(ctx, sourceID) + if err != nil { + return ReplayReceipt{}, err + } + metaKey, _ := store.RunMetaKey(sourceID) + failedKey, _ := store.FailedKey(sourceID) + var random [16]byte + if _, err := rand.Read(random[:]); err != nil { + return ReplayReceipt{}, err + } + targetID := "replay-" + hex.EncodeToString(random[:]) + // An index written before the reservation makes the pending source + // discoverable even if the process stops immediately after the Lua call. + if _, err = r.redis.Do(ctx, "SADD", store.Prefix()+"replay:pending", sourceID); err != nil { + return ReplayReceipt{}, err + } + v, err := r.redis.Eval(ctx, reserveReplay, 2, []string{metaKey, failedKey}, []any{targetID, requestedBy, maxRounds}) + if err != nil { + return ReplayReceipt{}, err + } + result := v.String() + // An unsuccessful reservation leaves an inert discovery entry. Never + // remove it here: a concurrent caller might have reserved the source + // after this script returned, and that reservation needs recovery. + switch result { + case "missing": + return ReplayReceipt{}, contract.ErrNotFound + case "running": + return ReplayReceipt{}, contract.ErrConflict + case "empty": + return ReplayReceipt{}, ErrNoFailures + } + if !strings.HasPrefix(result, "replay-") { + return ReplayReceipt{}, fmt.Errorf("unexpected replay reservation result") + } + go r.process(store, sourceID) + return ReplayReceipt{ReplayRunID: result, Status: "accepted"}, nil +} + +// Start reconciles pending sources after restarts and missed goroutine wakes. +// Every process may run it: the per-source Redis lease elects one writer. +func (r *SimpleReplay) Start(ctx context.Context) { + go func() { + ticker := time.NewTicker(10 * time.Second) + defer ticker.Stop() + for { + r.reconcile(ctx) + select { + case <-ctx.Done(): + return + case <-ticker.C: + } + } + }() +} + +func (r *SimpleReplay) reconcile(ctx context.Context) { + for _, store := range r.stores { + v, err := r.redis.Do(ctx, "SMEMBERS", store.Prefix()+"replay:pending") + if err != nil { + continue + } + for _, id := range v.Strings() { + if ctx.Err() != nil { + return + } + key, keyErr := store.RunMetaKey(id) + if keyErr != nil { + continue + } + reservation, getErr := r.redis.Do(ctx, "HGET", key, "replayTargetId") + // SADD happens before the reservation so even a crash during + // acceptance cannot lose work. Unreserved entries are inert. + if getErr != nil || reservation == nil || reservation.String() == "" { + continue + } + state, stateErr := r.redis.Do(ctx, "HGET", key, "replayState") + if stateErr != nil || state == nil || state.String() == "needs_attention" { + continue + } + go r.process(store, id) + } + } +} + +func (r *SimpleReplay) Status(ctx context.Context, targetID string) (ReplayStatus, error) { + store, target, err := r.find(ctx, targetID) + if err != nil { + // The target may not exist yet: locate its durable source reservation. + if !errors.Is(err, contract.ErrNotFound) { + return ReplayStatus{}, err + } + for _, candidate := range r.stores { + v, e := r.redis.Do(ctx, "SMEMBERS", candidate.Prefix()+"replay:pending") + if e != nil { + return ReplayStatus{}, e + } + for _, sourceID := range v.Strings() { + key, _ := candidate.RunMetaKey(sourceID) + value, e := r.redis.Do(ctx, "HGET", key, "replayTargetId") + if e == nil && value != nil && value.String() == targetID { + store = candidate + target.SourceRunID = sourceID + break + } + } + } + if store == nil { + return ReplayStatus{}, contract.ErrNotFound + } + } + if target.SourceRunID == "" { + return ReplayStatus{}, contract.ErrNotFound + } + sourceKey, _ := store.RunMetaKey(target.SourceRunID) + failureKey, _ := store.FailedKey(target.SourceRunID) + values, err := r.redis.Do(ctx, "HMGET", sourceKey, "replayTargetId", "replayState", "replayError", "replayEnqueued") + if err != nil { + return ReplayStatus{}, err + } + fields := values.Vars() + if len(fields) != 4 || fields[0].String() != targetID { + return ReplayStatus{}, contract.ErrNotFound + } + remaining, err := r.redis.Do(ctx, "HLEN", failureKey) + if err != nil { + return ReplayStatus{}, err + } + return ReplayStatus{ReplayRunID: targetID, SourceRunID: target.SourceRunID, Status: fields[1].String(), Error: fields[2].String(), Enqueued: fields[3].Int64(), Remaining: remaining.Int64(), RunStatus: target.Status}, nil +} + +func (r *SimpleReplay) process(store *zset.Store, sourceID string) { + ctx, cancel := context.WithTimeout(context.Background(), time.Hour) + defer cancel() + // Keep one process per source; batches are bounded so a short lease is + // renewed before it expires. A lost lease stops cleanup at the next batch. + lockKey := store.Prefix() + "{" + sourceID + "}:replay-lock" + var token [16]byte + if _, err := rand.Read(token[:]); err != nil { + return + } + owner := hex.EncodeToString(token[:]) + locked, err := store.AcquireLock(ctx, lockKey, owner, time.Minute) + if err != nil || !locked { + return + } + defer func() { _, _ = store.ReleaseLock(context.Background(), lockKey, owner) }() + if err := r.handoff(ctx, store, sourceID, lockKey, owner); err != nil { + // The time slice ended, not the replay: the reconciler will resume + // from remaining source records under a new lease. + if errors.Is(err, context.DeadlineExceeded) || errors.Is(err, context.Canceled) { + return + } + sourceKey, _ := store.RunMetaKey(sourceID) + // A stale owner must not overwrite a newer worker's status. + _, _ = r.redis.Eval(context.Background(), replayAttention, 2, []string{sourceKey, lockKey}, []any{owner}) + } +} + +func (r *SimpleReplay) handoff(ctx context.Context, store *zset.Store, sourceID, lockKey, owner string) error { + sourceKey, _ := store.RunMetaKey(sourceID) + failedKey, _ := store.FailedKey(sourceID) + meta, err := store.GetRunMeta(ctx, sourceID) + if err != nil { + return err + } + v, err := r.redis.Do(ctx, "HGET", sourceKey, "replayTargetId") + if err != nil { + return err + } + if v == nil || v.String() == "" { + return fmt.Errorf("replay reservation missing") + } + targetID := v.String() + // New reservations persist their budget. Reservations made before this + // field existed used one round; a restart must not apply today's default + // to those already accepted runs. + storedMaxRounds, err := r.redis.Do(ctx, "HGET", sourceKey, "replayMaxRound") + if err != nil { + return err + } + maxRounds := legacyReplayMaxRounds + if storedMaxRounds != nil && storedMaxRounds.String() != "" { + maxRounds = storedMaxRounds.Int() + } + if maxRounds <= 0 { + return fmt.Errorf("replay reservation has invalid round budget") + } + if maxRounds > MaxReplayRounds { + return fmt.Errorf("replay reservation exceeds round budget limit") + } + // CreateRun is idempotent for the main hash but writes extension fields + // again. Read and validate an existing target BEFORE CreateRun, otherwise + // recovery could silently replace its accepted maxRound. + existing, err := store.GetRunMeta(ctx, targetID) + if err != nil && !errors.Is(err, zset.ErrRunNotFound) { + return err + } + if err == nil { + if storedMaxRounds == nil || storedMaxRounds.String() == "" { + maxRounds = existing.MaxRound + } + if maxRounds <= 0 || maxRounds > MaxReplayRounds || existing.SourceRunID != sourceID || existing.SyncType != "replay" || existing.MaxRound != maxRounds || existing.Operation != meta.Operation || existing.QueryMonth != meta.QueryMonth { + return fmt.Errorf("replay target metadata is inconsistent") + } + } + target := zset.RunMeta{RunID: targetID, Operation: meta.Operation, QueryMonth: meta.QueryMonth, SyncType: "replay", SourceRunID: sourceID, ReplayCycle: meta.ReplayCycle + 1, TriggerSource: "replay", MaxRound: maxRounds} + if err := store.CreateRun(ctx, target); err != nil { + return err + } + actual, err := store.GetRunMeta(ctx, targetID) + if err != nil || actual.SourceRunID != sourceID || actual.SyncType != "replay" || actual.MaxRound != maxRounds || actual.Operation != meta.Operation || actual.QueryMonth != meta.QueryMonth { + return fmt.Errorf("replay target metadata is inconsistent") + } + // The target task-state is the admission receipt during recovery. It may + // not expire while source failures are still being transferred. + targetKey, _ := store.RunMetaKey(targetID) + stateKey, _ := store.TaskStateKey(targetID) + if !actual.ProducerDone { + if _, err := r.redis.Do(ctx, "PERSIST", targetKey); err != nil { + return err + } + if _, err := r.redis.Do(ctx, "PERSIST", stateKey); err != nil { + return err + } + } + _, err = r.redis.Do(ctx, "HSET", sourceKey, "replayState", "processing", "replayError", "") + if err != nil { + return err + } + for { + if ctx.Err() != nil { + return ctx.Err() + } + valid, err := store.RenewLock(ctx, lockKey, owner, time.Minute) + if err != nil || !valid { + return fmt.Errorf("replay lease lost: %w", err) + } + // HSCAN is not a snapshot. Complete a bounded cursor pass, then + // recheck HLEN and repeat from zero if deletion changed the scan. + var cursor uint64 + for { + v, e := r.redis.Do(ctx, "HSCAN", failedKey, cursor, "COUNT", 100) + if e != nil { + return e + } + var scanPairs []*gvar.Var + cursor, scanPairs, e = parseScanResult(v) + if e != nil { + return e + } + if len(scanPairs) > 0 { + for i := 0; i < len(scanPairs); i += 2 { + valid, renewErr := store.RenewLock(ctx, lockKey, owner, time.Minute) + if renewErr != nil || !valid { + return fmt.Errorf("replay lease lost") + } + field, raw := scanPairs[i].String(), scanPairs[i+1].String() + var rec zset.FailureRecord + if json.Unmarshal([]byte(raw), &rec) != nil || rec.TaskID == "" || rec.CardID <= 0 || rec.ICCID == "" || rec.ChanKey == "" || rec.TempKey == "" || field != rec.TaskID || (rec.LatestRunID != "" && rec.LatestRunID != sourceID) { + return fmt.Errorf("source failure record cannot be replayed") + } + operation := rec.Operation + if operation == "" { + operation = meta.Operation + } + // Source task identity (not just card ID) prevents collisions + // between distinct failures for the same card. + taskID := targetID + ":" + rec.TaskID + queryMonth := rec.QueryMonth + if queryMonth == "" { + queryMonth = meta.QueryMonth + } + task := zset.Task{RunID: targetID, TaskID: taskID, Operation: operation, QueryMonth: queryMonth, CardID: rec.CardID, ICCID: rec.ICCID, MSISDN: rec.MSISDN, IMSI: rec.IMSI, ChanKey: rec.ChanKey, TempKey: rec.TempKey, PackageCode: rec.PackageCode} + if _, e = store.EnqueueRunBatch(ctx, targetID, rec.ChanKey, []zset.Task{task}, time.Now()); e != nil { + return e + } + confirmed, e := r.redis.Do(ctx, "HEXISTS", stateKey, taskID) + if e != nil || confirmed == nil || confirmed.Int() != 1 { + return fmt.Errorf("target enqueue unconfirmed: %w", e) + } + removed, e := r.redis.Eval(ctx, cleanupReplayFailure, 3, []string{sourceKey, failedKey, lockKey}, []any{field, raw, targetID, owner}) + if e != nil { + return e + } + if removed.Int() == -1 { + return fmt.Errorf("source failure changed during replay") + } + } + } + if cursor == 0 { + break + } + } + remaining, err := r.redis.Do(ctx, "HLEN", failedKey) + if err != nil { + return err + } + if remaining.Int64() == 0 { + break + } + } + // Persist producerDone only after every source record was confirmed. + if err := store.MarkProducerDone(ctx, targetID); err != nil { + return err + } + if _, err := r.redis.Do(ctx, "HSET", sourceKey, "replayState", "completed", "replayError", ""); err != nil { + return err + } + // The source summary has already been archived. Keep the small source + // metadata for lookup but restore a bounded retention after handoff. + if _, err := r.redis.Do(ctx, "EXPIRE", sourceKey, 30*24*3600); err != nil { + return err + } + _, err = r.redis.Do(ctx, "SREM", store.Prefix()+"replay:pending", sourceID) + return err +} + +const replayAttention = ` +if redis.call('GET',KEYS[2])~=ARGV[1] then return 0 end +redis.call('HSET',KEYS[1],'replayState','needs_attention','replayError','Replay handoff stopped; check Worker logs') +return 1 +` + +const reserveReplay = ` +local id=redis.call('HGET',KEYS[1],'replayTargetId') +if id then return id end +local status=redis.call('HGET',KEYS[1],'status') +if not status then return 'missing' end +if status~='completed' and status~='completed_with_failures' then return 'running' end +if redis.call('HLEN',KEYS[2])==0 then return 'empty' end +redis.call('HSET',KEYS[1],'replayTargetId',ARGV[1],'replayState','accepted','replayRequestedBy',ARGV[2],'replayEnqueued',0,'replayMaxRound',ARGV[3]) +redis.call('PERSIST',KEYS[1]); redis.call('PERSIST',KEYS[2]) +return ARGV[1] +` +const cleanupReplayFailure = ` +if redis.call('HGET',KEYS[1],'replayTargetId')~=ARGV[3] or redis.call('GET',KEYS[3])~=ARGV[4] then return -1 end +local value=redis.call('HGET',KEYS[2],ARGV[1]) +if not value then return 0 end +if value~=ARGV[2] then return -1 end +redis.call('HDEL',KEYS[2],ARGV[1]); redis.call('HINCRBY',KEYS[1],'replayEnqueued',1) +redis.call('HSET',KEYS[1],'pendingFailureCount',redis.call('HLEN',KEYS[2])) +return 1 +` diff --git a/internal/fullsync/runtime/simple_replay_invalid_test.go b/internal/fullsync/runtime/simple_replay_invalid_test.go new file mode 100644 index 0000000..c83ad79 --- /dev/null +++ b/internal/fullsync/runtime/simple_replay_invalid_test.go @@ -0,0 +1,59 @@ +package runtime + +import ( + "context" + "encoding/json" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +func TestSimpleReplayRetainsInvalidFailureAndResumesSameTarget(t *testing.T) { + redis := testutil.Redis(t) + ctx := context.Background() + store, _ := zset.New(redis, "sim-status:") + key, _ := store.RunMetaKey("source") + failed, _ := store.FailedKey("source") + _, _ = redis.Do(ctx, "HSET", key, "runId", "source", "status", "completed_with_failures", "producerDone", "1", "currentRound", "1", "failedCount", "2") + valid := zset.FailureRecord{TaskID: "a", CardID: 1, ICCID: "card", ChanKey: "ch", TempKey: "template"} + payload, _ := json.Marshal(valid) + _, _ = redis.Do(ctx, "HSET", failed, "a", string(payload), "bad", `{"taskId":"bad"}`) + svc, _ := NewSimpleReplay(store) + accepted, err := svc.Request(ctx, "source", "42", 0) + if err != nil { + t.Fatal(err) + } + deadline := time.Now().Add(3 * time.Second) + for time.Now().Before(deadline) { + state, err := svc.Status(ctx, accepted.ReplayRunID) + if err == nil && state.Status == "needs_attention" { + break + } + time.Sleep(20 * time.Millisecond) + } + state, _ := svc.Status(ctx, accepted.ReplayRunID) + if state.Status != "needs_attention" || state.Remaining < 1 { + t.Fatalf("unexpected status: %+v", state) + } + // No unconfirmed record may be deleted. Retry the SAME target after repair. + repaired := zset.FailureRecord{TaskID: "bad", CardID: 2, ICCID: "card2", ChanKey: "ch", TempKey: "template"} + payload, _ = json.Marshal(repaired) + _, _ = redis.Do(ctx, "HSET", failed, "bad", string(payload)) + repeat, err := svc.Request(ctx, "source", "42", 0) + if err != nil || repeat.ReplayRunID != accepted.ReplayRunID { + t.Fatalf("repeat: %+v %v", repeat, err) + } + deadline = time.Now().Add(3 * time.Second) + for time.Now().Before(deadline) { + state, _ = svc.Status(ctx, accepted.ReplayRunID) + if state.Status == "completed" { + break + } + time.Sleep(20 * time.Millisecond) + } + if state.Status != "completed" || state.Enqueued != 2 || state.Remaining != 0 { + t.Fatalf("resume: %+v", state) + } +} diff --git a/internal/fullsync/runtime/simple_replay_max_rounds_test.go b/internal/fullsync/runtime/simple_replay_max_rounds_test.go new file mode 100644 index 0000000..7e48c70 --- /dev/null +++ b/internal/fullsync/runtime/simple_replay_max_rounds_test.go @@ -0,0 +1,121 @@ +package runtime + +import ( + "context" + "encoding/json" + "fmt" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +// seedReplaySource writes a completed source run with n failures into store. +func seedReplaySource(t *testing.T, redis zset.RedisClient, store *zset.Store, name string, n int) { + t.Helper() + ctx := context.Background() + key, _ := store.RunMetaKey(name) + failed, _ := store.FailedKey(name) + if _, err := redis.Do(ctx, "HSET", key, "runId", name, "operation", "usage", "status", "completed_with_failures", "producerDone", "1", "currentRound", "1", "failedCount", fmt.Sprint(n)); err != nil { + t.Fatal(err) + } + for i := 1; i <= n; i++ { + rec := zset.FailureRecord{TaskID: fmt.Sprintf("task-%d", i), CardID: int64(i), ICCID: fmt.Sprintf("iccid-%d", i), ChanKey: "ch", TempKey: "template", Operation: "usage", FailureType: "mobile_busy"} + payload, _ := json.Marshal(rec) + if _, err := redis.Do(ctx, "HSET", failed, rec.TaskID, string(payload)); err != nil { + t.Fatal(err) + } + } +} + +// awaitTargetMaxRound waits for the handoff to create the target and returns +// the persisted MaxRound of the replayed Run. +func awaitTargetMaxRound(t *testing.T, svc *SimpleReplay, store *zset.Store, replayRunID string) int { + t.Helper() + ctx := context.Background() + deadline := time.Now().Add(4 * time.Second) + for time.Now().Before(deadline) { + meta, err := store.GetRunMeta(ctx, replayRunID) + if err == nil && meta.MaxRound > 0 && meta.ProducerDone { + return meta.MaxRound + } + time.Sleep(20 * time.Millisecond) + } + meta, err := store.GetRunMeta(ctx, replayRunID) + t.Fatalf("target never settled: %+v %v", meta, err) + return 0 +} + +func TestReplayMaxRoundsPrefersRequestThenConfig(t *testing.T) { + redis := testutil.Redis(t) + ctx := context.Background() + store, _ := zset.New(redis, "flow:") + + // Use a default that differs from DefaultReplayMaxRounds so the assertion + // proves the injected config value is used, not the built-in constant. + svc, err := NewSimpleReplayWithDefault(5, store) + if err != nil { + t.Fatal(err) + } + + seedReplaySource(t, redis, store, "flow:default-source", 2) + receipt, err := svc.Request(ctx, "flow:default-source", "42", 0) + if err != nil { + t.Fatal(err) + } + if got := awaitTargetMaxRound(t, svc, store, receipt.ReplayRunID); got != 5 { + t.Fatalf("unspecified request must inherit config default 5, got %d", got) + } + + seedReplaySource(t, redis, store, "flow:override-source", 2) + override, err := svc.Request(ctx, "flow:override-source", "42", 7) + if err != nil { + t.Fatal(err) + } + if got := awaitTargetMaxRound(t, svc, store, override.ReplayRunID); got != 7 { + t.Fatalf("explicit request must win over config, got %d", got) + } +} + +func TestReplayMaxRoundsRejectsOutOfRange(t *testing.T) { + redis := testutil.Redis(t) + ctx := context.Background() + store, _ := zset.New(redis, "flow:") + svc, _ := NewSimpleReplay(store) + seedReplaySource(t, redis, store, "flow:source", 1) + + for _, requested := range []int{-1, MaxReplayRounds + 1} { + if _, err := svc.Request(ctx, "flow:source", "42", requested); err != contract.ErrInvalidArgument { + t.Fatalf("maxRounds=%d must be rejected, got %v", requested, err) + } + } + // The rejected requests must not have reserved the source. + if got, err := redis.Do(ctx, "HGET", "flow:{flow:source}:meta", "replayTargetId"); err == nil && got != nil && got.String() != "" { + t.Fatalf("rejected request must not reserve the source: %v", got.String()) + } +} + +// A worker restart re-enters through reconcile() without caller arguments, so +// the budget must survive on the reservation rather than come from the caller. +func TestReplayMaxRoundsSurvivesRestartRecovery(t *testing.T) { + redis := testutil.Redis(t) + ctx := context.Background() + store, _ := zset.New(redis, "flow:") + seedReplaySource(t, redis, store, "flow:source", 2) + + // Reserve with an explicit budget, then simulate a restart by building a + // fresh service whose configured default differs from the reservation. + first, _ := NewSimpleReplayWithDefault(1, store) + receipt, err := first.Request(ctx, "flow:source", "42", 5) + if err != nil { + t.Fatal(err) + } + restarted, _ := NewSimpleReplayWithDefault(9, store) + restarted.reconcile(ctx) + + if got := awaitTargetMaxRound(t, restarted, store, receipt.ReplayRunID); got != 5 { + t.Fatalf("recovery must use the persisted budget 5, not the new default 9: got %d", got) + } +} diff --git a/internal/fullsync/runtime/simple_replay_package_test.go b/internal/fullsync/runtime/simple_replay_package_test.go new file mode 100644 index 0000000..68f180b --- /dev/null +++ b/internal/fullsync/runtime/simple_replay_package_test.go @@ -0,0 +1,82 @@ +package runtime + +import ( + "context" + "encoding/json" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +// Exercise the real task -> failure pool -> replayed task path for both +// TDengine usage chains; a producer-only assertion misses lost package codes. +func TestReplayDailyAndMonthlyFailurePoolPreservesPackageCode(t *testing.T) { + for _, tc := range []struct { + name, operation, period string + }{ + {"daily", "previous_day_usage_full_sync", "20261009"}, + {"monthly", "previous_month_usage_full_sync", "202609"}, + } { + t.Run(tc.name, func(t *testing.T) { + redis := testutil.Redis(t) + ctx := context.Background() + store, err := zset.New(redis, tc.name+":") + if err != nil { + t.Fatal(err) + } + sourceID := tc.name + "-source" + if err := store.CreateRun(ctx, zset.RunMeta{RunID: sourceID, Operation: tc.operation, QueryMonth: tc.period}); err != nil { + t.Fatal(err) + } + task := zset.Task{TaskID: sourceID + ":card-42", RunID: sourceID, Operation: tc.operation, QueryMonth: tc.period, CardID: 42, ICCID: "89860042", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "PKG-42"} + if n, err := store.EnqueueRunBatch(ctx, sourceID, task.ChanKey, []zset.Task{task}, time.Now().Add(-time.Second)); err != nil || n != 1 { + t.Fatalf("enqueue source task: %d, %v", n, err) + } + if err := store.MarkProducerDone(ctx, sourceID); err != nil { + t.Fatal(err) + } + now := time.Now() + claimed, err := store.ClaimRun(ctx, sourceID, task.ChanKey, now, now.Add(time.Minute), 1) + if err != nil || len(claimed) != 1 { + t.Fatalf("claim source task: %d, %v", len(claimed), err) + } + if done, err := store.FinalizeRunTask(ctx, claimed[0], zset.TaskOutcome{ErrorClass: "business", Code: 500, Message: "retry by pool"}); err != nil || !done { + t.Fatalf("write failure pool: %v, %v", done, err) + } + failedKey, _ := store.FailedKey(sourceID) + value, err := redis.Do(ctx, "HGET", failedKey, task.TaskID) + if err != nil || value == nil { + t.Fatalf("failure record missing: %v", err) + } + var failure zset.FailureRecord + if err := json.Unmarshal([]byte(value.String()), &failure); err != nil || failure.PackageCode != task.PackageCode { + t.Fatalf("failure pool package code: %+v, %v", failure, err) + } + if done, err := store.CompleteRunIfIdle(ctx, sourceID); err != nil || !done { + t.Fatalf("complete failed source: %v, %v", done, err) + } + svc, err := NewSimpleReplay(store) + if err != nil { + t.Fatal(err) + } + receipt, err := svc.Request(ctx, sourceID, "42", 0) + if err != nil { + t.Fatal(err) + } + if got := awaitTargetMaxRound(t, svc, store, receipt.ReplayRunID); got != DefaultReplayMaxRounds { + t.Fatalf("unexpected replay budget: %d", got) + } + now = time.Now() + replayed, err := store.ClaimRun(ctx, receipt.ReplayRunID, task.ChanKey, now, now.Add(time.Minute), 1) + if err != nil || len(replayed) != 1 { + t.Fatalf("claim replayed task: %d, %v", len(replayed), err) + } + got := replayed[0].Task + if got.PackageCode != task.PackageCode || got.QueryMonth != tc.period || got.Operation != tc.operation { + t.Fatalf("replayed task lost package or period: %+v", got) + } + }) + } +} diff --git a/internal/fullsync/runtime/simple_replay_recovery_test.go b/internal/fullsync/runtime/simple_replay_recovery_test.go new file mode 100644 index 0000000..8fe97f0 --- /dev/null +++ b/internal/fullsync/runtime/simple_replay_recovery_test.go @@ -0,0 +1,64 @@ +package runtime + +import ( + "context" + "encoding/json" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +func TestSimpleReplayRecoversEnqueuedButNotCleanedFailure(t *testing.T) { + redis := testutil.Redis(t) + ctx := context.Background() + store, _ := zset.New(redis, "flow:") + sourceKey, _ := store.RunMetaKey("source") + failedKey, _ := store.FailedKey("source") + _, _ = redis.Do(ctx, "HSET", sourceKey, "runId", "source", "status", "completed_with_failures", "producerDone", "1", "operation", "usage", "currentRound", "1", "failedCount", "1", "replayTargetId", "replay-existing", "replayState", "processing", "replayEnqueued", "0") + rec := zset.FailureRecord{TaskID: "source:usage:1", CardID: 1, ICCID: "card", ChanKey: "ch", TempKey: "template", Operation: "usage", LatestRunID: "source"} + raw, _ := json.Marshal(rec) + _, _ = redis.Do(ctx, "HSET", failedKey, rec.TaskID, string(raw)) + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "replay-existing", Operation: "usage", SyncType: "replay", SourceRunID: "source", ReplayCycle: 1, MaxRound: 1}); err != nil { + t.Fatal(err) + } + if v, err := redis.Do(ctx, "HGET", sourceKey, "replayMaxRound"); err != nil || (v != nil && v.String() != "") { + t.Fatalf("legacy reservation unexpectedly has a stored budget: %v, %v", v, err) + } + task := zset.Task{RunID: "replay-existing", TaskID: "replay-existing:" + rec.TaskID, CardID: rec.CardID, ICCID: rec.ICCID, Operation: "usage", ChanKey: "ch", TempKey: "template"} + if count, err := store.EnqueueRunBatch(ctx, task.RunID, task.ChanKey, []zset.Task{task}, time.Now()); err != nil || count != 1 { + t.Fatalf("seed enqueue = %d, %v", count, err) + } + svc, _ := NewSimpleReplayWithDefault(9, store) + svc.process(store, "source") + meta, err := store.GetRunMeta(ctx, "replay-existing") + if err != nil || meta.ActiveCount != 1 || !meta.ProducerDone || meta.MaxRound != 1 { + t.Fatalf("legacy target budget or enqueue changed after recovery: %+v %v", meta, err) + } + state, err := svc.Status(ctx, "replay-existing") + if err != nil || state.Status != "completed" || state.Enqueued != 1 || state.Remaining != 0 { + t.Fatalf("recovery state: %+v %v", state, err) + } +} + +func TestSimpleReplayRecoversLegacyReservationBeforeTargetCreation(t *testing.T) { + redis := testutil.Redis(t) + ctx := context.Background() + store, _ := zset.New(redis, "flow:") + seedReplaySource(t, redis, store, "source", 1) + sourceKey, _ := store.RunMetaKey("source") + if _, err := redis.Do(ctx, "HSET", sourceKey, "replayTargetId", "replay-old-reservation", "replayState", "accepted", "replayEnqueued", "0"); err != nil { + t.Fatal(err) + } + svc, _ := NewSimpleReplayWithDefault(9, store) + svc.process(store, "source") + meta, err := store.GetRunMeta(ctx, "replay-old-reservation") + if err != nil || meta.MaxRound != 1 || !meta.ProducerDone || meta.ActiveCount != 1 { + t.Fatalf("legacy reservation must retain the old one-round budget: %+v %v", meta, err) + } + state, err := svc.Status(ctx, meta.RunID) + if err != nil || state.Status != "completed" || state.Enqueued != 1 || state.Remaining != 0 { + t.Fatalf("legacy reservation did not complete: %+v %v", state, err) + } +} diff --git a/internal/fullsync/runtime/simple_replay_test.go b/internal/fullsync/runtime/simple_replay_test.go new file mode 100644 index 0000000..3126ce3 --- /dev/null +++ b/internal/fullsync/runtime/simple_replay_test.go @@ -0,0 +1,96 @@ +package runtime + +import ( + "context" + "encoding/json" + "fmt" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +func TestSimpleReplayBothNamespacesAndRecovery(t *testing.T) { + redis := testutil.Redis(t) + ctx := context.Background() + traffic, _ := zset.New(redis, "flow:") + status, _ := zset.New(redis, "sim-status:") + for _, store := range []*zset.Store{traffic, status} { + name := store.Prefix() + "source" + key, _ := store.RunMetaKey(name) + failed, _ := store.FailedKey(name) + _, err := redis.Do(ctx, "HSET", key, "runId", name, "operation", "operation", "status", "completed_with_failures", "producerDone", "1", "failedCount", "3", "currentRound", "1") + if err != nil { + t.Fatal(err) + } + for i := 1; i <= 3; i++ { + rec := zset.FailureRecord{TaskID: fmt.Sprintf("task-%d", i), CardID: int64(i), ICCID: fmt.Sprintf("iccid-%d", i), ChanKey: "ch", TempKey: "template", Operation: "operation", FailureType: "business"} + payload, _ := json.Marshal(rec) + if _, err := redis.Do(ctx, "HSET", failed, rec.TaskID, string(payload)); err != nil { + t.Fatal(err) + } + } + } + svc, _ := NewSimpleReplay(traffic, status) + for _, prefix := range []string{"flow:", "sim-status:"} { + source := prefix + "source" + receipt, err := svc.Request(ctx, source, "42", 0) + if err != nil { + t.Fatal(err) + } + repeat, err := svc.Request(ctx, source, "42", 0) + if err != nil || repeat.ReplayRunID != receipt.ReplayRunID { + t.Fatalf("repeat = %+v, %v", repeat, err) + } + // Recovery competes with the initial goroutine but must keep its ID. + svc.reconcile(ctx) + deadline := time.Now().Add(4 * time.Second) + for time.Now().Before(deadline) { + state, err := svc.Status(ctx, receipt.ReplayRunID) + if err == nil && state.Status == "completed" { + if state.Remaining != 0 || state.Enqueued != 3 { + t.Fatalf("state = %+v", state) + } + store := traffic + if prefix == "sim-status:" { + store = status + } + meta, err := store.GetRunMeta(ctx, receipt.ReplayRunID) + if err != nil || meta.MaxRound != DefaultReplayMaxRounds || meta.SourceRunID != source || !meta.ProducerDone || meta.ActiveCount != 3 { + t.Fatalf("target meta = %+v, %v", meta, err) + } + sourceMeta, err := store.GetRunMeta(ctx, source) + if err != nil || sourceMeta.FailedCount != 3 { + t.Fatalf("source meta = %+v, %v", sourceMeta, err) + } + break + } + time.Sleep(20 * time.Millisecond) + } + state, _ := svc.Status(ctx, receipt.ReplayRunID) + if state.Status != "completed" { + t.Fatalf("handoff timed out: %+v", state) + } + } +} + +func TestSimpleReplayRejectsRunningAndEmpty(t *testing.T) { + redis := testutil.Redis(t) + store, _ := zset.New(redis, "flow:") + svc, _ := NewSimpleReplay(store) + ctx := context.Background() + if _, err := svc.Request(ctx, "missing", "1", 0); err != contract.ErrNotFound { + t.Fatal(err) + } + key, _ := store.RunMetaKey("source") + _, _ = redis.Do(ctx, "HSET", key, "status", "running") + if _, err := svc.Request(ctx, "source", "1", 0); err != contract.ErrConflict { + t.Fatal(err) + } + _, _ = redis.Do(ctx, "HSET", key, "status", "completed_with_failures") + if _, err := svc.Request(ctx, "source", "1", 0); err != ErrNoFailures { + t.Fatal(err) + } +} diff --git a/internal/fullsync/supply/README.md b/internal/fullsync/supply/README.md new file mode 100644 index 0000000..e80340b --- /dev/null +++ b/internal/fullsync/supply/README.md @@ -0,0 +1,17 @@ +# 流量首轮供料观测 + +配置 `trafficUsageFullSync.supplyObservation.enabled` 默认 false。仅 Run 首轮 channel 扫描生效；与状态同步、Replay、legacy 队列无关。采样 1 秒，快照 10 秒，单批最多 64 个目标，快照最大 256 KiB。保持原分页、扫描并发、水位、消费者 QPS 与重试策略。 + +队列只读 Lua 区分到期可领取库存、租约（含过期租约）与未来任务，不读取任务正文。生产者事件保留当前页未确认任务数；消费者处理窗口包含 QPS 等待、请求、写库和结算。两个相邻同类样本开始估算区间；超过 2 秒中断连续统计并记录缺口。启动前已知供料等待与首次新增后的确认缺料分开；查询期间没有已知待入队任务时只记待确认空闲。阶段时间是同期阶段而非因果证明。连续 5 秒仅作为 Bot 排查提示。 + +元数据可选 `supplyObservation` 使用 schemaVersion=1，归档顶层 version 仍为 1。关闭配置没有观察协程或遥测命令。非法遥测不影响业务归档。最终刷新发生在 producerDone 前；写入需校验 producer owner 及非终态。当前 owner 校验 Lua 涉及既有全局 producer 锁，适用现有 standalone Redis；Cluster CROSSSLOT 会使遥测降级，不应宣称已支持跨槽原子写。快照截断保留 Run 总量及最长等待前 100 个通道（若仍超限进一步缩减），并标明 partial/truncated。重启保留落盘累计值，不跨尝试拼接区间。 + +## 验证记录 + +- worker fullsync（archive/contract/httpapi/runtime/supply）、producer/monthlyusage、queue/zset、consumer/zset、worker 测试通过。 +- 真实隔离 Redis：130 通道分批、未来任务/到期库存/过期租约、旧 owner/终态拒写、非法可选 JSON 兼容。 +- scanner 替身：空目标被登记且无首次入队、不误报确认缺料、最终快照保存；注入遥测写失败仍完成 producerDone。 +- 可控时间：阶段累计、处理与库存排除、unknown、采样缺口、并发事件、256 KiB 排名与恢复总量。 +- `go build ./...` 已通过。 +- 本机 CGO 关闭且未发现 C 编译器，race 尚未执行成功。 +- 全仓测试非全绿； diff --git a/internal/fullsync/supply/observation.go b/internal/fullsync/supply/observation.go new file mode 100644 index 0000000..073313f --- /dev/null +++ b/internal/fullsync/supply/observation.go @@ -0,0 +1,287 @@ +// Package supply holds bounded, per-channel telemetry independent of card volume. +package supply + +import ( + "sort" + "sync" + "time" +) + +const SchemaVersion = 1 + +type Wait struct { + Count int64 `json:"count"` + TotalMS int64 `json:"totalMs"` + LongestMS int64 `json:"longestMs"` +} +type Channel struct { + Key string `json:"chanKey"` + FirstQueryMS *int64 `json:"firstQueryMs"` + FirstEnqueueMS *int64 `json:"firstEnqueueMs"` + Startup Wait `json:"startupWait"` + Confirmed Wait `json:"confirmedWait"` + Unconfirmed Wait `json:"unconfirmedIdle"` + Consumer Wait `json:"consumerStartupWait"` + UnknownMS int64 `json:"unknownMs"` + EmptyClaims int64 `json:"emptyClaims"` + ValidSamples int64 `json:"validSamples"` + MissingSamples int64 `json:"missingSamples"` + Stages map[string]int64 `json:"producerStageMs"` + Completeness string `json:"completeness"` + Termination string `json:"termination,omitempty"` + CoverageStart time.Time `json:"coverageStart"` + CoverageEnd time.Time `json:"coverageEnd"` +} +type Summary struct { + Targets int `json:"targetChannels"` + Valid int `json:"validChannels"` + Confirmed int `json:"confirmedChannels"` + Unknown int `json:"unknownChannels"` + UnknownMS int64 `json:"unknownChannelMs"` + UnconfirmedMS int64 `json:"unconfirmedChannelMs"` + ConsumerMS int64 `json:"consumerStartupChannelMs"` + TotalMS int64 `json:"confirmedChannelMs"` + LongestMS int64 `json:"longestMs"` + Worst string `json:"worstChannel,omitempty"` +} +type Snapshot struct { + SchemaVersion int `json:"schemaVersion"` + Attempt string `json:"attempt"` + ResolutionMS int `json:"resolutionMs"` + Partial bool `json:"partial"` + Truncated bool `json:"truncated"` + Summary Summary `json:"summary"` + Channels []Channel `json:"channels"` +} +type View struct { + Version uint64 + Pending int + Processing int + Stage string + Ended bool +} +type Sample struct { + Ready int64 + Leases int64 + Future int64 + Eligible bool + Known bool +} +type state struct { + Channel + View + last time.Time + kind string + continuous int64 + open bool +} +type Observer struct { + mu sync.Mutex + started time.Time + attempt string + partial bool + retained Summary + truncated bool + channels map[string]*state +} + +func New(attempt string, started time.Time, keys []string) *Observer { + o := &Observer{attempt: attempt, started: started, channels: make(map[string]*state, len(keys))} + for _, k := range keys { + o.channels[k] = &state{Channel: Channel{Key: k, Stages: map[string]int64{}, CoverageStart: started}, View: View{Stage: "initializing"}} + } + return o +} +func (o *Observer) Event(key, stage string, pending int, added bool, ended bool, now time.Time) { + o.mu.Lock() + defer o.mu.Unlock() + s := o.channels[key] + if s == nil { + return + } + s.Version++ + s.Stage = stage + s.Pending = pending + if stage == "sql_query" && s.FirstQueryMS == nil { + v := now.Sub(o.started).Milliseconds() + s.FirstQueryMS = &v + } + if added && s.FirstEnqueueMS == nil { + v := now.Sub(o.started).Milliseconds() + s.FirstEnqueueMS = &v + s.close() + } + if ended { + s.Ended = true + s.Termination = stage + s.CoverageEnd = now + s.close() + } +} +func (o *Observer) Empty(key string) { + o.mu.Lock() + defer o.mu.Unlock() + if s := o.channels[key]; s != nil && !s.Ended { + s.EmptyClaims++ + } +} +func (o *Observer) Processing(key string, delta int) { + o.mu.Lock() + defer o.mu.Unlock() + if s := o.channels[key]; s != nil { + s.Version++ + s.Processing = max(0, s.Processing+delta) + if delta > 0 { + s.close() + } + } +} +func (o *Observer) Views() map[string]View { + o.mu.Lock() + defer o.mu.Unlock() + out := make(map[string]View, len(o.channels)) + for k, s := range o.channels { + out[k] = s.View + } + return out +} +func (s *state) close() { s.kind = ""; s.continuous = 0; s.open = false; s.last = time.Time{} } +func (o *Observer) Partial() { o.mu.Lock(); o.partial = true; o.mu.Unlock() } +func (o *Observer) Apply(key string, before View, sample Sample, now time.Time) { + o.mu.Lock() + defer o.mu.Unlock() + s := o.channels[key] + if s == nil || s.Ended { + return + } + kind := "unknown" + if sample.Known && before.Version == s.Version { + s.ValidSamples++ + switch { + case !sample.Eligible: + s.close() + s.CoverageEnd = now + return + case sample.Ready > 0: + if s.Processing == 0 && sample.Leases == 0 { + kind = "consumer" + } else { + kind = "stock" + } + case s.Processing > 0 || sample.Leases > 0: + kind = "processing" + case sample.Future > 0: + kind = "scheduled" + case s.Pending > 0: + if s.FirstEnqueueMS == nil { + kind = "startup" + } else { + kind = "confirmed" + } + default: + kind = "unconfirmed" + } + } else { + s.MissingSamples++ + o.partial = true + } + elapsed := now.Sub(s.last) + if !s.last.IsZero() && elapsed > 2*time.Second { + s.MissingSamples += int64(elapsed/time.Second) - 1 + o.partial = true + s.close() + } + if s.kind == kind && !s.last.IsZero() { + ms := elapsed.Milliseconds() + if ms < 0 { + ms = 0 + } + s.continuous += ms + var w *Wait + switch kind { + case "startup": + w = &s.Startup + case "confirmed": + w = &s.Confirmed + case "unconfirmed": + w = &s.Unconfirmed + case "consumer": + w = &s.Consumer + case "unknown": + s.UnknownMS += ms + } + if w != nil { + if !s.open { + w.Count++ + s.open = true + } + w.TotalMS += ms + if s.continuous > w.LongestMS { + w.LongestMS = s.continuous + } + if kind == "startup" || kind == "confirmed" { + s.Stages[s.Stage] += ms + } + } + } else { + s.continuous = 0 + s.open = false + } + s.kind = kind + s.last = now + s.CoverageEnd = now +} +func (o *Observer) Snapshot() Snapshot { + o.mu.Lock() + defer o.mu.Unlock() + out := Snapshot{SchemaVersion: 1, Attempt: o.attempt, ResolutionMS: 1000, Partial: o.partial, Truncated: o.truncated, Channels: make([]Channel, 0, len(o.channels))} + out.Summary.Targets = len(o.channels) + for _, s := range o.channels { + c := s.Channel + c.Completeness = "observing" + if s.Ended { + c.Completeness = "complete" + } + if c.MissingSamples > 0 || c.UnknownMS > 0 || c.ValidSamples == 0 || c.Termination == "cancelled" { + c.Completeness = "partial" + } + c.Stages = make(map[string]int64, len(s.Stages)) + for k, v := range s.Stages { + c.Stages[k] = v + } + out.Channels = append(out.Channels, c) + if c.ValidSamples == 0 { + out.Partial = true + } + out.Summary.UnknownMS += c.UnknownMS + out.Summary.UnconfirmedMS += c.Unconfirmed.TotalMS + out.Summary.ConsumerMS += c.Consumer.TotalMS + if c.ValidSamples > 0 { + out.Summary.Valid++ + } + total := c.Startup.TotalMS + c.Confirmed.TotalMS + longest := max(c.Startup.LongestMS, c.Confirmed.LongestMS) + if total > 0 { + out.Summary.Confirmed++ + } + if c.MissingSamples > 0 || c.UnknownMS > 0 || c.ValidSamples == 0 { + out.Summary.Unknown++ + } + out.Summary.TotalMS += total + if longest > out.Summary.LongestMS { + out.Summary.LongestMS = longest + out.Summary.Worst = c.Key + } + } + out.Summary.Confirmed = min(out.Summary.Targets, out.Summary.Confirmed+o.retained.Confirmed) + out.Summary.TotalMS += o.retained.TotalMS + out.Summary.UnknownMS += o.retained.UnknownMS + out.Summary.UnconfirmedMS += o.retained.UnconfirmedMS + out.Summary.ConsumerMS += o.retained.ConsumerMS + if o.retained.LongestMS > out.Summary.LongestMS { + out.Summary.LongestMS = o.retained.LongestMS + out.Summary.Worst = o.retained.Worst + } + sort.Slice(out.Channels, func(i, j int) bool { return out.Channels[i].Key < out.Channels[j].Key }) + return out +} diff --git a/internal/fullsync/supply/observation_test.go b/internal/fullsync/supply/observation_test.go new file mode 100644 index 0000000..68dd69b --- /dev/null +++ b/internal/fullsync/supply/observation_test.go @@ -0,0 +1,134 @@ +package supply + +import ( + "encoding/json" + "fmt" + "strings" + "sync" + "testing" + "time" +) + +func TestSnapshotLimitPreservesSummaryAndRanking(t *testing.T) { + keys := make([]string, 700) + for i := range keys { + keys[i] = fmt.Sprintf("%04d-%s", i, strings.Repeat("x", 200)) + } + start := time.Unix(100, 0) + o := New("attempt", start, keys) + for i, k := range keys { + o.Event(k, "global_capacity", 1, false, false, start) + o.Apply(k, o.Views()[k], Sample{Known: true, Eligible: true}, start.Add(time.Second)) + o.Apply(k, o.Views()[k], Sample{Known: true, Eligible: true}, start.Add(time.Duration(1000+i)*time.Millisecond+time.Second)) + } + body, err := o.Encode() + if err != nil { + t.Fatal(err) + } + var s Snapshot + if err = json.Unmarshal(body, &s); err != nil { + t.Fatal(err) + } + if len(body) > 256*1024 || !s.Truncated || len(s.Channels) != 100 || s.Summary.Targets != 700 || s.Channels[0].Key != keys[699] { + t.Fatalf("invalid bounded snapshot bytes=%d %+v", len(body), s.Summary) + } + restored := New("new", start.Add(time.Hour), keys) + restored.Restore(s) + r := restored.Snapshot() + if !r.Partial || !r.Truncated || r.Summary.TotalMS != s.Summary.TotalMS { + t.Fatalf("lost historical totals %+v %+v", s.Summary, r.Summary) + } +} + +func TestConfirmedIntervalsAndStages(t *testing.T) { + start := time.Unix(100, 0) + o := New("attempt", start, []string{"a", "empty"}) + o.Event("a", "global_capacity", 2, false, false, start) + tick := func(n int) { + v := o.Views()["a"] + o.Apply("a", v, Sample{Known: true, Eligible: true}, start.Add(time.Duration(n)*time.Second)) + } + tick(1) + tick(2) + o.Event("a", "redis_write", 2, false, false, start.Add(2*time.Second)) + tick(3) + s := o.Snapshot() + c := s.Channels[0] + if c.Startup.Count != 1 || c.Startup.TotalMS != 2000 || c.Stages["global_capacity"] != 1000 || c.Stages["redis_write"] != 1000 { + t.Fatalf("unexpected %+v", c) + } + o.Processing("a", 1) + tick(4) + tick(5) + if o.Snapshot().Channels[0].Startup.TotalMS != 2000 { + t.Fatal("processing counted as idle") + } +} +func TestUncertainAndGap(t *testing.T) { + start := time.Unix(100, 0) + o := New("a", start, []string{"a"}) + v := o.Views()["a"] + o.Event("a", "sql_query", 0, false, false, start) + o.Apply("a", v, Sample{Known: true, Eligible: true}, start.Add(time.Second)) + o.Apply("a", o.Views()["a"], Sample{Known: false}, start.Add(2*time.Second)) + o.Apply("a", o.Views()["a"], Sample{Known: true, Eligible: true}, start.Add(6*time.Second)) + s := o.Snapshot() + if !s.Partial || s.Channels[0].UnknownMS != 1000 || s.Channels[0].MissingSamples != 5 || s.Summary.Confirmed != 0 { + t.Fatalf("unexpected %+v", s) + } +} +func TestSupplyExclusions(t *testing.T) { + for _, tc := range []struct { + name string + sample Sample + processing bool + }{{"stock", Sample{Known: true, Eligible: true, Ready: 1}, false}, {"lease", Sample{Known: true, Eligible: true, Leases: 1}, false}, {"ended_round_or_blocked", Sample{Known: true, Eligible: false}, false}, {"consumer", Sample{Known: true, Eligible: true}, true}, {"unknown", Sample{Eligible: true}, false}} { + t.Run(tc.name, func(t *testing.T) { + start := time.Unix(100, 0) + o := New("a", start, []string{"a"}) + o.Event("a", "global_capacity", 100, false, false, start) + if tc.processing { + o.Processing("a", 1) + } + for i := 1; i <= 3; i++ { + o.Apply("a", o.Views()["a"], tc.sample, start.Add(time.Duration(i)*time.Second)) + } + if o.Snapshot().Summary.Confirmed != 0 { + t.Fatal("excluded sample counted as supply idle") + } + }) + } +} +func TestConcurrentEvents(t *testing.T) { + start := time.Unix(100, 0) + o := New("a", start, []string{"a"}) + var wg sync.WaitGroup + for i := 0; i < 8; i++ { + wg.Add(1) + go func() { + defer wg.Done() + for n := 0; n < 100; n++ { + o.Event("a", "sql_query", 0, false, false, start) + o.Processing("a", 1) + o.Empty("a") + o.Apply("a", o.Views()["a"], Sample{Known: true, Eligible: true}, start) + o.Snapshot() + o.Processing("a", -1) + } + }() + } + wg.Wait() +} + +func TestEmptyNeverConfirms(t *testing.T) { + start := time.Unix(100, 0) + o := New("a", start, []string{"a"}) + for i := 1; i <= 3; i++ { + o.Apply("a", o.Views()["a"], Sample{Known: true, Eligible: true}, start.Add(time.Duration(i)*time.Second)) + } + o.Event("a", "empty", 0, false, true, start.Add(3*time.Second)) + s := o.Snapshot() + if s.Summary.Confirmed != 0 || s.Channels[0].Unconfirmed.TotalMS != 2000 { + t.Fatalf("unexpected %+v", s) + } +} diff --git a/internal/fullsync/supply/tracker.go b/internal/fullsync/supply/tracker.go new file mode 100644 index 0000000..6da5201 --- /dev/null +++ b/internal/fullsync/supply/tracker.go @@ -0,0 +1,90 @@ +package supply + +import ( + "encoding/json" + "fmt" + "sort" + "sync" + "sync/atomic" +) + +// Tracker routes optional consumer events to the local producer attempt. +type Tracker struct { + mu sync.RWMutex + runs map[string]*Observer + LocalConsumer bool + ConsumerActive atomic.Bool +} + +func NewTracker(localConsumer bool) *Tracker { + return &Tracker{runs: map[string]*Observer{}, LocalConsumer: localConsumer} +} +func (t *Tracker) Put(run string, o *Observer) { t.mu.Lock(); t.runs[run] = o; t.mu.Unlock() } +func (t *Tracker) Remove(run string) { t.mu.Lock(); delete(t.runs, run); t.mu.Unlock() } +func (t *Tracker) Empty(run, key string) { + t.mu.RLock() + defer t.mu.RUnlock() + if o := t.runs[run]; o != nil { + o.Empty(key) + } +} +func (t *Tracker) Processing(run, key string, delta int) { + t.mu.RLock() + defer t.mu.RUnlock() + if o := t.runs[run]; o != nil { + o.Processing(key, delta) + } +} +func (o *Observer) Restore(previous Snapshot) { + o.mu.Lock() + defer o.mu.Unlock() + o.partial = true + o.retained = previous.Summary + o.truncated = previous.Truncated + for _, c := range previous.Channels { + if s := o.channels[c.Key]; s != nil { + if c.Startup.TotalMS+c.Confirmed.TotalMS > 0 { + o.retained.Confirmed-- + } + o.retained.TotalMS -= c.Startup.TotalMS + c.Confirmed.TotalMS + o.retained.UnknownMS -= c.UnknownMS + o.retained.UnconfirmedMS -= c.Unconfirmed.TotalMS + o.retained.ConsumerMS -= c.Consumer.TotalMS + s.Channel = c + if s.Stages == nil { + s.Stages = map[string]int64{} + } + s.Termination = "" + s.MissingSamples++ + s.close() + } + } +} +func (o *Observer) Encode() ([]byte, error) { + snapshot := o.Snapshot() + encoded, err := json.Marshal(snapshot) + if err != nil || len(encoded) <= 256*1024 { + return encoded, err + } + snapshot.Truncated = true + snapshot.Partial = true + sort.Slice(snapshot.Channels, func(i, j int) bool { + a, b := snapshot.Channels[i], snapshot.Channels[j] + av, bv := max(a.Startup.LongestMS, a.Confirmed.LongestMS), max(b.Startup.LongestMS, b.Confirmed.LongestMS) + if av == bv { + return a.Key < b.Key + } + return av > bv + }) + snapshot.Channels = snapshot.Channels[:min(100, len(snapshot.Channels))] + for { + encoded, err = json.Marshal(snapshot) + if err != nil || len(encoded) <= 256*1024 { + return encoded, err + } + if len(snapshot.Channels) == 0 { + return nil, fmt.Errorf("supply summary exceeds snapshot limit") + } + snapshot.Channels = snapshot.Channels[:len(snapshot.Channels)-1] + } +} diff --git a/internal/fullsync/supply/validation.go b/internal/fullsync/supply/validation.go new file mode 100644 index 0000000..b1b9615 --- /dev/null +++ b/internal/fullsync/supply/validation.go @@ -0,0 +1,27 @@ +package supply + +// Valid checks only telemetry. Callers must degrade invalid observations without +// rejecting the business run or its archive. +func (s Snapshot) Valid() bool { + if s.SchemaVersion != 1 || s.ResolutionMS != 1000 || s.Summary.Targets < 0 || s.Summary.Valid < 0 || s.Summary.Valid > s.Summary.Targets || s.Summary.Confirmed < 0 || s.Summary.Confirmed > s.Summary.Targets || s.Summary.Unknown < 0 || s.Summary.Unknown > s.Summary.Targets || s.Summary.TotalMS < 0 || s.Summary.LongestMS < 0 || s.Summary.UnknownMS < 0 || s.Summary.UnconfirmedMS < 0 || s.Summary.ConsumerMS < 0 { + return false + } + seen := make(map[string]bool, len(s.Channels)) + for _, c := range s.Channels { + if c.Key == "" || seen[c.Key] || c.ValidSamples < 0 || c.MissingSamples < 0 || c.UnknownMS < 0 { + return false + } + seen[c.Key] = true + for _, w := range []Wait{c.Startup, c.Confirmed, c.Unconfirmed, c.Consumer} { + if w.Count < 0 || w.TotalMS < 0 || w.LongestMS < 0 || w.LongestMS > w.TotalMS { + return false + } + } + for _, v := range c.Stages { + if v < 0 { + return false + } + } + } + return len(s.Channels) <= s.Summary.Targets +} diff --git a/internal/library/rabbitmq/delivery.go b/internal/library/rabbitmq/delivery.go new file mode 100644 index 0000000..d640307 --- /dev/null +++ b/internal/library/rabbitmq/delivery.go @@ -0,0 +1,22 @@ +package rabbitmq + +import "context" + +type DeliveryMetadata struct { + DeliveryTag uint64 + Redelivered bool + Consumer string + Queue string + Headers map[string]any +} + +type deliveryMetadataContextKey struct{} + +func WithDeliveryMetadata(ctx context.Context, metadata DeliveryMetadata) context.Context { + return context.WithValue(ctx, deliveryMetadataContextKey{}, metadata) +} + +func DeliveryMetadataFromContext(ctx context.Context) (DeliveryMetadata, bool) { + metadata, ok := ctx.Value(deliveryMetadataContextKey{}).(DeliveryMetadata) + return metadata, ok +} diff --git a/internal/library/rabbitmq/rabbitmq.go b/internal/library/rabbitmq/rabbitmq.go new file mode 100644 index 0000000..076a8b1 --- /dev/null +++ b/internal/library/rabbitmq/rabbitmq.go @@ -0,0 +1,425 @@ +package rabbitmq + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "sync" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + amqp "github.com/rabbitmq/amqp091-go" +) + +type Config struct { + Enabled bool + Dsn string +} + +type channelLifecycle interface { + Cancel(consumer string, noWait bool) error + Close() error + IsClosed() bool +} + +type managedConsumer struct { + channel channelLifecycle + tag string + done chan struct{} +} + +// Handler 是 RabbitMQ 消费处理函数。 +// 返回 nil 时消息 Ack；返回 error 时消息 Nack 并重新入队，等待后续重试。 +type Handler func(ctx context.Context, body []byte) error + +// JSONHandler 是 RabbitMQ JSON 消息的业务处理函数。 +type JSONHandler[T any] func(ctx context.Context, message T) error + +var ( + conn *amqp.Connection + connMu sync.Mutex + consumers []*managedConsumer + consumerMu sync.Mutex + closeMu sync.Mutex +) + +func GetConfig(ctx context.Context) Config { + return Config{ + Enabled: g.Cfg().MustGet(ctx, "rabbitmq.enabled", false).Bool(), + Dsn: g.Cfg().MustGet(ctx, "rabbitmq.dsn", "").String(), + } +} + +// Enabled 判断 RabbitMQ 是否启用且 DSN 是否已配置。 +func Enabled(ctx context.Context) bool { + cfg := GetConfig(ctx) + return cfg.Enabled && cfg.Dsn != "" +} + +// Conn 懒加载 RabbitMQ 连接，供多个生产者/消费者复用。 +func Conn(ctx context.Context) (*amqp.Connection, error) { + cfg := GetConfig(ctx) + if !cfg.Enabled { + return nil, gerror.New("rabbitmq is disabled") + } + if cfg.Dsn == "" { + return nil, gerror.New("rabbitmq dsn is empty") + } + + connMu.Lock() + defer connMu.Unlock() + if conn != nil && !conn.IsClosed() { + return conn, nil + } + + c, err := amqp.Dial(cfg.Dsn) + if err != nil { + return nil, gerror.Wrap(err, "rabbitmq connect failed") + } + conn = c + return conn, nil +} + +// Channel 基于共享连接创建独立 channel。 +func Channel(ctx context.Context) (*amqp.Channel, error) { + c, err := Conn(ctx) + if err != nil { + return nil, err + } + ch, err := c.Channel() + if err != nil { + return nil, gerror.Wrap(err, "rabbitmq open channel failed") + } + return ch, nil +} + +// DeclareQueue 声明一个持久化队列。 +func DeclareQueue(ctx context.Context, queue string) error { + ch, err := Channel(ctx) + if err != nil { + return err + } + defer ch.Close() + return declareQueue(ch, queue) +} + +// Publish 将结构体序列化为 JSON 后发布到指定队列。 +func Publish(ctx context.Context, queue string, data any) error { + body, err := json.Marshal(data) + if err != nil { + return gerror.Wrap(err, "rabbitmq marshal message failed") + } + return PublishBytes(ctx, queue, body) +} + +// PublishDelayed 将结构体序列化为 JSON 后发布到延迟队列。 +// 消息在 delayQueue 中等待 delay 后，通过死信路由进入 targetQueue。 +func PublishDelayed(ctx context.Context, delayQueue, targetQueue string, delay time.Duration, data any) error { + body, err := json.Marshal(data) + if err != nil { + return gerror.Wrap(err, "rabbitmq marshal delayed message failed") + } + return PublishDelayedBytes(ctx, delayQueue, targetQueue, delay, body) +} + +// PublishBytes 发布原始消息体到指定队列。 +func PublishBytes(ctx context.Context, queue string, body []byte) error { + ch, err := Channel(ctx) + if err != nil { + return err + } + defer ch.Close() + + // 发布前也声明一次队列，避免生产者先启动时队列不存在。 + if err = declareQueue(ch, queue); err != nil { + return err + } + return ch.PublishWithContext(ctx, "", queue, false, false, amqp.Publishing{ + ContentType: "application/json", + DeliveryMode: amqp.Persistent, + Timestamp: time.Now(), + Body: body, + }) +} + +// PublishDelayedBytes 发布原始消息体到延迟队列。 +func PublishDelayedBytes(ctx context.Context, delayQueue, targetQueue string, delay time.Duration, body []byte) error { + if delay <= 0 { + return gerror.New("rabbitmq delayed message delay must be positive") + } + ch, err := Channel(ctx) + if err != nil { + return err + } + defer ch.Close() + + if err = declareQueue(ch, targetQueue); err != nil { + return err + } + if err = declareDelayQueue(ch, delayQueue, targetQueue); err != nil { + return err + } + return ch.PublishWithContext(ctx, "", delayQueue, false, false, amqp.Publishing{ + ContentType: "application/json", + DeliveryMode: amqp.Persistent, + Expiration: fmt.Sprintf("%d", delay.Milliseconds()), + Timestamp: time.Now(), + Body: body, + }) +} + +// PublishDelayedBytesWithHeaders 发布带消息头的原始消息体到延迟队列。 +func PublishDelayedBytesWithHeaders(ctx context.Context, delayQueue, targetQueue string, delay time.Duration, body []byte, headers amqp.Table) error { + if delay <= 0 { + return gerror.New("rabbitmq delayed message delay must be positive") + } + ch, err := Channel(ctx) + if err != nil { + return err + } + defer ch.Close() + + if err = declareQueue(ch, targetQueue); err != nil { + return err + } + if err = declareDelayQueue(ch, delayQueue, targetQueue); err != nil { + return err + } + return publishWithConfirm(ctx, ch, delayQueue, amqp.Publishing{ + ContentType: "application/json", + DeliveryMode: amqp.Persistent, + Expiration: fmt.Sprintf("%d", delay.Milliseconds()), + Timestamp: time.Now(), + Headers: headers, + Body: body, + }) +} + +// PublishBytesWithHeaders 发布带消息头的原始消息体到指定队列。 +func PublishBytesWithHeaders(ctx context.Context, queue string, body []byte, headers amqp.Table) error { + ch, err := Channel(ctx) + if err != nil { + return err + } + defer ch.Close() + + if err = declareQueue(ch, queue); err != nil { + return err + } + return publishWithConfirm(ctx, ch, queue, amqp.Publishing{ + ContentType: "application/json", + DeliveryMode: amqp.Persistent, + Timestamp: time.Now(), + Headers: headers, + Body: body, + }) +} + +func publishWithConfirm(ctx context.Context, ch *amqp.Channel, queue string, message amqp.Publishing) error { + if err := ch.Confirm(false); err != nil { + return gerror.Wrap(err, "rabbitmq enable publisher confirm failed") + } + confirms := ch.NotifyPublish(make(chan amqp.Confirmation, 1)) + if err := ch.PublishWithContext(ctx, "", queue, false, false, message); err != nil { + return gerror.Wrapf(err, "rabbitmq publish failed, queue:%s", queue) + } + select { + case confirmation, ok := <-confirms: + if !ok { + return gerror.Newf("rabbitmq publisher confirm channel closed, queue:%s", queue) + } + if !confirmation.Ack { + return gerror.Newf("rabbitmq message was not confirmed, queue:%s", queue) + } + return nil + case <-ctx.Done(): + return gerror.Wrapf(ctx.Err(), "wait rabbitmq publisher confirm failed, queue:%s", queue) + } +} + +// Consume 启动消费者，使用手动 Ack，handler 成功才确认消息。 +func Consume(ctx context.Context, queue, consumer string, handler Handler) error { + ch, err := Channel(ctx) + if err != nil { + return err + } + if err = declareQueue(ch, queue); err != nil { + _ = ch.Close() + return err + } + if err = ch.Qos(1, 0, false); err != nil { + _ = ch.Close() + return gerror.Wrap(err, "rabbitmq set qos failed") + } + + // autoAck=false，业务处理成功后手动 Ack，处理失败时 Nack 并重新入队。 + deliveries, err := ch.Consume(queue, consumer, false, false, false, false, nil) + if err != nil { + _ = ch.Close() + return gerror.Wrap(err, "rabbitmq consume failed") + } + + consumerState := &managedConsumer{ + channel: ch, + tag: consumer, + done: make(chan struct{}), + } + registerConsumer(consumerState) + go func() { + defer close(consumerState.done) + for { + if ctx.Err() != nil { + return + } + select { + case <-ctx.Done(): + return + case delivery, ok := <-deliveries: + if !ok { + return + } + consumerTag := delivery.ConsumerTag + if consumerTag == "" { + consumerTag = consumer + } + deliveryCtx := WithDeliveryMetadata(ctx, DeliveryMetadata{ + DeliveryTag: delivery.DeliveryTag, + Redelivered: delivery.Redelivered, + Consumer: consumerTag, + Queue: queue, + Headers: cloneHeaders(delivery.Headers), + }) + if handlerErr := handler(deliveryCtx, delivery.Body); handlerErr != nil { + g.Log().Errorf(deliveryCtx, "rabbitmq message Nack, queue:%s, consumer:%s, deliveryTag:%d, redelivered:%t, err:%+v", queue, consumerTag, delivery.DeliveryTag, delivery.Redelivered, handlerErr) + _ = delivery.Nack(false, true) + continue + } + // handler 内部已经完成业务处理或重新发布下一次重试消息，这里只确认当前消息。 + g.Log().Infof(deliveryCtx, "rabbitmq message Ack, queue:%s, consumer:%s, deliveryTag:%d, redelivered:%t", queue, consumerTag, delivery.DeliveryTag, delivery.Redelivered) + _ = delivery.Ack(false) + } + } + }() + return nil +} + +// ConsumeJSON 启动 JSON 消费者，统一完成反序列化和 Ack/Nack 处理。 +func ConsumeJSON[T any](ctx context.Context, queue, consumer string, handler JSONHandler[T]) error { + return Consume(ctx, queue, consumer, func(ctx context.Context, body []byte) error { + var message T + if err := json.Unmarshal(body, &message); err != nil { + g.Log().Errorf(ctx, "rabbitmq message dropped, invalid json, queue:%s, body:%s, err:%+v", queue, string(body), err) + return nil + } + return handler(ctx, message) + }) +} + +// Close 关闭消费者 channel 和共享连接。 +func Close(ctx context.Context) error { + closeMu.Lock() + defer closeMu.Unlock() + + consumerMu.Lock() + registeredConsumers := consumers + consumers = nil + consumerMu.Unlock() + + var closeErrors []error + for _, consumer := range registeredConsumers { + if consumer.channel.IsClosed() { + continue + } + if err := consumer.channel.Cancel(consumer.tag, false); !isClosedError(err) { + closeErrors = append(closeErrors, gerror.Wrapf(err, "cancel RabbitMQ consumer failed, consumer:%s", consumer.tag)) + } + } + + if err := waitForConsumers(ctx, registeredConsumers); err != nil { + closeErrors = append(closeErrors, err) + } + + for _, consumer := range registeredConsumers { + if consumer.channel.IsClosed() { + continue + } + if err := consumer.channel.Close(); !isClosedError(err) { + closeErrors = append(closeErrors, gerror.Wrapf(err, "close RabbitMQ channel failed, consumer:%s", consumer.tag)) + } + } + + if err := closeConnection(); err != nil { + closeErrors = append(closeErrors, err) + } + return errors.Join(closeErrors...) +} + +// declareQueue 统一声明持久化队列，生产者和消费者都会调用，保证队列存在。 +func declareQueue(ch *amqp.Channel, queue string) error { + _, err := ch.QueueDeclare(queue, true, false, false, false, nil) + if err != nil { + return gerror.Wrapf(err, "rabbitmq declare queue failed, queue:%s", queue) + } + return nil +} + +// declareDelayQueue 声明 TTL 延迟队列，到期后死信转发到目标队列。 +func declareDelayQueue(ch *amqp.Channel, delayQueue, targetQueue string) error { + _, err := ch.QueueDeclare(delayQueue, true, false, false, false, amqp.Table{ + "x-dead-letter-exchange": "", + "x-dead-letter-routing-key": targetQueue, + }) + if err != nil { + return gerror.Wrapf(err, "rabbitmq declare delay queue failed, queue:%s", delayQueue) + } + return nil +} + +// registerCloser 记录消费者 channel，服务退出时统一关闭。 +func registerConsumer(consumer *managedConsumer) { + consumerMu.Lock() + defer consumerMu.Unlock() + consumers = append(consumers, consumer) +} + +func waitForConsumers(ctx context.Context, registeredConsumers []*managedConsumer) error { + for _, consumer := range registeredConsumers { + select { + case <-consumer.done: + case <-ctx.Done(): + return gerror.Wrap(ctx.Err(), "wait for RabbitMQ consumers failed") + } + } + return nil +} + +func closeConnection() error { + connMu.Lock() + defer connMu.Unlock() + if conn == nil || conn.IsClosed() { + conn = nil + return nil + } + if err := conn.Close(); !isClosedError(err) { + return gerror.Wrap(err, "close RabbitMQ connection failed") + } + conn = nil + return nil +} + +func cloneHeaders(headers amqp.Table) map[string]any { + if len(headers) == 0 { + return nil + } + cloned := make(map[string]any, len(headers)) + for key, value := range headers { + cloned[key] = value + } + return cloned +} + +func isClosedError(err error) bool { + return err == nil || errors.Is(err, amqp.ErrClosed) +} diff --git a/internal/library/rabbitmq/rabbitmq_test.go b/internal/library/rabbitmq/rabbitmq_test.go new file mode 100644 index 0000000..8747e87 --- /dev/null +++ b/internal/library/rabbitmq/rabbitmq_test.go @@ -0,0 +1,151 @@ +package rabbitmq + +import ( + "context" + "reflect" + "sync" + "testing" +) + +type fakeChannel struct { + mu sync.Mutex + cancelCalls int + closeCalls int + closed bool +} + +func (f *fakeChannel) Cancel(string, bool) error { + f.mu.Lock() + defer f.mu.Unlock() + f.cancelCalls++ + return nil +} + +func (f *fakeChannel) Close() error { + f.mu.Lock() + defer f.mu.Unlock() + f.closeCalls++ + f.closed = true + return nil +} + +func (f *fakeChannel) IsClosed() bool { + f.mu.Lock() + defer f.mu.Unlock() + return f.closed +} + +func TestCloseIsIdempotent(t *testing.T) { + resetRabbitMQState(t) + + channel := &fakeChannel{} + done := make(chan struct{}) + close(done) + registerConsumer(&managedConsumer{ + channel: channel, + tag: "test-consumer", + done: done, + }) + + if err := Close(context.Background()); err != nil { + t.Fatalf("first Close() error = %v", err) + } + ctx, cancel := context.WithCancel(context.Background()) + cancel() + if err := Close(ctx); err != nil { + t.Fatalf("second Close() error = %v", err) + } + + channel.mu.Lock() + defer channel.mu.Unlock() + if channel.cancelCalls != 1 { + t.Fatalf("Cancel() calls = %d, want 1", channel.cancelCalls) + } + if channel.closeCalls != 1 { + t.Fatalf("Close() calls = %d, want 1", channel.closeCalls) + } +} + +func TestCloseSkipsAlreadyClosedChannel(t *testing.T) { + resetRabbitMQState(t) + + channel := &fakeChannel{closed: true} + done := make(chan struct{}) + close(done) + registerConsumer(&managedConsumer{ + channel: channel, + tag: "closed-consumer", + done: done, + }) + + if err := Close(context.Background()); err != nil { + t.Fatalf("Close() error = %v", err) + } + + channel.mu.Lock() + defer channel.mu.Unlock() + if channel.cancelCalls != 0 || channel.closeCalls != 0 { + t.Fatalf("closed channel calls: Cancel=%d Close=%d, want 0", channel.cancelCalls, channel.closeCalls) + } +} + +func TestCloseHonorsContextWhileWaiting(t *testing.T) { + resetRabbitMQState(t) + + channel := &fakeChannel{} + registerConsumer(&managedConsumer{ + channel: channel, + tag: "blocked-consumer", + done: make(chan struct{}), + }) + + ctx, cancel := context.WithCancel(context.Background()) + cancel() + if err := Close(ctx); err == nil { + t.Fatal("Close() error = nil, want context cancellation error") + } + + channel.mu.Lock() + defer channel.mu.Unlock() + if channel.closeCalls != 1 { + t.Fatalf("Close() calls = %d, want forced cleanup", channel.closeCalls) + } +} + +func TestDeliveryMetadataContext(t *testing.T) { + expected := DeliveryMetadata{ + DeliveryTag: 42, + Redelivered: true, + Consumer: "batch-consumer", + Queue: "iotlcm.batch.execute", + } + ctx := WithDeliveryMetadata(context.Background(), expected) + actual, ok := DeliveryMetadataFromContext(ctx) + if !ok { + t.Fatal("delivery metadata not found in context") + } + if !reflect.DeepEqual(actual, expected) { + t.Fatalf("delivery metadata = %+v, want %+v", actual, expected) + } + if _, ok = DeliveryMetadataFromContext(context.Background()); ok { + t.Fatal("unexpected delivery metadata in empty context") + } +} + +func resetRabbitMQState(t *testing.T) { + t.Helper() + consumerMu.Lock() + consumers = nil + consumerMu.Unlock() + connMu.Lock() + conn = nil + connMu.Unlock() + t.Cleanup(func() { + consumerMu.Lock() + consumers = nil + consumerMu.Unlock() + connMu.Lock() + conn = nil + connMu.Unlock() + }) +} diff --git a/internal/library/tdengine/tdengine.go b/internal/library/tdengine/tdengine.go new file mode 100644 index 0000000..eef40a5 --- /dev/null +++ b/internal/library/tdengine/tdengine.go @@ -0,0 +1,205 @@ +package tdengine + +import ( + "context" + "database/sql" + "sync" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + _ "github.com/taosdata/driver-go/v3/taosRestful" + _ "github.com/taosdata/driver-go/v3/taosWS" +) + +const ( + // dailyStable 和 monthlyStable 是 TDengine 中已经提前创建好的超级表名称。 + // 业务启动时只连接和读写，不负责自动建库建表。 + dailyStable = "t_sim_card_usage_daily" + monthlyStable = "t_sim_card_usage_monthly" + poolDailyStable = "t_flow_pool_usage_daily" + poolMonthlyStable = "t_flow_pool_usage_monthly" + poolCardCountDailyStable = "t_flow_pool_card_count_daily" + poolCardCountMonthlyStable = "t_flow_pool_card_count_monthly" + defaultBatchSize = 200 + // defaultDailyDbName / defaultMonthlyDbName 是配置缺失时的兑底库名。 + // 日流量与月流量分库（duration 不同，而 duration 只能建库时确定）。 + defaultDailyDbName = "iotlcm_daily" + defaultMonthlyDbName = "iotlcm_monthly" +) + +// Config 是项目内 TDengine 连接和流量超级表配置。 +// +// 日流量与月流量分库存储（iotlcm_daily / iotlcm_monthly）：两库的 duration 需要 +// 不同——日流量按天分片、月流量按月分片——而 TDengine 的 duration 只能建库时确定、 +// 事后无法 alter。DbName 保留为兼容位，指向不支持分库的流量池等表的落库位置。 +type Config struct { + Enabled bool + DriverName string + Dsn string + DbName string + DailyDbName string + MonthlyDbName string + DailyStable string + MonthlyStable string + PoolDailyStable string + PoolMonthlyStable string + PoolCardCountDailyStable string + PoolCardCountMonthlyStable string + MaxOpenConns int + MaxIdleConns int + BatchSize int +} + +// pools 按库名缓存连接池。DSN 不携带库名（形如 ws(host:6041)/）， +// 因此同一个 DSN 可以服务多个 database，日/月分库无需两套连接串。 +var ( + poolsMu sync.Mutex + pools = map[string]*sql.DB{} +) + +// GetConfig 从 GoFrame 配置中读取 TDengine 配置。 +// 表名保持在代码中固定，避免业务配置文件里暴露过多 TDengine 内部结构。 +func GetConfig(ctx context.Context) Config { + database := g.Cfg().MustGet(ctx, "tsdb.database", "TdEngine").String() + // dbName 只是连接池的键名与兑底值，不决定实际访问哪个库： + // DSN 末尾留空、库名由 SQL 里的 库名.表名 指定。 + // 默认值取日库，避免兑底到一个不存在的库名。 + dbName := g.Cfg().MustGet(ctx, "tsdb.tdengine.dbName", defaultDailyDbName).String() + // dailyDbName / monthlyDbName 未配置时回退到 dbName。 + dailyDbName := g.Cfg().MustGet(ctx, "tsdb.tdengine.dailyDbName", dbName).String() + monthlyDbName := g.Cfg().MustGet(ctx, "tsdb.tdengine.monthlyDbName", dbName).String() + return Config{ + Enabled: database == "TdEngine", + DriverName: g.Cfg().MustGet(ctx, "tsdb.tdengine.type", "taosWS").String(), + Dsn: g.Cfg().MustGet(ctx, "tsdb.tdengine.dsn", "").String(), + DbName: dbName, + DailyDbName: dailyDbName, + MonthlyDbName: monthlyDbName, + DailyStable: dailyStable, + MonthlyStable: monthlyStable, + PoolDailyStable: poolDailyStable, + PoolMonthlyStable: poolMonthlyStable, + PoolCardCountDailyStable: poolCardCountDailyStable, + PoolCardCountMonthlyStable: poolCardCountMonthlyStable, + MaxOpenConns: g.Cfg().MustGet(ctx, "tsdb.tdengine.maxOpenConns", 100).Int(), + MaxIdleConns: g.Cfg().MustGet(ctx, "tsdb.tdengine.maxIdleConns", 10).Int(), + BatchSize: g.Cfg().MustGet(ctx, "tsdb.tdengine.batchSize", defaultBatchSize).Int(), + } +} + +// DbNameFor 返回指定超级表应该写入的 database 名。 +// +// 日流量与月流量分库：两库的 TDengine duration 不同（日库按天分片、月库按月分片）， +// 而 duration 只能建库时确定。未单独配置 dailyDbName/monthlyDbName 时回退到 DbName。 +func (c Config) DbNameFor(stable string) string { + switch stable { + case c.DailyStable, c.PoolDailyStable, c.PoolCardCountDailyStable: + if c.DailyDbName != "" { + return c.DailyDbName + } + case c.MonthlyStable, c.PoolMonthlyStable, c.PoolCardCountMonthlyStable: + if c.MonthlyDbName != "" { + return c.MonthlyDbName + } + } + return c.DbName +} + +// Enabled 只判断当前项目是否启用 TDengine，并且 DSN 是否已经配置。 +func Enabled(ctx context.Context) bool { + cfg := GetConfig(ctx) + return cfg.Enabled && cfg.Dsn != "" +} + +// DB 懒加载并复用默认库（DbName）的连接池。 +// 注意：这里不会创建 database/stable，库表需要先通过 SQL 脚本准备好。 +func DB(ctx context.Context) (*sql.DB, error) { + return DBForDb(ctx, GetConfig(ctx).DbName) +} + +// DBForDb 返回指定 database 的连接池；同一库名复用同一个池。 +func DBForDb(ctx context.Context, dbName string) (*sql.DB, error) { + cfg := GetConfig(ctx) + if !cfg.Enabled { + return nil, gerror.New("tdengine is disabled") + } + if cfg.Dsn == "" { + return nil, gerror.New("tdengine dsn is empty") + } + if dbName == "" { + dbName = cfg.DbName + } + + poolsMu.Lock() + defer poolsMu.Unlock() + if pool, ok := pools[dbName]; ok { + return pool, nil + } + + driverName := cfg.DriverName + if driverName == "" { + driverName = "taosWS" + } + pool, err := sql.Open(driverName, cfg.Dsn) + if err != nil { + return nil, err + } + pool.SetMaxIdleConns(cfg.MaxIdleConns) + pool.SetMaxOpenConns(cfg.MaxOpenConns) + if err := pool.PingContext(ctx); err != nil { + _ = pool.Close() + return nil, err + } + pools[dbName] = pool + return pool, nil +} + +// Exec 执行 TDengine 默认库的写入、更新、删除类 SQL。 +func Exec(ctx context.Context, query string, args ...any) (sql.Result, error) { + conn, err := DB(ctx) + if err != nil { + return nil, err + } + return conn.ExecContext(ctx, query, args...) +} + +// ExecForDb 在指定 database 的连接池上执行写入类 SQL。 +func ExecForDb(ctx context.Context, dbName, query string, args ...any) (sql.Result, error) { + conn, err := DBForDb(ctx, dbName) + if err != nil { + return nil, err + } + return conn.ExecContext(ctx, query, args...) +} + +// Query 执行 TDengine 查询 SQL，调用方负责关闭返回的 rows。 +func Query(ctx context.Context, query string, args ...any) (*sql.Rows, error) { + conn, err := DB(ctx) + if err != nil { + return nil, err + } + return conn.QueryContext(ctx, query, args...) +} + +// QueryRow 执行 TDengine 单行查询 SQL。 +func QueryRow(ctx context.Context, query string, args ...any) (*sql.Row, error) { + conn, err := DB(ctx) + if err != nil { + return nil, err + } + return conn.QueryRowContext(ctx, query, args...), nil +} + +// Close 在应用退出时关闭所有 TDengine 连接池。 +func Close() error { + poolsMu.Lock() + defer poolsMu.Unlock() + var firstErr error + for name, pool := range pools { + if err := pool.Close(); err != nil && firstErr == nil { + firstErr = err + } + delete(pools, name) + } + return firstErr +} diff --git a/internal/model/channel_refresh.go b/internal/model/channel_refresh.go new file mode 100644 index 0000000..9f539ac --- /dev/null +++ b/internal/model/channel_refresh.go @@ -0,0 +1,11 @@ +package model + +// ChannelRefreshMessage requests an asynchronous refresh of channel instances. +// The payload is shared with admin-svr's publisher. +type ChannelRefreshMessage struct { + Type string `json:"type"` + ChanKey string `json:"chanKey,omitempty"` + TempKey string `json:"tempKey,omitempty"` + OldChanKey string `json:"oldChanKey,omitempty"` + NewChanKey string `json:"newChanKey,omitempty"` +} diff --git a/internal/model/do/batch_detail.go b/internal/model/do/batch_detail.go new file mode 100644 index 0000000..92e4277 --- /dev/null +++ b/internal/model/do/batch_detail.go @@ -0,0 +1,32 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package do + +import ( + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" +) + +// BatchDetail is the golang structure of table batch_detail for DAO operations like Where/Data. +type BatchDetail struct { + g.Meta `orm:"table:batch_detail, do:true"` + Id any // 主键ID + BatchNo any // 批次编号 + BatchType any // 批次类型 枚举（与主表一致） + DetailStatus any // 明细异步处理状态 0-待处理 1-处理中 2-成功 3-失败 + BlankOutStatus any // 作废状态 0-是 1-否 + RowNo any // 行号 + ItemId any // 项ID：卡ICCID/设备编码 + ErrorCode any // 异常编码 + ErrorReason any // 异常原因 + Snapshot *gjson.Json // 信息快照 + TenantId any // 租户ID + CreatedDept any // 创建部门ID + CreatedBy any // 创建者ID + CreatedAt *gtime.Time // 创建时间 + UpdatedBy any // 更新者ID + UpdatedAt *gtime.Time // 更新时间 +} diff --git a/internal/model/do/batch_main.go b/internal/model/do/batch_main.go new file mode 100644 index 0000000..4b3eb94 --- /dev/null +++ b/internal/model/do/batch_main.go @@ -0,0 +1,47 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package do + +import ( + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" +) + +// BatchMain is the golang structure of table batch_main for DAO operations like Where/Data. +type BatchMain struct { + g.Meta `orm:"table:batch_main, do:true"` + Id any // 主键ID + BatchNo any // 批次编号 + OrderSn any // 订单编号 + BatchType any // 批次类型 枚举 + BatchStatus any // 批次状态 枚举 + BatchAsyncStatus any // 异步处理状态 + SourceType any // 来源方式 枚举 + SourceBatchNo any // 来源批次编号 + SourceOssId any // 源文件OSS ID + ResultOssId any // 结果文件OSS ID + IsCrossDept any // 是否跨级操作机构 + TargetDeptId any // 目标机构编号 + DeptId any // 机构编号 + TotalCount any // 总数量 + ValidCount any // 有效数量 + SuccessCount any // 成功数量 + FailCount any // 失败数量 + RunResult any // 运行结果 + FailReason any // 失败原因 + BatchInfo *gjson.Json // 批次信息：入库/出库展示字段 + Remark *gjson.Json // 备注扩展字段 + StartTime *gtime.Time // 开始时间 + EndTime *gtime.Time // 结束时间 + TenantId any // 租户ID + CreatedDept any // 创建部门ID + CreatedBy any // 创建者ID + CreatedAt *gtime.Time // 创建时间 + UpdatedBy any // 更新者ID + UpdatedAt *gtime.Time // 更新时间 + SourceChannel any // 发起渠道:console/mini_program/api/system/legacy + ExecutionMode any // 操作规模:0未知1单卡2批量 +} diff --git a/internal/model/do/sys_dept.go b/internal/model/do/sys_dept.go new file mode 100644 index 0000000..afe65aa --- /dev/null +++ b/internal/model/do/sys_dept.go @@ -0,0 +1,39 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package do + +import ( + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" +) + +// SysDept is the golang structure of table sys_dept for DAO operations like Where/Data. +type SysDept struct { + g.Meta `orm:"table:sys_dept, do:true"` + DeptId any // 部门id + TenantId any // 租户编号 + ParentId any // 父部门id + Ancestors any // 祖级列表 + DeptKey any // 机构编号 + DeptName any // 部门名称 + DeptType any // 部门类型(0:部门 1:公司) + DeptCategory any // 部门类别编码 + OrderNum any // 显示顺序 + Leader any // 负责人 + Phone any // 联系电话 + Email any // 邮箱 + Status any // 部门状态（0正常 1停用） + CreditCode any // 统一社会信用代码 + AreaCode any // 行政区域代码 + Address any // 地址 + DeptRole any // 机构默认角色ID + CreatedDept any // 创建部门 + CreatedBy any // 创建者 + CreatedAt *gtime.Time // 创建时间 + UpdatedBy any // 更新者 + UpdatedAt *gtime.Time // 更新时间 + DeletedBy any // 删除人 + DeletedAt *gtime.Time // 删除时间 +} diff --git a/internal/model/do/sys_oss.go b/internal/model/do/sys_oss.go new file mode 100644 index 0000000..a5f05b4 --- /dev/null +++ b/internal/model/do/sys_oss.go @@ -0,0 +1,32 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package do + +import ( + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" +) + +// SysOss is the golang structure of table sys_oss for DAO operations like Where/Data. +type SysOss struct { + g.Meta `orm:"table:sys_oss, do:true"` + OssId any // 对象存储主键 + TenantId any // 租户编号 + FileName any // 文件名 + OriginalName any // 原名 + FileSuffix any // 文件后缀名 + Path any // 存储路径 + Url any // URL地址 + CreatedDept any // 创建部门 + CreatedAt *gtime.Time // 创建时间 + CreatedBy any // 创建者 + UpdatedAt *gtime.Time // 更新时间 + UpdatedBy any // 更新者 + Service any // 服务商 + Md5 any // 文件MD5 + FileSize any // 文件大小 + FileCrc16 any // 文件Crc16 + FileSum any // 文件校验和 +} diff --git a/internal/model/do/t_message_log.go b/internal/model/do/t_message_log.go new file mode 100644 index 0000000..c60da08 --- /dev/null +++ b/internal/model/do/t_message_log.go @@ -0,0 +1,31 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package do + +import ( + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" +) + +// TMessageLog is the golang structure of table t_message_log for DAO operations like Where/Data. +type TMessageLog struct { + g.Meta `orm:"table:t_message_log, do:true"` + MessageLogId any // 主键ID + MessageType any // 消息类型 + SendUserType any // 发送用户类型 + UserIds *gjson.Json // 接收用户ID集合(JSON数组) + MessageTitle any // 消息标题 + MessageContent any // 消息内容 + MessageLink any // 跳转链接地址 + SendTime *gtime.Time // 发送时间 + IsRecall any // 是否撤回 + TenantId any // 租户ID + CreatedDept any // 创建部门 + CreatedBy any // 创建者 + CreatedAt *gtime.Time // 创建时间 + UpdatedBy any // 更新者 + UpdatedAt *gtime.Time // 更新时间 +} diff --git a/internal/model/do/t_resource_batch_download.go b/internal/model/do/t_resource_batch_download.go new file mode 100644 index 0000000..1478714 --- /dev/null +++ b/internal/model/do/t_resource_batch_download.go @@ -0,0 +1,48 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package do + +import ( + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" +) + +// TResourceBatchDownload is the golang structure of table t_resource_batch_download for DAO operations like Where/Data. +type TResourceBatchDownload struct { + g.Meta `orm:"table:t_resource_batch_download, do:true"` + Id any // 主键ID + BatchNo any // 批次编号（全局唯一） + TaskName any // 任务名称 + ResourceType any // 资源类型 枚举 + ExportScene any // 导出场景 枚举 + QueryParams *gjson.Json // 导出查询条件快照 + TotalCount any // 预计导出总条数 + SuccessCount any // 已成功导出条数 + FailCount any // 失败条数 + DownloadStatus any // 下载状态 枚举 + OssId any // OSS文件ID + FileName any // 导出文件名 + FileExt any // 文件扩展名 + FileUrl any // 文件访问地址 + FilePath any // 服务器本地路径 + FileSize any // 文件大小（字节） + FileMd5 any // 文件MD5 + DownloadCount any // 下载次数 + LastDownloadTime *gtime.Time // 最后下载时间 + MaxDownloadCount any // 最大下载次数限制 + ErrorMessage any // 失败原因 + DeptId any // 所属机构ID + StartTime *gtime.Time // 任务开始时间 + FinishTime *gtime.Time // 任务完成时间 + ExpireTime *gtime.Time // 文件过期时间 + Remark any // 备注 + TenantId any // 租户ID + CreatedDept any // 创建部门ID + CreatedBy any // 创建者ID + CreatedAt *gtime.Time // 创建时间 + UpdatedBy any // 更新者ID + UpdatedAt *gtime.Time // 更新时间 +} diff --git a/internal/model/do/t_sim_card_cancellation.go b/internal/model/do/t_sim_card_cancellation.go new file mode 100644 index 0000000..4c9caa3 --- /dev/null +++ b/internal/model/do/t_sim_card_cancellation.go @@ -0,0 +1,72 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package do + +import ( + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" +) + +// TSimCardCancellation is the golang structure of table t_sim_card_cancellation for DAO operations like Where/Data. +type TSimCardCancellation struct { + g.Meta `orm:"table:t_sim_card_cancellation, do:true"` + Id any // 主键ID + Iccid any // ICCID + Msisdn any // MSISDN/卡号 + Imsi any // IMSI + Imei any // IMEI + RegionCode any // 归属地 + DeptId any // 渠道客户ID + CurrentPackageCode any // 当前套餐编号 + PackageStatus any // 套餐状态：0 无套餐，1待生效，2生效中，3已过期 + AccessType any // 接入方式：ipv4、ipv6、ipv4_ipv6 + BossCardStatus any // 运营商卡状态 枚举 + CancelBatchNo any // 系统销户批次号 + CancelledAt *gtime.Time // 系统销户完成时间 + CardType any // 卡类型 枚举 + BossOpenAt *gtime.Time // 运营商开户日期 + BossActivateAt *gtime.Time // 运营商激活日期 + IsBossActivateEnabled any // 是否按运营商激活减去赠送时长方：1是，0否 + InboundBatchNo any // 入库批次 + OutboundBatchNo any // 出库批次 + AllocationBatchNo any // 划拨批次 + IsFlowPool any // 是否为池卡：1是，0否 + FlowPoolCode any // 流量池编号 + FlowPkgBeginAt *gtime.Time // 当前套餐生效时间 + FlowPkgPresentAt *gtime.Time // 套餐赠送结束时间 + FlowPkgRealPresentAt *gtime.Time // 实际赠送结束时间 + FlowPkgEndAt *gtime.Time // 当前套餐预计到期时间 + DataServiceStatus any // 数据服务状态：0启用,1停用 + PackageTotalFlowMb any // 套餐总流量MB + UsedFlowMb any // 已用流量MB + TrafficCutoffStatus any // 达量断网配置状态：0未配置，1已开启，2已关闭 + TrafficCutoffThreshold any // 达量断网阈值 + SmsServiceStatus any // 短信服务状态：0启用,1停用 + PackageTotalSms any // 套餐短信总条数 + UsedSms any // 已用短信条数 + VoiceServiceStatus any // 语音服务状态：0启用,1停用 + PackageTotalVoice any // 套餐语音总分钟数 + UsedVoice any // 已用语音分钟数 + RoamServiceStatus any // 国漫服务状态：0启用,1停用 + PackageTotalRoamFlowMb any // 套餐国际流量MB + UsedRoamFlowMb any // 已用国际流量MB + StatusSyncTime *gtime.Time // 状态最后一次同步时间 + FlowSyncTime *gtime.Time // 流量最后一次同步时间 + SmsSyncTime *gtime.Time // 短信最后一次同步时间 + VoiceSyncTime *gtime.Time // 语音最后一次同步时间 + RoamSyncTime *gtime.Time // 国漫最后一次同步时间 + ExtraFeeEnabled any // 是否收取附加费：0是，1否 + ExtraFeeMultiple any // 附加费用倍数 + ExtraFee any // 附加金额 + ChanKey any // 通道编号 + NetModel any // 网络类型 枚举 + Remark any // 备注 + TenantId any // 租户ID + CreatedDept any // 创建部门 + CreatedBy any // 创建者 + CreatedAt *gtime.Time // 创建时间 + UpdatedBy any // 更新者 + UpdatedAt *gtime.Time // 更新时间 +} diff --git a/internal/model/do/t_sim_card_info.go b/internal/model/do/t_sim_card_info.go new file mode 100644 index 0000000..fc86863 --- /dev/null +++ b/internal/model/do/t_sim_card_info.go @@ -0,0 +1,73 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package do + +import ( + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" +) + +// TSimCardInfo is the golang structure of table t_sim_card_info for DAO operations like Where/Data. +type TSimCardInfo struct { + g.Meta `orm:"table:t_sim_card_info, do:true"` + Id any // 主键ID + Iccid any // ICCID + Msisdn any // MSISDN/卡号 + Imsi any // IMSI + Imei any // IMEI + RegionCode any // 归属地 + DeptId any // 渠道客户ID + CurrentPackageCode any // 当前套餐编号 + PackageStatus any // 套餐状态：0 无套餐，1待生效，2生效中，3已过期 + AccessType any // 接入方式：ipv4、ipv6、ipv4_ipv6 + BossCardStatus any // 运营商卡状态 枚举 + CardType any // 卡类型 枚举 + BossOpenAt *gtime.Time // 运营商开户日期 + BossActivateAt *gtime.Time // 运营商激活日期 + IsBossActivateEnabled any // 是否按运营商激活减去赠送时长方：1是，0否 + InboundBatchNo any // 入库批次 + OutboundBatchNo any // 出库批次 + AllocationBatchNo any // 划拨批次 + IsFlowPool any // 是否为池卡：1是，0否 + FlowPoolCode any // 流量池编号 + OutStockTime *gtime.Time // 出库时间 + FlowPkgBeginAt *gtime.Time // 当前套餐生效时间 + FlowPkgPresentAt *gtime.Time // 套餐赠送结束时间 + FlowPkgRealPresentAt *gtime.Time // 实际赠送结束时间 + FlowPkgEndAt *gtime.Time // 当前套餐预计到期时间 + DataServiceStatus any // 数据服务状态：1启用、0禁用 + PackageTotalFlowMb any // 套餐总流量MB + UsedFlowMb any // 已用流量MB + TrafficCutoffStatus any // 达量断网配置状态：0未配置，1已开启，2已关闭 + TrafficCutoffThreshold any // 达量断网阈值 + SmsServiceStatus any // 短信服务状态：1启用、0禁用 + PackageTotalSms any // 套餐短信总条数 + UsedSms any // 已用短信条数 + VoiceServiceStatus any // 语音服务状态：1启用、0禁用 + PackageTotalVoice any // 套餐语音总分钟数 + UsedVoice any // 已用语音分钟数 + RoamServiceStatus any // 国漫服务状态：1启用、0禁用 + PackageTotalRoamFlowMb any // 套餐国际流量MB + UsedRoamFlowMb any // 已用国际流量MB + StatusSyncTime *gtime.Time // 状态最后一次同步时间 + FlowSyncTime *gtime.Time // 流量最后一次同步时间 + SmsSyncTime *gtime.Time // 短信最后一次同步时间 + VoiceSyncTime *gtime.Time // 语音最后一次同步时间 + RoamSyncTime *gtime.Time // 国漫最后一次同步时间 + ImeiSyncTime *gtime.Time // IMEI 最后一次轮询同步时间 + ImeiUpdateTime *gtime.Time // IMEI 值最后一次实际变更时间 + ExtraFeeEnabled any // 是否收取附加费：1是，0否 + ExtraFeeMultiple any // 附加费用倍数 + ExtraFee any // 附加金额 + ChanKey any // 通道编号 + NetModel any // 网络类型：4G、5G、NB-IoT + Remark any // 备注 + TenantId any // 租户ID + CreatedDept any // 创建部门 + CreatedBy any // 创建者 + CreatedAt *gtime.Time // 创建时间 + UpdatedBy any // 更新者 + UpdatedAt *gtime.Time // 更新时间 +} diff --git a/internal/model/do/t_sim_card_traffic_cutoff_extension.go b/internal/model/do/t_sim_card_traffic_cutoff_extension.go new file mode 100644 index 0000000..4d217ce --- /dev/null +++ b/internal/model/do/t_sim_card_traffic_cutoff_extension.go @@ -0,0 +1,15 @@ +package do + +import "github.com/gogf/gf/v2/frame/g" + +// TSimCardTrafficCutoffConfig contains writable traffic-cutoff extension fields. +type TSimCardTrafficCutoffConfig struct { + g.Meta `orm:"table:t_sim_card_info, do:true"` + DataServiceStatus any `orm:"data_service_status"` + StatusSyncTime any `orm:"status_sync_time"` + TrafficCutoffStatus any `orm:"traffic_cutoff_status"` + TrafficCutoffThreshold any `orm:"traffic_cutoff_threshold"` + TrafficCutoffCrossMonthHold any `orm:"traffic_cutoff_cross_month_hold"` + TrafficCutoffTriggeredAt any `orm:"traffic_cutoff_triggered_at"` + UpdatedBy any `orm:"updated_by"` +} diff --git a/internal/model/do/t_sim_card_traffic_cutoff_extension_test.go b/internal/model/do/t_sim_card_traffic_cutoff_extension_test.go new file mode 100644 index 0000000..0e73acb --- /dev/null +++ b/internal/model/do/t_sim_card_traffic_cutoff_extension_test.go @@ -0,0 +1,19 @@ +package do + +import ( + "reflect" + "strings" + "testing" +) + +func TestTSimCardTrafficCutoffConfigUsesDoMetadata(t *testing.T) { + modelType := reflect.TypeOf(TSimCardTrafficCutoffConfig{}) + metaField, ok := modelType.FieldByName("Meta") + if !ok { + t.Fatal("TSimCardTrafficCutoffConfig must embed g.Meta") + } + ormTag := metaField.Tag.Get("orm") + if !strings.Contains(ormTag, "table:t_sim_card_info") || !strings.Contains(ormTag, "do:true") { + t.Fatalf("unexpected g.Meta orm tag: %q", ormTag) + } +} diff --git a/internal/model/do/t_sim_flow_pool.go b/internal/model/do/t_sim_flow_pool.go new file mode 100644 index 0000000..ae01873 --- /dev/null +++ b/internal/model/do/t_sim_flow_pool.go @@ -0,0 +1,37 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package do + +import ( + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" +) + +// TSimFlowPool is the golang structure of table t_sim_flow_pool for DAO operations like Where/Data. +type TSimFlowPool struct { + g.Meta `orm:"table:t_sim_flow_pool, do:true"` + Id any // 主键ID + FlowPoolCode any // 流量池编号 + DeptId any // 客户编号 + PackageCodes *gjson.Json // 套餐编号列表（冗余） + PoolPackageCode any // 池套餐编码 + CardCount any // 关联卡数量 + CardFlowMb any // 套餐卡流量MB + PoolPackageFlowMb any // 套内总流量MB + AddOnFlowMb any // 加油包总流量MB + TotalFlowMb any // 总流量MB(套内+加油包) + UsedFlowMb any // 已使用流量MB + RemainFlowMb any // 剩余流量MB + LastStatTime *gtime.Time // 最后流量统计时间 + PoolStatus any // 状态：0 正常、1 停用、2 锁定 + Remark any // 备注 + TenantId any // 租户ID + CreatedDept any // 创建部门 + CreatedBy any // 创建者 + CreatedAt *gtime.Time // 创建时间 + UpdatedBy any // 更新者 + UpdatedAt *gtime.Time // 更新时间 +} diff --git a/internal/model/do/t_user_message.go b/internal/model/do/t_user_message.go new file mode 100644 index 0000000..2f661dc --- /dev/null +++ b/internal/model/do/t_user_message.go @@ -0,0 +1,25 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package do + +import ( + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" +) + +// TUserMessage is the golang structure of table t_user_message for DAO operations like Where/Data. +type TUserMessage struct { + g.Meta `orm:"table:t_user_message, do:true"` + MessageId any // 主键ID + MessageLogId any // 消息日志ID + UserId any // 用户ID + IsRead any // 是否已读 + TenantId any // 租户ID + CreatedDept any // 创建部门 + CreatedBy any // 创建者 + CreatedAt *gtime.Time // 创建时间 + UpdatedBy any // 更新者 + UpdatedAt *gtime.Time // 更新时间 +} diff --git a/internal/model/do/t_warn_config.go b/internal/model/do/t_warn_config.go new file mode 100644 index 0000000..35bf3e3 --- /dev/null +++ b/internal/model/do/t_warn_config.go @@ -0,0 +1,38 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package do + +import ( + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" +) + +// TWarnConfig is the golang structure of table t_warn_config for DAO operations like Where/Data. +type TWarnConfig struct { + g.Meta `orm:"table:t_warn_config, do:true"` + Id any // 主键ID + TenantId any // 租户ID + BatchNo any // 批次编号 + ItemType any // 对象类型:1卡,2流量池 + ItemId any // 对象ID:单卡为t_sim_card_info.id,流量池为t_sim_flow_pool.id + ItemCode any // 对象编码快照:ICCID/MSISDN/流量池编码 + WarnUnit any // 预警单位:1MB,2百分比,3天数,4月数 + FirstWarn any // 第一档预警阈值,单位由warn_unit决定 + SecondWarn any // 第二档预警阈值,单位由warn_unit决定 + ThirdWarn any // 第三档预警阈值,单位由warn_unit决定 + WarnMethods any // 预警方式:0短信,1邮箱,2电话,3平台消息;多选逗号分隔 + WarnPhones any // 预警手机号,多个逗号分隔 + WarnEmails any // 预警邮箱,多个逗号分隔 + AutoAction any // 自动动作:0无,2断网;第一期只保存配置不自动执行 + IsEnabled any // 是否启用:1启用,0停用 + WxTemplateId any // 微信订阅消息模板ID + Remark any // 备注 + CreatedDept any // 创建部门 + CreatedBy any // 创建者 + CreatedAt *gtime.Time // 创建时间 + UpdatedBy any // 更新者 + UpdatedAt *gtime.Time // 更新时间 + AppNotifyEnabled any // 是否开启小程序预警通知:0关闭,1开启 +} diff --git a/internal/model/do/t_warn_log.go b/internal/model/do/t_warn_log.go new file mode 100644 index 0000000..322576c --- /dev/null +++ b/internal/model/do/t_warn_log.go @@ -0,0 +1,39 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package do + +import ( + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" +) + +// TWarnLog is the golang structure of table t_warn_log for DAO operations like Where/Data. +type TWarnLog struct { + g.Meta `orm:"table:t_warn_log, do:true"` + Id any // 主键ID + TenantId any // 租户ID + ConfigId any // 预警配置ID + ItemType any // 对象类型:1单卡,2流量池 + ItemId any // 对象ID:单卡为t_sim_card_info.id,流量池为t_sim_flow_pool.id + ItemCode any // 对象编码快照:ICCID/MSISDN/流量池编码 + WarnStage any // 触发档位:1第一档,2第二档,3第三档 + WarnLevel any // 预警级别:1提示,2注意,3警告,4危险,5紧急 + WarnUnit any // 预警单位/维度:1MB,2百分比,3天数,4月数 + ThresholdValue any // 触发阈值,单位由warn_unit决定 + ActualValue any // 触发时实际值,单位由warn_unit决定 + TotalFlowMb any // 触发时总流量MB;流量类预警使用 + RemainFlowMb any // 触发时剩余流量MB;流量类预警使用 + WarnMethods any // 实际预警方式快照 + WarnPhones any // 实际预警手机号快照 + WarnEmails any // 实际预警邮箱快照 + MessageLogId any // 平台消息ID;聚合通知时可能为空 + WarnTime *gtime.Time // 预警时间 + Remark any // 备注 + CreatedDept any // 创建部门 + CreatedBy any // 创建者 + CreatedAt *gtime.Time // 创建时间 + UpdatedBy any // 更新者 + UpdatedAt *gtime.Time // 更新时间 +} diff --git a/internal/model/do/t_wo_main.go b/internal/model/do/t_wo_main.go new file mode 100644 index 0000000..831c38d --- /dev/null +++ b/internal/model/do/t_wo_main.go @@ -0,0 +1,45 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package do + +import ( + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" +) + +// TWoMain is the golang structure of table t_wo_main for DAO operations like Where/Data. +type TWoMain struct { + g.Meta `orm:"table:t_wo_main, do:true"` + Id any // 主键ID + WoNo any // 工单编号 + WoType any // 工单类型 0卡,1模组,2设备,3订单,4余额,5提现 + RelatedId any // 关联业务ID + Title any // 工单标题 + Content any // 问题描述 + Attachment any // 附件 + CreatorType any // 创建人类型：1终端用户，2代理商 + CreatorDeptId any // 创建机构ID + CreatorUserId any // 创建用户ID + CreatorName any // 创建人姓名 + CreatorPhone any // 创建人手机号 + AssignDeptId any // 分派方机构ID + AssignUserId any // 分派人用户ID + CurrentStatus any // 当前处理状态：1已分派，2处理中，3已解决，4已完结_关单，5已取消 + HandlerDeptId any // 处理方机构ID + HandlerUserId any // 处理人用户ID + FirstResponseTime *gtime.Time // 首次响应时间 + ResolveTime *gtime.Time // 确认时间 + CloseTime *gtime.Time // 关闭时间 + SatisfactionLevel any // 满意度：1-5分 + EvalContent any // 评价内容 + EvalTime *gtime.Time // 评价时间 + Remark any // 备注 + TenantId any // 租户ID + CreatedDept any // 创建部门 + CreatedBy any // 创建者 + CreatedAt *gtime.Time // 创建时间 + UpdatedBy any // 更新者 + UpdatedAt *gtime.Time // 更新时间 +} diff --git a/internal/model/do/t_wo_record.go b/internal/model/do/t_wo_record.go new file mode 100644 index 0000000..3e5bd9b --- /dev/null +++ b/internal/model/do/t_wo_record.go @@ -0,0 +1,30 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package do + +import ( + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" +) + +// TWoRecord is the golang structure of table t_wo_record for DAO operations like Where/Data. +type TWoRecord struct { + g.Meta `orm:"table:t_wo_record, do:true"` + Id any // 主键ID + WoNo any // 工单编号 + OperationType any // 操作类型：0创建，1分派，2处理，3完成，4关单，5取消 + OperationDeptId any // 操作机构ID + OperationUserId any // 操作用户ID + TargetDeptId any // 目标部门ID + TargetUserId any // 目标用户ID + BeforeStatus any // 变更前状态 + AfterStatus any // 变更后状态 + TenantId any // 租户ID + CreatedDept any // 创建部门 + CreatedBy any // 创建者 + CreatedAt *gtime.Time // 创建时间 + UpdatedBy any // 更新者 + UpdatedAt *gtime.Time // 更新时间 +} diff --git a/internal/model/entity/batch_detail.go b/internal/model/entity/batch_detail.go new file mode 100644 index 0000000..b301e4d --- /dev/null +++ b/internal/model/entity/batch_detail.go @@ -0,0 +1,30 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package entity + +import ( + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" +) + +// BatchDetail is the golang structure for table batch_detail. +type BatchDetail struct { + Id int64 `json:"id" orm:"id" description:"主键ID"` + BatchNo string `json:"batchNo" orm:"batch_no" description:"批次编号"` + BatchType int `json:"batchType" orm:"batch_type" description:"批次类型 枚举（与主表一致）"` + DetailStatus int `json:"detailStatus" orm:"detail_status" description:"明细异步处理状态 0-待处理 1-处理中 2-成功 3-失败"` + BlankOutStatus int `json:"blankOutStatus" orm:"blank_out_status" description:"作废状态 0-是 1-否"` + RowNo int `json:"rowNo" orm:"row_no" description:"行号"` + ItemId string `json:"itemId" orm:"item_id" description:"项ID：卡ICCID/设备编码"` + ErrorCode string `json:"errorCode" orm:"error_code" description:"异常编码"` + ErrorReason string `json:"errorReason" orm:"error_reason" description:"异常原因"` + Snapshot *gjson.Json `json:"snapshot" orm:"snapshot" description:"信息快照"` + TenantId string `json:"tenantId" orm:"tenant_id" description:"租户ID"` + CreatedDept int64 `json:"createdDept" orm:"created_dept" description:"创建部门ID"` + CreatedBy int64 `json:"createdBy" orm:"created_by" description:"创建者ID"` + CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" description:"创建时间"` + UpdatedBy int64 `json:"updatedBy" orm:"updated_by" description:"更新者ID"` + UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" description:"更新时间"` +} diff --git a/internal/model/entity/batch_main.go b/internal/model/entity/batch_main.go new file mode 100644 index 0000000..3d2c057 --- /dev/null +++ b/internal/model/entity/batch_main.go @@ -0,0 +1,45 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package entity + +import ( + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" +) + +// BatchMain is the golang structure for table batch_main. +type BatchMain struct { + Id int64 `json:"id" orm:"id" description:"主键ID"` + BatchNo string `json:"batchNo" orm:"batch_no" description:"批次编号"` + OrderSn string `json:"orderSn" orm:"order_sn" description:"订单编号"` + BatchType int `json:"batchType" orm:"batch_type" description:"批次类型 枚举"` + BatchStatus int `json:"batchStatus" orm:"batch_status" description:"批次状态 枚举"` + BatchAsyncStatus int `json:"batchAsyncStatus" orm:"batch_async_status" description:"异步处理状态"` + SourceType int `json:"sourceType" orm:"source_type" description:"来源方式 枚举"` + SourceBatchNo string `json:"sourceBatchNo" orm:"source_batch_no" description:"来源批次编号"` + SourceOssId int64 `json:"sourceOssId" orm:"source_oss_id" description:"源文件OSS ID"` + ResultOssId int64 `json:"resultOssId" orm:"result_oss_id" description:"结果文件OSS ID"` + IsCrossDept int `json:"isCrossDept" orm:"is_cross_dept" description:"是否跨级操作机构"` + TargetDeptId int64 `json:"targetDeptId" orm:"target_dept_id" description:"目标机构编号"` + DeptId int64 `json:"deptId" orm:"dept_id" description:"机构编号"` + TotalCount int `json:"totalCount" orm:"total_count" description:"总数量"` + ValidCount int `json:"validCount" orm:"valid_count" description:"有效数量"` + SuccessCount int `json:"successCount" orm:"success_count" description:"成功数量"` + FailCount int `json:"failCount" orm:"fail_count" description:"失败数量"` + RunResult int `json:"runResult" orm:"run_result" description:"运行结果"` + FailReason string `json:"failReason" orm:"fail_reason" description:"失败原因"` + BatchInfo *gjson.Json `json:"batchInfo" orm:"batch_info" description:"批次信息：入库/出库展示字段"` + Remark *gjson.Json `json:"remark" orm:"remark" description:"备注扩展字段"` + StartTime *gtime.Time `json:"startTime" orm:"start_time" description:"开始时间"` + EndTime *gtime.Time `json:"endTime" orm:"end_time" description:"结束时间"` + TenantId string `json:"tenantId" orm:"tenant_id" description:"租户ID"` + CreatedDept int64 `json:"createdDept" orm:"created_dept" description:"创建部门ID"` + CreatedBy int64 `json:"createdBy" orm:"created_by" description:"创建者ID"` + CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" description:"创建时间"` + UpdatedBy int64 `json:"updatedBy" orm:"updated_by" description:"更新者ID"` + UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" description:"更新时间"` + SourceChannel string `json:"sourceChannel" orm:"source_channel" description:"发起渠道:console/mini_program/api/system/legacy"` + ExecutionMode uint `json:"executionMode" orm:"execution_mode" description:"操作规模:0未知1单卡2批量"` +} diff --git a/internal/model/entity/sys_dept.go b/internal/model/entity/sys_dept.go new file mode 100644 index 0000000..605de83 --- /dev/null +++ b/internal/model/entity/sys_dept.go @@ -0,0 +1,37 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package entity + +import ( + "github.com/gogf/gf/v2/os/gtime" +) + +// SysDept is the golang structure for table sys_dept. +type SysDept struct { + DeptId int64 `json:"deptId" orm:"dept_id" description:"部门id"` + TenantId string `json:"tenantId" orm:"tenant_id" description:"租户编号"` + ParentId int64 `json:"parentId" orm:"parent_id" description:"父部门id"` + Ancestors string `json:"ancestors" orm:"ancestors" description:"祖级列表"` + DeptKey string `json:"deptKey" orm:"dept_key" description:"机构编号"` + DeptName string `json:"deptName" orm:"dept_name" description:"部门名称"` + DeptType int `json:"deptType" orm:"dept_type" description:"部门类型(0:部门 1:公司)"` + DeptCategory string `json:"deptCategory" orm:"dept_category" description:"部门类别编码"` + OrderNum int `json:"orderNum" orm:"order_num" description:"显示顺序"` + Leader int64 `json:"leader" orm:"leader" description:"负责人"` + Phone string `json:"phone" orm:"phone" description:"联系电话"` + Email string `json:"email" orm:"email" description:"邮箱"` + Status string `json:"status" orm:"status" description:"部门状态（0正常 1停用）"` + CreditCode string `json:"creditCode" orm:"credit_code" description:"统一社会信用代码"` + AreaCode string `json:"areaCode" orm:"area_code" description:"行政区域代码"` + Address string `json:"address" orm:"address" description:"地址"` + DeptRole int64 `json:"deptRole" orm:"dept_role" description:"机构默认角色ID"` + CreatedDept int64 `json:"createdDept" orm:"created_dept" description:"创建部门"` + CreatedBy int64 `json:"createdBy" orm:"created_by" description:"创建者"` + CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" description:"创建时间"` + UpdatedBy int64 `json:"updatedBy" orm:"updated_by" description:"更新者"` + UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" description:"更新时间"` + DeletedBy int64 `json:"deletedBy" orm:"deleted_by" description:"删除人"` + DeletedAt *gtime.Time `json:"deletedAt" orm:"deleted_at" description:"删除时间"` +} diff --git a/internal/model/entity/sys_oss.go b/internal/model/entity/sys_oss.go new file mode 100644 index 0000000..5e6fbec --- /dev/null +++ b/internal/model/entity/sys_oss.go @@ -0,0 +1,30 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package entity + +import ( + "github.com/gogf/gf/v2/os/gtime" +) + +// SysOss is the golang structure for table sys_oss. +type SysOss struct { + OssId int64 `json:"ossId" orm:"oss_id" description:"对象存储主键"` + TenantId string `json:"tenantId" orm:"tenant_id" description:"租户编号"` + FileName string `json:"fileName" orm:"file_name" description:"文件名"` + OriginalName string `json:"originalName" orm:"original_name" description:"原名"` + FileSuffix string `json:"fileSuffix" orm:"file_suffix" description:"文件后缀名"` + Path string `json:"path" orm:"path" description:"存储路径"` + Url string `json:"url" orm:"url" description:"URL地址"` + CreatedDept int64 `json:"createdDept" orm:"created_dept" description:"创建部门"` + CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" description:"创建时间"` + CreatedBy int64 `json:"createdBy" orm:"created_by" description:"创建者"` + UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" description:"更新时间"` + UpdatedBy int64 `json:"updatedBy" orm:"updated_by" description:"更新者"` + Service string `json:"service" orm:"service" description:"服务商"` + Md5 string `json:"md5" orm:"md5" description:"文件MD5"` + FileSize int `json:"fileSize" orm:"file_size" description:"文件大小"` + FileCrc16 int `json:"fileCrc16" orm:"file_crc16" description:"文件Crc16"` + FileSum int `json:"fileSum" orm:"file_sum" description:"文件校验和"` +} diff --git a/internal/model/entity/t_message_log.go b/internal/model/entity/t_message_log.go new file mode 100644 index 0000000..a7eb33c --- /dev/null +++ b/internal/model/entity/t_message_log.go @@ -0,0 +1,29 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package entity + +import ( + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" +) + +// TMessageLog is the golang structure for table t_message_log. +type TMessageLog struct { + MessageLogId uint64 `json:"messageLogId" orm:"message_log_id" description:"主键ID"` + MessageType uint `json:"messageType" orm:"message_type" description:"消息类型"` + SendUserType uint `json:"sendUserType" orm:"send_user_type" description:"发送用户类型"` + UserIds *gjson.Json `json:"userIds" orm:"user_ids" description:"接收用户ID集合(JSON数组)"` + MessageTitle string `json:"messageTitle" orm:"message_title" description:"消息标题"` + MessageContent string `json:"messageContent" orm:"message_content" description:"消息内容"` + MessageLink string `json:"messageLink" orm:"message_link" description:"跳转链接地址"` + SendTime *gtime.Time `json:"sendTime" orm:"send_time" description:"发送时间"` + IsRecall uint `json:"isRecall" orm:"is_recall" description:"是否撤回"` + TenantId string `json:"tenantId" orm:"tenant_id" description:"租户ID"` + CreatedDept int64 `json:"createdDept" orm:"created_dept" description:"创建部门"` + CreatedBy int64 `json:"createdBy" orm:"created_by" description:"创建者"` + CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" description:"创建时间"` + UpdatedBy int64 `json:"updatedBy" orm:"updated_by" description:"更新者"` + UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" description:"更新时间"` +} diff --git a/internal/model/entity/t_resource_batch_download.go b/internal/model/entity/t_resource_batch_download.go new file mode 100644 index 0000000..bdd7b89 --- /dev/null +++ b/internal/model/entity/t_resource_batch_download.go @@ -0,0 +1,46 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package entity + +import ( + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" +) + +// TResourceBatchDownload is the golang structure for table t_resource_batch_download. +type TResourceBatchDownload struct { + Id int64 `json:"id" orm:"id" description:"主键ID"` + BatchNo string `json:"batchNo" orm:"batch_no" description:"批次编号（全局唯一）"` + TaskName string `json:"taskName" orm:"task_name" description:"任务名称"` + ResourceType int `json:"resourceType" orm:"resource_type" description:"资源类型 枚举"` + ExportScene int `json:"exportScene" orm:"export_scene" description:"导出场景 枚举"` + QueryParams *gjson.Json `json:"queryParams" orm:"query_params" description:"导出查询条件快照"` + TotalCount int `json:"totalCount" orm:"total_count" description:"预计导出总条数"` + SuccessCount int `json:"successCount" orm:"success_count" description:"已成功导出条数"` + FailCount int `json:"failCount" orm:"fail_count" description:"失败条数"` + DownloadStatus int `json:"downloadStatus" orm:"download_status" description:"下载状态 枚举"` + OssId int64 `json:"ossId" orm:"oss_id" description:"OSS文件ID"` + FileName string `json:"fileName" orm:"file_name" description:"导出文件名"` + FileExt string `json:"fileExt" orm:"file_ext" description:"文件扩展名"` + FileUrl string `json:"fileUrl" orm:"file_url" description:"文件访问地址"` + FilePath string `json:"filePath" orm:"file_path" description:"服务器本地路径"` + FileSize int64 `json:"fileSize" orm:"file_size" description:"文件大小（字节）"` + FileMd5 string `json:"fileMd5" orm:"file_md5" description:"文件MD5"` + DownloadCount int `json:"downloadCount" orm:"download_count" description:"下载次数"` + LastDownloadTime *gtime.Time `json:"lastDownloadTime" orm:"last_download_time" description:"最后下载时间"` + MaxDownloadCount int `json:"maxDownloadCount" orm:"max_download_count" description:"最大下载次数限制"` + ErrorMessage string `json:"errorMessage" orm:"error_message" description:"失败原因"` + DeptId int64 `json:"deptId" orm:"dept_id" description:"所属机构ID"` + StartTime *gtime.Time `json:"startTime" orm:"start_time" description:"任务开始时间"` + FinishTime *gtime.Time `json:"finishTime" orm:"finish_time" description:"任务完成时间"` + ExpireTime *gtime.Time `json:"expireTime" orm:"expire_time" description:"文件过期时间"` + Remark string `json:"remark" orm:"remark" description:"备注"` + TenantId string `json:"tenantId" orm:"tenant_id" description:"租户ID"` + CreatedDept int64 `json:"createdDept" orm:"created_dept" description:"创建部门ID"` + CreatedBy int64 `json:"createdBy" orm:"created_by" description:"创建者ID"` + CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" description:"创建时间"` + UpdatedBy int64 `json:"updatedBy" orm:"updated_by" description:"更新者ID"` + UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" description:"更新时间"` +} diff --git a/internal/model/entity/t_sim_card_cancellation.go b/internal/model/entity/t_sim_card_cancellation.go new file mode 100644 index 0000000..dd62575 --- /dev/null +++ b/internal/model/entity/t_sim_card_cancellation.go @@ -0,0 +1,70 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package entity + +import ( + "github.com/gogf/gf/v2/os/gtime" +) + +// TSimCardCancellation is the golang structure for table t_sim_card_cancellation. +type TSimCardCancellation struct { + Id int64 `json:"id" orm:"id" ` // 主键ID + Iccid string `json:"iccid" orm:"iccid" ` // ICCID + Msisdn string `json:"msisdn" orm:"msisdn" ` // MSISDN/卡号 + Imsi string `json:"imsi" orm:"imsi" ` // IMSI + Imei string `json:"imei" orm:"imei" ` // IMEI + RegionCode string `json:"regionCode" orm:"region_code" ` // 归属地 + DeptId int64 `json:"deptId" orm:"dept_id" ` // 渠道客户ID + CurrentPackageCode string `json:"currentPackageCode" orm:"current_package_code" ` // 当前套餐编号 + PackageStatus int `json:"packageStatus" orm:"package_status" ` // 套餐状态：0 无套餐，1待生效，2生效中，3已过期 + AccessType int `json:"accessType" orm:"access_type" ` // 接入方式：ipv4、ipv6、ipv4_ipv6 + BossCardStatus int `json:"bossCardStatus" orm:"boss_card_status" ` // 运营商卡状态 枚举 + CancelBatchNo string `json:"cancelBatchNo" orm:"cancel_batch_no" ` // 系统销户批次号 + CancelledAt *gtime.Time `json:"cancelledAt" orm:"cancelled_at" ` // 系统销户完成时间 + CardType int `json:"cardType" orm:"card_type" ` // 卡类型 枚举 + BossOpenAt *gtime.Time `json:"bossOpenAt" orm:"boss_open_at" ` // 运营商开户日期 + BossActivateAt *gtime.Time `json:"bossActivateAt" orm:"boss_activate_at" ` // 运营商激活日期 + IsBossActivateEnabled int `json:"isBossActivateEnabled" orm:"is_boss_activate_enabled" ` // 是否按运营商激活减去赠送时长方：1是，0否 + InboundBatchNo string `json:"inboundBatchNo" orm:"inbound_batch_no" ` // 入库批次 + OutboundBatchNo string `json:"outboundBatchNo" orm:"outbound_batch_no" ` // 出库批次 + AllocationBatchNo string `json:"allocationBatchNo" orm:"allocation_batch_no" ` // 划拨批次 + IsFlowPool int `json:"isFlowPool" orm:"is_flow_pool" ` // 是否为池卡：1是，0否 + FlowPoolCode string `json:"flowPoolCode" orm:"flow_pool_code" ` // 流量池编号 + FlowPkgBeginAt *gtime.Time `json:"flowPkgBeginAt" orm:"flow_pkg_begin_at" ` // 当前套餐生效时间 + FlowPkgPresentAt *gtime.Time `json:"flowPkgPresentAt" orm:"flow_pkg_present_at" ` // 套餐赠送结束时间 + FlowPkgRealPresentAt *gtime.Time `json:"flowPkgRealPresentAt" orm:"flow_pkg_real_present_at" ` // 实际赠送结束时间 + FlowPkgEndAt *gtime.Time `json:"flowPkgEndAt" orm:"flow_pkg_end_at" ` // 当前套餐预计到期时间 + DataServiceStatus int `json:"dataServiceStatus" orm:"data_service_status" ` // 数据服务状态：0启用,1停用 + PackageTotalFlowMb float64 `json:"packageTotalFlowMb" orm:"package_total_flow_mb" ` // 套餐总流量MB + UsedFlowMb float64 `json:"usedFlowMb" orm:"used_flow_mb" ` // 已用流量MB + TrafficCutoffStatus int `json:"trafficCutoffStatus" orm:"traffic_cutoff_status" ` // 达量断网配置状态：0未配置，1已开启，2已关闭 + TrafficCutoffThreshold float64 `json:"trafficCutoffThreshold" orm:"traffic_cutoff_threshold" ` // 达量断网阈值 + SmsServiceStatus int `json:"smsServiceStatus" orm:"sms_service_status" ` // 短信服务状态：0启用,1停用 + PackageTotalSms int `json:"packageTotalSms" orm:"package_total_sms" ` // 套餐短信总条数 + UsedSms int `json:"usedSms" orm:"used_sms" ` // 已用短信条数 + VoiceServiceStatus int `json:"voiceServiceStatus" orm:"voice_service_status" ` // 语音服务状态：0启用,1停用 + PackageTotalVoice int `json:"packageTotalVoice" orm:"package_total_voice" ` // 套餐语音总分钟数 + UsedVoice int `json:"usedVoice" orm:"used_voice" ` // 已用语音分钟数 + RoamServiceStatus int `json:"roamServiceStatus" orm:"roam_service_status" ` // 国漫服务状态：0启用,1停用 + PackageTotalRoamFlowMb float64 `json:"packageTotalRoamFlowMb" orm:"package_total_roam_flow_mb" ` // 套餐国际流量MB + UsedRoamFlowMb float64 `json:"usedRoamFlowMb" orm:"used_roam_flow_mb" ` // 已用国际流量MB + StatusSyncTime *gtime.Time `json:"statusSyncTime" orm:"status_sync_time" ` // 状态最后一次同步时间 + FlowSyncTime *gtime.Time `json:"flowSyncTime" orm:"flow_sync_time" ` // 流量最后一次同步时间 + SmsSyncTime *gtime.Time `json:"smsSyncTime" orm:"sms_sync_time" ` // 短信最后一次同步时间 + VoiceSyncTime *gtime.Time `json:"voiceSyncTime" orm:"voice_sync_time" ` // 语音最后一次同步时间 + RoamSyncTime *gtime.Time `json:"roamSyncTime" orm:"roam_sync_time" ` // 国漫最后一次同步时间 + ExtraFeeEnabled int `json:"extraFeeEnabled" orm:"extra_fee_enabled" ` // 是否收取附加费：0是，1否 + ExtraFeeMultiple float64 `json:"extraFeeMultiple" orm:"extra_fee_multiple" ` // 附加费用倍数 + ExtraFee float64 `json:"extraFee" orm:"extra_fee" ` // 附加金额 + ChanKey string `json:"chanKey" orm:"chan_key" ` // 通道编号 + NetModel int `json:"netModel" orm:"net_model" ` // 网络类型 枚举 + Remark string `json:"remark" orm:"remark" ` // 备注 + TenantId string `json:"tenantId" orm:"tenant_id" ` // 租户ID + CreatedDept int64 `json:"createdDept" orm:"created_dept" ` // 创建部门 + CreatedBy int64 `json:"createdBy" orm:"created_by" ` // 创建者 + CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" ` // 创建时间 + UpdatedBy int64 `json:"updatedBy" orm:"updated_by" ` // 更新者 + UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" ` // 更新时间 +} diff --git a/internal/model/entity/t_sim_card_info.go b/internal/model/entity/t_sim_card_info.go new file mode 100644 index 0000000..47cc89d --- /dev/null +++ b/internal/model/entity/t_sim_card_info.go @@ -0,0 +1,71 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package entity + +import ( + "github.com/gogf/gf/v2/os/gtime" +) + +// TSimCardInfo is the golang structure for table t_sim_card_info. +type TSimCardInfo struct { + Id int64 `json:"id" orm:"id" description:"主键ID"` + Iccid string `json:"iccid" orm:"iccid" description:"ICCID"` + Msisdn string `json:"msisdn" orm:"msisdn" description:"MSISDN/卡号"` + Imsi string `json:"imsi" orm:"imsi" description:"IMSI"` + Imei string `json:"imei" orm:"imei" description:"IMEI"` + RegionCode string `json:"regionCode" orm:"region_code" description:"归属地"` + DeptId int64 `json:"deptId" orm:"dept_id" description:"渠道客户ID"` + CurrentPackageCode string `json:"currentPackageCode" orm:"current_package_code" description:"当前套餐编号"` + PackageStatus int `json:"packageStatus" orm:"package_status" description:"套餐状态：0 无套餐，1待生效，2生效中，3已过期"` + AccessType int `json:"accessType" orm:"access_type" description:"接入方式：ipv4、ipv6、ipv4_ipv6"` + BossCardStatus int `json:"bossCardStatus" orm:"boss_card_status" description:"运营商卡状态 枚举"` + CardType int `json:"cardType" orm:"card_type" description:"卡类型 枚举"` + BossOpenAt *gtime.Time `json:"bossOpenAt" orm:"boss_open_at" description:"运营商开户日期"` + BossActivateAt *gtime.Time `json:"bossActivateAt" orm:"boss_activate_at" description:"运营商激活日期"` + IsBossActivateEnabled int `json:"isBossActivateEnabled" orm:"is_boss_activate_enabled" description:"是否按运营商激活减去赠送时长方：1是，0否"` + InboundBatchNo string `json:"inboundBatchNo" orm:"inbound_batch_no" description:"入库批次"` + OutboundBatchNo string `json:"outboundBatchNo" orm:"outbound_batch_no" description:"出库批次"` + AllocationBatchNo string `json:"allocationBatchNo" orm:"allocation_batch_no" description:"划拨批次"` + IsFlowPool int `json:"isFlowPool" orm:"is_flow_pool" description:"是否为池卡：1是，0否"` + FlowPoolCode string `json:"flowPoolCode" orm:"flow_pool_code" description:"流量池编号"` + OutStockTime *gtime.Time `json:"outStockTime" orm:"out_stock_time" description:"出库时间"` + FlowPkgBeginAt *gtime.Time `json:"flowPkgBeginAt" orm:"flow_pkg_begin_at" description:"当前套餐生效时间"` + FlowPkgPresentAt *gtime.Time `json:"flowPkgPresentAt" orm:"flow_pkg_present_at" description:"套餐赠送结束时间"` + FlowPkgRealPresentAt *gtime.Time `json:"flowPkgRealPresentAt" orm:"flow_pkg_real_present_at" description:"实际赠送结束时间"` + FlowPkgEndAt *gtime.Time `json:"flowPkgEndAt" orm:"flow_pkg_end_at" description:"当前套餐预计到期时间"` + DataServiceStatus int `json:"dataServiceStatus" orm:"data_service_status" description:"数据服务状态：1启用、0禁用"` + PackageTotalFlowMb float64 `json:"packageTotalFlowMb" orm:"package_total_flow_mb" description:"套餐总流量MB"` + UsedFlowMb float64 `json:"usedFlowMb" orm:"used_flow_mb" description:"已用流量MB"` + TrafficCutoffStatus int `json:"trafficCutoffStatus" orm:"traffic_cutoff_status" description:"达量断网配置状态：0未配置，1已开启，2已关闭"` + TrafficCutoffThreshold float64 `json:"trafficCutoffThreshold" orm:"traffic_cutoff_threshold" description:"达量断网阈值"` + SmsServiceStatus int `json:"smsServiceStatus" orm:"sms_service_status" description:"短信服务状态：1启用、0禁用"` + PackageTotalSms int `json:"packageTotalSms" orm:"package_total_sms" description:"套餐短信总条数"` + UsedSms int `json:"usedSms" orm:"used_sms" description:"已用短信条数"` + VoiceServiceStatus int `json:"voiceServiceStatus" orm:"voice_service_status" description:"语音服务状态：1启用、0禁用"` + PackageTotalVoice int `json:"packageTotalVoice" orm:"package_total_voice" description:"套餐语音总分钟数"` + UsedVoice int `json:"usedVoice" orm:"used_voice" description:"已用语音分钟数"` + RoamServiceStatus int `json:"roamServiceStatus" orm:"roam_service_status" description:"国漫服务状态：1启用、0禁用"` + PackageTotalRoamFlowMb float64 `json:"packageTotalRoamFlowMb" orm:"package_total_roam_flow_mb" description:"套餐国际流量MB"` + UsedRoamFlowMb float64 `json:"usedRoamFlowMb" orm:"used_roam_flow_mb" description:"已用国际流量MB"` + StatusSyncTime *gtime.Time `json:"statusSyncTime" orm:"status_sync_time" description:"状态最后一次同步时间"` + FlowSyncTime *gtime.Time `json:"flowSyncTime" orm:"flow_sync_time" description:"流量最后一次同步时间"` + SmsSyncTime *gtime.Time `json:"smsSyncTime" orm:"sms_sync_time" description:"短信最后一次同步时间"` + VoiceSyncTime *gtime.Time `json:"voiceSyncTime" orm:"voice_sync_time" description:"语音最后一次同步时间"` + RoamSyncTime *gtime.Time `json:"roamSyncTime" orm:"roam_sync_time" description:"国漫最后一次同步时间"` + ImeiSyncTime *gtime.Time `json:"imeiSyncTime" orm:"imei_sync_time" description:"IMEI 最后一次轮询同步时间"` + ImeiUpdateTime *gtime.Time `json:"imeiUpdateTime" orm:"imei_update_time" description:"IMEI 值最后一次实际变更时间"` + ExtraFeeEnabled int `json:"extraFeeEnabled" orm:"extra_fee_enabled" description:"是否收取附加费：1是，0否"` + ExtraFeeMultiple float64 `json:"extraFeeMultiple" orm:"extra_fee_multiple" description:"附加费用倍数"` + ExtraFee float64 `json:"extraFee" orm:"extra_fee" description:"附加金额"` + ChanKey string `json:"chanKey" orm:"chan_key" description:"通道编号"` + NetModel int `json:"netModel" orm:"net_model" description:"网络类型：4G、5G、NB-IoT"` + Remark string `json:"remark" orm:"remark" description:"备注"` + TenantId string `json:"tenantId" orm:"tenant_id" description:"租户ID"` + CreatedDept int64 `json:"createdDept" orm:"created_dept" description:"创建部门"` + CreatedBy int64 `json:"createdBy" orm:"created_by" description:"创建者"` + CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" description:"创建时间"` + UpdatedBy int64 `json:"updatedBy" orm:"updated_by" description:"更新者"` + UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" description:"更新时间"` +} diff --git a/internal/model/entity/t_sim_flow_pool.go b/internal/model/entity/t_sim_flow_pool.go new file mode 100644 index 0000000..bdcbc11 --- /dev/null +++ b/internal/model/entity/t_sim_flow_pool.go @@ -0,0 +1,35 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package entity + +import ( + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" +) + +// TSimFlowPool is the golang structure for table t_sim_flow_pool. +type TSimFlowPool struct { + Id int64 `json:"id" orm:"id" description:"主键ID"` + FlowPoolCode string `json:"flowPoolCode" orm:"flow_pool_code" description:"流量池编号"` + DeptId int64 `json:"deptId" orm:"dept_id" description:"客户编号"` + PackageCodes *gjson.Json `json:"packageCodes" orm:"package_codes" description:"套餐编号列表（冗余）"` + PoolPackageCode string `json:"poolPackageCode" orm:"pool_package_code" description:"池套餐编码"` + CardCount int64 `json:"cardCount" orm:"card_count" description:"关联卡数量"` + CardFlowMb float64 `json:"cardFlowMb" orm:"card_flow_mb" description:"套餐卡流量MB"` + PoolPackageFlowMb float64 `json:"poolPackageFlowMb" orm:"pool_package_flow_mb" description:"套内总流量MB"` + AddOnFlowMb float64 `json:"addOnFlowMb" orm:"add_on_flow_mb" description:"加油包总流量MB"` + TotalFlowMb float64 `json:"totalFlowMb" orm:"total_flow_mb" description:"总流量MB(套内+加油包)"` + UsedFlowMb float64 `json:"usedFlowMb" orm:"used_flow_mb" description:"已使用流量MB"` + RemainFlowMb float64 `json:"remainFlowMb" orm:"remain_flow_mb" description:"剩余流量MB"` + LastStatTime *gtime.Time `json:"lastStatTime" orm:"last_stat_time" description:"最后流量统计时间"` + PoolStatus int `json:"poolStatus" orm:"pool_status" description:"状态：0 正常、1 停用、2 锁定"` + Remark string `json:"remark" orm:"remark" description:"备注"` + TenantId string `json:"tenantId" orm:"tenant_id" description:"租户ID"` + CreatedDept int64 `json:"createdDept" orm:"created_dept" description:"创建部门"` + CreatedBy int64 `json:"createdBy" orm:"created_by" description:"创建者"` + CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" description:"创建时间"` + UpdatedBy int64 `json:"updatedBy" orm:"updated_by" description:"更新者"` + UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" description:"更新时间"` +} diff --git a/internal/model/entity/t_user_message.go b/internal/model/entity/t_user_message.go new file mode 100644 index 0000000..0b07a70 --- /dev/null +++ b/internal/model/entity/t_user_message.go @@ -0,0 +1,23 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package entity + +import ( + "github.com/gogf/gf/v2/os/gtime" +) + +// TUserMessage is the golang structure for table t_user_message. +type TUserMessage struct { + MessageId uint64 `json:"messageId" orm:"message_id" description:"主键ID"` + MessageLogId uint64 `json:"messageLogId" orm:"message_log_id" description:"消息日志ID"` + UserId uint64 `json:"userId" orm:"user_id" description:"用户ID"` + IsRead uint `json:"isRead" orm:"is_read" description:"是否已读"` + TenantId string `json:"tenantId" orm:"tenant_id" description:"租户ID"` + CreatedDept int64 `json:"createdDept" orm:"created_dept" description:"创建部门"` + CreatedBy int64 `json:"createdBy" orm:"created_by" description:"创建者"` + CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" description:"创建时间"` + UpdatedBy int64 `json:"updatedBy" orm:"updated_by" description:"更新者"` + UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" description:"更新时间"` +} diff --git a/internal/model/entity/t_warn_config.go b/internal/model/entity/t_warn_config.go new file mode 100644 index 0000000..b24f343 --- /dev/null +++ b/internal/model/entity/t_warn_config.go @@ -0,0 +1,36 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package entity + +import ( + "github.com/gogf/gf/v2/os/gtime" +) + +// TWarnConfig is the golang structure for table t_warn_config. +type TWarnConfig struct { + Id int64 `json:"id" orm:"id" description:"主键ID"` + TenantId string `json:"tenantId" orm:"tenant_id" description:"租户ID"` + BatchNo string `json:"batchNo" orm:"batch_no" description:"批次编号"` + ItemType int `json:"itemType" orm:"item_type" description:"对象类型:1卡,2流量池"` + ItemId int64 `json:"itemId" orm:"item_id" description:"对象ID:单卡为t_sim_card_info.id,流量池为t_sim_flow_pool.id"` + ItemCode string `json:"itemCode" orm:"item_code" description:"对象编码快照:ICCID/MSISDN/流量池编码"` + WarnUnit int `json:"warnUnit" orm:"warn_unit" description:"预警单位:1MB,2百分比,3天数,4月数"` + FirstWarn float64 `json:"firstWarn" orm:"first_warn" description:"第一档预警阈值,单位由warn_unit决定"` + SecondWarn float64 `json:"secondWarn" orm:"second_warn" description:"第二档预警阈值,单位由warn_unit决定"` + ThirdWarn float64 `json:"thirdWarn" orm:"third_warn" description:"第三档预警阈值,单位由warn_unit决定"` + WarnMethods string `json:"warnMethods" orm:"warn_methods" description:"预警方式:0短信,1邮箱,2电话,3平台消息;多选逗号分隔"` + WarnPhones string `json:"warnPhones" orm:"warn_phones" description:"预警手机号,多个逗号分隔"` + WarnEmails string `json:"warnEmails" orm:"warn_emails" description:"预警邮箱,多个逗号分隔"` + AutoAction int `json:"autoAction" orm:"auto_action" description:"自动动作:0无,2断网;第一期只保存配置不自动执行"` + IsEnabled int `json:"isEnabled" orm:"is_enabled" description:"是否启用:1启用,0停用"` + WxTemplateId string `json:"wxTemplateId" orm:"wx_template_id" description:"微信订阅消息模板ID"` + Remark string `json:"remark" orm:"remark" description:"备注"` + CreatedDept int64 `json:"createdDept" orm:"created_dept" description:"创建部门"` + CreatedBy int64 `json:"createdBy" orm:"created_by" description:"创建者"` + CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" description:"创建时间"` + UpdatedBy int64 `json:"updatedBy" orm:"updated_by" description:"更新者"` + UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" description:"更新时间"` + AppNotifyEnabled int `json:"appNotifyEnabled" orm:"app_notify_enabled" description:"是否开启小程序预警通知:0关闭,1开启"` +} diff --git a/internal/model/entity/t_warn_log.go b/internal/model/entity/t_warn_log.go new file mode 100644 index 0000000..f1f832f --- /dev/null +++ b/internal/model/entity/t_warn_log.go @@ -0,0 +1,37 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package entity + +import ( + "github.com/gogf/gf/v2/os/gtime" +) + +// TWarnLog is the golang structure for table t_warn_log. +type TWarnLog struct { + Id int64 `json:"id" orm:"id" description:"主键ID"` + TenantId string `json:"tenantId" orm:"tenant_id" description:"租户ID"` + ConfigId int64 `json:"configId" orm:"config_id" description:"预警配置ID"` + ItemType int `json:"itemType" orm:"item_type" description:"对象类型:1单卡,2流量池"` + ItemId int64 `json:"itemId" orm:"item_id" description:"对象ID:单卡为t_sim_card_info.id,流量池为t_sim_flow_pool.id"` + ItemCode string `json:"itemCode" orm:"item_code" description:"对象编码快照:ICCID/MSISDN/流量池编码"` + WarnStage int `json:"warnStage" orm:"warn_stage" description:"触发档位:1第一档,2第二档,3第三档"` + WarnLevel int `json:"warnLevel" orm:"warn_level" description:"预警级别:1提示,2注意,3警告,4危险,5紧急"` + WarnUnit int `json:"warnUnit" orm:"warn_unit" description:"预警单位/维度:1MB,2百分比,3天数,4月数"` + ThresholdValue float64 `json:"thresholdValue" orm:"threshold_value" description:"触发阈值,单位由warn_unit决定"` + ActualValue float64 `json:"actualValue" orm:"actual_value" description:"触发时实际值,单位由warn_unit决定"` + TotalFlowMb float64 `json:"totalFlowMb" orm:"total_flow_mb" description:"触发时总流量MB;流量类预警使用"` + RemainFlowMb float64 `json:"remainFlowMb" orm:"remain_flow_mb" description:"触发时剩余流量MB;流量类预警使用"` + WarnMethods string `json:"warnMethods" orm:"warn_methods" description:"实际预警方式快照"` + WarnPhones string `json:"warnPhones" orm:"warn_phones" description:"实际预警手机号快照"` + WarnEmails string `json:"warnEmails" orm:"warn_emails" description:"实际预警邮箱快照"` + MessageLogId int64 `json:"messageLogId" orm:"message_log_id" description:"平台消息ID;聚合通知时可能为空"` + WarnTime *gtime.Time `json:"warnTime" orm:"warn_time" description:"预警时间"` + Remark string `json:"remark" orm:"remark" description:"备注"` + CreatedDept int64 `json:"createdDept" orm:"created_dept" description:"创建部门"` + CreatedBy int64 `json:"createdBy" orm:"created_by" description:"创建者"` + CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" description:"创建时间"` + UpdatedBy int64 `json:"updatedBy" orm:"updated_by" description:"更新者"` + UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" description:"更新时间"` +} diff --git a/internal/model/entity/t_wo_main.go b/internal/model/entity/t_wo_main.go new file mode 100644 index 0000000..747a5a8 --- /dev/null +++ b/internal/model/entity/t_wo_main.go @@ -0,0 +1,43 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package entity + +import ( + "github.com/gogf/gf/v2/os/gtime" +) + +// TWoMain is the golang structure for table t_wo_main. +type TWoMain struct { + Id int64 `json:"id" orm:"id" description:"主键ID"` + WoNo string `json:"woNo" orm:"wo_no" description:"工单编号"` + WoType int `json:"woType" orm:"wo_type" description:"工单类型 0卡,1模组,2设备,3订单,4余额,5提现"` + RelatedId int64 `json:"relatedId" orm:"related_id" description:"关联业务ID"` + Title string `json:"title" orm:"title" description:"工单标题"` + Content string `json:"content" orm:"content" description:"问题描述"` + Attachment string `json:"attachment" orm:"attachment" description:"附件"` + CreatorType int `json:"creatorType" orm:"creator_type" description:"创建人类型：1终端用户，2代理商"` + CreatorDeptId int64 `json:"creatorDeptId" orm:"creator_dept_id" description:"创建机构ID"` + CreatorUserId int64 `json:"creatorUserId" orm:"creator_user_id" description:"创建用户ID"` + CreatorName string `json:"creatorName" orm:"creator_name" description:"创建人姓名"` + CreatorPhone string `json:"creatorPhone" orm:"creator_phone" description:"创建人手机号"` + AssignDeptId int64 `json:"assignDeptId" orm:"assign_dept_id" description:"分派方机构ID"` + AssignUserId int64 `json:"assignUserId" orm:"assign_user_id" description:"分派人用户ID"` + CurrentStatus int `json:"currentStatus" orm:"current_status" description:"当前处理状态：1已分派，2处理中，3已解决，4已完结_关单，5已取消"` + HandlerDeptId int64 `json:"handlerDeptId" orm:"handler_dept_id" description:"处理方机构ID"` + HandlerUserId int64 `json:"handlerUserId" orm:"handler_user_id" description:"处理人用户ID"` + FirstResponseTime *gtime.Time `json:"firstResponseTime" orm:"first_response_time" description:"首次响应时间"` + ResolveTime *gtime.Time `json:"resolveTime" orm:"resolve_time" description:"确认时间"` + CloseTime *gtime.Time `json:"closeTime" orm:"close_time" description:"关闭时间"` + SatisfactionLevel int `json:"satisfactionLevel" orm:"satisfaction_level" description:"满意度：1-5分"` + EvalContent string `json:"evalContent" orm:"eval_content" description:"评价内容"` + EvalTime *gtime.Time `json:"evalTime" orm:"eval_time" description:"评价时间"` + Remark string `json:"remark" orm:"remark" description:"备注"` + TenantId string `json:"tenantId" orm:"tenant_id" description:"租户ID"` + CreatedDept int64 `json:"createdDept" orm:"created_dept" description:"创建部门"` + CreatedBy int64 `json:"createdBy" orm:"created_by" description:"创建者"` + CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" description:"创建时间"` + UpdatedBy int64 `json:"updatedBy" orm:"updated_by" description:"更新者"` + UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" description:"更新时间"` +} diff --git a/internal/model/entity/t_wo_record.go b/internal/model/entity/t_wo_record.go new file mode 100644 index 0000000..347e54b --- /dev/null +++ b/internal/model/entity/t_wo_record.go @@ -0,0 +1,28 @@ +// ================================================================================= +// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT. +// ================================================================================= + +package entity + +import ( + "github.com/gogf/gf/v2/os/gtime" +) + +// TWoRecord is the golang structure for table t_wo_record. +type TWoRecord struct { + Id int64 `json:"id" orm:"id" description:"主键ID"` + WoNo string `json:"woNo" orm:"wo_no" description:"工单编号"` + OperationType int `json:"operationType" orm:"operation_type" description:"操作类型：0创建，1分派，2处理，3完成，4关单，5取消"` + OperationDeptId int64 `json:"operationDeptId" orm:"operation_dept_id" description:"操作机构ID"` + OperationUserId int64 `json:"operationUserId" orm:"operation_user_id" description:"操作用户ID"` + TargetDeptId int64 `json:"targetDeptId" orm:"target_dept_id" description:"目标部门ID"` + TargetUserId int64 `json:"targetUserId" orm:"target_user_id" description:"目标用户ID"` + BeforeStatus int `json:"beforeStatus" orm:"before_status" description:"变更前状态"` + AfterStatus int `json:"afterStatus" orm:"after_status" description:"变更后状态"` + TenantId string `json:"tenantId" orm:"tenant_id" description:"租户ID"` + CreatedDept int64 `json:"createdDept" orm:"created_dept" description:"创建部门"` + CreatedBy int64 `json:"createdBy" orm:"created_by" description:"创建者"` + CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" description:"创建时间"` + UpdatedBy int64 `json:"updatedBy" orm:"updated_by" description:"更新者"` + UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" description:"更新时间"` +} diff --git a/internal/model/imei_sync.go b/internal/model/imei_sync.go new file mode 100644 index 0000000..261aa45 --- /dev/null +++ b/internal/model/imei_sync.go @@ -0,0 +1,16 @@ +package model + +// ImeiQueueStartMessage asks the Worker to scan the card table and run a full +// IMEI synchronization. The Admin service does not perform upstream calls for +// this path. A nil Scope (or one whose channels normalize to empty) is a full +// sync. +// +// The json tags must stay byte-identical with the Admin-side definition in +// iotlcm-admin-svr (internal/model/imei_sync.go); the two services each keep +// their own copy by existing convention. +type ImeiQueueStartMessage struct { + RunID string `json:"runId"` + Operation string `json:"operation"` + TriggeredAt string `json:"triggeredAt"` + Scope *SyncScope `json:"scope,omitempty"` +} diff --git a/internal/model/package_customer_sync.go b/internal/model/package_customer_sync.go new file mode 100644 index 0000000..7dbbf84 --- /dev/null +++ b/internal/model/package_customer_sync.go @@ -0,0 +1,9 @@ +package model + +type PackageCustomerSyncMessage struct { + TenantId string `json:"tenantId"` + DeptId int64 `json:"deptId"` + UserId int64 `json:"userId"` + Trigger string `json:"trigger"` + SourceBatchNo string `json:"sourceBatchNo"` +} diff --git a/internal/model/sim_status_sync.go b/internal/model/sim_status_sync.go new file mode 100644 index 0000000..c90a745 --- /dev/null +++ b/internal/model/sim_status_sync.go @@ -0,0 +1,28 @@ +package model + +// SimStatusQueueStartMessage asks the Worker to scan the card table and run a +// full status synchronization. The Admin service does not perform upstream +// calls for this path. A nil Scope (or one whose channels normalize to empty) +// is a full sync. +type SimStatusQueueStartMessage struct { + RunID string `json:"runId"` + Operation string `json:"operation"` + TriggeredAt string `json:"triggeredAt"` + Scope *SyncScope `json:"scope,omitempty"` +} + +// SimStatusQueueTaskMessage is the card-level form used by business actions. +// The complete card routing information is carried so the Worker can enqueue +// it without doing a synchronous Admin lookup. +type SimStatusQueueTaskMessage struct { + RequestID string `json:"requestId"` + RunID string `json:"runId"` + Operation string `json:"operation"` + CardID int64 `json:"cardId"` + ICCID string `json:"iccid"` + MSISDN string `json:"msisdn"` + IMSI string `json:"imsi"` + ChanKey string `json:"chanKey"` + TempKey string `json:"tempKey"` + Operator int `json:"operator"` +} diff --git a/internal/model/sync_scope.go b/internal/model/sync_scope.go new file mode 100644 index 0000000..8f41e95 --- /dev/null +++ b/internal/model/sync_scope.go @@ -0,0 +1,149 @@ +package model + +import ( + "fmt" + "sort" + "strings" +) + +// SyncScope narrows one full-sync run to an explicit channel whitelist. +// +// A nil scope, or one whose channel list normalizes to empty, means "full +// sync" and must behave exactly like a run started before this field existed. +// TriggerSource is descriptive only: it is recorded on the run for operators +// and never changes scheduling or retry behaviour. +type SyncScope struct { + Channels []string `json:"channels,omitempty"` + TriggerSource string `json:"triggerSource,omitempty"` +} + +const ( + // MaxScopeChannels bounds one partial-sync request. It is a guard against + // accidental "select everything" payloads, not a capacity limit. + MaxScopeChannels = 200 + // MaxChanKeyLength matches t_sim_channels.chan_key varchar(32). + MaxChanKeyLength = 32 +) + +// Trigger source values recorded on the run. +const ( + TriggerSourceScheduled = "scheduled" + TriggerSourceManual = "manual" + TriggerSourceAPI = "api" + TriggerSourceReplay = "replay" +) + +// NormalizeScope validates a requested scope and returns its canonical form. +// +// Canonical form is: nil for a full sync, otherwise a scope whose channel list +// is deduplicated and sorted. Sorting makes ScopeDigest stable across requests +// that select the same channels in a different order. +func NormalizeScope(scope *SyncScope) (*SyncScope, error) { + if scope == nil { + return nil, nil + } + triggerSource := strings.TrimSpace(scope.TriggerSource) + switch triggerSource { + case "", TriggerSourceScheduled, TriggerSourceManual, TriggerSourceAPI, TriggerSourceReplay: + default: + return nil, fmt.Errorf("invalid triggerSource %q", scope.TriggerSource) + } + if len(scope.Channels) == 0 { + return nil, nil + } + if len(scope.Channels) > MaxScopeChannels { + return nil, fmt.Errorf("scope has %d channels, maximum is %d", len(scope.Channels), MaxScopeChannels) + } + + seen := make(map[string]struct{}, len(scope.Channels)) + channels := make([]string, 0, len(scope.Channels)) + for _, raw := range scope.Channels { + chanKey := strings.TrimSpace(raw) + if err := ValidateChanKey(chanKey); err != nil { + return nil, err + } + if _, exists := seen[chanKey]; exists { + continue + } + seen[chanKey] = struct{}{} + channels = append(channels, chanKey) + } + if len(channels) == 0 { + return nil, nil + } + sort.Strings(channels) + + if triggerSource == "" { + triggerSource = TriggerSourceManual + } + return &SyncScope{Channels: channels, TriggerSource: triggerSource}, nil +} + +// ValidateChanKey rejects values that cannot exist in t_sim_channels.chan_key. +// The whitelist character set keeps the value safe to log and to embed in a +// Redis key without escaping. +func ValidateChanKey(chanKey string) error { + if chanKey == "" { + return fmt.Errorf("chanKey is empty") + } + if len(chanKey) > MaxChanKeyLength { + return fmt.Errorf("chanKey %q exceeds %d characters", chanKey, MaxChanKeyLength) + } + for _, r := range chanKey { + switch { + case r >= 'a' && r <= 'z': + case r >= 'A' && r <= 'Z': + case r >= '0' && r <= '9': + case r == '-' || r == '_': + default: + return fmt.Errorf("chanKey %q contains invalid character %q", chanKey, r) + } + } + return nil +} + +// ChannelSet returns the scope channels as a lookup set, or nil for a full +// sync. Callers treat nil as "no filtering". +func (s *SyncScope) ChannelSet() map[string]struct{} { + if s == nil || len(s.Channels) == 0 { + return nil + } + set := make(map[string]struct{}, len(s.Channels)) + for _, chanKey := range s.Channels { + set[chanKey] = struct{}{} + } + return set +} + +// Allows reports whether a card on chanKey belongs to this scope. A nil scope +// allows every channel. +func (s *SyncScope) Allows(chanKey string) bool { + if s == nil || len(s.Channels) == 0 { + return true + } + for _, allowed := range s.Channels { + if allowed == chanKey { + return true + } + } + return false +} + +// ChannelCount is the number of selected channels; zero means full sync. +func (s *SyncScope) ChannelCount() int { + if s == nil { + return 0 + } + return len(s.Channels) +} + +// ChannelSample returns at most limit channels for logging and run metadata. +func (s *SyncScope) ChannelSample(limit int) []string { + if s == nil || limit <= 0 { + return nil + } + if len(s.Channels) <= limit { + return s.Channels + } + return s.Channels[:limit] +} diff --git a/internal/model/sync_scope_test.go b/internal/model/sync_scope_test.go new file mode 100644 index 0000000..dc96fa7 --- /dev/null +++ b/internal/model/sync_scope_test.go @@ -0,0 +1,141 @@ +package model + +import ( + "strings" + "testing" +) + +func TestNormalizeScopeNilAndEmptyMeanFullSync(t *testing.T) { + for name, scope := range map[string]*SyncScope{ + "nil": nil, + "no channels": {}, + "empty slice": {Channels: []string{}}, + } { + got, err := NormalizeScope(scope) + if err != nil { + t.Fatalf("%s: unexpected error: %v", name, err) + } + if got != nil { + t.Fatalf("%s: expected nil scope for full sync, got %+v", name, got) + } + if got.ChannelCount() != 0 || !got.Allows("anything") { + t.Fatalf("%s: nil scope must allow every channel", name) + } + } +} + +func TestNormalizeScopeDeduplicatesAndSorts(t *testing.T) { + got, err := NormalizeScope(&SyncScope{Channels: []string{"217", "173", "217", "189"}}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + want := []string{"173", "189", "217"} + if len(got.Channels) != len(want) { + t.Fatalf("channels = %v, want %v", got.Channels, want) + } + for i := range want { + if got.Channels[i] != want[i] { + t.Fatalf("channels = %v, want %v (sorted)", got.Channels, want) + } + } + // Sorting keeps ScopeDigest stable for requests listing the same channels + // in a different order. + reordered, err := NormalizeScope(&SyncScope{Channels: []string{"189", "217", "173"}}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if strings.Join(reordered.Channels, ",") != strings.Join(got.Channels, ",") { + t.Fatalf("order-dependent normalization: %v vs %v", reordered.Channels, got.Channels) + } +} + +func TestNormalizeScopeTrimsWhitespace(t *testing.T) { + got, err := NormalizeScope(&SyncScope{Channels: []string{" 217 ", "173"}}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if got.Allows("217") != true { + t.Fatalf("whitespace-padded chanKey was not trimmed: %+v", got) + } +} + +func TestNormalizeScopeRejectsInvalidInput(t *testing.T) { + cases := map[string]*SyncScope{ + "too many channels": {Channels: make([]string, MaxScopeChannels+1)}, + "chanKey too long": {Channels: []string{strings.Repeat("a", MaxChanKeyLength+1)}}, + "chanKey with slash": {Channels: []string{"a/b"}}, + "chanKey with space": {Channels: []string{"a b"}}, + "chanKey with unicode": {Channels: []string{"通道"}}, + "chanKey with brace": {Channels: []string{"a{b}"}}, + "empty chanKey": {Channels: []string{""}}, + "blank chanKey": {Channels: []string{" "}}, + "unknown triggerSource": {Channels: []string{"217"}, TriggerSource: "cron"}, + } + for name, scope := range cases { + if _, err := NormalizeScope(scope); err == nil { + t.Fatalf("%s: expected an error", name) + } + } +} + +func TestNormalizeScopeAcceptsBoundaries(t *testing.T) { + // Exactly the maximum count must pass. + channels := make([]string, MaxScopeChannels) + for i := range channels { + channels[i] = "chan-" + strings.Repeat("0", i%10) + string(rune('a'+i%26)) + } + if _, err := NormalizeScope(&SyncScope{Channels: channels}); err != nil { + t.Fatalf("exactly %d channels must be accepted: %v", MaxScopeChannels, err) + } + // A chanKey of exactly the maximum length must pass. + if err := ValidateChanKey(strings.Repeat("a", MaxChanKeyLength)); err != nil { + t.Fatalf("chanKey of exactly %d characters must be accepted: %v", MaxChanKeyLength, err) + } +} + +func TestNormalizeScopeDefaultsTriggerSource(t *testing.T) { + got, err := NormalizeScope(&SyncScope{Channels: []string{"217"}}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if got.TriggerSource != TriggerSourceManual { + t.Fatalf("triggerSource = %q, want %q", got.TriggerSource, TriggerSourceManual) + } +} + +func TestNormalizeScopeKeepsExplicitTriggerSource(t *testing.T) { + got, err := NormalizeScope(&SyncScope{Channels: []string{"217"}, TriggerSource: TriggerSourceAPI}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if got.TriggerSource != TriggerSourceAPI { + t.Fatalf("triggerSource = %q, want %q", got.TriggerSource, TriggerSourceAPI) + } +} + +func TestScopeChannelSetAndSample(t *testing.T) { + scope, err := NormalizeScope(&SyncScope{Channels: []string{"217", "173"}}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + set := scope.ChannelSet() + if _, ok := set["173"]; !ok { + t.Fatalf("channel set missing 173: %v", set) + } + if len(set) != 2 { + t.Fatalf("channel set = %v, want 2 entries", set) + } + if sample := scope.ChannelSample(1); len(sample) != 1 || sample[0] != "173" { + t.Fatalf("sample = %v, want [173]", sample) + } + if sample := scope.ChannelSample(10); len(sample) != 2 { + t.Fatalf("sample = %v, want both channels", sample) + } + var nilScope *SyncScope + if nilScope.ChannelSet() != nil { + t.Fatalf("nil scope must return a nil channel set") + } + if nilScope.ChannelSample(10) != nil { + t.Fatalf("nil scope must return a nil sample") + } +} diff --git a/internal/model/telecom_template.go b/internal/model/telecom_template.go new file mode 100644 index 0000000..b981002 --- /dev/null +++ b/internal/model/telecom_template.go @@ -0,0 +1,12 @@ +package model + +import "strings" + +func IsTelecomTemplate(tempKey string) bool { + switch strings.ToLower(strings.TrimSpace(tempKey)) { + case "sim_dx_v1", "sim_dx_v3": + return true + default: + return false + } +} diff --git a/internal/model/traffic_usage.go b/internal/model/traffic_usage.go new file mode 100644 index 0000000..8ec4f78 --- /dev/null +++ b/internal/model/traffic_usage.go @@ -0,0 +1,19 @@ +package model + +import "time" + +type TrafficUsageBase struct { + Iccid string + UsageMb float64 + PackageCode string +} + +type TrafficUsageDailyItem struct { + TrafficUsageBase + Date time.Time +} + +type TrafficUsageMonthlyItem struct { + TrafficUsageBase + Month time.Time +} diff --git a/internal/model/traffic_usage_sync.go b/internal/model/traffic_usage_sync.go new file mode 100644 index 0000000..4193f91 --- /dev/null +++ b/internal/model/traffic_usage_sync.go @@ -0,0 +1,79 @@ +package model + +// TrafficUsageSimCard is the basic card data needed when writing usage to TDengine. +type TrafficUsageSimCard struct { + Iccid string `json:"iccid"` + PackageCode string `json:"packageCode"` +} + +// TrafficUsageDailyRetryMessage is the retry message for daily traffic usage. +type TrafficUsageDailyRetryMessage struct { + UsageMb *float64 `json:"usageMb,omitempty"` + Attempt int `json:"attempt,omitempty"` + LastError string `json:"lastError,omitempty"` + SimKey *SimKey `json:"simKey"` + Card TrafficUsageSimCard `json:"card"` + // QueryDate format: yyyyMMdd. + QueryDate string `json:"queryDate"` +} + +// TrafficUsageMonthlyRetryMessage is the retry message for monthly traffic usage. +type TrafficUsageMonthlyRetryMessage struct { + UsageMb *float64 `json:"usageMb,omitempty"` + Attempt int `json:"attempt,omitempty"` + LastError string `json:"lastError,omitempty"` + SimKey *SimKey `json:"simKey"` + Card TrafficUsageSimCard `json:"card"` + // QueryDate format: yyyyMM. + QueryDate string `json:"queryDate"` +} + +// TrafficUsageMonthlyStartMessage starts one full monthly usage run. +// Admin publishes this command; Worker owns the card scan and Redis production. +// A nil Scope (or one whose channels normalize to empty) is a full sync. +type TrafficUsageMonthlyStartMessage struct { + RunID string `json:"runId"` + Operation string `json:"operation"` + QueryMonth string `json:"queryMonth"` + TriggeredAt string `json:"triggeredAt"` + Scope *SyncScope `json:"scope,omitempty"` +} + +// TrafficUsageDailyStartMessage starts one full daily usage run. +// Admin publishes this command; Worker owns the card scan and Redis production. +// A nil Scope (or one whose channels normalize to empty) is a full sync. +// +// QueryDate is yyyyMMdd. The producer carries it through zset.Task.QueryMonth, +// which is a period identifier rather than a month name; see that field's doc. +type TrafficUsageDailyStartMessage struct { + RunID string `json:"runId"` + Operation string `json:"operation"` + QueryDate string `json:"queryDate"` + TriggeredAt string `json:"triggeredAt"` + Scope *SyncScope `json:"scope,omitempty"` +} + +// TrafficUsageMonthlyHistoryStartMessage starts one full monthly usage history +// run. It writes the TDengine monthly table, which is a different target from +// TrafficUsageMonthlyStartMessage (the current-month snapshot written to MySQL). +// +// QueryMonth is yyyyMM and names the month being backfilled, which is the +// previous month rather than the current one. +type TrafficUsageMonthlyHistoryStartMessage struct { + RunID string `json:"runId"` + Operation string `json:"operation"` + QueryMonth string `json:"queryMonth"` + TriggeredAt string `json:"triggeredAt"` + Scope *SyncScope `json:"scope,omitempty"` +} + +// OperationPreviousDayUsageFullSync identifies a daily usage run. It is the one +// string that Admin publishes, Worker produces, and contract.SyncKindForOperation +// maps to the traffic_usage_daily sync kind, so it lives here rather than in +// either side's logic package. +const OperationPreviousDayUsageFullSync = "previous_day_usage_full_sync" + +// OperationPreviousMonthUsageFullSync identifies a monthly usage history run. +// It is deliberately distinct from current_month_usage_full_sync, which fills +// the MySQL current-month snapshot instead of the TDengine monthly table. +const OperationPreviousMonthUsageFullSync = "previous_month_usage_full_sync" diff --git a/internal/model/upstream_chan.go b/internal/model/upstream_chan.go new file mode 100644 index 0000000..b9c788f --- /dev/null +++ b/internal/model/upstream_chan.go @@ -0,0 +1,110 @@ +package model + +import "time" + +type UpstreamChanConfig struct { + Address string `json:"address"` + Timeout time.Duration `json:"timeout"` +} + +type SimKey struct { + ChanKey string `json:"chanKey"` + ICCID string `json:"iccid"` + IMSI string `json:"imsi"` + MSISDN string `json:"msisdn"` +} + +type SimBaseReq[T any] struct { + SimKeys []*SimKey `json:"simKeys"` + ReqData T `json:"reqData"` +} + +// SimInputReqNullDate is used by current-month real-time usage calls. The +// upstream determines the current month at request time. +type SimInputReqNullDate struct{} + +type SimDataUsageInput = SimBaseReq[SimInputReqNullDate] + +type SimDataUsageItem struct { + DataUsage float64 `json:"dataUsage"` +} + +type SimBaseRes[T any] struct { + Code int `json:"code"` + Message string `json:"message"` + ErrCode string `json:"errCode"` + SimKey *SimKey `json:"simKey"` + ResData T `json:"resData"` +} + +type SimBasicInfoInput = SimBaseReq[SimInputReqNullDate] + +type SimBasicInfoItem struct { + ICCID string `json:"iccid"` + IMSI string `json:"imsi"` + MSISDN string `json:"msisdn"` + ActiveDate string `json:"activeDate"` + OpenDate string `json:"openDate"` + Remark string `json:"remark"` +} + +type SimDataUsageMonthlyInput = SimBaseReq[SimDataUsageMonthlyInputItem] +type SimDataUsageMonthlyInputItem struct { + QueryDate string `json:"queryDate"` +} +type SimDataUsageMonthlyOutputItem struct { + DeadLine string `json:"deadLine"` + DataAmount float64 `json:"dataAmount"` +} + +type SimDataUsageDailyInput = SimBaseReq[SimDataUsageDailyInputItem] +type SimDataUsageDailyInputItem struct { + QueryDate string `json:"queryDate"` +} +type SimDataUsageDailyOutputItem struct { + QueryDate string `json:"queryDate"` + DataAmount float64 `json:"dataAmount"` +} + +// SimCardStatusInput is the request envelope for carrier status queries. +type SimCardStatusInput = SimBaseReq[SimInputReqNullDate] + +// SimImeiInput is the request envelope for terminal IMEI queries. +type SimImeiInput = SimBaseReq[SimInputReqNullDate] + +// SimImeiItem is the terminal IMEI returned for one SIM card. +// +// A carrier may answer successfully yet return an empty value (for example a +// card whose device never reported an IMEI, or a template whose upstream +// endpoint does not expose IMEI at all). Callers must therefore distinguish +// "query succeeded with an empty value" from "query failed". +type SimImeiItem struct { + Imei string `json:"imei"` +} + +// SimCardStatusItem is the carrier status returned for one SIM card. +type SimCardStatusItem struct { + CardStatus int `json:"cardStatus"` + LastChangeDate string `json:"lastChangeDate"` +} + +type SimCardStatusChangeInput = SimBaseReq[SimCardStatusChangeInputItem] + +type SimCardStatusChangeInputItem struct { + OperType string `json:"operType"` +} + +type SimCardStatusChangeOutputItem struct { + OrderNum string `json:"orderNum"` +} + +type SimCardApnChangeInput = SimBaseReq[SimCardApnChangeInputItem] + +type SimCardApnChangeInputItem struct { + OperType string `json:"operType"` + ApnName string `json:"apnName"` +} + +type SimCardApnChangeOutputItem struct { + OrderNum string `json:"orderNum"` +} diff --git a/internal/model/work_order_queue.go b/internal/model/work_order_queue.go new file mode 100644 index 0000000..062974c --- /dev/null +++ b/internal/model/work_order_queue.go @@ -0,0 +1,13 @@ +package model + +// WorkOrderAutoCloseMessage is consumed after the auto-close waiting window. +type WorkOrderAutoCloseMessage struct { + WoNo string `json:"woNo"` + ResolveTimeTs int64 `json:"resolveTimeTs"` +} + +// BalanceRechargeTimeoutMessage is consumed after the Alipay recharge window. +type BalanceRechargeTimeoutMessage struct { + OutTradeNo string `json:"outTradeNo"` + RetryCount int `json:"retryCount"` +} diff --git a/internal/model/worker_diagnostic.go b/internal/model/worker_diagnostic.go new file mode 100644 index 0000000..fb8ca06 --- /dev/null +++ b/internal/model/worker_diagnostic.go @@ -0,0 +1,8 @@ +package model + +// WorkerDiagnosticMessage verifies the Admin-to-Worker RabbitMQ path. +type WorkerDiagnosticMessage struct { + TraceID string `json:"traceId"` + SentAt int64 `json:"sentAt"` + Message string `json:"message"` +} diff --git a/internal/packed/packed.go b/internal/packed/packed.go new file mode 100644 index 0000000..e20ab1e --- /dev/null +++ b/internal/packed/packed.go @@ -0,0 +1 @@ +package packed diff --git a/internal/producer/dailyusage/blocked_zombie_test.go b/internal/producer/dailyusage/blocked_zombie_test.go new file mode 100644 index 0000000..808f94a --- /dev/null +++ b/internal/producer/dailyusage/blocked_zombie_test.go @@ -0,0 +1,206 @@ +package dailyusage + +import ( + "context" + "errors" + "reflect" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +// Pause the real producer after its first enqueue, before MarkProducerDone. +type interruptedScanner struct { + *datasetScanner + paused chan struct{} +} + +func TestInterruptedRunReclamationGuards(t *testing.T) { + for _, mode := range []string{"serial", "channel"} { + for _, status := range []string{"producing", "running"} { + for _, guard := range []string{"no_lock", "foreign_lock", "no_current_run", "same_run", "producer_done", "replay"} { + t.Run(mode+"/"+status+"/"+guard, func(t *testing.T) { + ctx := contextForTest(t) + store := testStore(t) + p := testProducer(t, scannerFor(nil, 0), store, Config{ScanMode: mode}) + meta := zset.RunMeta{RunID: "guarded", Status: status, ProducerDone: guard == "producer_done"} + if guard == "replay" { + meta.SyncType = "replay" + } + if err := store.CreateRun(ctx, meta); err != nil { + t.Fatal(err) + } + current := "fresh" + owner := p.owner + if guard == "same_run" { + current = meta.RunID + } + if guard == "no_current_run" { + current = "" + } + if guard == "foreign_lock" { + owner = "other-owner" + } + if guard != "no_lock" { + if ok, err := store.AcquireLockOrTakeover(ctx, p.config.RunLockKey, owner, current, p.config.RunLockTTL); err != nil || !ok { + t.Fatalf("acquire=%t err=%v", ok, err) + } + } + key, _ := store.RunMetaKey(meta.RunID) + before, err := store.RedisClient().Do(ctx, "HGETALL", key) + if err != nil { + t.Fatal(err) + } + pending, err := p.hasPendingTasks(ctx, current) + if err != nil || pending != (guard != "same_run") { + t.Fatalf("guard=%s pending=%t err=%v", guard, pending, err) + } + after, err := store.RedisClient().Do(ctx, "HGETALL", key) + if err != nil || !reflect.DeepEqual(before.MapStrStr(), after.MapStrStr()) { + t.Fatalf("guarded metadata changed: before=%v after=%v err=%v", before, after, err) + } + }) + } + } + } +} + +func (s *interruptedScanner) Scan(ctx context.Context, last, upper int64, size int) ([]Card, error) { + if last > 0 { + close(s.paused) + <-ctx.Done() + return nil, ctx.Err() + } + return s.datasetScanner.Scan(ctx, last, upper, size) +} + +func (s *interruptedScanner) ScanChannel(ctx context.Context, key string, last int64, size int) ([]Card, error) { + if last > 0 { + close(s.paused) + <-ctx.Done() + return nil, ctx.Err() + } + return s.datasetScanner.ScanChannel(ctx, key, last, size) +} + +func TestBlockedInterruptedProducerRecovery(t *testing.T) { + for _, mode := range []string{"serial", "channel"} { + for _, backlog := range []string{"none", "queued", "leased", "waiting"} { + t.Run(mode+"/"+backlog, func(t *testing.T) { + ctx := contextForTest(t) + store := testStore(t) + scanner := &interruptedScanner{datasetScanner: scannerFor([]Card{{ID: 1, ChanKey: "blocked"}, {ID: 2, ChanKey: "blocked"}}, 2), paused: make(chan struct{})} + old := testProducer(t, scanner, store, Config{ScanMode: mode, PageSize: 1}) + oldCtx, interrupt := context.WithCancel(ctx) + defer interrupt() + result := make(chan error, 1) + go func() { result <- old.Produce(oldCtx, scopedMessage("interrupted")) }() + select { + case <-scanner.paused: + case err := <-result: + t.Fatalf("producer exited before pause: %v", err) + case <-ctx.Done(): + t.Fatal(ctx.Err()) + } + claim := func(channel string) zset.ClaimedTask { + t.Helper() + tasks, err := store.ClaimRun(ctx, "interrupted", channel, time.Now().Add(time.Second), time.Now().Add(time.Minute), 1) + if err != nil || len(tasks) != 1 { + t.Fatalf("claim %s=%v err=%v", channel, tasks, err) + } + return tasks[0] + } + if n, err := store.BlockRunScopedAsFailed(ctx, claim("blocked"), zset.TaskOutcome{Code: 797, ErrCode: "S00004"}); err != nil || n != 1 { + t.Fatalf("blocked=%d err=%v", n, err) + } + var remaining zset.ClaimedTask + if backlog != "none" { + tasks := []zset.Task{{TaskID: "remaining", RunID: "interrupted", ChanKey: "other"}} + if n, err := store.EnqueueRunBatch(ctx, "interrupted", "other", tasks, time.Now()); err != nil || n != 1 { + t.Fatalf("enqueue=%d err=%v", n, err) + } + if backlog != "queued" { + remaining = claim("other") + } + if backlog == "waiting" { + if ok, err := store.FinalizeRunTask(ctx, remaining, zset.TaskOutcome{Retry: true}); err != nil || !ok { + t.Fatalf("defer=%t err=%v", ok, err) + } + } + } + before, err := store.GetRunMeta(ctx, "interrupted") + if err != nil || before.Status != "running" || before.ProducerDone || before.FailedCount != 1 { + t.Fatalf("blocked metadata=%+v err=%v", before, err) + } + fresh := testProducer(t, scannerFor([]Card{{ID: 3, ChanKey: "fresh"}}, 3), store, Config{ScanMode: mode}) + if err := fresh.Produce(ctx, scopedMessage("fresh")); !errors.Is(err, ErrAlreadyRunning) { + t.Fatalf("live producer lock must prevent reclamation: %v", err) + } + requireMeta(t, store, "interrupted", false, before.ActiveCount) + interrupt() + if err := <-result; !errors.Is(err, context.Canceled) { + t.Fatalf("interruption=%v", err) + } + lock, err := store.RedisClient().Do(ctx, "EXISTS", old.config.RunLockKey) + if err != nil || lock.Int() != 0 { + t.Fatalf("old lock=%v err=%v", lock, err) + } + err = fresh.Produce(ctx, scopedMessage("fresh")) + if backlog == "none" && err != nil || backlog != "none" && !errors.Is(err, ErrAlreadyRunning) { + t.Fatalf("fresh start with %s backlog: %v", backlog, err) + } + after, err := store.GetRunMeta(ctx, "interrupted") + if err != nil || !after.ProducerDone || after.Status != "running" || after.ActiveCount != before.ActiveCount || after.InflightCount != before.InflightCount || after.NextRoundCount != before.NextRoundCount || after.FailedCount != before.FailedCount { + t.Fatalf("reclaimed metadata=%+v before=%+v err=%v", after, before, err) + } + if backlog != "none" { + if _, err := store.GetRunMeta(ctx, "fresh"); !errors.Is(err, zset.ErrRunNotFound) { + t.Fatalf("backlog must prevent creating fresh run: %v", err) + } + if complete, err := store.CompleteRunIfIdle(ctx, "interrupted"); err != nil || complete { + t.Fatalf("premature completion=%t err=%v", complete, err) + } + if backlog == "waiting" { + if advanced, err := store.AdvanceRunRound(ctx, "interrupted"); err != nil || !advanced { + t.Fatalf("advance=%t err=%v", advanced, err) + } + } + if backlog != "leased" { + remaining = claim("other") + } + if ok, err := store.FinalizeRunTask(ctx, remaining, zset.TaskOutcome{Success: true}); err != nil || !ok { + t.Fatalf("drain=%t err=%v", ok, err) + } + } + if complete, err := store.CompleteRunIfIdle(ctx, "interrupted", time.Hour); err != nil || !complete { + t.Fatalf("normal completion=%t err=%v", complete, err) + } + closed, err := store.GetRunMeta(ctx, "interrupted") + if err != nil || closed.Status != "completed_with_failures" || closed.FailedCount != 1 || closed.ActiveCount != 0 || closed.InflightCount != 0 || closed.NextRoundCount != 0 { + t.Fatalf("closed metadata=%+v err=%v", closed, err) + } + metaKey, _ := store.RunMetaKey("interrupted") + failedKey, _ := store.FailedKey("interrupted") + stateKey, _ := store.TaskStateKey("interrupted") + channelsKey, _ := store.RunChannelsKey("interrupted") + for _, key := range []string{metaKey, failedKey, stateKey, channelsKey} { + ttl, err := store.RedisClient().Do(ctx, "PTTL", key) + if err != nil || ttl.Int64() <= 0 || ttl.Int64() > time.Hour.Milliseconds() { + t.Fatalf("retention %s=%v err=%v", key, ttl, err) + } + } + if backlog != "none" { + if err := fresh.Produce(ctx, scopedMessage("fresh")); err != nil { + t.Fatalf("fresh start after draining: %v", err) + } + } + requireMeta(t, store, "fresh", true, 1) + active, err := store.ActiveRuns(ctx) + if err != nil || len(active) != 1 || active[0] != "fresh" { + t.Fatalf("only closed run must leave active index: %v err=%v", active, err) + } + }) + } + } +} diff --git a/internal/producer/dailyusage/channel_producer.go b/internal/producer/dailyusage/channel_producer.go new file mode 100644 index 0000000..dfb5f7a --- /dev/null +++ b/internal/producer/dailyusage/channel_producer.go @@ -0,0 +1,491 @@ +package dailyusage + +import ( + "context" + "errors" + "fmt" + "sort" + "sync" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +var errProducerDone = gerror.New("run production already closed") + +func producerStateError(state zset.ProducerState) error { + if state.Status == "completed" || state.Status == "completed_with_failures" { + return ErrRunFinished + } + if state.ProducerDone { + return errProducerDone + } + if state.Status != "producing" && state.Status != "running" { + return gerror.Newf("unexpected producer run status %q", state.Status) + } + return nil +} + +// produceChannels owns all per-attempt coordination. There is one worker per +// target, NOT a pool of four long-lived channel workers. SQL tokens and the +// cancellable admission gate never span sleeps or goroutine joins. +func (p *Producer) produceChannels(ctx context.Context, message model.TrafficUsageDailyStartMessage, scope *model.SyncScope, scanUpperID int64, startedAt time.Time, total *productionStats) error { + scanner := p.scanner.(channelScanner) + metadata, err := scanner.ListChannels(ctx) + if err != nil { + return gerror.Wrap(err, "initialize channel scan") + } + state, err := p.store.ReadProducerState(ctx, message.RunID, "") + if err != nil { + return err + } + if err = producerStateError(state); err != nil { + return err + } + targets := make(map[string]channelMetadata, len(metadata)) + known := make(map[string]bool, len(metadata)) + for _, channel := range metadata { + if channel.ChanKey == "" || !scope.Allows(channel.ChanKey) { + continue + } + if known[channel.ChanKey] { + return gerror.Newf("duplicate channel metadata %q", channel.ChanKey) + } + known[channel.ChanKey] = true + reason := "" + if channel.SyncEnabled { + reason = "sync_disabled" + } else if channel.TempKey == "" { + reason = "missing_channel_template" + } + if reason != "" { + total.skippedChannels++ + g.Log().Warningf(ctx, "event=daily_usage_channel status=skipped runId=%s chanKey=%s reason=%s skipUnit=channel", message.RunID, channel.ChanKey, reason) + continue + } + targets[channel.ChanKey] = channel + } + if scope != nil { + for _, chanKey := range scope.Channels { + if !known[chanKey] { + total.skippedChannels++ + g.Log().Warningf(ctx, "event=daily_usage_channel status=skipped runId=%s chanKey=%s reason=unknown_channel skipUnit=channel", message.RunID, chanKey) + } + } + } + g.Log().Infof(ctx, "event=daily_usage_channel_scan status=started runId=%s targetChannels=%d scanUpperID=%d scanConcurrency=%d pageSize=%d skipUnit=channel", message.RunID, len(targets), scanUpperID, p.config.ScanConcurrency, p.config.PageSize) + keys := make([]string, 0, len(targets)) + for key := range targets { + keys = append(keys, key) + } + sort.Strings(keys) + finishObservation := p.startObservation(ctx, message.RunID, keys, startedAt) + defer finishObservation() + var ( + workers sync.WaitGroup + slots = make(chan struct{}, p.config.ScanConcurrency) + gate = make(chan struct{}, 1) + metrics = newScanActivity() + results = make([]*channelScanStats, len(keys)) + ) + pool, hasPool := scanner.(interface { + poolWait(context.Context) (int64, int64) + }) + var poolCount, poolNanos int64 + if hasPool { + poolCount, poolNanos = pool.poolWait(ctx) + } + workCtx, cancel := context.WithCancelCause(ctx) + defer cancel(nil) + for i, key := range keys { + stats := &channelScanStats{chanKey: key, lastSummary: time.Now()} + results[i] = stats + workers.Add(1) + go func(channel channelMetadata, stats *channelScanStats) { + defer workers.Done() + scanErr := p.scanOneChannel(workCtx, scanner, message, channel, scanUpperID, startedAt, slots, gate, metrics, stats) + if scanErr != nil { + cancel(scanErr) + } + reason := stats.stopReason + if reason == "" { + reason = "scan_complete" + } + if scanErr != nil { + reason = "cancelled" + } + p.observe(channel.ChanKey, reason, 0, false, true) + stats.log(workCtx, message.RunID, "ended", startedAt, scanErr) + }(targets[key], stats) + } + workers.Wait() + var summary channelScanStats + for _, stats := range results { + total.pages += stats.scanPages + total.scanned += stats.eligible + total.eligible += stats.eligible + total.enqueued += stats.added + total.stoppedTasks += stats.rejected + if stats.eligible > 0 { + total.channels[stats.chanKey] = struct{}{} + } + summary.merge(stats) + } + metrics.log(ctx, message.RunID, &summary) + if hasPool { + count, nanos := pool.poolWait(ctx) + g.Log().Infof(ctx, "event=daily_usage_scan_pool runId=%s sharedPoolWaitCountDelta=%d sharedPoolWaitMsDelta=%d attribution=shared_pool", message.RunID, count-poolCount, (nanos-poolNanos)/int64(time.Millisecond)) + } + return context.Cause(workCtx) +} + +func acquireContext(ctx context.Context, token chan struct{}) error { + select { + case <-ctx.Done(): + return ctx.Err() + case token <- struct{}{}: + if err := ctx.Err(); err != nil { + <-token + return err + } + return nil + } +} + +func (p *Producer) scanOneChannel(ctx context.Context, scanner channelScanner, message model.TrafficUsageDailyStartMessage, channel channelMetadata, scanUpperID int64, startedAt time.Time, slots, gate chan struct{}, metrics *scanActivity, stats *channelScanStats) error { + for stats.cursor < scanUpperID { + // Detect a blocked/closed run before querying, including empty channels. + p.observe(channel.ChanKey, "redis_read", 0, false, false) + state, err := p.store.ReadProducerState(ctx, message.RunID, channel.ChanKey) + if err != nil { + return err + } + if err = producerStateError(state); err != nil { + return err + } + if state.Blocked { + // No page was read: the unknown unscanned remainder has no card + // count. Never turn it into a fabricated success/failure total. + stats.stopReason = "blocked_unscanned_remainder" + return nil + } + p.observe(channel.ChanKey, "sql_slot", 0, false, false) + waitStart := time.Now() + if err = acquireContext(ctx, slots); err != nil { + return err + } + stats.slotWait += time.Since(waitStart) + metrics.begin() + p.observe(channel.ChanKey, "sql_query", 0, false, false) + queryStart := time.Now() + cards, scanErr := scanner.ScanChannel(ctx, channel.ChanKey, stats.cursor, p.config.PageSize) + elapsed := time.Since(queryStart) + metrics.end() + <-slots // release immediately after results have been materialized + stats.sqlTime += elapsed + stats.sqlLatency.observe(elapsed) + stats.scanPages++ + if scanErr != nil { + return gerror.Wrapf(scanErr, "scan channel %s after %d", channel.ChanKey, stats.cursor) + } + if len(cards) > p.config.PageSize { + return gerror.New("scanner exceeded page size") + } + if len(cards) == 0 { + stats.stopReason = "empty_result" + return nil + } + var ( + pageCursor = stats.cursor + overUpper bool + tasks = make([]zset.Task, 0, len(cards)) + ) + for i, card := range cards { + if card.ChanKey != channel.ChanKey || card.ID <= pageCursor { + return gerror.New("channel scanner returned unordered or foreign cards") + } + if card.ID > scanUpperID { + stats.overUpper += int64(len(cards) - i) + overUpper = true + break + } + pageCursor = card.ID + tasks = append(tasks, zset.Task{TaskID: zset.StableTaskID(message.RunID, message.Operation, card.ID), RunID: message.RunID, Operation: message.Operation, QueryMonth: message.QueryDate, ChanKey: channel.ChanKey, TempKey: channel.TempKey, Operator: channel.Operator, CardID: card.ID, ICCID: card.ICCID, MSISDN: card.MSISDN, IMSI: card.IMSI, PackageCode: card.PackageCode}) + } + p.observe(channel.ChanKey, "page_ready", len(tasks), false, false) + stats.eligible += int64(len(tasks)) + blocked, err := p.admitChannelPage(ctx, message.RunID, tasks, startedAt, gate, stats) + if err != nil { + return err // page cursor is not committed, even after a partial write + } + if blocked { + stats.stopReason = "blocked" + return nil + } + stats.cursor = pageCursor + stats.progress(ctx, message.RunID, startedAt) + if overUpper || stats.cursor >= scanUpperID { + stats.stopReason = "upper_bound" + return nil + } + // Do not use a short page as EOF: query-time eligibility may change. + } + return ctx.Err() +} + +// admitChannelPage admits only new TaskIDs, in capacity-sized slices. Each +// worker retains at most this page. No capacity reservation survives the gate. +func (p *Producer) admitChannelPage(ctx context.Context, runID string, tasks []zset.Task, startedAt time.Time, gate chan struct{}, stats *channelScanStats) (bool, error) { + for len(tasks) > 0 { + p.observe(stats.chanKey, "admission_gate", len(tasks), false, false) + gateStart := time.Now() + if err := acquireContext(ctx, gate); err != nil { + return false, err + } + stats.gateWait += time.Since(gateStart) + blocked, waitKind, remaining, err := p.admitChannelSlice(ctx, runID, tasks, startedAt, stats) + <-gate + if err != nil || blocked { + return blocked, err + } + tasks = remaining + p.observe(stats.chanKey, "page_ready", len(tasks), false, false) + if waitKind != "" { + p.observe(stats.chanKey, waitKind+"_capacity", len(tasks), false, false) + waitStart := time.Now() + err = sleepContext(ctx, p.config.PollInterval) + if waitKind == "global" { + stats.globalWait += time.Since(waitStart) + } else { + stats.channelWait += time.Since(waitStart) + } + stats.progress(ctx, runID, startedAt) + if err != nil { + return false, err + } + } + } + return false, ctx.Err() +} + +// Caller holds the short admission gate; there is no sleep in this method. +func (p *Producer) admitChannelSlice(ctx context.Context, runID string, tasks []zset.Task, startedAt time.Time, stats *channelScanStats) (blocked bool, waitKind string, remaining []zset.Task, err error) { + p.observe(stats.chanKey, "redis_read", len(tasks), false, false) + readStart := time.Now() + state, err := p.store.ReadProducerState(ctx, runID, stats.chanKey) + if err != nil { + return false, "", tasks, err + } + if err = producerStateError(state); err != nil { + return false, "", tasks, err + } + existing, err := p.store.ExistingRunTasks(ctx, runID, tasks) + stats.stateTime += time.Since(readStart) + if err != nil { + return false, "", tasks, err + } + pending := tasks[:0] + for i, exists := range existing { + if exists { + stats.duplicates++ + } else { + pending = append(pending, tasks[i]) + } + } + tasks = pending + p.observe(stats.chanKey, "redis_read", len(tasks), false, false) + if state.Blocked { + stats.rejected += int64(len(tasks)) + return true, "", nil, nil + } + if len(tasks) == 0 { + return false, "", nil, nil + } + space := int64(len(tasks)) + if limit := p.config.GlobalHighWaterMark; limit > 0 { + space = min(space, limit-state.ActiveCount) // inflight is already included + if space <= 0 { + return false, "global", tasks, nil + } + } + if limit := p.config.ChannelHighWaterMark; limit > 0 { + key, keyErr := p.store.RoundActiveKey(runID, state.CurrentRound, stats.chanKey) + if keyErr != nil { + return false, "", tasks, keyErr + } + readStart = time.Now() + size, sizeErr := p.store.SizeKey(ctx, key) + stats.stateTime += time.Since(readStart) + if sizeErr != nil { + return false, "", tasks, sizeErr + } + space = min(space, limit-size) + if space <= 0 { + return false, "channel", tasks, nil + } + } + batch := tasks[:int(space)] + p.observe(stats.chanKey, "redis_write", len(tasks), false, false) + enqueueStart := time.Now() + added, err := p.store.EnqueueRunBatch(ctx, runID, stats.chanKey, batch, time.Now()) + stats.enqueueTime += time.Since(enqueueStart) + stats.enqueueBatches++ + if err != nil { + return false, "", tasks, err // uncertain responses require idempotent rescan + } + if added < 0 || added > len(batch) { + return false, "", tasks, gerror.New("invalid enqueue count") + } + stats.added += int64(added) + p.observe(stats.chanKey, "redis_write", len(tasks)-len(batch), added > 0, false) + if added > 0 && !stats.firstEnqueueRecorded { + stats.firstEnqueueRecorded = true + stats.firstEnqueue = time.Since(startedAt) + stats.log(ctx, runID, "first_enqueue", startedAt, nil) + } + if added < len(batch) { + readStart = time.Now() + confirmed, checkErr := p.store.ExistingRunTasks(ctx, runID, batch) + if checkErr != nil { + return false, "", tasks, checkErr + } + state, checkErr = p.store.ReadProducerState(ctx, runID, stats.chanKey) + stats.stateTime += time.Since(readStart) + if checkErr != nil { + return false, "", tasks, checkErr + } + if checkErr = producerStateError(state); checkErr != nil { + return false, "", tasks, checkErr + } + present := 0 + for _, exists := range confirmed { + if exists { + present++ + } + } + if present < added { + return false, "", tasks, gerror.New("enqueue count exceeds confirmed task states") + } + stats.duplicates += int64(present - added) + if state.Blocked { + stats.rejected += int64(len(tasks) - present) + return true, "", nil, nil + } + if present != len(batch) { + return false, "", tasks, gerror.Newf("unexplained enqueue rejection: added=%d submitted=%d confirmed=%d", added, len(batch), present) + } + } + return false, "", tasks[len(batch):], nil +} + +// Retained only to keep errors.Is semantics visible at the call site. +func isProductionClosed(err error) bool { return errors.Is(err, errProducerDone) } + +func durationMS(d time.Duration) float64 { return float64(d) / float64(time.Millisecond) } + +// latencyHistogram is fixed-size. Reported percentiles are bucket upper bounds, +// never exact samples or a server-side SLA; the last bucket is unbounded. +var latencyBounds = [...]time.Duration{time.Millisecond, 2 * time.Millisecond, 5 * time.Millisecond, 10 * time.Millisecond, 20 * time.Millisecond, 50 * time.Millisecond, 100 * time.Millisecond, 200 * time.Millisecond, 500 * time.Millisecond, time.Second, 2 * time.Second, 5 * time.Second, 10 * time.Second, 30 * time.Second, time.Minute} + +type latencyHistogram struct{ counts [16]int64 } + +func (h *latencyHistogram) observe(d time.Duration) { + i := 0 + for i < len(latencyBounds) && d > latencyBounds[i] { + i++ + } + h.counts[i]++ +} +func (h *latencyHistogram) percentile(percent int64) string { + var total, cumulative int64 + for _, count := range h.counts { + total += count + } + if total == 0 { + return "n/a" + } + target := (total*percent + 99) / 100 + for i, count := range h.counts { + cumulative += count + if cumulative >= target { + if i == len(latencyBounds) { + return ">60000" + } + return fmt.Sprintf("%.0f", durationMS(latencyBounds[i])) + } + } + return "n/a" +} + +type channelScanStats struct { + firstEnqueueRecorded bool + chanKey, stopReason string + cursor, scanPages, eligible, overUpper, enqueueBatches, added, duplicates, rejected int64 + sqlTime, slotWait, gateWait, globalWait, channelWait, stateTime, enqueueTime, firstEnqueue time.Duration + sqlLatency latencyHistogram + lastSummary time.Time +} + +func (s *channelScanStats) log(ctx context.Context, runID, status string, startedAt time.Time, err error) { + errorText := "" + if err != nil { + errorText = err.Error() + } + g.Log().Infof(ctx, "event=daily_usage_channel_scan status=%s runId=%s chanKey=%s cursor=%d scanPages=%d eligible=%d overUpper=%d enqueueBatches=%d added=%d duplicates=%d rejected=%d sqlMs=%.3f sqlP50UpperMs=%s sqlP95UpperMs=%s slotWaitMs=%.3f admissionWaitMs=%.3f globalCapacityWaitMs=%.3f channelCapacityWaitMs=%.3f stateReadMs=%.3f enqueueMs=%.3f firstEnqueueMs=%.3f wallMs=%d stopReason=%s error=%q", status, runID, s.chanKey, s.cursor, s.scanPages, s.eligible, s.overUpper, s.enqueueBatches, s.added, s.duplicates, s.rejected, durationMS(s.sqlTime), s.sqlLatency.percentile(50), s.sqlLatency.percentile(95), durationMS(s.slotWait), durationMS(s.gateWait), durationMS(s.globalWait), durationMS(s.channelWait), durationMS(s.stateTime), durationMS(s.enqueueTime), durationMS(s.firstEnqueue), time.Since(startedAt).Milliseconds(), s.stopReason, errorText) +} +func (s *channelScanStats) progress(ctx context.Context, runID string, startedAt time.Time) { + if time.Since(s.lastSummary) >= time.Minute { + s.lastSummary = time.Now() + s.log(ctx, runID, "progress", startedAt, nil) + } +} +func (s *channelScanStats) merge(other *channelScanStats) { + s.scanPages += other.scanPages + s.eligible += other.eligible + s.overUpper += other.overUpper + s.enqueueBatches += other.enqueueBatches + s.added += other.added + s.duplicates += other.duplicates + s.rejected += other.rejected + s.sqlTime += other.sqlTime + s.slotWait += other.slotWait + s.gateWait += other.gateWait + s.globalWait += other.globalWait + s.channelWait += other.channelWait + s.stateTime += other.stateTime + s.enqueueTime += other.enqueueTime + for i, count := range other.sqlLatency.counts { + s.sqlLatency.counts[i] += count + } +} + +type scanActivity struct { + mu sync.Mutex + started, changed time.Time + running, peak int + area time.Duration +} + +func newScanActivity() *scanActivity { + now := time.Now() + return &scanActivity{started: now, changed: now} +} +func (s *scanActivity) change(delta int) { + s.mu.Lock() + defer s.mu.Unlock() + now := time.Now() + s.area += time.Duration(s.running) * now.Sub(s.changed) + s.changed = now + s.running += delta + s.peak = max(s.peak, s.running) +} +func (s *scanActivity) begin() { s.change(1) } +func (s *scanActivity) end() { s.change(-1) } +func (s *scanActivity) log(ctx context.Context, runID string, stats *channelScanStats) { + wall := time.Since(s.started) + g.Log().Infof(ctx, "event=daily_usage_scan_summary runId=%s wallMs=%d sqlInFlightPeak=%d sqlInFlightAvg=%.4f scanPages=%d enqueueBatches=%d eligible=%d overUpper=%d added=%d duplicates=%d rejected=%d sqlCumulativeMs=%.3f sqlP50UpperMs=%s sqlP95UpperMs=%s slotWaitCumulativeMs=%.3f admissionWaitCumulativeMs=%.3f globalCapacityWaitCumulativeMs=%.3f channelCapacityWaitCumulativeMs=%.3f stateReadCumulativeMs=%.3f enqueueCumulativeMs=%.3f", runID, wall.Milliseconds(), s.peak, float64(s.area)/float64(wall), stats.scanPages, stats.enqueueBatches, stats.eligible, stats.overUpper, stats.added, stats.duplicates, stats.rejected, durationMS(stats.sqlTime), stats.sqlLatency.percentile(50), stats.sqlLatency.percentile(95), durationMS(stats.slotWait), durationMS(stats.gateWait), durationMS(stats.globalWait), durationMS(stats.channelWait), durationMS(stats.stateTime), durationMS(stats.enqueueTime)) +} diff --git a/internal/producer/dailyusage/channel_producer_test.go b/internal/producer/dailyusage/channel_producer_test.go new file mode 100644 index 0000000..0f52318 --- /dev/null +++ b/internal/producer/dailyusage/channel_producer_test.go @@ -0,0 +1,539 @@ +package dailyusage + +import ( + "context" + "errors" + "fmt" + "reflect" + "sort" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" + "github.com/gogf/gf/v2/container/gvar" +) + +// datasetScanner applies the SQL eligibility predicate before LIMIT; metadata +// eligibility is applied by the producer. Optional hooks simulate SQL latency, +// live additions and failures without accessing an application database. +type datasetScanner struct { + cards []Card + channels []channelMetadata + upper int64 + hook func(context.Context, string, int64) error + queries atomic.Int32 + lists atomic.Int32 + inflight atomic.Int32 + peak atomic.Int32 +} + +func (s *datasetScanner) SnapshotMaxID(context.Context) (int64, error) { return s.upper, nil } +func (s *datasetScanner) ListChannels(context.Context) ([]channelMetadata, error) { + s.lists.Add(1) + return s.channels, nil +} +func (s *datasetScanner) Scan(ctx context.Context, last, upper int64, size int) ([]Card, error) { + var rows []Card + for _, card := range s.cards { + if card.ID > last && card.ID <= upper { + for _, channel := range s.channels { + if channel.ChanKey == card.ChanKey { + card.TempKey = channel.TempKey + card.SyncEnabled = channel.SyncEnabled + card.Operator = channel.Operator + } + } + rows = append(rows, card) + if len(rows) == size { + break + } + } + } + return rows, nil +} +func (s *datasetScanner) ScanChannel(ctx context.Context, key string, last int64, size int) ([]Card, error) { + s.queries.Add(1) + running := s.inflight.Add(1) + defer s.inflight.Add(-1) + for { + old := s.peak.Load() + if running <= old || s.peak.CompareAndSwap(old, running) { + break + } + } + if s.hook != nil { + if err := s.hook(ctx, key, last); err != nil { + return nil, err + } + } + var rows []Card + for _, card := range s.cards { + if card.ChanKey == key && card.ID > last { + rows = append(rows, card) + if len(rows) == size { + break + } + } + } + return rows, nil +} +func scannerFor(cards []Card, upper int64) *datasetScanner { + keys := map[string]bool{} + s := &datasetScanner{cards: cards, upper: upper} + for _, card := range cards { + if card.ChanKey != "" && !keys[card.ChanKey] { + keys[card.ChanKey] = true + s.channels = append(s.channels, channelMetadata{ChanKey: card.ChanKey, TempKey: "sim_yd_v5", Operator: 1}) + } + } + return s +} +func testStore(t *testing.T) *zset.Store { + t.Helper() + store, err := zset.New(testutil.Redis(t), "flow:") + if err != nil { + t.Fatal(err) + } + return store +} +func testProducer(t *testing.T, scanner CardScanner, store *zset.Store, config Config) *Producer { + t.Helper() + config.RoundRetryEnabled = true + if config.ScanMode == "" { + config.ScanMode = "channel" + } + p, err := New(scanner, store, config) + if err != nil { + t.Fatal(err) + } + return p +} +func contextForTest(t *testing.T) context.Context { + t.Helper() + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + t.Cleanup(cancel) + return ctx +} +func taskIDs(t *testing.T, store *zset.Store, run string) []string { + t.Helper() + key, _ := store.TaskStateKey(run) + result, err := store.RedisClient().Do(context.Background(), "HKEYS", key) + if err != nil { + t.Fatal(err) + } + ids := result.Strings() + sort.Strings(ids) + return ids +} +func requireMeta(t *testing.T, store *zset.Store, run string, done bool, active int64) { + t.Helper() + meta, err := store.GetRunMeta(context.Background(), run) + if err != nil { + t.Fatal(err) + } + if meta.ProducerDone != done || meta.ActiveCount != active { + t.Fatalf("meta=%+v, want done=%t active=%d", meta, done, active) + } +} +func waitUntil(t *testing.T, ctx context.Context, condition func() bool) { + t.Helper() + for !condition() { + select { + case <-ctx.Done(): + t.Fatal(ctx.Err()) + case <-time.After(time.Millisecond): + } + } +} + +func TestChannelModeSettings(t *testing.T) { + store, _ := zset.New(&producerRedis{}, "flow:") + p, err := New(&fakeScanner{}, store, Config{}) + if err != nil || p.config.ScanMode != "serial" || p.config.ScanConcurrency != 4 { + t.Fatalf("defaults: p=%+v err=%v", p, err) + } + for _, config := range []Config{{ScanMode: "parallel"}, {ScanConcurrency: -1}, {ScanMode: "channel"}} { + if _, err = New(&fakeScanner{}, store, config); err == nil { + t.Fatalf("accepted invalid config/capability %+v", config) + } + } +} + +func TestChannelScanFiniteMemoryBoundary(t *testing.T) { + for _, test := range []struct { + name string + upper int64 + cards []Card + ids []int64 + queries int32 + }{ + {"zero upper", 0, []Card{{ID: 1, ChanKey: "a"}}, nil, 0}, + {"exact page", 4, []Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "a"}, {ID: 3, ChanKey: "a"}, {ID: 4, ChanKey: "a"}}, []int64{1, 2, 3, 4}, 2}, + {"tail then empty", 9, []Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "a"}, {ID: 3, ChanKey: "a"}}, []int64{1, 2, 3}, 3}, + {"first outside", 3, []Card{{ID: 4, ChanKey: "a"}, {ID: 5, ChanKey: "a"}}, nil, 1}, + {"mixed final page", 3, []Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "a"}, {ID: 3, ChanKey: "a"}, {ID: 4, ChanKey: "a"}, {ID: 5, ChanKey: "a"}}, []int64{1, 2, 3}, 2}, + {"empty channel", 9, nil, nil, 1}, + } { + t.Run(test.name, func(t *testing.T) { + store := testStore(t) + scanner := scannerFor(test.cards, test.upper) + if len(scanner.channels) == 0 { + scanner.channels = []channelMetadata{{ChanKey: "a", TempKey: "sim_yd_v5"}} + } + p := testProducer(t, scanner, store, Config{PageSize: 2}) + if err := p.Produce(contextForTest(t), scopedMessage("boundary")); err != nil { + t.Fatal(err) + } + var want []string + for _, id := range test.ids { + want = append(want, zset.StableTaskID("boundary", operation, id)) + } + sort.Strings(want) + got := taskIDs(t, store, "boundary") + if len(got) != len(want) || (len(want) > 0 && !reflect.DeepEqual(got, want)) { + t.Fatalf("got=%v want=%v", got, want) + } + if scanner.queries.Load() != test.queries { + t.Fatalf("queries=%d want=%d", scanner.queries.Load(), test.queries) + } + requireMeta(t, store, "boundary", true, int64(len(want))) + }) + } +} + +func TestChannelScanLiveHigherIDsDoNotExtendRun(t *testing.T) { + store := testStore(t) + scanner := scannerFor([]Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "a"}, {ID: 3, ChanKey: "a"}}, 3) + scanner.hook = func(_ context.Context, _ string, last int64) error { + if last > 0 { + scanner.cards = append(scanner.cards, Card{ID: 100 + last, ChanKey: "a"}) + } + return nil + } + p := testProducer(t, scanner, store, Config{PageSize: 2}) + if err := p.Produce(contextForTest(t), scopedMessage("live")); err != nil { + t.Fatal(err) + } + if len(taskIDs(t, store, "live")) != 3 || scanner.queries.Load() != 2 { + t.Fatal("higher IDs extended finite range") + } +} + +func TestChannelScanSerialTaskIDSetParity(t *testing.T) { + for _, scope := range [][]string{nil, {"a", "b"}, {"disabled", "unknown"}} { + t.Run(fmt.Sprint(scope), func(t *testing.T) { + store := testStore(t) + cards := []Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "b"}, {ID: 3, ChanKey: "a"}, {ID: 4, ChanKey: "disabled"}, {ID: 5, ChanKey: "no-template"}, {ID: 6, ChanKey: "unknown"}, {ID: 7, ChanKey: "b"}, {ID: 8, ChanKey: "a"}} + var sets [][]string + for _, mode := range []string{"serial", "channel"} { + scanner := scannerFor(cards, 7) + scanner.channels = []channelMetadata{{ChanKey: "a", TempKey: "sim_yd_v5", Operator: 1}, {ChanKey: "b", TempKey: "sim_dx_v1", Operator: 2}, {ChanKey: "disabled", TempKey: "sim_yd_v5", SyncEnabled: true}, {ChanKey: "no-template"}} + p := testProducer(t, scanner, store, Config{ScanMode: mode, PageSize: 2}) + run := "parity-" + mode + if err := p.Produce(contextForTest(t), scopedMessage(run, scope...)); err != nil { + t.Fatal(err) + } + ids := taskIDs(t, store, run) + for i, id := range ids { + ids[i] = strings.TrimPrefix(id, run+":") + } + sets = append(sets, ids) + if mode == "channel" && scanner.lists.Load() != 1 { + t.Fatal("metadata not loaded once") + } + _, _ = store.RedisClient().Do(context.Background(), "SREM", store.ActiveRunsKey(), run) + } + if !reflect.DeepEqual(sets[0], sets[1]) { + t.Fatalf("serial=%v channel=%v", sets[0], sets[1]) + } + }) + } +} + +func TestChannelSQLSlotsReleasedWhileChannelsWait(t *testing.T) { + store := testStore(t) + ctx := contextForTest(t) + var cards []Card + for i := 0; i < 9; i++ { + cards = append(cards, Card{ID: int64(i + 1), ChanKey: fmt.Sprint(i)}, Card{ID: int64(i + 10), ChanKey: fmt.Sprint(i)}) + } + sort.Slice(cards, func(i, j int) bool { return cards[i].ID < cards[j].ID }) + scanner := scannerFor(cards, 100) + scanner.hook = func(ctx context.Context, _ string, _ int64) error { return sleepContext(ctx, 5*time.Millisecond) } + p := testProducer(t, scanner, store, Config{PageSize: 2, ScanConcurrency: 4, ChannelHighWaterMark: 1, GlobalHighWaterMark: 100, PollInterval: time.Millisecond}) + result := make(chan error, 1) + go func() { result <- p.Produce(ctx, scopedMessage("isolation")) }() + waitUntil(t, ctx, func() bool { return len(taskIDs(t, store, "isolation")) == 9 }) + if peak := scanner.peak.Load(); peak > 4 || peak < 2 { + t.Fatalf("SQL peak=%d", peak) + } + // All nine channels got a first task despite the first four waiting at + // channel capacity. SQL slots are free while every channel is paused. + if scanner.inflight.Load() != 0 { + t.Fatal("SQL tokens held during channel backpressure") + } + for _, channel := range scanner.channels { + if _, err := store.BlockRunScoped(ctx, "isolation", channel.ChanKey); err != nil { + t.Fatal(err) + } + } + if err := <-result; err != nil { + t.Fatal(err) + } + meta, err := store.GetRunMeta(ctx, "isolation") + if err != nil || !meta.ProducerDone { + t.Fatalf("meta=%+v err=%v", meta, err) + } +} + +func TestChannelRemainingCapacityAndDuplicateAdmission(t *testing.T) { + store := testStore(t) + ctx := contextForTest(t) + scanner := scannerFor(nil, 0) + p := testProducer(t, scanner, store, Config{GlobalHighWaterMark: 3, ChannelHighWaterMark: 2, PollInterval: time.Millisecond}) + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "capacity", Status: "producing"}); err != nil { + t.Fatal(err) + } + // Existing success and waiting states still deduplicate and consume no new + // capacity. Active includes leases: inflight must not be added again. + stateKey, _ := store.TaskStateKey("capacity") + metaKey, _ := store.RunMetaKey("capacity") + activeKey, _ := store.RoundActiveKey("capacity", 1, "a") + _, _ = store.RedisClient().Do(ctx, "HSET", stateKey, "one", `{"status":"success"}`, "two", `{"status":"waiting"}`) + _, _ = store.RedisClient().Do(ctx, "HSET", metaKey, "activeCount", 2, "inflightCount", 2) + stats := &channelScanStats{chanKey: "a", lastSummary: time.Now()} + tasks := []zset.Task{{TaskID: "one", RunID: "capacity", ChanKey: "a"}, {TaskID: "two", RunID: "capacity", ChanKey: "a"}, {TaskID: "three", RunID: "capacity", ChanKey: "a"}, {TaskID: "four", RunID: "capacity", ChanKey: "a"}} + gate := make(chan struct{}, 1) + result := make(chan error, 1) + go func() { _, err := p.admitChannelPage(ctx, "capacity", tasks, time.Now(), gate, stats); result <- err }() + waitUntil(t, ctx, func() bool { meta, e := store.GetRunMeta(ctx, "capacity"); return e == nil && meta.ActiveCount == 3 }) + if size, err := store.SizeKey(ctx, activeKey); err != nil || size != 1 { + t.Fatalf("initial slice size=%d err=%v", size, err) + } + _, _ = store.RedisClient().Do(ctx, "HSET", metaKey, "activeCount", 0, "inflightCount", 0) + if err := <-result; err != nil { + t.Fatal(err) + } + if stats.duplicates != 2 || stats.added != 2 || stats.enqueueBatches != 2 || stats.globalWait == 0 { + t.Fatalf("stats=%+v", stats) + } +} + +// faultRedis forwards to a real isolated Redis, optionally returning a lost +// response after Lua succeeded. Never mocks the Lua semantics under test. +type faultRedis struct { + zset.RedisClient + failEnqueue atomic.Int32 + failRead atomic.Bool + failRenew atomic.Bool + lostResponse bool + reject bool +} + +func (r *faultRedis) Do(ctx context.Context, command string, args ...any) (*gvar.Var, error) { + if r.failRead.Load() && command == "HMGET" { + return nil, errors.New("Redis read failure") + } + return r.RedisClient.Do(ctx, command, args...) +} +func (r *faultRedis) Eval(ctx context.Context, script string, n int64, keys []string, args []any) (*gvar.Var, error) { + if r.failRenew.Load() && strings.Contains(script, "PEXPIRE") && strings.Contains(script, "owner ~=") { + return gvar.New(0), nil + } + if n == 4 && strings.Contains(script, "HEXISTS") { + if r.reject { + return gvar.New(0), nil + } + if r.failEnqueue.CompareAndSwap(1, 0) { + if r.lostResponse { + if _, err := r.RedisClient.Eval(ctx, script, n, keys, args); err != nil { + return nil, err + } + } + return nil, errors.New("enqueue response unavailable") + } + } + return r.RedisClient.Eval(ctx, script, n, keys, args) +} + +func TestChannelRescanPartialAndUncertainWrites(t *testing.T) { + for _, uncertain := range []bool{false, true} { + t.Run(fmt.Sprint(uncertain), func(t *testing.T) { + client := &faultRedis{RedisClient: testutil.Redis(t), lostResponse: uncertain} + store, _ := zset.New(client, "flow:") + scanner := scannerFor([]Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "a"}, {ID: 3, ChanKey: "a"}, {ID: 4, ChanKey: "a"}}, 4) + // Fail the second page: first page must survive; uncertain success is + // confirmed only on a same-Run rescan, with stable IDs and no double count. + scanner.hook = func(_ context.Context, _ string, last int64) error { + if last == 2 { + client.failEnqueue.Store(1) + } + return nil + } + p := testProducer(t, scanner, store, Config{PageSize: 2}) + if err := p.Produce(contextForTest(t), scopedMessage("rescan")); err == nil { + t.Fatal("expected enqueue failure") + } + active := int64(2) + if uncertain { + active = 4 + } + requireMeta(t, store, "rescan", false, active) + scanner.hook = nil + if err := p.Produce(contextForTest(t), scopedMessage("rescan")); err != nil { + t.Fatal(err) + } + requireMeta(t, store, "rescan", true, 4) + if len(taskIDs(t, store, "rescan")) != 4 { + t.Fatal("wrong stable task set") + } + before := scanner.queries.Load() + if err := p.Produce(contextForTest(t), scopedMessage("rescan")); err != nil { + t.Fatal(err) + } + if before != scanner.queries.Load() { + t.Fatal("producerDone redelivery rescanned cards") + } + }) + } +} + +func TestChannelFailureCancelsAllWorkersWithoutProducerDone(t *testing.T) { + for _, kind := range []string{"sql", "redis", "unexplained_rejection", "lease"} { + t.Run(kind, func(t *testing.T) { + client := &faultRedis{RedisClient: testutil.Redis(t)} + store, _ := zset.New(client, "flow:") + cards := []Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "b"}, {ID: 3, ChanKey: "c"}} + scanner := scannerFor(cards, 10) + allStarted := make(chan struct{}) + var started atomic.Int32 + var once sync.Once + scanner.hook = func(ctx context.Context, key string, _ int64) error { + if started.Add(1) >= 3 { + once.Do(func() { close(allStarted) }) + } + select { + case <-allStarted: + case <-ctx.Done(): + return ctx.Err() + } + if kind == "lease" { + client.failRenew.Store(true) + <-ctx.Done() + return ctx.Err() + } + if key == "a" { + switch kind { + case "sql": + return errors.New("SQL failed") + case "redis": + client.failRead.Store(true) + case "unexplained_rejection": + client.reject = true + } + return nil + } + <-ctx.Done() + return ctx.Err() + } + p := testProducer(t, scanner, store, Config{PageSize: 2, RunLockTTL: 90 * time.Millisecond}) + if err := p.Produce(contextForTest(t), scopedMessage("failure")); err == nil { + t.Fatal("expected fatal failure") + } + if scanner.inflight.Load() != 0 { + t.Fatal("scanner goroutines leaked") + } + meta, err := store.GetRunMeta(context.Background(), "failure") + if err != nil || meta.ProducerDone { + t.Fatalf("partial scan marked done: %+v err=%v", meta, err) + } + }) + } +} + +func TestChannelScanStateStopsAndNoTarget(t *testing.T) { + for _, kind := range []string{"terminal", "producerDone", "blocked", "missing", "no-target", "no-cards"} { + t.Run(kind, func(t *testing.T) { + store := testStore(t) + scanner := scannerFor([]Card{{ID: 1, ChanKey: "a"}}, 2) + if kind == "no-target" { + scanner.channels[0].SyncEnabled = true + } + if kind == "no-cards" { + scanner.cards = nil + } + p := testProducer(t, scanner, store, Config{}) + if kind != "no-target" && kind != "no-cards" { + if err := store.CreateRun(context.Background(), zset.RunMeta{RunID: "stop", Status: "producing"}); err != nil { + t.Fatal(err) + } + key, _ := store.RunMetaKey("stop") + switch kind { + case "terminal": + _, _ = store.RedisClient().Do(context.Background(), "HSET", key, "status", "completed") + case "producerDone": + if err := store.MarkProducerDone(context.Background(), "stop"); err != nil { + t.Fatal(err) + } + case "blocked": + _, _ = store.BlockRunScoped(context.Background(), "stop", "a") + case "missing": + scanner.hook = func(ctx context.Context, _ string, _ int64) error { + _, err := store.RedisClient().Do(ctx, "DEL", key) + return err + } + } + } + err := p.Produce(contextForTest(t), scopedMessage("stop")) + if kind == "terminal" { + if !errors.Is(err, ErrRunFinished) { + t.Fatalf("err=%v", err) + } + } else if kind == "missing" { + if !errors.Is(err, zset.ErrRunNotFound) { + t.Fatalf("err=%v", err) + } + } else if err != nil { + t.Fatal(err) + } + if len(taskIDs(t, store, "stop")) != 0 { + t.Fatal("stopped/empty run enqueued tasks") + } + }) + } +} + +func TestChannelOldNonRunPathRemainsSerial(t *testing.T) { + redis := &producerRedis{batches: make(map[string]int)} + store, _ := zset.New(redis, "flow:") + scanner := scannerFor([]Card{{ID: 1, ChanKey: "a"}}, 1) + p, err := New(scanner, store, Config{ScanMode: "channel", RoundRetryEnabled: false}) + if err != nil { + t.Fatal(err) + } + if err = p.Produce(context.Background(), scopedMessage("legacy")); err != nil { + t.Fatal(err) + } + if scanner.queries.Load() != 0 || scanner.lists.Load() != 0 { + t.Fatal("non-Run path invoked channel scanner") + } +} + +const operation = FullSyncOperation + +type fakeScanner struct{} + +func (*fakeScanner) SnapshotMaxID(context.Context) (int64, error) { return 0, nil } +func (*fakeScanner) Scan(context.Context, int64, int64, int) ([]Card, error) { return nil, nil } +func scopedMessage(run string, channels ...string) model.TrafficUsageDailyStartMessage { + m := model.TrafficUsageDailyStartMessage{RunID: run, QueryDate: "20261012", Operation: FullSyncOperation} + if len(channels) > 0 { + m.Scope = &model.SyncScope{Channels: channels} + } + return m +} diff --git a/internal/producer/dailyusage/channel_scanner.go b/internal/producer/dailyusage/channel_scanner.go new file mode 100644 index 0000000..780f22e --- /dev/null +++ b/internal/producer/dailyusage/channel_scanner.go @@ -0,0 +1,81 @@ +package dailyusage + +import ( + "context" + + "github.com/gogf/gf/v2/errors/gerror" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +// channelScanner augments, rather than replaces, the legacy scanner contract. +// ListChannels is called once before any workers start. ScanChannel must not +// mutate the legacy DatabaseScanner.channelMetadata cache. +type channelScanner interface { + ListChannels(context.Context) ([]channelMetadata, error) + ScanChannel(context.Context, string, int64, int) ([]Card, error) +} + +// This exact MySQL query shape is a release gate. Do not silently remove hints, +// add an SQL upper bound, or move eligibility filtering after LIMIT. +const channelPageSQL = `SELECT c.id, c.iccid, c.msisdn, c.imsi, c.chan_key, c.current_package_code +FROM t_sim_card_info AS c FORCE INDEX (idx_card_chan_key) +STRAIGHT_JOIN sys_dept AS d + ON c.dept_id = d.dept_id + AND c.tenant_id = d.tenant_id +WHERE c.chan_key = ? + AND c.id > ? + AND c.iccid <> '' + AND c.current_package_code <> '' + AND d.deleted_at IS NULL +ORDER BY c.id ASC +LIMIT ?` + +func (s *DatabaseScanner) ListChannels(ctx context.Context) ([]channelMetadata, error) { + var metadata []channelMetadata + if err := dao.TSimCardInfo.DB().Model("t_sim_channels").Ctx(ctx).Fields("chan_key,temp_key,sync_enabled,operator").Scan(&metadata); err != nil { + return nil, gerror.Wrap(err, "list channel metadata") + } + return metadata, nil +} + +func (s *DatabaseScanner) ScanChannel(ctx context.Context, chanKey string, lastID int64, pageSize int) ([]Card, error) { + if chanKey == "" || lastID < 0 || pageSize <= 0 { + return nil, gerror.New("invalid channel pagination parameters") + } + var rows []entity.TSimCardInfo + // Use the card DAO's database group, not a separately configured connection. + if err := dao.TSimCardInfo.DB().Raw(channelPageSQL, chanKey, lastID, pageSize).Ctx(ctx).Scan(&rows); err != nil { + return nil, gerror.Wrap(err, "scan channel cards") + } + cards := make([]Card, 0, len(rows)) + for _, row := range rows { + cards = append(cards, Card{ID: row.Id, ICCID: row.Iccid, MSISDN: row.Msisdn, IMSI: row.Imsi, ChanKey: row.ChanKey, PackageCode: row.CurrentPackageCode}) + } + return cards, nil +} + +// Pool counters are shared with other worker traffic. Their deltas are direct +// observations, not an attribution of all connection waits to this producer. +func (*DatabaseScanner) poolWait(ctx context.Context) (int64, int64) { + var count, nanos int64 + for _, item := range dao.TSimCardInfo.DB().Stats(ctx) { + stats := item.Stats() + count += stats.WaitCount + nanos += int64(stats.WaitDuration) + } + return count, nanos +} + +// ValidateScanSettings is strict for values read from configuration. The Go +// constructor first fills omitted zero values; explicit YAML zero is rejected. +func ValidateScanSettings(mode string, concurrency int) error { + if mode != "serial" && mode != "channel" { + return gerror.Newf("invalid trafficUsageDailySync.producer.scanMode %q (serial or channel required)", mode) + } + if concurrency <= 0 { + return gerror.New("trafficUsageDailySync.producer.scanConcurrency must be positive") + } + return nil +} diff --git a/internal/producer/dailyusage/channel_scanner_test.go b/internal/producer/dailyusage/channel_scanner_test.go new file mode 100644 index 0000000..a71b6f7 --- /dev/null +++ b/internal/producer/dailyusage/channel_scanner_test.go @@ -0,0 +1,123 @@ +package dailyusage + +import ( + "context" + "reflect" + "sort" + "strings" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + _ "github.com/gogf/gf/contrib/drivers/sqlite/v2" + "github.com/gogf/gf/v2/database/gdb" +) + +// SQLite runs the SAME predicate and ordering with only the two MySQL hints +// removed. This checks full eligibility/TaskID parity, not MySQL access plans. +// Production EXPLAIN must still validate the unmodified channelPageSQL. +type sqliteChannelScanner struct{ DatabaseScanner } + +func (s *sqliteChannelScanner) ScanChannel(ctx context.Context, key string, last int64, size int) ([]Card, error) { + query := strings.ReplaceAll(channelPageSQL, "FORCE INDEX (idx_card_chan_key)", "") + query = strings.ReplaceAll(query, "STRAIGHT_JOIN", "INNER JOIN") + var rows []struct { + ID int64 `orm:"id"` + ICCID string `orm:"iccid"` + MSISDN string `orm:"msisdn"` + IMSI string `orm:"imsi"` + ChanKey string `orm:"chan_key"` + PackageCode string `orm:"current_package_code"` + } + if err := dao.TSimCardInfo.DB().Raw(query, key, last, size).Ctx(ctx).Scan(&rows); err != nil { + return nil, err + } + cards := make([]Card, 0, len(rows)) + for _, row := range rows { + cards = append(cards, Card{ID: row.ID, ICCID: row.ICCID, MSISDN: row.MSISDN, IMSI: row.IMSI, ChanKey: row.ChanKey, PackageCode: row.PackageCode}) + } + return cards, nil +} + +func TestDatabaseQualificationAndCompleteTaskIDParity(t *testing.T) { + // No application DB/config: isolated SQLite file and loopback Redis only. + if err := gdb.SetConfig(gdb.Config{"default": gdb.ConfigGroup{{Type: "sqlite", Link: "sqlite::@file(" + t.TempDir() + "/cards.db)"}}}); err != nil { + t.Fatal(err) + } + ctx := contextForTest(t) + db := dao.TSimCardInfo.DB() + t.Cleanup(func() { _ = db.Close(context.Background()) }) + for _, sql := range []string{ + `CREATE TABLE sys_dept (dept_id INTEGER PRIMARY KEY, tenant_id TEXT, deleted_at DATETIME)`, + `CREATE TABLE t_sim_card_info (id INTEGER PRIMARY KEY, iccid TEXT, msisdn TEXT, imsi TEXT, chan_key TEXT, current_package_code TEXT, dept_id INTEGER, tenant_id TEXT)`, + `CREATE INDEX idx_card_chan_key ON t_sim_card_info(chan_key)`, + `CREATE TABLE t_sim_channels (chan_key TEXT, temp_key TEXT, sync_enabled INTEGER, operator INTEGER)`, + `INSERT INTO sys_dept VALUES (1,'t1',NULL),(2,'t2',NULL),(3,'t1','2026-01-01')`, + `INSERT INTO t_sim_channels VALUES ('a','sim_yd_v5',0,1),('b','sim_dx_v1',0,2),('disabled','sim_yd_v5',1,1),('no-template','',0,1)`, + `INSERT INTO t_sim_card_info VALUES + (1,'i1','m1','s1','a','pkg',1,'t1'), + (2,'i2','m2','s2','b','pkg',2,'t2'), + (3,'i3','','','a','pkg',1,'t1'), + (4,'','','','a','pkg',1,'t1'), + (5,'i5','','','a','',1,'t1'), + (6,'i6','','','a','pkg',1,'wrong-tenant'), + (7,'i7','','','a','pkg',3,'t1'), + (8,'i8','','','b','pkg',99,'t2'), + (9,'i9','','','disabled','pkg',1,'t1'), + (10,'i10','','','no-template','pkg',1,'t1'), + (11,'i11','','','unknown','pkg',1,'t1'), + (12,'i12','','','','pkg',1,'t1'), + (13,'i13','','','b','pkg',2,'t2')`, + } { + if _, err := db.Exec(ctx, sql); err != nil { + t.Fatal(err) + } + } + store := testStore(t) + for _, scope := range [][]string{nil, {"a"}, {"b", "disabled", "no-template", "unknown"}, {"unknown"}} { + var sets [][]string + for _, mode := range []string{"serial", "channel"} { + scanner := &sqliteChannelScanner{} + p := testProducer(t, scanner, store, Config{ScanMode: mode, PageSize: 2}) + run := "db-" + mode + if err := p.Produce(ctx, scopedMessage(run, scope...)); err != nil { + t.Fatal(err) + } + ids := taskIDs(t, store, run) + for i, id := range ids { + ids[i] = strings.TrimPrefix(id, run+":") + } + sort.Strings(ids) + sets = append(sets, ids) + stateKey, _ := store.TaskStateKey(run) + metaKey, _ := store.RunMetaKey(run) + chanKey, _ := store.RunChannelsKey(run) + _, _ = store.RedisClient().Do(ctx, "DEL", stateKey, metaKey, chanKey) + for _, key := range []string{"a", "b"} { + activeKey, _ := store.RoundActiveKey(run, 1, key) + _, _ = store.RedisClient().Do(ctx, "DEL", activeKey) + } + _, _ = store.RedisClient().Do(ctx, "SREM", store.ActiveRunsKey(), run) + } + if !reflect.DeepEqual(sets[0], sets[1]) { + t.Fatalf("scope=%v serial=%v channel=%v", scope, sets[0], sets[1]) + } + if scope == nil { + want := []string{operation + ":1", operation + ":13", operation + ":2", operation + ":3"} + sort.Strings(want) + if !reflect.DeepEqual(sets[0], want) { + t.Fatalf("qualification set=%v want=%v", sets[0], want) + } + } + } +} + +func TestChannelSQLShapeAndParameters(t *testing.T) { + for _, term := range []string{"FORCE INDEX (idx_card_chan_key)", "STRAIGHT_JOIN sys_dept", "c.chan_key = ?", "c.id > ?", "c.iccid <> ''", "c.current_package_code", "c.current_package_code <> ''", "d.deleted_at IS NULL", "c.tenant_id = d.tenant_id", "ORDER BY c.id ASC", "LIMIT ?"} { + if !strings.Contains(channelPageSQL, term) { + t.Fatalf("missing %q", term) + } + } + if strings.Contains(channelPageSQL, "<=") || strings.Count(channelPageSQL, "?") != 3 { + t.Fatal("SQL contains upper bound or changed bind parameter count") + } +} diff --git a/internal/producer/dailyusage/parallel_regression_test.go b/internal/producer/dailyusage/parallel_regression_test.go new file mode 100644 index 0000000..49ccdeb --- /dev/null +++ b/internal/producer/dailyusage/parallel_regression_test.go @@ -0,0 +1,102 @@ +package dailyusage + +import ( + "context" + "errors" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +func TestChannelBackpressureDoesNotStarveHighIDChannel(t *testing.T) { + for _, mode := range []string{"serial", "channel"} { + t.Run(mode, func(t *testing.T) { + store := testStore(t) + ctx, cancel := context.WithCancel(contextForTest(t)) + defer cancel() + scanner := scannerFor([]Card{{ID: 1, ChanKey: "slow"}, {ID: 2, ChanKey: "slow"}, {ID: 3, ChanKey: "slow"}, {ID: 1000, ChanKey: "fast"}}, 1000) + p := testProducer(t, scanner, store, Config{ScanMode: mode, PageSize: 1, ScanConcurrency: 1, ChannelHighWaterMark: 1, GlobalHighWaterMark: 10, PollInterval: time.Millisecond}) + result := make(chan error, 1) + started := time.Now() + go func() { result <- p.Produce(ctx, scopedMessage("starvation")) }() + slowKey, _ := store.RoundActiveKey("starvation", 1, "slow") + fastKey, _ := store.RoundActiveKey("starvation", 1, "fast") + waitUntil(t, ctx, func() bool { n, e := store.SizeKey(ctx, slowKey); return e == nil && n == 1 }) + if mode == "channel" { + waitUntil(t, ctx, func() bool { n, e := store.SizeKey(ctx, fastKey); return e == nil && n == 1 }) + } else { + // No consumer drains slow. The serial cursor cannot reach card 1000. + time.Sleep(30 * time.Millisecond) + if n, err := store.SizeKey(ctx, fastKey); err != nil || n != 0 { + t.Fatalf("serial fast size=%d err=%v", n, err) + } + } + slow, _ := store.SizeKey(ctx, slowKey) + fast, _ := store.SizeKey(ctx, fastKey) + t.Logf("mode=%s slowQueued=%d fastQueued=%d scanConcurrency=1 channelHWM=1 elapsedMs=%d", mode, slow, fast, time.Since(started).Milliseconds()) + cancel() + if err := <-result; !errors.Is(err, context.Canceled) { + t.Fatalf("cancellation err=%v", err) + } + requireMeta(t, store, "starvation", false, slow+fast) + }) + } +} + +func TestChannelPayloadRetainsPeriodPackageAndIdentifiers(t *testing.T) { + store := testStore(t) + ctx := contextForTest(t) + card := Card{ID: 10, ICCID: "iccid", MSISDN: "msisdn", IMSI: "imsi", ChanKey: "a", PackageCode: "pkg-history"} + p := testProducer(t, scannerFor([]Card{card}, card.ID), store, Config{PageSize: 1}) + message := scopedMessage("payload", "a") + if err := p.Produce(ctx, message); err != nil { + t.Fatal(err) + } + claimed, err := store.ClaimRun(ctx, message.RunID, "a", time.Now().Add(time.Second), time.Now().Add(time.Minute), 1) + if err != nil || len(claimed) != 1 { + t.Fatalf("claimed=%v err=%v", claimed, err) + } + task := claimed[0].Task + if task.TaskID != zset.StableTaskID(message.RunID, FullSyncOperation, card.ID) || task.Operation != FullSyncOperation || task.QueryMonth != message.QueryDate || task.PackageCode != card.PackageCode || task.ICCID != card.ICCID || task.MSISDN != card.MSISDN || task.IMSI != card.IMSI || task.TempKey != "sim_yd_v5" || task.Operator != 1 { + t.Fatalf("task=%+v", task) + } +} + +func TestZombieWithBacklogDrainsBeforeCompletion(t *testing.T) { + p, store := newZombieTestProducer(t) + ctx := contextForTest(t) + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "zombie-backlog"}); err != nil { + t.Fatal(err) + } + tasks := []zset.Task{{TaskID: "zombie-card", RunID: "zombie-backlog", ChanKey: "a"}} + if _, err := store.EnqueueRunBatch(ctx, "zombie-backlog", "a", tasks, time.Now()); err != nil { + t.Fatal(err) + } + if pending, err := p.hasPendingTasks(ctx, "new"); err != nil || !pending { + t.Fatalf("backlog must retain backpressure: pending=%t err=%v", pending, err) + } + requireMeta(t, store, "zombie-backlog", true, 1) + if complete, err := store.CompleteRunIfIdle(ctx, "zombie-backlog"); err != nil || complete { + t.Fatalf("premature complete=%t err=%v", complete, err) + } + claimed, err := store.ClaimRun(ctx, "zombie-backlog", "a", time.Now().Add(time.Second), time.Now().Add(time.Minute), 1) + if err != nil || len(claimed) != 1 { + t.Fatalf("claimed=%v err=%v", claimed, err) + } + if ok, err := store.FinalizeRunTask(ctx, claimed[0], zset.TaskOutcome{Success: true}); err != nil || !ok { + t.Fatalf("finalize=%t err=%v", ok, err) + } + if complete, err := store.CompleteRunIfIdle(ctx, "zombie-backlog", time.Hour); err != nil || !complete { + t.Fatalf("complete=%t err=%v", complete, err) + } + meta, _ := store.RunMetaKey("zombie-backlog") + ttl, err := store.RedisClient().Do(ctx, "TTL", meta) + if err != nil || ttl.Int() <= 0 { + t.Fatalf("retention ttl=%v err=%v", ttl, err) + } + active, err := store.ActiveRuns(ctx) + if err != nil || len(active) != 0 { + t.Fatalf("active=%v err=%v", active, err) + } +} diff --git a/internal/producer/dailyusage/producer.go b/internal/producer/dailyusage/producer.go new file mode 100644 index 0000000..1a59792 --- /dev/null +++ b/internal/producer/dailyusage/producer.go @@ -0,0 +1,739 @@ +// Package dailyusage produces the previous-day usage full-sync run. +// +// It is the daily-usage twin of producer/monthlyusage and producer/simstatus: +// it scans the eligible cards once, groups them by channel, and enqueues one +// Redis ZSet task per card. The zset consumer owns pacing, retries, failure +// pooling, archiving and replay. +// +// The only structural difference from the monthly producer is that the run +// period is a day (yyyyMMdd) rather than a month (yyyyMM). That value travels +// in zset.Task.QueryMonth, which is a period identifier rather than a month +// name; see that field's documentation. +package dailyusage + +import ( + "context" + "crypto/rand" + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + "strings" + "sync/atomic" + "time" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +// FullSyncOperation identifies a daily usage run in run metadata, archives and +// replay. contract.SyncKindForOperation maps it to traffic_usage_daily. +const FullSyncOperation = model.OperationPreviousDayUsageFullSync + +// scopeSampleLimit bounds how many channel names are copied into run metadata. +// The run hash and its archive JSON must stay bounded regardless of how many +// channels a scope selects. +const scopeSampleLimit = 10 + +// scopeDigest is a stable short fingerprint of the selected channels. It lets +// operators confirm two runs covered the same scope without storing the list. +func scopeDigest(scope *model.SyncScope) string { + if scope == nil || len(scope.Channels) == 0 { + return "" + } + digest := sha256.Sum256([]byte(strings.Join(scope.Channels, "\n"))) + return hex.EncodeToString(digest[:8]) +} + +type Config struct { + ScanMode string + ScanConcurrency int + PageSize int + GlobalHighWaterMark int64 + ChannelHighWaterMark int64 + PollInterval time.Duration + RunLockKey string + RunLockTTL time.Duration + RoundRetryEnabled bool + MaxRounds int + FailureRetention time.Duration +} + +func (c Config) withDefaults() Config { + if c.ScanMode == "" { + c.ScanMode = "serial" + } + if c.ScanConcurrency == 0 { + c.ScanConcurrency = 4 + } + if c.PageSize <= 0 { + c.PageSize = 100 + } + if c.PollInterval <= 0 { + c.PollInterval = 2 * time.Second + } + if c.RunLockKey == "" { + c.RunLockKey = "daily:producer:active" + } + if c.RunLockTTL <= 0 { + c.RunLockTTL = 5 * time.Minute + } + if c.MaxRounds <= 0 { + c.MaxRounds = 3 + } + if c.FailureRetention <= 0 { + c.FailureRetention = 30 * 24 * time.Hour + } + return c +} + +type CardScanner interface { + SnapshotMaxID(context.Context) (int64, error) + Scan(context.Context, int64, int64, int) ([]Card, error) +} + +type Card struct { + ID int64 + ICCID string + MSISDN string + IMSI string + ChanKey string + TempKey string + PackageCode string + SyncEnabled bool + Operator int +} + +type channelMetadata struct { + ChanKey string `orm:"chan_key"` + TempKey string `orm:"temp_key"` + SyncEnabled bool `orm:"sync_enabled"` + Operator int `orm:"operator"` +} + +type DatabaseScanner struct { + channelMetadata map[string]channelMetadata +} + +func (s *DatabaseScanner) SnapshotMaxID(ctx context.Context) (int64, error) { + var card entity.TSimCardInfo + cardTable := dao.TSimCardInfo.Table() + columns := dao.TSimCardInfo.Columns() + if err := dailyUsageEligibleCardModel(ctx). + Fields(cardTable + "." + columns.Id + " AS " + columns.Id). + OrderDesc(cardTable + "." + columns.Id). + Limit(1). + Scan(&card); err != nil { + return 0, fmt.Errorf("read card snapshot max id: %w", err) + } + return card.Id, nil +} + +func (s *DatabaseScanner) Scan(ctx context.Context, lastID, maxID int64, pageSize int) ([]Card, error) { + if maxID <= lastID { + return nil, nil + } + if lastID == 0 || s.channelMetadata == nil { + var metadata []channelMetadata + if err := g.DB().Model("t_sim_channels").Fields("chan_key,temp_key,sync_enabled,operator").Scan(&metadata); err != nil { + return nil, fmt.Errorf("scan channel metadata: %w", err) + } + s.channelMetadata = make(map[string]channelMetadata, len(metadata)) + for _, item := range metadata { + s.channelMetadata[item.ChanKey] = item + } + } + var rows []entity.TSimCardInfo + columns := dao.TSimCardInfo.Columns() + cardTable := dao.TSimCardInfo.Table() + query := dailyUsageEligibleCardModel(ctx). + Fields( + cardTable+"."+columns.Id+" AS "+columns.Id, + cardTable+"."+columns.Iccid+" AS "+columns.Iccid, + cardTable+"."+columns.Msisdn+" AS "+columns.Msisdn, + cardTable+"."+columns.Imsi+" AS "+columns.Imsi, + cardTable+"."+columns.ChanKey+" AS "+columns.ChanKey, + // The daily usage child table stores the package code and the + // diagnosis page reads it back, so it must be carried on the task. + cardTable+"."+columns.CurrentPackageCode+" AS "+columns.CurrentPackageCode, + ). + WhereGT(cardTable+"."+columns.Id, lastID). + WhereLTE(cardTable+"."+columns.Id, maxID). + OrderAsc(cardTable + "." + columns.Id). + Limit(pageSize) + if err := query.Scan(&rows); err != nil { + return nil, fmt.Errorf("scan card information: %w", err) + } + cards := make([]Card, 0, len(rows)) + for _, row := range rows { + channel := s.channelMetadata[row.ChanKey] + cards = append(cards, Card{ + ID: row.Id, + ICCID: row.Iccid, + MSISDN: row.Msisdn, + IMSI: row.Imsi, + ChanKey: row.ChanKey, + TempKey: channel.TempKey, + PackageCode: row.CurrentPackageCode, + SyncEnabled: channel.SyncEnabled, + Operator: channel.Operator, + }) + } + return cards, nil +} + +// dailyUsageEligibleCardModel mirrors the monthly usage eligibility rule and the +// legacy admin SyncDaily scan: the card must belong to a live department, carry +// an ICCID and channel, and have a current package code. +func dailyUsageEligibleCardModel(ctx context.Context) *gdb.Model { + cardTable := dao.TSimCardInfo.Table() + deptTable := dao.SysDept.Table() + cardColumns := dao.TSimCardInfo.Columns() + deptColumns := dao.SysDept.Columns() + + return dao.TSimCardInfo.Ctx(ctx). + InnerJoin(deptTable, cardTable+"."+cardColumns.DeptId+" = "+deptTable+"."+deptColumns.DeptId+" AND "+cardTable+"."+cardColumns.TenantId+" = "+deptTable+"."+deptColumns.TenantId). + Where(cardTable+"."+cardColumns.Iccid+" <> ?", ""). + Where(cardTable+"."+cardColumns.ChanKey+" <> ?", ""). + Where(cardTable+"."+cardColumns.CurrentPackageCode+" <> ?", ""). + WhereNull(deptTable + "." + deptColumns.DeletedAt) +} + +type Producer struct { + scanner CardScanner + store *zset.Store + config Config + running atomic.Int32 + owner string + observation *supplyObservation +} + +type productionStats struct { + pages int64 + scanned int64 + eligible int64 + enqueued int64 + skippedEmptyChan int64 + skippedNoTemplate int64 + skippedSyncDisabled int64 + skippedOutOfScope int64 + stoppedTasks int64 + channels map[string]struct{} + scopeChannels int + skippedChannels int +} + +func (s *productionStats) channelCount() int { + return len(s.channels) +} + +func (s *productionStats) log(ctx context.Context, status, runID, queryDate, stage, reason string, startedAt time.Time, maxID int64, err error) { + message := fmt.Sprintf("event=daily_usage_producer status=%s runId=%s queryDate=%s snapshotMaxId=%d pages=%d scanned=%d eligible=%d enqueued=%d skippedEmptyChan=%d skippedNoTemplate=%d skippedSyncDisabled=%d skippedOutOfScope=%d stoppedTasks=%d channelCount=%d scopeChannels=%d skippedChannels=%d durationMs=%d", status, runID, queryDate, maxID, s.pages, s.scanned, s.eligible, s.enqueued, s.skippedEmptyChan, s.skippedNoTemplate, s.skippedSyncDisabled, s.skippedOutOfScope, s.stoppedTasks, s.channelCount(), s.scopeChannels, s.skippedChannels, time.Since(startedAt).Milliseconds()) + if s.scanned > 0 { + message += fmt.Sprintf(" cardsPerSec=%.2f", float64(s.scanned)/time.Since(startedAt).Seconds()) + } + if stage != "" { + message += " stage=" + stage + } + if reason != "" { + message += " reason=" + reason + } + if err != nil { + message += fmt.Sprintf(" error=%q", err.Error()) + } + if status == "failed" { + g.Log().Error(ctx, message) + return + } + if status == "skipped" { + g.Log().Warning(ctx, message) + return + } + g.Log().Info(ctx, message) +} + +func New(scanner CardScanner, store *zset.Store, config Config) (*Producer, error) { + if scanner == nil || store == nil { + return nil, fmt.Errorf("daily usage producer dependencies are incomplete") + } + config = config.withDefaults() + if err := ValidateScanSettings(config.ScanMode, config.ScanConcurrency); err != nil { + return nil, err + } + if config.ScanMode == "channel" { + if _, ok := scanner.(channelScanner); !ok { + return nil, fmt.Errorf("channel scan requires ListChannels and ScanChannel") + } + } + return &Producer{scanner: scanner, store: store, config: config, owner: newOwner()}, nil +} + +var ErrAlreadyRunning = fmt.Errorf("daily usage producer already running") + +// ErrRunFinished is returned when a redelivered start message names a run that +// already reached a terminal state. The run must not be resurrected; callers +// should log and ack instead of retrying. +var ErrRunFinished = fmt.Errorf("daily usage run already finished") + +// runAlreadyFinished reports whether runID exists and has already completed. +// A missing run is not an error: the message may be the first delivery. +func (p *Producer) runAlreadyFinished(ctx context.Context, runID string) (bool, error) { + meta, err := p.store.GetRunMeta(ctx, runID) + if err != nil { + if errors.Is(err, zset.ErrRunNotFound) { + return false, nil + } + return false, err + } + return meta.Status == "completed" || meta.Status == "completed_with_failures", nil +} + +// Produce scans the eligible cards once and enqueues one task per card. +// +// message.QueryDate (yyyyMMdd) is stored in zset.Task.QueryMonth, the queue's +// period-identifier field, so the zset store, run metadata, archive, replay and +// Lua scripts need no change for a daily run. +func (p *Producer) Produce(ctx context.Context, message model.TrafficUsageDailyStartMessage) (err error) { + startedAt := time.Now() + stats := productionStats{channels: make(map[string]struct{})} + if !p.running.CompareAndSwap(0, 1) { + stats.log(ctx, "skipped", message.RunID, message.QueryDate, "precheck", "already_running", startedAt, 0, nil) + return ErrAlreadyRunning + } + defer p.running.Store(0) + + if message.RunID == "" { + err = fmt.Errorf("runId is empty") + stats.log(ctx, "failed", message.RunID, message.QueryDate, "validate", "", startedAt, 0, err) + return err + } + if message.QueryDate == "" { + err = fmt.Errorf("queryDate is empty") + stats.log(ctx, "failed", message.RunID, message.QueryDate, "validate", "", startedAt, 0, err) + return err + } + scope, err := model.NormalizeScope(message.Scope) + if err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryDate, "validate", "invalid_scope", startedAt, 0, err) + return err + } + stats.scopeChannels = scope.ChannelCount() + // A redelivered start message must not resurrect a run that already + // finished; that would create a second producer pass over the same RunID. + if p.config.RoundRetryEnabled { + if finished, checkErr := p.runAlreadyFinished(ctx, message.RunID); checkErr != nil { + stats.log(ctx, "failed", message.RunID, message.QueryDate, "run_state", "", startedAt, 0, checkErr) + return checkErr + } else if finished { + stats.log(ctx, "skipped", message.RunID, message.QueryDate, "run_state", "run_finished", startedAt, 0, nil) + return ErrRunFinished + } + } + acquired, err := p.store.AcquireLockOrTakeover(ctx, p.config.RunLockKey, p.owner, message.RunID, p.config.RunLockTTL) + if err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryDate, "lock", "", startedAt, 0, err) + return err + } + if !acquired { + stats.log(ctx, "skipped", message.RunID, message.QueryDate, "lock", "already_running", startedAt, 0, nil) + return ErrAlreadyRunning + } + runCtx, cancel := context.WithCancel(ctx) + defer cancel() + defer func() { + releaseCtx, releaseCancel := context.WithTimeout(context.Background(), time.Second) + defer releaseCancel() + _, _ = p.store.ReleaseLock(releaseCtx, p.config.RunLockKey, p.owner) + }() + pending, err := p.hasPendingTasks(ctx, message.RunID) + if err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryDate, "pending_check", "", startedAt, 0, err) + return err + } + if pending { + stats.log(ctx, "skipped", message.RunID, message.QueryDate, "pending_check", "pending_tasks", startedAt, 0, nil) + return ErrAlreadyRunning + } + renewErr := make(chan error, 1) + go p.renewLock(runCtx, cancel, renewErr) + ctx = runCtx + if message.Operation == "" { + message.Operation = FullSyncOperation + } + if p.config.RoundRetryEnabled { + if err = p.store.CreateRun(ctx, zset.RunMeta{ + RunID: message.RunID, + Operation: message.Operation, + QueryMonth: message.QueryDate, + Status: "producing", + CurrentRound: 1, + ScopeChannels: scope.ChannelCount(), + ScopeDigest: scopeDigest(scope), + ScopeChannelsSample: strings.Join(scope.ChannelSample(scopeSampleLimit), ","), + }); err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryDate, "run_init", "", startedAt, 0, err) + return err + } + } + maxID, err := p.scanner.SnapshotMaxID(ctx) + if err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryDate, "snapshot", "", startedAt, 0, err) + return err + } + g.Log().Infof(ctx, "event=daily_usage_producer status=started runId=%s queryDate=%s snapshotMaxId=%d scopeChannels=%d scanMode=%s scanConcurrency=%d pageSize=%d", message.RunID, message.QueryDate, maxID, scope.ChannelCount(), p.config.ScanMode, p.config.ScanConcurrency, p.config.PageSize) + stage := "scan" + defer func() { + if err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryDate, stage, "", startedAt, maxID, err) + return + } + stats.log(ctx, "completed", message.RunID, message.QueryDate, "", "", startedAt, maxID, nil) + }() + if p.config.RoundRetryEnabled && p.config.ScanMode == "channel" { + stage = "channel_scan" + if err = p.produceChannels(ctx, message, scope, maxID, startedAt, &stats); err != nil { + if isProductionClosed(err) { + // A same-Run redelivery after producerDone leaves consumption alone. + err = nil + return nil + } + return err + } + // Verify ownership immediately before closing production. The periodic + // renewer still cancels scanners and capacity waits on lease loss. + var owned bool + owned, err = p.store.RenewLock(ctx, p.config.RunLockKey, p.owner, p.config.RunLockTTL) + if err != nil { + return err + } + if !owned { + return fmt.Errorf("producer lock token is no longer owned") + } + if err = ctx.Err(); err != nil { + return err + } + stage = "producer_done" + err = p.store.MarkProducerDone(ctx, message.RunID) + return err + } + lastID := int64(0) + for lastID < maxID { + stage = "capacity" + if err = p.waitForCapacity(ctx, message.RunID); err != nil { + return err + } + stage = "scan" + cards, scanErr := p.scanner.Scan(ctx, lastID, maxID, p.config.PageSize) + if scanErr != nil { + err = scanErr + return err + } + if len(cards) == 0 { + break + } + stats.pages++ + stats.scanned += int64(len(cards)) + groupedTasks := make(map[string][]zset.Task) + for _, card := range cards { + lastID = card.ID + if card.ChanKey == "" { + stats.skippedEmptyChan++ + g.Log().Warningf(ctx, "event=daily_usage_card status=skipped stage=validate cardId=%d reason=empty_chan_key", card.ID) + continue + } + // Scope is applied before the sync_enabled and template checks so a + // partial run only ever reports on the channels it was asked for. + if !scope.Allows(card.ChanKey) { + stats.skippedOutOfScope++ + continue + } + if card.SyncEnabled { + stats.skippedSyncDisabled++ + g.Log().Warningf(ctx, "event=daily_usage_card status=skipped stage=validate cardId=%d chanKey=%s reason=sync_disabled", card.ID, card.ChanKey) + continue + } + if card.TempKey == "" { + stats.skippedNoTemplate++ + g.Log().Warningf(ctx, "event=daily_usage_card status=skipped stage=validate cardId=%d chanKey=%s reason=missing_channel_template", card.ID, card.ChanKey) + continue + } + stats.eligible++ + stats.channels[card.ChanKey] = struct{}{} + task := zset.Task{ + TaskID: zset.StableTaskID(message.RunID, message.Operation, card.ID), + RunID: message.RunID, + Operation: message.Operation, + QueryMonth: message.QueryDate, + ChanKey: card.ChanKey, + TempKey: card.TempKey, + Operator: card.Operator, + CardID: card.ID, + ICCID: card.ICCID, + MSISDN: card.MSISDN, + IMSI: card.IMSI, + PackageCode: card.PackageCode, + } + groupedTasks[card.ChanKey] = append(groupedTasks[card.ChanKey], task) + } + now := time.Now() + for chanKey, tasks := range groupedTasks { + stage = "enqueue" + if err = p.waitForChannelCapacity(ctx, message.RunID, chanKey); err != nil { + return err + } + var added int + var enqueueErr error + if p.config.RoundRetryEnabled { + added, enqueueErr = p.store.EnqueueRunBatch(ctx, message.RunID, chanKey, tasks, now) + } else { + added, enqueueErr = p.store.EnqueueBatch(ctx, chanKey, tasks, now) + } + if enqueueErr != nil { + err = enqueueErr + return err + } + stats.enqueued += int64(added) + if added < len(tasks) { + stats.stoppedTasks += int64(len(tasks) - added) + } + if added == 0 { + g.Log().Warningf(ctx, "event=daily_usage_channel status=skipped runId=%s queryDate=%s chanKey=%s count=%d reason=stopped", message.RunID, message.QueryDate, chanKey, len(tasks)) + } + } + } + if p.config.RoundRetryEnabled { + if err = p.store.MarkProducerDone(ctx, message.RunID); err != nil { + return err + } + } + return nil +} + +func (p *Producer) hasPendingTasks(ctx context.Context, currentRunID ...string) (bool, error) { + currentID := "" + if len(currentRunID) > 0 { + currentID = currentRunID[0] + } + if p.config.RoundRetryEnabled { + runs, err := p.store.ActiveRuns(ctx) + if err != nil { + return false, err + } + for _, runID := range runs { + if runID == currentID { + // A redelivered message for the same run may resume a producer + // that stopped before producerDone. Stable task IDs make the + // rescan idempotent; other runs still provide backpressure. + continue + } + meta, metaErr := p.store.GetRunMeta(ctx, runID) + if metaErr != nil { + // The active index is a global key written outside the run-scoped + // Lua transaction, so it can list a run whose hash is already gone + // (expired, evicted, or cleaned by hand). Returning an error here + // would surface as a non-retryable failure and make the consumer + // redeliver this request forever. Treat it as a stale index entry + // and drop it. + if errors.Is(metaErr, zset.ErrRunNotFound) { + if _, removeErr := p.store.RedisClient().Do(ctx, "SREM", p.store.ActiveRunsKey(), runID); removeErr != nil { + return false, removeErr + } + g.Log().Warningf(ctx, "event=daily_usage_producer status=skipped stage=pending_check reason=stale_index runId=%s", runID) + continue + } + return false, metaErr + } + if meta.Status != "producing" && meta.Status != "running" { + continue + } + // Channel blocking can set running before production ends. Only + // producerDone identifies consumer-owned work; replay has its own lock. + if meta.ProducerDone || meta.SyncType == "replay" { + return true, nil + } + // Recheck the unique producer lease immediately before reclaiming + // another run. A lost lease or an unspecified current run is not proof + // that the old producer has stopped. + if currentID == "" { + return true, nil + } + owned, lockErr := p.store.RenewLock(ctx, p.config.RunLockKey, p.owner, p.config.RunLockTTL) + if lockErr != nil { + return false, lockErr + } + if !owned { + return true, nil + } + // Close production, not the run: consumers retain queued, leased and + // next-round work and apply retention/index cleanup on normal completion. + reclaimed, reclaimErr := p.store.MarkProducerAbandoned(ctx, runID) + if reclaimErr != nil { + return false, reclaimErr + } + g.Log().Warningf(ctx, "event=daily_usage_producer status=skipped stage=pending_check reason=zombie_run runId=%s reclaimed=%t activeCount=%d inflightCount=%d nextRoundCount=%d", runID, reclaimed, meta.ActiveCount, meta.InflightCount, meta.NextRoundCount) + if !reclaimed || meta.ActiveCount > 0 || meta.InflightCount > 0 || meta.NextRoundCount > 0 { + return true, nil + } + } + } + channels, err := p.store.ActiveChannels(ctx) + if err != nil { + return false, err + } + for _, chanKey := range channels { + size, sizeErr := p.store.Size(ctx, chanKey) + if sizeErr != nil { + return false, sizeErr + } + if size > 0 { + return true, nil + } + } + return false, nil +} + +func (p *Producer) renewLock(ctx context.Context, cancel context.CancelFunc, errors chan<- error) { + interval := p.config.RunLockTTL / 3 + if interval <= 0 { + interval = time.Second + } + ticker := time.NewTicker(interval) + defer ticker.Stop() + for { + select { + case <-ctx.Done(): + return + case <-ticker.C: + ok, err := p.store.RenewLock(ctx, p.config.RunLockKey, p.owner, p.config.RunLockTTL) + if err != nil || !ok { + if err == nil { + err = fmt.Errorf("producer lock token is no longer owned") + } + select { + case errors <- fmt.Errorf("renew producer lock: %w", err): + default: + } + cancel() + return + } + } + } +} + +func newOwner() string { + var bytes [12]byte + if _, err := rand.Read(bytes[:]); err == nil { + return "producer-" + hex.EncodeToString(bytes[:]) + } + return fmt.Sprintf("producer-%d", time.Now().UnixNano()) +} + +func (p *Producer) waitForCapacity(ctx context.Context, runID ...string) error { + if p.config.GlobalHighWaterMark <= 0 { + return nil + } + for { + if p.config.RoundRetryEnabled && len(runID) > 0 && runID[0] != "" { + meta, err := p.store.GetRunMeta(ctx, runID[0]) + if err != nil { + return err + } + channels, err := p.store.RunChannels(ctx, runID[0]) + if err != nil { + return err + } + var total int64 + for _, chanKey := range channels { + key, keyErr := p.store.RoundActiveKey(runID[0], meta.CurrentRound, chanKey) + if keyErr != nil { + return keyErr + } + size, sizeErr := p.store.SizeKey(ctx, key) + if sizeErr != nil { + return sizeErr + } + total += size + } + if total < p.config.GlobalHighWaterMark { + return nil + } + if err = sleepContext(ctx, p.config.PollInterval); err != nil { + return err + } + continue + } + channels, err := p.store.ActiveChannels(ctx) + if err != nil { + return err + } + var total int64 + for _, chanKey := range channels { + size, sizeErr := p.store.Size(ctx, chanKey) + if sizeErr != nil { + return sizeErr + } + total += size + } + if total < p.config.GlobalHighWaterMark { + return nil + } + if err = sleepContext(ctx, p.config.PollInterval); err != nil { + return err + } + } +} + +func (p *Producer) waitForChannelCapacity(ctx context.Context, runIDAndChan ...string) error { + if p.config.ChannelHighWaterMark <= 0 { + return nil + } + chanKey := "" + runID := "" + if len(runIDAndChan) == 1 { + chanKey = runIDAndChan[0] + } else if len(runIDAndChan) >= 2 { + runID, chanKey = runIDAndChan[0], runIDAndChan[1] + } + for { + var size int64 + var err error + if p.config.RoundRetryEnabled && runID != "" { + meta, metaErr := p.store.GetRunMeta(ctx, runID) + if metaErr != nil { + return metaErr + } + key, keyErr := p.store.RoundActiveKey(runID, meta.CurrentRound, chanKey) + if keyErr != nil { + return keyErr + } + size, err = p.store.SizeKey(ctx, key) + } else { + size, err = p.store.Size(ctx, chanKey) + } + if err != nil { + return err + } + if size < p.config.ChannelHighWaterMark { + return nil + } + if err = sleepContext(ctx, p.config.PollInterval); err != nil { + return err + } + } +} + +func sleepContext(ctx context.Context, duration time.Duration) error { + timer := time.NewTimer(duration) + defer timer.Stop() + select { + case <-timer.C: + return nil + case <-ctx.Done(): + return ctx.Err() + } +} diff --git a/internal/producer/dailyusage/producer_test.go b/internal/producer/dailyusage/producer_test.go new file mode 100644 index 0000000..ee279b6 --- /dev/null +++ b/internal/producer/dailyusage/producer_test.go @@ -0,0 +1,256 @@ +package dailyusage + +import ( + "context" + "fmt" + "strings" + "sync" + "testing" + + "github.com/gogf/gf/v2/container/gvar" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +type pageScanner struct { + cards []Card +} + +func (s *pageScanner) SnapshotMaxID(context.Context) (int64, error) { + if len(s.cards) == 0 { + return 0, nil + } + return s.cards[len(s.cards)-1].ID, nil +} + +func (s *pageScanner) Scan(_ context.Context, lastID, _ int64, _ int) ([]Card, error) { + if lastID != 0 { + return nil, nil + } + return s.cards, nil +} + +// producerRedis records the enqueued members so a test can assert both the +// batch shape and the exact task payload (period and package code). +type producerRedis struct { + mu sync.Mutex + members []string + batches map[string]int +} + +func (r *producerRedis) Do(_ context.Context, command string, _ ...any) (*gvar.Var, error) { + switch strings.ToUpper(command) { + case "SET": + return gvar.New("OK"), nil + case "SMEMBERS": + return gvar.New([]string{}), nil + case "HGETALL": + // No stored run metadata: the producer treats this as a first delivery. + return gvar.New(map[string]string{}), nil + default: + return nil, fmt.Errorf("unexpected command %s", command) + } +} + +func (r *producerRedis) Eval(_ context.Context, _ string, numKeys int64, keys []string, args []any) (*gvar.Var, error) { + if numKeys == 1 { + return gvar.New(int64(1)), nil + } + if numKeys != 3 { + return nil, fmt.Errorf("unexpected key count %d", numKeys) + } + r.mu.Lock() + defer r.mu.Unlock() + if r.batches == nil { + r.batches = make(map[string]int) + } + count := len(args) - 3 + r.batches[keys[0]] += count + for _, arg := range args[3:] { + if member, ok := arg.(string); ok { + r.members = append(r.members, member) + } + } + return gvar.New(count), nil +} + +func newProducer(t *testing.T, cards []Card) (*Producer, *producerRedis) { + t.Helper() + redis := &producerRedis{batches: make(map[string]int)} + store, err := zset.New(redis, "daily:") + if err != nil { + t.Fatal(err) + } + producer, err := New(&pageScanner{cards: cards}, store, Config{PageSize: 100}) + if err != nil { + t.Fatal(err) + } + return producer, redis +} + +func dailyMessage(runID string, channels ...string) model.TrafficUsageDailyStartMessage { + message := model.TrafficUsageDailyStartMessage{ + RunID: runID, + Operation: FullSyncOperation, + QueryDate: "20261009", + TriggeredAt: "2026-10-10T06:00:00+08:00", + } + if len(channels) > 0 { + message.Scope = &model.SyncScope{Channels: channels} + } + return message +} + +func TestProduceCarriesQueryDateAndPackageCode(t *testing.T) { + producer, redis := newProducer(t, []Card{ + {ID: 1, ICCID: "89860001", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "PKG-A", Operator: 1}, + }) + + if err := producer.Produce(context.Background(), dailyMessage("daily-1")); err != nil { + t.Fatal(err) + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.members) != 1 { + t.Fatalf("members = %+v, want exactly one task", redis.members) + } + member := redis.members[0] + // The period rides in queryMonth because the queue only copies that field. + if !strings.Contains(member, `"queryMonth":"20261009"`) { + t.Fatalf("task member lost the daily period: %s", member) + } + // The package code must reach the TDengine child table via the task. + if !strings.Contains(member, `"packageCode":"PKG-A"`) { + t.Fatalf("task member lost the package code: %s", member) + } + if !strings.Contains(member, `"operation":"`+FullSyncOperation+`"`) { + t.Fatalf("task member lost the operation: %s", member) + } +} + +func TestProduceUsesStableTaskIDForIdempotency(t *testing.T) { + producer, _ := newProducer(t, []Card{ + {ID: 42, ICCID: "89860042", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "PKG-A"}, + }) + want := zset.StableTaskID("daily-1", FullSyncOperation, 42) + if got := zset.StableTaskID("daily-1", FullSyncOperation, 42); got != want { + t.Fatalf("StableTaskID is not deterministic: %s vs %s", got, want) + } + if err := producer.Produce(context.Background(), dailyMessage("daily-1")); err != nil { + t.Fatal(err) + } +} + +func TestProduceScopeEnqueuesOnlySelectedChannels(t *testing.T) { + producer, redis := newProducer(t, []Card{ + {ID: 1, ICCID: "1", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "P"}, + {ID: 2, ICCID: "2", ChanKey: "173", TempKey: "sim_yd_v5", PackageCode: "P"}, + {ID: 3, ICCID: "3", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "P"}, + {ID: 4, ICCID: "4", ChanKey: "189", TempKey: "sim_yd_v5", PackageCode: "P"}, + }) + + if err := producer.Produce(context.Background(), dailyMessage("daily-scoped", "217")); err != nil { + t.Fatal(err) + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.batches) != 1 { + t.Fatalf("enqueue batches = %+v, want only the 217 channel", redis.batches) + } + if redis.batches["daily:217"] != 2 { + t.Fatalf("enqueue batches = %+v, want 2 tasks on daily:217", redis.batches) + } +} + +func TestProduceFullScopeKeepsEveryChannel(t *testing.T) { + producer, redis := newProducer(t, []Card{ + {ID: 1, ICCID: "1", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "P"}, + {ID: 2, ICCID: "2", ChanKey: "173", TempKey: "sim_yd_v5", PackageCode: "P"}, + }) + + if err := producer.Produce(context.Background(), dailyMessage("daily-full")); err != nil { + t.Fatal(err) + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.batches) != 2 { + t.Fatalf("enqueue batches = %+v, want both channels for a full sync", redis.batches) + } +} + +func TestProduceScopeDoesNotOverrideSyncDisabled(t *testing.T) { + producer, redis := newProducer(t, []Card{ + {ID: 1, ICCID: "1", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "P"}, + {ID: 2, ICCID: "2", ChanKey: "173", TempKey: "sim_yd_v5", PackageCode: "P", SyncEnabled: true}, + }) + + if err := producer.Produce(context.Background(), dailyMessage("daily-scoped", "173")); err != nil { + t.Fatal(err) + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.batches) != 0 { + t.Fatalf("enqueue batches = %+v, want none (selected channel is sync-disabled)", redis.batches) + } +} + +func TestProduceRejectsMissingPeriod(t *testing.T) { + producer, redis := newProducer(t, []Card{ + {ID: 1, ICCID: "1", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "P"}, + }) + message := dailyMessage("daily-bad") + message.QueryDate = "" + if err := producer.Produce(context.Background(), message); err == nil { + t.Fatal("expected a missing queryDate to be rejected") + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.batches) != 0 { + t.Fatalf("rejected message must not enqueue: %+v", redis.batches) + } +} + +func TestProduceRejectsInvalidScope(t *testing.T) { + producer, redis := newProducer(t, []Card{ + {ID: 1, ICCID: "1", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "P"}, + }) + if err := producer.Produce(context.Background(), dailyMessage("daily-bad", "invalid/chan")); err == nil { + t.Fatal("expected invalid chanKey to be rejected") + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.batches) != 0 { + t.Fatalf("rejected scope must not enqueue: %+v", redis.batches) + } +} + +func TestProduceScopeDigestIsOrderIndependent(t *testing.T) { + first, err := model.NormalizeScope(&model.SyncScope{Channels: []string{"173", "217"}}) + if err != nil { + t.Fatal(err) + } + second, err := model.NormalizeScope(&model.SyncScope{Channels: []string{"217", "173"}}) + if err != nil { + t.Fatal(err) + } + if scopeDigest(first) != scopeDigest(second) { + t.Fatalf("digest depends on channel order: %s vs %s", scopeDigest(first), scopeDigest(second)) + } + if scopeDigest(nil) != "" { + t.Fatal("a full sync must have an empty digest") + } +} + +func TestRunAlreadyFinishedTreatsMissingRunAsFirstDelivery(t *testing.T) { + producer, _ := newProducer(t, []Card{ + {ID: 1, ICCID: "1", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "P"}, + }) + finished, err := producer.runAlreadyFinished(context.Background(), "fresh") + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if finished { + t.Fatal("a run without stored metadata must not be reported as finished") + } +} diff --git a/internal/producer/dailyusage/supply_observation.go b/internal/producer/dailyusage/supply_observation.go new file mode 100644 index 0000000..d9db9fd --- /dev/null +++ b/internal/producer/dailyusage/supply_observation.go @@ -0,0 +1,174 @@ +package dailyusage + +import ( + "context" + "sync" + "time" + + "github.com/gogf/gf/v2/frame/g" +) + +// Producer-local telemetry: no run/archive schema changes and no assertion of +// consumer readiness. Empty queues alone do not prove consumer starvation. +type supplyChannel struct { + stage string + pending int + version uint64 + ended bool + firstQuery, firstEnqueue *int64 + changed time.Time + stageDuration map[string]time.Duration + validSamples, missingSamples int64 +} + +type supplyObservation struct { + mu sync.Mutex + started time.Time + channels map[string]*supplyChannel +} + +func (o *supplyObservation) event(key, stage string, pending int, added, ended bool, now time.Time) { + o.mu.Lock() + defer o.mu.Unlock() + s := o.channels[key] + if s == nil { + return + } + s.stageDuration[s.stage] += now.Sub(s.changed) + s.changed, s.stage, s.pending, s.ended = now, stage, pending, ended + s.version++ + if stage == "sql_query" && s.firstQuery == nil { + v := now.Sub(o.started).Milliseconds() + s.firstQuery = &v + } + if added && s.firstEnqueue == nil { + v := now.Sub(o.started).Milliseconds() + s.firstEnqueue = &v + } +} + +// Preserve fractional milliseconds across transitions; round only for logs. +func (s *supplyChannel) stageMilliseconds(now time.Time) map[string]int64 { + if !s.ended { + s.stageDuration[s.stage] += now.Sub(s.changed) + s.changed = now + } + ms := make(map[string]int64, len(s.stageDuration)) + for stage, duration := range s.stageDuration { + ms[stage] = duration.Milliseconds() + } + return ms +} + +func (p *Producer) observe(key, stage string, pending int, added, ended bool) { + if p.observation != nil { + p.observation.event(key, stage, pending, added, ended, time.Now()) + } +} + +// Queue-only samples mirror e6ec5b8: cardinalities, never task bodies. All keys +// share the run hash tag, and round/producer state is checked in the same Lua. +// Leases are token HASHes, not ZSets; future scores include leased members. +const supplySampleScript = ` +local run=redis.call('HMGET',KEYS[1],'runId','status','currentRound','producerDone','syncType') +local known=run[1] and run[2] and run[3] and run[4] +local eligible=known and (run[2]=='producing' or run[2]=='running') and run[3]=='1' and run[4]=='0' and run[5]~='replay' +local blocked=redis.call('HGET',KEYS[1],'blocked:'..ARGV[2]) +return {known and 1 or 0,(eligible and blocked~='1') and 1 or 0,redis.call('ZCOUNT',KEYS[2],'-inf',ARGV[1]),redis.call('HLEN',KEYS[3]),redis.call('ZCOUNT',KEYS[2],'('..ARGV[1],'+inf')} +` + +func (p *Producer) sampleSupply(ctx context.Context, run, key string) ([]int64, error) { + meta, err := p.store.RunMetaKey(run) + if err != nil { + return nil, err + } + active, err := p.store.RoundActiveKey(run, 1, key) + if err != nil { + return nil, err + } + lease, err := p.store.RoundLeaseKey(run, 1, key) + if err != nil { + return nil, err + } + result, err := p.store.RedisClient().Eval(ctx, supplySampleScript, 3, []string{meta, active, lease}, []any{time.Now().UnixMilli(), key}) + if err != nil { + return nil, err + } + return result.Int64s(), nil +} + +func (p *Producer) startObservation(ctx context.Context, run string, keys []string, started time.Time) func() { + o := &supplyObservation{started: started, channels: make(map[string]*supplyChannel, len(keys))} + for _, key := range keys { + o.channels[key] = &supplyChannel{stage: "initializing", changed: started, stageDuration: make(map[string]time.Duration)} + } + p.observation = o + stop, done := make(chan struct{}), make(chan struct{}) + go func() { + defer close(done) + ticker := time.NewTicker(time.Second) + defer ticker.Stop() + for { + select { + case <-ctx.Done(): + return + case <-stop: + return + case <-ticker.C: + for _, key := range keys { + select { + case <-stop: + return + case <-ctx.Done(): + return + default: + } + o.mu.Lock() + s := o.channels[key] + version, ended := s.version, s.ended + o.mu.Unlock() + if ended { + continue + } + sampleCtx, cancel := context.WithTimeout(ctx, 500*time.Millisecond) + values, err := p.sampleSupply(sampleCtx, run, key) + cancel() + o.mu.Lock() + known := err == nil && len(values) == 5 && values[0] == 1 && s.version == version && !s.ended + stage, pending := s.stage, s.pending + if known { + s.validSamples++ + } else { + s.missingSamples++ + } + o.mu.Unlock() + // Logging must not hold the mutex used by producer progress. + if known { + g.Log().Infof(ctx, "event=daily_usage_supply status=sample runId=%s chanKey=%s stage=%s pending=%d firstRoundEligible=%t ready=%d leases=%d futureIncludingLeases=%d consumerEligibility=unknown", run, key, stage, pending, values[1] == 1, values[2], values[3], values[4]) + } else { + g.Log().Infof(ctx, "event=daily_usage_supply status=unknown runId=%s chanKey=%s stage=%s pending=%d consumerEligibility=unknown", run, key, stage, pending) + } + } + } + } + }() + return func() { + close(stop) + <-done + o.mu.Lock() + defer o.mu.Unlock() + for _, key := range keys { + s := o.channels[key] + stageMS := s.stageMilliseconds(time.Now()) + query, enqueue := int64(-1), int64(-1) + if s.firstQuery != nil { + query = *s.firstQuery + } + if s.firstEnqueue != nil { + enqueue = *s.firstEnqueue + } + g.Log().Infof(ctx, "event=daily_usage_supply status=ended runId=%s chanKey=%s firstQueryMs=%d firstEnqueueMs=%d stageMs=%v validSamples=%d missingSamples=%d terminalStage=%s consumerEligibility=unknown", run, key, query, enqueue, stageMS, s.validSamples, s.missingSamples, s.stage) + } + p.observation = nil + } +} diff --git a/internal/producer/dailyusage/supply_observation_test.go b/internal/producer/dailyusage/supply_observation_test.go new file mode 100644 index 0000000..9e5755a --- /dev/null +++ b/internal/producer/dailyusage/supply_observation_test.go @@ -0,0 +1,110 @@ +package dailyusage + +import ( + "reflect" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +func TestLocalSupplyObservationFirstEventsAndStages(t *testing.T) { + started := time.Now() + o := &supplyObservation{started: started, channels: map[string]*supplyChannel{"a": {stage: "initializing", changed: started, stageDuration: map[string]time.Duration{}}}} + o.event("a", "sql_query", 0, false, false, started.Add(10*time.Millisecond)) + o.event("a", "page_ready", 2, false, false, started.Add(20*time.Millisecond)) + o.event("a", "redis_write", 1, true, false, started.Add(30*time.Millisecond)) + o.event("a", "channel_capacity", 1, false, false, started.Add(40*time.Millisecond)) + o.event("a", "sql_query", 0, false, false, started.Add(50*time.Millisecond)) + o.event("a", "redis_write", 0, true, false, started.Add(60*time.Millisecond)) + o.event("a", "upper_bound", 0, false, true, started.Add(70*time.Millisecond)) + o.event("unknown", "ignored", 0, false, false, started) + s := o.channels["a"] + if s.firstQuery == nil || *s.firstQuery != 10 || s.firstEnqueue == nil || *s.firstEnqueue != 30 || !s.ended || s.pending != 0 || s.version != 7 || s.stageDuration["channel_capacity"] != 10*time.Millisecond || s.stageDuration["sql_query"] != 20*time.Millisecond { + t.Fatalf("observation=%+v", s) + } +} + +func TestSupplyObservationAccumulatesSubMillisecondStages(t *testing.T) { + started := time.Unix(0, 0) + o := &supplyObservation{started: started, channels: map[string]*supplyChannel{"a": {stage: "redis_read", changed: started, stageDuration: map[string]time.Duration{}}}} + for i := 1; i <= 100000; i++ { + stage := "redis_read" + if i%2 == 1 { + stage = "redis_write" + } + o.event("a", stage, 0, false, i == 100000, started.Add(time.Duration(i)*500*time.Microsecond)) + } + if got := o.channels["a"].stageMilliseconds(started.Add(time.Hour)); got["redis_read"] != 25000 || got["redis_write"] != 25000 { + t.Fatalf("stageMs=%v, want 25000ms per stage", got) + } +} + +func TestSupplyObservationFinalFlushPreservesFraction(t *testing.T) { + started := time.Unix(0, 0) + o := &supplyObservation{started: started, channels: map[string]*supplyChannel{"a": {stage: "redis_write", changed: started, stageDuration: map[string]time.Duration{}}}} + o.event("a", "redis_read", 0, false, false, started.Add(600*time.Microsecond)) + o.event("a", "redis_write", 0, false, false, started.Add(1200*time.Microsecond)) + s := o.channels["a"] + if got := s.stageMilliseconds(started.Add(1800 * time.Microsecond)); got["redis_write"] != 1 || got["redis_read"] != 0 { + t.Fatalf("final stageMs=%v", got) + } + if s.stageDuration["redis_write"] != 1200*time.Microsecond || s.stageDuration["redis_read"] != 600*time.Microsecond { + t.Fatalf("fractional duration lost: %v", s.stageDuration) + } +} + +func TestSupplySamplesDistinguishReadyLeaseFutureAndClosedRound(t *testing.T) { + store := testStore(t) + ctx := contextForTest(t) + p := testProducer(t, scannerFor(nil, 0), store, Config{}) + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "supply"}); err != nil { + t.Fatal(err) + } + active, _ := store.RoundActiveKey("supply", 1, "a") + lease, _ := store.RoundLeaseKey("supply", 1, "a") + _, _ = store.RedisClient().Do(ctx, "ZADD", active, time.Now().Add(-time.Minute).UnixMilli(), "ready", time.Now().Add(time.Hour).UnixMilli(), "future") + _, _ = store.RedisClient().Do(ctx, "HSET", lease, "leased", "token") + values, err := p.sampleSupply(ctx, "supply", "a") + if err != nil || !reflect.DeepEqual(values, []int64{1, 1, 1, 1, 1}) { + t.Fatalf("sample=%v err=%v", values, err) + } + meta, _ := store.RunMetaKey("supply") + for _, args := range [][]any{{"blocked:a", 1}, {"blocked:a", 0, "producerDone", 1}, {"producerDone", 0, "currentRound", 2}, {"currentRound", 1, "syncType", "replay"}} { + _, _ = store.RedisClient().Do(ctx, "HSET", append([]any{meta}, args...)...) + values, err = p.sampleSupply(ctx, "supply", "a") + if err != nil || values[0] != 1 || values[1] != 0 { + t.Fatalf("ineligible sample=%v err=%v", values, err) + } + } + values, err = p.sampleSupply(ctx, "missing", "a") + if err != nil || values[0] != 0 || values[1] != 0 { + t.Fatalf("missing sample=%v err=%v", values, err) + } +} + +func TestSupplySamplerTracksWaitingChannelsAndStops(t *testing.T) { + store := testStore(t) + ctx := contextForTest(t) + p := testProducer(t, scannerFor(nil, 0), store, Config{}) + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "sampling"}); err != nil { + t.Fatal(err) + } + finish := p.startObservation(ctx, "sampling", []string{"a", "b"}, time.Now()) + defer finish() + o := p.observation + p.observe("a", "channel_capacity", 1, false, false) + p.observe("b", "sql_slot", 0, false, false) + waitUntil(t, ctx, func() bool { + o.mu.Lock() + defer o.mu.Unlock() + return o.channels["a"].validSamples > 0 && o.channels["b"].validSamples > 0 + }) + p.observe("a", "cancelled", 0, false, true) + p.observe("b", "cancelled", 0, false, true) + o.mu.Lock() + defer o.mu.Unlock() + if o.channels["a"].stageDuration["channel_capacity"] < 900*time.Millisecond || o.channels["b"].stageDuration["sql_slot"] < 900*time.Millisecond { + t.Fatalf("missing wait attribution: a=%+v b=%+v", o.channels["a"], o.channels["b"]) + } +} diff --git a/internal/producer/dailyusage/zombie_test.go b/internal/producer/dailyusage/zombie_test.go new file mode 100644 index 0000000..e80aed7 --- /dev/null +++ b/internal/producer/dailyusage/zombie_test.go @@ -0,0 +1,179 @@ +package dailyusage + +import ( + "context" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +func newZombieTestProducer(t *testing.T) (*Producer, *zset.Store) { + t.Helper() + store, err := zset.New(testutil.Redis(t), "flow:") + if err != nil { + t.Fatal(err) + } + producer, err := New(&pageScanner{cards: nil}, store, Config{PageSize: 100, RoundRetryEnabled: true}) + if err != nil { + t.Fatal(err) + } + ctx := context.Background() + if acquired, err := store.AcquireLockOrTakeover(ctx, producer.config.RunLockKey, producer.owner, "pending-check", producer.config.RunLockTTL); err != nil || !acquired { + t.Fatalf("acquire producer lock=%t err=%v", acquired, err) + } + t.Cleanup(func() { _, _ = store.ReleaseLock(ctx, producer.config.RunLockKey, producer.owner) }) + return producer, store +} + +// TestHasPendingTasksReclaimsZombieRun guards the "zombie run" regression. +// +// A producer that dies before MarkProducerDone (for example the container is +// replaced mid-run) leaves its run in status=producing with producerDone=0. +// completeRunIfIdle and advanceRunRound both require producerDone=1, so such a +// run can never be closed, never leaves the active index, and never gets the +// retention TTL. Treating it as backpressure rejected every later request +// forever; merely warning about it leaked keys and an index entry indefinitely. +func TestHasPendingTasksReclaimsZombieRun(t *testing.T) { + producer, store := newZombieTestProducer(t) + ctx := context.Background() + + // A genuinely in-flight run (producerDone set, consumer working) must keep + // providing backpressure. + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "running-run", Status: "running", ProducerDone: true, CurrentRound: 1}); err != nil { + t.Fatal(err) + } + pending, err := producer.hasPendingTasks(ctx, "new-run") + if err != nil { + t.Fatal(err) + } + if !pending { + t.Fatal("a running run must still provide backpressure") + } + if _, err := store.RedisClient().Do(ctx, "SREM", store.ActiveRunsKey(), "running-run"); err != nil { + t.Fatal(err) + } + + // A zombie run must not block a new request, and must be promoted so the + // normal lifecycle can drain and close it. + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "zombie-run", Status: "producing", ProducerDone: false, CurrentRound: 1}); err != nil { + t.Fatal(err) + } + pending, err = producer.hasPendingTasks(ctx, "new-run-2") + if err != nil { + t.Fatal(err) + } + if pending { + t.Fatal("a producing run with producerDone=0 must not block new requests (zombie run)") + } + + meta, err := store.GetRunMeta(ctx, "zombie-run") + if err != nil { + t.Fatal(err) + } + if !meta.ProducerDone { + t.Fatal("the zombie run must be promoted (producerDone=1) so it can be closed") + } + if meta.Status != "running" { + t.Fatalf("promoted run status = %q, want running", meta.Status) + } + // Promotion makes the run closable, which is what applies the retention TTL + // and releases the active index entry. + if completed, err := store.CompleteRunIfIdle(ctx, "zombie-run"); err != nil || !completed { + t.Fatalf("promoted run must be closable: completed=%v err=%v", completed, err) + } +} + +// TestHasPendingTasksNeverReclaimsReplayRun guards the boundary: a replay target +// is also created as "producing", but it is driven by the replay engine under +// its own lock, so its producerDone is not the producer's to set. +func TestHasPendingTasksNeverReclaimsReplayRun(t *testing.T) { + producer, store := newZombieTestProducer(t) + ctx := context.Background() + + if err := store.CreateRun(ctx, zset.RunMeta{ + RunID: "replay-run", Status: "producing", ProducerDone: false, CurrentRound: 1, + SyncType: "replay", SourceRunID: "source-run", ReplayCycle: 1, + }); err != nil { + t.Fatal(err) + } + if pending, err := producer.hasPendingTasks(ctx, "new-run"); err != nil || !pending { + t.Fatalf("replay must retain backpressure: pending=%t err=%v", pending, err) + } + meta, err := store.GetRunMeta(ctx, "replay-run") + if err != nil { + t.Fatal(err) + } + if meta.ProducerDone { + t.Fatal("a replay run must not be promoted by the full-sync producer") + } + if meta.Status != "producing" { + t.Fatalf("replay run status = %q, want producing", meta.Status) + } +} + +// TestHasPendingTasksDropsStaleIndexEntry covers a second unrecoverable path: +// the active index can list a run whose hash is already gone. Returning that as +// an error made the request fail permanently (the consumer redelivers it +// forever), which is the same "only fixable by hand-cleaning Redis" failure. +func TestHasPendingTasksDropsStaleIndexEntry(t *testing.T) { + producer, store := newZombieTestProducer(t) + ctx := context.Background() + + // Index a run that has no metadata hash. + if _, err := store.RedisClient().Do(ctx, "SADD", store.ActiveRunsKey(), "ghost-run"); err != nil { + t.Fatal(err) + } + pending, err := producer.hasPendingTasks(ctx, "new-run") + if err != nil { + t.Fatalf("a stale index entry must not fail the request: %v", err) + } + if pending { + t.Fatal("a stale index entry must not provide backpressure") + } + // The stale entry is removed so it does not have to be handled again. + members, err := store.ActiveRuns(ctx) + if err != nil { + t.Fatal(err) + } + for _, member := range members { + if member == "ghost-run" { + t.Fatal("the stale index entry must be removed") + } + } +} + +// TestProduceCreatesRunAfterZombie covers the end-to-end path through Produce: +// a zombie run must not stop a new run from being created. +func TestProduceCreatesRunAfterZombie(t *testing.T) { + store, err := zset.New(testutil.Redis(t), "flow:") + if err != nil { + t.Fatal(err) + } + ctx := context.Background() + producer, err := New(&pageScanner{cards: []Card{ + {ID: 1, ChanKey: "channel-a", TempKey: "sim_yd_v5", Operator: 1}, + }}, store, Config{PageSize: 100, RoundRetryEnabled: true}) + if err != nil { + t.Fatal(err) + } + + // A zombie run from a crashed producer. + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "zombie-run", Status: "producing", ProducerDone: false, CurrentRound: 1}); err != nil { + t.Fatal(err) + } + + // A new request must still produce its own run. + if err := producer.Produce(ctx, scopedMessage("fresh-run", "channel-a")); err != nil { + t.Fatalf("a zombie run must not stop a new run: %v", err) + } + meta, err := store.GetRunMeta(ctx, "fresh-run") + if err != nil { + t.Fatal(err) + } + // The producer enqueues into the active ZSet; ActiveCount is the count it + // maintains. EnqueuedCount is an accounting field the producer does not set. + if !meta.ProducerDone || meta.ActiveCount != 1 { + t.Fatalf("fresh run metadata = %+v, want producerDone with 1 active task", meta) + } +} diff --git a/internal/producer/imei/lease_client.go b/internal/producer/imei/lease_client.go new file mode 100644 index 0000000..6138fbf --- /dev/null +++ b/internal/producer/imei/lease_client.go @@ -0,0 +1,95 @@ +package imei + +import ( + "context" + "strings" + "time" + + "github.com/gogf/gf/v2/container/gvar" + "github.com/gogf/gf/v2/database/gredis" + "github.com/gogf/gf/v2/errors/gerror" + goredis "github.com/redis/go-redis/v9" + "github.com/redis/go-redis/v9/maintnotifications" + + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +type leaseClientFactory func() (queue.RedisClient, func() error, error) + +// A custom client must explicitly supply the same bounded-I/O contract. There +// is no fallback to context cancellation on an unknown Redis implementation. +type boundedLeaseProvider interface { + newBoundedLeaseClient(time.Duration) (queue.RedisClient, func() error, error) +} + +func newLeaseClientFactory(client queue.RedisClient, config *gredis.Config, budget time.Duration) (leaseClientFactory, error) { + if budget <= 0 { + return nil, gerror.New("IMEI lease I/O budget must be positive") + } + if provider, ok := client.(boundedLeaseProvider); ok { + return func() (queue.RedisClient, func() error, error) { return provider.newBoundedLeaseClient(budget) }, nil + } + getter, ok := client.(interface{ GetAdapter() gredis.Adapter }) + if !ok || getter.GetAdapter() == nil { + return nil, gerror.New("IMEI producer requires a bounded GoFrame Redis adapter") + } + raw, ok := getter.GetAdapter().Client().(*goredis.Client) + if !ok { + return nil, gerror.New("IMEI producer does not support Redis Cluster or an unknown Redis adapter") + } + opts := *raw.Options() + if opts.Addr == "FailoverClient" { + if config == nil || config.MasterName == "" || config.Cluster || config.SlaveOnly { + return nil, gerror.New("IMEI Sentinel lease client requires its master configuration") + } + // Rebuild Sentinel discovery too: sharing the old dialer would retain + // its unbounded Sentinel socket reads even if master I/O were bounded. + cfg := *config + return func() (queue.RedisClient, func() error, error) { + client := goredis.NewFailoverClient(&goredis.FailoverOptions{ + MasterName: cfg.MasterName, SentinelAddrs: sentinelAddresses(cfg.Address), + SentinelUsername: cfg.SentinelUser, SentinelPassword: cfg.SentinelPass, + Username: cfg.User, Password: cfg.Pass, DB: cfg.Db, Protocol: cfg.Protocol, + TLSConfig: opts.TLSConfig, + DialTimeout: budget, ReadTimeout: budget, WriteTimeout: budget, PoolTimeout: budget, + ContextTimeoutEnabled: true, MaxRetries: -1, DialerRetries: 1, + PoolSize: 1, MaxActiveConns: 1, + }) + return &leaseRedis{client}, client.Close, nil + }, nil + } + // Own a separate pool so a timeout/close never closes a consumer's socket. + opts.DialTimeout, opts.ReadTimeout, opts.WriteTimeout, opts.PoolTimeout = budget, budget, budget, budget + opts.ContextTimeoutEnabled = true + // Maintenance handoffs can relax socket deadlines; leases must never do so. + opts.MaintNotificationsConfig = &maintnotifications.Config{Mode: maintnotifications.ModeDisabled} + opts.MaxRetries, opts.DialerRetries = -1, 1 + opts.PoolSize, opts.MaxActiveConns, opts.MinIdleConns, opts.MaxIdleConns = 1, 1, 0, 1 + opts.Dialer = goredis.NewDialer(&opts) + return func() (queue.RedisClient, func() error, error) { + client := goredis.NewClient(&opts) + return &leaseRedis{client}, client.Close, nil + }, nil +} + +func sentinelAddresses(address string) []string { + var addresses []string + for _, addr := range strings.Split(address, ",") { + if addr = strings.TrimSpace(addr); addr != "" { + addresses = append(addresses, addr) + } + } + return addresses +} + +type leaseRedis struct{ client *goredis.Client } + +func (r *leaseRedis) Do(ctx context.Context, command string, args ...any) (*gvar.Var, error) { + result, err := r.client.Do(ctx, append([]any{command}, args...)...).Result() + return gvar.New(result), err +} + +func (r *leaseRedis) Eval(ctx context.Context, script string, _ int64, keys []string, args []any) (*gvar.Var, error) { + result, err := r.client.Eval(ctx, script, keys, args...).Result() + return gvar.New(result), err +} diff --git a/internal/producer/imei/lease_io_test.go b/internal/producer/imei/lease_io_test.go new file mode 100644 index 0000000..ba4f9c9 --- /dev/null +++ b/internal/producer/imei/lease_io_test.go @@ -0,0 +1,576 @@ +package imei + +import ( + "bufio" + "context" + "errors" + "io" + "net" + "os" + "os/exec" + "path/filepath" + "strconv" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" + "github.com/gogf/gf/v2/database/gredis" + goredis "github.com/redis/go-redis/v9" +) + +// This proxy withholds a real Redis request/reply without closing its TCP socket. +func readLeaseRESP(r *bufio.Reader) ([]byte, error) { + line, err := r.ReadBytes('\n') + if err != nil { + return nil, err + } + switch line[0] { + case '$': + n, err := strconv.Atoi(strings.TrimSpace(string(line[1:]))) + if err != nil { + return nil, err + } + if n >= 0 { + payload := make([]byte, n+2) + if _, err := io.ReadFull(r, payload); err != nil { + return nil, err + } + line = append(line, payload...) + } + case '*', '%', '~', '>', '|': + n, err := strconv.Atoi(strings.TrimSpace(string(line[1:]))) + if err != nil { + return nil, err + } + if line[0] == '%' || line[0] == '|' { + n *= 2 + } + for i := 0; i < n; i++ { + payload, err := readLeaseRESP(r) + if err != nil { + return nil, err + } + line = append(line, payload...) + } + } + return line, nil +} + +func TestProduceBoundsRealRedisRenewalIO(t *testing.T) { + for _, mode := range []string{"standalone", "sentinel"} { + t.Run(mode, func(t *testing.T) { testProduceBoundsRealRedisRenewalIO(t, mode) }) + } +} + +func testProduceBoundsRealRedisRenewalIO(t *testing.T, mode string) { + for _, callerCancel := range []bool{false, true} { + t.Run(strconv.FormatBool(callerCancel), func(t *testing.T) { + backend := testutil.Redis(t) + real, held, release := leaseRedisProxy(t, backend, "redis.call('PEXPIRE'", true) + // Release the gate even on a red-test failure so no worker leaks. + defer release() + opts := real.GetAdapter().Client().(*goredis.Client).Options() + if opts.ReadTimeout != 0 || opts.WriteTimeout != 0 || opts.ContextTimeoutEnabled { + t.Fatal("test must start with production-default unbounded socket I/O") + } + cause := errors.New("snapshot failed independently") + ctx, cancel := context.WithCancelCause(context.Background()) + defer cancel(nil) + scanner := &lockTestScanner{snapshot: func(workCtx context.Context) (int64, error) { + select { + case <-held: + if callerCancel { + cancel(cause) + <-workCtx.Done() + return 0, workCtx.Err() + } + return 0, cause + case <-workCtx.Done(): + return 0, workCtx.Err() + } + }} + store, err := queue.New(real, "imei:") + if err != nil { + t.Fatal(err) + } + config := Config{RunLockTTL: time.Second} + if mode == "sentinel" { + real, config.LeaseRedisConfig = sentinelLeaseRedis(t, real) + store, err = queue.New(real, "imei:") + if err != nil { + t.Fatal(err) + } + } + producer, err := New(scanner, store, config) + if err != nil { + t.Fatal(err) + } + result := make(chan error, 1) + go func() { result <- producer.Produce(ctx, model.ImeiQueueStartMessage{RunID: "lock-test"}) }() + select { + case <-held: + case <-time.After(2 * time.Second): + t.Fatal("proxy did not hold renewal response") + } + select { + case err := <-result: + if !errors.Is(err, cause) { + t.Fatalf("lost original cause: %v", err) + } + case <-time.After(600 * time.Millisecond): + release() + <-result + t.Fatal("canceled renewal socket I/O blocked cleanup beyond its lease budget") + } + if producer.running.Load() != 0 { + t.Fatal("running flag was not reset") + } + lock, err := backend.Do(context.Background(), "GET", producer.config.RunLockKey) + if err != nil || !lock.IsNil() { + t.Fatalf("lock not released: %v, %v", lock, err) + } + if opts.ReadTimeout != 0 || opts.WriteTimeout != 0 { + t.Fatal("lease client mutated the shared production client") + } + // A fresh run must start without opening the stalled gate. + producer.scanner = cardScanner{} + if err := producer.Produce(context.Background(), model.ImeiQueueStartMessage{RunID: "next-run"}); err != nil { + t.Fatalf("producer cannot restart: %v", err) + } + }) + } +} + +func TestProduceFencesDelayedRealRedisWrites(t *testing.T) { + for _, mode := range []string{"standalone", "sentinel"} { + t.Run(mode, func(t *testing.T) { testProduceFencesDelayedRealRedisWrites(t, mode) }) + } +} + +func testProduceFencesDelayedRealRedisWrites(t *testing.T, mode string) { + for _, stage := range []string{"enqueue", "producer_done"} { + t.Run(stage, func(t *testing.T) { + backend := testutil.Redis(t) + match := "local taskID=ARGV[i]" + cards := []Card{{ID: 1, ICCID: "a", ChanKey: "ch", TempKey: "sim_yd_v5"}} + if stage == "producer_done" { + match = "'producerCompletedAt'" + cards = nil + } + real, held, release := leaseRedisProxy(t, backend, match, false) + defer release() + workContexts := make(chan context.Context, 1) + scanner := &lockTestScanner{cardScanner: cardScanner{cards: cards}} + scanner.snapshot = func(ctx context.Context) (int64, error) { workContexts <- ctx; return int64(len(cards)), nil } + store, err := queue.New(real, "imei:") + if err != nil { + t.Fatal(err) + } + config := Config{RunLockTTL: time.Second} + if mode == "sentinel" { + real, config.LeaseRedisConfig = sentinelLeaseRedis(t, real) + store, err = queue.New(real, "imei:") + if err != nil { + t.Fatal(err) + } + } + producer, err := New(scanner, store, config) + if err != nil { + t.Fatal(err) + } + result := startLockTestProducer(t, producer) + select { + case <-held: + case <-time.After(2 * time.Second): + t.Fatal("proxy did not hold old-owner EVAL") + } + workCtx := <-workContexts + observer, err := queue.New(backend, "imei:") + if err != nil { + t.Fatal(err) + } + ok, err := observer.AcquireLockOrTakeover(context.Background(), producer.config.RunLockKey, "new-owner", "lock-test", time.Minute) + if err != nil || !ok { + t.Fatalf("takeover: %v, %v", ok, err) + } + select { + case <-workCtx.Done(): + case <-time.After(2 * time.Second): + t.Fatal("real renewal did not detect takeover") + } + if !errors.Is(context.Cause(workCtx), ErrAlreadyRunning) { + t.Fatalf("wrong loss cause: %v", context.Cause(workCtx)) + } + release() + select { + case err := <-result: + if !errors.Is(err, ErrAlreadyRunning) { + t.Fatalf("wrong result: %v", err) + } + case <-time.After(2 * time.Second): + t.Fatal("producer did not finish") + } + meta, err := observer.GetRunMeta(context.Background(), "lock-test") + if err != nil || meta.ActiveCount != 0 || meta.ProducerDone { + t.Fatalf("stale owner changed run state: %+v, %v", meta, err) + } + channels, err := observer.RunChannels(context.Background(), "lock-test") + if err != nil || len(channels) != 0 { + t.Fatalf("stale owner indexed channel: %v, %v", channels, err) + } + ok, err = observer.RenewLock(context.Background(), producer.config.RunLockKey, "new-owner", time.Minute) + if err != nil || !ok { + t.Fatalf("old cleanup removed new lease: %v, %v", ok, err) + } + added, err := observer.EnqueueRunBatch(context.Background(), "lock-test", "ch", []queue.Task{{TaskID: "next", RunID: "lock-test", ChanKey: "ch", CardID: 2}}, time.Now()) + if err != nil || added != 1 { + t.Fatalf("new work rejected: added=%d, %v", added, err) + } + }) + } +} + +func leaseRedisProxy(t *testing.T, upstream *gredis.Redis, match string, holdReply bool) (*gredis.Redis, <-chan struct{}, func()) { + t.Helper() + raw := upstream.GetAdapter().Client().(*goredis.Client) + listener, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + held, gate := make(chan struct{}), make(chan struct{}) + var heldOnce, releaseOnce sync.Once + release := func() { releaseOnce.Do(func() { close(gate) }) } + var connections sync.Map + go func() { + for { + down, err := listener.Accept() + if err != nil { + return + } + connections.Store(down, struct{}{}) + go func() { + defer down.Close() + defer connections.Delete(down) + up, err := net.Dial("tcp", raw.Options().Addr) + if err != nil { + return + } + defer up.Close() + dr, ur := bufio.NewReader(down), bufio.NewReader(up) + for { + request, err := readLeaseRESP(dr) + if err != nil { + return + } + holding := strings.Contains(string(request), match) + if holding && !holdReply { + heldOnce.Do(func() { close(held) }) + <-gate + } + if _, err := up.Write(request); err != nil { + return + } + response, err := readLeaseRESP(ur) + if err != nil { + return + } + if holding && holdReply { + heldOnce.Do(func() { close(held) }) + <-gate + } + if _, err := down.Write(response); err != nil { + return + } + } + }() + } + }() + t.Cleanup(func() { + release() + _ = listener.Close() + connections.Range(func(key, _ any) bool { _ = key.(net.Conn).Close(); return true }) + }) + client, err := gredis.New(&gredis.Config{Address: listener.Addr().String(), DialTimeout: 100 * time.Millisecond, Protocol: 2}) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = client.Close(context.Background()) }) + if _, err := client.Do(context.Background(), "PING"); err != nil { + t.Fatal(err) + } + opts := client.GetAdapter().Client().(*goredis.Client).Options() + t.Logf("real driver options: readTimeout=%s writeTimeout=%s contextTimeoutEnabled=%v", opts.ReadTimeout, opts.WriteTimeout, opts.ContextTimeoutEnabled) + return client, held, release +} + +// A real disposable Sentinel discovers the proxy as its master; neither client +// configuration nor server state comes from the application's Redis settings. +func sentinelLeaseRedis(t *testing.T, master *gredis.Redis) (*gredis.Redis, *gredis.Config) { + t.Helper() + binary := os.Getenv("REDIS_SERVER_BINARY") + if binary == "" { + var err error + binary, err = exec.LookPath("redis-server") + if err != nil { + t.Skip("Sentinel tests require redis-server") + } + } + listener, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + address := listener.Addr().String() + _, port, _ := net.SplitHostPort(address) + _ = listener.Close() + masterAddress := master.GetAdapter().Client().(*goredis.Client).Options().Addr + host, masterPort, err := net.SplitHostPort(masterAddress) + if err != nil { + t.Fatal(err) + } + dir := t.TempDir() + path := filepath.Join(dir, "sentinel.conf") + contents := "bind 127.0.0.1\nport " + port + "\nsentinel monitor lease-test " + host + " " + masterPort + " 1\nsentinel down-after-milliseconds lease-test 60000\n" + if err := os.WriteFile(path, []byte(contents), 0600); err != nil { + t.Fatal(err) + } + command := exec.Command(binary, filepath.Base(path), "--sentinel") + command.Dir = dir + logPath := filepath.Join(dir, "sentinel.log") + logFile, err := os.Create(logPath) + if err != nil { + t.Fatal(err) + } + defer logFile.Close() + command.Stdout, command.Stderr = logFile, logFile + if err := command.Start(); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = command.Process.Kill(); _ = command.Wait() }) + probe, err := gredis.New(&gredis.Config{Address: address, ReadTimeout: 100 * time.Millisecond, WriteTimeout: 100 * time.Millisecond, DialTimeout: 100 * time.Millisecond, Protocol: 2}) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = probe.Close(context.Background()) }) + for deadline := time.Now().Add(5 * time.Second); ; { + if _, err = probe.Do(context.Background(), "SENTINEL", "get-master-addr-by-name", "lease-test"); err == nil { + break + } + if time.Now().After(deadline) { + log, _ := os.ReadFile(logPath) + t.Fatalf("isolated Sentinel did not start: %v; %s", err, log) + } + time.Sleep(20 * time.Millisecond) + } + config := &gredis.Config{Address: address, MasterName: "lease-test", DialTimeout: 100 * time.Millisecond, Protocol: 2} + client, err := gredis.New(config) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = client.Close(context.Background()) }) + return client, config +} + +func TestLeaseClientBoundsSentinelDiscoveryIO(t *testing.T) { + master := testutil.Redis(t) + _, config := sentinelLeaseRedis(t, master) + sentinel, err := gredis.New(&gredis.Config{Address: config.Address, ReadTimeout: time.Second, WriteTimeout: time.Second, Protocol: 2}) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = sentinel.Close(context.Background()) }) + proxy, held, release := leaseRedisProxy(t, sentinel, "get-master-addr-by-name", true) + defer release() + config.Address = proxy.GetAdapter().Client().(*goredis.Client).Options().Addr + shared, err := gredis.New(config) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = shared.Close(context.Background()) }) + factory, err := newLeaseClientFactory(shared, config, 100*time.Millisecond) + if err != nil { + t.Fatal(err) + } + client, closeClient, err := factory() + if err != nil { + t.Fatal(err) + } + defer closeClient() + result := make(chan error, 1) + go func() { _, err := client.Do(context.Background(), "PING"); result <- err }() + select { + case <-held: + case <-time.After(2 * time.Second): + t.Fatal("Sentinel discovery request was not held") + } + select { + case err := <-result: + if err == nil { + t.Fatal("stalled Sentinel discovery unexpectedly succeeded") + } + case <-time.After(600 * time.Millisecond): + release() + <-result + t.Fatal("Sentinel discovery has no socket I/O upper bound") + } +} + +func TestLeaseClientRejectsClusterAndUnknownAdapter(t *testing.T) { + cluster, err := gredis.New(&gredis.Config{Address: "127.0.0.1:1", Cluster: true}) + if err != nil { + t.Fatal(err) + } + defer cluster.Close(context.Background()) + store, err := queue.New(cluster, "imei:") + if err != nil { + t.Fatal(err) + } + if _, err := New(cardScanner{}, store, Config{}); err == nil || !strings.Contains(err.Error(), "Cluster") { + t.Fatalf("Cluster must be rejected at initialization before any write: %v", err) + } + if _, err := newLeaseClientFactory(&leaseRedis{}, nil, time.Second); err == nil { + t.Fatal("unknown adapter must not fall back to unbounded context-only I/O") + } +} + +func TestProduceBoundsReleaseIO(t *testing.T) { + backend := testutil.Redis(t) + real, held, release := leaseRedisProxy(t, backend, "redis.call('DEL', KEYS[1])", true) + defer release() + store, err := queue.New(real, "imei:") + if err != nil { + t.Fatal(err) + } + producer, err := New(cardScanner{}, store, Config{RunLockTTL: time.Second}) + if err != nil { + t.Fatal(err) + } + result := startLockTestProducer(t, producer) + select { + case <-held: + case <-time.After(2 * time.Second): + t.Fatal("release response was not held") + } + select { + case err := <-result: + if err != nil { + t.Fatal(err) + } + case <-time.After(600 * time.Millisecond): + release() + <-result + t.Fatal("release socket I/O blocked cleanup") + } + if producer.running.Load() != 0 { + t.Fatal("cleanup did not reset running") + } +} + +func TestLeaseClientBoundsSocketWriteIO(t *testing.T) { + backend := testutil.Redis(t) + upstream := backend.GetAdapter().Client().(*goredis.Client).Options().Addr + listener, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + var stopReading atomic.Bool + held, gate := make(chan struct{}), make(chan struct{}) + var releaseOnce sync.Once + release := func() { releaseOnce.Do(func() { close(gate) }) } + defer release() + t.Cleanup(func() { _ = listener.Close() }) + go func() { + down, err := listener.Accept() + if err != nil { + return + } + defer down.Close() + up, err := net.Dial("tcp", upstream) + if err != nil { + return + } + defer up.Close() + dr, ur := bufio.NewReader(down), bufio.NewReader(up) + for { + // Read only the array header once armed: the large EVAL payload + // cannot enter the upstream or fit in TCP buffers, so Write blocks. + header, err := dr.ReadBytes('\n') + if err != nil { + return + } + if stopReading.Load() { + close(held) + <-gate + return + } + n, err := strconv.Atoi(strings.TrimSpace(string(header[1:]))) + if err != nil { + return + } + for i := 0; i < n; i++ { + part, err := readLeaseRESP(dr) + if err != nil { + return + } + header = append(header, part...) + } + if _, err := up.Write(header); err != nil { + return + } + response, err := readLeaseRESP(ur) + if err != nil { + return + } + if _, err := down.Write(response); err != nil { + return + } + } + }() + shared, err := gredis.New(&gredis.Config{Address: listener.Addr().String(), Protocol: 2}) + if err != nil { + t.Fatal(err) + } + defer shared.Close(context.Background()) + factory, err := newLeaseClientFactory(shared, nil, 100*time.Millisecond) + if err != nil { + t.Fatal(err) + } + client, closeClient, err := factory() + if err != nil { + t.Fatal(err) + } + defer closeClient() + if _, err := client.Do(context.Background(), "PING"); err != nil { + t.Fatal(err) + } + opts := client.(*leaseRedis).client.Options() + if opts.ReadTimeout != 100*time.Millisecond || opts.WriteTimeout != 100*time.Millisecond || !opts.ContextTimeoutEnabled || opts.MaxRetries != 0 || opts.DialerRetries != 1 { + t.Fatalf("lease client has unsafe timeout/retry options") + } + stopReading.Store(true) + result := make(chan error, 1) + go func() { + _, err := client.Eval(context.Background(), "return 1 --"+strings.Repeat("x", 32<<20), 0, nil, nil) + result <- err + }() + select { + case <-held: + case <-time.After(2 * time.Second): + t.Fatal("write proxy did not stop reading") + } + select { + case err := <-result: + var timeout net.Error + if !errors.As(err, &timeout) || !timeout.Timeout() || !strings.Contains(err.Error(), "write") { + t.Fatalf("expected real socket write timeout, got %v", err) + } + case <-time.After(600 * time.Millisecond): + release() + <-result + t.Fatal("socket write has no upper bound") + } +} diff --git a/internal/producer/imei/lock_cancel_test.go b/internal/producer/imei/lock_cancel_test.go new file mode 100644 index 0000000..d6df29f --- /dev/null +++ b/internal/producer/imei/lock_cancel_test.go @@ -0,0 +1,405 @@ +package imei + +import ( + "context" + "errors" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/gogf/gf/v2/container/gvar" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +type lockTestRedis struct { + queue.RedisClient + do func(context.Context, string, []any) (*gvar.Var, error) + eval func(context.Context, string, int64, []string, []any) (*gvar.Var, error) +} + +func (r *lockTestRedis) newBoundedLeaseClient(budget time.Duration) (queue.RedisClient, func() error, error) { + factory, err := newLeaseClientFactory(r.RedisClient, nil, budget) + if err != nil { + return nil, nil, err + } + client, closeClient, err := factory() + if err != nil { + return nil, nil, err + } + return &lockTestRedis{RedisClient: client, + do: func(ctx context.Context, command string, args []any) (*gvar.Var, error) { + if r.do != nil { + return r.do(ctx, command, args) + } + return client.Do(ctx, command, args...) + }, + eval: func(ctx context.Context, script string, numKeys int64, keys []string, args []any) (*gvar.Var, error) { + if r.eval != nil { + return r.eval(ctx, script, numKeys, keys, args) + } + return client.Eval(ctx, script, numKeys, keys, args) + }, + }, closeClient, nil +} + +func (r *lockTestRedis) Do(ctx context.Context, command string, args ...any) (*gvar.Var, error) { + if r.do != nil { + return r.do(ctx, command, args) + } + return r.RedisClient.Do(ctx, command, args...) +} + +func (r *lockTestRedis) Eval(ctx context.Context, script string, numKeys int64, keys []string, args []any) (*gvar.Var, error) { + if r.eval != nil { + return r.eval(ctx, script, numKeys, keys, args) + } + return r.RedisClient.Eval(ctx, script, numKeys, keys, args) +} + +type lockTestScanner struct { + cardScanner + scanCalls atomic.Int32 + snapshot func(context.Context) (int64, error) + scan func(context.Context, int64, int64, int) ([]Card, error) +} + +func (s *lockTestScanner) SnapshotMaxID(ctx context.Context) (int64, error) { + if s.snapshot != nil { + return s.snapshot(ctx) + } + return s.cardScanner.SnapshotMaxID(ctx) +} + +func (s *lockTestScanner) Scan(ctx context.Context, last, max int64, limit int) ([]Card, error) { + s.scanCalls.Add(1) + if s.scan != nil { + return s.scan(ctx, last, max, limit) + } + return s.cardScanner.Scan(ctx, last, max, limit) +} + +func startLockTestProducer(t *testing.T, producer *Producer) <-chan error { + t.Helper() + ctx, cancel := context.WithCancel(context.Background()) + result := make(chan error, 1) + done := make(chan struct{}) + go func() { + defer close(done) + result <- producer.Produce(ctx, model.ImeiQueueStartMessage{RunID: "lock-test"}) + }() + t.Cleanup(func() { + cancel() + select { + case <-done: + case <-time.After(3 * time.Second): + t.Error("producer goroutine did not stop after cleanup cancellation") + } + }) + return result +} + +func TestProduceStopsOnLockLossWhileWaitingForCapacity(t *testing.T) { + backendErr := errors.New("renewal backend unavailable") + for _, test := range []struct { + name string + err error + }{ + {name: "taken_over", err: ErrAlreadyRunning}, + {name: "redis_error", err: backendErr}, + } { + t.Run(test.name, func(t *testing.T) { + var ( + client = &lockTestRedis{RedisClient: testutil.Redis(t)} + waiting = make(chan struct{}) + failRenew = make(chan struct{}) + waitingOnce sync.Once + capacityRead atomic.Bool + scanner = &lockTestScanner{cardScanner: cardScanner{cards: []Card{ + {ID: 1, ICCID: "a", ChanKey: "ch", TempKey: "sim_yd_v5"}, + {ID: 2, ICCID: "b", ChanKey: "ch", TempKey: "sim_yd_v5"}, + {ID: 3, ICCID: "c", ChanKey: "ch", TempKey: "sim_yd_v5"}, + }}} + ) + store, err := queue.New(client, "imei:") + if err != nil { + t.Fatal(err) + } + producer, err := New(scanner, store, Config{ + PageSize: 2, GlobalHighWaterMark: 1, + RunLockTTL: 300 * time.Millisecond, PollInterval: time.Hour, + }) + if err != nil { + t.Fatal(err) + } + client.do = func(ctx context.Context, command string, args []any) (*gvar.Var, error) { + result, err := client.RedisClient.Do(ctx, command, args...) + if err == nil && command == "SMEMBERS" && args[0] == store.ActiveRunsKey() { + capacityRead.Store(true) + } + if err == nil && command == "HGETALL" && capacityRead.Load() && result.MapStrStr()["activeCount"] == "2" { + waitingOnce.Do(func() { close(waiting) }) + } + return result, err + } + client.eval = func(ctx context.Context, script string, numKeys int64, keys []string, args []any) (*gvar.Var, error) { + if strings.Contains(script, "redis.call('PEXPIRE'") { + select { + case <-failRenew: + case <-ctx.Done(): + return nil, ctx.Err() + } + if test.err == backendErr { + return nil, backendErr + } + } + return client.RedisClient.Eval(ctx, script, numKeys, keys, args) + } + result := startLockTestProducer(t, producer) + select { + case <-waiting: + case <-time.After(2 * time.Second): + t.Fatal("producer did not reach backpressure") + } + // The long poll cannot finish naturally; renewal failure must wake it. + select { + case err := <-result: + t.Fatalf("producer exited before lock loss: %v", err) + case <-time.After(20 * time.Millisecond): + } + if test.err == ErrAlreadyRunning { + ok, err := store.AcquireLockOrTakeover(context.Background(), producer.config.RunLockKey, "new-owner", "lock-test", time.Minute) + if err != nil || !ok { + t.Fatalf("take over lock: acquired=%v err=%v", ok, err) + } + } + close(failRenew) + select { + case err := <-result: + if !errors.Is(err, test.err) { + t.Fatalf("producer error = %v, want renewal cause %v", err, test.err) + } + case <-time.After(2 * time.Second): + t.Fatal("lock loss did not interrupt backpressure before the next capacity poll") + } + if calls := scanner.scanCalls.Load(); calls != 1 { + t.Fatalf("scan calls = %d, want only the first page", calls) + } + meta, err := store.GetRunMeta(context.Background(), "lock-test") + if err != nil || meta.ProducerDone || meta.ActiveCount != 2 { + t.Fatalf("run must remain incomplete without additional tasks: meta=%+v err=%v", meta, err) + } + if producer.running.Load() != 0 { + t.Fatal("producer running flag was not reset") + } + if test.err == ErrAlreadyRunning { + ok, err := store.RenewLock(context.Background(), producer.config.RunLockKey, "new-owner", time.Minute) + if err != nil || !ok { + t.Fatalf("old producer released the new owner's lock: renewed=%v err=%v", ok, err) + } + } + }) + } +} + +func TestProduceCancelsLockedOperationsWithRenewalCause(t *testing.T) { + for _, stage := range []string{"run_init", "snapshot", "scan", "enqueue", "producer_done"} { + t.Run(stage, func(t *testing.T) { + var ( + client = &lockTestRedis{RedisClient: testutil.Redis(t)} + entered = make(chan context.Context, 1) + failRenew = make(chan struct{}) + cause = errors.New("renewal failed during " + stage) + scanner = &lockTestScanner{cardScanner: cardScanner{cards: []Card{ + {ID: 1, ICCID: "a", ChanKey: "ch", TempKey: "sim_yd_v5"}, + }}} + ) + block := func(ctx context.Context) error { + entered <- ctx + <-ctx.Done() + return ctx.Err() + } + if stage == "snapshot" { + scanner.snapshot = func(ctx context.Context) (int64, error) { return 0, block(ctx) } + } + if stage == "scan" { + scanner.scan = func(ctx context.Context, _, _ int64, _ int) ([]Card, error) { return nil, block(ctx) } + } + store, err := queue.New(client, "imei:") + if err != nil { + t.Fatal(err) + } + producer, err := New(scanner, store, Config{RunLockTTL: 300 * time.Millisecond}) + if err != nil { + t.Fatal(err) + } + client.eval = func(ctx context.Context, script string, numKeys int64, keys []string, args []any) (*gvar.Var, error) { + if strings.Contains(script, "redis.call('PEXPIRE'") { + select { + case <-failRenew: + return nil, cause + case <-ctx.Done(): + return nil, ctx.Err() + } + } + if stage == "run_init" && strings.Contains(script, "'createdAt', ARGV[12]") || + stage == "enqueue" && strings.Contains(script, "local taskID=ARGV[i]") || + stage == "producer_done" && strings.Contains(script, "'producerCompletedAt'") { + return nil, block(ctx) + } + return client.RedisClient.Eval(ctx, script, numKeys, keys, args) + } + result := startLockTestProducer(t, producer) + var workCtx context.Context + select { + case workCtx = <-entered: + case <-time.After(2 * time.Second): + t.Fatal("producer did not enter " + stage) + } + close(failRenew) + select { + case err := <-result: + if !errors.Is(err, cause) || !errors.Is(context.Cause(workCtx), cause) { + t.Fatalf("renewal cause lost: result=%v context cause=%v", err, context.Cause(workCtx)) + } + case <-time.After(2 * time.Second): + t.Fatal("lock loss did not interrupt " + stage) + } + if stage == "run_init" { + if scanner.scanCalls.Load() != 0 { + t.Fatal("producer scanned after run initialization failed") + } + return + } + meta, err := store.GetRunMeta(context.Background(), "lock-test") + wantActive := int64(0) + if stage == "producer_done" { + wantActive = 1 + } + if err != nil || meta.ProducerDone || meta.ActiveCount != wantActive { + t.Fatalf("unexpected work after lock loss: meta=%+v err=%v", meta, err) + } + }) + } +} + +func TestProduceJoinsRenewalBeforeReleasingLock(t *testing.T) { + scanErr := errors.New("snapshot failed") + for _, test := range []struct { + name string + err error + }{ + {name: "success"}, + {name: "scanner_error", err: scanErr}, + } { + t.Run(test.name, func(t *testing.T) { + var ( + client = &lockTestRedis{RedisClient: testutil.Redis(t)} + renewEntered = make(chan struct{}) + renewCanceled = make(chan struct{}) + allowRenewReturn = make(chan struct{}) + resumeSnapshot = make(chan struct{}) + released = make(chan struct{}, 1) + allowReturnOnce sync.Once + scanner = &lockTestScanner{} + ) + scanner.snapshot = func(ctx context.Context) (int64, error) { + select { + case <-resumeSnapshot: + return 0, test.err + case <-ctx.Done(): + return 0, ctx.Err() + } + } + store, err := queue.New(client, "imei:") + if err != nil { + t.Fatal(err) + } + producer, err := New(scanner, store, Config{RunLockTTL: 300 * time.Millisecond}) + if err != nil { + t.Fatal(err) + } + client.eval = func(ctx context.Context, script string, numKeys int64, keys []string, args []any) (*gvar.Var, error) { + if strings.Contains(script, "redis.call('PEXPIRE'") { + close(renewEntered) + <-ctx.Done() + close(renewCanceled) + <-allowRenewReturn + return nil, ctx.Err() + } + if len(keys) == 1 && keys[0] == producer.config.RunLockKey && len(args) == 1 { + released <- struct{}{} + } + return client.RedisClient.Eval(ctx, script, numKeys, keys, args) + } + result := startLockTestProducer(t, producer) + t.Cleanup(func() { allowReturnOnce.Do(func() { close(allowRenewReturn) }) }) + select { + case <-renewEntered: + case <-time.After(2 * time.Second): + t.Fatal("renewal did not start") + } + close(resumeSnapshot) + select { + case <-renewCanceled: + case <-time.After(2 * time.Second): + t.Fatal("producer did not cancel renewal on exit") + } + // Keep the in-flight renewal alive to verify cleanup actually joins it. + select { + case err := <-result: + t.Fatalf("producer returned before renewal exited: %v", err) + case <-released: + t.Fatal("producer released the lock before renewal exited") + case <-time.After(20 * time.Millisecond): + } + allowReturnOnce.Do(func() { close(allowRenewReturn) }) + select { + case err := <-result: + if !errors.Is(err, test.err) { + t.Fatalf("cleanup changed the production error: got %v, want %v", err, test.err) + } + case <-time.After(2 * time.Second): + t.Fatal("producer did not finish after renewal exited") + } + select { + case <-released: + default: + t.Fatal("producer did not release the lock after joining renewal") + } + }) + } +} + +func TestProducePreservesCallerCancellationCause(t *testing.T) { + producer, store := newProducer(t, nil) + ctx, cancel := context.WithCancelCause(context.Background()) + defer cancel(nil) + cause := errors.New("caller stopped the run") + producer.scanner = &lockTestScanner{snapshot: func(ctx context.Context) (int64, error) { + cancel(cause) + <-ctx.Done() + return 0, ctx.Err() + }} + if err := producer.Produce(ctx, model.ImeiQueueStartMessage{RunID: "caller-canceled"}); !errors.Is(err, cause) { + t.Fatalf("caller cancellation cause lost: %v", err) + } + meta, err := store.GetRunMeta(context.Background(), "caller-canceled") + if err != nil || meta.ProducerDone || producer.running.Load() != 0 { + t.Fatalf("canceled producer did not clean up: meta=%+v err=%v running=%d", meta, err, producer.running.Load()) + } +} + +func TestWaitForCapacityHonorsCancellationWithoutWatermark(t *testing.T) { + producer := &Producer{} + ctx, cancel := context.WithCancelCause(context.Background()) + cause := errors.New("lock lost") + cancel(cause) + if err := producer.waitForCapacity(ctx); !errors.Is(err, cause) { + t.Fatalf("disabled watermark ignored cancellation: %v", err) + } +} diff --git a/internal/producer/imei/producer.go b/internal/producer/imei/producer.go new file mode 100644 index 0000000..b779b75 --- /dev/null +++ b/internal/producer/imei/producer.go @@ -0,0 +1,498 @@ +package imei + +import ( + "context" + "crypto/rand" + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + "strings" + "sync/atomic" + "time" + + "github.com/gogf/gf/v2/database/gredis" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +// FullSyncOperation is recorded on the run and used by SyncKindForOperation to +// classify the run for the admin monitor. +const FullSyncOperation = "imei_full_sync" + +// scopeSampleLimit bounds how many channel names are copied into run metadata. +const scopeSampleLimit = 10 + +// productionStats mirrors the status producer so every full-sync path reports a +// single comparable summary line. +type productionStats struct { + pages int64 + scanned int64 + enqueued int64 + skippedEmptyChan int64 + skippedNoTemplate int64 + skippedSyncDisabled int64 + skippedOutOfScope int64 + skippedNoImei int64 + scopeChannels int + channels map[string]struct{} +} + +func newProductionStats(scopeChannels int) productionStats { + return productionStats{channels: make(map[string]struct{}), scopeChannels: scopeChannels} +} + +func (s *productionStats) log(ctx context.Context, status, runID, stage, reason string, startedAt time.Time, maxID int64, err error) { + message := fmt.Sprintf("event=imei_queue_producer status=%s runId=%s snapshotMaxId=%d pages=%d scanned=%d enqueued=%d skippedEmptyChan=%d skippedNoTemplate=%d skippedSyncDisabled=%d skippedOutOfScope=%d skippedNoImei=%d channelCount=%d scopeChannels=%d durationMs=%d", status, runID, maxID, s.pages, s.scanned, s.enqueued, s.skippedEmptyChan, s.skippedNoTemplate, s.skippedSyncDisabled, s.skippedOutOfScope, s.skippedNoImei, len(s.channels), s.scopeChannels, time.Since(startedAt).Milliseconds()) + if s.scanned > 0 { + message += fmt.Sprintf(" cardsPerSec=%.2f", float64(s.scanned)/time.Since(startedAt).Seconds()) + } + if stage != "" { + message += " stage=" + stage + } + if reason != "" { + message += " reason=" + reason + } + if err != nil { + message += fmt.Sprintf(" error=%q", err.Error()) + } + switch status { + case "failed": + g.Log().Error(ctx, message) + case "skipped": + g.Log().Warning(ctx, message) + default: + g.Log().Info(ctx, message) + } +} + +type Config struct { + PageSize int + RunLockKey string + RunLockTTL time.Duration + PollInterval time.Duration + GlobalHighWaterMark int64 + // LeaseRedisConfig is required only for Sentinel discovery. Standalone + // settings come from the actual adapter; Cluster is rejected in New. + LeaseRedisConfig *gredis.Config +} + +func (c Config) withDefaults() Config { + if c.PageSize <= 0 { + c.PageSize = 1000 + } + if c.RunLockKey == "" { + c.RunLockKey = "imei:producer:active" + } + if c.RunLockTTL <= 0 { + c.RunLockTTL = 5 * time.Minute + } + if c.PollInterval <= 0 { + c.PollInterval = time.Second + } + return c +} + +type Card struct { + ID int64 + ICCID string + MSISDN string + IMSI string + ChanKey string + TempKey string + Operator int + SyncEnabled bool + Imei string +} + +type Scanner interface { + SnapshotMaxID(context.Context) (int64, error) + Scan(context.Context, int64, int64, int) ([]Card, error) +} + +type channelMetadata struct { + ChanKey string `orm:"chan_key"` + TempKey string `orm:"temp_key"` + Operator int `orm:"operator"` + SyncEnabled bool `orm:"sync_enabled"` +} + +// DatabaseScanner uses keyset pagination so a large IMEI run does not hold the +// entire card table in memory and does not degrade through OFFSET scans. +type DatabaseScanner struct { + metadata map[string]channelMetadata +} + +func (s *DatabaseScanner) SnapshotMaxID(ctx context.Context) (int64, error) { + var card entity.TSimCardInfo + columns := dao.TSimCardInfo.Columns() + if err := dao.TSimCardInfo.Ctx(ctx). + Fields(columns.Id). + OrderDesc(columns.Id). + Limit(1). + Scan(&card); err != nil { + return 0, fmt.Errorf("read imei snapshot max id: %w", err) + } + return card.Id, nil +} + +func (s *DatabaseScanner) Scan(ctx context.Context, lastID, maxID int64, pageSize int) ([]Card, error) { + if maxID <= lastID { + return nil, nil + } + if s.metadata == nil { + var rows []channelMetadata + if err := g.DB().Model("t_sim_channels").Ctx(ctx).Fields("chan_key,temp_key,sync_enabled,operator").Scan(&rows); err != nil { + return nil, fmt.Errorf("scan imei channel metadata: %w", err) + } + s.metadata = make(map[string]channelMetadata, len(rows)) + for _, row := range rows { + s.metadata[row.ChanKey] = row + } + } + columns := dao.TSimCardInfo.Columns() + var rows []entity.TSimCardInfo + if err := dao.TSimCardInfo.Ctx(ctx). + Fields(columns.Id, columns.Iccid, columns.Msisdn, columns.Imsi, columns.ChanKey, columns.Imei). + WhereGT(columns.Id, lastID). + WhereLTE(columns.Id, maxID). + OrderAsc(columns.Id). + Limit(pageSize). + Scan(&rows); err != nil { + return nil, fmt.Errorf("scan imei cards: %w", err) + } + cards := make([]Card, 0, len(rows)) + for _, row := range rows { + if row.Id <= 0 || row.Iccid == "" || row.ChanKey == "" { + // Keep the page cursor moving even when a row cannot become a + // runnable task. The producer advances by the highest returned ID. + if row.Id > 0 { + cards = append(cards, Card{ID: row.Id}) + } + continue + } + metadata := s.metadata[row.ChanKey] + cards = append(cards, Card{ID: row.Id, ICCID: row.Iccid, MSISDN: row.Msisdn, IMSI: row.Imsi, ChanKey: row.ChanKey, TempKey: metadata.TempKey, Operator: metadata.Operator, SyncEnabled: metadata.SyncEnabled, Imei: row.Imei}) + } + return cards, nil +} + +type Producer struct { + scanner Scanner + store *queue.Store + config Config + running atomic.Int32 + owner string + leaseFactory leaseClientFactory +} + +var ErrAlreadyRunning = fmt.Errorf("imei producer already running") + +// ErrRunFinished is returned when a redelivered start message names a run that +// already reached a terminal state. The run must not be resurrected. +var ErrRunFinished = fmt.Errorf("imei run already finished") + +// runAlreadyFinished reports whether runID exists and has already completed. +// A missing run is not an error: the message may be the first delivery. +func (p *Producer) runAlreadyFinished(ctx context.Context, runID string) (bool, error) { + meta, err := p.store.GetRunMeta(ctx, runID) + if err != nil { + if strings.Contains(err.Error(), "not found") { + return false, nil + } + return false, err + } + return meta.Status == "completed" || meta.Status == "completed_with_failures", nil +} + +func New(scanner Scanner, store *queue.Store, config Config) (*Producer, error) { + if scanner == nil || store == nil { + return nil, fmt.Errorf("imei producer dependencies are incomplete") + } + config = config.withDefaults() + factory, err := newLeaseClientFactory(store.RedisClient(), config.LeaseRedisConfig, leaseIOBudget(config.RunLockTTL)) + if err != nil { + return nil, err + } + return &Producer{scanner: scanner, store: store, config: config, owner: newOwner(), leaseFactory: factory}, nil +} + +func leaseIOBudget(ttl time.Duration) time.Duration { + return min(time.Second, ttl/6) +} + +func (p *Producer) Produce(ctx context.Context, message model.ImeiQueueStartMessage) (err error) { + startedAt := time.Now() + if message.RunID == "" { + return fmt.Errorf("imei sync runId is empty") + } + operation := message.Operation + if operation == "" { + operation = FullSyncOperation + } + scope, err := model.NormalizeScope(message.Scope) + if err != nil { + failedStats := newProductionStats(0) + failedStats.log(ctx, "failed", message.RunID, "validate", "invalid_scope", startedAt, 0, err) + return err + } + stats := newProductionStats(scope.ChannelCount()) + if !p.running.CompareAndSwap(0, 1) { + stats.log(ctx, "skipped", message.RunID, "precheck", "already_running", startedAt, 0, nil) + return ErrAlreadyRunning + } + defer p.running.Store(0) + leaseClient, closeLeaseClient, err := p.leaseFactory() + if err != nil { + return err + } + defer closeLeaseClient() + leaseStore, err := queue.New(leaseClient, p.store.Prefix()) + if err != nil { + return err + } + // A redelivered start message must not resurrect a run that already + // finished; that would create a second producer pass over the same RunID. + if finished, checkErr := p.runAlreadyFinished(ctx, message.RunID); checkErr != nil { + stats.log(ctx, "failed", message.RunID, "run_state", "", startedAt, 0, checkErr) + return checkErr + } else if finished { + stats.log(ctx, "skipped", message.RunID, "run_state", "run_finished", startedAt, 0, nil) + return ErrRunFinished + } + // Tokens are per lease, not per Producer: a delayed command from an earlier + // attempt must never match a later lease on this same Producer instance. + p.owner = newOwner() + acquireCtx, cancelAcquire := context.WithTimeout(ctx, leaseIOBudget(p.config.RunLockTTL)) + acquired, err := leaseStore.AcquireLockOrTakeover(acquireCtx, p.config.RunLockKey, p.owner, message.RunID, p.config.RunLockTTL) + cancelAcquire() + if err != nil { + stats.log(ctx, "failed", message.RunID, "lock", "", startedAt, 0, err) + return err + } + if !acquired { + stats.log(ctx, "skipped", message.RunID, "lock", "already_running", startedAt, 0, nil) + return ErrAlreadyRunning + } + defer func() { + releaseCtx, cancel := context.WithTimeout(context.Background(), leaseIOBudget(p.config.RunLockTTL)) + defer cancel() + _, _ = leaseStore.ReleaseLock(releaseCtx, p.config.RunLockKey, p.owner) + }() + workStore, err := p.store.WithIMEIProducerLease(p.config.RunLockKey, p.owner, message.RunID) + if err != nil { + return err + } + lockCtx, cancelLock := context.WithCancelCause(ctx) + // Stop renewal separately so normal cleanup does not become a lock-loss cause. + renewCtx, stopRenew := context.WithCancel(lockCtx) + renewDone := make(chan struct{}) + var ( + maxID int64 + stage = "run_init" + ) + go func() { + defer close(renewDone) + p.renewLock(renewCtx, cancelLock, leaseStore) + }() + defer func() { + stopRenew() + <-renewDone + // Join before reading the cause and releasing the lease: a failed renewal + // must not be masked by a scanner/Redis context-cancellation error. + if cause := context.Cause(lockCtx); cause != nil { + err = cause + } + if errors.Is(err, queue.ErrProducerLeaseLost) { + err = ErrAlreadyRunning + } + cancelLock(nil) + if err != nil { + stats.log(ctx, "failed", message.RunID, stage, "", startedAt, maxID, err) + return + } + stats.log(ctx, "completed", message.RunID, "", "", startedAt, maxID, nil) + }() + if err = workStore.CreateRun(lockCtx, queue.RunMeta{ + RunID: message.RunID, + Operation: operation, + Status: "producing", + CurrentRound: 1, + ScopeChannels: scope.ChannelCount(), + ScopeDigest: scopeDigest(scope), + ScopeChannelsSample: strings.Join(scope.ChannelSample(scopeSampleLimit), ","), + }); err != nil { + return err + } + stage = "snapshot" + if err = lockCtx.Err(); err != nil { + return err + } + maxID, err = p.scanner.SnapshotMaxID(lockCtx) + if err != nil { + return err + } + g.Log().Infof(ctx, "event=imei_queue_producer status=started runId=%s snapshotMaxId=%d scopeChannels=%d", message.RunID, maxID, scope.ChannelCount()) + lastID := int64(0) + for lastID < maxID { + if err = lockCtx.Err(); err != nil { + return err + } + stage = "scan" + cards, scanErr := p.scanner.Scan(lockCtx, lastID, maxID, p.config.PageSize) + if scanErr != nil { + return scanErr + } + if len(cards) == 0 { + break + } + stats.pages++ + stats.scanned += int64(len(cards)) + groups := make(map[string][]queue.Task) + for _, card := range cards { + if err = lockCtx.Err(); err != nil { + return err + } + if card.ID > lastID { + lastID = card.ID + } + if card.ChanKey == "" { + stats.skippedEmptyChan++ + continue + } + // Scope is applied before the template and sync_enabled checks so a + // partial run only reports on the channels it was asked for. + if !scope.Allows(card.ChanKey) { + stats.skippedOutOfScope++ + continue + } + if card.TempKey == "" { + stats.skippedNoTemplate++ + continue + } + if card.SyncEnabled { + // sync_enabled=1(否/不参与) → skip + stats.skippedSyncDisabled++ + continue + } + // skippedNoImei counts cards that already have a value. They are + // still enqueued (this is a full re-poll, so a device swap is + // detected), but the count makes the "how much of this pass is + // actually filling gaps" question answerable from one log line. + if strings.TrimSpace(card.Imei) != "" { + stats.skippedNoImei++ + } + stats.channels[card.ChanKey] = struct{}{} + task := queue.Task{TaskID: queue.StableTaskID(message.RunID, operation, card.ID), RunID: message.RunID, Operation: operation, ChanKey: card.ChanKey, TempKey: card.TempKey, Operator: card.Operator, CardID: card.ID, ICCID: card.ICCID, MSISDN: card.MSISDN, IMSI: card.IMSI} + groups[card.ChanKey] = append(groups[card.ChanKey], task) + } + for chanKey, tasks := range groups { + stage = "enqueue" + if err = lockCtx.Err(); err != nil { + return err + } + added, enqueueErr := workStore.EnqueueRunBatch(lockCtx, message.RunID, chanKey, tasks, time.Now()) + if enqueueErr != nil { + return fmt.Errorf("enqueue imei channel %s: %w", chanKey, enqueueErr) + } + stats.enqueued += int64(added) + } + stage = "capacity" + if err = p.waitForCapacity(lockCtx); err != nil { + return err + } + } + stage = "producer_done" + if err = lockCtx.Err(); err != nil { + return err + } + return workStore.MarkProducerDone(lockCtx, message.RunID) +} + +// scopeDigest is a stable short fingerprint of the selected channels. +func scopeDigest(scope *model.SyncScope) string { + if scope == nil || len(scope.Channels) == 0 { + return "" + } + digest := sha256.Sum256([]byte(strings.Join(scope.Channels, "\n"))) + return hex.EncodeToString(digest[:8]) +} + +func (p *Producer) renewLock(ctx context.Context, cancel context.CancelCauseFunc, leaseStore *queue.Store) { + interval := p.config.RunLockTTL / 3 + if interval <= 0 { + interval = time.Second + } + ticker := time.NewTicker(interval) + defer ticker.Stop() + for { + select { + case <-ctx.Done(): + return + case <-ticker.C: + if ctx.Err() != nil { + return + } + renewCtx, stop := context.WithTimeout(ctx, leaseIOBudget(p.config.RunLockTTL)) + ok, err := leaseStore.RenewLock(renewCtx, p.config.RunLockKey, p.owner, p.config.RunLockTTL) + stop() + if ctx.Err() != nil { + return + } + if err == nil && !ok { + err = ErrAlreadyRunning + } + if err != nil { + cancel(err) + return + } + } + } +} + +func (p *Producer) waitForCapacity(ctx context.Context) error { + if ctx.Err() != nil { + return context.Cause(ctx) + } + if p.config.GlobalHighWaterMark <= 0 { + return nil + } + for { + if ctx.Err() != nil { + return context.Cause(ctx) + } + runs, err := p.store.ActiveRuns(ctx) + if err != nil { + return err + } + var total int64 + for _, runID := range runs { + meta, sizeErr := p.store.GetRunMeta(ctx, runID) + if sizeErr != nil { + return sizeErr + } + // ActiveCount includes leased tasks; do not count InflightCount twice. + total += meta.ActiveCount + meta.NextRoundCount + } + if total <= p.config.GlobalHighWaterMark { + return nil + } + select { + case <-ctx.Done(): + return context.Cause(ctx) + case <-time.After(p.config.PollInterval): + } + } +} + +func newOwner() string { + var b [12]byte + if _, err := rand.Read(b[:]); err == nil { + return hex.EncodeToString(b[:]) + } + return fmt.Sprintf("%d", time.Now().UnixNano()) +} diff --git a/internal/producer/imei/producer_test.go b/internal/producer/imei/producer_test.go new file mode 100644 index 0000000..705c65c --- /dev/null +++ b/internal/producer/imei/producer_test.go @@ -0,0 +1,262 @@ +package imei + +import ( + "context" + "errors" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +type cardScanner struct{ cards []Card } + +func (s cardScanner) SnapshotMaxID(context.Context) (int64, error) { return int64(len(s.cards)), nil } +func (s cardScanner) Scan(_ context.Context, last, max int64, limit int) ([]Card, error) { + end := last + int64(limit) + if end > max { + end = max + } + return s.cards[last:end], nil +} + +func newProducer(t *testing.T, cards []Card, config ...Config) (*Producer, *queue.Store) { + t.Helper() + redis := testutil.Redis(t) + store, err := queue.New(redis, "imei:") + if err != nil { + t.Fatal(err) + } + cfg := Config{PageSize: 1} + if len(config) > 0 { + cfg = config[0] + } + producer, err := New(cardScanner{cards: cards}, store, cfg) + if err != nil { + t.Fatal(err) + } + return producer, store +} + +// TestProduceEnqueuesEligibleCards 覆盖 §7 T1：全量入队， +// 跳过无通道、无模板、不参与轮询的卡。 +func TestProduceEnqueuesEligibleCards(t *testing.T) { + ctx := context.Background() + producer, store := newProducer(t, []Card{ + {ID: 1, ICCID: "a", ChanKey: "ch", TempKey: "sim_yd_v5"}, // 合格 + {ID: 2, ICCID: "b", ChanKey: "ch", TempKey: "sim_yd_v5", SyncEnabled: true}, // sync_enabled=1 跳过 + {ID: 3, ICCID: "c", ChanKey: "ch"}, // 无模板 + {ID: 4, ICCID: "d", ChanKey: "ch", TempKey: "sim_yd_v5"}, // 合格 + }) + + if err := producer.Produce(ctx, model.ImeiQueueStartMessage{RunID: "full"}); err != nil { + t.Fatal(err) + } + meta, err := store.GetRunMeta(ctx, "full") + if err != nil { + t.Fatal(err) + } + if !meta.ProducerDone { + t.Fatal("producerDone 应为 true") + } + // 用 ActiveCount 而非 EnqueuedCount：后者是 RunMeta 上的既有死字段 + // （只有 RoundStats.EnqueuedCount 会被真实写入），状态链路同样如此。 + if meta.ActiveCount != 2 { + t.Fatalf("activeCount = %d, want 2", meta.ActiveCount) + } + claims, err := store.ClaimRun(ctx, "full", "ch", time.Now(), time.Now().Add(time.Minute), 10) + if err != nil { + t.Fatal(err) + } + if len(claims) != 2 { + t.Fatalf("claims = %d, want 2", len(claims)) + } + for _, claim := range claims { + if claim.Task.CardID != 1 && claim.Task.CardID != 4 { + t.Fatalf("不应入队 card %d", claim.Task.CardID) + } + if claim.Task.Operation != FullSyncOperation { + t.Fatalf("operation = %q, want %q", claim.Task.Operation, FullSyncOperation) + } + } +} + +// TestProduceRePollIncludesCardsThatAlreadyHaveImei 是用户决策的直接回归测试： +// 全量重查（而非只查空值卡），已有 IMEI 的卡也必须入队，否则无法发现换机。 +func TestProduceRePollIncludesCardsThatAlreadyHaveImei(t *testing.T) { + ctx := context.Background() + producer, store := newProducer(t, []Card{ + {ID: 1, ICCID: "a", ChanKey: "ch", TempKey: "sim_yd_v5", Imei: "865466070817587"}, + {ID: 2, ICCID: "b", ChanKey: "ch", TempKey: "sim_yd_v5", Imei: ""}, + }) + + if err := producer.Produce(ctx, model.ImeiQueueStartMessage{RunID: "repoll"}); err != nil { + t.Fatal(err) + } + meta, err := store.GetRunMeta(ctx, "repoll") + if err != nil { + t.Fatal(err) + } + if meta.ActiveCount != 2 { + t.Fatalf("全量重查应入队全部合格卡（含已有值），activeCount = %d, want 2", meta.ActiveCount) + } +} + +// TestProduceAppliesChannelScope 覆盖 §7 T2：按通道裁剪。 +func TestProduceAppliesChannelScope(t *testing.T) { + ctx := context.Background() + producer, store := newProducer(t, []Card{ + {ID: 1, ICCID: "a", ChanKey: "173", TempKey: "sim_yd_v5"}, + {ID: 2, ICCID: "b", ChanKey: "189", TempKey: "sim_yd_v5"}, + {ID: 3, ICCID: "c", ChanKey: "173", TempKey: "sim_yd_v5"}, + }) + + scope := &model.SyncScope{Channels: []string{"173"}, TriggerSource: model.TriggerSourceManual} + if err := producer.Produce(ctx, model.ImeiQueueStartMessage{RunID: "scoped", Scope: scope}); err != nil { + t.Fatal(err) + } + meta, err := store.GetRunMeta(ctx, "scoped") + if err != nil { + t.Fatal(err) + } + if meta.ActiveCount != 2 { + t.Fatalf("activeCount = %d, want 2（只入 173 通道）", meta.ActiveCount) + } + if meta.ScopeChannels != 1 { + t.Fatalf("scopeChannels = %d, want 1", meta.ScopeChannels) + } + if meta.ScopeDigest == "" { + t.Fatal("scopeDigest 不应为空") + } + + claims, err := store.ClaimRun(ctx, "scoped", "173", time.Now(), time.Now().Add(time.Minute), 10) + if err != nil { + t.Fatal(err) + } + if len(claims) != 2 { + t.Fatalf("173 通道应有 2 张卡，实际 %d", len(claims)) + } +} + +// TestProduceRejectsInvalidScope 非法 scope 应在建 Run 前失败。 +func TestProduceRejectsInvalidScope(t *testing.T) { + ctx := context.Background() + producer, store := newProducer(t, []Card{{ID: 1, ICCID: "a", ChanKey: "ch", TempKey: "sim_yd_v5"}}) + + // 超过 200 个通道 + tooMany := make([]string, 0, model.MaxScopeChannels+1) + for i := 0; i <= model.MaxScopeChannels; i++ { + tooMany = append(tooMany, "ch"+string(rune('a'+i%26))) + } + // 通道名重复会被去重，故用不重复的 + tooMany = tooMany[:0] + for i := 0; i <= model.MaxScopeChannels; i++ { + tooMany = append(tooMany, "ch"+padInt(i)) + } + err := producer.Produce(ctx, model.ImeiQueueStartMessage{ + RunID: "bad-scope", + Scope: &model.SyncScope{Channels: tooMany}, + }) + if err == nil { + t.Fatal("超限 scope 应被拒绝") + } + if _, runErr := store.GetRunMeta(ctx, "bad-scope"); runErr == nil { + t.Fatal("非法 scope 不应创建 Run") + } +} + +// TestProduceIsIdempotentForSameRun 同一 RunID 重复投递不重复入队。 +func TestProduceIsIdempotentForSameRun(t *testing.T) { + ctx := context.Background() + producer, store := newProducer(t, []Card{ + {ID: 1, ICCID: "a", ChanKey: "ch", TempKey: "sim_yd_v5"}, + {ID: 2, ICCID: "b", ChanKey: "ch", TempKey: "sim_yd_v5"}, + }) + + for i := 0; i < 2; i++ { + if err := producer.Produce(ctx, model.ImeiQueueStartMessage{RunID: "dup"}); err != nil { + t.Fatal(err) + } + } + meta, err := store.GetRunMeta(ctx, "dup") + if err != nil { + t.Fatal(err) + } + if meta.ActiveCount != 2 { + t.Fatalf("activeCount = %d, want 2（重复投递不应翻倍）", meta.ActiveCount) + } +} + +// TestProduceSkipsFinishedRun 终态 Run 不被重投消息复活。 +func TestProduceSkipsFinishedRun(t *testing.T) { + ctx := context.Background() + producer, store := newProducer(t, []Card{{ID: 1, ICCID: "a", ChanKey: "ch", TempKey: "sim_yd_v5"}}) + + if err := producer.Produce(ctx, model.ImeiQueueStartMessage{RunID: "done"}); err != nil { + t.Fatal(err) + } + // 排空队列让 Run 进入终态：先领取并终结任务，再收尾。 + claims, err := store.ClaimRun(ctx, "done", "ch", time.Now(), time.Now().Add(time.Minute), 10) + if err != nil || len(claims) != 1 { + t.Fatalf("claims=%v err=%v", claims, err) + } + if _, err = store.FinalizeRunTasks(ctx, claims, []queue.TaskOutcome{{Success: true}}); err != nil { + t.Fatal(err) + } + if completed, err := store.CompleteRunIfIdle(ctx, "done"); err != nil || !completed { + t.Fatalf("complete run: completed=%v err=%v", completed, err) + } + + err = producer.Produce(ctx, model.ImeiQueueStartMessage{RunID: "done"}) + if !errors.Is(err, ErrRunFinished) { + t.Fatalf("终态 Run 重投应返回 ErrRunFinished，实际 %v", err) + } +} + +// TestProduceRejectsEmptyRunID 空 RunID 直接失败。 +func TestProduceRejectsEmptyRunID(t *testing.T) { + ctx := context.Background() + producer, _ := newProducer(t, []Card{{ID: 1, ICCID: "a", ChanKey: "ch", TempKey: "sim_yd_v5"}}) + if err := producer.Produce(ctx, model.ImeiQueueStartMessage{}); err == nil { + t.Fatal("空 RunID 应被拒绝") + } +} + +// TestScopeDigestIsStableAcrossOrdering 摘要对通道顺序不敏感， +// 使「同一批通道以不同顺序请求」得到同一个 digest。 +// +// 排序由 NormalizeScope 完成，所以必须走归一化后再算 digest +// （直接对未排序输入调 scopeDigest 得到不同结果属预期行为）。 +func TestScopeDigestIsStableAcrossOrdering(t *testing.T) { + a, err := model.NormalizeScope(&model.SyncScope{Channels: []string{"173", "189", "216"}}) + if err != nil { + t.Fatal(err) + } + b, err := model.NormalizeScope(&model.SyncScope{Channels: []string{"216", "173", "189"}}) + if err != nil { + t.Fatal(err) + } + if scopeDigest(a) != scopeDigest(b) { + t.Fatalf("digest 应对顺序不敏感: %q vs %q", scopeDigest(a), scopeDigest(b)) + } + if scopeDigest(a) == "" { + t.Fatal("非空 scope 的 digest 不应为空") + } + if scopeDigest(nil) != "" { + t.Fatal("全量同步的 digest 应为空") + } +} + +func padInt(value int) string { + if value == 0 { + return "0" + } + digits := make([]byte, 0, 8) + for value > 0 { + digits = append([]byte{byte('0' + value%10)}, digits...) + value /= 10 + } + return string(digits) +} diff --git a/internal/producer/monthlyusage/channel_producer.go b/internal/producer/monthlyusage/channel_producer.go new file mode 100644 index 0000000..98cf475 --- /dev/null +++ b/internal/producer/monthlyusage/channel_producer.go @@ -0,0 +1,493 @@ +package monthlyusage + +import ( + "context" + "errors" + "fmt" + "sort" + "sync" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +var errProducerDone = gerror.New("run production already closed") + +func producerStateError(state zset.ProducerState) error { + if state.Status == "completed" || state.Status == "completed_with_failures" { + return ErrRunFinished + } + if state.ProducerDone { + return errProducerDone + } + if state.Status != "producing" && state.Status != "running" { + return gerror.Newf("unexpected producer run status %q", state.Status) + } + return nil +} + +// produceChannels owns all per-attempt coordination. There is one worker per +// target, NOT a pool of four long-lived channel workers. SQL tokens and the +// cancellable admission gate never span sleeps or goroutine joins. +func (p *Producer) produceChannels(ctx context.Context, message model.TrafficUsageMonthlyStartMessage, scope *model.SyncScope, scanUpperID int64, startedAt time.Time, total *productionStats) error { + scanner := p.scanner.(channelScanner) + metadata, err := scanner.ListChannels(ctx) + if err != nil { + return gerror.Wrap(err, "initialize channel scan") + } + state, err := p.store.ReadProducerState(ctx, message.RunID, "") + if err != nil { + return err + } + if err = producerStateError(state); err != nil { + return err + } + targets := make(map[string]channelMetadata, len(metadata)) + known := make(map[string]bool, len(metadata)) + for _, channel := range metadata { + if channel.ChanKey == "" || !scope.Allows(channel.ChanKey) { + continue + } + if known[channel.ChanKey] { + return gerror.Newf("duplicate channel metadata %q", channel.ChanKey) + } + known[channel.ChanKey] = true + reason := "" + if channel.SyncEnabled { + reason = "sync_disabled" + } else if channel.TempKey == "" { + reason = "missing_channel_template" + } + if reason != "" { + total.skippedChannels++ + g.Log().Warningf(ctx, "event=monthly_usage_channel status=skipped runId=%s chanKey=%s reason=%s skipUnit=channel", message.RunID, channel.ChanKey, reason) + continue + } + targets[channel.ChanKey] = channel + } + if scope != nil { + for _, chanKey := range scope.Channels { + if !known[chanKey] { + total.skippedChannels++ + g.Log().Warningf(ctx, "event=monthly_usage_channel status=skipped runId=%s chanKey=%s reason=unknown_channel skipUnit=channel", message.RunID, chanKey) + } + } + } + g.Log().Infof(ctx, "event=monthly_usage_channel_scan status=started runId=%s targetChannels=%d scanUpperID=%d scanConcurrency=%d pageSize=%d skipUnit=channel", message.RunID, len(targets), scanUpperID, p.config.ScanConcurrency, p.config.PageSize) + keys := make([]string, 0, len(targets)) + for key := range targets { + keys = append(keys, key) + } + sort.Strings(keys) + finishObservation := p.startObservation(ctx, message.RunID, keys, startedAt) + observationSuccess := false + defer func() { finishObservation(observationSuccess) }() + var ( + workers sync.WaitGroup + slots = make(chan struct{}, p.config.ScanConcurrency) + gate = make(chan struct{}, 1) + metrics = newScanActivity() + results = make([]*channelScanStats, len(keys)) + ) + pool, hasPool := scanner.(interface { + poolWait(context.Context) (int64, int64) + }) + var poolCount, poolNanos int64 + if hasPool { + poolCount, poolNanos = pool.poolWait(ctx) + } + workCtx, cancel := context.WithCancelCause(ctx) + defer cancel(nil) + for i, key := range keys { + stats := &channelScanStats{chanKey: key, lastSummary: time.Now()} + results[i] = stats + workers.Add(1) + go func(channel channelMetadata, stats *channelScanStats) { + defer workers.Done() + scanErr := p.scanOneChannel(workCtx, scanner, message, channel, scanUpperID, startedAt, slots, gate, metrics, stats) + if scanErr != nil { + cancel(scanErr) + } + reason := stats.stopReason + if reason == "" { + reason = "scan_complete" + } + if scanErr != nil { + reason = "cancelled" + } + p.observe(channel.ChanKey, reason, 0, false, true) + stats.log(workCtx, message.RunID, "ended", startedAt, scanErr) + }(targets[key], stats) + } + workers.Wait() + var summary channelScanStats + for _, stats := range results { + total.pages += stats.scanPages + total.scanned += stats.eligible + total.eligible += stats.eligible + total.enqueued += stats.added + total.stoppedTasks += stats.rejected + if stats.eligible > 0 { + total.channels[stats.chanKey] = struct{}{} + } + summary.merge(stats) + } + metrics.log(ctx, message.RunID, &summary) + if hasPool { + count, nanos := pool.poolWait(ctx) + g.Log().Infof(ctx, "event=monthly_usage_scan_pool runId=%s sharedPoolWaitCountDelta=%d sharedPoolWaitMsDelta=%d attribution=shared_pool", message.RunID, count-poolCount, (nanos-poolNanos)/int64(time.Millisecond)) + } + observationSuccess = context.Cause(workCtx) == nil + return context.Cause(workCtx) +} + +func acquireContext(ctx context.Context, token chan struct{}) error { + select { + case <-ctx.Done(): + return ctx.Err() + case token <- struct{}{}: + if err := ctx.Err(); err != nil { + <-token + return err + } + return nil + } +} + +func (p *Producer) scanOneChannel(ctx context.Context, scanner channelScanner, message model.TrafficUsageMonthlyStartMessage, channel channelMetadata, scanUpperID int64, startedAt time.Time, slots, gate chan struct{}, metrics *scanActivity, stats *channelScanStats) error { + for stats.cursor < scanUpperID { + // Detect a blocked/closed run before querying, including empty channels. + p.observe(channel.ChanKey, "redis_read", 0, false, false) + state, err := p.store.ReadProducerState(ctx, message.RunID, channel.ChanKey) + if err != nil { + return err + } + if err = producerStateError(state); err != nil { + return err + } + if state.Blocked { + // No page was read: the unknown unscanned remainder has no card + // count. Never turn it into a fabricated success/failure total. + stats.stopReason = "blocked_unscanned_remainder" + return nil + } + p.observe(channel.ChanKey, "sql_slot", 0, false, false) + waitStart := time.Now() + if err = acquireContext(ctx, slots); err != nil { + return err + } + stats.slotWait += time.Since(waitStart) + metrics.begin() + p.observe(channel.ChanKey, "sql_query", 0, false, false) + queryStart := time.Now() + cards, scanErr := scanner.ScanChannel(ctx, channel.ChanKey, stats.cursor, p.config.PageSize) + elapsed := time.Since(queryStart) + metrics.end() + <-slots // release immediately after results have been materialized + stats.sqlTime += elapsed + stats.sqlLatency.observe(elapsed) + stats.scanPages++ + if scanErr != nil { + return gerror.Wrapf(scanErr, "scan channel %s after %d", channel.ChanKey, stats.cursor) + } + if len(cards) > p.config.PageSize { + return gerror.New("scanner exceeded page size") + } + if len(cards) == 0 { + stats.stopReason = "empty_result" + return nil + } + var ( + pageCursor = stats.cursor + overUpper bool + tasks = make([]zset.Task, 0, len(cards)) + ) + for i, card := range cards { + if card.ChanKey != channel.ChanKey || card.ID <= pageCursor { + return gerror.New("channel scanner returned unordered or foreign cards") + } + if card.ID > scanUpperID { + stats.overUpper += int64(len(cards) - i) + overUpper = true + break + } + pageCursor = card.ID + tasks = append(tasks, zset.Task{TaskID: zset.StableTaskID(message.RunID, message.Operation, card.ID), RunID: message.RunID, Operation: message.Operation, QueryMonth: message.QueryMonth, ChanKey: channel.ChanKey, TempKey: channel.TempKey, Operator: channel.Operator, CardID: card.ID, ICCID: card.ICCID, MSISDN: card.MSISDN, IMSI: card.IMSI}) + } + p.observe(channel.ChanKey, "page_ready", len(tasks), false, false) + stats.eligible += int64(len(tasks)) + blocked, err := p.admitChannelPage(ctx, message.RunID, tasks, startedAt, gate, stats) + if err != nil { + return err // page cursor is not committed, even after a partial write + } + if blocked { + stats.stopReason = "blocked" + return nil + } + stats.cursor = pageCursor + stats.progress(ctx, message.RunID, startedAt) + if overUpper || stats.cursor >= scanUpperID { + stats.stopReason = "upper_bound" + return nil + } + // Do not use a short page as EOF: query-time eligibility may change. + } + return ctx.Err() +} + +// admitChannelPage admits only new TaskIDs, in capacity-sized slices. Each +// worker retains at most this page. No capacity reservation survives the gate. +func (p *Producer) admitChannelPage(ctx context.Context, runID string, tasks []zset.Task, startedAt time.Time, gate chan struct{}, stats *channelScanStats) (bool, error) { + for len(tasks) > 0 { + p.observe(stats.chanKey, "admission_gate", len(tasks), false, false) + gateStart := time.Now() + if err := acquireContext(ctx, gate); err != nil { + return false, err + } + stats.gateWait += time.Since(gateStart) + blocked, waitKind, remaining, err := p.admitChannelSlice(ctx, runID, tasks, startedAt, stats) + <-gate + if err != nil || blocked { + return blocked, err + } + tasks = remaining + p.observe(stats.chanKey, "page_ready", len(tasks), false, false) + if waitKind != "" { + p.observe(stats.chanKey, waitKind+"_capacity", len(tasks), false, false) + waitStart := time.Now() + err = sleepContext(ctx, p.config.PollInterval) + if waitKind == "global" { + stats.globalWait += time.Since(waitStart) + } else { + stats.channelWait += time.Since(waitStart) + } + stats.progress(ctx, runID, startedAt) + if err != nil { + return false, err + } + } + } + return false, ctx.Err() +} + +// Caller holds the short admission gate; there is no sleep in this method. +func (p *Producer) admitChannelSlice(ctx context.Context, runID string, tasks []zset.Task, startedAt time.Time, stats *channelScanStats) (blocked bool, waitKind string, remaining []zset.Task, err error) { + p.observe(stats.chanKey, "redis_read", len(tasks), false, false) + readStart := time.Now() + state, err := p.store.ReadProducerState(ctx, runID, stats.chanKey) + if err != nil { + return false, "", tasks, err + } + if err = producerStateError(state); err != nil { + return false, "", tasks, err + } + existing, err := p.store.ExistingRunTasks(ctx, runID, tasks) + stats.stateTime += time.Since(readStart) + if err != nil { + return false, "", tasks, err + } + pending := tasks[:0] + for i, exists := range existing { + if exists { + stats.duplicates++ + } else { + pending = append(pending, tasks[i]) + } + } + tasks = pending + p.observe(stats.chanKey, "redis_read", len(tasks), false, false) + if state.Blocked { + stats.rejected += int64(len(tasks)) + return true, "", nil, nil + } + if len(tasks) == 0 { + return false, "", nil, nil + } + space := int64(len(tasks)) + if limit := p.config.GlobalHighWaterMark; limit > 0 { + space = min(space, limit-state.ActiveCount) // inflight is already included + if space <= 0 { + return false, "global", tasks, nil + } + } + if limit := p.config.ChannelHighWaterMark; limit > 0 { + key, keyErr := p.store.RoundActiveKey(runID, state.CurrentRound, stats.chanKey) + if keyErr != nil { + return false, "", tasks, keyErr + } + readStart = time.Now() + size, sizeErr := p.store.SizeKey(ctx, key) + stats.stateTime += time.Since(readStart) + if sizeErr != nil { + return false, "", tasks, sizeErr + } + space = min(space, limit-size) + if space <= 0 { + return false, "channel", tasks, nil + } + } + batch := tasks[:int(space)] + p.observe(stats.chanKey, "redis_write", len(tasks), false, false) + enqueueStart := time.Now() + added, err := p.store.EnqueueRunBatch(ctx, runID, stats.chanKey, batch, time.Now()) + stats.enqueueTime += time.Since(enqueueStart) + stats.enqueueBatches++ + if err != nil { + return false, "", tasks, err // uncertain responses require idempotent rescan + } + if added < 0 || added > len(batch) { + return false, "", tasks, gerror.New("invalid enqueue count") + } + stats.added += int64(added) + p.observe(stats.chanKey, "redis_write", len(tasks)-len(batch), added > 0, false) + if added > 0 && !stats.firstEnqueueRecorded { + stats.firstEnqueueRecorded = true + stats.firstEnqueue = time.Since(startedAt) + stats.log(ctx, runID, "first_enqueue", startedAt, nil) + } + if added < len(batch) { + readStart = time.Now() + confirmed, checkErr := p.store.ExistingRunTasks(ctx, runID, batch) + if checkErr != nil { + return false, "", tasks, checkErr + } + state, checkErr = p.store.ReadProducerState(ctx, runID, stats.chanKey) + stats.stateTime += time.Since(readStart) + if checkErr != nil { + return false, "", tasks, checkErr + } + if checkErr = producerStateError(state); checkErr != nil { + return false, "", tasks, checkErr + } + present := 0 + for _, exists := range confirmed { + if exists { + present++ + } + } + if present < added { + return false, "", tasks, gerror.New("enqueue count exceeds confirmed task states") + } + stats.duplicates += int64(present - added) + if state.Blocked { + stats.rejected += int64(len(tasks) - present) + return true, "", nil, nil + } + if present != len(batch) { + return false, "", tasks, gerror.Newf("unexplained enqueue rejection: added=%d submitted=%d confirmed=%d", added, len(batch), present) + } + } + return false, "", tasks[len(batch):], nil +} + +// Retained only to keep errors.Is semantics visible at the call site. +func isProductionClosed(err error) bool { return errors.Is(err, errProducerDone) } + +func durationMS(d time.Duration) float64 { return float64(d) / float64(time.Millisecond) } + +// latencyHistogram is fixed-size. Reported percentiles are bucket upper bounds, +// never exact samples or a server-side SLA; the last bucket is unbounded. +var latencyBounds = [...]time.Duration{time.Millisecond, 2 * time.Millisecond, 5 * time.Millisecond, 10 * time.Millisecond, 20 * time.Millisecond, 50 * time.Millisecond, 100 * time.Millisecond, 200 * time.Millisecond, 500 * time.Millisecond, time.Second, 2 * time.Second, 5 * time.Second, 10 * time.Second, 30 * time.Second, time.Minute} + +type latencyHistogram struct{ counts [16]int64 } + +func (h *latencyHistogram) observe(d time.Duration) { + i := 0 + for i < len(latencyBounds) && d > latencyBounds[i] { + i++ + } + h.counts[i]++ +} +func (h *latencyHistogram) percentile(percent int64) string { + var total, cumulative int64 + for _, count := range h.counts { + total += count + } + if total == 0 { + return "n/a" + } + target := (total*percent + 99) / 100 + for i, count := range h.counts { + cumulative += count + if cumulative >= target { + if i == len(latencyBounds) { + return ">60000" + } + return fmt.Sprintf("%.0f", durationMS(latencyBounds[i])) + } + } + return "n/a" +} + +type channelScanStats struct { + firstEnqueueRecorded bool + chanKey, stopReason string + cursor, scanPages, eligible, overUpper, enqueueBatches, added, duplicates, rejected int64 + sqlTime, slotWait, gateWait, globalWait, channelWait, stateTime, enqueueTime, firstEnqueue time.Duration + sqlLatency latencyHistogram + lastSummary time.Time +} + +func (s *channelScanStats) log(ctx context.Context, runID, status string, startedAt time.Time, err error) { + errorText := "" + if err != nil { + errorText = err.Error() + } + g.Log().Infof(ctx, "event=monthly_usage_channel_scan status=%s runId=%s chanKey=%s cursor=%d scanPages=%d eligible=%d overUpper=%d enqueueBatches=%d added=%d duplicates=%d rejected=%d sqlMs=%.3f sqlP50UpperMs=%s sqlP95UpperMs=%s slotWaitMs=%.3f admissionWaitMs=%.3f globalCapacityWaitMs=%.3f channelCapacityWaitMs=%.3f stateReadMs=%.3f enqueueMs=%.3f firstEnqueueMs=%.3f wallMs=%d stopReason=%s error=%q", status, runID, s.chanKey, s.cursor, s.scanPages, s.eligible, s.overUpper, s.enqueueBatches, s.added, s.duplicates, s.rejected, durationMS(s.sqlTime), s.sqlLatency.percentile(50), s.sqlLatency.percentile(95), durationMS(s.slotWait), durationMS(s.gateWait), durationMS(s.globalWait), durationMS(s.channelWait), durationMS(s.stateTime), durationMS(s.enqueueTime), durationMS(s.firstEnqueue), time.Since(startedAt).Milliseconds(), s.stopReason, errorText) +} +func (s *channelScanStats) progress(ctx context.Context, runID string, startedAt time.Time) { + if time.Since(s.lastSummary) >= time.Minute { + s.lastSummary = time.Now() + s.log(ctx, runID, "progress", startedAt, nil) + } +} +func (s *channelScanStats) merge(other *channelScanStats) { + s.scanPages += other.scanPages + s.eligible += other.eligible + s.overUpper += other.overUpper + s.enqueueBatches += other.enqueueBatches + s.added += other.added + s.duplicates += other.duplicates + s.rejected += other.rejected + s.sqlTime += other.sqlTime + s.slotWait += other.slotWait + s.gateWait += other.gateWait + s.globalWait += other.globalWait + s.channelWait += other.channelWait + s.stateTime += other.stateTime + s.enqueueTime += other.enqueueTime + for i, count := range other.sqlLatency.counts { + s.sqlLatency.counts[i] += count + } +} + +type scanActivity struct { + mu sync.Mutex + started, changed time.Time + running, peak int + area time.Duration +} + +func newScanActivity() *scanActivity { + now := time.Now() + return &scanActivity{started: now, changed: now} +} +func (s *scanActivity) change(delta int) { + s.mu.Lock() + defer s.mu.Unlock() + now := time.Now() + s.area += time.Duration(s.running) * now.Sub(s.changed) + s.changed = now + s.running += delta + s.peak = max(s.peak, s.running) +} +func (s *scanActivity) begin() { s.change(1) } +func (s *scanActivity) end() { s.change(-1) } +func (s *scanActivity) log(ctx context.Context, runID string, stats *channelScanStats) { + wall := time.Since(s.started) + g.Log().Infof(ctx, "event=monthly_usage_scan_summary runId=%s wallMs=%d sqlInFlightPeak=%d sqlInFlightAvg=%.4f scanPages=%d enqueueBatches=%d eligible=%d overUpper=%d added=%d duplicates=%d rejected=%d sqlCumulativeMs=%.3f sqlP50UpperMs=%s sqlP95UpperMs=%s slotWaitCumulativeMs=%.3f admissionWaitCumulativeMs=%.3f globalCapacityWaitCumulativeMs=%.3f channelCapacityWaitCumulativeMs=%.3f stateReadCumulativeMs=%.3f enqueueCumulativeMs=%.3f", runID, wall.Milliseconds(), s.peak, float64(s.area)/float64(wall), stats.scanPages, stats.enqueueBatches, stats.eligible, stats.overUpper, stats.added, stats.duplicates, stats.rejected, durationMS(stats.sqlTime), stats.sqlLatency.percentile(50), stats.sqlLatency.percentile(95), durationMS(stats.slotWait), durationMS(stats.gateWait), durationMS(stats.globalWait), durationMS(stats.channelWait), durationMS(stats.stateTime), durationMS(stats.enqueueTime)) +} diff --git a/internal/producer/monthlyusage/channel_producer_test.go b/internal/producer/monthlyusage/channel_producer_test.go new file mode 100644 index 0000000..68cfe38 --- /dev/null +++ b/internal/producer/monthlyusage/channel_producer_test.go @@ -0,0 +1,522 @@ +package monthlyusage + +import ( + "context" + "errors" + "fmt" + "reflect" + "sort" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" + "github.com/gogf/gf/v2/container/gvar" +) + +// datasetScanner applies the SQL eligibility predicate before LIMIT; metadata +// eligibility is applied by the producer. Optional hooks simulate SQL latency, +// live additions and failures without accessing an application database. +type datasetScanner struct { + cards []Card + channels []channelMetadata + upper int64 + hook func(context.Context, string, int64) error + queries atomic.Int32 + lists atomic.Int32 + inflight atomic.Int32 + peak atomic.Int32 +} + +func (s *datasetScanner) SnapshotMaxID(context.Context) (int64, error) { return s.upper, nil } +func (s *datasetScanner) ListChannels(context.Context) ([]channelMetadata, error) { + s.lists.Add(1) + return s.channels, nil +} +func (s *datasetScanner) Scan(ctx context.Context, last, upper int64, size int) ([]Card, error) { + var rows []Card + for _, card := range s.cards { + if card.ID > last && card.ID <= upper { + for _, channel := range s.channels { + if channel.ChanKey == card.ChanKey { + card.TempKey = channel.TempKey + card.SyncEnabled = channel.SyncEnabled + card.Operator = channel.Operator + } + } + rows = append(rows, card) + if len(rows) == size { + break + } + } + } + return rows, nil +} +func (s *datasetScanner) ScanChannel(ctx context.Context, key string, last int64, size int) ([]Card, error) { + s.queries.Add(1) + running := s.inflight.Add(1) + defer s.inflight.Add(-1) + for { + old := s.peak.Load() + if running <= old || s.peak.CompareAndSwap(old, running) { + break + } + } + if s.hook != nil { + if err := s.hook(ctx, key, last); err != nil { + return nil, err + } + } + var rows []Card + for _, card := range s.cards { + if card.ChanKey == key && card.ID > last { + rows = append(rows, card) + if len(rows) == size { + break + } + } + } + return rows, nil +} +func scannerFor(cards []Card, upper int64) *datasetScanner { + keys := map[string]bool{} + s := &datasetScanner{cards: cards, upper: upper} + for _, card := range cards { + if card.ChanKey != "" && !keys[card.ChanKey] { + keys[card.ChanKey] = true + s.channels = append(s.channels, channelMetadata{ChanKey: card.ChanKey, TempKey: "sim_yd_v5", Operator: 1}) + } + } + return s +} +func testStore(t *testing.T) *zset.Store { + t.Helper() + store, err := zset.New(testutil.Redis(t), "flow:") + if err != nil { + t.Fatal(err) + } + return store +} +func testProducer(t *testing.T, scanner CardScanner, store *zset.Store, config Config) *Producer { + t.Helper() + config.RoundRetryEnabled = true + p, err := New(scanner, store, config) + if err != nil { + t.Fatal(err) + } + return p +} +func contextForTest(t *testing.T) context.Context { + t.Helper() + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + t.Cleanup(cancel) + return ctx +} +func taskIDs(t *testing.T, store *zset.Store, run string) []string { + t.Helper() + key, _ := store.TaskStateKey(run) + result, err := store.RedisClient().Do(context.Background(), "HKEYS", key) + if err != nil { + t.Fatal(err) + } + ids := result.Strings() + sort.Strings(ids) + return ids +} +func requireMeta(t *testing.T, store *zset.Store, run string, done bool, active int64) { + t.Helper() + meta, err := store.GetRunMeta(context.Background(), run) + if err != nil { + t.Fatal(err) + } + if meta.ProducerDone != done || meta.ActiveCount != active { + t.Fatalf("meta=%+v, want done=%t active=%d", meta, done, active) + } +} +func waitUntil(t *testing.T, ctx context.Context, condition func() bool) { + t.Helper() + for !condition() { + select { + case <-ctx.Done(): + t.Fatal(ctx.Err()) + case <-time.After(time.Millisecond): + } + } +} + +func TestChannelModeSettings(t *testing.T) { + store, _ := zset.New(&producerRedis{}, "flow:") + p, err := New(&fakeScanner{}, store, Config{}) + if err != nil || p.config.ScanConcurrency != 4 { + t.Fatalf("defaults: p=%+v err=%v", p, err) + } + for _, config := range []Config{{ScanConcurrency: -1}} { + if _, err = New(&fakeScanner{}, store, config); err == nil { + t.Fatalf("accepted invalid config/capability %+v", config) + } + } +} + +func TestChannelScanFiniteMemoryBoundary(t *testing.T) { + for _, test := range []struct { + name string + upper int64 + cards []Card + ids []int64 + queries int32 + }{ + {"zero upper", 0, []Card{{ID: 1, ChanKey: "a"}}, nil, 0}, + {"exact page", 4, []Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "a"}, {ID: 3, ChanKey: "a"}, {ID: 4, ChanKey: "a"}}, []int64{1, 2, 3, 4}, 2}, + {"tail then empty", 9, []Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "a"}, {ID: 3, ChanKey: "a"}}, []int64{1, 2, 3}, 3}, + {"first outside", 3, []Card{{ID: 4, ChanKey: "a"}, {ID: 5, ChanKey: "a"}}, nil, 1}, + {"mixed final page", 3, []Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "a"}, {ID: 3, ChanKey: "a"}, {ID: 4, ChanKey: "a"}, {ID: 5, ChanKey: "a"}}, []int64{1, 2, 3}, 2}, + {"empty channel", 9, nil, nil, 1}, + } { + t.Run(test.name, func(t *testing.T) { + store := testStore(t) + scanner := scannerFor(test.cards, test.upper) + if len(scanner.channels) == 0 { + scanner.channels = []channelMetadata{{ChanKey: "a", TempKey: "sim_yd_v5"}} + } + p := testProducer(t, scanner, store, Config{PageSize: 2}) + if err := p.Produce(contextForTest(t), scopedMessage("boundary")); err != nil { + t.Fatal(err) + } + var want []string + for _, id := range test.ids { + want = append(want, zset.StableTaskID("boundary", operation, id)) + } + sort.Strings(want) + got := taskIDs(t, store, "boundary") + if len(got) != len(want) || (len(want) > 0 && !reflect.DeepEqual(got, want)) { + t.Fatalf("got=%v want=%v", got, want) + } + if scanner.queries.Load() != test.queries { + t.Fatalf("queries=%d want=%d", scanner.queries.Load(), test.queries) + } + requireMeta(t, store, "boundary", true, int64(len(want))) + }) + } +} + +func TestChannelScanLiveHigherIDsDoNotExtendRun(t *testing.T) { + store := testStore(t) + scanner := scannerFor([]Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "a"}, {ID: 3, ChanKey: "a"}}, 3) + scanner.hook = func(_ context.Context, _ string, last int64) error { + if last > 0 { + scanner.cards = append(scanner.cards, Card{ID: 100 + last, ChanKey: "a"}) + } + return nil + } + p := testProducer(t, scanner, store, Config{PageSize: 2}) + if err := p.Produce(contextForTest(t), scopedMessage("live")); err != nil { + t.Fatal(err) + } + if len(taskIDs(t, store, "live")) != 3 || scanner.queries.Load() != 2 { + t.Fatal("higher IDs extended finite range") + } +} + +func TestChannelScanSerialTaskIDSetParity(t *testing.T) { + for _, scope := range [][]string{nil, {"a", "b"}, {"disabled", "unknown"}} { + t.Run(fmt.Sprint(scope), func(t *testing.T) { + store := testStore(t) + cards := []Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "b"}, {ID: 3, ChanKey: "a"}, {ID: 4, ChanKey: "disabled"}, {ID: 5, ChanKey: "no-template"}, {ID: 6, ChanKey: "unknown"}, {ID: 7, ChanKey: "b"}, {ID: 8, ChanKey: "a"}} + var sets [][]string + for _, mode := range []string{"serial", "channel"} { + scanner := scannerFor(cards, 7) + scanner.channels = []channelMetadata{{ChanKey: "a", TempKey: "sim_yd_v5", Operator: 1}, {ChanKey: "b", TempKey: "sim_dx_v1", Operator: 2}, {ChanKey: "disabled", TempKey: "sim_yd_v5", SyncEnabled: true}, {ChanKey: "no-template"}} + p := testProducer(t, scanner, store, Config{PageSize: 2}) + run := "parity-" + mode + if err := p.Produce(contextForTest(t), scopedMessage(run, scope...)); err != nil { + t.Fatal(err) + } + ids := taskIDs(t, store, run) + for i, id := range ids { + ids[i] = strings.TrimPrefix(id, run+":") + } + sets = append(sets, ids) + if mode == "channel" && scanner.lists.Load() != 1 { + t.Fatal("metadata not loaded once") + } + _, _ = store.RedisClient().Do(context.Background(), "SREM", store.ActiveRunsKey(), run) + } + if !reflect.DeepEqual(sets[0], sets[1]) { + t.Fatalf("serial=%v channel=%v", sets[0], sets[1]) + } + }) + } +} + +func TestChannelSQLSlotsReleasedWhileChannelsWait(t *testing.T) { + store := testStore(t) + ctx := contextForTest(t) + var cards []Card + for i := 0; i < 9; i++ { + cards = append(cards, Card{ID: int64(i + 1), ChanKey: fmt.Sprint(i)}, Card{ID: int64(i + 10), ChanKey: fmt.Sprint(i)}) + } + sort.Slice(cards, func(i, j int) bool { return cards[i].ID < cards[j].ID }) + scanner := scannerFor(cards, 100) + scanner.hook = func(ctx context.Context, _ string, _ int64) error { return sleepContext(ctx, 5*time.Millisecond) } + p := testProducer(t, scanner, store, Config{PageSize: 2, ScanConcurrency: 4, ChannelHighWaterMark: 1, GlobalHighWaterMark: 100, PollInterval: time.Millisecond}) + result := make(chan error, 1) + go func() { result <- p.Produce(ctx, scopedMessage("isolation")) }() + waitUntil(t, ctx, func() bool { return len(taskIDs(t, store, "isolation")) == 9 }) + if peak := scanner.peak.Load(); peak > 4 || peak < 2 { + t.Fatalf("SQL peak=%d", peak) + } + // All nine channels got a first task despite the first four waiting at + // channel capacity. SQL slots are free while every channel is paused. + if scanner.inflight.Load() != 0 { + t.Fatal("SQL tokens held during channel backpressure") + } + for _, channel := range scanner.channels { + if _, err := store.BlockRunScoped(ctx, "isolation", channel.ChanKey); err != nil { + t.Fatal(err) + } + } + if err := <-result; err != nil { + t.Fatal(err) + } + meta, err := store.GetRunMeta(ctx, "isolation") + if err != nil || !meta.ProducerDone { + t.Fatalf("meta=%+v err=%v", meta, err) + } +} + +func TestChannelRemainingCapacityAndDuplicateAdmission(t *testing.T) { + store := testStore(t) + ctx := contextForTest(t) + scanner := scannerFor(nil, 0) + p := testProducer(t, scanner, store, Config{GlobalHighWaterMark: 3, ChannelHighWaterMark: 2, PollInterval: time.Millisecond}) + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "capacity", Status: "producing"}); err != nil { + t.Fatal(err) + } + // Existing success and waiting states still deduplicate and consume no new + // capacity. Active includes leases: inflight must not be added again. + stateKey, _ := store.TaskStateKey("capacity") + metaKey, _ := store.RunMetaKey("capacity") + activeKey, _ := store.RoundActiveKey("capacity", 1, "a") + _, _ = store.RedisClient().Do(ctx, "HSET", stateKey, "one", `{"status":"success"}`, "two", `{"status":"waiting"}`) + _, _ = store.RedisClient().Do(ctx, "HSET", metaKey, "activeCount", 2, "inflightCount", 2) + stats := &channelScanStats{chanKey: "a", lastSummary: time.Now()} + tasks := []zset.Task{{TaskID: "one", RunID: "capacity", ChanKey: "a"}, {TaskID: "two", RunID: "capacity", ChanKey: "a"}, {TaskID: "three", RunID: "capacity", ChanKey: "a"}, {TaskID: "four", RunID: "capacity", ChanKey: "a"}} + gate := make(chan struct{}, 1) + result := make(chan error, 1) + go func() { _, err := p.admitChannelPage(ctx, "capacity", tasks, time.Now(), gate, stats); result <- err }() + waitUntil(t, ctx, func() bool { meta, e := store.GetRunMeta(ctx, "capacity"); return e == nil && meta.ActiveCount == 3 }) + if size, err := store.SizeKey(ctx, activeKey); err != nil || size != 1 { + t.Fatalf("initial slice size=%d err=%v", size, err) + } + _, _ = store.RedisClient().Do(ctx, "HSET", metaKey, "activeCount", 0, "inflightCount", 0) + if err := <-result; err != nil { + t.Fatal(err) + } + if stats.duplicates != 2 || stats.added != 2 || stats.enqueueBatches != 2 || stats.globalWait == 0 { + t.Fatalf("stats=%+v", stats) + } +} + +// faultRedis forwards to a real isolated Redis, optionally returning a lost +// response after Lua succeeded. Never mocks the Lua semantics under test. +type faultRedis struct { + zset.RedisClient + failEnqueue atomic.Int32 + failRead atomic.Bool + failRenew atomic.Bool + lostResponse bool + reject bool +} + +func (r *faultRedis) Do(ctx context.Context, command string, args ...any) (*gvar.Var, error) { + if r.failRead.Load() && command == "HMGET" { + return nil, errors.New("Redis read failure") + } + return r.RedisClient.Do(ctx, command, args...) +} +func (r *faultRedis) Eval(ctx context.Context, script string, n int64, keys []string, args []any) (*gvar.Var, error) { + if r.failRenew.Load() && strings.Contains(script, "PEXPIRE") && strings.Contains(script, "owner ~=") { + return gvar.New(0), nil + } + if n == 4 && strings.Contains(script, "HEXISTS") { + if r.reject { + return gvar.New(0), nil + } + if r.failEnqueue.CompareAndSwap(1, 0) { + if r.lostResponse { + if _, err := r.RedisClient.Eval(ctx, script, n, keys, args); err != nil { + return nil, err + } + } + return nil, errors.New("enqueue response unavailable") + } + } + return r.RedisClient.Eval(ctx, script, n, keys, args) +} + +func TestChannelRescanPartialAndUncertainWrites(t *testing.T) { + for _, uncertain := range []bool{false, true} { + t.Run(fmt.Sprint(uncertain), func(t *testing.T) { + client := &faultRedis{RedisClient: testutil.Redis(t), lostResponse: uncertain} + store, _ := zset.New(client, "flow:") + scanner := scannerFor([]Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "a"}, {ID: 3, ChanKey: "a"}, {ID: 4, ChanKey: "a"}}, 4) + // Fail the second page: first page must survive; uncertain success is + // confirmed only on a same-Run rescan, with stable IDs and no double count. + scanner.hook = func(_ context.Context, _ string, last int64) error { + if last == 2 { + client.failEnqueue.Store(1) + } + return nil + } + p := testProducer(t, scanner, store, Config{PageSize: 2}) + if err := p.Produce(contextForTest(t), scopedMessage("rescan")); err == nil { + t.Fatal("expected enqueue failure") + } + active := int64(2) + if uncertain { + active = 4 + } + requireMeta(t, store, "rescan", false, active) + scanner.hook = nil + if err := p.Produce(contextForTest(t), scopedMessage("rescan")); err != nil { + t.Fatal(err) + } + requireMeta(t, store, "rescan", true, 4) + if len(taskIDs(t, store, "rescan")) != 4 { + t.Fatal("wrong stable task set") + } + before := scanner.queries.Load() + if err := p.Produce(contextForTest(t), scopedMessage("rescan")); err != nil { + t.Fatal(err) + } + if before != scanner.queries.Load() { + t.Fatal("producerDone redelivery rescanned cards") + } + }) + } +} + +func TestChannelFailureCancelsAllWorkersWithoutProducerDone(t *testing.T) { + for _, kind := range []string{"sql", "redis", "unexplained_rejection", "lease"} { + t.Run(kind, func(t *testing.T) { + client := &faultRedis{RedisClient: testutil.Redis(t)} + store, _ := zset.New(client, "flow:") + cards := []Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "b"}, {ID: 3, ChanKey: "c"}} + scanner := scannerFor(cards, 10) + allStarted := make(chan struct{}) + var started atomic.Int32 + var once sync.Once + scanner.hook = func(ctx context.Context, key string, _ int64) error { + if started.Add(1) >= 3 { + once.Do(func() { close(allStarted) }) + } + select { + case <-allStarted: + case <-ctx.Done(): + return ctx.Err() + } + if kind == "lease" { + client.failRenew.Store(true) + <-ctx.Done() + return ctx.Err() + } + if key == "a" { + switch kind { + case "sql": + return errors.New("SQL failed") + case "redis": + client.failRead.Store(true) + case "unexplained_rejection": + client.reject = true + } + return nil + } + <-ctx.Done() + return ctx.Err() + } + p := testProducer(t, scanner, store, Config{PageSize: 2, RunLockTTL: 90 * time.Millisecond}) + if err := p.Produce(contextForTest(t), scopedMessage("failure")); err == nil { + t.Fatal("expected fatal failure") + } + if scanner.inflight.Load() != 0 { + t.Fatal("scanner goroutines leaked") + } + meta, err := store.GetRunMeta(context.Background(), "failure") + if err != nil || meta.ProducerDone { + t.Fatalf("partial scan marked done: %+v err=%v", meta, err) + } + }) + } +} + +func TestChannelScanStateStopsAndNoTarget(t *testing.T) { + for _, kind := range []string{"terminal", "producerDone", "blocked", "missing", "no-target", "no-cards"} { + t.Run(kind, func(t *testing.T) { + store := testStore(t) + scanner := scannerFor([]Card{{ID: 1, ChanKey: "a"}}, 2) + if kind == "no-target" { + scanner.channels[0].SyncEnabled = true + } + if kind == "no-cards" { + scanner.cards = nil + } + p := testProducer(t, scanner, store, Config{}) + if kind != "no-target" && kind != "no-cards" { + if err := store.CreateRun(context.Background(), zset.RunMeta{RunID: "stop", Status: "producing"}); err != nil { + t.Fatal(err) + } + key, _ := store.RunMetaKey("stop") + switch kind { + case "terminal": + _, _ = store.RedisClient().Do(context.Background(), "HSET", key, "status", "completed") + case "producerDone": + if err := store.MarkProducerDone(context.Background(), "stop"); err != nil { + t.Fatal(err) + } + case "blocked": + _, _ = store.BlockRunScoped(context.Background(), "stop", "a") + case "missing": + scanner.hook = func(ctx context.Context, _ string, _ int64) error { + _, err := store.RedisClient().Do(ctx, "DEL", key) + return err + } + } + } + err := p.Produce(contextForTest(t), scopedMessage("stop")) + if kind == "terminal" { + if !errors.Is(err, ErrRunFinished) { + t.Fatalf("err=%v", err) + } + } else if kind == "missing" { + if !errors.Is(err, zset.ErrRunNotFound) { + t.Fatalf("err=%v", err) + } + } else if err != nil { + t.Fatal(err) + } + if len(taskIDs(t, store, "stop")) != 0 { + t.Fatal("stopped/empty run enqueued tasks") + } + }) + } +} + +// Omitted retry/scan-mode settings must still use the channel Run producer. +func TestDefaultProducerUsesChannelRun(t *testing.T) { + store := testStore(t) + scanner := scannerFor([]Card{{ID: 1, ChanKey: "a"}}, 1) + p, err := New(scanner, store, Config{}) + if err != nil { + t.Fatal(err) + } + if err = p.Produce(contextForTest(t), scopedMessage("default-channel")); err != nil { + t.Fatal(err) + } + if scanner.queries.Load() == 0 || scanner.lists.Load() != 1 { + t.Fatal("default did not use channel scan") + } + requireMeta(t, store, "default-channel", true, 1) +} diff --git a/internal/producer/monthlyusage/channel_scanner.go b/internal/producer/monthlyusage/channel_scanner.go new file mode 100644 index 0000000..9a11dfa --- /dev/null +++ b/internal/producer/monthlyusage/channel_scanner.go @@ -0,0 +1,78 @@ +package monthlyusage + +import ( + "context" + + "github.com/gogf/gf/v2/errors/gerror" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +// channelScanner augments, rather than replaces, the legacy scanner contract. +// ListChannels is called once before any workers start. ScanChannel must not +// mutate the legacy DatabaseScanner.channelMetadata cache. +type channelScanner interface { + ListChannels(context.Context) ([]channelMetadata, error) + ScanChannel(context.Context, string, int64, int) ([]Card, error) +} + +// This exact MySQL query shape is a release gate. Do not silently remove hints, +// add an SQL upper bound, or move eligibility filtering after LIMIT. +const channelPageSQL = `SELECT c.id, c.iccid, c.msisdn, c.imsi, c.chan_key +FROM t_sim_card_info AS c FORCE INDEX (idx_card_chan_key) +STRAIGHT_JOIN sys_dept AS d + ON c.dept_id = d.dept_id + AND c.tenant_id = d.tenant_id +WHERE c.chan_key = ? + AND c.id > ? + AND c.iccid <> '' + AND c.current_package_code <> '' + AND d.deleted_at IS NULL +ORDER BY c.id ASC +LIMIT ?` + +func (s *DatabaseScanner) ListChannels(ctx context.Context) ([]channelMetadata, error) { + var metadata []channelMetadata + if err := dao.TSimCardInfo.DB().Model("t_sim_channels").Ctx(ctx).Fields("chan_key,temp_key,sync_enabled,operator").Scan(&metadata); err != nil { + return nil, gerror.Wrap(err, "list channel metadata") + } + return metadata, nil +} + +func (s *DatabaseScanner) ScanChannel(ctx context.Context, chanKey string, lastID int64, pageSize int) ([]Card, error) { + if chanKey == "" || lastID < 0 || pageSize <= 0 { + return nil, gerror.New("invalid channel pagination parameters") + } + var rows []entity.TSimCardInfo + // Use the card DAO's database group, not a separately configured connection. + if err := dao.TSimCardInfo.DB().Raw(channelPageSQL, chanKey, lastID, pageSize).Ctx(ctx).Scan(&rows); err != nil { + return nil, gerror.Wrap(err, "scan channel cards") + } + cards := make([]Card, 0, len(rows)) + for _, row := range rows { + cards = append(cards, Card{ID: row.Id, ICCID: row.Iccid, MSISDN: row.Msisdn, IMSI: row.Imsi, ChanKey: row.ChanKey}) + } + return cards, nil +} + +// Pool counters are shared with other worker traffic. Their deltas are direct +// observations, not an attribution of all connection waits to this producer. +func (*DatabaseScanner) poolWait(ctx context.Context) (int64, int64) { + var count, nanos int64 + for _, item := range dao.TSimCardInfo.DB().Stats(ctx) { + stats := item.Stats() + count += stats.WaitCount + nanos += int64(stats.WaitDuration) + } + return count, nanos +} + +// ValidateScanSettings is strict for values read from configuration. The Go +// constructor first fills omitted zero values; explicit YAML zero is rejected. +func ValidateScanSettings(concurrency int) error { + if concurrency <= 0 { + return gerror.New("trafficUsageFullSync.producer.scanConcurrency must be positive") + } + return nil +} diff --git a/internal/producer/monthlyusage/channel_scanner_test.go b/internal/producer/monthlyusage/channel_scanner_test.go new file mode 100644 index 0000000..82678f9 --- /dev/null +++ b/internal/producer/monthlyusage/channel_scanner_test.go @@ -0,0 +1,122 @@ +package monthlyusage + +import ( + "context" + "reflect" + "sort" + "strings" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + _ "github.com/gogf/gf/contrib/drivers/sqlite/v2" + "github.com/gogf/gf/v2/database/gdb" +) + +// SQLite runs the SAME predicate and ordering with only the two MySQL hints +// removed. This checks full eligibility/TaskID parity, not MySQL access plans. +// Production EXPLAIN must still validate the unmodified channelPageSQL. +type sqliteChannelScanner struct{ DatabaseScanner } + +func (s *sqliteChannelScanner) ScanChannel(ctx context.Context, key string, last int64, size int) ([]Card, error) { + query := strings.ReplaceAll(channelPageSQL, "FORCE INDEX (idx_card_chan_key)", "") + query = strings.ReplaceAll(query, "STRAIGHT_JOIN", "INNER JOIN") + var rows []struct { + ID int64 `orm:"id"` + ICCID string `orm:"iccid"` + MSISDN string `orm:"msisdn"` + IMSI string `orm:"imsi"` + ChanKey string `orm:"chan_key"` + } + if err := dao.TSimCardInfo.DB().Raw(query, key, last, size).Ctx(ctx).Scan(&rows); err != nil { + return nil, err + } + cards := make([]Card, 0, len(rows)) + for _, row := range rows { + cards = append(cards, Card{ID: row.ID, ICCID: row.ICCID, MSISDN: row.MSISDN, IMSI: row.IMSI, ChanKey: row.ChanKey}) + } + return cards, nil +} + +func TestDatabaseQualificationAndCompleteTaskIDParity(t *testing.T) { + // No application DB/config: isolated SQLite file and loopback Redis only. + if err := gdb.SetConfig(gdb.Config{"default": gdb.ConfigGroup{{Type: "sqlite", Link: "sqlite::@file(" + t.TempDir() + "/cards.db)"}}}); err != nil { + t.Fatal(err) + } + ctx := contextForTest(t) + db := dao.TSimCardInfo.DB() + t.Cleanup(func() { _ = db.Close(context.Background()) }) + for _, sql := range []string{ + `CREATE TABLE sys_dept (dept_id INTEGER PRIMARY KEY, tenant_id TEXT, deleted_at DATETIME)`, + `CREATE TABLE t_sim_card_info (id INTEGER PRIMARY KEY, iccid TEXT, msisdn TEXT, imsi TEXT, chan_key TEXT, current_package_code TEXT, dept_id INTEGER, tenant_id TEXT)`, + `CREATE INDEX idx_card_chan_key ON t_sim_card_info(chan_key)`, + `CREATE TABLE t_sim_channels (chan_key TEXT, temp_key TEXT, sync_enabled INTEGER, operator INTEGER)`, + `INSERT INTO sys_dept VALUES (1,'t1',NULL),(2,'t2',NULL),(3,'t1','2026-01-01')`, + `INSERT INTO t_sim_channels VALUES ('a','sim_yd_v5',0,1),('b','sim_dx_v1',0,2),('disabled','sim_yd_v5',1,1),('no-template','',0,1)`, + `INSERT INTO t_sim_card_info VALUES + (1,'i1','m1','s1','a','pkg',1,'t1'), + (2,'i2','m2','s2','b','pkg',2,'t2'), + (3,'i3','','','a','pkg',1,'t1'), + (4,'','','','a','pkg',1,'t1'), + (5,'i5','','','a','',1,'t1'), + (6,'i6','','','a','pkg',1,'wrong-tenant'), + (7,'i7','','','a','pkg',3,'t1'), + (8,'i8','','','b','pkg',99,'t2'), + (9,'i9','','','disabled','pkg',1,'t1'), + (10,'i10','','','no-template','pkg',1,'t1'), + (11,'i11','','','unknown','pkg',1,'t1'), + (12,'i12','','','','pkg',1,'t1'), + (13,'i13','','','b','pkg',2,'t2')`, + } { + if _, err := db.Exec(ctx, sql); err != nil { + t.Fatal(err) + } + } + store := testStore(t) + for _, scope := range [][]string{nil, {"a"}, {"b", "disabled", "no-template", "unknown"}, {"unknown"}} { + var sets [][]string + for _, mode := range []string{"serial", "channel"} { + scanner := &sqliteChannelScanner{} + p := testProducer(t, scanner, store, Config{PageSize: 2}) + run := "db-" + mode + if err := p.Produce(ctx, scopedMessage(run, scope...)); err != nil { + t.Fatal(err) + } + ids := taskIDs(t, store, run) + for i, id := range ids { + ids[i] = strings.TrimPrefix(id, run+":") + } + sort.Strings(ids) + sets = append(sets, ids) + stateKey, _ := store.TaskStateKey(run) + metaKey, _ := store.RunMetaKey(run) + chanKey, _ := store.RunChannelsKey(run) + _, _ = store.RedisClient().Do(ctx, "DEL", stateKey, metaKey, chanKey) + for _, key := range []string{"a", "b"} { + activeKey, _ := store.RoundActiveKey(run, 1, key) + _, _ = store.RedisClient().Do(ctx, "DEL", activeKey) + } + _, _ = store.RedisClient().Do(ctx, "SREM", store.ActiveRunsKey(), run) + } + if !reflect.DeepEqual(sets[0], sets[1]) { + t.Fatalf("scope=%v serial=%v channel=%v", scope, sets[0], sets[1]) + } + if scope == nil { + want := []string{operation + ":1", operation + ":13", operation + ":2", operation + ":3"} + sort.Strings(want) + if !reflect.DeepEqual(sets[0], want) { + t.Fatalf("qualification set=%v want=%v", sets[0], want) + } + } + } +} + +func TestChannelSQLShapeAndParameters(t *testing.T) { + for _, term := range []string{"FORCE INDEX (idx_card_chan_key)", "STRAIGHT_JOIN sys_dept", "c.chan_key = ?", "c.id > ?", "c.iccid <> ''", "c.current_package_code <> ''", "d.deleted_at IS NULL", "c.tenant_id = d.tenant_id", "ORDER BY c.id ASC", "LIMIT ?"} { + if !strings.Contains(channelPageSQL, term) { + t.Fatalf("missing %q", term) + } + } + if strings.Contains(channelPageSQL, "<=") || strings.Count(channelPageSQL, "?") != 3 { + t.Fatal("SQL contains upper bound or changed bind parameter count") + } +} diff --git a/internal/producer/monthlyusage/producer.go b/internal/producer/monthlyusage/producer.go new file mode 100644 index 0000000..adb9fc9 --- /dev/null +++ b/internal/producer/monthlyusage/producer.go @@ -0,0 +1,485 @@ +package monthlyusage + +import ( + "context" + "crypto/rand" + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + "strings" + "sync/atomic" + "time" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +const operation = "current_month_usage_full_sync" + +// scopeSampleLimit bounds how many channel names are copied into run metadata. +// The run hash and its archive JSON must stay bounded regardless of how many +// channels a scope selects. +const scopeSampleLimit = 10 + +// scopeDigest is a stable short fingerprint of the selected channels. It lets +// operators confirm two runs covered the same scope without storing the list. +func scopeDigest(scope *model.SyncScope) string { + if scope == nil || len(scope.Channels) == 0 { + return "" + } + digest := sha256.Sum256([]byte(strings.Join(scope.Channels, "\n"))) + return hex.EncodeToString(digest[:8]) +} + +type Config struct { + SupplyObservation *supply.Tracker + // Empty/zero mean omitted for Go callers; startup config validates explicit values. + ScanConcurrency int + PageSize int + GlobalHighWaterMark int64 + ChannelHighWaterMark int64 + PollInterval time.Duration + RunLockKey string + RunLockTTL time.Duration + RoundRetryEnabled bool + MaxRounds int + FailureRetention time.Duration +} + +func (c Config) withDefaults() Config { + c.RoundRetryEnabled = true + if c.ScanConcurrency == 0 { + c.ScanConcurrency = 4 + } + if c.PageSize <= 0 { + c.PageSize = 100 + } + if c.PollInterval <= 0 { + c.PollInterval = 2 * time.Second + } + if c.RunLockKey == "" { + c.RunLockKey = "flow:producer:active" + } + if c.RunLockTTL <= 0 { + c.RunLockTTL = 5 * time.Minute + } + if c.MaxRounds <= 0 { + c.MaxRounds = 5 + } + if c.FailureRetention <= 0 { + c.FailureRetention = 30 * 24 * time.Hour + } + return c +} + +type CardScanner interface { + SnapshotMaxID(context.Context) (int64, error) +} + +type Card struct { + ID int64 + ICCID string + MSISDN string + IMSI string + ChanKey string + TempKey string + SyncEnabled bool + Operator int +} + +type channelMetadata struct { + ChanKey string `orm:"chan_key"` + TempKey string `orm:"temp_key"` + SyncEnabled bool `orm:"sync_enabled"` + Operator int `orm:"operator"` +} + +type DatabaseScanner struct{} + +func (s *DatabaseScanner) SnapshotMaxID(ctx context.Context) (int64, error) { + var card entity.TSimCardInfo + cardTable := dao.TSimCardInfo.Table() + columns := dao.TSimCardInfo.Columns() + if err := monthlyUsageEligibleCardModel(ctx). + Fields(cardTable + "." + columns.Id + " AS " + columns.Id). + OrderDesc(cardTable + "." + columns.Id). + Limit(1). + Scan(&card); err != nil { + return 0, fmt.Errorf("read card snapshot max id: %w", err) + } + return card.Id, nil +} + +func monthlyUsageEligibleCardModel(ctx context.Context) *gdb.Model { + cardTable := dao.TSimCardInfo.Table() + deptTable := dao.SysDept.Table() + cardColumns := dao.TSimCardInfo.Columns() + deptColumns := dao.SysDept.Columns() + + return dao.TSimCardInfo.Ctx(ctx). + InnerJoin(deptTable, cardTable+"."+cardColumns.DeptId+" = "+deptTable+"."+deptColumns.DeptId+" AND "+cardTable+"."+cardColumns.TenantId+" = "+deptTable+"."+deptColumns.TenantId). + Where(cardTable+"."+cardColumns.Iccid+" <> ?", ""). + Where(cardTable+"."+cardColumns.ChanKey+" <> ?", ""). + Where(cardTable+"."+cardColumns.CurrentPackageCode+" <> ?", ""). + WhereNull(deptTable + "." + deptColumns.DeletedAt) +} + +type Producer struct { + scanner CardScanner + store *zset.Store + config Config + running atomic.Int32 + owner string + observation *supply.Observer +} + +type productionStats struct { + pages int64 + scanned int64 + eligible int64 + enqueued int64 + skippedEmptyChan int64 + skippedNoTemplate int64 + skippedSyncDisabled int64 + skippedOutOfScope int64 + skippedChannels int64 + stoppedTasks int64 + channels map[string]struct{} + scopeChannels int +} + +func (s *productionStats) channelCount() int { + return len(s.channels) +} + +func (s *productionStats) log(ctx context.Context, status, runID, queryMonth, stage, reason string, startedAt time.Time, maxID int64, err error) { + message := fmt.Sprintf("event=monthly_usage_producer status=%s runId=%s queryMonth=%s scanUpperID=%d pages=%d scanned=%d eligible=%d enqueued=%d skippedEmptyChan=%d skippedNoTemplate=%d skippedSyncDisabled=%d skippedOutOfScope=%d stoppedTasks=%d channelCount=%d scopeChannels=%d durationMs=%d", status, runID, queryMonth, maxID, s.pages, s.scanned, s.eligible, s.enqueued, s.skippedEmptyChan, s.skippedNoTemplate, s.skippedSyncDisabled, s.skippedOutOfScope, s.stoppedTasks, s.channelCount(), s.scopeChannels, time.Since(startedAt).Milliseconds()) + message += fmt.Sprintf(" skippedChannels=%d", s.skippedChannels) + if s.scanned > 0 { + message += fmt.Sprintf(" cardsPerSec=%.2f", float64(s.scanned)/time.Since(startedAt).Seconds()) + } + if stage != "" { + message += " stage=" + stage + } + if reason != "" { + message += " reason=" + reason + } + if err != nil { + message += fmt.Sprintf(" error=%q", err.Error()) + } + if status == "failed" { + g.Log().Error(ctx, message) + return + } + if status == "skipped" { + g.Log().Warning(ctx, message) + return + } + g.Log().Info(ctx, message) +} + +func New(scanner CardScanner, store *zset.Store, config Config) (*Producer, error) { + if scanner == nil || store == nil { + return nil, fmt.Errorf("monthly usage producer dependencies are incomplete") + } + config = config.withDefaults() + if err := ValidateScanSettings(config.ScanConcurrency); err != nil { + return nil, err + } + if config.RoundRetryEnabled { + if _, ok := scanner.(channelScanner); !ok { + return nil, fmt.Errorf("channel scan requires ListChannels and ScanChannel capabilities") + } + } + return &Producer{scanner: scanner, store: store, config: config, owner: newOwner()}, nil +} + +var ErrAlreadyRunning = fmt.Errorf("monthly usage producer already running") + +// ErrRunFinished is returned when a redelivered start message names a run that +// already reached a terminal state. The run must not be resurrected; callers +// should log and ack instead of retrying. +var ErrRunFinished = fmt.Errorf("monthly usage run already finished") + +// runAlreadyFinished reports whether runID exists and has already completed. +// A missing run is not an error: the message may be the first delivery. +func (p *Producer) runAlreadyFinished(ctx context.Context, runID string) (bool, error) { + meta, err := p.store.GetRunMeta(ctx, runID) + if err != nil { + if strings.Contains(err.Error(), "not found") { + return false, nil + } + return false, err + } + return meta.Status == "completed" || meta.Status == "completed_with_failures", nil +} + +func (p *Producer) Produce(ctx context.Context, message model.TrafficUsageMonthlyStartMessage) (err error) { + startedAt := time.Now() + stats := productionStats{channels: make(map[string]struct{})} + if !p.running.CompareAndSwap(0, 1) { + stats.log(ctx, "skipped", message.RunID, message.QueryMonth, "precheck", "already_running", startedAt, 0, nil) + return ErrAlreadyRunning + } + defer p.running.Store(0) + + if message.RunID == "" { + err = fmt.Errorf("runId is empty") + stats.log(ctx, "failed", message.RunID, message.QueryMonth, "validate", "", startedAt, 0, err) + return err + } + scope, err := model.NormalizeScope(message.Scope) + if err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryMonth, "validate", "invalid_scope", startedAt, 0, err) + return err + } + stats.scopeChannels = scope.ChannelCount() + // A redelivered start message must not resurrect a run that already + // finished; that would create a second producer pass over the same RunID. + if p.config.RoundRetryEnabled { + if finished, checkErr := p.runAlreadyFinished(ctx, message.RunID); checkErr != nil { + stats.log(ctx, "failed", message.RunID, message.QueryMonth, "run_state", "", startedAt, 0, checkErr) + return checkErr + } else if finished { + stats.log(ctx, "skipped", message.RunID, message.QueryMonth, "run_state", "run_finished", startedAt, 0, nil) + return ErrRunFinished + } + } + acquired, err := p.store.AcquireLockOrTakeover(ctx, p.config.RunLockKey, p.owner, message.RunID, p.config.RunLockTTL) + if err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryMonth, "lock", "", startedAt, 0, err) + return err + } + if !acquired { + stats.log(ctx, "skipped", message.RunID, message.QueryMonth, "lock", "already_running", startedAt, 0, nil) + return ErrAlreadyRunning + } + runCtx, cancel := context.WithCancel(ctx) + defer cancel() + defer func() { + releaseCtx, releaseCancel := context.WithTimeout(context.Background(), time.Second) + defer releaseCancel() + _, _ = p.store.ReleaseLock(releaseCtx, p.config.RunLockKey, p.owner) + }() + pending, err := p.hasPendingTasks(ctx, message.RunID) + if err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryMonth, "pending_check", "", startedAt, 0, err) + return err + } + if pending { + stats.log(ctx, "skipped", message.RunID, message.QueryMonth, "pending_check", "pending_tasks", startedAt, 0, nil) + return ErrAlreadyRunning + } + renewErr := make(chan error, 1) + renewDone := make(chan struct{}) + go func() { + defer close(renewDone) + p.renewLock(runCtx, cancel, renewErr) + }() + defer func() { + cancel() + <-renewDone + select { + case lockErr := <-renewErr: + err = lockErr + default: + } + }() + ctx = runCtx + if message.Operation == "" { + message.Operation = operation + } + if p.config.RoundRetryEnabled { + if err = p.store.CreateRun(ctx, zset.RunMeta{ + RunID: message.RunID, + Operation: message.Operation, + QueryMonth: message.QueryMonth, + Status: "producing", + CurrentRound: 1, + ScopeChannels: scope.ChannelCount(), + ScopeDigest: scopeDigest(scope), + ScopeChannelsSample: strings.Join(scope.ChannelSample(scopeSampleLimit), ","), + }); err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryMonth, "run_init", "", startedAt, 0, err) + return err + } + } + maxID, err := p.scanner.SnapshotMaxID(ctx) + if err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryMonth, "snapshot", "", startedAt, 0, err) + return err + } + g.Log().Infof(ctx, "event=monthly_usage_producer status=started runId=%s queryMonth=%s scanUpperID=%d scopeChannels=%d scanMode=channel scanConcurrency=%d pageSize=%d", message.RunID, message.QueryMonth, maxID, scope.ChannelCount(), p.config.ScanConcurrency, p.config.PageSize) + stage := "scan" + defer func() { + if err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryMonth, stage, "", startedAt, maxID, err) + return + } + stats.log(ctx, "completed", message.RunID, message.QueryMonth, "", "", startedAt, maxID, nil) + }() + if p.config.RoundRetryEnabled { + stage = "channel_scan" + if err = p.produceChannels(ctx, message, scope, maxID, startedAt, &stats); err != nil { + if isProductionClosed(err) { + // A same-Run redelivery after producerDone leaves consumption alone. + return nil + } + return err + } + // Verify ownership immediately before closing production. The periodic + // renewer still cancels scanners and capacity waits on lease loss. + var owned bool + owned, err = p.store.RenewLock(ctx, p.config.RunLockKey, p.owner, p.config.RunLockTTL) + if err != nil { + return err + } + if !owned { + return fmt.Errorf("producer lock token is no longer owned") + } + if err = ctx.Err(); err != nil { + return err + } + stage = "producer_done" + err = p.store.MarkProducerDone(ctx, message.RunID) + return err + } + return fmt.Errorf("monthly usage full sync requires Run queue; legacy serial production has been removed") +} + +// isRunNotFound reports whether err means the run's metadata hash is absent, as +// opposed to a genuine read failure. +func isRunNotFound(err error) bool { + return errors.Is(err, zset.ErrRunNotFound) +} + +func (p *Producer) hasPendingTasks(ctx context.Context, currentRunID ...string) (bool, error) { + currentID := "" + if len(currentRunID) > 0 { + currentID = currentRunID[0] + } + if p.config.RoundRetryEnabled { + runs, err := p.store.ActiveRuns(ctx) + if err != nil { + return false, err + } + for _, runID := range runs { + if runID == currentID { + // A redelivered message for the same run may resume a producer + // that stopped before producerDone. Stable task IDs make the + // rescan idempotent; other runs still provide backpressure. + continue + } + meta, metaErr := p.store.GetRunMeta(ctx, runID) + if metaErr != nil { + // The active index is a global key written outside the run-scoped + // Lua transaction, so it can list a run whose hash is already gone + // (expired, evicted, or cleaned by hand). Returning an error here + // would surface as a non-retryable failure and make the consumer + // redeliver this request forever. Treat it as a stale index entry + // and drop it. + if isRunNotFound(metaErr) { + if _, removeErr := p.store.RedisClient().Do(ctx, "SREM", p.store.ActiveRunsKey(), runID); removeErr != nil { + return false, removeErr + } + g.Log().Warningf(ctx, "event=monthly_usage_producer status=skipped stage=pending_check reason=stale_index runId=%s", runID) + continue + } + return false, metaErr + } + // A "running" run has already finished producing (producerDone=1) + // and the consumer owns the remaining work: it is genuinely in-flight + // and must keep providing backpressure. + if meta.Status == "running" { + return true, nil + } + // A "producing" run for another ID cannot have a live producer. This + // function only runs after AcquireLockOrTakeover succeeded on the + // single producer lock, so any live producer would still hold it and + // we could not have acquired it. Such a run is a zombie left behind by + // a producer that died before MarkProducerDone (for example the + // process was replaced mid-run). + // + // Promote it so the normal lifecycle can drain and close it. Blocking + // would reject every later request forever, and merely warning would + // leak the run's keys and its active-index entry indefinitely. + reclaimed, reclaimErr := p.store.MarkProducerAbandoned(ctx, runID) + if reclaimErr != nil { + return false, reclaimErr + } + g.Log().Warningf(ctx, "event=monthly_usage_producer status=skipped stage=pending_check reason=zombie_run runId=%s reclaimed=%t activeCount=%d inflightCount=%d nextRoundCount=%d", runID, reclaimed, meta.ActiveCount, meta.InflightCount, meta.NextRoundCount) + } + } + channels, err := p.store.ActiveChannels(ctx) + if err != nil { + return false, err + } + for _, chanKey := range channels { + size, sizeErr := p.store.Size(ctx, chanKey) + if sizeErr != nil { + return false, sizeErr + } + if size > 0 { + return true, nil + } + } + return false, nil +} + +func (p *Producer) renewLock(ctx context.Context, cancel context.CancelFunc, errors chan<- error) { + interval := p.config.RunLockTTL / 3 + if interval <= 0 { + interval = time.Second + } + ticker := time.NewTicker(interval) + defer ticker.Stop() + for { + select { + case <-ctx.Done(): + return + case <-ticker.C: + ok, err := p.store.RenewLock(ctx, p.config.RunLockKey, p.owner, p.config.RunLockTTL) + if err != nil || !ok { + // Cancellation during a renewal RPC is shutdown, not lease loss. + if ctx.Err() != nil { + return + } + if err == nil { + err = fmt.Errorf("producer lock token is no longer owned") + } + select { + case errors <- fmt.Errorf("renew producer lock: %w", err): + default: + } + cancel() + return + } + } + } +} + +func newOwner() string { + var bytes [12]byte + if _, err := rand.Read(bytes[:]); err == nil { + return "producer-" + hex.EncodeToString(bytes[:]) + } + return fmt.Sprintf("producer-%d", time.Now().UnixNano()) +} + +func sleepContext(ctx context.Context, duration time.Duration) error { + timer := time.NewTimer(duration) + defer timer.Stop() + select { + case <-timer.C: + return nil + case <-ctx.Done(): + return ctx.Err() + } +} diff --git a/internal/producer/monthlyusage/producer_test.go b/internal/producer/monthlyusage/producer_test.go new file mode 100644 index 0000000..4377ade --- /dev/null +++ b/internal/producer/monthlyusage/producer_test.go @@ -0,0 +1,168 @@ +package monthlyusage + +import ( + "context" + "fmt" + "strings" + "sync" + "testing" + + "github.com/gogf/gf/v2/container/gvar" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +type pendingRedis struct { + channels []string + sizes map[string]int64 + failKey string +} + +func (r *pendingRedis) Do(_ context.Context, command string, args ...any) (*gvar.Var, error) { + switch strings.ToUpper(command) { + case "SMEMBERS": + return gvar.New(r.channels), nil + case "ZCARD": + key := fmt.Sprint(args[0]) + if key == r.failKey { + return nil, fmt.Errorf("redis unavailable") + } + return gvar.New(r.sizes[key]), nil + default: + return nil, fmt.Errorf("unexpected command %s", command) + } +} + +func (r *pendingRedis) Eval(_ context.Context, script string, numKeys int64, keys []string, args []any) (*gvar.Var, error) { + return nil, fmt.Errorf("unexpected Eval call") +} + +func TestHasPendingTasks(t *testing.T) { + tests := []struct { + name string + redis *pendingRedis + want bool + wantErr bool + }{ + { + name: "all registered channels are empty", + redis: &pendingRedis{ + channels: []string{"mobile-a", "telecom-a"}, + sizes: map[string]int64{ + "flow:mobile-a": 0, + "flow:telecom-a": 0, + }, + }, + }, + { + name: "previous run still has tasks", + redis: &pendingRedis{ + channels: []string{"mobile-a", "telecom-a"}, + sizes: map[string]int64{ + "flow:mobile-a": 0, + "flow:telecom-a": 3, + }, + }, + want: true, + }, + { + name: "redis error is not treated as empty", + redis: &pendingRedis{ + channels: []string{"mobile-a"}, + sizes: map[string]int64{}, + failKey: "flow:mobile-a", + }, + wantErr: true, + }, + } + + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + store, err := zset.New(test.redis, "flow:") + if err != nil { + t.Fatal(err) + } + producer, err := New(&fakeScanner{}, store, Config{}) + if err != nil { + t.Fatal(err) + } + got, err := func() (bool, error) { + producer.config.RoundRetryEnabled = false + return producer.hasPendingTasks(context.Background()) + }() + if (err != nil) != test.wantErr { + t.Fatalf("hasPendingTasks() error = %v, wantErr %v", err, test.wantErr) + } + if got != test.want { + t.Fatalf("hasPendingTasks() = %v, want %v", got, test.want) + } + }) + } +} + +type fakeScanner struct{} + +func (*fakeScanner) SnapshotMaxID(context.Context) (int64, error) { return 0, nil } +func (*fakeScanner) Scan(context.Context, int64, int64, int) ([]Card, error) { + return nil, nil +} + +func (*fakeScanner) ListChannels(context.Context) ([]channelMetadata, error) { return nil, nil } +func (*fakeScanner) ScanChannel(context.Context, string, int64, int) ([]Card, error) { return nil, nil } + +type pageScanner struct { + cards []Card +} + +func (s *pageScanner) SnapshotMaxID(context.Context) (int64, error) { + if len(s.cards) == 0 { + return 0, nil + } + return s.cards[len(s.cards)-1].ID, nil +} + +func (s *pageScanner) Scan(_ context.Context, lastID, _ int64, _ int) ([]Card, error) { + if lastID != 0 { + return nil, nil + } + return s.cards, nil +} + +type producerRedis struct { + mu sync.Mutex + enqueueBatches map[string]int +} + +func (r *producerRedis) Do(_ context.Context, command string, _ ...any) (*gvar.Var, error) { + switch strings.ToUpper(command) { + case "SET": + return gvar.New("OK"), nil + case "SMEMBERS": + return gvar.New([]string{}), nil + case "HGETALL": + // No stored run metadata: the producer treats this as a first delivery. + return gvar.New(map[string]string{}), nil + default: + return nil, fmt.Errorf("unexpected command %s", command) + } +} + +func (r *producerRedis) Eval(_ context.Context, _ string, numKeys int64, keys []string, args []any) (*gvar.Var, error) { + if numKeys == 1 { + return gvar.New(int64(1)), nil + } + if numKeys != 3 { + return nil, fmt.Errorf("unexpected key count %d", numKeys) + } + r.mu.Lock() + r.enqueueBatches[keys[0]] = len(args) - 3 + r.mu.Unlock() + return gvar.New(len(args) - 3), nil +} + +func (s *pageScanner) ListChannels(ctx context.Context) ([]channelMetadata, error) { + return scannerFor(s.cards, 0).ListChannels(ctx) +} +func (s *pageScanner) ScanChannel(ctx context.Context, key string, last int64, size int) ([]Card, error) { + return scannerFor(s.cards, 0).ScanChannel(ctx, key, last, size) +} diff --git a/internal/producer/monthlyusage/scope_test.go b/internal/producer/monthlyusage/scope_test.go new file mode 100644 index 0000000..58a69b2 --- /dev/null +++ b/internal/producer/monthlyusage/scope_test.go @@ -0,0 +1,249 @@ +package monthlyusage + +import ( + "context" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +func newScopeProducer(t *testing.T, cards []Card) (*Producer, *producerRedis) { + t.Helper() + store := testStore(t) + scanner := scannerFor(cards, cards[len(cards)-1].ID) + for i := range scanner.channels { + for _, c := range cards { + if c.ChanKey == scanner.channels[i].ChanKey { + scanner.channels[i].TempKey = c.TempKey + scanner.channels[i].SyncEnabled = c.SyncEnabled + break + } + } + } + p, err := New(scanner, store, Config{PageSize: 100}) + if err != nil { + t.Fatal(err) + } + return p, &producerRedis{enqueueBatches: map[string]int{}} +} +func scopedMessage(runID string, channels ...string) model.TrafficUsageMonthlyStartMessage { + return model.TrafficUsageMonthlyStartMessage{ + RunID: runID, + Operation: "current_month_usage_full_sync", + QueryMonth: "202608", + Scope: &model.SyncScope{Channels: channels}, + } +} + +func TestProduceScopeEnqueuesOnlySelectedChannels(t *testing.T) { + producer, redis := newScopeProducer(t, []Card{ + {ID: 1, ChanKey: "217", TempKey: "sim_yd_v5", Operator: 1}, + {ID: 2, ChanKey: "173", TempKey: "sim_yd_v5", Operator: 1}, + {ID: 3, ChanKey: "217", TempKey: "sim_yd_v5", Operator: 1}, + {ID: 4, ChanKey: "189", TempKey: "sim_yd_v5", Operator: 1}, + }) + + if err := producer.Produce(context.Background(), scopedMessage("scoped", "217")); err != nil { + t.Fatal(err) + } + redis.enqueueBatches = map[string]int{} + for _, ch := range []string{"217", "173", "189", "ghost-channel", "a", "b", "mobile-a", "mobile-b"} { + key, _ := producer.store.RoundActiveKey("scoped", 1, ch) + n, _ := producer.store.SizeKey(context.Background(), key) + if n > 0 { + redis.enqueueBatches["flow:"+ch] = int(n) + } + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.enqueueBatches) != 1 { + t.Fatalf("enqueue batches = %+v, want only the 217 channel", redis.enqueueBatches) + } + if redis.enqueueBatches["flow:217"] != 2 { + t.Fatalf("enqueue batches = %+v, want 2 tasks on flow:217", redis.enqueueBatches) + } +} + +func TestProduceScopeMultipleChannels(t *testing.T) { + producer, redis := newScopeProducer(t, []Card{ + {ID: 1, ChanKey: "217", TempKey: "sim_yd_v5", Operator: 1}, + {ID: 2, ChanKey: "173", TempKey: "sim_yd_v5", Operator: 1}, + {ID: 3, ChanKey: "189", TempKey: "sim_yd_v5", Operator: 1}, + }) + + if err := producer.Produce(context.Background(), scopedMessage("scoped", "189", "217")); err != nil { + t.Fatal(err) + } + redis.enqueueBatches = map[string]int{} + for _, ch := range []string{"217", "173", "189", "ghost-channel", "a", "b", "mobile-a", "mobile-b"} { + key, _ := producer.store.RoundActiveKey("scoped", 1, ch) + n, _ := producer.store.SizeKey(context.Background(), key) + if n > 0 { + redis.enqueueBatches["flow:"+ch] = int(n) + } + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.enqueueBatches) != 2 { + t.Fatalf("enqueue batches = %+v, want 217 and 189", redis.enqueueBatches) + } + if _, ok := redis.enqueueBatches["flow:173"]; ok { + t.Fatalf("channel 173 was enqueued despite not being selected: %+v", redis.enqueueBatches) + } +} + +func TestProduceFullScopeKeepsEveryChannel(t *testing.T) { + producer, redis := newScopeProducer(t, []Card{ + {ID: 1, ChanKey: "217", TempKey: "sim_yd_v5", Operator: 1}, + {ID: 2, ChanKey: "173", TempKey: "sim_yd_v5", Operator: 1}, + }) + + if err := producer.Produce(context.Background(), model.TrafficUsageMonthlyStartMessage{ + RunID: "full", + Operation: "current_month_usage_full_sync", + QueryMonth: "202608", + }); err != nil { + t.Fatal(err) + } + redis.enqueueBatches = map[string]int{} + for _, ch := range []string{"217", "173", "189", "ghost-channel", "a", "b", "mobile-a", "mobile-b"} { + key, _ := producer.store.RoundActiveKey("full", 1, ch) + n, _ := producer.store.SizeKey(context.Background(), key) + if n > 0 { + redis.enqueueBatches["flow:"+ch] = int(n) + } + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.enqueueBatches) != 2 { + t.Fatalf("enqueue batches = %+v, want both channels for a full sync", redis.enqueueBatches) + } +} + +func TestProduceScopeCombinesWithSyncDisabledGuard(t *testing.T) { + producer, redis := newScopeProducer(t, []Card{ + {ID: 1, ChanKey: "217", TempKey: "sim_yd_v5", Operator: 1}, + // Selected but excluded by sync_enabled: scope must not override it. + {ID: 2, ChanKey: "173", TempKey: "sim_yd_v5", SyncEnabled: true, Operator: 1}, + }) + + if err := producer.Produce(context.Background(), scopedMessage("scoped", "173")); err != nil { + t.Fatal(err) + } + redis.enqueueBatches = map[string]int{} + for _, ch := range []string{"217", "173", "189", "ghost-channel", "a", "b", "mobile-a", "mobile-b"} { + key, _ := producer.store.RoundActiveKey("scoped", 1, ch) + n, _ := producer.store.SizeKey(context.Background(), key) + if n > 0 { + redis.enqueueBatches["flow:"+ch] = int(n) + } + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.enqueueBatches) != 0 { + t.Fatalf("enqueue batches = %+v, want none (selected channel is sync-disabled)", redis.enqueueBatches) + } +} + +func TestProduceScopeSelectingUnknownChannelEnqueuesNothing(t *testing.T) { + producer, redis := newScopeProducer(t, []Card{ + {ID: 1, ChanKey: "217", TempKey: "sim_yd_v5", Operator: 1}, + }) + + if err := producer.Produce(context.Background(), scopedMessage("scoped", "does-not-exist")); err != nil { + t.Fatal(err) + } + redis.enqueueBatches = map[string]int{} + for _, ch := range []string{"217", "173", "189", "ghost-channel", "a", "b", "mobile-a", "mobile-b"} { + key, _ := producer.store.RoundActiveKey("scoped", 1, ch) + n, _ := producer.store.SizeKey(context.Background(), key) + if n > 0 { + redis.enqueueBatches["flow:"+ch] = int(n) + } + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.enqueueBatches) != 0 { + t.Fatalf("enqueue batches = %+v, want none", redis.enqueueBatches) + } +} + +func TestProduceRejectsInvalidScope(t *testing.T) { + producer, redis := newScopeProducer(t, []Card{ + {ID: 1, ChanKey: "217", TempKey: "sim_yd_v5", Operator: 1}, + }) + + if err := producer.Produce(context.Background(), scopedMessage("bad", "invalid/chan")); err == nil { + t.Fatal("expected invalid chanKey to be rejected") + } + redis.enqueueBatches = map[string]int{} + for _, ch := range []string{"217", "173", "189", "ghost-channel", "a", "b", "mobile-a", "mobile-b"} { + key, _ := producer.store.RoundActiveKey("scoped", 1, ch) + n, _ := producer.store.SizeKey(context.Background(), key) + if n > 0 { + redis.enqueueBatches["flow:"+ch] = int(n) + } + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.enqueueBatches) != 0 { + t.Fatalf("rejected scope must not enqueue: %+v", redis.enqueueBatches) + } +} + +func TestProduceScopeDeduplicatesChannels(t *testing.T) { + producer, redis := newScopeProducer(t, []Card{ + {ID: 1, ChanKey: "217", TempKey: "sim_yd_v5", Operator: 1}, + }) + + if err := producer.Produce(context.Background(), scopedMessage("scoped", "217", "217")); err != nil { + t.Fatal(err) + } + redis.enqueueBatches = map[string]int{} + for _, ch := range []string{"217", "173", "189", "ghost-channel", "a", "b", "mobile-a", "mobile-b"} { + key, _ := producer.store.RoundActiveKey("scoped", 1, ch) + n, _ := producer.store.SizeKey(context.Background(), key) + if n > 0 { + redis.enqueueBatches["flow:"+ch] = int(n) + } + } + redis.mu.Lock() + defer redis.mu.Unlock() + if redis.enqueueBatches["flow:217"] != 1 { + t.Fatalf("enqueue batches = %+v, want 1 task despite a duplicated chanKey", redis.enqueueBatches) + } +} + +func TestScopeDigestIsOrderIndependent(t *testing.T) { + first := &model.SyncScope{Channels: []string{"173", "217"}} + second := &model.SyncScope{Channels: []string{"217", "173"}} + // Normalization sorts the channels, so both requests produce one digest. + normalizedFirst, err := model.NormalizeScope(first) + if err != nil { + t.Fatal(err) + } + normalizedSecond, err := model.NormalizeScope(second) + if err != nil { + t.Fatal(err) + } + if scopeDigest(normalizedFirst) != scopeDigest(normalizedSecond) { + t.Fatalf("digest depends on channel order: %s vs %s", scopeDigest(normalizedFirst), scopeDigest(normalizedSecond)) + } + if scopeDigest(nil) != "" { + t.Fatalf("full sync must have an empty digest") + } +} + +func TestRunAlreadyFinishedTreatsMissingRunAsFirstDelivery(t *testing.T) { + producer, _ := newScopeProducer(t, []Card{ + {ID: 1, ChanKey: "217", TempKey: "sim_yd_v5", Operator: 1}, + }) + // The fake Redis returns no metadata, which must not be treated as finished. + finished, err := producer.runAlreadyFinished(context.Background(), "fresh") + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if finished { + t.Fatal("a run without stored metadata must not be reported as finished") + } +} diff --git a/internal/producer/monthlyusage/supply_observation.go b/internal/producer/monthlyusage/supply_observation.go new file mode 100644 index 0000000..82645f8 --- /dev/null +++ b/internal/producer/monthlyusage/supply_observation.go @@ -0,0 +1,90 @@ +package monthlyusage + +import ( + "context" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply" + "github.com/gogf/gf/v2/frame/g" + "time" +) + +func (p *Producer) observe(key, stage string, pending int, added, ended bool) { + if p.observation != nil { + p.observation.Event(key, stage, pending, added, ended, time.Now()) + } +} +func (p *Producer) startObservation(ctx context.Context, run string, keys []string, started time.Time) func(bool) { + tracker := p.config.SupplyObservation + if tracker == nil { + return func(bool) {} + } + o := supply.New(newOwner(), started, keys) + p.observation = o + readCtx, readCancel := context.WithTimeout(context.Background(), 500*time.Millisecond) + meta, err := p.store.GetRunMeta(readCtx, run) + readCancel() + if err != nil { + o.Partial() + } else if meta.SupplyObservation != nil { + o.Restore(*meta.SupplyObservation) + } + tracker.Put(run, o) + stop := make(chan struct{}) + done := make(chan struct{}) + flush := func() { + writeCtx, cancel := context.WithTimeout(context.Background(), 500*time.Millisecond) + defer cancel() + encoded, e := o.Encode() + if e == nil { + var saved bool + saved, e = p.store.SaveSupply(writeCtx, run, p.config.RunLockKey, p.owner, encoded) + if !saved && e == nil { + o.Partial() + } + } + if e != nil { + o.Partial() + g.Log().Warningf(writeCtx, "event=supply_observation status=partial runId=%s error=%q", run, e.Error()) + } + } + go func() { + defer close(done) + sampleTicker := time.NewTicker(time.Second) + defer sampleTicker.Stop() + flushTicker := time.NewTicker(10 * time.Second) + defer flushTicker.Stop() + for { + select { + case <-stop: + return + case <-ctx.Done(): + o.Partial() + return + case <-flushTicker.C: + flush() + case <-sampleTicker.C: + views := o.Views() + sampleCtx, cancel := context.WithTimeout(context.Background(), 500*time.Millisecond) + samples, e := p.store.ReadSupply(sampleCtx, run, keys, time.Now()) + cancel() + now := time.Now() + for _, key := range keys { + s := samples[key] + if e != nil || !tracker.LocalConsumer || !tracker.ConsumerActive.Load() { + s.Known = false + } + o.Apply(key, views[key], s, now) + } + } + } + }() + return func(success bool) { + close(stop) + <-done + if !success { + o.Partial() + } + flush() + tracker.Remove(run) + p.observation = nil + } +} diff --git a/internal/producer/monthlyusage/supply_observation_test.go b/internal/producer/monthlyusage/supply_observation_test.go new file mode 100644 index 0000000..fd812fe --- /dev/null +++ b/internal/producer/monthlyusage/supply_observation_test.go @@ -0,0 +1,69 @@ +package monthlyusage + +import ( + "context" + "fmt" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" + "github.com/gogf/gf/v2/container/gvar" + "strings" + "testing" + "time" +) + +type telemetryFailRedis struct{ zset.RedisClient } + +func (r telemetryFailRedis) Eval(ctx context.Context, script string, n int64, keys []string, args []any) (*gvar.Var, error) { + if strings.Contains(script, "'supplyObservation'") { + return nil, fmt.Errorf("injected telemetry write failure") + } + return r.RedisClient.Eval(ctx, script, n, keys, args) +} +func TestSupplyFlushFailureDoesNotBlockProducerDone(t *testing.T) { + client := testutil.Redis(t) + store, err := zset.New(telemetryFailRedis{client}, "flow:") + if err != nil { + t.Fatal(err) + } + scanner := scannerFor([]Card{{ID: 1, ChanKey: "a"}}, 1) + p := testProducer(t, scanner, store, Config{SupplyObservation: supply.NewTracker(true)}) + if err = p.Produce(context.Background(), model.TrafficUsageMonthlyStartMessage{RunID: "telemetry-failure"}); err != nil { + t.Fatal(err) + } + requireMeta(t, store, "telemetry-failure", true, 1) +} + +func TestSupplyObservesEmptyTargetAndFinalFlush(t *testing.T) { + store := testStore(t) + scanner := scannerFor([]Card{{ID: 1, ChanKey: "a"}}, 2) + scanner.channels = append(scanner.channels, channelMetadata{ChanKey: "empty", TempKey: "sim_yd_v5"}) + tracker := supply.NewTracker(true) + tracker.ConsumerActive.Store(true) + scanner.hook = func(ctx context.Context, key string, last int64) error { + if key == "empty" { + return sleepContext(ctx, 2200*time.Millisecond) + } + return nil + } + p := testProducer(t, scanner, store, Config{SupplyObservation: tracker}) + if err := p.Produce(context.Background(), model.TrafficUsageMonthlyStartMessage{RunID: "supply-end", QueryMonth: "202610"}); err != nil { + t.Fatal(err) + } + m, err := store.GetRunMeta(context.Background(), "supply-end") + if err != nil { + t.Fatal(err) + } + if !m.ProducerDone || m.SupplyObservation == nil { + t.Fatalf("missing final observation %+v", m) + } + snapshot := m.SupplyObservation + if snapshot.Summary.Targets != 2 || snapshot.Summary.Confirmed != 0 { + t.Fatalf("bad summary %+v", snapshot) + } + c := snapshot.Channels[1] + if c.Key != "empty" || c.FirstEnqueueMS != nil || c.Termination != "empty_result" || c.Unconfirmed.TotalMS < 900 { + t.Fatalf("bad empty target %+v", c) + } +} diff --git a/internal/producer/monthlyusage/zombie_test.go b/internal/producer/monthlyusage/zombie_test.go new file mode 100644 index 0000000..4d878a3 --- /dev/null +++ b/internal/producer/monthlyusage/zombie_test.go @@ -0,0 +1,174 @@ +package monthlyusage + +import ( + "context" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +func newZombieTestProducer(t *testing.T) (*Producer, *zset.Store) { + t.Helper() + store, err := zset.New(testutil.Redis(t), "flow:") + if err != nil { + t.Fatal(err) + } + producer, err := New(&pageScanner{cards: nil}, store, Config{PageSize: 100, RoundRetryEnabled: true}) + if err != nil { + t.Fatal(err) + } + return producer, store +} + +// TestHasPendingTasksReclaimsZombieRun guards the "zombie run" regression. +// +// A producer that dies before MarkProducerDone (for example the container is +// replaced mid-run) leaves its run in status=producing with producerDone=0. +// completeRunIfIdle and advanceRunRound both require producerDone=1, so such a +// run can never be closed, never leaves the active index, and never gets the +// retention TTL. Treating it as backpressure rejected every later request +// forever; merely warning about it leaked keys and an index entry indefinitely. +func TestHasPendingTasksReclaimsZombieRun(t *testing.T) { + producer, store := newZombieTestProducer(t) + ctx := context.Background() + + // A genuinely in-flight run (producerDone set, consumer working) must keep + // providing backpressure. + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "running-run", Status: "running", ProducerDone: true, CurrentRound: 1}); err != nil { + t.Fatal(err) + } + pending, err := producer.hasPendingTasks(ctx, "new-run") + if err != nil { + t.Fatal(err) + } + if !pending { + t.Fatal("a running run must still provide backpressure") + } + if _, err := store.RedisClient().Do(ctx, "SREM", store.ActiveRunsKey(), "running-run"); err != nil { + t.Fatal(err) + } + + // A zombie run must not block a new request, and must be promoted so the + // normal lifecycle can drain and close it. + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "zombie-run", Status: "producing", ProducerDone: false, CurrentRound: 1}); err != nil { + t.Fatal(err) + } + pending, err = producer.hasPendingTasks(ctx, "new-run-2") + if err != nil { + t.Fatal(err) + } + if pending { + t.Fatal("a producing run with producerDone=0 must not block new requests (zombie run)") + } + + meta, err := store.GetRunMeta(ctx, "zombie-run") + if err != nil { + t.Fatal(err) + } + if !meta.ProducerDone { + t.Fatal("the zombie run must be promoted (producerDone=1) so it can be closed") + } + if meta.Status != "running" { + t.Fatalf("promoted run status = %q, want running", meta.Status) + } + // Promotion makes the run closable, which is what applies the retention TTL + // and releases the active index entry. + if completed, err := store.CompleteRunIfIdle(ctx, "zombie-run"); err != nil || !completed { + t.Fatalf("promoted run must be closable: completed=%v err=%v", completed, err) + } +} + +// TestHasPendingTasksNeverReclaimsReplayRun guards the boundary: a replay target +// is also created as "producing", but it is driven by the replay engine under +// its own lock, so its producerDone is not the producer's to set. +func TestHasPendingTasksNeverReclaimsReplayRun(t *testing.T) { + producer, store := newZombieTestProducer(t) + ctx := context.Background() + + if err := store.CreateRun(ctx, zset.RunMeta{ + RunID: "replay-run", Status: "producing", ProducerDone: false, CurrentRound: 1, + SyncType: "replay", SourceRunID: "source-run", ReplayCycle: 1, + }); err != nil { + t.Fatal(err) + } + if _, err := producer.hasPendingTasks(ctx, "new-run"); err != nil { + t.Fatal(err) + } + meta, err := store.GetRunMeta(ctx, "replay-run") + if err != nil { + t.Fatal(err) + } + if meta.ProducerDone { + t.Fatal("a replay run must not be promoted by the full-sync producer") + } + if meta.Status != "producing" { + t.Fatalf("replay run status = %q, want producing", meta.Status) + } +} + +// TestHasPendingTasksDropsStaleIndexEntry covers a second unrecoverable path: +// the active index can list a run whose hash is already gone. Returning that as +// an error made the request fail permanently (the consumer redelivers it +// forever), which is the same "only fixable by hand-cleaning Redis" failure. +func TestHasPendingTasksDropsStaleIndexEntry(t *testing.T) { + producer, store := newZombieTestProducer(t) + ctx := context.Background() + + // Index a run that has no metadata hash. + if _, err := store.RedisClient().Do(ctx, "SADD", store.ActiveRunsKey(), "ghost-run"); err != nil { + t.Fatal(err) + } + pending, err := producer.hasPendingTasks(ctx, "new-run") + if err != nil { + t.Fatalf("a stale index entry must not fail the request: %v", err) + } + if pending { + t.Fatal("a stale index entry must not provide backpressure") + } + // The stale entry is removed so it does not have to be handled again. + members, err := store.ActiveRuns(ctx) + if err != nil { + t.Fatal(err) + } + for _, member := range members { + if member == "ghost-run" { + t.Fatal("the stale index entry must be removed") + } + } +} + +// TestProduceCreatesRunAfterZombie covers the end-to-end path through Produce: +// a zombie run must not stop a new run from being created. +func TestProduceCreatesRunAfterZombie(t *testing.T) { + store, err := zset.New(testutil.Redis(t), "flow:") + if err != nil { + t.Fatal(err) + } + ctx := context.Background() + producer, err := New(&pageScanner{cards: []Card{ + {ID: 1, ChanKey: "channel-a", TempKey: "sim_yd_v5", Operator: 1}, + }}, store, Config{PageSize: 100, RoundRetryEnabled: true}) + if err != nil { + t.Fatal(err) + } + + // A zombie run from a crashed producer. + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "zombie-run", Status: "producing", ProducerDone: false, CurrentRound: 1}); err != nil { + t.Fatal(err) + } + + // A new request must still produce its own run. + if err := producer.Produce(ctx, scopedMessage("fresh-run", "channel-a")); err != nil { + t.Fatalf("a zombie run must not stop a new run: %v", err) + } + meta, err := store.GetRunMeta(ctx, "fresh-run") + if err != nil { + t.Fatal(err) + } + // The producer enqueues into the active ZSet; ActiveCount is the count it + // maintains. EnqueuedCount is an accounting field the producer does not set. + if !meta.ProducerDone || meta.ActiveCount != 1 { + t.Fatalf("fresh run metadata = %+v, want producerDone with 1 active task", meta) + } +} diff --git a/internal/producer/monthlyusagehistory/blocked_zombie_test.go b/internal/producer/monthlyusagehistory/blocked_zombie_test.go new file mode 100644 index 0000000..0b5f594 --- /dev/null +++ b/internal/producer/monthlyusagehistory/blocked_zombie_test.go @@ -0,0 +1,206 @@ +package monthlyusagehistory + +import ( + "context" + "errors" + "reflect" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +// Pause the real producer after its first enqueue, before MarkProducerDone. +type interruptedScanner struct { + *datasetScanner + paused chan struct{} +} + +func TestInterruptedRunReclamationGuards(t *testing.T) { + for _, mode := range []string{"serial", "channel"} { + for _, status := range []string{"producing", "running"} { + for _, guard := range []string{"no_lock", "foreign_lock", "no_current_run", "same_run", "producer_done", "replay"} { + t.Run(mode+"/"+status+"/"+guard, func(t *testing.T) { + ctx := contextForTest(t) + store := testStore(t) + p := testProducer(t, scannerFor(nil, 0), store, Config{ScanMode: mode}) + meta := zset.RunMeta{RunID: "guarded", Status: status, ProducerDone: guard == "producer_done"} + if guard == "replay" { + meta.SyncType = "replay" + } + if err := store.CreateRun(ctx, meta); err != nil { + t.Fatal(err) + } + current := "fresh" + owner := p.owner + if guard == "same_run" { + current = meta.RunID + } + if guard == "no_current_run" { + current = "" + } + if guard == "foreign_lock" { + owner = "other-owner" + } + if guard != "no_lock" { + if ok, err := store.AcquireLockOrTakeover(ctx, p.config.RunLockKey, owner, current, p.config.RunLockTTL); err != nil || !ok { + t.Fatalf("acquire=%t err=%v", ok, err) + } + } + key, _ := store.RunMetaKey(meta.RunID) + before, err := store.RedisClient().Do(ctx, "HGETALL", key) + if err != nil { + t.Fatal(err) + } + pending, err := p.hasPendingTasks(ctx, current) + if err != nil || pending != (guard != "same_run") { + t.Fatalf("guard=%s pending=%t err=%v", guard, pending, err) + } + after, err := store.RedisClient().Do(ctx, "HGETALL", key) + if err != nil || !reflect.DeepEqual(before.MapStrStr(), after.MapStrStr()) { + t.Fatalf("guarded metadata changed: before=%v after=%v err=%v", before, after, err) + } + }) + } + } + } +} + +func (s *interruptedScanner) Scan(ctx context.Context, last, upper int64, size int) ([]Card, error) { + if last > 0 { + close(s.paused) + <-ctx.Done() + return nil, ctx.Err() + } + return s.datasetScanner.Scan(ctx, last, upper, size) +} + +func (s *interruptedScanner) ScanChannel(ctx context.Context, key string, last int64, size int) ([]Card, error) { + if last > 0 { + close(s.paused) + <-ctx.Done() + return nil, ctx.Err() + } + return s.datasetScanner.ScanChannel(ctx, key, last, size) +} + +func TestBlockedInterruptedProducerRecovery(t *testing.T) { + for _, mode := range []string{"serial", "channel"} { + for _, backlog := range []string{"none", "queued", "leased", "waiting"} { + t.Run(mode+"/"+backlog, func(t *testing.T) { + ctx := contextForTest(t) + store := testStore(t) + scanner := &interruptedScanner{datasetScanner: scannerFor([]Card{{ID: 1, ChanKey: "blocked"}, {ID: 2, ChanKey: "blocked"}}, 2), paused: make(chan struct{})} + old := testProducer(t, scanner, store, Config{ScanMode: mode, PageSize: 1}) + oldCtx, interrupt := context.WithCancel(ctx) + defer interrupt() + result := make(chan error, 1) + go func() { result <- old.Produce(oldCtx, scopedMessage("interrupted")) }() + select { + case <-scanner.paused: + case err := <-result: + t.Fatalf("producer exited before pause: %v", err) + case <-ctx.Done(): + t.Fatal(ctx.Err()) + } + claim := func(channel string) zset.ClaimedTask { + t.Helper() + tasks, err := store.ClaimRun(ctx, "interrupted", channel, time.Now().Add(time.Second), time.Now().Add(time.Minute), 1) + if err != nil || len(tasks) != 1 { + t.Fatalf("claim %s=%v err=%v", channel, tasks, err) + } + return tasks[0] + } + if n, err := store.BlockRunScopedAsFailed(ctx, claim("blocked"), zset.TaskOutcome{Code: 797, ErrCode: "S00004"}); err != nil || n != 1 { + t.Fatalf("blocked=%d err=%v", n, err) + } + var remaining zset.ClaimedTask + if backlog != "none" { + tasks := []zset.Task{{TaskID: "remaining", RunID: "interrupted", ChanKey: "other"}} + if n, err := store.EnqueueRunBatch(ctx, "interrupted", "other", tasks, time.Now()); err != nil || n != 1 { + t.Fatalf("enqueue=%d err=%v", n, err) + } + if backlog != "queued" { + remaining = claim("other") + } + if backlog == "waiting" { + if ok, err := store.FinalizeRunTask(ctx, remaining, zset.TaskOutcome{Retry: true}); err != nil || !ok { + t.Fatalf("defer=%t err=%v", ok, err) + } + } + } + before, err := store.GetRunMeta(ctx, "interrupted") + if err != nil || before.Status != "running" || before.ProducerDone || before.FailedCount != 1 { + t.Fatalf("blocked metadata=%+v err=%v", before, err) + } + fresh := testProducer(t, scannerFor([]Card{{ID: 3, ChanKey: "fresh"}}, 3), store, Config{ScanMode: mode}) + if err := fresh.Produce(ctx, scopedMessage("fresh")); !errors.Is(err, ErrAlreadyRunning) { + t.Fatalf("live producer lock must prevent reclamation: %v", err) + } + requireMeta(t, store, "interrupted", false, before.ActiveCount) + interrupt() + if err := <-result; !errors.Is(err, context.Canceled) { + t.Fatalf("interruption=%v", err) + } + lock, err := store.RedisClient().Do(ctx, "EXISTS", old.config.RunLockKey) + if err != nil || lock.Int() != 0 { + t.Fatalf("old lock=%v err=%v", lock, err) + } + err = fresh.Produce(ctx, scopedMessage("fresh")) + if backlog == "none" && err != nil || backlog != "none" && !errors.Is(err, ErrAlreadyRunning) { + t.Fatalf("fresh start with %s backlog: %v", backlog, err) + } + after, err := store.GetRunMeta(ctx, "interrupted") + if err != nil || !after.ProducerDone || after.Status != "running" || after.ActiveCount != before.ActiveCount || after.InflightCount != before.InflightCount || after.NextRoundCount != before.NextRoundCount || after.FailedCount != before.FailedCount { + t.Fatalf("reclaimed metadata=%+v before=%+v err=%v", after, before, err) + } + if backlog != "none" { + if _, err := store.GetRunMeta(ctx, "fresh"); !errors.Is(err, zset.ErrRunNotFound) { + t.Fatalf("backlog must prevent creating fresh run: %v", err) + } + if complete, err := store.CompleteRunIfIdle(ctx, "interrupted"); err != nil || complete { + t.Fatalf("premature completion=%t err=%v", complete, err) + } + if backlog == "waiting" { + if advanced, err := store.AdvanceRunRound(ctx, "interrupted"); err != nil || !advanced { + t.Fatalf("advance=%t err=%v", advanced, err) + } + } + if backlog != "leased" { + remaining = claim("other") + } + if ok, err := store.FinalizeRunTask(ctx, remaining, zset.TaskOutcome{Success: true}); err != nil || !ok { + t.Fatalf("drain=%t err=%v", ok, err) + } + } + if complete, err := store.CompleteRunIfIdle(ctx, "interrupted", time.Hour); err != nil || !complete { + t.Fatalf("normal completion=%t err=%v", complete, err) + } + closed, err := store.GetRunMeta(ctx, "interrupted") + if err != nil || closed.Status != "completed_with_failures" || closed.FailedCount != 1 || closed.ActiveCount != 0 || closed.InflightCount != 0 || closed.NextRoundCount != 0 { + t.Fatalf("closed metadata=%+v err=%v", closed, err) + } + metaKey, _ := store.RunMetaKey("interrupted") + failedKey, _ := store.FailedKey("interrupted") + stateKey, _ := store.TaskStateKey("interrupted") + channelsKey, _ := store.RunChannelsKey("interrupted") + for _, key := range []string{metaKey, failedKey, stateKey, channelsKey} { + ttl, err := store.RedisClient().Do(ctx, "PTTL", key) + if err != nil || ttl.Int64() <= 0 || ttl.Int64() > time.Hour.Milliseconds() { + t.Fatalf("retention %s=%v err=%v", key, ttl, err) + } + } + if backlog != "none" { + if err := fresh.Produce(ctx, scopedMessage("fresh")); err != nil { + t.Fatalf("fresh start after draining: %v", err) + } + } + requireMeta(t, store, "fresh", true, 1) + active, err := store.ActiveRuns(ctx) + if err != nil || len(active) != 1 || active[0] != "fresh" { + t.Fatalf("only closed run must leave active index: %v err=%v", active, err) + } + }) + } + } +} diff --git a/internal/producer/monthlyusagehistory/channel_producer.go b/internal/producer/monthlyusagehistory/channel_producer.go new file mode 100644 index 0000000..fd709d9 --- /dev/null +++ b/internal/producer/monthlyusagehistory/channel_producer.go @@ -0,0 +1,491 @@ +package monthlyusagehistory + +import ( + "context" + "errors" + "fmt" + "sort" + "sync" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +var errProducerDone = gerror.New("run production already closed") + +func producerStateError(state zset.ProducerState) error { + if state.Status == "completed" || state.Status == "completed_with_failures" { + return ErrRunFinished + } + if state.ProducerDone { + return errProducerDone + } + if state.Status != "producing" && state.Status != "running" { + return gerror.Newf("unexpected producer run status %q", state.Status) + } + return nil +} + +// produceChannels owns all per-attempt coordination. There is one worker per +// target, NOT a pool of four long-lived channel workers. SQL tokens and the +// cancellable admission gate never span sleeps or goroutine joins. +func (p *Producer) produceChannels(ctx context.Context, message model.TrafficUsageMonthlyHistoryStartMessage, scope *model.SyncScope, scanUpperID int64, startedAt time.Time, total *productionStats) error { + scanner := p.scanner.(channelScanner) + metadata, err := scanner.ListChannels(ctx) + if err != nil { + return gerror.Wrap(err, "initialize channel scan") + } + state, err := p.store.ReadProducerState(ctx, message.RunID, "") + if err != nil { + return err + } + if err = producerStateError(state); err != nil { + return err + } + targets := make(map[string]channelMetadata, len(metadata)) + known := make(map[string]bool, len(metadata)) + for _, channel := range metadata { + if channel.ChanKey == "" || !scope.Allows(channel.ChanKey) { + continue + } + if known[channel.ChanKey] { + return gerror.Newf("duplicate channel metadata %q", channel.ChanKey) + } + known[channel.ChanKey] = true + reason := "" + if channel.SyncEnabled { + reason = "sync_disabled" + } else if channel.TempKey == "" { + reason = "missing_channel_template" + } + if reason != "" { + total.skippedChannels++ + g.Log().Warningf(ctx, "event=monthly_usage_history_channel status=skipped runId=%s chanKey=%s reason=%s skipUnit=channel", message.RunID, channel.ChanKey, reason) + continue + } + targets[channel.ChanKey] = channel + } + if scope != nil { + for _, chanKey := range scope.Channels { + if !known[chanKey] { + total.skippedChannels++ + g.Log().Warningf(ctx, "event=monthly_usage_history_channel status=skipped runId=%s chanKey=%s reason=unknown_channel skipUnit=channel", message.RunID, chanKey) + } + } + } + g.Log().Infof(ctx, "event=monthly_usage_history_channel_scan status=started runId=%s targetChannels=%d scanUpperID=%d scanConcurrency=%d pageSize=%d skipUnit=channel", message.RunID, len(targets), scanUpperID, p.config.ScanConcurrency, p.config.PageSize) + keys := make([]string, 0, len(targets)) + for key := range targets { + keys = append(keys, key) + } + sort.Strings(keys) + finishObservation := p.startObservation(ctx, message.RunID, keys, startedAt) + defer finishObservation() + var ( + workers sync.WaitGroup + slots = make(chan struct{}, p.config.ScanConcurrency) + gate = make(chan struct{}, 1) + metrics = newScanActivity() + results = make([]*channelScanStats, len(keys)) + ) + pool, hasPool := scanner.(interface { + poolWait(context.Context) (int64, int64) + }) + var poolCount, poolNanos int64 + if hasPool { + poolCount, poolNanos = pool.poolWait(ctx) + } + workCtx, cancel := context.WithCancelCause(ctx) + defer cancel(nil) + for i, key := range keys { + stats := &channelScanStats{chanKey: key, lastSummary: time.Now()} + results[i] = stats + workers.Add(1) + go func(channel channelMetadata, stats *channelScanStats) { + defer workers.Done() + scanErr := p.scanOneChannel(workCtx, scanner, message, channel, scanUpperID, startedAt, slots, gate, metrics, stats) + if scanErr != nil { + cancel(scanErr) + } + reason := stats.stopReason + if reason == "" { + reason = "scan_complete" + } + if scanErr != nil { + reason = "cancelled" + } + p.observe(channel.ChanKey, reason, 0, false, true) + stats.log(workCtx, message.RunID, "ended", startedAt, scanErr) + }(targets[key], stats) + } + workers.Wait() + var summary channelScanStats + for _, stats := range results { + total.pages += stats.scanPages + total.scanned += stats.eligible + total.eligible += stats.eligible + total.enqueued += stats.added + total.stoppedTasks += stats.rejected + if stats.eligible > 0 { + total.channels[stats.chanKey] = struct{}{} + } + summary.merge(stats) + } + metrics.log(ctx, message.RunID, &summary) + if hasPool { + count, nanos := pool.poolWait(ctx) + g.Log().Infof(ctx, "event=monthly_usage_history_scan_pool runId=%s sharedPoolWaitCountDelta=%d sharedPoolWaitMsDelta=%d attribution=shared_pool", message.RunID, count-poolCount, (nanos-poolNanos)/int64(time.Millisecond)) + } + return context.Cause(workCtx) +} + +func acquireContext(ctx context.Context, token chan struct{}) error { + select { + case <-ctx.Done(): + return ctx.Err() + case token <- struct{}{}: + if err := ctx.Err(); err != nil { + <-token + return err + } + return nil + } +} + +func (p *Producer) scanOneChannel(ctx context.Context, scanner channelScanner, message model.TrafficUsageMonthlyHistoryStartMessage, channel channelMetadata, scanUpperID int64, startedAt time.Time, slots, gate chan struct{}, metrics *scanActivity, stats *channelScanStats) error { + for stats.cursor < scanUpperID { + // Detect a blocked/closed run before querying, including empty channels. + p.observe(channel.ChanKey, "redis_read", 0, false, false) + state, err := p.store.ReadProducerState(ctx, message.RunID, channel.ChanKey) + if err != nil { + return err + } + if err = producerStateError(state); err != nil { + return err + } + if state.Blocked { + // No page was read: the unknown unscanned remainder has no card + // count. Never turn it into a fabricated success/failure total. + stats.stopReason = "blocked_unscanned_remainder" + return nil + } + p.observe(channel.ChanKey, "sql_slot", 0, false, false) + waitStart := time.Now() + if err = acquireContext(ctx, slots); err != nil { + return err + } + stats.slotWait += time.Since(waitStart) + metrics.begin() + p.observe(channel.ChanKey, "sql_query", 0, false, false) + queryStart := time.Now() + cards, scanErr := scanner.ScanChannel(ctx, channel.ChanKey, stats.cursor, p.config.PageSize) + elapsed := time.Since(queryStart) + metrics.end() + <-slots // release immediately after results have been materialized + stats.sqlTime += elapsed + stats.sqlLatency.observe(elapsed) + stats.scanPages++ + if scanErr != nil { + return gerror.Wrapf(scanErr, "scan channel %s after %d", channel.ChanKey, stats.cursor) + } + if len(cards) > p.config.PageSize { + return gerror.New("scanner exceeded page size") + } + if len(cards) == 0 { + stats.stopReason = "empty_result" + return nil + } + var ( + pageCursor = stats.cursor + overUpper bool + tasks = make([]zset.Task, 0, len(cards)) + ) + for i, card := range cards { + if card.ChanKey != channel.ChanKey || card.ID <= pageCursor { + return gerror.New("channel scanner returned unordered or foreign cards") + } + if card.ID > scanUpperID { + stats.overUpper += int64(len(cards) - i) + overUpper = true + break + } + pageCursor = card.ID + tasks = append(tasks, zset.Task{TaskID: zset.StableTaskID(message.RunID, message.Operation, card.ID), RunID: message.RunID, Operation: message.Operation, QueryMonth: message.QueryMonth, ChanKey: channel.ChanKey, TempKey: channel.TempKey, Operator: channel.Operator, CardID: card.ID, ICCID: card.ICCID, MSISDN: card.MSISDN, IMSI: card.IMSI, PackageCode: card.PackageCode}) + } + p.observe(channel.ChanKey, "page_ready", len(tasks), false, false) + stats.eligible += int64(len(tasks)) + blocked, err := p.admitChannelPage(ctx, message.RunID, tasks, startedAt, gate, stats) + if err != nil { + return err // page cursor is not committed, even after a partial write + } + if blocked { + stats.stopReason = "blocked" + return nil + } + stats.cursor = pageCursor + stats.progress(ctx, message.RunID, startedAt) + if overUpper || stats.cursor >= scanUpperID { + stats.stopReason = "upper_bound" + return nil + } + // Do not use a short page as EOF: query-time eligibility may change. + } + return ctx.Err() +} + +// admitChannelPage admits only new TaskIDs, in capacity-sized slices. Each +// worker retains at most this page. No capacity reservation survives the gate. +func (p *Producer) admitChannelPage(ctx context.Context, runID string, tasks []zset.Task, startedAt time.Time, gate chan struct{}, stats *channelScanStats) (bool, error) { + for len(tasks) > 0 { + p.observe(stats.chanKey, "admission_gate", len(tasks), false, false) + gateStart := time.Now() + if err := acquireContext(ctx, gate); err != nil { + return false, err + } + stats.gateWait += time.Since(gateStart) + blocked, waitKind, remaining, err := p.admitChannelSlice(ctx, runID, tasks, startedAt, stats) + <-gate + if err != nil || blocked { + return blocked, err + } + tasks = remaining + p.observe(stats.chanKey, "page_ready", len(tasks), false, false) + if waitKind != "" { + p.observe(stats.chanKey, waitKind+"_capacity", len(tasks), false, false) + waitStart := time.Now() + err = sleepContext(ctx, p.config.PollInterval) + if waitKind == "global" { + stats.globalWait += time.Since(waitStart) + } else { + stats.channelWait += time.Since(waitStart) + } + stats.progress(ctx, runID, startedAt) + if err != nil { + return false, err + } + } + } + return false, ctx.Err() +} + +// Caller holds the short admission gate; there is no sleep in this method. +func (p *Producer) admitChannelSlice(ctx context.Context, runID string, tasks []zset.Task, startedAt time.Time, stats *channelScanStats) (blocked bool, waitKind string, remaining []zset.Task, err error) { + p.observe(stats.chanKey, "redis_read", len(tasks), false, false) + readStart := time.Now() + state, err := p.store.ReadProducerState(ctx, runID, stats.chanKey) + if err != nil { + return false, "", tasks, err + } + if err = producerStateError(state); err != nil { + return false, "", tasks, err + } + existing, err := p.store.ExistingRunTasks(ctx, runID, tasks) + stats.stateTime += time.Since(readStart) + if err != nil { + return false, "", tasks, err + } + pending := tasks[:0] + for i, exists := range existing { + if exists { + stats.duplicates++ + } else { + pending = append(pending, tasks[i]) + } + } + tasks = pending + p.observe(stats.chanKey, "redis_read", len(tasks), false, false) + if state.Blocked { + stats.rejected += int64(len(tasks)) + return true, "", nil, nil + } + if len(tasks) == 0 { + return false, "", nil, nil + } + space := int64(len(tasks)) + if limit := p.config.GlobalHighWaterMark; limit > 0 { + space = min(space, limit-state.ActiveCount) // inflight is already included + if space <= 0 { + return false, "global", tasks, nil + } + } + if limit := p.config.ChannelHighWaterMark; limit > 0 { + key, keyErr := p.store.RoundActiveKey(runID, state.CurrentRound, stats.chanKey) + if keyErr != nil { + return false, "", tasks, keyErr + } + readStart = time.Now() + size, sizeErr := p.store.SizeKey(ctx, key) + stats.stateTime += time.Since(readStart) + if sizeErr != nil { + return false, "", tasks, sizeErr + } + space = min(space, limit-size) + if space <= 0 { + return false, "channel", tasks, nil + } + } + batch := tasks[:int(space)] + p.observe(stats.chanKey, "redis_write", len(tasks), false, false) + enqueueStart := time.Now() + added, err := p.store.EnqueueRunBatch(ctx, runID, stats.chanKey, batch, time.Now()) + stats.enqueueTime += time.Since(enqueueStart) + stats.enqueueBatches++ + if err != nil { + return false, "", tasks, err // uncertain responses require idempotent rescan + } + if added < 0 || added > len(batch) { + return false, "", tasks, gerror.New("invalid enqueue count") + } + stats.added += int64(added) + p.observe(stats.chanKey, "redis_write", len(tasks)-len(batch), added > 0, false) + if added > 0 && !stats.firstEnqueueRecorded { + stats.firstEnqueueRecorded = true + stats.firstEnqueue = time.Since(startedAt) + stats.log(ctx, runID, "first_enqueue", startedAt, nil) + } + if added < len(batch) { + readStart = time.Now() + confirmed, checkErr := p.store.ExistingRunTasks(ctx, runID, batch) + if checkErr != nil { + return false, "", tasks, checkErr + } + state, checkErr = p.store.ReadProducerState(ctx, runID, stats.chanKey) + stats.stateTime += time.Since(readStart) + if checkErr != nil { + return false, "", tasks, checkErr + } + if checkErr = producerStateError(state); checkErr != nil { + return false, "", tasks, checkErr + } + present := 0 + for _, exists := range confirmed { + if exists { + present++ + } + } + if present < added { + return false, "", tasks, gerror.New("enqueue count exceeds confirmed task states") + } + stats.duplicates += int64(present - added) + if state.Blocked { + stats.rejected += int64(len(tasks) - present) + return true, "", nil, nil + } + if present != len(batch) { + return false, "", tasks, gerror.Newf("unexplained enqueue rejection: added=%d submitted=%d confirmed=%d", added, len(batch), present) + } + } + return false, "", tasks[len(batch):], nil +} + +// Retained only to keep errors.Is semantics visible at the call site. +func isProductionClosed(err error) bool { return errors.Is(err, errProducerDone) } + +func durationMS(d time.Duration) float64 { return float64(d) / float64(time.Millisecond) } + +// latencyHistogram is fixed-size. Reported percentiles are bucket upper bounds, +// never exact samples or a server-side SLA; the last bucket is unbounded. +var latencyBounds = [...]time.Duration{time.Millisecond, 2 * time.Millisecond, 5 * time.Millisecond, 10 * time.Millisecond, 20 * time.Millisecond, 50 * time.Millisecond, 100 * time.Millisecond, 200 * time.Millisecond, 500 * time.Millisecond, time.Second, 2 * time.Second, 5 * time.Second, 10 * time.Second, 30 * time.Second, time.Minute} + +type latencyHistogram struct{ counts [16]int64 } + +func (h *latencyHistogram) observe(d time.Duration) { + i := 0 + for i < len(latencyBounds) && d > latencyBounds[i] { + i++ + } + h.counts[i]++ +} +func (h *latencyHistogram) percentile(percent int64) string { + var total, cumulative int64 + for _, count := range h.counts { + total += count + } + if total == 0 { + return "n/a" + } + target := (total*percent + 99) / 100 + for i, count := range h.counts { + cumulative += count + if cumulative >= target { + if i == len(latencyBounds) { + return ">60000" + } + return fmt.Sprintf("%.0f", durationMS(latencyBounds[i])) + } + } + return "n/a" +} + +type channelScanStats struct { + firstEnqueueRecorded bool + chanKey, stopReason string + cursor, scanPages, eligible, overUpper, enqueueBatches, added, duplicates, rejected int64 + sqlTime, slotWait, gateWait, globalWait, channelWait, stateTime, enqueueTime, firstEnqueue time.Duration + sqlLatency latencyHistogram + lastSummary time.Time +} + +func (s *channelScanStats) log(ctx context.Context, runID, status string, startedAt time.Time, err error) { + errorText := "" + if err != nil { + errorText = err.Error() + } + g.Log().Infof(ctx, "event=monthly_usage_history_channel_scan status=%s runId=%s chanKey=%s cursor=%d scanPages=%d eligible=%d overUpper=%d enqueueBatches=%d added=%d duplicates=%d rejected=%d sqlMs=%.3f sqlP50UpperMs=%s sqlP95UpperMs=%s slotWaitMs=%.3f admissionWaitMs=%.3f globalCapacityWaitMs=%.3f channelCapacityWaitMs=%.3f stateReadMs=%.3f enqueueMs=%.3f firstEnqueueMs=%.3f wallMs=%d stopReason=%s error=%q", status, runID, s.chanKey, s.cursor, s.scanPages, s.eligible, s.overUpper, s.enqueueBatches, s.added, s.duplicates, s.rejected, durationMS(s.sqlTime), s.sqlLatency.percentile(50), s.sqlLatency.percentile(95), durationMS(s.slotWait), durationMS(s.gateWait), durationMS(s.globalWait), durationMS(s.channelWait), durationMS(s.stateTime), durationMS(s.enqueueTime), durationMS(s.firstEnqueue), time.Since(startedAt).Milliseconds(), s.stopReason, errorText) +} +func (s *channelScanStats) progress(ctx context.Context, runID string, startedAt time.Time) { + if time.Since(s.lastSummary) >= time.Minute { + s.lastSummary = time.Now() + s.log(ctx, runID, "progress", startedAt, nil) + } +} +func (s *channelScanStats) merge(other *channelScanStats) { + s.scanPages += other.scanPages + s.eligible += other.eligible + s.overUpper += other.overUpper + s.enqueueBatches += other.enqueueBatches + s.added += other.added + s.duplicates += other.duplicates + s.rejected += other.rejected + s.sqlTime += other.sqlTime + s.slotWait += other.slotWait + s.gateWait += other.gateWait + s.globalWait += other.globalWait + s.channelWait += other.channelWait + s.stateTime += other.stateTime + s.enqueueTime += other.enqueueTime + for i, count := range other.sqlLatency.counts { + s.sqlLatency.counts[i] += count + } +} + +type scanActivity struct { + mu sync.Mutex + started, changed time.Time + running, peak int + area time.Duration +} + +func newScanActivity() *scanActivity { + now := time.Now() + return &scanActivity{started: now, changed: now} +} +func (s *scanActivity) change(delta int) { + s.mu.Lock() + defer s.mu.Unlock() + now := time.Now() + s.area += time.Duration(s.running) * now.Sub(s.changed) + s.changed = now + s.running += delta + s.peak = max(s.peak, s.running) +} +func (s *scanActivity) begin() { s.change(1) } +func (s *scanActivity) end() { s.change(-1) } +func (s *scanActivity) log(ctx context.Context, runID string, stats *channelScanStats) { + wall := time.Since(s.started) + g.Log().Infof(ctx, "event=monthly_usage_history_scan_summary runId=%s wallMs=%d sqlInFlightPeak=%d sqlInFlightAvg=%.4f scanPages=%d enqueueBatches=%d eligible=%d overUpper=%d added=%d duplicates=%d rejected=%d sqlCumulativeMs=%.3f sqlP50UpperMs=%s sqlP95UpperMs=%s slotWaitCumulativeMs=%.3f admissionWaitCumulativeMs=%.3f globalCapacityWaitCumulativeMs=%.3f channelCapacityWaitCumulativeMs=%.3f stateReadCumulativeMs=%.3f enqueueCumulativeMs=%.3f", runID, wall.Milliseconds(), s.peak, float64(s.area)/float64(wall), stats.scanPages, stats.enqueueBatches, stats.eligible, stats.overUpper, stats.added, stats.duplicates, stats.rejected, durationMS(stats.sqlTime), stats.sqlLatency.percentile(50), stats.sqlLatency.percentile(95), durationMS(stats.slotWait), durationMS(stats.gateWait), durationMS(stats.globalWait), durationMS(stats.channelWait), durationMS(stats.stateTime), durationMS(stats.enqueueTime)) +} diff --git a/internal/producer/monthlyusagehistory/channel_producer_test.go b/internal/producer/monthlyusagehistory/channel_producer_test.go new file mode 100644 index 0000000..54349c8 --- /dev/null +++ b/internal/producer/monthlyusagehistory/channel_producer_test.go @@ -0,0 +1,539 @@ +package monthlyusagehistory + +import ( + "context" + "errors" + "fmt" + "reflect" + "sort" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" + "github.com/gogf/gf/v2/container/gvar" +) + +// datasetScanner applies the SQL eligibility predicate before LIMIT; metadata +// eligibility is applied by the producer. Optional hooks simulate SQL latency, +// live additions and failures without accessing an application database. +type datasetScanner struct { + cards []Card + channels []channelMetadata + upper int64 + hook func(context.Context, string, int64) error + queries atomic.Int32 + lists atomic.Int32 + inflight atomic.Int32 + peak atomic.Int32 +} + +func (s *datasetScanner) SnapshotMaxID(context.Context) (int64, error) { return s.upper, nil } +func (s *datasetScanner) ListChannels(context.Context) ([]channelMetadata, error) { + s.lists.Add(1) + return s.channels, nil +} +func (s *datasetScanner) Scan(ctx context.Context, last, upper int64, size int) ([]Card, error) { + var rows []Card + for _, card := range s.cards { + if card.ID > last && card.ID <= upper { + for _, channel := range s.channels { + if channel.ChanKey == card.ChanKey { + card.TempKey = channel.TempKey + card.SyncEnabled = channel.SyncEnabled + card.Operator = channel.Operator + } + } + rows = append(rows, card) + if len(rows) == size { + break + } + } + } + return rows, nil +} +func (s *datasetScanner) ScanChannel(ctx context.Context, key string, last int64, size int) ([]Card, error) { + s.queries.Add(1) + running := s.inflight.Add(1) + defer s.inflight.Add(-1) + for { + old := s.peak.Load() + if running <= old || s.peak.CompareAndSwap(old, running) { + break + } + } + if s.hook != nil { + if err := s.hook(ctx, key, last); err != nil { + return nil, err + } + } + var rows []Card + for _, card := range s.cards { + if card.ChanKey == key && card.ID > last { + rows = append(rows, card) + if len(rows) == size { + break + } + } + } + return rows, nil +} +func scannerFor(cards []Card, upper int64) *datasetScanner { + keys := map[string]bool{} + s := &datasetScanner{cards: cards, upper: upper} + for _, card := range cards { + if card.ChanKey != "" && !keys[card.ChanKey] { + keys[card.ChanKey] = true + s.channels = append(s.channels, channelMetadata{ChanKey: card.ChanKey, TempKey: "sim_yd_v5", Operator: 1}) + } + } + return s +} +func testStore(t *testing.T) *zset.Store { + t.Helper() + store, err := zset.New(testutil.Redis(t), "flow:") + if err != nil { + t.Fatal(err) + } + return store +} +func testProducer(t *testing.T, scanner CardScanner, store *zset.Store, config Config) *Producer { + t.Helper() + config.RoundRetryEnabled = true + if config.ScanMode == "" { + config.ScanMode = "channel" + } + p, err := New(scanner, store, config) + if err != nil { + t.Fatal(err) + } + return p +} +func contextForTest(t *testing.T) context.Context { + t.Helper() + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + t.Cleanup(cancel) + return ctx +} +func taskIDs(t *testing.T, store *zset.Store, run string) []string { + t.Helper() + key, _ := store.TaskStateKey(run) + result, err := store.RedisClient().Do(context.Background(), "HKEYS", key) + if err != nil { + t.Fatal(err) + } + ids := result.Strings() + sort.Strings(ids) + return ids +} +func requireMeta(t *testing.T, store *zset.Store, run string, done bool, active int64) { + t.Helper() + meta, err := store.GetRunMeta(context.Background(), run) + if err != nil { + t.Fatal(err) + } + if meta.ProducerDone != done || meta.ActiveCount != active { + t.Fatalf("meta=%+v, want done=%t active=%d", meta, done, active) + } +} +func waitUntil(t *testing.T, ctx context.Context, condition func() bool) { + t.Helper() + for !condition() { + select { + case <-ctx.Done(): + t.Fatal(ctx.Err()) + case <-time.After(time.Millisecond): + } + } +} + +func TestChannelModeSettings(t *testing.T) { + store, _ := zset.New(&producerRedis{}, "flow:") + p, err := New(&fakeScanner{}, store, Config{}) + if err != nil || p.config.ScanMode != "serial" || p.config.ScanConcurrency != 4 { + t.Fatalf("defaults: p=%+v err=%v", p, err) + } + for _, config := range []Config{{ScanMode: "parallel"}, {ScanConcurrency: -1}, {ScanMode: "channel"}} { + if _, err = New(&fakeScanner{}, store, config); err == nil { + t.Fatalf("accepted invalid config/capability %+v", config) + } + } +} + +func TestChannelScanFiniteMemoryBoundary(t *testing.T) { + for _, test := range []struct { + name string + upper int64 + cards []Card + ids []int64 + queries int32 + }{ + {"zero upper", 0, []Card{{ID: 1, ChanKey: "a"}}, nil, 0}, + {"exact page", 4, []Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "a"}, {ID: 3, ChanKey: "a"}, {ID: 4, ChanKey: "a"}}, []int64{1, 2, 3, 4}, 2}, + {"tail then empty", 9, []Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "a"}, {ID: 3, ChanKey: "a"}}, []int64{1, 2, 3}, 3}, + {"first outside", 3, []Card{{ID: 4, ChanKey: "a"}, {ID: 5, ChanKey: "a"}}, nil, 1}, + {"mixed final page", 3, []Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "a"}, {ID: 3, ChanKey: "a"}, {ID: 4, ChanKey: "a"}, {ID: 5, ChanKey: "a"}}, []int64{1, 2, 3}, 2}, + {"empty channel", 9, nil, nil, 1}, + } { + t.Run(test.name, func(t *testing.T) { + store := testStore(t) + scanner := scannerFor(test.cards, test.upper) + if len(scanner.channels) == 0 { + scanner.channels = []channelMetadata{{ChanKey: "a", TempKey: "sim_yd_v5"}} + } + p := testProducer(t, scanner, store, Config{PageSize: 2}) + if err := p.Produce(contextForTest(t), scopedMessage("boundary")); err != nil { + t.Fatal(err) + } + var want []string + for _, id := range test.ids { + want = append(want, zset.StableTaskID("boundary", operation, id)) + } + sort.Strings(want) + got := taskIDs(t, store, "boundary") + if len(got) != len(want) || (len(want) > 0 && !reflect.DeepEqual(got, want)) { + t.Fatalf("got=%v want=%v", got, want) + } + if scanner.queries.Load() != test.queries { + t.Fatalf("queries=%d want=%d", scanner.queries.Load(), test.queries) + } + requireMeta(t, store, "boundary", true, int64(len(want))) + }) + } +} + +func TestChannelScanLiveHigherIDsDoNotExtendRun(t *testing.T) { + store := testStore(t) + scanner := scannerFor([]Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "a"}, {ID: 3, ChanKey: "a"}}, 3) + scanner.hook = func(_ context.Context, _ string, last int64) error { + if last > 0 { + scanner.cards = append(scanner.cards, Card{ID: 100 + last, ChanKey: "a"}) + } + return nil + } + p := testProducer(t, scanner, store, Config{PageSize: 2}) + if err := p.Produce(contextForTest(t), scopedMessage("live")); err != nil { + t.Fatal(err) + } + if len(taskIDs(t, store, "live")) != 3 || scanner.queries.Load() != 2 { + t.Fatal("higher IDs extended finite range") + } +} + +func TestChannelScanSerialTaskIDSetParity(t *testing.T) { + for _, scope := range [][]string{nil, {"a", "b"}, {"disabled", "unknown"}} { + t.Run(fmt.Sprint(scope), func(t *testing.T) { + store := testStore(t) + cards := []Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "b"}, {ID: 3, ChanKey: "a"}, {ID: 4, ChanKey: "disabled"}, {ID: 5, ChanKey: "no-template"}, {ID: 6, ChanKey: "unknown"}, {ID: 7, ChanKey: "b"}, {ID: 8, ChanKey: "a"}} + var sets [][]string + for _, mode := range []string{"serial", "channel"} { + scanner := scannerFor(cards, 7) + scanner.channels = []channelMetadata{{ChanKey: "a", TempKey: "sim_yd_v5", Operator: 1}, {ChanKey: "b", TempKey: "sim_dx_v1", Operator: 2}, {ChanKey: "disabled", TempKey: "sim_yd_v5", SyncEnabled: true}, {ChanKey: "no-template"}} + p := testProducer(t, scanner, store, Config{ScanMode: mode, PageSize: 2}) + run := "parity-" + mode + if err := p.Produce(contextForTest(t), scopedMessage(run, scope...)); err != nil { + t.Fatal(err) + } + ids := taskIDs(t, store, run) + for i, id := range ids { + ids[i] = strings.TrimPrefix(id, run+":") + } + sets = append(sets, ids) + if mode == "channel" && scanner.lists.Load() != 1 { + t.Fatal("metadata not loaded once") + } + _, _ = store.RedisClient().Do(context.Background(), "SREM", store.ActiveRunsKey(), run) + } + if !reflect.DeepEqual(sets[0], sets[1]) { + t.Fatalf("serial=%v channel=%v", sets[0], sets[1]) + } + }) + } +} + +func TestChannelSQLSlotsReleasedWhileChannelsWait(t *testing.T) { + store := testStore(t) + ctx := contextForTest(t) + var cards []Card + for i := 0; i < 9; i++ { + cards = append(cards, Card{ID: int64(i + 1), ChanKey: fmt.Sprint(i)}, Card{ID: int64(i + 10), ChanKey: fmt.Sprint(i)}) + } + sort.Slice(cards, func(i, j int) bool { return cards[i].ID < cards[j].ID }) + scanner := scannerFor(cards, 100) + scanner.hook = func(ctx context.Context, _ string, _ int64) error { return sleepContext(ctx, 5*time.Millisecond) } + p := testProducer(t, scanner, store, Config{PageSize: 2, ScanConcurrency: 4, ChannelHighWaterMark: 1, GlobalHighWaterMark: 100, PollInterval: time.Millisecond}) + result := make(chan error, 1) + go func() { result <- p.Produce(ctx, scopedMessage("isolation")) }() + waitUntil(t, ctx, func() bool { return len(taskIDs(t, store, "isolation")) == 9 }) + if peak := scanner.peak.Load(); peak > 4 || peak < 2 { + t.Fatalf("SQL peak=%d", peak) + } + // All nine channels got a first task despite the first four waiting at + // channel capacity. SQL slots are free while every channel is paused. + if scanner.inflight.Load() != 0 { + t.Fatal("SQL tokens held during channel backpressure") + } + for _, channel := range scanner.channels { + if _, err := store.BlockRunScoped(ctx, "isolation", channel.ChanKey); err != nil { + t.Fatal(err) + } + } + if err := <-result; err != nil { + t.Fatal(err) + } + meta, err := store.GetRunMeta(ctx, "isolation") + if err != nil || !meta.ProducerDone { + t.Fatalf("meta=%+v err=%v", meta, err) + } +} + +func TestChannelRemainingCapacityAndDuplicateAdmission(t *testing.T) { + store := testStore(t) + ctx := contextForTest(t) + scanner := scannerFor(nil, 0) + p := testProducer(t, scanner, store, Config{GlobalHighWaterMark: 3, ChannelHighWaterMark: 2, PollInterval: time.Millisecond}) + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "capacity", Status: "producing"}); err != nil { + t.Fatal(err) + } + // Existing success and waiting states still deduplicate and consume no new + // capacity. Active includes leases: inflight must not be added again. + stateKey, _ := store.TaskStateKey("capacity") + metaKey, _ := store.RunMetaKey("capacity") + activeKey, _ := store.RoundActiveKey("capacity", 1, "a") + _, _ = store.RedisClient().Do(ctx, "HSET", stateKey, "one", `{"status":"success"}`, "two", `{"status":"waiting"}`) + _, _ = store.RedisClient().Do(ctx, "HSET", metaKey, "activeCount", 2, "inflightCount", 2) + stats := &channelScanStats{chanKey: "a", lastSummary: time.Now()} + tasks := []zset.Task{{TaskID: "one", RunID: "capacity", ChanKey: "a"}, {TaskID: "two", RunID: "capacity", ChanKey: "a"}, {TaskID: "three", RunID: "capacity", ChanKey: "a"}, {TaskID: "four", RunID: "capacity", ChanKey: "a"}} + gate := make(chan struct{}, 1) + result := make(chan error, 1) + go func() { _, err := p.admitChannelPage(ctx, "capacity", tasks, time.Now(), gate, stats); result <- err }() + waitUntil(t, ctx, func() bool { meta, e := store.GetRunMeta(ctx, "capacity"); return e == nil && meta.ActiveCount == 3 }) + if size, err := store.SizeKey(ctx, activeKey); err != nil || size != 1 { + t.Fatalf("initial slice size=%d err=%v", size, err) + } + _, _ = store.RedisClient().Do(ctx, "HSET", metaKey, "activeCount", 0, "inflightCount", 0) + if err := <-result; err != nil { + t.Fatal(err) + } + if stats.duplicates != 2 || stats.added != 2 || stats.enqueueBatches != 2 || stats.globalWait == 0 { + t.Fatalf("stats=%+v", stats) + } +} + +// faultRedis forwards to a real isolated Redis, optionally returning a lost +// response after Lua succeeded. Never mocks the Lua semantics under test. +type faultRedis struct { + zset.RedisClient + failEnqueue atomic.Int32 + failRead atomic.Bool + failRenew atomic.Bool + lostResponse bool + reject bool +} + +func (r *faultRedis) Do(ctx context.Context, command string, args ...any) (*gvar.Var, error) { + if r.failRead.Load() && command == "HMGET" { + return nil, errors.New("Redis read failure") + } + return r.RedisClient.Do(ctx, command, args...) +} +func (r *faultRedis) Eval(ctx context.Context, script string, n int64, keys []string, args []any) (*gvar.Var, error) { + if r.failRenew.Load() && strings.Contains(script, "PEXPIRE") && strings.Contains(script, "owner ~=") { + return gvar.New(0), nil + } + if n == 4 && strings.Contains(script, "HEXISTS") { + if r.reject { + return gvar.New(0), nil + } + if r.failEnqueue.CompareAndSwap(1, 0) { + if r.lostResponse { + if _, err := r.RedisClient.Eval(ctx, script, n, keys, args); err != nil { + return nil, err + } + } + return nil, errors.New("enqueue response unavailable") + } + } + return r.RedisClient.Eval(ctx, script, n, keys, args) +} + +func TestChannelRescanPartialAndUncertainWrites(t *testing.T) { + for _, uncertain := range []bool{false, true} { + t.Run(fmt.Sprint(uncertain), func(t *testing.T) { + client := &faultRedis{RedisClient: testutil.Redis(t), lostResponse: uncertain} + store, _ := zset.New(client, "flow:") + scanner := scannerFor([]Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "a"}, {ID: 3, ChanKey: "a"}, {ID: 4, ChanKey: "a"}}, 4) + // Fail the second page: first page must survive; uncertain success is + // confirmed only on a same-Run rescan, with stable IDs and no double count. + scanner.hook = func(_ context.Context, _ string, last int64) error { + if last == 2 { + client.failEnqueue.Store(1) + } + return nil + } + p := testProducer(t, scanner, store, Config{PageSize: 2}) + if err := p.Produce(contextForTest(t), scopedMessage("rescan")); err == nil { + t.Fatal("expected enqueue failure") + } + active := int64(2) + if uncertain { + active = 4 + } + requireMeta(t, store, "rescan", false, active) + scanner.hook = nil + if err := p.Produce(contextForTest(t), scopedMessage("rescan")); err != nil { + t.Fatal(err) + } + requireMeta(t, store, "rescan", true, 4) + if len(taskIDs(t, store, "rescan")) != 4 { + t.Fatal("wrong stable task set") + } + before := scanner.queries.Load() + if err := p.Produce(contextForTest(t), scopedMessage("rescan")); err != nil { + t.Fatal(err) + } + if before != scanner.queries.Load() { + t.Fatal("producerDone redelivery rescanned cards") + } + }) + } +} + +func TestChannelFailureCancelsAllWorkersWithoutProducerDone(t *testing.T) { + for _, kind := range []string{"sql", "redis", "unexplained_rejection", "lease"} { + t.Run(kind, func(t *testing.T) { + client := &faultRedis{RedisClient: testutil.Redis(t)} + store, _ := zset.New(client, "flow:") + cards := []Card{{ID: 1, ChanKey: "a"}, {ID: 2, ChanKey: "b"}, {ID: 3, ChanKey: "c"}} + scanner := scannerFor(cards, 10) + allStarted := make(chan struct{}) + var started atomic.Int32 + var once sync.Once + scanner.hook = func(ctx context.Context, key string, _ int64) error { + if started.Add(1) >= 3 { + once.Do(func() { close(allStarted) }) + } + select { + case <-allStarted: + case <-ctx.Done(): + return ctx.Err() + } + if kind == "lease" { + client.failRenew.Store(true) + <-ctx.Done() + return ctx.Err() + } + if key == "a" { + switch kind { + case "sql": + return errors.New("SQL failed") + case "redis": + client.failRead.Store(true) + case "unexplained_rejection": + client.reject = true + } + return nil + } + <-ctx.Done() + return ctx.Err() + } + p := testProducer(t, scanner, store, Config{PageSize: 2, RunLockTTL: 90 * time.Millisecond}) + if err := p.Produce(contextForTest(t), scopedMessage("failure")); err == nil { + t.Fatal("expected fatal failure") + } + if scanner.inflight.Load() != 0 { + t.Fatal("scanner goroutines leaked") + } + meta, err := store.GetRunMeta(context.Background(), "failure") + if err != nil || meta.ProducerDone { + t.Fatalf("partial scan marked done: %+v err=%v", meta, err) + } + }) + } +} + +func TestChannelScanStateStopsAndNoTarget(t *testing.T) { + for _, kind := range []string{"terminal", "producerDone", "blocked", "missing", "no-target", "no-cards"} { + t.Run(kind, func(t *testing.T) { + store := testStore(t) + scanner := scannerFor([]Card{{ID: 1, ChanKey: "a"}}, 2) + if kind == "no-target" { + scanner.channels[0].SyncEnabled = true + } + if kind == "no-cards" { + scanner.cards = nil + } + p := testProducer(t, scanner, store, Config{}) + if kind != "no-target" && kind != "no-cards" { + if err := store.CreateRun(context.Background(), zset.RunMeta{RunID: "stop", Status: "producing"}); err != nil { + t.Fatal(err) + } + key, _ := store.RunMetaKey("stop") + switch kind { + case "terminal": + _, _ = store.RedisClient().Do(context.Background(), "HSET", key, "status", "completed") + case "producerDone": + if err := store.MarkProducerDone(context.Background(), "stop"); err != nil { + t.Fatal(err) + } + case "blocked": + _, _ = store.BlockRunScoped(context.Background(), "stop", "a") + case "missing": + scanner.hook = func(ctx context.Context, _ string, _ int64) error { + _, err := store.RedisClient().Do(ctx, "DEL", key) + return err + } + } + } + err := p.Produce(contextForTest(t), scopedMessage("stop")) + if kind == "terminal" { + if !errors.Is(err, ErrRunFinished) { + t.Fatalf("err=%v", err) + } + } else if kind == "missing" { + if !errors.Is(err, zset.ErrRunNotFound) { + t.Fatalf("err=%v", err) + } + } else if err != nil { + t.Fatal(err) + } + if len(taskIDs(t, store, "stop")) != 0 { + t.Fatal("stopped/empty run enqueued tasks") + } + }) + } +} + +func TestChannelOldNonRunPathRemainsSerial(t *testing.T) { + redis := &producerRedis{batches: make(map[string]int)} + store, _ := zset.New(redis, "flow:") + scanner := scannerFor([]Card{{ID: 1, ChanKey: "a"}}, 1) + p, err := New(scanner, store, Config{ScanMode: "channel", RoundRetryEnabled: false}) + if err != nil { + t.Fatal(err) + } + if err = p.Produce(context.Background(), scopedMessage("legacy")); err != nil { + t.Fatal(err) + } + if scanner.queries.Load() != 0 || scanner.lists.Load() != 0 { + t.Fatal("non-Run path invoked channel scanner") + } +} + +const operation = FullSyncOperation + +type fakeScanner struct{} + +func (*fakeScanner) SnapshotMaxID(context.Context) (int64, error) { return 0, nil } +func (*fakeScanner) Scan(context.Context, int64, int64, int) ([]Card, error) { return nil, nil } +func scopedMessage(run string, channels ...string) model.TrafficUsageMonthlyHistoryStartMessage { + m := model.TrafficUsageMonthlyHistoryStartMessage{RunID: run, QueryMonth: "202609", Operation: FullSyncOperation} + if len(channels) > 0 { + m.Scope = &model.SyncScope{Channels: channels} + } + return m +} diff --git a/internal/producer/monthlyusagehistory/channel_scanner.go b/internal/producer/monthlyusagehistory/channel_scanner.go new file mode 100644 index 0000000..54299e9 --- /dev/null +++ b/internal/producer/monthlyusagehistory/channel_scanner.go @@ -0,0 +1,81 @@ +package monthlyusagehistory + +import ( + "context" + + "github.com/gogf/gf/v2/errors/gerror" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +// channelScanner augments, rather than replaces, the legacy scanner contract. +// ListChannels is called once before any workers start. ScanChannel must not +// mutate the legacy DatabaseScanner.channelMetadata cache. +type channelScanner interface { + ListChannels(context.Context) ([]channelMetadata, error) + ScanChannel(context.Context, string, int64, int) ([]Card, error) +} + +// This exact MySQL query shape is a release gate. Do not silently remove hints, +// add an SQL upper bound, or move eligibility filtering after LIMIT. +const channelPageSQL = `SELECT c.id, c.iccid, c.msisdn, c.imsi, c.chan_key, c.current_package_code +FROM t_sim_card_info AS c FORCE INDEX (idx_card_chan_key) +STRAIGHT_JOIN sys_dept AS d + ON c.dept_id = d.dept_id + AND c.tenant_id = d.tenant_id +WHERE c.chan_key = ? + AND c.id > ? + AND c.iccid <> '' + AND c.current_package_code <> '' + AND d.deleted_at IS NULL +ORDER BY c.id ASC +LIMIT ?` + +func (s *DatabaseScanner) ListChannels(ctx context.Context) ([]channelMetadata, error) { + var metadata []channelMetadata + if err := dao.TSimCardInfo.DB().Model("t_sim_channels").Ctx(ctx).Fields("chan_key,temp_key,sync_enabled,operator").Scan(&metadata); err != nil { + return nil, gerror.Wrap(err, "list channel metadata") + } + return metadata, nil +} + +func (s *DatabaseScanner) ScanChannel(ctx context.Context, chanKey string, lastID int64, pageSize int) ([]Card, error) { + if chanKey == "" || lastID < 0 || pageSize <= 0 { + return nil, gerror.New("invalid channel pagination parameters") + } + var rows []entity.TSimCardInfo + // Use the card DAO's database group, not a separately configured connection. + if err := dao.TSimCardInfo.DB().Raw(channelPageSQL, chanKey, lastID, pageSize).Ctx(ctx).Scan(&rows); err != nil { + return nil, gerror.Wrap(err, "scan channel cards") + } + cards := make([]Card, 0, len(rows)) + for _, row := range rows { + cards = append(cards, Card{ID: row.Id, ICCID: row.Iccid, MSISDN: row.Msisdn, IMSI: row.Imsi, ChanKey: row.ChanKey, PackageCode: row.CurrentPackageCode}) + } + return cards, nil +} + +// Pool counters are shared with other worker traffic. Their deltas are direct +// observations, not an attribution of all connection waits to this producer. +func (*DatabaseScanner) poolWait(ctx context.Context) (int64, int64) { + var count, nanos int64 + for _, item := range dao.TSimCardInfo.DB().Stats(ctx) { + stats := item.Stats() + count += stats.WaitCount + nanos += int64(stats.WaitDuration) + } + return count, nanos +} + +// ValidateScanSettings is strict for values read from configuration. The Go +// constructor first fills omitted zero values; explicit YAML zero is rejected. +func ValidateScanSettings(mode string, concurrency int) error { + if mode != "serial" && mode != "channel" { + return gerror.Newf("invalid trafficUsageMonthlyHistorySync.producer.scanMode %q (serial or channel required)", mode) + } + if concurrency <= 0 { + return gerror.New("trafficUsageMonthlyHistorySync.producer.scanConcurrency must be positive") + } + return nil +} diff --git a/internal/producer/monthlyusagehistory/channel_scanner_test.go b/internal/producer/monthlyusagehistory/channel_scanner_test.go new file mode 100644 index 0000000..fc31a6a --- /dev/null +++ b/internal/producer/monthlyusagehistory/channel_scanner_test.go @@ -0,0 +1,123 @@ +package monthlyusagehistory + +import ( + "context" + "reflect" + "sort" + "strings" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + _ "github.com/gogf/gf/contrib/drivers/sqlite/v2" + "github.com/gogf/gf/v2/database/gdb" +) + +// SQLite runs the SAME predicate and ordering with only the two MySQL hints +// removed. This checks full eligibility/TaskID parity, not MySQL access plans. +// Production EXPLAIN must still validate the unmodified channelPageSQL. +type sqliteChannelScanner struct{ DatabaseScanner } + +func (s *sqliteChannelScanner) ScanChannel(ctx context.Context, key string, last int64, size int) ([]Card, error) { + query := strings.ReplaceAll(channelPageSQL, "FORCE INDEX (idx_card_chan_key)", "") + query = strings.ReplaceAll(query, "STRAIGHT_JOIN", "INNER JOIN") + var rows []struct { + ID int64 `orm:"id"` + ICCID string `orm:"iccid"` + MSISDN string `orm:"msisdn"` + IMSI string `orm:"imsi"` + ChanKey string `orm:"chan_key"` + PackageCode string `orm:"current_package_code"` + } + if err := dao.TSimCardInfo.DB().Raw(query, key, last, size).Ctx(ctx).Scan(&rows); err != nil { + return nil, err + } + cards := make([]Card, 0, len(rows)) + for _, row := range rows { + cards = append(cards, Card{ID: row.ID, ICCID: row.ICCID, MSISDN: row.MSISDN, IMSI: row.IMSI, ChanKey: row.ChanKey, PackageCode: row.PackageCode}) + } + return cards, nil +} + +func TestDatabaseQualificationAndCompleteTaskIDParity(t *testing.T) { + // No application DB/config: isolated SQLite file and loopback Redis only. + if err := gdb.SetConfig(gdb.Config{"default": gdb.ConfigGroup{{Type: "sqlite", Link: "sqlite::@file(" + t.TempDir() + "/cards.db)"}}}); err != nil { + t.Fatal(err) + } + ctx := contextForTest(t) + db := dao.TSimCardInfo.DB() + t.Cleanup(func() { _ = db.Close(context.Background()) }) + for _, sql := range []string{ + `CREATE TABLE sys_dept (dept_id INTEGER PRIMARY KEY, tenant_id TEXT, deleted_at DATETIME)`, + `CREATE TABLE t_sim_card_info (id INTEGER PRIMARY KEY, iccid TEXT, msisdn TEXT, imsi TEXT, chan_key TEXT, current_package_code TEXT, dept_id INTEGER, tenant_id TEXT)`, + `CREATE INDEX idx_card_chan_key ON t_sim_card_info(chan_key)`, + `CREATE TABLE t_sim_channels (chan_key TEXT, temp_key TEXT, sync_enabled INTEGER, operator INTEGER)`, + `INSERT INTO sys_dept VALUES (1,'t1',NULL),(2,'t2',NULL),(3,'t1','2026-01-01')`, + `INSERT INTO t_sim_channels VALUES ('a','sim_yd_v5',0,1),('b','sim_dx_v1',0,2),('disabled','sim_yd_v5',1,1),('no-template','',0,1)`, + `INSERT INTO t_sim_card_info VALUES + (1,'i1','m1','s1','a','pkg',1,'t1'), + (2,'i2','m2','s2','b','pkg',2,'t2'), + (3,'i3','','','a','pkg',1,'t1'), + (4,'','','','a','pkg',1,'t1'), + (5,'i5','','','a','',1,'t1'), + (6,'i6','','','a','pkg',1,'wrong-tenant'), + (7,'i7','','','a','pkg',3,'t1'), + (8,'i8','','','b','pkg',99,'t2'), + (9,'i9','','','disabled','pkg',1,'t1'), + (10,'i10','','','no-template','pkg',1,'t1'), + (11,'i11','','','unknown','pkg',1,'t1'), + (12,'i12','','','','pkg',1,'t1'), + (13,'i13','','','b','pkg',2,'t2')`, + } { + if _, err := db.Exec(ctx, sql); err != nil { + t.Fatal(err) + } + } + store := testStore(t) + for _, scope := range [][]string{nil, {"a"}, {"b", "disabled", "no-template", "unknown"}, {"unknown"}} { + var sets [][]string + for _, mode := range []string{"serial", "channel"} { + scanner := &sqliteChannelScanner{} + p := testProducer(t, scanner, store, Config{ScanMode: mode, PageSize: 2}) + run := "db-" + mode + if err := p.Produce(ctx, scopedMessage(run, scope...)); err != nil { + t.Fatal(err) + } + ids := taskIDs(t, store, run) + for i, id := range ids { + ids[i] = strings.TrimPrefix(id, run+":") + } + sort.Strings(ids) + sets = append(sets, ids) + stateKey, _ := store.TaskStateKey(run) + metaKey, _ := store.RunMetaKey(run) + chanKey, _ := store.RunChannelsKey(run) + _, _ = store.RedisClient().Do(ctx, "DEL", stateKey, metaKey, chanKey) + for _, key := range []string{"a", "b"} { + activeKey, _ := store.RoundActiveKey(run, 1, key) + _, _ = store.RedisClient().Do(ctx, "DEL", activeKey) + } + _, _ = store.RedisClient().Do(ctx, "SREM", store.ActiveRunsKey(), run) + } + if !reflect.DeepEqual(sets[0], sets[1]) { + t.Fatalf("scope=%v serial=%v channel=%v", scope, sets[0], sets[1]) + } + if scope == nil { + want := []string{operation + ":1", operation + ":13", operation + ":2", operation + ":3"} + sort.Strings(want) + if !reflect.DeepEqual(sets[0], want) { + t.Fatalf("qualification set=%v want=%v", sets[0], want) + } + } + } +} + +func TestChannelSQLShapeAndParameters(t *testing.T) { + for _, term := range []string{"FORCE INDEX (idx_card_chan_key)", "STRAIGHT_JOIN sys_dept", "c.chan_key = ?", "c.id > ?", "c.iccid <> ''", "c.current_package_code", "c.current_package_code <> ''", "d.deleted_at IS NULL", "c.tenant_id = d.tenant_id", "ORDER BY c.id ASC", "LIMIT ?"} { + if !strings.Contains(channelPageSQL, term) { + t.Fatalf("missing %q", term) + } + } + if strings.Contains(channelPageSQL, "<=") || strings.Count(channelPageSQL, "?") != 3 { + t.Fatal("SQL contains upper bound or changed bind parameter count") + } +} diff --git a/internal/producer/monthlyusagehistory/parallel_regression_test.go b/internal/producer/monthlyusagehistory/parallel_regression_test.go new file mode 100644 index 0000000..4243149 --- /dev/null +++ b/internal/producer/monthlyusagehistory/parallel_regression_test.go @@ -0,0 +1,102 @@ +package monthlyusagehistory + +import ( + "context" + "errors" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +func TestChannelBackpressureDoesNotStarveHighIDChannel(t *testing.T) { + for _, mode := range []string{"serial", "channel"} { + t.Run(mode, func(t *testing.T) { + store := testStore(t) + ctx, cancel := context.WithCancel(contextForTest(t)) + defer cancel() + scanner := scannerFor([]Card{{ID: 1, ChanKey: "slow"}, {ID: 2, ChanKey: "slow"}, {ID: 3, ChanKey: "slow"}, {ID: 1000, ChanKey: "fast"}}, 1000) + p := testProducer(t, scanner, store, Config{ScanMode: mode, PageSize: 1, ScanConcurrency: 1, ChannelHighWaterMark: 1, GlobalHighWaterMark: 10, PollInterval: time.Millisecond}) + result := make(chan error, 1) + started := time.Now() + go func() { result <- p.Produce(ctx, scopedMessage("starvation")) }() + slowKey, _ := store.RoundActiveKey("starvation", 1, "slow") + fastKey, _ := store.RoundActiveKey("starvation", 1, "fast") + waitUntil(t, ctx, func() bool { n, e := store.SizeKey(ctx, slowKey); return e == nil && n == 1 }) + if mode == "channel" { + waitUntil(t, ctx, func() bool { n, e := store.SizeKey(ctx, fastKey); return e == nil && n == 1 }) + } else { + // No consumer drains slow. The serial cursor cannot reach card 1000. + time.Sleep(30 * time.Millisecond) + if n, err := store.SizeKey(ctx, fastKey); err != nil || n != 0 { + t.Fatalf("serial fast size=%d err=%v", n, err) + } + } + slow, _ := store.SizeKey(ctx, slowKey) + fast, _ := store.SizeKey(ctx, fastKey) + t.Logf("mode=%s slowQueued=%d fastQueued=%d scanConcurrency=1 channelHWM=1 elapsedMs=%d", mode, slow, fast, time.Since(started).Milliseconds()) + cancel() + if err := <-result; !errors.Is(err, context.Canceled) { + t.Fatalf("cancellation err=%v", err) + } + requireMeta(t, store, "starvation", false, slow+fast) + }) + } +} + +func TestChannelPayloadRetainsPeriodPackageAndIdentifiers(t *testing.T) { + store := testStore(t) + ctx := contextForTest(t) + card := Card{ID: 10, ICCID: "iccid", MSISDN: "msisdn", IMSI: "imsi", ChanKey: "a", PackageCode: "pkg-history"} + p := testProducer(t, scannerFor([]Card{card}, card.ID), store, Config{PageSize: 1}) + message := scopedMessage("payload", "a") + if err := p.Produce(ctx, message); err != nil { + t.Fatal(err) + } + claimed, err := store.ClaimRun(ctx, message.RunID, "a", time.Now().Add(time.Second), time.Now().Add(time.Minute), 1) + if err != nil || len(claimed) != 1 { + t.Fatalf("claimed=%v err=%v", claimed, err) + } + task := claimed[0].Task + if task.TaskID != zset.StableTaskID(message.RunID, FullSyncOperation, card.ID) || task.Operation != FullSyncOperation || task.QueryMonth != message.QueryMonth || task.PackageCode != card.PackageCode || task.ICCID != card.ICCID || task.MSISDN != card.MSISDN || task.IMSI != card.IMSI || task.TempKey != "sim_yd_v5" || task.Operator != 1 { + t.Fatalf("task=%+v", task) + } +} + +func TestZombieWithBacklogDrainsBeforeCompletion(t *testing.T) { + p, store := newZombieTestProducer(t) + ctx := contextForTest(t) + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "zombie-backlog"}); err != nil { + t.Fatal(err) + } + tasks := []zset.Task{{TaskID: "zombie-card", RunID: "zombie-backlog", ChanKey: "a"}} + if _, err := store.EnqueueRunBatch(ctx, "zombie-backlog", "a", tasks, time.Now()); err != nil { + t.Fatal(err) + } + if pending, err := p.hasPendingTasks(ctx, "new"); err != nil || !pending { + t.Fatalf("backlog must retain backpressure: pending=%t err=%v", pending, err) + } + requireMeta(t, store, "zombie-backlog", true, 1) + if complete, err := store.CompleteRunIfIdle(ctx, "zombie-backlog"); err != nil || complete { + t.Fatalf("premature complete=%t err=%v", complete, err) + } + claimed, err := store.ClaimRun(ctx, "zombie-backlog", "a", time.Now().Add(time.Second), time.Now().Add(time.Minute), 1) + if err != nil || len(claimed) != 1 { + t.Fatalf("claimed=%v err=%v", claimed, err) + } + if ok, err := store.FinalizeRunTask(ctx, claimed[0], zset.TaskOutcome{Success: true}); err != nil || !ok { + t.Fatalf("finalize=%t err=%v", ok, err) + } + if complete, err := store.CompleteRunIfIdle(ctx, "zombie-backlog", time.Hour); err != nil || !complete { + t.Fatalf("complete=%t err=%v", complete, err) + } + meta, _ := store.RunMetaKey("zombie-backlog") + ttl, err := store.RedisClient().Do(ctx, "TTL", meta) + if err != nil || ttl.Int() <= 0 { + t.Fatalf("retention ttl=%v err=%v", ttl, err) + } + active, err := store.ActiveRuns(ctx) + if err != nil || len(active) != 0 { + t.Fatalf("active=%v err=%v", active, err) + } +} diff --git a/internal/producer/monthlyusagehistory/producer.go b/internal/producer/monthlyusagehistory/producer.go new file mode 100644 index 0000000..9ff42ab --- /dev/null +++ b/internal/producer/monthlyusagehistory/producer.go @@ -0,0 +1,741 @@ +// Package monthlyusagehistory produces the previous-month usage history run. +// +// It writes the TDengine monthly table, a different target from +// producer/monthlyusage (which fills the MySQL current-month snapshot). Apart +// from the period and the destination it is structurally identical: scan the +// eligible cards once, group by channel, and enqueue one Redis ZSet task per +// card. The zset consumer owns pacing, retries, failure pooling, archiving and +// replay. +// +// The run period is a month (yyyyMM) and travels in zset.Task.QueryMonth, which +// is a period identifier rather than a month name; see that field's +// documentation. +package monthlyusagehistory + +import ( + "context" + "crypto/rand" + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + "strings" + "sync/atomic" + "time" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +// FullSyncOperation identifies a monthly usage history run in run metadata, +// archives and replay. contract.SyncKindForOperation maps it to +// traffic_usage_monthly. +const FullSyncOperation = model.OperationPreviousMonthUsageFullSync + +// scopeSampleLimit bounds how many channel names are copied into run metadata. +// The run hash and its archive JSON must stay bounded regardless of how many +// channels a scope selects. +const scopeSampleLimit = 10 + +// scopeDigest is a stable short fingerprint of the selected channels. It lets +// operators confirm two runs covered the same scope without storing the list. +func scopeDigest(scope *model.SyncScope) string { + if scope == nil || len(scope.Channels) == 0 { + return "" + } + digest := sha256.Sum256([]byte(strings.Join(scope.Channels, "\n"))) + return hex.EncodeToString(digest[:8]) +} + +type Config struct { + ScanMode string + ScanConcurrency int + PageSize int + GlobalHighWaterMark int64 + ChannelHighWaterMark int64 + PollInterval time.Duration + RunLockKey string + RunLockTTL time.Duration + RoundRetryEnabled bool + MaxRounds int + FailureRetention time.Duration +} + +func (c Config) withDefaults() Config { + if c.ScanMode == "" { + c.ScanMode = "serial" + } + if c.ScanConcurrency == 0 { + c.ScanConcurrency = 4 + } + if c.PageSize <= 0 { + c.PageSize = 100 + } + if c.PollInterval <= 0 { + c.PollInterval = 2 * time.Second + } + if c.RunLockKey == "" { + c.RunLockKey = "monthly-history:producer:active" + } + if c.RunLockTTL <= 0 { + c.RunLockTTL = 5 * time.Minute + } + if c.MaxRounds <= 0 { + c.MaxRounds = 3 + } + if c.FailureRetention <= 0 { + c.FailureRetention = 30 * 24 * time.Hour + } + return c +} + +type CardScanner interface { + SnapshotMaxID(context.Context) (int64, error) + Scan(context.Context, int64, int64, int) ([]Card, error) +} + +type Card struct { + ID int64 + ICCID string + MSISDN string + IMSI string + ChanKey string + TempKey string + PackageCode string + SyncEnabled bool + Operator int +} + +type channelMetadata struct { + ChanKey string `orm:"chan_key"` + TempKey string `orm:"temp_key"` + SyncEnabled bool `orm:"sync_enabled"` + Operator int `orm:"operator"` +} + +type DatabaseScanner struct { + channelMetadata map[string]channelMetadata +} + +func (s *DatabaseScanner) SnapshotMaxID(ctx context.Context) (int64, error) { + var card entity.TSimCardInfo + cardTable := dao.TSimCardInfo.Table() + columns := dao.TSimCardInfo.Columns() + if err := monthlyHistoryEligibleCardModel(ctx). + Fields(cardTable + "." + columns.Id + " AS " + columns.Id). + OrderDesc(cardTable + "." + columns.Id). + Limit(1). + Scan(&card); err != nil { + return 0, fmt.Errorf("read card snapshot max id: %w", err) + } + return card.Id, nil +} + +func (s *DatabaseScanner) Scan(ctx context.Context, lastID, maxID int64, pageSize int) ([]Card, error) { + if maxID <= lastID { + return nil, nil + } + if lastID == 0 || s.channelMetadata == nil { + var metadata []channelMetadata + if err := g.DB().Model("t_sim_channels").Fields("chan_key,temp_key,sync_enabled,operator").Scan(&metadata); err != nil { + return nil, fmt.Errorf("scan channel metadata: %w", err) + } + s.channelMetadata = make(map[string]channelMetadata, len(metadata)) + for _, item := range metadata { + s.channelMetadata[item.ChanKey] = item + } + } + var rows []entity.TSimCardInfo + columns := dao.TSimCardInfo.Columns() + cardTable := dao.TSimCardInfo.Table() + query := monthlyHistoryEligibleCardModel(ctx). + Fields( + cardTable+"."+columns.Id+" AS "+columns.Id, + cardTable+"."+columns.Iccid+" AS "+columns.Iccid, + cardTable+"."+columns.Msisdn+" AS "+columns.Msisdn, + cardTable+"."+columns.Imsi+" AS "+columns.Imsi, + cardTable+"."+columns.ChanKey+" AS "+columns.ChanKey, + // The monthly usage child table stores the package code and the + // diagnosis page reads it back, so it must be carried on the task. + cardTable+"."+columns.CurrentPackageCode+" AS "+columns.CurrentPackageCode, + ). + WhereGT(cardTable+"."+columns.Id, lastID). + WhereLTE(cardTable+"."+columns.Id, maxID). + OrderAsc(cardTable + "." + columns.Id). + Limit(pageSize) + if err := query.Scan(&rows); err != nil { + return nil, fmt.Errorf("scan card information: %w", err) + } + cards := make([]Card, 0, len(rows)) + for _, row := range rows { + channel := s.channelMetadata[row.ChanKey] + cards = append(cards, Card{ + ID: row.Id, + ICCID: row.Iccid, + MSISDN: row.Msisdn, + IMSI: row.Imsi, + ChanKey: row.ChanKey, + TempKey: channel.TempKey, + PackageCode: row.CurrentPackageCode, + SyncEnabled: channel.SyncEnabled, + Operator: channel.Operator, + }) + } + return cards, nil +} + +// monthlyHistoryEligibleCardModel mirrors the monthly usage eligibility rule and +// the legacy admin SyncMonthly scan: the card must belong to a live department, +// carry an ICCID and channel, and have a current package code. +func monthlyHistoryEligibleCardModel(ctx context.Context) *gdb.Model { + cardTable := dao.TSimCardInfo.Table() + deptTable := dao.SysDept.Table() + cardColumns := dao.TSimCardInfo.Columns() + deptColumns := dao.SysDept.Columns() + + return dao.TSimCardInfo.Ctx(ctx). + InnerJoin(deptTable, cardTable+"."+cardColumns.DeptId+" = "+deptTable+"."+deptColumns.DeptId+" AND "+cardTable+"."+cardColumns.TenantId+" = "+deptTable+"."+deptColumns.TenantId). + Where(cardTable+"."+cardColumns.Iccid+" <> ?", ""). + Where(cardTable+"."+cardColumns.ChanKey+" <> ?", ""). + Where(cardTable+"."+cardColumns.CurrentPackageCode+" <> ?", ""). + WhereNull(deptTable + "." + deptColumns.DeletedAt) +} + +type Producer struct { + scanner CardScanner + store *zset.Store + config Config + running atomic.Int32 + owner string + observation *supplyObservation +} + +type productionStats struct { + pages int64 + scanned int64 + eligible int64 + enqueued int64 + skippedEmptyChan int64 + skippedNoTemplate int64 + skippedSyncDisabled int64 + skippedOutOfScope int64 + stoppedTasks int64 + channels map[string]struct{} + scopeChannels int + skippedChannels int +} + +func (s *productionStats) channelCount() int { + return len(s.channels) +} + +func (s *productionStats) log(ctx context.Context, status, runID, queryMonth, stage, reason string, startedAt time.Time, maxID int64, err error) { + message := fmt.Sprintf("event=monthly_usage_history_producer status=%s runId=%s queryMonth=%s snapshotMaxId=%d pages=%d scanned=%d eligible=%d enqueued=%d skippedEmptyChan=%d skippedNoTemplate=%d skippedSyncDisabled=%d skippedOutOfScope=%d stoppedTasks=%d channelCount=%d scopeChannels=%d skippedChannels=%d durationMs=%d", status, runID, queryMonth, maxID, s.pages, s.scanned, s.eligible, s.enqueued, s.skippedEmptyChan, s.skippedNoTemplate, s.skippedSyncDisabled, s.skippedOutOfScope, s.stoppedTasks, s.channelCount(), s.scopeChannels, s.skippedChannels, time.Since(startedAt).Milliseconds()) + if s.scanned > 0 { + message += fmt.Sprintf(" cardsPerSec=%.2f", float64(s.scanned)/time.Since(startedAt).Seconds()) + } + if stage != "" { + message += " stage=" + stage + } + if reason != "" { + message += " reason=" + reason + } + if err != nil { + message += fmt.Sprintf(" error=%q", err.Error()) + } + if status == "failed" { + g.Log().Error(ctx, message) + return + } + if status == "skipped" { + g.Log().Warning(ctx, message) + return + } + g.Log().Info(ctx, message) +} + +func New(scanner CardScanner, store *zset.Store, config Config) (*Producer, error) { + if scanner == nil || store == nil { + return nil, fmt.Errorf("monthly usage history producer dependencies are incomplete") + } + config = config.withDefaults() + if err := ValidateScanSettings(config.ScanMode, config.ScanConcurrency); err != nil { + return nil, err + } + if config.ScanMode == "channel" { + if _, ok := scanner.(channelScanner); !ok { + return nil, fmt.Errorf("channel scan requires ListChannels and ScanChannel") + } + } + return &Producer{scanner: scanner, store: store, config: config, owner: newOwner()}, nil +} + +var ErrAlreadyRunning = fmt.Errorf("monthly usage history producer already running") + +// ErrRunFinished is returned when a redelivered start message names a run that +// already reached a terminal state. The run must not be resurrected; callers +// should log and ack instead of retrying. +var ErrRunFinished = fmt.Errorf("monthly usage history run already finished") + +// runAlreadyFinished reports whether runID exists and has already completed. +// A missing run is not an error: the message may be the first delivery. +func (p *Producer) runAlreadyFinished(ctx context.Context, runID string) (bool, error) { + meta, err := p.store.GetRunMeta(ctx, runID) + if err != nil { + if errors.Is(err, zset.ErrRunNotFound) { + return false, nil + } + return false, err + } + return meta.Status == "completed" || meta.Status == "completed_with_failures", nil +} + +// Produce scans the eligible cards once and enqueues one task per card. +// +// message.QueryMonth (yyyyMM) is stored in zset.Task.QueryMonth, the queue's +// period-identifier field, so the zset store, run metadata, archive, replay and +// Lua scripts need no change for a monthly history run. +func (p *Producer) Produce(ctx context.Context, message model.TrafficUsageMonthlyHistoryStartMessage) (err error) { + startedAt := time.Now() + stats := productionStats{channels: make(map[string]struct{})} + if !p.running.CompareAndSwap(0, 1) { + stats.log(ctx, "skipped", message.RunID, message.QueryMonth, "precheck", "already_running", startedAt, 0, nil) + return ErrAlreadyRunning + } + defer p.running.Store(0) + + if message.RunID == "" { + err = fmt.Errorf("runId is empty") + stats.log(ctx, "failed", message.RunID, message.QueryMonth, "validate", "", startedAt, 0, err) + return err + } + if message.QueryMonth == "" { + err = fmt.Errorf("queryMonth is empty") + stats.log(ctx, "failed", message.RunID, message.QueryMonth, "validate", "", startedAt, 0, err) + return err + } + scope, err := model.NormalizeScope(message.Scope) + if err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryMonth, "validate", "invalid_scope", startedAt, 0, err) + return err + } + stats.scopeChannels = scope.ChannelCount() + // A redelivered start message must not resurrect a run that already + // finished; that would create a second producer pass over the same RunID. + if p.config.RoundRetryEnabled { + if finished, checkErr := p.runAlreadyFinished(ctx, message.RunID); checkErr != nil { + stats.log(ctx, "failed", message.RunID, message.QueryMonth, "run_state", "", startedAt, 0, checkErr) + return checkErr + } else if finished { + stats.log(ctx, "skipped", message.RunID, message.QueryMonth, "run_state", "run_finished", startedAt, 0, nil) + return ErrRunFinished + } + } + acquired, err := p.store.AcquireLockOrTakeover(ctx, p.config.RunLockKey, p.owner, message.RunID, p.config.RunLockTTL) + if err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryMonth, "lock", "", startedAt, 0, err) + return err + } + if !acquired { + stats.log(ctx, "skipped", message.RunID, message.QueryMonth, "lock", "already_running", startedAt, 0, nil) + return ErrAlreadyRunning + } + runCtx, cancel := context.WithCancel(ctx) + defer cancel() + defer func() { + releaseCtx, releaseCancel := context.WithTimeout(context.Background(), time.Second) + defer releaseCancel() + _, _ = p.store.ReleaseLock(releaseCtx, p.config.RunLockKey, p.owner) + }() + pending, err := p.hasPendingTasks(ctx, message.RunID) + if err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryMonth, "pending_check", "", startedAt, 0, err) + return err + } + if pending { + stats.log(ctx, "skipped", message.RunID, message.QueryMonth, "pending_check", "pending_tasks", startedAt, 0, nil) + return ErrAlreadyRunning + } + renewErr := make(chan error, 1) + go p.renewLock(runCtx, cancel, renewErr) + ctx = runCtx + if message.Operation == "" { + message.Operation = FullSyncOperation + } + if p.config.RoundRetryEnabled { + if err = p.store.CreateRun(ctx, zset.RunMeta{ + RunID: message.RunID, + Operation: message.Operation, + QueryMonth: message.QueryMonth, + Status: "producing", + CurrentRound: 1, + ScopeChannels: scope.ChannelCount(), + ScopeDigest: scopeDigest(scope), + ScopeChannelsSample: strings.Join(scope.ChannelSample(scopeSampleLimit), ","), + }); err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryMonth, "run_init", "", startedAt, 0, err) + return err + } + } + maxID, err := p.scanner.SnapshotMaxID(ctx) + if err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryMonth, "snapshot", "", startedAt, 0, err) + return err + } + g.Log().Infof(ctx, "event=monthly_usage_history_producer status=started runId=%s queryMonth=%s snapshotMaxId=%d scopeChannels=%d scanMode=%s scanConcurrency=%d pageSize=%d", message.RunID, message.QueryMonth, maxID, scope.ChannelCount(), p.config.ScanMode, p.config.ScanConcurrency, p.config.PageSize) + stage := "scan" + defer func() { + if err != nil { + stats.log(ctx, "failed", message.RunID, message.QueryMonth, stage, "", startedAt, maxID, err) + return + } + stats.log(ctx, "completed", message.RunID, message.QueryMonth, "", "", startedAt, maxID, nil) + }() + if p.config.RoundRetryEnabled && p.config.ScanMode == "channel" { + stage = "channel_scan" + if err = p.produceChannels(ctx, message, scope, maxID, startedAt, &stats); err != nil { + if isProductionClosed(err) { + // A same-Run redelivery after producerDone leaves consumption alone. + err = nil + return nil + } + return err + } + // Verify ownership immediately before closing production. The periodic + // renewer still cancels scanners and capacity waits on lease loss. + var owned bool + owned, err = p.store.RenewLock(ctx, p.config.RunLockKey, p.owner, p.config.RunLockTTL) + if err != nil { + return err + } + if !owned { + return fmt.Errorf("producer lock token is no longer owned") + } + if err = ctx.Err(); err != nil { + return err + } + stage = "producer_done" + err = p.store.MarkProducerDone(ctx, message.RunID) + return err + } + lastID := int64(0) + for lastID < maxID { + stage = "capacity" + if err = p.waitForCapacity(ctx, message.RunID); err != nil { + return err + } + stage = "scan" + cards, scanErr := p.scanner.Scan(ctx, lastID, maxID, p.config.PageSize) + if scanErr != nil { + err = scanErr + return err + } + if len(cards) == 0 { + break + } + stats.pages++ + stats.scanned += int64(len(cards)) + groupedTasks := make(map[string][]zset.Task) + for _, card := range cards { + lastID = card.ID + if card.ChanKey == "" { + stats.skippedEmptyChan++ + g.Log().Warningf(ctx, "event=monthly_usage_history_card status=skipped stage=validate cardId=%d reason=empty_chan_key", card.ID) + continue + } + // Scope is applied before the sync_enabled and template checks so a + // partial run only ever reports on the channels it was asked for. + if !scope.Allows(card.ChanKey) { + stats.skippedOutOfScope++ + continue + } + if card.SyncEnabled { + stats.skippedSyncDisabled++ + g.Log().Warningf(ctx, "event=monthly_usage_history_card status=skipped stage=validate cardId=%d chanKey=%s reason=sync_disabled", card.ID, card.ChanKey) + continue + } + if card.TempKey == "" { + stats.skippedNoTemplate++ + g.Log().Warningf(ctx, "event=monthly_usage_history_card status=skipped stage=validate cardId=%d chanKey=%s reason=missing_channel_template", card.ID, card.ChanKey) + continue + } + stats.eligible++ + stats.channels[card.ChanKey] = struct{}{} + task := zset.Task{ + TaskID: zset.StableTaskID(message.RunID, message.Operation, card.ID), + RunID: message.RunID, + Operation: message.Operation, + QueryMonth: message.QueryMonth, + ChanKey: card.ChanKey, + TempKey: card.TempKey, + Operator: card.Operator, + CardID: card.ID, + ICCID: card.ICCID, + MSISDN: card.MSISDN, + IMSI: card.IMSI, + PackageCode: card.PackageCode, + } + groupedTasks[card.ChanKey] = append(groupedTasks[card.ChanKey], task) + } + now := time.Now() + for chanKey, tasks := range groupedTasks { + stage = "enqueue" + if err = p.waitForChannelCapacity(ctx, message.RunID, chanKey); err != nil { + return err + } + var added int + var enqueueErr error + if p.config.RoundRetryEnabled { + added, enqueueErr = p.store.EnqueueRunBatch(ctx, message.RunID, chanKey, tasks, now) + } else { + added, enqueueErr = p.store.EnqueueBatch(ctx, chanKey, tasks, now) + } + if enqueueErr != nil { + err = enqueueErr + return err + } + stats.enqueued += int64(added) + if added < len(tasks) { + stats.stoppedTasks += int64(len(tasks) - added) + } + if added == 0 { + g.Log().Warningf(ctx, "event=monthly_usage_history_channel status=skipped runId=%s queryMonth=%s chanKey=%s count=%d reason=stopped", message.RunID, message.QueryMonth, chanKey, len(tasks)) + } + } + } + if p.config.RoundRetryEnabled { + if err = p.store.MarkProducerDone(ctx, message.RunID); err != nil { + return err + } + } + return nil +} + +func (p *Producer) hasPendingTasks(ctx context.Context, currentRunID ...string) (bool, error) { + currentID := "" + if len(currentRunID) > 0 { + currentID = currentRunID[0] + } + if p.config.RoundRetryEnabled { + runs, err := p.store.ActiveRuns(ctx) + if err != nil { + return false, err + } + for _, runID := range runs { + if runID == currentID { + // A redelivered message for the same run may resume a producer + // that stopped before producerDone. Stable task IDs make the + // rescan idempotent; other runs still provide backpressure. + continue + } + meta, metaErr := p.store.GetRunMeta(ctx, runID) + if metaErr != nil { + // The active index is a global key written outside the run-scoped + // Lua transaction, so it can list a run whose hash is already gone + // (expired, evicted, or cleaned by hand). Returning an error here + // would surface as a non-retryable failure and make the consumer + // redeliver this request forever. Treat it as a stale index entry + // and drop it. + if errors.Is(metaErr, zset.ErrRunNotFound) { + if _, removeErr := p.store.RedisClient().Do(ctx, "SREM", p.store.ActiveRunsKey(), runID); removeErr != nil { + return false, removeErr + } + g.Log().Warningf(ctx, "event=monthly_usage_history_producer status=skipped stage=pending_check reason=stale_index runId=%s", runID) + continue + } + return false, metaErr + } + if meta.Status != "producing" && meta.Status != "running" { + continue + } + // Channel blocking can set running before production ends. Only + // producerDone identifies consumer-owned work; replay has its own lock. + if meta.ProducerDone || meta.SyncType == "replay" { + return true, nil + } + // Recheck the unique producer lease immediately before reclaiming + // another run. A lost lease or an unspecified current run is not proof + // that the old producer has stopped. + if currentID == "" { + return true, nil + } + owned, lockErr := p.store.RenewLock(ctx, p.config.RunLockKey, p.owner, p.config.RunLockTTL) + if lockErr != nil { + return false, lockErr + } + if !owned { + return true, nil + } + // Close production, not the run: consumers retain queued, leased and + // next-round work and apply retention/index cleanup on normal completion. + reclaimed, reclaimErr := p.store.MarkProducerAbandoned(ctx, runID) + if reclaimErr != nil { + return false, reclaimErr + } + g.Log().Warningf(ctx, "event=monthly_usage_history_producer status=skipped stage=pending_check reason=zombie_run runId=%s reclaimed=%t activeCount=%d inflightCount=%d nextRoundCount=%d", runID, reclaimed, meta.ActiveCount, meta.InflightCount, meta.NextRoundCount) + if !reclaimed || meta.ActiveCount > 0 || meta.InflightCount > 0 || meta.NextRoundCount > 0 { + return true, nil + } + } + } + channels, err := p.store.ActiveChannels(ctx) + if err != nil { + return false, err + } + for _, chanKey := range channels { + size, sizeErr := p.store.Size(ctx, chanKey) + if sizeErr != nil { + return false, sizeErr + } + if size > 0 { + return true, nil + } + } + return false, nil +} + +func (p *Producer) renewLock(ctx context.Context, cancel context.CancelFunc, errors chan<- error) { + interval := p.config.RunLockTTL / 3 + if interval <= 0 { + interval = time.Second + } + ticker := time.NewTicker(interval) + defer ticker.Stop() + for { + select { + case <-ctx.Done(): + return + case <-ticker.C: + ok, err := p.store.RenewLock(ctx, p.config.RunLockKey, p.owner, p.config.RunLockTTL) + if err != nil || !ok { + if err == nil { + err = fmt.Errorf("producer lock token is no longer owned") + } + select { + case errors <- fmt.Errorf("renew producer lock: %w", err): + default: + } + cancel() + return + } + } + } +} + +func newOwner() string { + var bytes [12]byte + if _, err := rand.Read(bytes[:]); err == nil { + return "producer-" + hex.EncodeToString(bytes[:]) + } + return fmt.Sprintf("producer-%d", time.Now().UnixNano()) +} + +func (p *Producer) waitForCapacity(ctx context.Context, runID ...string) error { + if p.config.GlobalHighWaterMark <= 0 { + return nil + } + for { + if p.config.RoundRetryEnabled && len(runID) > 0 && runID[0] != "" { + meta, err := p.store.GetRunMeta(ctx, runID[0]) + if err != nil { + return err + } + channels, err := p.store.RunChannels(ctx, runID[0]) + if err != nil { + return err + } + var total int64 + for _, chanKey := range channels { + key, keyErr := p.store.RoundActiveKey(runID[0], meta.CurrentRound, chanKey) + if keyErr != nil { + return keyErr + } + size, sizeErr := p.store.SizeKey(ctx, key) + if sizeErr != nil { + return sizeErr + } + total += size + } + if total < p.config.GlobalHighWaterMark { + return nil + } + if err = sleepContext(ctx, p.config.PollInterval); err != nil { + return err + } + continue + } + channels, err := p.store.ActiveChannels(ctx) + if err != nil { + return err + } + var total int64 + for _, chanKey := range channels { + size, sizeErr := p.store.Size(ctx, chanKey) + if sizeErr != nil { + return sizeErr + } + total += size + } + if total < p.config.GlobalHighWaterMark { + return nil + } + if err = sleepContext(ctx, p.config.PollInterval); err != nil { + return err + } + } +} + +func (p *Producer) waitForChannelCapacity(ctx context.Context, runIDAndChan ...string) error { + if p.config.ChannelHighWaterMark <= 0 { + return nil + } + chanKey := "" + runID := "" + if len(runIDAndChan) == 1 { + chanKey = runIDAndChan[0] + } else if len(runIDAndChan) >= 2 { + runID, chanKey = runIDAndChan[0], runIDAndChan[1] + } + for { + var size int64 + var err error + if p.config.RoundRetryEnabled && runID != "" { + meta, metaErr := p.store.GetRunMeta(ctx, runID) + if metaErr != nil { + return metaErr + } + key, keyErr := p.store.RoundActiveKey(runID, meta.CurrentRound, chanKey) + if keyErr != nil { + return keyErr + } + size, err = p.store.SizeKey(ctx, key) + } else { + size, err = p.store.Size(ctx, chanKey) + } + if err != nil { + return err + } + if size < p.config.ChannelHighWaterMark { + return nil + } + if err = sleepContext(ctx, p.config.PollInterval); err != nil { + return err + } + } +} + +func sleepContext(ctx context.Context, duration time.Duration) error { + timer := time.NewTimer(duration) + defer timer.Stop() + select { + case <-timer.C: + return nil + case <-ctx.Done(): + return ctx.Err() + } +} diff --git a/internal/producer/monthlyusagehistory/producer_test.go b/internal/producer/monthlyusagehistory/producer_test.go new file mode 100644 index 0000000..2962589 --- /dev/null +++ b/internal/producer/monthlyusagehistory/producer_test.go @@ -0,0 +1,256 @@ +package monthlyusagehistory + +import ( + "context" + "fmt" + "strings" + "sync" + "testing" + + "github.com/gogf/gf/v2/container/gvar" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +type pageScanner struct { + cards []Card +} + +func (s *pageScanner) SnapshotMaxID(context.Context) (int64, error) { + if len(s.cards) == 0 { + return 0, nil + } + return s.cards[len(s.cards)-1].ID, nil +} + +func (s *pageScanner) Scan(_ context.Context, lastID, _ int64, _ int) ([]Card, error) { + if lastID != 0 { + return nil, nil + } + return s.cards, nil +} + +// producerRedis records the enqueued members so a test can assert both the +// batch shape and the exact task payload (period and package code). +type producerRedis struct { + mu sync.Mutex + members []string + batches map[string]int +} + +func (r *producerRedis) Do(_ context.Context, command string, _ ...any) (*gvar.Var, error) { + switch strings.ToUpper(command) { + case "SET": + return gvar.New("OK"), nil + case "SMEMBERS": + return gvar.New([]string{}), nil + case "HGETALL": + // No stored run metadata: the producer treats this as a first delivery. + return gvar.New(map[string]string{}), nil + default: + return nil, fmt.Errorf("unexpected command %s", command) + } +} + +func (r *producerRedis) Eval(_ context.Context, _ string, numKeys int64, keys []string, args []any) (*gvar.Var, error) { + if numKeys == 1 { + return gvar.New(int64(1)), nil + } + if numKeys != 3 { + return nil, fmt.Errorf("unexpected key count %d", numKeys) + } + r.mu.Lock() + defer r.mu.Unlock() + if r.batches == nil { + r.batches = make(map[string]int) + } + count := len(args) - 3 + r.batches[keys[0]] += count + for _, arg := range args[3:] { + if member, ok := arg.(string); ok { + r.members = append(r.members, member) + } + } + return gvar.New(count), nil +} + +func newProducer(t *testing.T, cards []Card) (*Producer, *producerRedis) { + t.Helper() + redis := &producerRedis{batches: make(map[string]int)} + store, err := zset.New(redis, "monthly-history:") + if err != nil { + t.Fatal(err) + } + producer, err := New(&pageScanner{cards: cards}, store, Config{PageSize: 100}) + if err != nil { + t.Fatal(err) + } + return producer, redis +} + +func monthlyMessage(runID string, channels ...string) model.TrafficUsageMonthlyHistoryStartMessage { + message := model.TrafficUsageMonthlyHistoryStartMessage{ + RunID: runID, + Operation: FullSyncOperation, + QueryMonth: "202609", + TriggeredAt: "2026-10-10T06:00:00+08:00", + } + if len(channels) > 0 { + message.Scope = &model.SyncScope{Channels: channels} + } + return message +} + +func TestProduceCarriesQueryMonthAndPackageCode(t *testing.T) { + producer, redis := newProducer(t, []Card{ + {ID: 1, ICCID: "89860001", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "PKG-A", Operator: 1}, + }) + + if err := producer.Produce(context.Background(), monthlyMessage("monthly-1")); err != nil { + t.Fatal(err) + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.members) != 1 { + t.Fatalf("members = %+v, want exactly one task", redis.members) + } + member := redis.members[0] + // The period rides in queryMonth because the queue only copies that field. + if !strings.Contains(member, `"queryMonth":"202609"`) { + t.Fatalf("task member lost the monthly period: %s", member) + } + // The package code must reach the TDengine child table via the task. + if !strings.Contains(member, `"packageCode":"PKG-A"`) { + t.Fatalf("task member lost the package code: %s", member) + } + if !strings.Contains(member, `"operation":"`+FullSyncOperation+`"`) { + t.Fatalf("task member lost the operation: %s", member) + } +} + +func TestProduceUsesStableTaskIDForIdempotency(t *testing.T) { + producer, _ := newProducer(t, []Card{ + {ID: 42, ICCID: "89860042", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "PKG-A"}, + }) + want := zset.StableTaskID("monthly-1", FullSyncOperation, 42) + if got := zset.StableTaskID("monthly-1", FullSyncOperation, 42); got != want { + t.Fatalf("StableTaskID is not deterministic: %s vs %s", got, want) + } + if err := producer.Produce(context.Background(), monthlyMessage("monthly-1")); err != nil { + t.Fatal(err) + } +} + +func TestProduceScopeEnqueuesOnlySelectedChannels(t *testing.T) { + producer, redis := newProducer(t, []Card{ + {ID: 1, ICCID: "1", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "P"}, + {ID: 2, ICCID: "2", ChanKey: "173", TempKey: "sim_yd_v5", PackageCode: "P"}, + {ID: 3, ICCID: "3", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "P"}, + {ID: 4, ICCID: "4", ChanKey: "189", TempKey: "sim_yd_v5", PackageCode: "P"}, + }) + + if err := producer.Produce(context.Background(), monthlyMessage("monthly-scoped", "217")); err != nil { + t.Fatal(err) + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.batches) != 1 { + t.Fatalf("enqueue batches = %+v, want only the 217 channel", redis.batches) + } + if redis.batches["monthly-history:217"] != 2 { + t.Fatalf("enqueue batches = %+v, want 2 tasks on monthly:217", redis.batches) + } +} + +func TestProduceFullScopeKeepsEveryChannel(t *testing.T) { + producer, redis := newProducer(t, []Card{ + {ID: 1, ICCID: "1", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "P"}, + {ID: 2, ICCID: "2", ChanKey: "173", TempKey: "sim_yd_v5", PackageCode: "P"}, + }) + + if err := producer.Produce(context.Background(), monthlyMessage("monthly-full")); err != nil { + t.Fatal(err) + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.batches) != 2 { + t.Fatalf("enqueue batches = %+v, want both channels for a full sync", redis.batches) + } +} + +func TestProduceScopeDoesNotOverrideSyncDisabled(t *testing.T) { + producer, redis := newProducer(t, []Card{ + {ID: 1, ICCID: "1", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "P"}, + {ID: 2, ICCID: "2", ChanKey: "173", TempKey: "sim_yd_v5", PackageCode: "P", SyncEnabled: true}, + }) + + if err := producer.Produce(context.Background(), monthlyMessage("monthly-scoped", "173")); err != nil { + t.Fatal(err) + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.batches) != 0 { + t.Fatalf("enqueue batches = %+v, want none (selected channel is sync-disabled)", redis.batches) + } +} + +func TestProduceRejectsMissingPeriod(t *testing.T) { + producer, redis := newProducer(t, []Card{ + {ID: 1, ICCID: "1", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "P"}, + }) + message := monthlyMessage("monthly-bad") + message.QueryMonth = "" + if err := producer.Produce(context.Background(), message); err == nil { + t.Fatal("expected a missing queryDate to be rejected") + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.batches) != 0 { + t.Fatalf("rejected message must not enqueue: %+v", redis.batches) + } +} + +func TestProduceRejectsInvalidScope(t *testing.T) { + producer, redis := newProducer(t, []Card{ + {ID: 1, ICCID: "1", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "P"}, + }) + if err := producer.Produce(context.Background(), monthlyMessage("monthly-bad", "invalid/chan")); err == nil { + t.Fatal("expected invalid chanKey to be rejected") + } + redis.mu.Lock() + defer redis.mu.Unlock() + if len(redis.batches) != 0 { + t.Fatalf("rejected scope must not enqueue: %+v", redis.batches) + } +} + +func TestProduceScopeDigestIsOrderIndependent(t *testing.T) { + first, err := model.NormalizeScope(&model.SyncScope{Channels: []string{"173", "217"}}) + if err != nil { + t.Fatal(err) + } + second, err := model.NormalizeScope(&model.SyncScope{Channels: []string{"217", "173"}}) + if err != nil { + t.Fatal(err) + } + if scopeDigest(first) != scopeDigest(second) { + t.Fatalf("digest depends on channel order: %s vs %s", scopeDigest(first), scopeDigest(second)) + } + if scopeDigest(nil) != "" { + t.Fatal("a full sync must have an empty digest") + } +} + +func TestRunAlreadyFinishedTreatsMissingRunAsFirstDelivery(t *testing.T) { + producer, _ := newProducer(t, []Card{ + {ID: 1, ICCID: "1", ChanKey: "217", TempKey: "sim_yd_v5", PackageCode: "P"}, + }) + finished, err := producer.runAlreadyFinished(context.Background(), "fresh") + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if finished { + t.Fatal("a run without stored metadata must not be reported as finished") + } +} diff --git a/internal/producer/monthlyusagehistory/supply_observation.go b/internal/producer/monthlyusagehistory/supply_observation.go new file mode 100644 index 0000000..b08d410 --- /dev/null +++ b/internal/producer/monthlyusagehistory/supply_observation.go @@ -0,0 +1,174 @@ +package monthlyusagehistory + +import ( + "context" + "sync" + "time" + + "github.com/gogf/gf/v2/frame/g" +) + +// Producer-local telemetry: no run/archive schema changes and no assertion of +// consumer readiness. Empty queues alone do not prove consumer starvation. +type supplyChannel struct { + stage string + pending int + version uint64 + ended bool + firstQuery, firstEnqueue *int64 + changed time.Time + stageDuration map[string]time.Duration + validSamples, missingSamples int64 +} + +type supplyObservation struct { + mu sync.Mutex + started time.Time + channels map[string]*supplyChannel +} + +func (o *supplyObservation) event(key, stage string, pending int, added, ended bool, now time.Time) { + o.mu.Lock() + defer o.mu.Unlock() + s := o.channels[key] + if s == nil { + return + } + s.stageDuration[s.stage] += now.Sub(s.changed) + s.changed, s.stage, s.pending, s.ended = now, stage, pending, ended + s.version++ + if stage == "sql_query" && s.firstQuery == nil { + v := now.Sub(o.started).Milliseconds() + s.firstQuery = &v + } + if added && s.firstEnqueue == nil { + v := now.Sub(o.started).Milliseconds() + s.firstEnqueue = &v + } +} + +// Preserve fractional milliseconds across transitions; round only for logs. +func (s *supplyChannel) stageMilliseconds(now time.Time) map[string]int64 { + if !s.ended { + s.stageDuration[s.stage] += now.Sub(s.changed) + s.changed = now + } + ms := make(map[string]int64, len(s.stageDuration)) + for stage, duration := range s.stageDuration { + ms[stage] = duration.Milliseconds() + } + return ms +} + +func (p *Producer) observe(key, stage string, pending int, added, ended bool) { + if p.observation != nil { + p.observation.event(key, stage, pending, added, ended, time.Now()) + } +} + +// Queue-only samples mirror e6ec5b8: cardinalities, never task bodies. All keys +// share the run hash tag, and round/producer state is checked in the same Lua. +// Leases are token HASHes, not ZSets; future scores include leased members. +const supplySampleScript = ` +local run=redis.call('HMGET',KEYS[1],'runId','status','currentRound','producerDone','syncType') +local known=run[1] and run[2] and run[3] and run[4] +local eligible=known and (run[2]=='producing' or run[2]=='running') and run[3]=='1' and run[4]=='0' and run[5]~='replay' +local blocked=redis.call('HGET',KEYS[1],'blocked:'..ARGV[2]) +return {known and 1 or 0,(eligible and blocked~='1') and 1 or 0,redis.call('ZCOUNT',KEYS[2],'-inf',ARGV[1]),redis.call('HLEN',KEYS[3]),redis.call('ZCOUNT',KEYS[2],'('..ARGV[1],'+inf')} +` + +func (p *Producer) sampleSupply(ctx context.Context, run, key string) ([]int64, error) { + meta, err := p.store.RunMetaKey(run) + if err != nil { + return nil, err + } + active, err := p.store.RoundActiveKey(run, 1, key) + if err != nil { + return nil, err + } + lease, err := p.store.RoundLeaseKey(run, 1, key) + if err != nil { + return nil, err + } + result, err := p.store.RedisClient().Eval(ctx, supplySampleScript, 3, []string{meta, active, lease}, []any{time.Now().UnixMilli(), key}) + if err != nil { + return nil, err + } + return result.Int64s(), nil +} + +func (p *Producer) startObservation(ctx context.Context, run string, keys []string, started time.Time) func() { + o := &supplyObservation{started: started, channels: make(map[string]*supplyChannel, len(keys))} + for _, key := range keys { + o.channels[key] = &supplyChannel{stage: "initializing", changed: started, stageDuration: make(map[string]time.Duration)} + } + p.observation = o + stop, done := make(chan struct{}), make(chan struct{}) + go func() { + defer close(done) + ticker := time.NewTicker(time.Second) + defer ticker.Stop() + for { + select { + case <-ctx.Done(): + return + case <-stop: + return + case <-ticker.C: + for _, key := range keys { + select { + case <-stop: + return + case <-ctx.Done(): + return + default: + } + o.mu.Lock() + s := o.channels[key] + version, ended := s.version, s.ended + o.mu.Unlock() + if ended { + continue + } + sampleCtx, cancel := context.WithTimeout(ctx, 500*time.Millisecond) + values, err := p.sampleSupply(sampleCtx, run, key) + cancel() + o.mu.Lock() + known := err == nil && len(values) == 5 && values[0] == 1 && s.version == version && !s.ended + stage, pending := s.stage, s.pending + if known { + s.validSamples++ + } else { + s.missingSamples++ + } + o.mu.Unlock() + // Logging must not hold the mutex used by producer progress. + if known { + g.Log().Infof(ctx, "event=monthly_usage_history_supply status=sample runId=%s chanKey=%s stage=%s pending=%d firstRoundEligible=%t ready=%d leases=%d futureIncludingLeases=%d consumerEligibility=unknown", run, key, stage, pending, values[1] == 1, values[2], values[3], values[4]) + } else { + g.Log().Infof(ctx, "event=monthly_usage_history_supply status=unknown runId=%s chanKey=%s stage=%s pending=%d consumerEligibility=unknown", run, key, stage, pending) + } + } + } + } + }() + return func() { + close(stop) + <-done + o.mu.Lock() + defer o.mu.Unlock() + for _, key := range keys { + s := o.channels[key] + stageMS := s.stageMilliseconds(time.Now()) + query, enqueue := int64(-1), int64(-1) + if s.firstQuery != nil { + query = *s.firstQuery + } + if s.firstEnqueue != nil { + enqueue = *s.firstEnqueue + } + g.Log().Infof(ctx, "event=monthly_usage_history_supply status=ended runId=%s chanKey=%s firstQueryMs=%d firstEnqueueMs=%d stageMs=%v validSamples=%d missingSamples=%d terminalStage=%s consumerEligibility=unknown", run, key, query, enqueue, stageMS, s.validSamples, s.missingSamples, s.stage) + } + p.observation = nil + } +} diff --git a/internal/producer/monthlyusagehistory/supply_observation_test.go b/internal/producer/monthlyusagehistory/supply_observation_test.go new file mode 100644 index 0000000..6907754 --- /dev/null +++ b/internal/producer/monthlyusagehistory/supply_observation_test.go @@ -0,0 +1,110 @@ +package monthlyusagehistory + +import ( + "reflect" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +func TestLocalSupplyObservationFirstEventsAndStages(t *testing.T) { + started := time.Now() + o := &supplyObservation{started: started, channels: map[string]*supplyChannel{"a": {stage: "initializing", changed: started, stageDuration: map[string]time.Duration{}}}} + o.event("a", "sql_query", 0, false, false, started.Add(10*time.Millisecond)) + o.event("a", "page_ready", 2, false, false, started.Add(20*time.Millisecond)) + o.event("a", "redis_write", 1, true, false, started.Add(30*time.Millisecond)) + o.event("a", "channel_capacity", 1, false, false, started.Add(40*time.Millisecond)) + o.event("a", "sql_query", 0, false, false, started.Add(50*time.Millisecond)) + o.event("a", "redis_write", 0, true, false, started.Add(60*time.Millisecond)) + o.event("a", "upper_bound", 0, false, true, started.Add(70*time.Millisecond)) + o.event("unknown", "ignored", 0, false, false, started) + s := o.channels["a"] + if s.firstQuery == nil || *s.firstQuery != 10 || s.firstEnqueue == nil || *s.firstEnqueue != 30 || !s.ended || s.pending != 0 || s.version != 7 || s.stageDuration["channel_capacity"] != 10*time.Millisecond || s.stageDuration["sql_query"] != 20*time.Millisecond { + t.Fatalf("observation=%+v", s) + } +} + +func TestSupplyObservationAccumulatesSubMillisecondStages(t *testing.T) { + started := time.Unix(0, 0) + o := &supplyObservation{started: started, channels: map[string]*supplyChannel{"a": {stage: "redis_read", changed: started, stageDuration: map[string]time.Duration{}}}} + for i := 1; i <= 100000; i++ { + stage := "redis_read" + if i%2 == 1 { + stage = "redis_write" + } + o.event("a", stage, 0, false, i == 100000, started.Add(time.Duration(i)*500*time.Microsecond)) + } + if got := o.channels["a"].stageMilliseconds(started.Add(time.Hour)); got["redis_read"] != 25000 || got["redis_write"] != 25000 { + t.Fatalf("stageMs=%v, want 25000ms per stage", got) + } +} + +func TestSupplyObservationFinalFlushPreservesFraction(t *testing.T) { + started := time.Unix(0, 0) + o := &supplyObservation{started: started, channels: map[string]*supplyChannel{"a": {stage: "redis_write", changed: started, stageDuration: map[string]time.Duration{}}}} + o.event("a", "redis_read", 0, false, false, started.Add(600*time.Microsecond)) + o.event("a", "redis_write", 0, false, false, started.Add(1200*time.Microsecond)) + s := o.channels["a"] + if got := s.stageMilliseconds(started.Add(1800 * time.Microsecond)); got["redis_write"] != 1 || got["redis_read"] != 0 { + t.Fatalf("final stageMs=%v", got) + } + if s.stageDuration["redis_write"] != 1200*time.Microsecond || s.stageDuration["redis_read"] != 600*time.Microsecond { + t.Fatalf("fractional duration lost: %v", s.stageDuration) + } +} + +func TestSupplySamplesDistinguishReadyLeaseFutureAndClosedRound(t *testing.T) { + store := testStore(t) + ctx := contextForTest(t) + p := testProducer(t, scannerFor(nil, 0), store, Config{}) + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "supply"}); err != nil { + t.Fatal(err) + } + active, _ := store.RoundActiveKey("supply", 1, "a") + lease, _ := store.RoundLeaseKey("supply", 1, "a") + _, _ = store.RedisClient().Do(ctx, "ZADD", active, time.Now().Add(-time.Minute).UnixMilli(), "ready", time.Now().Add(time.Hour).UnixMilli(), "future") + _, _ = store.RedisClient().Do(ctx, "HSET", lease, "leased", "token") + values, err := p.sampleSupply(ctx, "supply", "a") + if err != nil || !reflect.DeepEqual(values, []int64{1, 1, 1, 1, 1}) { + t.Fatalf("sample=%v err=%v", values, err) + } + meta, _ := store.RunMetaKey("supply") + for _, args := range [][]any{{"blocked:a", 1}, {"blocked:a", 0, "producerDone", 1}, {"producerDone", 0, "currentRound", 2}, {"currentRound", 1, "syncType", "replay"}} { + _, _ = store.RedisClient().Do(ctx, "HSET", append([]any{meta}, args...)...) + values, err = p.sampleSupply(ctx, "supply", "a") + if err != nil || values[0] != 1 || values[1] != 0 { + t.Fatalf("ineligible sample=%v err=%v", values, err) + } + } + values, err = p.sampleSupply(ctx, "missing", "a") + if err != nil || values[0] != 0 || values[1] != 0 { + t.Fatalf("missing sample=%v err=%v", values, err) + } +} + +func TestSupplySamplerTracksWaitingChannelsAndStops(t *testing.T) { + store := testStore(t) + ctx := contextForTest(t) + p := testProducer(t, scannerFor(nil, 0), store, Config{}) + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "sampling"}); err != nil { + t.Fatal(err) + } + finish := p.startObservation(ctx, "sampling", []string{"a", "b"}, time.Now()) + defer finish() + o := p.observation + p.observe("a", "channel_capacity", 1, false, false) + p.observe("b", "sql_slot", 0, false, false) + waitUntil(t, ctx, func() bool { + o.mu.Lock() + defer o.mu.Unlock() + return o.channels["a"].validSamples > 0 && o.channels["b"].validSamples > 0 + }) + p.observe("a", "cancelled", 0, false, true) + p.observe("b", "cancelled", 0, false, true) + o.mu.Lock() + defer o.mu.Unlock() + if o.channels["a"].stageDuration["channel_capacity"] < 900*time.Millisecond || o.channels["b"].stageDuration["sql_slot"] < 900*time.Millisecond { + t.Fatalf("missing wait attribution: a=%+v b=%+v", o.channels["a"], o.channels["b"]) + } +} diff --git a/internal/producer/monthlyusagehistory/zombie_test.go b/internal/producer/monthlyusagehistory/zombie_test.go new file mode 100644 index 0000000..35e110d --- /dev/null +++ b/internal/producer/monthlyusagehistory/zombie_test.go @@ -0,0 +1,179 @@ +package monthlyusagehistory + +import ( + "context" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +func newZombieTestProducer(t *testing.T) (*Producer, *zset.Store) { + t.Helper() + store, err := zset.New(testutil.Redis(t), "flow:") + if err != nil { + t.Fatal(err) + } + producer, err := New(&pageScanner{cards: nil}, store, Config{PageSize: 100, RoundRetryEnabled: true}) + if err != nil { + t.Fatal(err) + } + ctx := context.Background() + if acquired, err := store.AcquireLockOrTakeover(ctx, producer.config.RunLockKey, producer.owner, "pending-check", producer.config.RunLockTTL); err != nil || !acquired { + t.Fatalf("acquire producer lock=%t err=%v", acquired, err) + } + t.Cleanup(func() { _, _ = store.ReleaseLock(ctx, producer.config.RunLockKey, producer.owner) }) + return producer, store +} + +// TestHasPendingTasksReclaimsZombieRun guards the "zombie run" regression. +// +// A producer that dies before MarkProducerDone (for example the container is +// replaced mid-run) leaves its run in status=producing with producerDone=0. +// completeRunIfIdle and advanceRunRound both require producerDone=1, so such a +// run can never be closed, never leaves the active index, and never gets the +// retention TTL. Treating it as backpressure rejected every later request +// forever; merely warning about it leaked keys and an index entry indefinitely. +func TestHasPendingTasksReclaimsZombieRun(t *testing.T) { + producer, store := newZombieTestProducer(t) + ctx := context.Background() + + // A genuinely in-flight run (producerDone set, consumer working) must keep + // providing backpressure. + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "running-run", Status: "running", ProducerDone: true, CurrentRound: 1}); err != nil { + t.Fatal(err) + } + pending, err := producer.hasPendingTasks(ctx, "new-run") + if err != nil { + t.Fatal(err) + } + if !pending { + t.Fatal("a running run must still provide backpressure") + } + if _, err := store.RedisClient().Do(ctx, "SREM", store.ActiveRunsKey(), "running-run"); err != nil { + t.Fatal(err) + } + + // A zombie run must not block a new request, and must be promoted so the + // normal lifecycle can drain and close it. + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "zombie-run", Status: "producing", ProducerDone: false, CurrentRound: 1}); err != nil { + t.Fatal(err) + } + pending, err = producer.hasPendingTasks(ctx, "new-run-2") + if err != nil { + t.Fatal(err) + } + if pending { + t.Fatal("a producing run with producerDone=0 must not block new requests (zombie run)") + } + + meta, err := store.GetRunMeta(ctx, "zombie-run") + if err != nil { + t.Fatal(err) + } + if !meta.ProducerDone { + t.Fatal("the zombie run must be promoted (producerDone=1) so it can be closed") + } + if meta.Status != "running" { + t.Fatalf("promoted run status = %q, want running", meta.Status) + } + // Promotion makes the run closable, which is what applies the retention TTL + // and releases the active index entry. + if completed, err := store.CompleteRunIfIdle(ctx, "zombie-run"); err != nil || !completed { + t.Fatalf("promoted run must be closable: completed=%v err=%v", completed, err) + } +} + +// TestHasPendingTasksNeverReclaimsReplayRun guards the boundary: a replay target +// is also created as "producing", but it is driven by the replay engine under +// its own lock, so its producerDone is not the producer's to set. +func TestHasPendingTasksNeverReclaimsReplayRun(t *testing.T) { + producer, store := newZombieTestProducer(t) + ctx := context.Background() + + if err := store.CreateRun(ctx, zset.RunMeta{ + RunID: "replay-run", Status: "producing", ProducerDone: false, CurrentRound: 1, + SyncType: "replay", SourceRunID: "source-run", ReplayCycle: 1, + }); err != nil { + t.Fatal(err) + } + if pending, err := producer.hasPendingTasks(ctx, "new-run"); err != nil || !pending { + t.Fatalf("replay must retain backpressure: pending=%t err=%v", pending, err) + } + meta, err := store.GetRunMeta(ctx, "replay-run") + if err != nil { + t.Fatal(err) + } + if meta.ProducerDone { + t.Fatal("a replay run must not be promoted by the full-sync producer") + } + if meta.Status != "producing" { + t.Fatalf("replay run status = %q, want producing", meta.Status) + } +} + +// TestHasPendingTasksDropsStaleIndexEntry covers a second unrecoverable path: +// the active index can list a run whose hash is already gone. Returning that as +// an error made the request fail permanently (the consumer redelivers it +// forever), which is the same "only fixable by hand-cleaning Redis" failure. +func TestHasPendingTasksDropsStaleIndexEntry(t *testing.T) { + producer, store := newZombieTestProducer(t) + ctx := context.Background() + + // Index a run that has no metadata hash. + if _, err := store.RedisClient().Do(ctx, "SADD", store.ActiveRunsKey(), "ghost-run"); err != nil { + t.Fatal(err) + } + pending, err := producer.hasPendingTasks(ctx, "new-run") + if err != nil { + t.Fatalf("a stale index entry must not fail the request: %v", err) + } + if pending { + t.Fatal("a stale index entry must not provide backpressure") + } + // The stale entry is removed so it does not have to be handled again. + members, err := store.ActiveRuns(ctx) + if err != nil { + t.Fatal(err) + } + for _, member := range members { + if member == "ghost-run" { + t.Fatal("the stale index entry must be removed") + } + } +} + +// TestProduceCreatesRunAfterZombie covers the end-to-end path through Produce: +// a zombie run must not stop a new run from being created. +func TestProduceCreatesRunAfterZombie(t *testing.T) { + store, err := zset.New(testutil.Redis(t), "flow:") + if err != nil { + t.Fatal(err) + } + ctx := context.Background() + producer, err := New(&pageScanner{cards: []Card{ + {ID: 1, ChanKey: "channel-a", TempKey: "sim_yd_v5", Operator: 1}, + }}, store, Config{PageSize: 100, RoundRetryEnabled: true}) + if err != nil { + t.Fatal(err) + } + + // A zombie run from a crashed producer. + if err := store.CreateRun(ctx, zset.RunMeta{RunID: "zombie-run", Status: "producing", ProducerDone: false, CurrentRound: 1}); err != nil { + t.Fatal(err) + } + + // A new request must still produce its own run. + if err := producer.Produce(ctx, scopedMessage("fresh-run", "channel-a")); err != nil { + t.Fatalf("a zombie run must not stop a new run: %v", err) + } + meta, err := store.GetRunMeta(ctx, "fresh-run") + if err != nil { + t.Fatal(err) + } + // The producer enqueues into the active ZSet; ActiveCount is the count it + // maintains. EnqueuedCount is an accounting field the producer does not set. + if !meta.ProducerDone || meta.ActiveCount != 1 { + t.Fatalf("fresh run metadata = %+v, want producerDone with 1 active task", meta) + } +} diff --git a/internal/producer/simstatus/producer.go b/internal/producer/simstatus/producer.go new file mode 100644 index 0000000..1f9ea9f --- /dev/null +++ b/internal/producer/simstatus/producer.go @@ -0,0 +1,459 @@ +package simstatus + +import ( + "context" + "crypto/rand" + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + "strings" + "sync/atomic" + "time" + + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +const FullSyncOperation = "sim_status_queue_full_sync" + +// scopeSampleLimit bounds how many channel names are copied into run metadata. +const scopeSampleLimit = 10 + +// productionStats mirrors the monthly usage producer so both full-sync paths +// report a single comparable summary line. The status path previously had no +// producer summary at all, which made a scope change impossible to verify. +type productionStats struct { + pages int64 + scanned int64 + enqueued int64 + skippedEmptyChan int64 + skippedNoTemplate int64 + skippedSyncDisabled int64 + skippedOutOfScope int64 + scopeChannels int + channels map[string]struct{} +} + +func newProductionStats(scopeChannels int) productionStats { + return productionStats{channels: make(map[string]struct{}), scopeChannels: scopeChannels} +} + +func (s *productionStats) log(ctx context.Context, status, runID, stage, reason string, startedAt time.Time, maxID int64, err error) { + message := fmt.Sprintf("event=sim_status_queue_producer status=%s runId=%s snapshotMaxId=%d pages=%d scanned=%d enqueued=%d skippedEmptyChan=%d skippedNoTemplate=%d skippedSyncDisabled=%d skippedOutOfScope=%d channelCount=%d scopeChannels=%d durationMs=%d", status, runID, maxID, s.pages, s.scanned, s.enqueued, s.skippedEmptyChan, s.skippedNoTemplate, s.skippedSyncDisabled, s.skippedOutOfScope, len(s.channels), s.scopeChannels, time.Since(startedAt).Milliseconds()) + if s.scanned > 0 { + message += fmt.Sprintf(" cardsPerSec=%.2f", float64(s.scanned)/time.Since(startedAt).Seconds()) + } + if stage != "" { + message += " stage=" + stage + } + if reason != "" { + message += " reason=" + reason + } + if err != nil { + message += fmt.Sprintf(" error=%q", err.Error()) + } + switch status { + case "failed": + g.Log().Error(ctx, message) + case "skipped": + g.Log().Warning(ctx, message) + default: + g.Log().Info(ctx, message) + } +} + +type Config struct { + PageSize int + RunLockKey string + RunLockTTL time.Duration + PollInterval time.Duration + GlobalHighWaterMark int64 +} + +func (c Config) withDefaults() Config { + if c.PageSize <= 0 { + c.PageSize = 1000 + } + if c.RunLockKey == "" { + c.RunLockKey = "sim-status:producer:active" + } + if c.RunLockTTL <= 0 { + c.RunLockTTL = 5 * time.Minute + } + if c.PollInterval <= 0 { + c.PollInterval = time.Second + } + return c +} + +type Card struct { + ID int64 + ICCID string + MSISDN string + IMSI string + ChanKey string + TempKey string + Operator int + SyncEnabled bool +} + +type Scanner interface { + SnapshotMaxID(context.Context) (int64, error) + Scan(context.Context, int64, int64, int) ([]Card, error) +} + +type channelMetadata struct { + ChanKey string `orm:"chan_key"` + TempKey string `orm:"temp_key"` + Operator int `orm:"operator"` + SyncEnabled bool `orm:"sync_enabled"` +} + +// DatabaseScanner uses keyset pagination so a large status run does not hold +// the entire card table in memory and does not degrade through OFFSET scans. +type DatabaseScanner struct { + metadata map[string]channelMetadata +} + +func (s *DatabaseScanner) SnapshotMaxID(ctx context.Context) (int64, error) { + var card entity.TSimCardInfo + columns := dao.TSimCardInfo.Columns() + if err := dao.TSimCardInfo.Ctx(ctx). + Fields(columns.Id). + OrderDesc(columns.Id). + Limit(1). + Scan(&card); err != nil { + return 0, fmt.Errorf("read status snapshot max id: %w", err) + } + return card.Id, nil +} + +func (s *DatabaseScanner) Scan(ctx context.Context, lastID, maxID int64, pageSize int) ([]Card, error) { + if maxID <= lastID { + return nil, nil + } + if s.metadata == nil { + var rows []channelMetadata + if err := g.DB().Model("t_sim_channels").Fields("chan_key,temp_key,sync_enabled,operator").Scan(&rows); err != nil { + return nil, fmt.Errorf("scan status channel metadata: %w", err) + } + s.metadata = make(map[string]channelMetadata, len(rows)) + for _, row := range rows { + s.metadata[row.ChanKey] = row + } + } + columns := dao.TSimCardInfo.Columns() + var rows []entity.TSimCardInfo + if err := dao.TSimCardInfo.Ctx(ctx). + Fields(columns.Id, columns.Iccid, columns.Msisdn, columns.Imsi, columns.ChanKey). + WhereGT(columns.Id, lastID). + WhereLTE(columns.Id, maxID). + OrderAsc(columns.Id). + Limit(pageSize). + Scan(&rows); err != nil { + return nil, fmt.Errorf("scan status cards: %w", err) + } + cards := make([]Card, 0, len(rows)) + for _, row := range rows { + if row.Id <= 0 || row.Iccid == "" || row.ChanKey == "" { + // Keep the page cursor moving even when a row cannot become a + // runnable task. The producer advances by the highest returned ID. + if row.Id > 0 { + cards = append(cards, Card{ID: row.Id}) + } + continue + } + metadata := s.metadata[row.ChanKey] + cards = append(cards, Card{ID: row.Id, ICCID: row.Iccid, MSISDN: row.Msisdn, IMSI: row.Imsi, ChanKey: row.ChanKey, TempKey: metadata.TempKey, Operator: metadata.Operator, SyncEnabled: metadata.SyncEnabled}) + } + return cards, nil +} + +type Producer struct { + scanner Scanner + store *queue.Store + config Config + running atomic.Int32 + owner string +} + +var ErrAlreadyRunning = fmt.Errorf("sim status producer already running") + +// ErrRunFinished is returned when a redelivered start message names a run that +// already reached a terminal state. The run must not be resurrected. +var ErrRunFinished = fmt.Errorf("sim status run already finished") + +// runAlreadyFinished reports whether runID exists and has already completed. +// A missing run is not an error: the message may be the first delivery. +func (p *Producer) runAlreadyFinished(ctx context.Context, runID string) (bool, error) { + meta, err := p.store.GetRunMeta(ctx, runID) + if err != nil { + if strings.Contains(err.Error(), "not found") { + return false, nil + } + return false, err + } + return meta.Status == "completed" || meta.Status == "completed_with_failures", nil +} + +func New(scanner Scanner, store *queue.Store, config Config) (*Producer, error) { + if scanner == nil || store == nil { + return nil, fmt.Errorf("sim status producer dependencies are incomplete") + } + return &Producer{scanner: scanner, store: store, config: config.withDefaults(), owner: newOwner()}, nil +} + +func (p *Producer) Produce(ctx context.Context, message model.SimStatusQueueStartMessage) error { + startedAt := time.Now() + if message.RunID == "" { + return fmt.Errorf("status sync runId is empty") + } + operation := message.Operation + if operation == "" { + operation = FullSyncOperation + } + scope, err := model.NormalizeScope(message.Scope) + if err != nil { + failedStats := newProductionStats(0) + failedStats.log(ctx, "failed", message.RunID, "validate", "invalid_scope", startedAt, 0, err) + return err + } + stats := newProductionStats(scope.ChannelCount()) + if !p.running.CompareAndSwap(0, 1) { + stats.log(ctx, "skipped", message.RunID, "precheck", "already_running", startedAt, 0, nil) + return ErrAlreadyRunning + } + defer p.running.Store(0) + // A redelivered start message must not resurrect a run that already + // finished; that would create a second producer pass over the same RunID. + if finished, checkErr := p.runAlreadyFinished(ctx, message.RunID); checkErr != nil { + stats.log(ctx, "failed", message.RunID, "run_state", "", startedAt, 0, checkErr) + return checkErr + } else if finished { + stats.log(ctx, "skipped", message.RunID, "run_state", "run_finished", startedAt, 0, nil) + return ErrRunFinished + } + acquired, err := p.store.AcquireLockOrTakeover(ctx, p.config.RunLockKey, p.owner, message.RunID, p.config.RunLockTTL) + if err != nil { + stats.log(ctx, "failed", message.RunID, "lock", "", startedAt, 0, err) + return err + } + if !acquired { + stats.log(ctx, "skipped", message.RunID, "lock", "already_running", startedAt, 0, nil) + return ErrAlreadyRunning + } + defer func() { + releaseCtx, cancel := context.WithTimeout(context.Background(), time.Second) + defer cancel() + _, _ = p.store.ReleaseLock(releaseCtx, p.config.RunLockKey, p.owner) + }() + if err := p.store.CreateRun(ctx, queue.RunMeta{ + RunID: message.RunID, + Operation: operation, + Status: "producing", + CurrentRound: 1, + ScopeChannels: scope.ChannelCount(), + ScopeDigest: scopeDigest(scope), + ScopeChannelsSample: strings.Join(scope.ChannelSample(scopeSampleLimit), ","), + }); err != nil { + stats.log(ctx, "failed", message.RunID, "run_init", "", startedAt, 0, err) + return err + } + lockCtx, stopRenew := context.WithCancel(ctx) + renewErr := make(chan error, 1) + go p.renewLock(lockCtx, stopRenew, renewErr) + defer func() { + stopRenew() + select { + case <-renewErr: + default: + } + }() + maxID, err := p.scanner.SnapshotMaxID(ctx) + if err != nil { + stats.log(ctx, "failed", message.RunID, "snapshot", "", startedAt, 0, err) + return err + } + g.Log().Infof(ctx, "event=sim_status_queue_producer status=started runId=%s snapshotMaxId=%d scopeChannels=%d", message.RunID, maxID, scope.ChannelCount()) + stage := "scan" + defer func() { + if err != nil { + stats.log(ctx, "failed", message.RunID, stage, "", startedAt, maxID, err) + return + } + stats.log(ctx, "completed", message.RunID, "", "", startedAt, maxID, nil) + }() + lastID := int64(0) + for lastID < maxID { + select { + case err := <-renewErr: + return err + default: + } + stage = "scan" + cards, scanErr := p.scanner.Scan(ctx, lastID, maxID, p.config.PageSize) + if scanErr != nil { + return scanErr + } + if len(cards) == 0 { + break + } + stats.pages++ + stats.scanned += int64(len(cards)) + groups := make(map[string][]queue.Task) + for _, card := range cards { + if card.ID > lastID { + lastID = card.ID + } + if card.ChanKey == "" { + stats.skippedEmptyChan++ + continue + } + // Scope is applied before the template and sync_enabled checks so a + // partial run only reports on the channels it was asked for. + if !scope.Allows(card.ChanKey) { + stats.skippedOutOfScope++ + continue + } + if card.TempKey == "" { + stats.skippedNoTemplate++ + continue + } + if card.SyncEnabled { + // sync_enabled=1(否/不参与) → skip + stats.skippedSyncDisabled++ + continue + } + stats.channels[card.ChanKey] = struct{}{} + task := queue.Task{TaskID: queue.StableTaskID(message.RunID, operation, card.ID), RunID: message.RunID, Operation: operation, ChanKey: card.ChanKey, TempKey: card.TempKey, Operator: card.Operator, CardID: card.ID, ICCID: card.ICCID, MSISDN: card.MSISDN, IMSI: card.IMSI} + groups[card.ChanKey] = append(groups[card.ChanKey], task) + } + for chanKey, tasks := range groups { + stage = "enqueue" + added, enqueueErr := p.store.EnqueueRunBatch(ctx, message.RunID, chanKey, tasks, time.Now()) + if enqueueErr != nil { + return fmt.Errorf("enqueue status channel %s: %w", chanKey, enqueueErr) + } + stats.enqueued += int64(added) + } + stage = "capacity" + if err = p.waitForCapacity(ctx); err != nil { + return err + } + } + return p.store.MarkProducerDone(ctx, message.RunID) +} + +// scopeDigest is a stable short fingerprint of the selected channels. +func scopeDigest(scope *model.SyncScope) string { + if scope == nil || len(scope.Channels) == 0 { + return "" + } + digest := sha256.Sum256([]byte(strings.Join(scope.Channels, "\n"))) + return hex.EncodeToString(digest[:8]) +} + +func (p *Producer) renewLock(ctx context.Context, cancel context.CancelFunc, errors chan<- error) { + interval := p.config.RunLockTTL / 3 + if interval <= 0 { + interval = time.Second + } + ticker := time.NewTicker(interval) + defer ticker.Stop() + for { + select { + case <-ctx.Done(): + return + case <-ticker.C: + ok, err := p.store.RenewLock(ctx, p.config.RunLockKey, p.owner, p.config.RunLockTTL) + if err == nil && !ok { + err = ErrAlreadyRunning + } + if err != nil { + select { + case errors <- err: + default: + } + cancel() + return + } + } + } +} + +func (p *Producer) waitForCapacity(ctx context.Context) error { + if p.config.GlobalHighWaterMark <= 0 { + return nil + } + for { + runs, err := p.store.ActiveRuns(ctx) + if err != nil { + return err + } + var total int64 + for _, runID := range runs { + meta, sizeErr := p.store.GetRunMeta(ctx, runID) + if sizeErr != nil { + // The active index can list a run whose hash is already gone + // (expired, evicted, or cleaned by hand). Treat it as a stale + // index entry and drop it instead of failing the scan. + if errors.Is(sizeErr, queue.ErrRunNotFound) { + if _, removeErr := p.store.RedisClient().Do(ctx, "SREM", p.store.ActiveRunsKey(), runID); removeErr != nil { + return removeErr + } + g.Log().Warningf(ctx, "event=sim_status_queue_producer status=skipped stage=capacity reason=stale_index runId=%s", runID) + continue + } + return sizeErr + } + // ActiveCount includes leased tasks; do not count InflightCount twice. + total += meta.ActiveCount + meta.NextRoundCount + } + if total <= p.config.GlobalHighWaterMark { + return nil + } + select { + case <-ctx.Done(): + return ctx.Err() + case <-time.After(p.config.PollInterval): + } + } +} + +func (p *Producer) EnqueueCard(ctx context.Context, message model.SimStatusQueueTaskMessage) error { + if message.CardID <= 0 || message.ICCID == "" || message.ChanKey == "" || message.TempKey == "" { + return fmt.Errorf("invalid status sync task") + } + if message.RunID == "" { + message.RunID = "status-card-" + newOwner() + } + if message.Operation == "" { + message.Operation = "sim_status_queue_task" + } + if err := p.store.CreateRun(ctx, queue.RunMeta{ + RunID: message.RunID, + Operation: message.Operation, + Status: "producing", + CurrentRound: 1, + }); err != nil { + return err + } + task := queue.Task{TaskID: queue.StableTaskID(message.RunID, message.Operation, message.CardID), RunID: message.RunID, Operation: message.Operation, ChanKey: message.ChanKey, TempKey: message.TempKey, Operator: message.Operator, CardID: message.CardID, ICCID: message.ICCID, MSISDN: message.MSISDN, IMSI: message.IMSI} + if _, err := p.store.EnqueueRunBatch(ctx, message.RunID, message.ChanKey, []queue.Task{task}, time.Now()); err != nil { + return err + } + return p.store.MarkProducerDone(ctx, message.RunID) +} + +func newOwner() string { + var b [12]byte + if _, err := rand.Read(b[:]); err == nil { + return hex.EncodeToString(b[:]) + } + return fmt.Sprintf("%d", time.Now().UnixNano()) +} diff --git a/internal/producer/simstatus/producer_test.go b/internal/producer/simstatus/producer_test.go new file mode 100644 index 0000000..addf847 --- /dev/null +++ b/internal/producer/simstatus/producer_test.go @@ -0,0 +1,120 @@ +package simstatus + +import ( + "context" + "errors" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +type cardScanner struct{ cards []Card } + +func (s cardScanner) SnapshotMaxID(context.Context) (int64, error) { return int64(len(s.cards)), nil } +func (s cardScanner) Scan(_ context.Context, last, max int64, limit int) ([]Card, error) { + end := last + int64(limit) + if end > max { + end = max + } + return s.cards[last:end], nil +} + +func TestProducerUsesRunQueueForFullAndSingleCard(t *testing.T) { + redis := testutil.Redis(t) + ctx := context.Background() + store, err := queue.New(redis, "sim-status:") + if err != nil { + t.Fatal(err) + } + scanner := cardScanner{cards: []Card{ + {ID: 1, ICCID: "a", ChanKey: "ch", TempKey: "sim_yd_v5"}, + {ID: 2, ICCID: "b", ChanKey: "ch", TempKey: "sim_yd_v5", SyncEnabled: true}, + {ID: 3, ICCID: "c", ChanKey: "ch"}, + {ID: 4, ICCID: "d", ChanKey: "ch", TempKey: "sim_yd_v5"}, + }} + producer, err := New(scanner, store, Config{PageSize: 1}) + if err != nil { + t.Fatal(err) + } + for i := 0; i < 2; i++ { + if err = producer.Produce(ctx, model.SimStatusQueueStartMessage{RunID: "full"}); err != nil { + t.Fatal(err) + } + } + meta, err := store.GetRunMeta(ctx, "full") + if err != nil || !meta.ProducerDone || meta.ActiveCount != 2 || meta.Status != "running" { + t.Fatalf("full metadata=%+v err=%v", meta, err) + } + claims, err := store.ClaimRun(ctx, "full", "ch", time.Now(), time.Now().Add(time.Minute), 10) + if err != nil || len(claims) != 2 { + t.Fatalf("full claims=%+v err=%v", claims, err) + } + if n, err := store.Size(ctx, "ch"); err != nil || n != 0 { + t.Fatalf("legacy queue populated: %d %v", n, err) + } + message := model.SimStatusQueueTaskMessage{RunID: "single", CardID: 42, ICCID: "single", ChanKey: "ch", TempKey: "sim_yd_v5"} + for i := 0; i < 2; i++ { + if err = producer.EnqueueCard(ctx, message); err != nil { + t.Fatal(err) + } + } + meta, err = store.GetRunMeta(ctx, "single") + if err != nil || !meta.ProducerDone || meta.ActiveCount != 1 { + t.Fatalf("single metadata=%+v err=%v", meta, err) + } + claims, err = store.ClaimRun(ctx, "single", "ch", time.Now(), time.Now().Add(time.Minute), 10) + if err != nil || len(claims) != 1 { + t.Fatalf("single claims=%+v err=%v", claims, err) + } + if n, err := store.FinalizeRunTasks(ctx, claims, []queue.TaskOutcome{{Success: true}}, 1); err != nil || n != 1 { + t.Fatalf("single finalize: %d %v", n, err) + } + if ok, err := store.CompleteRunIfIdle(ctx, "single"); err != nil || !ok { + t.Fatalf("single complete: %v %v", ok, err) + } + if err = producer.EnqueueCard(ctx, message); err != nil { + t.Fatal(err) + } + meta, err = store.GetRunMeta(ctx, "single") + if err != nil || meta.Status != "completed" || meta.SuccessCount != 1 || meta.ActiveCount != 0 { + t.Fatalf("replayed single=%+v err=%v", meta, err) + } +} + +func TestProducerBackpressureIncludesLeasedRunTasks(t *testing.T) { + redis := testutil.Redis(t) + store, _ := queue.New(redis, "sim-status:") + ctx := context.Background() + producer, err := New(cardScanner{}, store, Config{GlobalHighWaterMark: 1, PollInterval: time.Millisecond}) + if err != nil { + t.Fatal(err) + } + if err = store.CreateRun(ctx, queue.RunMeta{RunID: "busy"}); err != nil { + t.Fatal(err) + } + tasks := []queue.Task{{RunID: "busy", TaskID: "1", ChanKey: "ch"}, {RunID: "busy", TaskID: "2", ChanKey: "ch"}} + if _, err = store.EnqueueRunBatch(ctx, "busy", "ch", tasks, time.Now()); err != nil { + t.Fatal(err) + } + claims, err := store.ClaimRun(ctx, "busy", "ch", time.Now(), time.Now().Add(time.Minute), 2) + if err != nil || len(claims) != 2 { + t.Fatalf("claim=%v err=%v", claims, err) + } + waitCtx, cancel := context.WithTimeout(ctx, 30*time.Millisecond) + defer cancel() + if err = producer.waitForCapacity(waitCtx); !errors.Is(err, context.DeadlineExceeded) { + t.Fatalf("did not wait on run backlog: %v", err) + } + if _, err = store.FinalizeRunTasks(ctx, claims[:1], []queue.TaskOutcome{{Success: true}}); err != nil { + t.Fatal(err) + } + // One remaining lease meets the watermark: InflightCount must not be added again. + waitCtx2, cancel2 := context.WithTimeout(ctx, time.Second) + defer cancel2() + if err = producer.waitForCapacity(waitCtx2); err != nil { + t.Fatalf("double-counted lease: %v", err) + } +} diff --git a/internal/producer/simstatus/scope_test.go b/internal/producer/simstatus/scope_test.go new file mode 100644 index 0000000..c308f20 --- /dev/null +++ b/internal/producer/simstatus/scope_test.go @@ -0,0 +1,223 @@ +package simstatus + +import ( + "context" + "errors" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +func newScopeTestProducer(t *testing.T, cards []Card) (*Producer, *queue.Store) { + t.Helper() + store, err := queue.New(testutil.Redis(t), "sim-status:") + if err != nil { + t.Fatal(err) + } + producer, err := New(cardScanner{cards: cards}, store, Config{PageSize: 10}) + if err != nil { + t.Fatal(err) + } + return producer, store +} + +func scopeMessage(runID string, channels ...string) model.SimStatusQueueStartMessage { + return model.SimStatusQueueStartMessage{ + RunID: runID, + Scope: &model.SyncScope{Channels: channels}, + } +} + +func TestProduceScopeEnqueuesOnlySelectedChannels(t *testing.T) { + cards := []Card{ + {ID: 1, ICCID: "a", ChanKey: "217", TempKey: "sim_yd_v5"}, + {ID: 2, ICCID: "b", ChanKey: "173", TempKey: "sim_yd_v5"}, + {ID: 3, ICCID: "c", ChanKey: "217", TempKey: "sim_yd_v5"}, + {ID: 4, ICCID: "d", ChanKey: "189", TempKey: "sim_yd_v5"}, + } + producer, store := newScopeTestProducer(t, cards) + ctx := context.Background() + + if err := producer.Produce(ctx, scopeMessage("scoped", "217")); err != nil { + t.Fatal(err) + } + + channels, err := store.RunChannels(ctx, "scoped") + if err != nil { + t.Fatal(err) + } + if len(channels) != 1 || channels[0] != "217" { + t.Fatalf("run channels = %v, want only 217", channels) + } + meta, err := store.GetRunMeta(ctx, "scoped") + if err != nil { + t.Fatal(err) + } + if meta.ScopeChannels != 1 { + t.Fatalf("scopeChannels = %d, want 1", meta.ScopeChannels) + } + if meta.ScopeDigest == "" { + t.Fatalf("scopeDigest must be recorded for a partial run") + } + if meta.ScopeChannelsSample != "217" { + t.Fatalf("scopeChannelsSample = %q, want 217", meta.ScopeChannelsSample) + } +} + +func TestProduceFullScopeRecordsNoScopeMetadata(t *testing.T) { + cards := []Card{ + {ID: 1, ICCID: "a", ChanKey: "217", TempKey: "sim_yd_v5"}, + {ID: 2, ICCID: "b", ChanKey: "173", TempKey: "sim_yd_v5"}, + } + producer, store := newScopeTestProducer(t, cards) + ctx := context.Background() + + if err := producer.Produce(ctx, model.SimStatusQueueStartMessage{RunID: "full"}); err != nil { + t.Fatal(err) + } + meta, err := store.GetRunMeta(ctx, "full") + if err != nil { + t.Fatal(err) + } + if meta.ScopeChannels != 0 || meta.ScopeDigest != "" || meta.ScopeChannelsSample != "" { + t.Fatalf("full run must not record scope metadata: %+v", meta) + } + channels, err := store.RunChannels(ctx, "full") + if err != nil { + t.Fatal(err) + } + if len(channels) != 2 { + t.Fatalf("run channels = %v, want both channels", channels) + } +} + +func TestProduceScopeCombinesWithSyncDisabledGuard(t *testing.T) { + cards := []Card{ + {ID: 1, ICCID: "a", ChanKey: "217", TempKey: "sim_yd_v5"}, + // Selected but excluded by the sync_enabled guard: scope must not + // override it, and the run must still complete cleanly. + {ID: 2, ICCID: "b", ChanKey: "173", TempKey: "sim_yd_v5", SyncEnabled: true}, + {ID: 3, ICCID: "c", ChanKey: "189", TempKey: "sim_yd_v5"}, + } + producer, store := newScopeTestProducer(t, cards) + ctx := context.Background() + + if err := producer.Produce(ctx, scopeMessage("scoped", "173")); err != nil { + t.Fatal(err) + } + channels, err := store.RunChannels(ctx, "scoped") + if err != nil { + t.Fatal(err) + } + if len(channels) != 0 { + t.Fatalf("run channels = %v, want none (only selected channel is sync-disabled)", channels) + } + meta, err := store.GetRunMeta(ctx, "scoped") + if err != nil { + t.Fatal(err) + } + // The producer never enqueues work for a sync-disabled channel. The run is + // closed by the consumer's idle check, which is simulated here. + if !meta.ProducerDone || meta.ActiveCount != 0 || meta.EnqueuedCount != 0 { + t.Fatalf("empty scoped run metadata = %+v", meta) + } + if completed, err := store.CompleteRunIfIdle(ctx, "scoped"); err != nil || !completed { + t.Fatalf("empty scoped run must be closable: completed=%v err=%v", completed, err) + } + meta, err = store.GetRunMeta(ctx, "scoped") + if err != nil { + t.Fatal(err) + } + if meta.Status != "completed" { + t.Fatalf("status = %q, want completed", meta.Status) + } +} + +func TestProduceScopeSelectingUnknownChannelProducesEmptyRun(t *testing.T) { + cards := []Card{ + {ID: 1, ICCID: "a", ChanKey: "217", TempKey: "sim_yd_v5"}, + } + producer, store := newScopeTestProducer(t, cards) + ctx := context.Background() + + // An unknown channel is not an error; it simply matches no card. + if err := producer.Produce(ctx, scopeMessage("scoped", "does-not-exist")); err != nil { + t.Fatal(err) + } + meta, err := store.GetRunMeta(ctx, "scoped") + if err != nil { + t.Fatal(err) + } + if !meta.ProducerDone || meta.EnqueuedCount != 0 || meta.ActiveCount != 0 { + t.Fatalf("unknown-channel run metadata = %+v", meta) + } + if meta.ScopeChannels != 1 { + t.Fatalf("scopeChannels = %d, want the requested 1", meta.ScopeChannels) + } + if completed, err := store.CompleteRunIfIdle(ctx, "scoped"); err != nil || !completed { + t.Fatalf("unknown-channel run must be closable: completed=%v err=%v", completed, err) + } +} + +func TestProduceRejectsInvalidScope(t *testing.T) { + producer, store := newScopeTestProducer(t, []Card{{ID: 1, ICCID: "a", ChanKey: "217", TempKey: "sim_yd_v5"}}) + ctx := context.Background() + + err := producer.Produce(ctx, scopeMessage("bad", "invalid/chan")) + if err == nil { + t.Fatal("expected invalid chanKey to be rejected") + } + // A rejected request must not leave a run behind. + if _, metaErr := store.GetRunMeta(ctx, "bad"); metaErr == nil { + t.Fatal("rejected scope created a run") + } +} + +func TestProduceScopeDeduplicatesChannels(t *testing.T) { + producer, store := newScopeTestProducer(t, []Card{ + {ID: 1, ICCID: "a", ChanKey: "217", TempKey: "sim_yd_v5"}, + }) + ctx := context.Background() + + if err := producer.Produce(ctx, scopeMessage("scoped", "217", "217", "217")); err != nil { + t.Fatal(err) + } + meta, err := store.GetRunMeta(ctx, "scoped") + if err != nil { + t.Fatal(err) + } + if meta.ScopeChannels != 1 { + t.Fatalf("scopeChannels = %d, want 1 after deduplication", meta.ScopeChannels) + } +} + +func TestProduceRejectsRedeliveryOfFinishedRun(t *testing.T) { + producer, store := newScopeTestProducer(t, []Card{ + {ID: 1, ICCID: "a", ChanKey: "217", TempKey: "sim_yd_v5"}, + }) + ctx := context.Background() + + if err := producer.Produce(ctx, model.SimStatusQueueStartMessage{RunID: "done"}); err != nil { + t.Fatal(err) + } + // Drain the queued work so the run reaches a terminal state. + claims, err := store.ClaimRun(ctx, "done", "217", time.Now(), time.Now().Add(time.Minute), 10) + if err != nil || len(claims) != 1 { + t.Fatalf("claims=%v err=%v", claims, err) + } + if _, err = store.FinalizeRunTasks(ctx, claims, []queue.TaskOutcome{{Success: true}}); err != nil { + t.Fatal(err) + } + if completed, err := store.CompleteRunIfIdle(ctx, "done"); err != nil || !completed { + t.Fatalf("complete run: completed=%v err=%v", completed, err) + } + + // A redelivered start message must not resurrect the finished run. + err = producer.Produce(ctx, model.SimStatusQueueStartMessage{RunID: "done"}) + if !errors.Is(err, ErrRunFinished) { + t.Fatalf("redelivery error = %v, want ErrRunFinished", err) + } +} diff --git a/internal/queue/zset/lock.go b/internal/queue/zset/lock.go new file mode 100644 index 0000000..6b074e5 --- /dev/null +++ b/internal/queue/zset/lock.go @@ -0,0 +1,171 @@ +package zset + +import ( + "context" + "encoding/json" + "fmt" + "time" +) + +// AcquireLock obtains a Redis lease only when the key is absent. +func (s *Store) AcquireLock(ctx context.Context, key, token string, ttl time.Duration) (bool, error) { + if key == "" || token == "" { + return false, fmt.Errorf("lock key and token are required") + } + if ttl <= 0 { + return false, fmt.Errorf("lock ttl must be positive") + } + result, err := s.redis.Do(ctx, "SET", key, token, "NX", "PX", ttl.Milliseconds()) + if err != nil { + return false, fmt.Errorf("acquire lock: %w", err) + } + return result.String() == "OK", nil +} + +// RenewLock extends the lease only when the token still owns the key. +func (s *Store) RenewLock(ctx context.Context, key, token string, ttl time.Duration) (bool, error) { + if key == "" || token == "" { + return false, fmt.Errorf("lock key and token are required") + } + if ttl <= 0 { + return false, fmt.Errorf("lock ttl must be positive") + } + result, err := s.redis.Eval(ctx, renewLockScript, 1, []string{key}, []any{token, ttl.Milliseconds()}) + if err != nil { + return false, fmt.Errorf("renew lock: %w", err) + } + return result.Int() == 1, nil +} + +// ReleaseLock removes the key only when the token still owns it. +func (s *Store) ReleaseLock(ctx context.Context, key, token string) (bool, error) { + if key == "" || token == "" { + return false, fmt.Errorf("lock key and token are required") + } + result, err := s.redis.Eval(ctx, releaseLockScript, 1, []string{key}, []any{token}) + if err != nil { + return false, fmt.Errorf("release lock: %w", err) + } + return result.Int() == 1, nil +} + +// AcquireLockOrTakeover acquires key for token, or takes it over when the +// existing lock cannot belong to a live owner of different work. +// +// This exists because a producer that dies (for example on a Redis EOF) leaves +// its lock behind for the remainder of the TTL. A redelivered start message for +// that same run would otherwise fail to acquire the lock, be reported as +// "already running", and be acked and dropped by the consumer - the request +// would disappear without ever reaching the idempotent rescan path that +// hasPendingTasks deliberately allows. +// +// Takeover rules, in order: +// +// 1. No lock: acquire it. +// 2. Structured lock for the same runID: take it over. Owner tokens are +// regenerated per process, so only the runID can identify "the same work". +// 3. Structured lock for a different runID: refuse. Another run is live. +// 4. Bare (legacy) token: take it over only when it is provably stale (see +// below). A bare value can only have been written by a build that predates +// structured locks. Refusing outright would strand the lock for the whole +// TTL and drop the request, which is the exact failure this method exists to +// prevent. Taking it over unconditionally would be unsafe: during a rolling +// deploy an older process still writes and renews a bare token, and stealing +// a live lock lets two producers run at once. +// +// The staleness test for a legacy lock: a live holder renews every TTL/3 (see +// Producer.renewLock), so its remaining TTL never falls below 2*TTL/3. A +// remaining TTL at or below TTL/2 therefore proves the holder stopped renewing. +// A lock with no expiry at all is treated as stale, because every writer in this +// codebase sets one. +// +// runID may be empty, in which case the call degrades to AcquireLock. +func (s *Store) AcquireLockOrTakeover(ctx context.Context, key, token, runID string, ttl time.Duration) (bool, error) { + if key == "" || token == "" { + return false, fmt.Errorf("lock key and token are required") + } + if ttl <= 0 { + return false, fmt.Errorf("lock ttl must be positive") + } + if runID == "" { + return s.AcquireLock(ctx, key, token, ttl) + } + value, err := marshalLockValue(lockValue{Owner: token, RunID: runID}) + if err != nil { + return false, err + } + result, err := s.redis.Eval(ctx, acquireOrTakeoverLockScript, 1, []string{key}, []any{value, ttl.Milliseconds()}) + if err != nil { + return false, fmt.Errorf("acquire or take over lock: %w", err) + } + return result.Int() == 1, nil +} + +// lockValue is the structured lock payload. Older locks hold a bare owner +// token; the scripts accept both so a rolling deploy is safe. +type lockValue struct { + Owner string `json:"owner"` + RunID string `json:"runId,omitempty"` +} + +func marshalLockValue(value lockValue) (string, error) { + encoded, err := json.Marshal(value) + if err != nil { + return "", fmt.Errorf("encode lock value: %w", err) + } + return string(encoded), nil +} + +// renewLockScript extends the lease when the token still owns the key. The +// stored value may be a bare legacy token or a structured lockValue. +const renewLockScript = ` +local current = redis.call('GET', KEYS[1]) +if not current then return 0 end +local ok, decoded = pcall(cjson.decode, current) +local owner = current +if ok and type(decoded) == 'table' and decoded['owner'] then owner = decoded['owner'] end +if owner ~= ARGV[1] then return 0 end +redis.call('PEXPIRE', KEYS[1], ARGV[2]) +return 1 +` + +// releaseLockScript deletes the key only when the token still owns it. +const releaseLockScript = ` +local current = redis.call('GET', KEYS[1]) +if not current then return 0 end +local ok, decoded = pcall(cjson.decode, current) +local owner = current +if ok and type(decoded) == 'table' and decoded['owner'] then owner = decoded['owner'] end +if owner ~= ARGV[1] then return 0 end +redis.call('DEL', KEYS[1]) +return 1 +` + +// acquireOrTakeoverLockScript sets the key when free, or replaces it when the +// existing lock either belongs to the same run or is a legacy bare token that +// cannot belong to a live structured-lock holder. Returns 1 on success. +const acquireOrTakeoverLockScript = ` +local current = redis.call('GET', KEYS[1]) +if not current then + redis.call('SET', KEYS[1], ARGV[1], 'PX', ARGV[2]) + return 1 +end +local okNext, wanted = pcall(cjson.decode, ARGV[1]) +if not okNext or type(wanted) ~= 'table' or wanted['runId'] == nil then return 0 end +local ok, decoded = pcall(cjson.decode, current) +if not ok or type(decoded) ~= 'table' then + -- Legacy bare token. A live holder renews every TTL/3, so its remaining TTL + -- never drops below 2*TTL/3; at or below TTL/2 the holder has stopped + -- renewing and the lock is stale. A missing expiry is stale too, because + -- every writer sets one. Refuse a still-live legacy lock so a rolling deploy + -- cannot end up with two producers. + local pttl = redis.call('PTTL', KEYS[1]) + if pttl > 0 and pttl > math.floor(tonumber(ARGV[2]) / 2) then return 0 end + redis.call('SET', KEYS[1], ARGV[1], 'PX', ARGV[2]) + return 1 +end +if decoded['runId'] == nil then return 0 end +if decoded['runId'] ~= wanted['runId'] then return 0 end +redis.call('SET', KEYS[1], ARGV[1], 'PX', ARGV[2]) +return 1 +` diff --git a/internal/queue/zset/lock_test.go b/internal/queue/zset/lock_test.go new file mode 100644 index 0000000..36a22d5 --- /dev/null +++ b/internal/queue/zset/lock_test.go @@ -0,0 +1,189 @@ +package zset + +import ( + "context" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +func newLockStore(t *testing.T) *Store { + t.Helper() + store, err := New(testutil.Redis(t), "flow:") + if err != nil { + t.Fatal(err) + } + return store +} + +func TestAcquireLockOrTakeoverBlocksOtherRuns(t *testing.T) { + store := newLockStore(t) + ctx := context.Background() + + acquired, err := store.AcquireLockOrTakeover(ctx, "flow:producer:active", "owner-a", "run-1", time.Minute) + if err != nil || !acquired { + t.Fatalf("first acquire: acquired=%v err=%v", acquired, err) + } + // A different run must not steal the lock. + acquired, err = store.AcquireLockOrTakeover(ctx, "flow:producer:active", "owner-b", "run-2", time.Minute) + if err != nil { + t.Fatal(err) + } + if acquired { + t.Fatal("a different run took over the lock") + } +} + +func TestAcquireLockOrTakeoverResumesSameRunAfterCrash(t *testing.T) { + store := newLockStore(t) + ctx := context.Background() + + // Simulate a producer that died without releasing its lock. + if acquired, err := store.AcquireLockOrTakeover(ctx, "flow:producer:active", "dead-owner", "run-1", time.Minute); err != nil || !acquired { + t.Fatalf("first acquire: acquired=%v err=%v", acquired, err) + } + // The redelivered message arrives with a fresh owner token but the same run. + acquired, err := store.AcquireLockOrTakeover(ctx, "flow:producer:active", "new-owner", "run-1", time.Minute) + if err != nil { + t.Fatal(err) + } + if !acquired { + t.Fatal("the same run could not resume after a producer crash") + } + // The new owner must now hold the lock for renew and release. + renewed, err := store.RenewLock(ctx, "flow:producer:active", "new-owner", time.Minute) + if err != nil || !renewed { + t.Fatalf("new owner cannot renew: renewed=%v err=%v", renewed, err) + } + released, err := store.ReleaseLock(ctx, "flow:producer:active", "new-owner") + if err != nil || !released { + t.Fatalf("new owner cannot release: released=%v err=%v", released, err) + } +} + +func TestAcquireLockOrTakeoverEmptyRunIDDegradesToAcquireLock(t *testing.T) { + store := newLockStore(t) + ctx := context.Background() + + if acquired, err := store.AcquireLockOrTakeover(ctx, "flow:producer:active", "owner-a", "", time.Minute); err != nil || !acquired { + t.Fatalf("first acquire: acquired=%v err=%v", acquired, err) + } + acquired, err := store.AcquireLockOrTakeover(ctx, "flow:producer:active", "owner-b", "", time.Minute) + if err != nil { + t.Fatal(err) + } + if acquired { + t.Fatal("an empty runID must not permit takeover") + } +} + +// TestAcquireLockOrTakeoverRefusesLiveLegacyLock covers the rolling-deploy case: +// an older build still holds a bare token and keeps renewing it. Stealing that +// lock would let two producers run at once, so it must be refused. +func TestAcquireLockOrTakeoverRefusesLiveLegacyLock(t *testing.T) { + store := newLockStore(t) + ctx := context.Background() + + // A live legacy holder renews every TTL/3, so its remaining TTL stays above + // TTL/2. AcquireLock writes the bare token with the full TTL, which is + // indistinguishable from a just-renewed lock. + if acquired, err := store.AcquireLock(ctx, "flow:producer:active", "legacy-owner", time.Minute); err != nil || !acquired { + t.Fatalf("legacy acquire: acquired=%v err=%v", acquired, err) + } + + acquired, err := store.AcquireLockOrTakeover(ctx, "flow:producer:active", "new-owner", "run-1", time.Minute) + if err != nil { + t.Fatal(err) + } + if acquired { + t.Fatal("a still-live legacy lock must not be taken over: that would let two producers run at once") + } + // The legacy owner keeps the lock. + if renewed, err := store.RenewLock(ctx, "flow:producer:active", "legacy-owner", time.Minute); err != nil || !renewed { + t.Fatalf("legacy owner lost the lock: renewed=%v err=%v", renewed, err) + } +} + +// TestAcquireLockOrTakeoverTakesOverStaleLegacyLock covers the case the takeover +// exists for: the legacy writer died, stopped renewing, and its TTL has fallen +// past the point a live holder could ever reach. +func TestAcquireLockOrTakeoverTakesOverStaleLegacyLock(t *testing.T) { + store := newLockStore(t) + ctx := context.Background() + + // A short TTL lets the test wait for the stale window without sleeping a + // full production TTL. After ~60% of the TTL has elapsed the remaining TTL is + // below TTL/2, which a renewing holder can never reach. + if acquired, err := store.AcquireLock(ctx, "flow:producer:active", "legacy-owner", 2*time.Second); err != nil || !acquired { + t.Fatalf("legacy acquire: acquired=%v err=%v", acquired, err) + } + time.Sleep(1300 * time.Millisecond) + + acquired, err := store.AcquireLockOrTakeover(ctx, "flow:producer:active", "new-owner", "run-1", time.Minute) + if err != nil { + t.Fatal(err) + } + if !acquired { + t.Fatal("a stale legacy lock must be taken over; refusing strands the key and drops the request") + } + // The new owner must now hold the lock, and the old token must not. + if renewed, err := store.RenewLock(ctx, "flow:producer:active", "new-owner", time.Minute); err != nil || !renewed { + t.Fatalf("new owner cannot renew: renewed=%v err=%v", renewed, err) + } + if released, err := store.ReleaseLock(ctx, "flow:producer:active", "legacy-owner"); err != nil || released { + t.Fatalf("legacy owner must no longer hold the lock: released=%v err=%v", released, err) + } +} + +// TestAcquireLockOrTakeoverTakesOverLegacyLockWithoutExpiry treats a legacy lock +// that has no expiry as stale: every writer in this codebase sets one. +func TestAcquireLockOrTakeoverTakesOverLegacyLockWithoutExpiry(t *testing.T) { + store := newLockStore(t) + ctx := context.Background() + + // Simulate a legacy lock written without a TTL. + if _, err := store.RedisClient().Do(ctx, "SET", "flow:producer:active", "legacy-owner"); err != nil { + t.Fatal(err) + } + acquired, err := store.AcquireLockOrTakeover(ctx, "flow:producer:active", "new-owner", "run-1", time.Minute) + if err != nil { + t.Fatal(err) + } + if !acquired { + t.Fatal("a legacy lock with no expiry must be treated as stale and taken over") + } +} + +func TestAcquireLockOrTakeoverBlocksDifferentStructuredRun(t *testing.T) { + store := newLockStore(t) + ctx := context.Background() + + // A structured lock for a different run means another run is genuinely live. + if acquired, err := store.AcquireLockOrTakeover(ctx, "flow:producer:active", "owner-a", "run-1", time.Minute); err != nil || !acquired { + t.Fatalf("first acquire: acquired=%v err=%v", acquired, err) + } + acquired, err := store.AcquireLockOrTakeover(ctx, "flow:producer:active", "owner-b", "run-2", time.Minute) + if err != nil { + t.Fatal(err) + } + if acquired { + t.Fatal("a structured lock held by a different run must not be taken over") + } +} + +func TestAcquireLockOrTakeoverRejectsNonOwnerRelease(t *testing.T) { + store := newLockStore(t) + ctx := context.Background() + + if acquired, err := store.AcquireLockOrTakeover(ctx, "flow:producer:active", "owner-a", "run-1", time.Minute); err != nil || !acquired { + t.Fatalf("acquire: acquired=%v err=%v", acquired, err) + } + released, err := store.ReleaseLock(ctx, "flow:producer:active", "owner-b") + if err != nil { + t.Fatal(err) + } + if released { + t.Fatal("a non-owner released the lock") + } +} diff --git a/internal/queue/zset/producer_abandoned_test.go b/internal/queue/zset/producer_abandoned_test.go new file mode 100644 index 0000000..2b1aec1 --- /dev/null +++ b/internal/queue/zset/producer_abandoned_test.go @@ -0,0 +1,77 @@ +package zset + +import ( + "context" + "reflect" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +func TestMarkProducerAbandonedAtomicGuards(t *testing.T) { + store, err := New(testutil.Redis(t), "abandoned:") + if err != nil { + t.Fatal(err) + } + ctx := context.Background() + for _, test := range []struct { + name, status, syncType string + done, want bool + }{ + {name: "producing", status: "producing", want: true}, + {name: "blocked_running", status: "running", want: true}, + {name: "producing_done", status: "producing", done: true}, + {name: "running_done", status: "running", done: true}, + {name: "producing_replay", status: "producing", syncType: "replay"}, + {name: "running_replay", status: "running", syncType: "replay"}, + {name: "completed", status: "completed"}, + {name: "completed_with_failures", status: "completed_with_failures"}, + {name: "unknown_status", status: "paused"}, + {name: "missing"}, + } { + t.Run(test.name, func(t *testing.T) { + key, _ := store.RunMetaKey(test.name) + if test.status != "" { + if err := store.CreateRun(ctx, RunMeta{RunID: test.name, Status: test.status, ProducerDone: test.done, SyncType: test.syncType, ActiveCount: 3, InflightCount: 1, NextRoundCount: 2, FailedCount: 4}); err != nil { + t.Fatal(err) + } + if _, err := store.RedisClient().Do(ctx, "HSET", key, "blocked:a", "1"); err != nil { + t.Fatal(err) + } + } + before, err := store.RedisClient().Do(ctx, "HGETALL", key) + if err != nil { + t.Fatal(err) + } + promoted, err := store.MarkProducerAbandoned(ctx, test.name) + if err != nil || promoted != test.want { + t.Fatalf("promoted=%t want=%t err=%v", promoted, test.want, err) + } + after, err := store.RedisClient().Do(ctx, "HGETALL", key) + if err != nil { + t.Fatal(err) + } + expected := before.MapStrStr() + actual := after.MapStrStr() + if test.want { + if actual["abandonedAt"] == "" || actual["abandonedAt"] != actual["producerCompletedAt"] { + t.Fatalf("missing abandonment timestamps: %v", actual) + } + expected["producerDone"] = "1" + expected["status"] = "running" + expected["abandonedAt"] = actual["abandonedAt"] + expected["producerCompletedAt"] = actual["producerCompletedAt"] + } + if !reflect.DeepEqual(expected, actual) { + t.Fatalf("unexpected metadata change: want=%v got=%v", expected, actual) + } + if again, err := store.MarkProducerAbandoned(ctx, test.name); err != nil || again { + t.Fatalf("repeated promotion=%t err=%v", again, err) + } + unchanged, err := store.RedisClient().Do(ctx, "HGETALL", key) + if err != nil || !reflect.DeepEqual(actual, unchanged.MapStrStr()) { + t.Fatalf("idempotent promotion changed metadata: %v err=%v", unchanged, err) + } + }) + } +} diff --git a/internal/queue/zset/producer_lease.go b/internal/queue/zset/producer_lease.go new file mode 100644 index 0000000..20b9c55 --- /dev/null +++ b/internal/queue/zset/producer_lease.go @@ -0,0 +1,64 @@ +package zset + +import ( + "context" + "strings" + + "github.com/gogf/gf/v2/container/gvar" + "github.com/gogf/gf/v2/errors/gerror" +) + +var ErrProducerLeaseLost = gerror.New("producer lease lost") + +// WithIMEIProducerLease returns a private producer view; the shared store and +// other producer/consumer protocols are unchanged. The IMEI caller must reject +// Redis Cluster before using this view: the global lock and run keys cross slots. +func (s *Store) WithIMEIProducerLease(key, owner, runID string) (*Store, error) { + if key == "" || owner == "" || runID == "" { + return nil, gerror.New("producer lease key, owner and runId are required") + } + view := *s + view.redis = &imeiProducerRedis{RedisClient: s.redis, key: key, owner: owner, runID: runID} + return &view, nil +} + +type imeiProducerRedis struct { + RedisClient + key, owner, runID string +} + +func (r *imeiProducerRedis) Eval(ctx context.Context, script string, numKeys int64, keys []string, args []any) (*gvar.Var, error) { + leaseKeys := append(append([]string(nil), keys...), r.key) + leaseArgs := append(append([]any(nil), args...), r.owner, r.runID) + result, err := r.RedisClient.Eval(ctx, imeiProducerLeaseGuard+script, numKeys+1, leaseKeys, leaseArgs) + if err != nil && strings.Contains(err.Error(), "IMEI_PRODUCER_LEASE_LOST") { + return nil, gerror.Wrap(ErrProducerLeaseLost, "reject stale IMEI producer write") + } + return result, err +} + +func (r *imeiProducerRedis) Do(ctx context.Context, command string, args ...any) (*gvar.Var, error) { + switch strings.ToUpper(command) { + case "HGETALL", "SMEMBERS": // The producer's metadata/capacity reads are not mutations. + return r.RedisClient.Do(ctx, command, args...) + case "SADD", "HSET": // CreateRun also writes the index and metadata extensions. + commandArgs := append([]any{command}, args...) + return r.Eval(ctx, "return redis.call(unpack(ARGV))", 0, nil, commandArgs) + default: + return nil, gerror.Newf("unsupported command in IMEI producer lease view: %s", command) + } +} + +// Remove guard-only arguments before executing the original queue script, so +// task-pair loops and key indexing retain their original semantics. +const imeiProducerLeaseGuard = ` +local leaseRunID = table.remove(ARGV) +local leaseOwner = table.remove(ARGV) +local leaseKey = table.remove(KEYS) +local current = redis.call('GET', leaseKey) +if not current then return redis.error_reply('IMEI_PRODUCER_LEASE_LOST') end +local ok, lease = pcall(cjson.decode, current) +if not ok or type(lease) ~= 'table' or lease['owner'] ~= leaseOwner or lease['runId'] ~= leaseRunID then + return redis.error_reply('IMEI_PRODUCER_LEASE_LOST') +end +` diff --git a/internal/queue/zset/producer_lease_test.go b/internal/queue/zset/producer_lease_test.go new file mode 100644 index 0000000..be78188 --- /dev/null +++ b/internal/queue/zset/producer_lease_test.go @@ -0,0 +1,94 @@ +package zset + +import ( + "context" + "errors" + "testing" + "time" +) + +func TestIMEIProducerLeaseRejectsStaleWrites(t *testing.T) { + for _, invalid := range []string{"missing", "expired", "other_owner", "other_run", "legacy", "invalid_json"} { + t.Run(invalid, func(t *testing.T) { + store := newLockStore(t) + ctx := context.Background() + const key = "imei:producer:active" + view, err := store.WithIMEIProducerLease(key, "owner", "run") + if err != nil { + t.Fatal(err) + } + if err := store.CreateRun(ctx, RunMeta{RunID: "run"}); err != nil { + t.Fatal(err) + } + switch invalid { + case "expired": + if _, err := store.redis.Do(ctx, "SET", key, `{"owner":"owner","runId":"run"}`, "PX", 1); err != nil { + t.Fatal(err) + } + time.Sleep(10 * time.Millisecond) + case "other_owner": + _, err = store.redis.Do(ctx, "SET", key, `{"owner":"new-owner","runId":"run"}`) + case "other_run": + _, err = store.redis.Do(ctx, "SET", key, `{"owner":"owner","runId":"other-run"}`) + case "legacy": + _, err = store.redis.Do(ctx, "SET", key, "owner") + case "invalid_json": + _, err = store.redis.Do(ctx, "SET", key, "{") + } + if err != nil { + t.Fatal(err) + } + _, err = view.EnqueueRunBatch(ctx, "run", "ch", []Task{{TaskID: "old", RunID: "run", ChanKey: "ch"}}, time.Now()) + if !errors.Is(err, ErrProducerLeaseLost) { + t.Fatalf("enqueue did not report lease loss: %v", err) + } + if err := view.MarkProducerDone(ctx, "run"); !errors.Is(err, ErrProducerLeaseLost) { + t.Fatalf("done did not report lease loss: %v", err) + } + if err := view.CreateRun(ctx, RunMeta{RunID: "run", ScopeDigest: "stale"}); !errors.Is(err, ErrProducerLeaseLost) { + t.Fatalf("metadata write did not report lease loss: %v", err) + } + meta, err := store.GetRunMeta(ctx, "run") + if err != nil || meta.ActiveCount != 0 || meta.ProducerDone || meta.ScopeDigest != "" { + t.Fatalf("stale state mutation: %+v, %v", meta, err) + } + channels, err := store.RunChannels(ctx, "run") + if err != nil || len(channels) != 0 { + t.Fatalf("stale channel mutation: %v, %v", channels, err) + } + // The private view never changes the consumer/other-producer store. + if added, err := store.EnqueueRunBatch(ctx, "run", "ch", []Task{{TaskID: "next", RunID: "run", ChanKey: "ch"}}, time.Now()); err != nil || added != 1 { + t.Fatalf("shared queue semantics changed: %d, %v", added, err) + } + }) + } +} + +func TestIMEIProducerLeasePreservesValidOwnerSemantics(t *testing.T) { + store := newLockStore(t) + ctx := context.Background() + const key = "imei:producer:active" + if ok, err := store.AcquireLockOrTakeover(ctx, key, "owner", "run", time.Minute); err != nil || !ok { + t.Fatalf("acquire: %v, %v", ok, err) + } + view, err := store.WithIMEIProducerLease(key, "owner", "run") + if err != nil { + t.Fatal(err) + } + if err := view.CreateRun(ctx, RunMeta{RunID: "run", ScopeDigest: "scope"}); err != nil { + t.Fatal(err) + } + tasks := []Task{{TaskID: "one", RunID: "run", ChanKey: "ch"}, {TaskID: "two", RunID: "run", ChanKey: "ch"}} + for _, want := range []int{2, 0} { + if added, err := view.EnqueueRunBatch(ctx, "run", "ch", tasks, time.Now()); err != nil || added != want { + t.Fatalf("idempotent batch pairs: %d, %v; want %d", added, err, want) + } + } + if err := view.MarkProducerDone(ctx, "run"); err != nil { + t.Fatal(err) + } + meta, err := store.GetRunMeta(ctx, "run") + if err != nil || meta.ActiveCount != 2 || !meta.ProducerDone || meta.ScopeDigest != "scope" { + t.Fatalf("valid owner state: %+v, %v", meta, err) + } +} diff --git a/internal/queue/zset/producer_state.go b/internal/queue/zset/producer_state.go new file mode 100644 index 0000000..823e5dd --- /dev/null +++ b/internal/queue/zset/producer_state.go @@ -0,0 +1,70 @@ +package zset + +import ( + "context" + + "github.com/gogf/gf/v2/errors/gerror" +) + +// ProducerState is a small internal read view, not a new Run wire format. +// ActiveCount includes inflight; never add InflightCount to capacity usage. +type ProducerState struct { + Status string + ProducerDone bool + CurrentRound int + ActiveCount int64 + Blocked bool +} + +func (s *Store) ReadProducerState(ctx context.Context, runID, chanKey string) (ProducerState, error) { + key, err := s.RunMetaKey(runID) + if err != nil { + return ProducerState{}, err + } + result, err := s.redis.Do(ctx, "HMGET", key, "runId", "status", "producerDone", "currentRound", "activeCount", "blocked:"+chanKey) + if err != nil { + return ProducerState{}, gerror.Wrap(err, "read producer state") + } + values := result.Strings() + if len(values) != 6 || values[0] == "" { + return ProducerState{}, gerror.Wrapf(ErrRunNotFound, "run %q", runID) + } + state := ProducerState{Status: values[1], ProducerDone: parseBool(values[2]), CurrentRound: parseInt(values[3]), ActiveCount: parseInt64(values[4]), Blocked: parseBool(values[5])} + if state.CurrentRound <= 0 || state.ActiveCount < 0 { + return ProducerState{}, gerror.New("invalid producer run counters") + } + return state, nil +} + +// ExistingRunTasks uses one bounded HMGET for a producer page. All task states +// (queued, leased, waiting, success and failed) count as already admitted. +func (s *Store) ExistingRunTasks(ctx context.Context, runID string, tasks []Task) ([]bool, error) { + if len(tasks) == 0 { + return nil, nil + } + key, err := s.TaskStateKey(runID) + if err != nil { + return nil, err + } + args := make([]any, 1, len(tasks)+1) + args[0] = key + for _, task := range tasks { + if task.TaskID == "" || task.RunID != runID { + return nil, gerror.New("invalid task identity in producer state read") + } + args = append(args, task.TaskID) + } + result, err := s.redis.Do(ctx, "HMGET", args...) + if err != nil { + return nil, gerror.Wrap(err, "read existing run tasks") + } + values := result.Strings() + if len(values) != len(tasks) { + return nil, gerror.New("incomplete HMGET task state response") + } + exists := make([]bool, len(tasks)) + for i, value := range values { + exists[i] = value != "" + } + return exists, nil +} diff --git a/internal/queue/zset/producer_state_test.go b/internal/queue/zset/producer_state_test.go new file mode 100644 index 0000000..513aee3 --- /dev/null +++ b/internal/queue/zset/producer_state_test.go @@ -0,0 +1,45 @@ +package zset + +import ( + "context" + "errors" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +func TestProducerStateReadView(t *testing.T) { + store, err := New(testutil.Redis(t), "producer-state:") + if err != nil { + t.Fatal(err) + } + ctx := context.Background() + if _, err = store.ReadProducerState(ctx, "missing", "a"); !errors.Is(err, ErrRunNotFound) { + t.Fatalf("missing run err=%v", err) + } + if err = store.CreateRun(ctx, RunMeta{RunID: "state", Status: "producing", ActiveCount: 3, InflightCount: 2}); err != nil { + t.Fatal(err) + } + metaKey, _ := store.RunMetaKey("state") + _, _ = store.RedisClient().Do(ctx, "HSET", metaKey, "blocked:a", "1") + state, err := store.ReadProducerState(ctx, "state", "a") + if err != nil || state.ActiveCount != 3 || state.ProducerDone || state.CurrentRound != 1 || !state.Blocked { + t.Fatalf("state=%+v err=%v", state, err) + } + // Every state deduplicates admission, including terminal and leased tasks. + statesKey, _ := store.TaskStateKey("state") + _, _ = store.RedisClient().Do(ctx, "HSET", statesKey, "success", "succeeded", "leased", "claimed", "waiting", "waiting_next_round") + exists, err := store.ExistingRunTasks(ctx, "state", []Task{{TaskID: "success", RunID: "state"}, {TaskID: "leased", RunID: "state"}, {TaskID: "waiting", RunID: "state"}, {TaskID: "new", RunID: "state"}}) + if err != nil || len(exists) != 4 || !exists[0] || !exists[1] || !exists[2] || exists[3] { + t.Fatalf("exists=%v err=%v", exists, err) + } + if _, err = store.ExistingRunTasks(ctx, "state", []Task{{RunID: "foreign", TaskID: "x"}}); err == nil { + t.Fatal("foreign task accepted") + } + for _, args := range [][]any{{"currentRound", 0}, {"currentRound", 1, "activeCount", -1}} { + _, _ = store.RedisClient().Do(ctx, "HSET", append([]any{metaKey}, args...)...) + if _, err = store.ReadProducerState(ctx, "state", "a"); err == nil { + t.Fatal("invalid run counters accepted") + } + } +} diff --git a/internal/queue/zset/run.go b/internal/queue/zset/run.go new file mode 100644 index 0000000..8d827ba --- /dev/null +++ b/internal/queue/zset/run.go @@ -0,0 +1,1156 @@ +package zset + +// This file contains the run-scoped queue protocol used by the full traffic +// usage synchronizer. The original channel-scoped API in store.go remains +// intentionally unchanged; run keys are separate so old, in-flight work can +// continue to be consumed while new producers use round barriers. + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "sort" + "strconv" + "strings" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "github.com/gogf/gf/v2/errors/gerror" +) + +// ErrRunNotFound reports that a run's metadata hash does not exist. Callers use +// errors.Is so a stale active-index entry can be told apart from a real read +// failure. +var ErrRunNotFound = errors.New("run not found") + +// RunMeta is the durable state for one full-sync run. Counts are maintained by +// the Lua transitions below rather than inferred from logs. +type RunMeta struct { + SupplyObservation *supply.Snapshot `json:"supplyObservation,omitempty"` + RunID string `json:"runId"` + Operation string `json:"operation"` + QueryMonth string `json:"queryMonth"` + Status string `json:"status"` + ProducerDone bool `json:"producerDone"` + CurrentRound int `json:"currentRound"` + ActiveCount int64 `json:"activeCount"` + InflightCount int64 `json:"inflightCount"` + NextRoundCount int64 `json:"nextRoundCount"` + SuccessCount int64 `json:"successCount"` + FailedCount int64 `json:"failedCount"` + CreatedAt time.Time `json:"createdAt"` + ProducerCompletedAt time.Time `json:"producerCompletedAt"` + CompletedAt time.Time `json:"completedAt"` + SyncType string `json:"syncType,omitempty"` + SourceRunID string `json:"sourceRunId,omitempty"` + ReplayCycle int `json:"replayCycle,omitempty"` + TriggerSource string `json:"triggerSource,omitempty"` + ScannedCount int64 `json:"scannedCount,omitempty"` + EnqueuedCount int64 `json:"enqueuedCount,omitempty"` + UpstreamAttemptCount int64 `json:"upstreamAttemptCount,omitempty"` + BatchCount int64 `json:"batchCount,omitempty"` + MaxRound int `json:"maxRound,omitempty"` + PendingFailureCount int64 `json:"pendingFailureCount,omitempty"` + StartedAt *time.Time `json:"startedAt,omitempty"` + ScanCompletedAt *time.Time `json:"scanCompletedAt,omitempty"` + ArchiveStatus string `json:"archiveStatus,omitempty"` + ArchiveVersion int `json:"archiveVersion,omitempty"` + ArchivePath string `json:"archivePath,omitempty"` + ArchiveChecksum string `json:"archiveChecksum,omitempty"` + ArchivedAt *time.Time `json:"archivedAt,omitempty"` + ReplayDisposition string `json:"replayDisposition,omitempty"` + // Scope fields describe an optional channel whitelist. ScopeChannels == 0 + // means the run covered every eligible channel (a full sync), which is also + // what legacy runs without these fields report. + ScopeChannels int `json:"scopeChannels,omitempty"` + ScopeDigest string `json:"scopeDigest,omitempty"` + ScopeChannelsSample string `json:"scopeChannelsSample,omitempty"` + RoundStats []RoundStats `json:"rounds,omitempty"` +} + +// RoundStats is the bounded aggregate persisted with a run. It deliberately +// contains no task identity or error detail so the Run hash stays small. +type RoundStats struct { + RunID string `json:"runId"` + Round int `json:"round"` + EnqueuedCount int64 `json:"enqueuedCount"` + ActiveCount int64 `json:"activeCount"` + NextRoundCount int64 `json:"nextRoundCount"` + SuccessCount int64 `json:"successCount"` + FailedCount int64 `json:"failedCount"` + UpstreamAttemptCount int64 `json:"upstreamAttemptCount"` + BatchCount int64 `json:"batchCount"` + StartedAt *time.Time `json:"startedAt,omitempty"` + CompletedAt *time.Time `json:"completedAt,omitempty"` +} + +// TaskState tracks attempts independently from the stable ZSet member. +type TaskState struct { + Round int `json:"round"` + Status string `json:"status"` + Attempts int `json:"attempts"` + LastCode int `json:"lastCode,omitempty"` + LastErrCode string `json:"lastErrCode,omitempty"` + LastMessage string `json:"lastMessage,omitempty"` + UpdatedAt time.Time `json:"updatedAt"` +} + +// FailureRecord is persisted for permanent failures and exhausted busy +// retries. It deliberately contains the task identity and response fields so +// an external reviewer can diagnose a run without the original queue member. +type FailureRecord struct { + Version int `json:"version"` + TaskID string `json:"taskId"` + OriginalRunID string `json:"originalRunId"` + LatestRunID string `json:"latestRunId"` + QueryMonth string `json:"queryMonth"` + Operation string `json:"operation"` + CardID int64 `json:"cardId"` + ICCID string `json:"iccid"` + MSISDN string `json:"msisdn"` + IMSI string `json:"imsi"` + ChanKey string `json:"chanKey"` + TempKey string `json:"tempKey"` + // PackageCode is additive; older failure records decode it as empty. It is + // preserved so a replayed daily-usage card can restore the original package + // code instead of writing an empty value into the TDengine child table. + PackageCode string `json:"packageCode,omitempty"` + Carrier string `json:"carrier"` + FailureType string `json:"failureType"` + Round int `json:"round"` + Attempts int `json:"attempts"` + Code int `json:"code"` + ErrCode string `json:"errCode"` + Message string `json:"message"` + ReviewStatus string `json:"reviewStatus"` + ReviewRemark string `json:"reviewRemark"` + ReviewedBy string `json:"reviewedBy"` + ReviewedAt string `json:"reviewedAt"` + RetryCycle int `json:"retryCycle"` + ReplayRunID string `json:"replayRunId"` + FirstFailedAt time.Time `json:"firstFailedAt"` + LastFailedAt time.Time `json:"lastFailedAt"` +} + +// TaskOutcome is supplied by a consumer when atomically finalizing a lease. +// ErrorClass is intentionally generic: the consumer may classify protocol +// responses without making the queue package depend on service packages. +type TaskOutcome struct { + Success bool + Retry bool + Retryable bool + ErrorClass string + FailureType string + Code int + ErrCode string + Message string + BlockChannel bool + // Covered marks a terminal outcome that answered definitively but carries + // no value ("the carrier has no record of this card", "no IMEI reported + // yet"). It is not a failure: the task leaves the queue like any other + // final outcome and counts as completed work, but it must never be written + // to the failure pool or counted in failedCount. The zero value preserves + // the historical behaviour, so a caller that does not classify the case is + // unaffected. + Covered bool + Failure *FailureRecord +} + +// taskStatusCovered is the task-state value persisted for a covered outcome. +// It is distinct from succeeded/failed because the failure-review page keys off +// the persisted state: a card with no carrier record did not fail, it simply +// has no value to show. +const taskStatusCovered = "covered" + +// Run key builders. Every key contains the same Redis hash-tag ({runId}), so +// multi-key Lua transitions remain valid when Redis Cluster is enabled. +func (s *Store) runTag(runID string) (string, error) { + if strings.TrimSpace(runID) == "" { + return "", fmt.Errorf("runId is empty") + } + return s.prefix + "{" + runID + "}", nil +} + +func (s *Store) RunMetaKey(runID string) (string, error) { + tag, err := s.runTag(runID) + if err != nil { + return "", err + } + return tag + ":meta", nil +} +func (s *Store) RunChannelsKey(runID string) (string, error) { + tag, err := s.runTag(runID) + if err != nil { + return "", err + } + return tag + ":channels", nil +} +func (s *Store) RoundActiveKey(runID string, round int, chanKey string) (string, error) { + tag, err := s.runTag(runID) + if err != nil { + return "", err + } + if round <= 0 { + return "", fmt.Errorf("round must be positive") + } + if strings.TrimSpace(chanKey) == "" { + return "", fmt.Errorf("chanKey is empty") + } + return fmt.Sprintf("%s:round:%d:active:%s", tag, round, chanKey), nil +} +func (s *Store) RoundWaitingKey(runID string, round int, chanKey string) (string, error) { + tag, err := s.runTag(runID) + if err != nil { + return "", err + } + if round <= 0 { + return "", fmt.Errorf("round must be positive") + } + if strings.TrimSpace(chanKey) == "" { + return "", fmt.Errorf("chanKey is empty") + } + return fmt.Sprintf("%s:round:%d:waiting:%s", tag, round, chanKey), nil +} +func (s *Store) RoundLeaseKey(runID string, round int, chanKey string) (string, error) { + tag, err := s.runTag(runID) + if err != nil { + return "", err + } + if round <= 0 { + return "", fmt.Errorf("round must be positive") + } + if strings.TrimSpace(chanKey) == "" { + return "", fmt.Errorf("chanKey is empty") + } + return fmt.Sprintf("%s:round:%d:lease:%s", tag, round, chanKey), nil +} +func (s *Store) TaskStateKey(runID string) (string, error) { + tag, err := s.runTag(runID) + if err != nil { + return "", err + } + return tag + ":task-state", nil +} +func (s *Store) FailedKey(runID string) (string, error) { + tag, err := s.runTag(runID) + if err != nil { + return "", err + } + return tag + ":failed", nil +} +func (s *Store) FailedRunsKey() string { return s.prefix + "failed:runs" } +func (s *Store) ActiveRunsKey() string { return s.prefix + "runs:active" } + +// Lower-case aliases keep key construction consistent with the legacy store +// helpers (leaseKey/blockedKey) and are useful to package-local tests. +func (s *Store) runMetaKey(runID string) (string, error) { return s.RunMetaKey(runID) } +func (s *Store) runChannelsKey(runID string) (string, error) { return s.RunChannelsKey(runID) } +func (s *Store) roundActiveKey(runID string, round int, chanKey string) (string, error) { + return s.RoundActiveKey(runID, round, chanKey) +} +func (s *Store) roundWaitingKey(runID string, round int, chanKey string) (string, error) { + return s.RoundWaitingKey(runID, round, chanKey) +} +func (s *Store) roundLeaseKey(runID string, round int, chanKey string) (string, error) { + return s.RoundLeaseKey(runID, round, chanKey) +} +func (s *Store) taskStateKey(runID string) (string, error) { return s.TaskStateKey(runID) } +func (s *Store) failedKey(runID string) (string, error) { return s.FailedKey(runID) } + +// CreateRun initializes metadata and the run channel set. Replaying the same +// run is idempotent: an existing metadata hash is left intact. +func (s *Store) CreateRun(ctx context.Context, meta RunMeta) error { + if strings.TrimSpace(meta.RunID) == "" { + return fmt.Errorf("runId is empty") + } + if meta.CurrentRound <= 0 { + meta.CurrentRound = 1 + } + if meta.Status == "" { + meta.Status = "producing" + } + if meta.CreatedAt.IsZero() { + meta.CreatedAt = time.Now() + } + metaKey, err := s.RunMetaKey(meta.RunID) + if err != nil { + return err + } + args := []any{meta.RunID, meta.Operation, meta.QueryMonth, meta.Status, boolString(meta.ProducerDone), meta.CurrentRound, meta.ActiveCount, meta.InflightCount, meta.NextRoundCount, meta.SuccessCount, meta.FailedCount, meta.CreatedAt.UTC().Format(time.RFC3339Nano)} + created, err := s.redis.Eval(ctx, createRunScript, 1, []string{metaKey}, args) + if err != nil { + return fmt.Errorf("create run: %w", err) + } + if created.Int() == 0 { + return fmt.Errorf("create run %q failed", meta.RunID) + } + if created.Int() == 2 { + // Do not resurrect a completed run in the active index when an old + // producer delivery is replayed, and reject accidental RunID reuse. + stored, getErr := s.GetRunMeta(ctx, meta.RunID) + if getErr != nil { + return getErr + } + if (meta.Operation != "" && stored.Operation != "" && stored.Operation != meta.Operation) || (meta.QueryMonth != "" && stored.QueryMonth != "" && stored.QueryMonth != meta.QueryMonth) { + return fmt.Errorf("run %q metadata does not match existing run", meta.RunID) + } + if stored.Status == "completed" || stored.Status == "completed_with_failures" { + return nil + } + } + // The active-run index is intentionally updated outside the run-key Lua + // script. It is a global key and therefore lives in a different Redis + // Cluster slot from flow:{runId}:* keys. + if _, err = s.redis.Do(ctx, "SADD", s.ActiveRunsKey(), meta.RunID); err != nil { + return fmt.Errorf("index active run: %w", err) + } + if err := s.setRunMetaExtensions(ctx, meta); err != nil { + return err + } + return nil +} + +func (s *Store) setRunMetaExtensions(ctx context.Context, meta RunMeta) error { + args := []any{meta.RunID} + add := func(name, value string) { + if value != "" { + args = append(args, name, value) + } + } + addInt := func(name string, value int64) { + if value != 0 { + args = append(args, name, strconv.FormatInt(value, 10)) + } + } + addIntN := func(name string, value int) { + if value != 0 { + args = append(args, name, strconv.Itoa(value)) + } + } + addTime := func(name string, value *time.Time) { + if value != nil && !value.IsZero() { + add(name, value.UTC().Format(time.RFC3339Nano)) + } + } + add("syncType", meta.SyncType) + add("sourceRunId", meta.SourceRunID) + addIntN("replayCycle", meta.ReplayCycle) + add("triggerSource", meta.TriggerSource) + addInt("scannedCount", meta.ScannedCount) + addInt("enqueuedCount", meta.EnqueuedCount) + addInt("upstreamAttemptCount", meta.UpstreamAttemptCount) + addInt("batchCount", meta.BatchCount) + addIntN("maxRound", meta.MaxRound) + addInt("pendingFailureCount", meta.PendingFailureCount) + addTime("startedAt", meta.StartedAt) + addTime("scanCompletedAt", meta.ScanCompletedAt) + add("archiveStatus", meta.ArchiveStatus) + addIntN("archiveVersion", meta.ArchiveVersion) + add("archivePath", meta.ArchivePath) + add("archiveChecksum", meta.ArchiveChecksum) + addTime("archivedAt", meta.ArchivedAt) + add("replayDisposition", meta.ReplayDisposition) + addIntN("scopeChannels", meta.ScopeChannels) + add("scopeDigest", meta.ScopeDigest) + add("scopeChannelsSample", meta.ScopeChannelsSample) + if len(args) == 1 { + return nil + } + hsetArgs := append([]any{s.mustRunMetaKey(meta.RunID)}, args[1:]...) + if _, err := s.redis.Do(ctx, "HSET", hsetArgs...); err != nil { + return fmt.Errorf("set run metadata extensions: %w", err) + } + return nil +} + +func (s *Store) mustRunMetaKey(runID string) string { key, _ := s.RunMetaKey(runID); return key } + +// EnqueueRunBatch writes tasks to the active ZSet for the run's current round. +// The task ID is used as a hash field to make retries/idempotent producer +// deliveries no-ops. It also initializes task-state to queued. +func (s *Store) EnqueueRunBatch(ctx context.Context, runID, chanKey string, tasks []Task, nextExecuteAt time.Time) (int, error) { + if len(tasks) == 0 { + return 0, nil + } + if strings.TrimSpace(runID) == "" { + return 0, fmt.Errorf("runId is empty") + } + if strings.TrimSpace(chanKey) == "" { + return 0, fmt.Errorf("chanKey is empty") + } + if nextExecuteAt.IsZero() { + nextExecuteAt = time.Now() + } + metaKey, err := s.RunMetaKey(runID) + if err != nil { + return 0, err + } + channelsKey, _ := s.RunChannelsKey(runID) + stateKey, _ := s.TaskStateKey(runID) + activeKey, _ := s.RoundActiveKey(runID, 1, chanKey) // script resolves round from metadata; key is replaced below + // The active key must be for currentRound. Read metadata before writing; the + // producer normally enqueues only during round one, while this also supports + // replay producers that initialize another round. + meta, getErr := s.GetRunMeta(ctx, runID) + if getErr != nil { + return 0, getErr + } + activeKey, _ = s.RoundActiveKey(runID, meta.CurrentRound, chanKey) + args := make([]any, 0, len(tasks)*2+4) + args = append(args, chanKey, nextExecuteAt.UnixMilli(), meta.CurrentRound, time.Now().UTC().Format(time.RFC3339Nano)) + for _, task := range tasks { + if task.TaskID == "" { + return 0, fmt.Errorf("taskId is empty") + } + if task.RunID != runID { + return 0, fmt.Errorf("task runId %q does not match runId %q", task.RunID, runID) + } + if task.ChanKey != chanKey { + return 0, fmt.Errorf("task chanKey %q does not match batch chanKey %q", task.ChanKey, chanKey) + } + task.NextExecuteAt = nextExecuteAt.UnixMilli() + member, marshalErr := json.Marshal(task) + if marshalErr != nil { + return 0, fmt.Errorf("marshal task: %w", marshalErr) + } + args = append(args, task.TaskID, string(member)) + } + result, err := s.redis.Eval(ctx, enqueueRunScript, 4, []string{metaKey, channelsKey, activeKey, stateKey}, args) + if err != nil { + return 0, fmt.Errorf("enqueue run batch: %w", err) + } + return result.Int(), nil +} + +func (s *Store) MarkProducerDone(ctx context.Context, runID string) error { + metaKey, err := s.RunMetaKey(runID) + if err != nil { + return err + } + result, err := s.redis.Eval(ctx, markProducerDoneScript, 1, []string{metaKey}, []any{time.Now().UTC().Format(time.RFC3339Nano)}) + if err != nil { + return fmt.Errorf("mark producer done: %w", err) + } + if result.Int() == 0 { + return fmt.Errorf("run %q not found", runID) + } + return nil +} + +// MarkProducerAbandoned promotes a run whose producer died before +// MarkProducerDone so the normal lifecycle can finish it. +// +// The caller must hold the unique full-sync producer lock and exclude its own +// RunID. A run left in "producing" or "running" with producerDone=0 cannot close: +// completeRunIfIdle and advanceRunRound both require producerDone=1. It would +// therefore stay in the active index forever, its keys would never receive the +// retention TTL, and the consumer would poll it on every round. Setting +// producerDone=1 lets the consumer drain whatever was already enqueued and then +// close the run normally, which applies the retention TTL and releases the +// index. +// +// It refuses to touch replay runs: a replay target is created as "producing" +// too, but it is driven by the replay engine under its own lock, so its +// producerDone is not this producer's to set. +// +// Returns true when the run was actually promoted. +func (s *Store) MarkProducerAbandoned(ctx context.Context, runID string) (bool, error) { + metaKey, err := s.RunMetaKey(runID) + if err != nil { + return false, err + } + result, err := s.redis.Eval(ctx, markProducerAbandonedScript, 1, []string{metaKey}, []any{time.Now().UTC().Format(time.RFC3339Nano)}) + if err != nil { + return false, fmt.Errorf("mark producer abandoned: %w", err) + } + return result.Int() == 1, nil +} + +func (s *Store) GetRunMeta(ctx context.Context, runID string) (RunMeta, error) { + key, err := s.RunMetaKey(runID) + if err != nil { + return RunMeta{}, err + } + result, err := s.redis.Do(ctx, "HGETALL", key) + if err != nil { + return RunMeta{}, fmt.Errorf("get run metadata: %w", err) + } + values := result.MapStrStr() + if len(values) == 0 { + return RunMeta{}, fmt.Errorf("%w: %s", ErrRunNotFound, runID) + } + meta := RunMeta{RunID: runID, Operation: values["operation"], QueryMonth: values["queryMonth"], Status: values["status"], ProducerDone: parseBool(values["producerDone"]), CurrentRound: parseInt(values["currentRound"]), ActiveCount: parseInt64(values["activeCount"]), InflightCount: parseInt64(values["inflightCount"]), NextRoundCount: parseInt64(values["nextRoundCount"]), SuccessCount: parseInt64(values["successCount"]), FailedCount: parseInt64(values["failedCount"])} + meta.CreatedAt = parseTime(values["createdAt"]) + meta.ProducerCompletedAt = parseTime(values["producerCompletedAt"]) + meta.CompletedAt = parseTime(values["completedAt"]) + meta.SyncType = values["syncType"] + meta.SourceRunID = values["sourceRunId"] + meta.ReplayCycle = parseInt(values["replayCycle"]) + meta.TriggerSource = values["triggerSource"] + meta.ScannedCount = parseInt64(values["scannedCount"]) + meta.EnqueuedCount = parseInt64(values["enqueuedCount"]) + meta.UpstreamAttemptCount = parseInt64(values["upstreamAttemptCount"]) + meta.BatchCount = parseInt64(values["batchCount"]) + meta.MaxRound = parseInt(values["maxRound"]) + meta.PendingFailureCount = parseInt64(values["pendingFailureCount"]) + meta.StartedAt = parseOptionalTime(values["startedAt"]) + meta.ScanCompletedAt = parseOptionalTime(values["scanCompletedAt"]) + meta.ArchiveStatus = values["archiveStatus"] + meta.ArchiveVersion = parseInt(values["archiveVersion"]) + meta.ArchivePath = values["archivePath"] + meta.ArchiveChecksum = values["archiveChecksum"] + meta.ArchivedAt = parseOptionalTime(values["archivedAt"]) + meta.ReplayDisposition = values["replayDisposition"] + if raw := values["supplyObservation"]; raw != "" { + var observation supply.Snapshot + if err := json.Unmarshal([]byte(raw), &observation); err != nil || !observation.Valid() { + observation = supply.Snapshot{SchemaVersion: 0, ResolutionMS: 1000, Partial: true} + } + meta.SupplyObservation = &observation + } + meta.ScopeChannels = parseInt(values["scopeChannels"]) + meta.ScopeDigest = values["scopeDigest"] + meta.ScopeChannelsSample = values["scopeChannelsSample"] + if raw := values["roundStats"]; raw != "" { + if err := json.Unmarshal([]byte(raw), &meta.RoundStats); err != nil { + return RunMeta{}, fmt.Errorf("decode run round stats: %w", err) + } + } + return meta, nil +} + +// SaveRoundStats replaces the bounded round summary JSON stored alongside the +// run metadata. The single run consumer is the writer, so one HSET is enough +// and avoids introducing another Redis key or coordination protocol. +func (s *Store) SaveRoundStats(ctx context.Context, runID string, stats []RoundStats) error { + key, err := s.RunMetaKey(runID) + if err != nil { + return err + } + payload, err := json.Marshal(stats) + if err != nil { + return fmt.Errorf("encode run round stats: %w", err) + } + if _, err := s.redis.Do(ctx, "HSET", key, "roundStats", string(payload)); err != nil { + return fmt.Errorf("save run round stats: %w", err) + } + return nil +} + +// GetRoundStats returns the persisted summaries in execution order. +func (s *Store) GetRoundStats(ctx context.Context, runID string) ([]RoundStats, error) { + meta, err := s.GetRunMeta(ctx, runID) + if err != nil { + return nil, err + } + sort.Slice(meta.RoundStats, func(i, j int) bool { return meta.RoundStats[i].Round < meta.RoundStats[j].Round }) + return meta.RoundStats, nil +} + +func (s *Store) ActiveRuns(ctx context.Context) ([]string, error) { + result, err := s.redis.Do(ctx, "SMEMBERS", s.ActiveRunsKey()) + if err != nil { + return nil, fmt.Errorf("list active runs: %w", err) + } + return result.Strings(), nil +} + +func (s *Store) RunChannels(ctx context.Context, runID string) ([]string, error) { + key, err := s.RunChannelsKey(runID) + if err != nil { + return nil, err + } + result, err := s.redis.Do(ctx, "SMEMBERS", key) + if err != nil { + return nil, fmt.Errorf("list run channels: %w", err) + } + return result.Strings(), nil +} + +// SizeKey returns the cardinality of an explicitly scoped queue key. It is +// used by the Run-aware producer for the same high-water backpressure policy +// as the legacy channel queues. +func (s *Store) SizeKey(ctx context.Context, key string) (int64, error) { + if strings.TrimSpace(key) == "" { + return 0, fmt.Errorf("queue key is empty") + } + result, err := s.redis.Do(ctx, "ZCARD", key) + if err != nil { + return 0, fmt.Errorf("get task count: %w", err) + } + return result.Int64(), nil +} + +// ClaimRun claims due members from the current round for one channel. +func (s *Store) ClaimRun(ctx context.Context, runID, chanKey string, now, leaseUntil time.Time, limit int) ([]ClaimedTask, error) { + if limit <= 0 { + return nil, nil + } + if now.IsZero() { + now = time.Now() + } + if leaseUntil.IsZero() || !leaseUntil.After(now) { + return nil, fmt.Errorf("leaseUntil must be after now") + } + meta, err := s.GetRunMeta(ctx, runID) + if err != nil { + return nil, err + } + activeKey, _ := s.RoundActiveKey(runID, meta.CurrentRound, chanKey) + leaseKey, _ := s.RoundLeaseKey(runID, meta.CurrentRound, chanKey) + metaKey, _ := s.RunMetaKey(runID) + stateKey, _ := s.TaskStateKey(runID) + result, err := s.redis.Eval(ctx, claimRunScript, 4, []string{activeKey, leaseKey, metaKey, stateKey}, []any{now.UnixMilli(), leaseUntil.UnixMilli(), limit, s.owner, meta.CurrentRound, now.UTC().Format(time.RFC3339Nano)}) + if err != nil { + return nil, fmt.Errorf("claim run tasks: %w", err) + } + values := result.Vars() + if len(values)%2 != 0 { + return nil, fmt.Errorf("claim returned odd number of values: %d", len(values)) + } + tasks := make([]ClaimedTask, 0, len(values)/2) + for i := 0; i < len(values); i += 2 { + member := values[i].String() + var task Task + if err := json.Unmarshal([]byte(member), &task); err != nil { + return nil, fmt.Errorf("decode claimed run task: %w", err) + } + tasks = append(tasks, ClaimedTask{Task: task, Member: member, LeaseToken: values[i+1].String(), LeaseUntil: leaseUntil.UnixMilli(), Round: meta.CurrentRound}) + } + return tasks, nil +} + +// RenewRunBatch extends leases for a Run-scoped claim. It mirrors the legacy +// RenewBatch ownership protocol while keeping the round and run hash-tagged +// keys together for Redis Cluster. +func (s *Store) RenewRunBatch(ctx context.Context, tasks []ClaimedTask, leaseUntil time.Time) (int, error) { + if len(tasks) == 0 { + return 0, nil + } + if leaseUntil.IsZero() { + return 0, fmt.Errorf("leaseUntil is zero") + } + first := tasks[0] + if first.Round <= 0 { + return 0, fmt.Errorf("run task round is missing") + } + activeKey, err := s.RoundActiveKey(first.Task.RunID, first.Round, first.Task.ChanKey) + if err != nil { + return 0, err + } + leaseKey, err := s.RoundLeaseKey(first.Task.RunID, first.Round, first.Task.ChanKey) + if err != nil { + return 0, err + } + args := make([]any, 0, len(tasks)*2+1) + args = append(args, leaseUntil.UnixMilli()) + for _, task := range tasks { + if task.Round != first.Round || task.Task.RunID != first.Task.RunID || task.Task.ChanKey != first.Task.ChanKey { + return 0, fmt.Errorf("renew run batch contains multiple runs, rounds, or channels") + } + args = append(args, task.Member, task.LeaseToken) + } + result, err := s.redis.Eval(ctx, renewBatchScript, 2, []string{activeKey, leaseKey}, args) + if err != nil { + return 0, fmt.Errorf("renew run task batch lease: %w", err) + } + return result.Int(), nil +} + +// FinalizeRunTask atomically removes a lease and updates success/failure state. +// A non-success outcome with explicit Retry set is deferred to the next +// round; Retryable alone is reserved for lease recovery. Permanent outcomes +// are written to the final failure hash. +func (s *Store) FinalizeRunTask(ctx context.Context, claimed ClaimedTask, outcome TaskOutcome, maxRounds ...int) (bool, error) { + count, err := s.FinalizeRunTasks(ctx, []ClaimedTask{claimed}, []TaskOutcome{outcome}, maxRounds...) + return count == 1, err +} + +// FinalizeRunTasks applies a whole claimed batch in one ownership-checked Lua +// transition. Every task must belong to the same run/round/channel. +func (s *Store) FinalizeRunTasks(ctx context.Context, claimed []ClaimedTask, outcomes []TaskOutcome, maxRounds ...int) (int, error) { + if len(claimed) == 0 { + return 0, nil + } + if len(claimed) != len(outcomes) { + return 0, fmt.Errorf("claimed and outcome lengths differ") + } + first := claimed[0] + if first.Task.RunID == "" { + return 0, fmt.Errorf("runId is empty") + } + runID := first.Task.RunID + round := first.Round + if round <= 0 { + round = 1 + } + activeKey, _ := s.RoundActiveKey(runID, round, first.Task.ChanKey) + leaseKey, _ := s.RoundLeaseKey(runID, round, first.Task.ChanKey) + metaKey, _ := s.RunMetaKey(runID) + stateKey, _ := s.TaskStateKey(runID) + failureKey, _ := s.FailedKey(runID) + waitingKey, _ := s.RoundWaitingKey(runID, round+1, first.Task.ChanKey) + keys := []string{activeKey, leaseKey, metaKey, stateKey, waitingKey, failureKey} + now := time.Now().UTC().Format(time.RFC3339Nano) + args := make([]any, 0, len(claimed)*9+2) + args = append(args, round, now) + maxRound := 5 + if len(maxRounds) > 0 && maxRounds[0] > 0 { + maxRound = maxRounds[0] + } + finalFailures := false + for i, task := range claimed { + if task.Round > 0 && task.Round != first.Round || task.Task.RunID != runID || task.Task.ChanKey != first.Task.ChanKey { + return 0, fmt.Errorf("finalize run batch contains multiple runs, rounds, or channels") + } + outcome := outcomes[i] + record := outcome.Failure + if record == nil { + record = failureFromOutcome(task.Task, outcome, round) + } + status := "succeeded" + switch { + case outcome.Covered: + status = taskStatusCovered + case !outcome.Success: + status = "failed" + } + // Retryable describes lease recovery for infrastructure failures; only + // the explicit Retry flag is allowed to cross the round barrier. + deferTask := !outcome.Success && outcome.Retry && !outcome.BlockChannel && round < maxRound + if !outcome.Success && !deferTask && outcome.ErrorClass == "mobile_busy" { + record.FailureType = "mobile_busy_exhausted" + record.Round = round + } + recordJSON, marshalErr := json.Marshal(record) + if marshalErr != nil { + return 0, fmt.Errorf("marshal failure record: %w", marshalErr) + } + if !deferTask && !outcome.Success && !outcome.Covered { + finalFailures = true + } + args = append(args, task.Member, task.LeaseToken, task.Task.TaskID, status, outcome.Code, outcome.ErrCode, outcome.Message, string(recordJSON), boolString(deferTask)) + } + result, err := s.redis.Eval(ctx, finalizeRunBatchScript, int64(len(keys)), keys, args) + if err != nil { + return 0, fmt.Errorf("finalize run task batch: %w", err) + } + count := result.Int() + if count == len(claimed) && finalFailures { + if _, err = s.redis.Do(ctx, "ZADD", s.FailedRunsKey(), time.Now().UnixMilli(), runID); err != nil { + return 0, fmt.Errorf("index failed run: %w", err) + } + } + return count, nil +} + +func (s *Store) finalizeRunTaskLegacy(ctx context.Context, claimed ClaimedTask, outcome TaskOutcome) (bool, error) { + if claimed.Task.RunID == "" { + return false, fmt.Errorf("runId is empty") + } + runID := claimed.Task.RunID + round := claimed.Round + if round <= 0 { + round = 1 + } + activeKey, _ := s.RoundActiveKey(runID, round, claimed.Task.ChanKey) + leaseKey, _ := s.RoundLeaseKey(runID, round, claimed.Task.ChanKey) + metaKey, _ := s.RunMetaKey(runID) + stateKey, _ := s.TaskStateKey(runID) + failureKey, _ := s.FailedKey(runID) + waitingKey, _ := s.RoundWaitingKey(runID, round+1, claimed.Task.ChanKey) + record := outcome.Failure + if record == nil { + record = failureFromOutcome(claimed.Task, outcome, round) + } + status := "succeeded" + switch { + case outcome.Covered: + status = taskStatusCovered + case !outcome.Success: + status = "failed" + } + deferTask := !outcome.Success && outcome.Retry && !outcome.BlockChannel && round < 5 + if !outcome.Success && !deferTask && outcome.ErrorClass == "mobile_busy" { + record.FailureType = "mobile_busy_exhausted" + record.Round = round + } + recordJSON, _ := json.Marshal(record) + args := []any{claimed.Member, claimed.LeaseToken, claimed.Task.TaskID, status, round, outcome.Code, outcome.ErrCode, outcome.Message, string(recordJSON), boolString(deferTask), time.Now().UTC().Format(time.RFC3339Nano)} + keys := []string{activeKey, leaseKey, metaKey, stateKey, waitingKey, failureKey} + result, err := s.redis.Eval(ctx, finalizeRunScript, int64(len(keys)), keys, args) + if err != nil { + return false, fmt.Errorf("finalize run task: %w", err) + } + if result.Int() != 1 { + return false, nil + } + if !deferTask && !outcome.Success && !outcome.Covered { + // The failed-run index is global and cannot be part of the run-key Lua + // transaction on Redis Cluster. The per-task failed hash and counters + // above are the atomic state; this index is only a discovery aid. + if _, err = s.redis.Do(ctx, "ZADD", s.FailedRunsKey(), time.Now().UnixMilli(), runID); err != nil { + return false, fmt.Errorf("index failed run: %w", err) + } + } + return true, nil +} + +func (s *Store) DeferRunTask(ctx context.Context, claimed ClaimedTask, failure FailureRecord) (bool, error) { + return s.FinalizeRunTask(ctx, claimed, TaskOutcome{Retry: true, Retryable: true, Failure: &failure, Code: failure.Code, ErrCode: failure.ErrCode, Message: failure.Message, FailureType: failure.FailureType}) +} + +func (s *Store) FailRunTask(ctx context.Context, claimed ClaimedTask, failure FailureRecord) (bool, error) { + return s.FinalizeRunTask(ctx, claimed, TaskOutcome{Failure: &failure, Code: failure.Code, ErrCode: failure.ErrCode, Message: failure.Message, FailureType: failure.FailureType}) +} + +// AdvanceRunRound atomically promotes waiting members only when the producer +// has completed and both active and inflight counts are zero. +func (s *Store) AdvanceRunRound(ctx context.Context, runID string, maxRounds ...int) (bool, error) { + meta, err := s.GetRunMeta(ctx, runID) + if err != nil { + return false, err + } + if meta.CurrentRound <= 0 { + meta.CurrentRound = 1 + } + channels, err := s.RunChannels(ctx, runID) + if err != nil { + return false, err + } + metaKey, _ := s.RunMetaKey(runID) + stateKey, _ := s.TaskStateKey(runID) + keys := []string{metaKey, stateKey} + for _, ch := range channels { + w, _ := s.RoundWaitingKey(runID, meta.CurrentRound+1, ch) + a, _ := s.RoundActiveKey(runID, meta.CurrentRound+1, ch) + keys = append(keys, w, a) + } + maxRound := 5 + if len(maxRounds) > 0 && maxRounds[0] > 0 { + maxRound = maxRounds[0] + } + args := []any{meta.CurrentRound, maxRound, time.Now().UTC().Format(time.RFC3339Nano)} + result, err := s.redis.Eval(ctx, advanceRoundScript, int64(len(keys)), keys, args) + if err != nil { + return false, fmt.Errorf("advance run round: %w", err) + } + return result.Int() == 1, nil +} + +// CompleteRunIfIdle closes a run when no active/inflight/waiting work remains. +func (s *Store) CompleteRunIfIdle(ctx context.Context, runID string, retention ...time.Duration) (bool, error) { + meta, err := s.GetRunMeta(ctx, runID) + if err != nil { + return false, err + } + channels, err := s.RunChannels(ctx, runID) + if err != nil { + return false, err + } + metaKey, _ := s.RunMetaKey(runID) + failedKey, _ := s.FailedKey(runID) + stateKey, _ := s.TaskStateKey(runID) + channelsKey, _ := s.RunChannelsKey(runID) + keys := []string{metaKey, failedKey, stateKey, channelsKey} + for _, ch := range channels { + a, _ := s.RoundActiveKey(runID, meta.CurrentRound, ch) + l, _ := s.RoundLeaseKey(runID, meta.CurrentRound, ch) + w, _ := s.RoundWaitingKey(runID, meta.CurrentRound+1, ch) + keys = append(keys, a, l, w) + } + keep := 30 * 24 * time.Hour + if len(retention) > 0 && retention[0] > 0 { + keep = retention[0] + } + result, err := s.redis.Eval(ctx, completeRunScript, int64(len(keys)), keys, []any{time.Now().UTC().Format(time.RFC3339Nano), int64(keep.Milliseconds())}) + if err != nil { + return false, fmt.Errorf("complete run: %w", err) + } + if result.Int() != 1 { + return false, nil + } + // Keep global indexes out of the run-scoped Lua transaction (see + // CreateRun). The metadata transition above is the authoritative CAS. + if _, err = s.redis.Do(ctx, "SREM", s.ActiveRunsKey(), runID); err != nil { + return false, fmt.Errorf("remove completed run index: %w", err) + } + if meta.FailedCount > 0 { + if _, err = s.redis.Do(ctx, "ZADD", s.FailedRunsKey(), time.Now().UnixMilli(), runID); err != nil { + return false, fmt.Errorf("index failed run: %w", err) + } + } + if s.archiveEnabled { + if _, err = s.redis.Do(ctx, "ZADD", s.prefix+"archive:pending", time.Now().UnixMilli(), runID); err != nil { + return false, fmt.Errorf("index archive pending run: %w", err) + } + } + return true, nil +} + +// BlockRunScoped is the run-queue counterpart of the legacy BlockRun API. It +// removes only this run's active, leased and waiting members on a channel and +// marks the channel blocked in run metadata. BlockRunRun is kept as a readable +// alias for callers that prefer runID-first argument ordering. +func (s *Store) BlockRunScoped(ctx context.Context, runID, chanKey string) (int64, error) { + meta, err := s.GetRunMeta(ctx, runID) + if err != nil { + return 0, err + } + active, _ := s.RoundActiveKey(runID, meta.CurrentRound, chanKey) + lease, _ := s.RoundLeaseKey(runID, meta.CurrentRound, chanKey) + waiting, _ := s.RoundWaitingKey(runID, meta.CurrentRound+1, chanKey) + metaKey, _ := s.RunMetaKey(runID) + stateKey, _ := s.TaskStateKey(runID) + result, err := s.redis.Eval(ctx, blockRunScopedScript, 5, []string{active, lease, waiting, metaKey, stateKey}, []any{runID, chanKey, time.Now().UTC().Format(time.RFC3339Nano)}) + if err != nil { + return 0, fmt.Errorf("block run channel: %w", err) + } + return result.Int64(), nil +} + +func (s *Store) BlockRunRun(ctx context.Context, runID, chanKey string) (int64, error) { + return s.BlockRunScoped(ctx, runID, chanKey) +} + +// BlockRunScopedAsFailed preserves a terminal failure for every unfinished +// status task on a blocked channel. A negative count means the triggering +// lease was lost; an obsolete consumer must not block another owner's work. +func (s *Store) BlockRunScopedAsFailed(ctx context.Context, trigger ClaimedTask, outcome TaskOutcome) (int64, error) { + if trigger.Round <= 0 || trigger.Task.RunID == "" || trigger.Task.ChanKey == "" || trigger.LeaseToken == "" { + return 0, gerror.New("channel block requires a run-scoped triggering lease") + } + var ( + runID = trigger.Task.RunID + chanKey = trigger.Task.ChanKey + round = trigger.Round + ) + active, _ := s.RoundActiveKey(runID, round, chanKey) + lease, _ := s.RoundLeaseKey(runID, round, chanKey) + waiting, _ := s.RoundWaitingKey(runID, round+1, chanKey) + metaKey, _ := s.RunMetaKey(runID) + stateKey, _ := s.TaskStateKey(runID) + failureKey, _ := s.FailedKey(runID) + outcome.FailureType = "channel_blocked" + record, err := json.Marshal(failureFromOutcome(trigger.Task, outcome, round)) + if err != nil { + return 0, gerror.Wrap(err, "marshal channel block failure") + } + result, err := s.redis.Eval(ctx, blockRunScopedScript, 6, + []string{active, lease, waiting, metaKey, stateKey, failureKey}, + []any{runID, chanKey, time.Now().UTC().Format(time.RFC3339Nano), "1", trigger.Member, trigger.LeaseToken, string(record)}) + if err != nil { + return 0, gerror.Wrap(err, "fail blocked run channel") + } + if result.Int64() > 0 { + if _, err = s.redis.Do(ctx, "ZADD", s.FailedRunsKey(), time.Now().UnixMilli(), runID); err != nil { + return 0, gerror.Wrap(err, "index blocked run failures") + } + } + return result.Int64(), nil +} + +func boolString(v bool) string { + if v { + return "1" + } + return "0" +} +func parseBool(v string) bool { return v == "1" || strings.EqualFold(v, "true") } +func parseInt(v string) int { n, _ := strconv.Atoi(v); return n } +func parseInt64(v string) int64 { n, _ := strconv.ParseInt(v, 10, 64); return n } +func parseTime(v string) time.Time { t, _ := time.Parse(time.RFC3339Nano, v); return t } +func parseOptionalTime(v string) *time.Time { + if v == "" { + return nil + } + t, err := time.Parse(time.RFC3339Nano, v) + if err != nil { + return nil + } + return &t +} + +func failureFromOutcome(task Task, o TaskOutcome, round int) *FailureRecord { + now := time.Now() + typ := o.FailureType + if typ == "" { + typ = o.ErrorClass + } + carrier := "generic" + if model.IsTelecomTemplate(task.TempKey) { + carrier = "telecom" + } else { + switch strings.ToLower(strings.TrimSpace(task.TempKey)) { + // The list must stay in sync with carrierMobile's template set in + // internal/consumer/zset/consumer.go; a missing entry silently labels the + // failure generic, which breaks carrier-based failure review and replay. + case "sim_yd_v5", "sim_yd_mock_v5", "sim_onelink_tengyu_v1", "sim_onelink_xuzhou_mobile_v1", "sim_onelink_lelian_v1", "sim_onelink_beixin_v1", "sim_onelink_qianxun_v1", "sim_onelink_henan_xinjing_v1", "sim_onelink_daihe_v1", "lanshang_8000", "lanshang_8003": + carrier = "mobile" + } + } + return &FailureRecord{Version: 1, TaskID: task.TaskID, OriginalRunID: task.RunID, LatestRunID: task.RunID, QueryMonth: task.QueryMonth, Operation: task.Operation, CardID: task.CardID, ICCID: task.ICCID, MSISDN: task.MSISDN, IMSI: task.IMSI, ChanKey: task.ChanKey, TempKey: task.TempKey, PackageCode: task.PackageCode, Carrier: carrier, FailureType: typ, Round: round, Attempts: round, Code: o.Code, ErrCode: o.ErrCode, Message: o.Message, ReviewStatus: "pending", FirstFailedAt: now, LastFailedAt: now} +} + +const createRunScript = ` +if redis.call('EXISTS', KEYS[1]) == 0 then + redis.call('HSET', KEYS[1], 'runId', ARGV[1], 'operation', ARGV[2], 'queryMonth', ARGV[3], 'status', ARGV[4], 'producerDone', ARGV[5], 'currentRound', ARGV[6], 'activeCount', ARGV[7], 'inflightCount', ARGV[8], 'nextRoundCount', ARGV[9], 'successCount', ARGV[10], 'failedCount', ARGV[11], 'createdAt', ARGV[12]) + return 1 +end +return 2 +` +const enqueueRunScript = ` +if redis.call('EXISTS', KEYS[1]) == 0 then return 0 end +if redis.call('HGET', KEYS[1], 'status') == 'completed' or redis.call('HGET', KEYS[1], 'status') == 'completed_with_failures' then return 0 end +if redis.call('HGET', KEYS[1], 'producerDone') == '1' then return 0 end +if redis.call('HGET', KEYS[1], 'blocked:' .. ARGV[1]) == '1' then return 0 end +redis.call('SADD', KEYS[2], ARGV[1]) +local added = 0 +for i=5,#ARGV,2 do + local taskID=ARGV[i]; local member=ARGV[i+1] + if redis.call('HEXISTS', KEYS[4], taskID) == 0 then + redis.call('ZADD', KEYS[3], ARGV[2], member) + redis.call('HSET', KEYS[4], taskID, cjson.encode({round=tonumber(ARGV[3]),status='queued',attempts=0,updatedAt=ARGV[4]})) + redis.call('HINCRBY', KEYS[1], 'activeCount', 1) + added=added+1 + end +end +return added +` +const markProducerDoneScript = ` +if redis.call('EXISTS', KEYS[1]) == 0 then return 0 end +local status=redis.call('HGET',KEYS[1],'status'); if status=='completed' or status=='completed_with_failures' then return 2 end +redis.call('HSET', KEYS[1], 'producerDone', '1', 'producerCompletedAt', ARGV[1]) +if redis.call('HGET', KEYS[1], 'status') == 'producing' then redis.call('HSET', KEYS[1], 'status', 'running') end +return 1 +` + +// markProducerAbandonedScript promotes only a run whose producer provably never +// finished: "producing" or channel-blocked "running", producerDone not set, +// and not a replay target. Callers establish ownership of the producer lock. +const markProducerAbandonedScript = ` +if redis.call('EXISTS', KEYS[1]) == 0 then return 0 end +local status = redis.call('HGET', KEYS[1], 'status') +if status ~= 'producing' and status ~= 'running' then return 0 end +if redis.call('HGET', KEYS[1], 'producerDone') == '1' then return 0 end +local syncType = redis.call('HGET', KEYS[1], 'syncType') +if syncType == 'replay' then return 0 end +redis.call('HSET', KEYS[1], 'producerDone', '1', 'producerCompletedAt', ARGV[1], 'abandonedAt', ARGV[1], 'status', 'running') +return 1 +` +const claimRunScript = ` +local status=redis.call('HGET',KEYS[3],'status') +if status=='completed' or status=='completed_with_failures' then return {} end +local members=redis.call('ZRANGEBYSCORE',KEYS[1],'-inf',ARGV[1],'LIMIT',0,ARGV[3]); local result={}; local serverTime=redis.call('TIME') +for i,member in ipairs(members) do + local previous=redis.call('HGET',KEYS[2],member) + if previous then redis.call('HINCRBY',KEYS[3],'inflightCount',-1) end + local token=ARGV[4]..':'..serverTime[1]..':'..serverTime[2]..':'..i + redis.call('ZADD',KEYS[1],ARGV[2],member); redis.call('HSET',KEYS[2],member,token) + local decoded=cjson.decode(member); local taskID=decoded.taskId + local old=redis.call('HGET',KEYS[4],taskID); local attempts=0 + if old then local ok,state=pcall(cjson.decode,old); if ok and state.attempts then attempts=state.attempts end end + redis.call('HSET',KEYS[4],taskID,cjson.encode({round=tonumber(ARGV[5]),status='claimed',attempts=attempts+1,updatedAt=ARGV[6]})) + table.insert(result,member); table.insert(result,token) +end +if #members>0 then redis.call('HINCRBY',KEYS[3],'inflightCount',#members) end +return result +` +const finalizeRunScript = ` +if redis.call('HGET',KEYS[2],ARGV[1]) ~= ARGV[2] then return 0 end +redis.call('ZREM',KEYS[1],ARGV[1]); redis.call('HDEL',KEYS[2],ARGV[1]); redis.call('HINCRBY',KEYS[3],'activeCount',-1); redis.call('HINCRBY',KEYS[3],'inflightCount',-1) +local attempts=tonumber(ARGV[5]) +local previous=redis.call('HGET',KEYS[4],ARGV[3]) +if previous then local ok,state=pcall(cjson.decode,previous); if ok and state.attempts then attempts=tonumber(state.attempts) or attempts end end +local taskStatus=ARGV[4] +if ARGV[10]=='1' then taskStatus='waiting_next_round' end +if taskStatus=='failed' then taskStatus='failed_pending_review' end +redis.call('HSET',KEYS[4],ARGV[3], cjson.encode({round=tonumber(ARGV[5]),status=taskStatus,attempts=attempts,lastCode=tonumber(ARGV[6]),lastErrCode=ARGV[7],lastMessage=ARGV[8],updatedAt=ARGV[11]})) +if ARGV[10]=='1' then redis.call('ZADD',KEYS[5],0,ARGV[1]); redis.call('HINCRBY',KEYS[3],'nextRoundCount',1) else if ARGV[4]=='succeeded' or ARGV[4]=='covered' then redis.call('HINCRBY',KEYS[3],'successCount',1) else local failureRecord=ARGV[9]; local ok,failure=pcall(cjson.decode,failureRecord); if ok and failure then failure.round=tonumber(ARGV[5]); failure.attempts=attempts; failureRecord=cjson.encode(failure) end; redis.call('HSET',KEYS[6],ARGV[3],failureRecord); redis.call('HINCRBY',KEYS[3],'failedCount',1) end end +return 1 +` +const finalizeRunBatchScript = ` +local count=(#ARGV-2)/9 +for i=3,#ARGV,9 do + if redis.call('HGET',KEYS[2],ARGV[i]) ~= ARGV[i+1] then return 0 end +end +for i=3,#ARGV,9 do + local member=ARGV[i]; local taskID=ARGV[i+2]; local status=ARGV[i+3]; local code=ARGV[i+4]; local errCode=ARGV[i+5]; local message=ARGV[i+6]; local record=ARGV[i+7]; local deferTask=ARGV[i+8] + redis.call('ZREM',KEYS[1],member); redis.call('HDEL',KEYS[2],member) + redis.call('HINCRBY',KEYS[3],'activeCount',-1); redis.call('HINCRBY',KEYS[3],'inflightCount',-1) + local attempts=tonumber(ARGV[1]); local previous=redis.call('HGET',KEYS[4],taskID) + if previous then local ok,state=pcall(cjson.decode,previous); if ok and state.attempts then attempts=tonumber(state.attempts) or attempts end end + local taskStatus=status; if deferTask=='1' then taskStatus='waiting_next_round' elseif taskStatus=='failed' then taskStatus='failed_pending_review' end + redis.call('HSET',KEYS[4],taskID,cjson.encode({round=tonumber(ARGV[1]),status=taskStatus,attempts=attempts,lastCode=tonumber(code),lastErrCode=errCode,lastMessage=message,updatedAt=ARGV[2]})) + if deferTask=='1' then redis.call('ZADD',KEYS[5],0,member); redis.call('HINCRBY',KEYS[3],'nextRoundCount',1) + elseif status=='succeeded' or status=='covered' then redis.call('HINCRBY',KEYS[3],'successCount',1) + else local ok,failure=pcall(cjson.decode,record); if ok and failure then failure.round=tonumber(ARGV[1]); failure.attempts=attempts; record=cjson.encode(failure) end; redis.call('HSET',KEYS[6],taskID,record); redis.call('HINCRBY',KEYS[3],'failedCount',1) end +end +return count +` +const advanceRoundScript = ` +if redis.call('HGET',KEYS[1],'producerDone') ~= '1' then return 0 end +local status=redis.call('HGET',KEYS[1],'status'); if status=='completed' or status=='completed_with_failures' then return 0 end +if tonumber(redis.call('HGET',KEYS[1],'currentRound') or '0') ~= tonumber(ARGV[1]) then return 0 end +if tonumber(ARGV[1]) >= tonumber(ARGV[2]) then return 0 end +if tonumber(redis.call('HGET',KEYS[1],'activeCount') or '0') ~= 0 or tonumber(redis.call('HGET',KEYS[1],'inflightCount') or '0') ~= 0 then return 0 end +local moved=0; local nextRound=tonumber(ARGV[1])+1 +for i=3,#KEYS,2 do local waiting=KEYS[i]; local active=KEYS[i+1]; local members=redis.call('ZRANGE',waiting,0,-1,'WITHSCORES'); for j=1,#members,2 do redis.call('ZADD',active,members[j+1],members[j]); local payload=cjson.decode(members[j]); local attempts=nextRound-1; local previous=redis.call('HGET',KEYS[2],payload.taskId); if previous then local ok,state=pcall(cjson.decode,previous); if ok and state.attempts then attempts=tonumber(state.attempts) or attempts end end; redis.call('HSET',KEYS[2],payload.taskId,cjson.encode({round=nextRound,status='queued',attempts=attempts,updatedAt=ARGV[3]})); moved=moved+1 end; redis.call('DEL',waiting); end +if moved==0 then return 0 end +redis.call('HSET',KEYS[1],'currentRound',nextRound,'activeCount',moved,'inflightCount',0,'nextRoundCount',0) +return 1 +` +const completeRunScript = ` +if redis.call('HGET',KEYS[1],'producerDone') ~= '1' then return 0 end +local existingStatus=redis.call('HGET',KEYS[1],'status'); if existingStatus=='completed' or existingStatus=='completed_with_failures' then return 0 end +if tonumber(redis.call('HGET',KEYS[1],'activeCount') or '0') ~= 0 or tonumber(redis.call('HGET',KEYS[1],'inflightCount') or '0') ~= 0 or tonumber(redis.call('HGET',KEYS[1],'nextRoundCount') or '0') ~= 0 then return 0 end +local failed=tonumber(redis.call('HGET',KEYS[1],'failedCount') or '0'); local status='completed'; if failed>0 then status='completed_with_failures'; end +redis.call('HSET',KEYS[1],'status',status,'completedAt',ARGV[1]); redis.call('PEXPIRE',KEYS[1],ARGV[2]); redis.call('PEXPIRE',KEYS[2],ARGV[2]); for i=3,#KEYS do redis.call('PEXPIRE',KEYS[i],ARGV[2]); end; return 1 +` + +const blockRunScopedScript = ` +local status=redis.call('HGET',KEYS[4],'status') +if status=='completed' or status=='completed_with_failures' then return 0 end +local countFailures=ARGV[4]=='1' +if countFailures and redis.call('HGET',KEYS[2],ARGV[5]) ~= ARGV[6] then return -1 end +local removed = 0 +local leased = 0 +local waiting = 0 +local function markBlocked(member) + local ok,payload=pcall(cjson.decode,member) + if not ok or not payload or not payload.taskId then return end + local round=0; local attempts=0; local previous=redis.call('HGET',KEYS[5],payload.taskId) + if previous then local stateOK,state=pcall(cjson.decode,previous); if stateOK and state then round=tonumber(state.round) or 0; attempts=tonumber(state.attempts) or 0 end end + redis.call('HSET',KEYS[5],payload.taskId,cjson.encode({round=round,status='channel_blocked',attempts=attempts,updatedAt=ARGV[3]})) + if countFailures then + local record=cjson.decode(ARGV[7]) + record.taskId=payload.taskId; record.cardId=payload.cardId; record.iccid=payload.iccid + record.msisdn=payload.msisdn; record.imsi=payload.imsi; record.operation=payload.operation + record.round=round; record.attempts=attempts + redis.call('HSET',KEYS[6],payload.taskId,cjson.encode(record)) + redis.call('HINCRBY',KEYS[4],'failedCount',1) + end +end +local activeMembers = redis.call('ZRANGE', KEYS[1], 0, -1) +for _, member in ipairs(activeMembers) do + if string.find(member, '"runId":"' .. ARGV[1] .. '"', 1, true) then + redis.call('ZREM', KEYS[1], member) + if redis.call('HDEL', KEYS[2], member) == 1 then leased = leased + 1 end + markBlocked(member) + removed = removed + 1 + end +end +local waitingMembers = redis.call('ZRANGE', KEYS[3], 0, -1) +for _, member in ipairs(waitingMembers) do + redis.call('ZREM', KEYS[3], member) + markBlocked(member) + waiting = waiting + 1 +end +redis.call('HINCRBY', KEYS[4], 'activeCount', -removed) +if leased > 0 then redis.call('HINCRBY', KEYS[4], 'inflightCount', -leased) end +if waiting > 0 then redis.call('HINCRBY', KEYS[4], 'nextRoundCount', -waiting) end +redis.call('HSET', KEYS[4], 'blocked:' .. ARGV[2], '1', 'status', 'running') +return removed + waiting +` diff --git a/internal/queue/zset/run_covered_test.go b/internal/queue/zset/run_covered_test.go new file mode 100644 index 0000000..a3743e1 --- /dev/null +++ b/internal/queue/zset/run_covered_test.go @@ -0,0 +1,124 @@ +package zset + +import ( + "context" + "fmt" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +func newCoveredRun(t *testing.T, prefix, run string, count int) (*Store, context.Context) { + t.Helper() + store, err := New(testutil.Redis(t), prefix) + if err != nil { + t.Fatal(err) + } + ctx := context.Background() + if err := store.CreateRun(ctx, RunMeta{RunID: run}); err != nil { + t.Fatal(err) + } + tasks := make([]Task, count) + for i := range tasks { + tasks[i] = Task{RunID: run, TaskID: fmt.Sprintf("%s:%d", run, i), CardID: int64(i + 1), ChanKey: "channel", TempKey: "sim_yd_v5", ICCID: fmt.Sprintf("card-%d", i)} + } + if n, err := store.EnqueueRunBatch(ctx, run, "channel", tasks, time.Now()); err != nil || n != count { + t.Fatalf("enqueue %d: %v", n, err) + } + if err := store.MarkProducerDone(ctx, run); err != nil { + t.Fatal(err) + } + return store, ctx +} + +// TestCoveredOutcomeStaysOutOfFailurePool pins the queue-side contract of a +// covered result: the task leaves the round like a success, the run counts it +// as completed work, and nothing is written to the failure hash or failedCount. +// Code 14000 ("carrier has no record of this card") reaches the queue through +// this path, so a regression here restores the 4900-entry failure pool seen in +// production on 2026-10-20. +func TestCoveredOutcomeStaysOutOfFailurePool(t *testing.T) { + store, ctx := newCoveredRun(t, "covered:", "run", 3) + now := time.Now() + claims, err := store.ClaimRun(ctx, "run", "channel", now, now.Add(time.Minute), 3) + if err != nil || len(claims) != 3 { + t.Fatalf("claim: %d %v", len(claims), err) + } + outcomes := []TaskOutcome{ + {Success: true}, + {Covered: true, ErrorClass: "no_record", Code: 14000, Message: "系统无记录，请检查输入参数是否正确"}, + {Covered: true, ErrorClass: "data_invalid", Message: "carrier returned no imei value"}, + } + if n, err := store.FinalizeRunTasks(ctx, claims, outcomes); err != nil || n != 3 { + t.Fatalf("finalize: %d %v", n, err) + } + + failureKey, _ := store.FailedKey("run") + size, err := store.RedisClient().Do(ctx, "HLEN", failureKey) + if err != nil || size.Int64() != 0 { + t.Fatalf("failure pool entries = %d err=%v, want 0", size.Int64(), err) + } + meta, err := store.GetRunMeta(ctx, "run") + if err != nil { + t.Fatal(err) + } + if meta.FailedCount != 0 || meta.SuccessCount != 3 { + t.Fatalf("counters: success=%d failed=%d, want 3/0", meta.SuccessCount, meta.FailedCount) + } + // The round was still advanced: covered tasks must free the capacity they + // claimed, otherwise the run can never complete. + if meta.ActiveCount != 0 || meta.InflightCount != 0 || meta.NextRoundCount != 0 { + t.Fatalf("queue capacity not released: %+v", meta) + } + + stateKey, _ := store.TaskStateKey("run") + state, err := store.RedisClient().Do(ctx, "HGET", stateKey, "run:1") + if err != nil { + t.Fatal(err) + } + values := state.MapStrAny() + if values["status"] != taskStatusCovered || fmt.Sprint(values["lastCode"]) != "14000" { + t.Fatalf("covered task state = %v", values) + } + + completed, err := store.CompleteRunIfIdle(ctx, "run") + if err != nil || !completed { + t.Fatalf("complete: %t %v", completed, err) + } + if meta, err = store.GetRunMeta(ctx, "run"); err != nil || meta.Status != "completed" { + t.Fatalf("status = %q err=%v, want completed (no failures)", meta.Status, err) + } +} + +// TestFailureOutcomeStillCountsAsFailure is the guard rail for the test above: +// only Covered is exempt. An ordinary terminal failure keeps the historical +// failure-pool behaviour. +func TestFailureOutcomeStillCountsAsFailure(t *testing.T) { + store, ctx := newCoveredRun(t, "covered-fail:", "run", 2) + now := time.Now() + claims, err := store.ClaimRun(ctx, "run", "channel", now, now.Add(time.Minute), 2) + if err != nil || len(claims) != 2 { + t.Fatalf("claim: %d %v", len(claims), err) + } + outcomes := []TaskOutcome{ + {Covered: true, ErrorClass: "no_record", Code: 14000}, + {ErrorClass: "business", FailureType: "business", Code: 11011, Message: "invalid card"}, + } + if n, err := store.FinalizeRunTasks(ctx, claims, outcomes); err != nil || n != 2 { + t.Fatalf("finalize: %d %v", n, err) + } + failureKey, _ := store.FailedKey("run") + size, err := store.RedisClient().Do(ctx, "HLEN", failureKey) + if err != nil || size.Int64() != 1 { + t.Fatalf("failure pool entries = %d err=%v, want 1", size.Int64(), err) + } + exists, err := store.RedisClient().Do(ctx, "HEXISTS", failureKey, "run:1") + if err != nil || exists.Int() != 1 { + t.Fatalf("the business failure is missing from the pool: %v %v", exists, err) + } + meta, err := store.GetRunMeta(ctx, "run") + if err != nil || meta.FailedCount != 1 || meta.SuccessCount != 1 { + t.Fatalf("counters: %+v err=%v", meta, err) + } +} diff --git a/internal/queue/zset/run_package_test.go b/internal/queue/zset/run_package_test.go new file mode 100644 index 0000000..37e7b4b --- /dev/null +++ b/internal/queue/zset/run_package_test.go @@ -0,0 +1,21 @@ +package zset + +import "testing" + +// TestFailureFromOutcomeCarriesPackageCode guards the daily-usage replay path: +// the TDengine child table stores the package code and the diagnosis page reads +// it, so a failed card must survive into the failure pool with its package code +// intact, or the replayed write would wipe it to empty. +func TestFailureFromOutcomeCarriesPackageCode(t *testing.T) { + record := failureFromOutcome( + Task{TaskID: "r:op:42", RunID: "r", Operation: "op", QueryMonth: "20261009", ChanKey: "217", TempKey: "sim_yd_v5", CardID: 42, ICCID: "89860042", PackageCode: "PKG-A"}, + TaskOutcome{Retryable: false, ErrorClass: "business", Code: 4242, Message: "boom"}, + 3, + ) + if record.PackageCode != "PKG-A" { + t.Fatalf("failure record lost the package code: %+v", record) + } + if record.QueryMonth != "20261009" { + t.Fatalf("failure record lost the daily period: %+v", record) + } +} diff --git a/internal/queue/zset/store.go b/internal/queue/zset/store.go new file mode 100644 index 0000000..5b49dcf --- /dev/null +++ b/internal/queue/zset/store.go @@ -0,0 +1,424 @@ +package zset + +import ( + "context" + "crypto/rand" + "encoding/hex" + "encoding/json" + "fmt" + "strings" + "time" + + "github.com/gogf/gf/v2/container/gvar" +) + +// RedisClient is the small Redis surface required by Store. Keeping this +// interface here makes the claim/ack protocol testable without a live Redis. +type RedisClient interface { + Do(ctx context.Context, command string, args ...any) (*gvar.Var, error) + Eval(ctx context.Context, script string, numKeys int64, keys []string, args []any) (*gvar.Var, error) +} + +const defaultPrefix = "flow:" + +// Task is the payload stored as a ZSet member. The member itself is retained +// so a task can be recovered after a consumer process exits. +type Task struct { + TaskID string `json:"taskId"` + RunID string `json:"runId"` + Operation string `json:"operation"` + // QueryMonth is the run's period identifier, not necessarily a month. + // Monthly usage sync stores yyyyMM; daily usage sync stores yyyyMMdd. + // The whole queue, run, archive and replay path only ever copies this + // string, so a period change needs no schema change. Do not parse it as a + // month without checking the operation first. + QueryMonth string `json:"queryMonth"` + ChanKey string `json:"chanKey"` + TempKey string `json:"tempKey"` + Operator int `json:"operator,omitempty"` + CardID int64 `json:"cardId"` + ICCID string `json:"iccid"` + MSISDN string `json:"msisdn"` + IMSI string `json:"imsi"` + // PackageCode is the card's current package at production time. Daily + // usage writes it to the TDengine child table where the diagnosis page + // reads it back. It is additive: older members simply decode as empty. + PackageCode string `json:"packageCode,omitempty"` + // NextExecuteAt is the ZSet score and is intentionally excluded from the + // member payload so retrying the same producer run remains idempotent. + NextExecuteAt int64 `json:"-"` +} + +// ClaimedTask is returned after a successful Lua lease claim. +type ClaimedTask struct { + Task Task + Member string + LeaseToken string + LeaseUntil int64 + // Round is populated by ClaimRun. Legacy channel-scoped claims leave it + // zero and continue using the original lease keys. + Round int +} + +// StableTaskID is the logical id used for idempotency across producer retries. +func StableTaskID(runID, operation string, cardID int64) string { + return fmt.Sprintf("%s:%s:%d", runID, operation, cardID) +} + +// Store owns the Redis key and lease protocol for one task namespace. +type Store struct { + redis RedisClient + prefix string + owner string + archiveEnabled bool +} + +// RedisClient exposes the narrow Redis surface for adjacent full-sync +// repositories. Callers must continue to use Store key builders for namespacing. +func (s *Store) RedisClient() RedisClient { return s.redis } + +// Prefix returns the configured Redis namespace prefix. +func (s *Store) Prefix() string { return s.prefix } + +// EnableArchiveIndex makes terminal runs discoverable by the optional JSON +// archive worker. It is opt-in so the legacy queue path remains unchanged. +func (s *Store) EnableArchiveIndex(enabled bool) { s.archiveEnabled = enabled } + +func New(redis RedisClient, prefix string) (*Store, error) { + if redis == nil { + return nil, fmt.Errorf("redis client is nil") + } + if strings.TrimSpace(prefix) == "" { + prefix = defaultPrefix + } + return &Store{redis: redis, prefix: prefix, owner: newOwner()}, nil +} + +func (s *Store) Owner() string { return s.owner } + +func (s *Store) Key(chanKey string) (string, error) { + if strings.TrimSpace(chanKey) == "" { + return "", fmt.Errorf("chanKey is empty") + } + return s.prefix + chanKey, nil +} + +func (s *Store) leaseKey(chanKey string) (string, error) { + if strings.TrimSpace(chanKey) == "" { + return "", fmt.Errorf("chanKey is empty") + } + return s.prefix + "lease:" + chanKey, nil +} + +func (s *Store) blockedKey(chanKey string) (string, error) { + if strings.TrimSpace(chanKey) == "" { + return "", fmt.Errorf("chanKey is empty") + } + return s.prefix + "blocked:" + chanKey, nil +} + +func (s *Store) ChannelsKey() string { return s.prefix + "channels" } + +func (s *Store) RegisterChannel(ctx context.Context, chanKey string) error { + if strings.TrimSpace(chanKey) == "" { + return fmt.Errorf("chanKey is empty") + } + if _, err := s.redis.Do(ctx, "SADD", s.ChannelsKey(), chanKey); err != nil { + return fmt.Errorf("register channel: %w", err) + } + return nil +} + +func (s *Store) ActiveChannels(ctx context.Context) ([]string, error) { + result, err := s.redis.Do(ctx, "SMEMBERS", s.ChannelsKey()) + if err != nil { + return nil, fmt.Errorf("list active channels: %w", err) + } + return result.Strings(), nil +} + +// Enqueue adds or replaces a task at the requested execution time. A +// deterministic TaskID lets a producer retry its write without creating a +// second logical task. +func (s *Store) Enqueue(ctx context.Context, task Task, nextExecuteAt time.Time) error { + _, err := s.EnqueueBatch(ctx, task.ChanKey, []Task{task}, nextExecuteAt) + return err +} + +// EnqueueBatch registers one channel and writes all of its tasks atomically. +// It returns zero when the channel has already been stopped for this run. +func (s *Store) EnqueueBatch(ctx context.Context, chanKey string, tasks []Task, nextExecuteAt time.Time) (int, error) { + if len(tasks) == 0 { + return 0, nil + } + key, err := s.Key(chanKey) + if err != nil { + return 0, err + } + blockedKey, err := s.blockedKey(chanKey) + if err != nil { + return 0, err + } + if nextExecuteAt.IsZero() { + nextExecuteAt = time.Now() + } + runID := tasks[0].RunID + args := make([]any, 0, len(tasks)+3) + args = append(args, chanKey, runID, nextExecuteAt.UnixMilli()) + for index := range tasks { + task := tasks[index] + if task.TaskID == "" { + return 0, fmt.Errorf("taskId is empty") + } + if task.ChanKey != chanKey { + return 0, fmt.Errorf("task chanKey %q does not match batch chanKey %q", task.ChanKey, chanKey) + } + if task.RunID != runID { + return 0, fmt.Errorf("task runId %q does not match batch runId %q", task.RunID, runID) + } + task.NextExecuteAt = nextExecuteAt.UnixMilli() + member, marshalErr := json.Marshal(task) + if marshalErr != nil { + return 0, fmt.Errorf("marshal task: %w", marshalErr) + } + args = append(args, string(member)) + } + result, err := s.redis.Eval(ctx, enqueueBatchScript, 3, []string{key, s.ChannelsKey(), blockedKey}, args) + if err != nil { + return 0, fmt.Errorf("enqueue task batch: %w", err) + } + return result.Int(), nil +} + +// Claim atomically leases due tasks. It intentionally does not remove the +// members: a process crash leaves them recoverable after the lease expires. +func (s *Store) Claim(ctx context.Context, chanKey string, now time.Time, leaseUntil time.Time, limit int) ([]ClaimedTask, error) { + if limit <= 0 { + return nil, nil + } + key, err := s.Key(chanKey) + if err != nil { + return nil, err + } + leaseKey, err := s.leaseKey(chanKey) + if err != nil { + return nil, err + } + if now.IsZero() { + now = time.Now() + } + if leaseUntil.IsZero() || !leaseUntil.After(now) { + return nil, fmt.Errorf("leaseUntil must be after now") + } + result, err := s.redis.Eval(ctx, claimScript, 2, []string{key, leaseKey}, []any{now.UnixMilli(), leaseUntil.UnixMilli(), limit, s.owner}) + if err != nil { + return nil, fmt.Errorf("claim tasks: %w", err) + } + values := result.Vars() + if len(values)%2 != 0 { + return nil, fmt.Errorf("claim returned odd number of values: %d", len(values)) + } + tasks := make([]ClaimedTask, 0, len(values)/2) + for i := 0; i < len(values); i += 2 { + member := values[i].String() + var task Task + if err := json.Unmarshal([]byte(member), &task); err != nil { + return nil, fmt.Errorf("decode claimed task: %w", err) + } + tasks = append(tasks, ClaimedTask{ + Task: task, + Member: member, + LeaseToken: values[i+1].String(), + LeaseUntil: leaseUntil.UnixMilli(), + }) + } + return tasks, nil +} + +// Renew extends a lease only when the caller still owns it. +func (s *Store) Renew(ctx context.Context, task ClaimedTask, leaseUntil time.Time) (bool, error) { + count, err := s.RenewBatch(ctx, []ClaimedTask{task}, leaseUntil) + return count == 1, err +} + +// RenewBatch extends every lease in one Redis round trip. The Lua script +// validates the complete batch before changing any score, so callers never +// continue with only part of a claimed batch. +func (s *Store) RenewBatch(ctx context.Context, tasks []ClaimedTask, leaseUntil time.Time) (int, error) { + if len(tasks) == 0 { + return 0, nil + } + if leaseUntil.IsZero() { + return 0, fmt.Errorf("leaseUntil is zero") + } + chanKey := tasks[0].Task.ChanKey + key, err := s.Key(chanKey) + if err != nil { + return 0, err + } + leaseKey, err := s.leaseKey(chanKey) + if err != nil { + return 0, err + } + args := make([]any, 0, len(tasks)*2+1) + args = append(args, leaseUntil.UnixMilli()) + for _, task := range tasks { + if task.Task.ChanKey != chanKey { + return 0, fmt.Errorf("renew batch contains multiple channels") + } + args = append(args, task.Member, task.LeaseToken) + } + result, err := s.redis.Eval(ctx, renewBatchScript, 2, []string{key, leaseKey}, args) + if err != nil { + return 0, fmt.Errorf("renew task batch lease: %w", err) + } + return result.Int(), nil +} + +// Ack removes a task only when the lease token still matches. A stale worker +// cannot delete a task reclaimed by another worker. +func (s *Store) Ack(ctx context.Context, task ClaimedTask) (bool, error) { + count, err := s.AckBatch(ctx, []ClaimedTask{task}) + return count == 1, err +} + +// AckBatch removes every task whose lease token is still owned by this +// consumer. A partial count means at least one lease expired or was reclaimed. +func (s *Store) AckBatch(ctx context.Context, tasks []ClaimedTask) (int, error) { + if len(tasks) == 0 { + return 0, nil + } + chanKey := tasks[0].Task.ChanKey + key, err := s.Key(chanKey) + if err != nil { + return 0, err + } + leaseKey, err := s.leaseKey(chanKey) + if err != nil { + return 0, err + } + args := make([]any, 0, len(tasks)*2) + for _, task := range tasks { + if task.Task.ChanKey != chanKey { + return 0, fmt.Errorf("ack batch contains multiple channels") + } + args = append(args, task.Member, task.LeaseToken) + } + result, err := s.redis.Eval(ctx, ackBatchScript, 2, []string{key, leaseKey}, args) + if err != nil { + return 0, fmt.Errorf("ack task batch: %w", err) + } + return result.Int(), nil +} + +// BlockRun stops one channel for the current run and removes its queued and +// leased tasks. Producer batches for the same run are rejected atomically. +func (s *Store) BlockRun(ctx context.Context, chanKey, runID string) (int64, error) { + if strings.TrimSpace(runID) == "" { + return 0, fmt.Errorf("runId is empty") + } + key, err := s.Key(chanKey) + if err != nil { + return 0, err + } + leaseKey, err := s.leaseKey(chanKey) + if err != nil { + return 0, err + } + blockedKey, err := s.blockedKey(chanKey) + if err != nil { + return 0, err + } + result, err := s.redis.Eval(ctx, blockRunScript, 3, []string{key, leaseKey, blockedKey}, []any{runID}) + if err != nil { + return 0, fmt.Errorf("block channel run: %w", err) + } + return result.Int64(), nil +} + +func (s *Store) Size(ctx context.Context, chanKey string) (int64, error) { + key, err := s.Key(chanKey) + if err != nil { + return 0, err + } + result, err := s.redis.Do(ctx, "ZCARD", key) + if err != nil { + return 0, fmt.Errorf("get task count: %w", err) + } + return result.Int64(), nil +} + +func newOwner() string { + var bytes [12]byte + if _, err := rand.Read(bytes[:]); err == nil { + return hex.EncodeToString(bytes[:]) + } + return fmt.Sprintf("consumer-%d", time.Now().UnixNano()) +} + +const claimScript = ` +local members = redis.call('ZRANGEBYSCORE', KEYS[1], '-inf', ARGV[1], 'LIMIT', 0, ARGV[3]) +local result = {} +local serverTime = redis.call('TIME') +for index, member in ipairs(members) do + local token = ARGV[4] .. ':' .. serverTime[1] .. ':' .. serverTime[2] .. ':' .. index + redis.call('ZADD', KEYS[1], ARGV[2], member) + redis.call('HSET', KEYS[2], member, token) + table.insert(result, member) + table.insert(result, token) +end +return result +` + +const enqueueBatchScript = ` +if redis.call('GET', KEYS[3]) == ARGV[2] then return 0 end +redis.call('SADD', KEYS[2], ARGV[1]) +for index = 4, #ARGV do + redis.call('ZADD', KEYS[1], ARGV[3], ARGV[index]) +end +return #ARGV - 3 +` + +const renewBatchScript = ` +for index = 2, #ARGV, 2 do + local member = ARGV[index] + local token = ARGV[index + 1] + if redis.call('HGET', KEYS[2], member) ~= token then return 0 end + if redis.call('ZSCORE', KEYS[1], member) == false then return 0 end +end +for index = 2, #ARGV, 2 do + redis.call('ZADD', KEYS[1], ARGV[1], ARGV[index]) +end +return (#ARGV - 1) / 2 +` + +const ackBatchScript = ` +local removed = 0 +for index = 1, #ARGV, 2 do + local member = ARGV[index] + local token = ARGV[index + 1] + if redis.call('HGET', KEYS[2], member) == token then + redis.call('ZREM', KEYS[1], member) + redis.call('HDEL', KEYS[2], member) + removed = removed + 1 + end +end +return removed +` + +const blockRunScript = ` +redis.call('SET', KEYS[3], ARGV[1]) +local pending = 0 +local members = redis.call('ZRANGE', KEYS[1], 0, -1) +for _, member in ipairs(members) do + -- Task members are JSON produced by this package. Match the complete + -- runId field so blocking run A cannot delete run B's work on the same + -- legacy channel queue. + if string.find(member, '"runId":"' .. ARGV[1] .. '"', 1, true) then + redis.call('ZREM', KEYS[1], member) + redis.call('HDEL', KEYS[2], member) + pending = pending + 1 + end +end +return pending +` diff --git a/internal/queue/zset/store_test.go b/internal/queue/zset/store_test.go new file mode 100644 index 0000000..0c84432 --- /dev/null +++ b/internal/queue/zset/store_test.go @@ -0,0 +1,274 @@ +package zset + +import ( + "context" + "encoding/json" + "strings" + "testing" + "time" + + "github.com/gogf/gf/v2/container/gvar" +) + +type redisCall struct { + command string + args []any +} + +type fakeRedis struct { + calls []redisCall + responses []*gvar.Var + err error +} + +func (f *fakeRedis) Do(_ context.Context, command string, args ...any) (*gvar.Var, error) { + f.calls = append(f.calls, redisCall{command: command, args: args}) + if f.err != nil { + return nil, f.err + } + if len(f.responses) == 0 { + return gvar.New(int64(1)), nil + } + response := f.responses[0] + f.responses = f.responses[1:] + return response, nil +} + +func (f *fakeRedis) Eval(_ context.Context, script string, numKeys int64, keys []string, args []any) (*gvar.Var, error) { + allArgs := []any{script, numKeys} + for _, k := range keys { + allArgs = append(allArgs, k) + } + allArgs = append(allArgs, args...) + f.calls = append(f.calls, redisCall{command: "EVAL", args: allArgs}) + if f.err != nil { + return nil, f.err + } + if len(f.responses) == 0 { + return gvar.New(int64(1)), nil + } + response := f.responses[0] + f.responses = f.responses[1:] + return response, nil +} + +func TestStoreUsesContractKeys(t *testing.T) { + store, err := New(&fakeRedis{}, "flow:") + if err != nil { + t.Fatal(err) + } + key, err := store.Key("mobile-a") + if err != nil { + t.Fatal(err) + } + if key != "flow:mobile-a" { + t.Fatalf("task key = %q, want flow:mobile-a", key) + } + lease, err := store.leaseKey("mobile-a") + if err != nil { + t.Fatal(err) + } + if lease != "flow:lease:mobile-a" { + t.Fatalf("lease key = %q, want flow:lease:mobile-a", lease) + } + if got := store.ChannelsKey(); got != "flow:channels" { + t.Fatalf("channels key = %q, want flow:channels", got) + } +} + +func TestStableTaskID(t *testing.T) { + if got := StableTaskID("run-1", "current_month_usage", 42); got != "run-1:current_month_usage:42" { + t.Fatalf("stable task id = %q", got) + } +} + +func TestRunKeysShareRedisHashTag(t *testing.T) { + store, err := New(&fakeRedis{}, "flow:") + if err != nil { + t.Fatal(err) + } + keys := []func() (string, error){ + func() (string, error) { return store.RunMetaKey("run-1") }, + func() (string, error) { return store.RoundActiveKey("run-1", 1, "mobile-a") }, + func() (string, error) { return store.RoundWaitingKey("run-1", 2, "mobile-a") }, + func() (string, error) { return store.RoundLeaseKey("run-1", 1, "mobile-a") }, + func() (string, error) { return store.TaskStateKey("run-1") }, + func() (string, error) { return store.FailedKey("run-1") }, + } + for _, build := range keys { + key, err := build() + if err != nil { + t.Fatal(err) + } + if !strings.Contains(key, "flow:{run-1}") { + t.Fatalf("run key %q does not contain hash tag", key) + } + } +} + +func TestEnqueueMemberExcludesSchedulingScore(t *testing.T) { + fake := &fakeRedis{responses: []*gvar.Var{gvar.New(int64(1))}} + store, err := New(fake, "flow:") + if err != nil { + t.Fatal(err) + } + task := Task{TaskID: "run:current_month_usage:1", ChanKey: "mobile-a", CardID: 1} + if err = store.Enqueue(context.Background(), task, time.UnixMilli(1234)); err != nil { + t.Fatal(err) + } + if len(fake.calls) != 1 || fake.calls[0].command != "EVAL" { + t.Fatalf("Redis calls = %+v", fake.calls) + } + member, ok := fake.calls[0].args[len(fake.calls[0].args)-1].(string) + if !ok { + t.Fatalf("member type = %T", fake.calls[0].args[len(fake.calls[0].args)-1]) + } + if strings.Contains(member, "nextExecuteAt") { + t.Fatalf("member contains volatile score: %s", member) + } +} + +func TestClaimDecodesMemberAndUsesLeaseKey(t *testing.T) { + task := Task{ + TaskID: "run-1:current_month_usage:42", + RunID: "run-1", + Operation: "current_month_usage", + QueryMonth: "202608", + ChanKey: "mobile-a", + CardID: 42, + ICCID: "8986000000000042", + } + member, err := json.Marshal(task) + if err != nil { + t.Fatal(err) + } + fake := &fakeRedis{responses: []*gvar.Var{gvar.New([]any{string(member), "worker:token"})}} + store, err := New(fake, "flow:") + if err != nil { + t.Fatal(err) + } + now := time.UnixMilli(1000) + leaseUntil := now.Add(time.Minute) + claimed, err := store.Claim(context.Background(), "mobile-a", now, leaseUntil, 1) + if err != nil { + t.Fatal(err) + } + if len(claimed) != 1 || claimed[0].Task.TaskID != task.TaskID { + t.Fatalf("claimed tasks = %+v", claimed) + } + if claimed[0].LeaseToken != "worker:token" { + t.Fatalf("lease token = %q", claimed[0].LeaseToken) + } + if len(fake.calls) != 1 || fake.calls[0].command != "EVAL" { + t.Fatalf("Redis calls = %+v", fake.calls) + } + args := fake.calls[0].args + if len(args) < 4 || args[3] != "flow:lease:mobile-a" { + t.Fatalf("claim args do not use lease key: %+v", args) + } +} + +func TestAckBatchUsesLuaOwnershipProtocol(t *testing.T) { + fake := &fakeRedis{} + store, err := New(fake, "flow:") + if err != nil { + t.Fatal(err) + } + claimed := ClaimedTask{ + Task: Task{TaskID: "run:current_month_usage:1", ChanKey: "mobile-a", CardID: 1}, + Member: `{"taskId":"run:current_month_usage:1","chanKey":"mobile-a","cardId":1}`, + LeaseToken: "token-1", + } + count, err := store.AckBatch(context.Background(), []ClaimedTask{claimed}) + if err != nil || count != 1 { + t.Fatalf("ack count = %d, err = %v", count, err) + } + renewed, err := store.Renew(context.Background(), claimed, time.Now().Add(time.Minute)) + if err != nil || !renewed { + t.Fatalf("renew = %t, err = %v", renewed, err) + } + if len(fake.calls) != 2 { + t.Fatalf("Redis calls = %d, want 2", len(fake.calls)) + } + for _, call := range fake.calls { + if call.command != "EVAL" { + t.Fatalf("command = %q, want EVAL", call.command) + } + script, ok := call.args[0].(string) + if !ok || !strings.Contains(script, "HGET") { + t.Fatalf("Lua script does not validate ownership: %v", call.args[0]) + } + } +} + +func TestRenewBatchUsesOneAtomicLuaCall(t *testing.T) { + fake := &fakeRedis{responses: []*gvar.Var{gvar.New(int64(2))}} + store, err := New(fake, "flow:") + if err != nil { + t.Fatal(err) + } + tasks := []ClaimedTask{ + {Task: Task{ChanKey: "mobile-a"}, Member: "member-1", LeaseToken: "token-1"}, + {Task: Task{ChanKey: "mobile-a"}, Member: "member-2", LeaseToken: "token-2"}, + } + count, err := store.RenewBatch(context.Background(), tasks, time.UnixMilli(5000)) + if err != nil || count != 2 { + t.Fatalf("RenewBatch() = %d, err = %v", count, err) + } + if len(fake.calls) != 1 || fake.calls[0].command != "EVAL" { + t.Fatalf("Redis calls = %+v, want one EVAL", fake.calls) + } + script := fake.calls[0].args[0].(string) + if !strings.Contains(script, "HGET") || !strings.Contains(script, "ZSCORE") { + t.Fatalf("renew script does not validate all leases: %s", script) + } +} + +func TestEnqueueBatchRegistersChannelAndWritesTasksOnce(t *testing.T) { + fake := &fakeRedis{responses: []*gvar.Var{gvar.New(int64(2))}} + store, err := New(fake, "flow:") + if err != nil { + t.Fatal(err) + } + tasks := []Task{ + {TaskID: "run:usage:1", RunID: "run", ChanKey: "mobile-a", CardID: 1}, + {TaskID: "run:usage:2", RunID: "run", ChanKey: "mobile-a", CardID: 2}, + } + added, err := store.EnqueueBatch(context.Background(), "mobile-a", tasks, time.UnixMilli(1234)) + if err != nil || added != 2 { + t.Fatalf("EnqueueBatch() = %d, err = %v", added, err) + } + if len(fake.calls) != 1 || fake.calls[0].command != "EVAL" { + t.Fatalf("Redis calls = %+v, want one EVAL", fake.calls) + } + if fake.calls[0].args[2] != "flow:mobile-a" || fake.calls[0].args[3] != "flow:channels" || fake.calls[0].args[4] != "flow:blocked:mobile-a" { + t.Fatalf("batch keys = %+v", fake.calls[0].args) + } +} + +func TestEnqueueBatchRejectsMixedChannels(t *testing.T) { + store, err := New(&fakeRedis{}, "flow:") + if err != nil { + t.Fatal(err) + } + _, err = store.EnqueueBatch(context.Background(), "mobile-a", []Task{{TaskID: "1", RunID: "run", ChanKey: "mobile-b"}}, time.Now()) + if err == nil { + t.Fatal("expected mixed channel batch error") + } +} + +func TestBlockRunUsesTaskLeaseAndBlockedKeys(t *testing.T) { + fake := &fakeRedis{responses: []*gvar.Var{gvar.New(int64(7))}} + store, err := New(fake, "flow:") + if err != nil { + t.Fatal(err) + } + removed, err := store.BlockRun(context.Background(), "telecom-a", "run-1") + if err != nil || removed != 7 { + t.Fatalf("BlockRun() = %d, err = %v", removed, err) + } + args := fake.calls[0].args + if args[2] != "flow:telecom-a" || args[3] != "flow:lease:telecom-a" || args[4] != "flow:blocked:telecom-a" { + t.Fatalf("block keys = %+v", args) + } +} diff --git a/internal/queue/zset/supply_observation.go b/internal/queue/zset/supply_observation.go new file mode 100644 index 0000000..6531648 --- /dev/null +++ b/internal/queue/zset/supply_observation.go @@ -0,0 +1,100 @@ +package zset + +import ( + "context" + "encoding/json" + "fmt" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply" + "time" +) + +// ReadSupply uses only queue cardinalities, never task bodies or task-state scans. +// All keys in a batch share the run hash slot. The returned round is verified +// inside the script so a round transition cannot mix queue generations. +func (s *Store) ReadSupply(ctx context.Context, runID string, channels []string, now time.Time) (map[string]supply.Sample, error) { + out := make(map[string]supply.Sample, len(channels)) + meta, err := s.RunMetaKey(runID) + if err != nil { + return nil, err + } + for offset := 0; offset < len(channels); offset += 64 { + batch := channels[offset:min(offset+64, len(channels))] + keys := []string{meta} + args := []any{now.UnixMilli()} + for _, ch := range batch { + active, e := s.RoundActiveKey(runID, 1, ch) + if e != nil { + return nil, e + } + lease, e := s.RoundLeaseKey(runID, 1, ch) + if e != nil { + return nil, e + } + keys = append(keys, active, lease) + args = append(args, ch) + } + result, e := s.redis.Eval(ctx, readSupplyScript, int64(len(keys)), keys, args) + if e != nil { + return nil, e + } + var values [][]int64 + if e = json.Unmarshal([]byte(result.String()), &values); e != nil { + return nil, e + } + if len(values) != len(batch) { + return nil, fmt.Errorf("incomplete supply response") + } + for i, ch := range batch { + v := values[i] + if len(v) != 5 { + return nil, fmt.Errorf("invalid supply response") + } + out[ch] = supply.Sample{Known: v[0] == 1, Eligible: v[1] == 1, Ready: v[2], Leases: v[3], Future: v[4]} + } + } + return out, nil +} + +const readSupplyScript = ` +local run=redis.call('HMGET',KEYS[1],'runId','status','currentRound','producerDone','syncType') +local known=run[1] and run[2] and run[3] and run[4] +local eligible=known and (run[2]=='producing' or run[2]=='running') and run[3]=='1' and run[4]=='0' and run[5]~='replay' +local out={} +for i=2,#KEYS,2 do + local ch=ARGV[(i+2)/2] + local blocked=redis.call('HGET',KEYS[1],'blocked:'..ch) + table.insert(out,{known and 1 or 0,(eligible and blocked~='1') and 1 or 0,redis.call('ZCOUNT',KEYS[i],'-inf',ARGV[1]),redis.call('ZCARD',KEYS[i+1]),redis.call('ZCOUNT',KEYS[i],'('..ARGV[1],'+inf')}) +end +return cjson.encode(out) +` + +// SaveSupply requires owner validation in the same Redis transaction as the +// write. The existing producer lock is global; this matches the standalone +// Redis deployment and deliberately fails closed on a cluster CROSSSLOT error. +func (s *Store) SaveSupply(ctx context.Context, runID, lockKey, owner string, snapshot []byte) (bool, error) { + if len(snapshot) > 256*1024 { + return false, fmt.Errorf("supply snapshot exceeds 256 KiB") + } + meta, err := s.RunMetaKey(runID) + if err != nil { + return false, err + } + result, err := s.redis.Eval(ctx, saveSupplyScript, 2, []string{meta, lockKey}, []any{owner, string(snapshot)}) + if err != nil { + return false, err + } + return result.Int() == 1, nil +} + +const saveSupplyScript = ` +local current=redis.call('GET',KEYS[2]) +if not current then return 0 end +local ok,decoded=pcall(cjson.decode,current) +local owner=current +if ok and type(decoded)=='table' then owner=decoded['owner'] end +if owner~=ARGV[1] then return 0 end +local status=redis.call('HGET',KEYS[1],'status') +if status~='producing' and status~='running' then return 0 end +redis.call('HSET',KEYS[1],'supplyObservation',ARGV[2]) +return 1 +` diff --git a/internal/queue/zset/supply_observation_test.go b/internal/queue/zset/supply_observation_test.go new file mode 100644 index 0000000..9606194 --- /dev/null +++ b/internal/queue/zset/supply_observation_test.go @@ -0,0 +1,78 @@ +package zset + +import ( + "context" + "fmt" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply" + "testing" + "time" +) + +func TestSupplyBatchAndLeaseView(t *testing.T) { + s := newLockStore(t) + ctx := context.Background() + run := "supply-test" + if err := s.CreateRun(ctx, RunMeta{RunID: run, Status: "producing", CurrentRound: 1}); err != nil { + t.Fatal(err) + } + keys := make([]string, 130) + now := time.Now() + for i := range keys { + keys[i] = fmt.Sprintf("channel-%03d", i) + } + a, _ := s.RoundActiveKey(run, 1, keys[0]) + l, _ := s.RoundLeaseKey(run, 1, keys[1]) + if _, err := s.redis.Do(ctx, "ZADD", a, now.UnixMilli(), "ready", now.Add(time.Hour).UnixMilli(), "future"); err != nil { + t.Fatal(err) + } + if _, err := s.redis.Do(ctx, "ZADD", l, now.Add(-time.Hour).UnixMilli(), "expired"); err != nil { + t.Fatal(err) + } + view, err := s.ReadSupply(ctx, run, keys, now) + if err != nil { + t.Fatal(err) + } + if len(view) != 130 || view[keys[0]].Ready != 1 || view[keys[1]].Leases != 1 || !view[keys[129]].Eligible { + t.Fatalf("bad view %+v", view) + } + meta, _ := s.RunMetaKey(run) + s.redis.Do(ctx, "HSET", meta, "blocked:"+keys[2], 1, "currentRound", 2) + view, err = s.ReadSupply(ctx, run, keys, now) + if err != nil { + t.Fatal(err) + } + for _, v := range view { + if v.Eligible { + t.Fatal("retry round eligible") + } + } +} +func TestSupplyOwnerAndTerminalGuard(t *testing.T) { + s := newLockStore(t) + ctx := context.Background() + run := "supply-guard" + s.CreateRun(ctx, RunMeta{RunID: run, Status: "producing", CurrentRound: 1}) + lock := "flow:producer:active" + s.AcquireLockOrTakeover(ctx, lock, "old", run, time.Minute) + encoded, _ := supply.New("old", time.Now(), []string{"a"}).Encode() + ok, err := s.SaveSupply(ctx, run, lock, "old", encoded) + if err != nil || !ok { + t.Fatalf("save %v %v", ok, err) + } + s.AcquireLockOrTakeover(ctx, lock, "new", run, time.Minute) + ok, err = s.SaveSupply(ctx, run, lock, "old", encoded) + if err != nil || ok { + t.Fatalf("old owner save %v %v", ok, err) + } + meta, _ := s.RunMetaKey(run) + s.redis.Do(ctx, "HSET", meta, "status", "completed") + ok, err = s.SaveSupply(ctx, run, lock, "new", encoded) + if err != nil || ok { + t.Fatalf("terminal save %v %v", ok, err) + } + s.redis.Do(ctx, "HSET", meta, "supplyObservation", "{broken") + m, err := s.GetRunMeta(ctx, run) + if err != nil || m.SupplyObservation == nil || !m.SupplyObservation.Partial { + t.Fatalf("invalid telemetry poisoned business %+v %v", m, err) + } +} diff --git a/internal/queue/zset/telecom_template_test.go b/internal/queue/zset/telecom_template_test.go new file mode 100644 index 0000000..c49644f --- /dev/null +++ b/internal/queue/zset/telecom_template_test.go @@ -0,0 +1,50 @@ +package zset + +import "testing" + +// TestFailureRecordUsesTelecomTemplateVersion covers the telecom branch and the +// generic fallback of the carrier label written into a FailureRecord. +func TestFailureRecordUsesTelecomTemplateVersion(t *testing.T) { + for _, key := range []string{"sim_dx_v1", " SIM_DX_V3 "} { + if got := failureFromOutcome(Task{TempKey: key}, TaskOutcome{}, 1); got.Carrier != "telecom" { + t.Fatalf("%q carrier = %q", key, got.Carrier) + } + } + if got := failureFromOutcome(Task{TempKey: "sim_dx_v2_future"}, TaskOutcome{}, 1); got.Carrier != "generic" { + t.Fatalf("unknown carrier = %q", got.Carrier) + } +} + +// TestFailureRecordUsesMobileTemplateSet pins the mobile template list. The list +// must stay in sync with carrierMobile's set in +// internal/consumer/zset/consumer.go: a template missing here is silently +// labelled generic, so failure review and replay would treat a real mobile +// channel as the slow generic pool. +// +// lanshang_8000 / lanshang_8003 are the regression this test was added for -- +// they are OneLink shared-token templates whose names omit the sim_onelink_ +// prefix, so they were omitted from both the QPS policy and this label. +func TestFailureRecordUsesMobileTemplateSet(t *testing.T) { + mobileKeys := []string{ + "sim_yd_v5", + "sim_yd_mock_v5", + "sim_onelink_tengyu_v1", + "sim_onelink_xuzhou_mobile_v1", + "sim_onelink_lelian_v1", + "sim_onelink_beixin_v1", + "sim_onelink_qianxun_v1", + "sim_onelink_henan_xinjing_v1", + "sim_onelink_daihe_v1", + "lanshang_8000", + "lanshang_8003", + } + for _, key := range mobileKeys { + if got := failureFromOutcome(Task{TempKey: key}, TaskOutcome{}, 1); got.Carrier != "mobile" { + t.Fatalf("%q carrier = %q, want mobile", key, got.Carrier) + } + } + // Case and padding must not change the classification. + if got := failureFromOutcome(Task{TempKey: " LANSHANG_8000 "}, TaskOutcome{}, 1); got.Carrier != "mobile" { + t.Fatalf("padded lanshang_8000 carrier = %q, want mobile", got.Carrier) + } +} diff --git a/internal/repository/batch/metadata_test.go b/internal/repository/batch/metadata_test.go new file mode 100644 index 0000000..70ad86a --- /dev/null +++ b/internal/repository/batch/metadata_test.go @@ -0,0 +1,35 @@ +package batch + +import ( + "context" + batchdomain "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "testing" +) + +func TestExecutionPreservesOriginMetadata(t *testing.T) { + ctx := context.Background() + r := newTestRepository(t, ctx) + insertBatch(t, ctx, r.db, "META", batchdomain.Type103) + if _, err := r.db.Model(dao.BatchMain.Table()).Ctx(ctx).Where("batch_no", "META").Data(do.BatchMain{SourceChannel: "mini_program", ExecutionMode: 1}).Update(); err != nil { + t.Fatal(err) + } + if claimed, err := r.MarkProcessing(ctx, "META", batchdomain.Type103); err != nil || !claimed { + t.Fatal(claimed, err) + } + if marked, err := r.MarkFailed(ctx, "META", batchdomain.Type103, "test failure"); err != nil || !marked { + t.Fatal(marked, err) + } + main, err := r.FindMain(ctx, "META") + if err != nil || main.SourceChannel != "mini_program" || main.ExecutionMode != 1 { + t.Fatalf("metadata changed: %+v err=%v", main, err) + } + if marked, err := r.MarkFailed(ctx, "META", batchdomain.Type103, "duplicate"); err != nil || marked { + t.Fatal(marked, err) + } + main, err = r.FindMain(ctx, "META") + if err != nil || main.SourceChannel != "mini_program" || main.ExecutionMode != 1 { + t.Fatal("duplicate mutated origin", err) + } +} diff --git a/internal/repository/batch/repository.go b/internal/repository/batch/repository.go new file mode 100644 index 0000000..60c9721 --- /dev/null +++ b/internal/repository/batch/repository.go @@ -0,0 +1,200 @@ +package batch + +import ( + "context" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/os/gtime" + + batchdomain "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +const ( + AsyncStatusSuccess = 3 + RunResultSuccess = 2 + batchDetailErrorReasonLimit = 512 +) + +type Repository struct { + db gdb.DB +} + +func New() *Repository { + return &Repository{db: dao.BatchMain.DB()} +} + +func (r *Repository) FindMain(ctx context.Context, batchNo string) (*entity.BatchMain, error) { + var main *entity.BatchMain + if err := r.db.Model(dao.BatchMain.Table()).Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Scan(&main); err != nil { + return nil, gerror.Wrap(err, "query batch main failed") + } + return main, nil +} + +func (r *Repository) MarkProcessing(ctx context.Context, batchNo string, batchType int) (claimed bool, err error) { + err = r.db.Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + mainColumns := dao.BatchMain.Columns() + result, updateErr := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(mainColumns.BatchNo, batchNo). + Where(mainColumns.BatchType, batchType). + Where(mainColumns.BatchAsyncStatus, batchdomain.AsyncStatusPending). + Data(do.BatchMain{ + BatchAsyncStatus: batchdomain.AsyncStatusProcessing, + RunResult: batchdomain.RunResultRunning, + StartTime: gtime.Now(), + }). + Update() + if updateErr != nil { + return gerror.Wrap(updateErr, "CAS batch main failed") + } + affected, affectedErr := result.RowsAffected() + if affectedErr != nil { + return gerror.Wrap(affectedErr, "read batch main CAS result failed") + } + if affected == 0 { + return nil + } + + detailColumns := dao.BatchDetail.Columns() + if _, updateErr = tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(detailColumns.BatchNo, batchNo). + Where(detailColumns.BatchType, batchType). + Where(detailColumns.DetailStatus, batchdomain.DetailStatusPending). + Data(do.BatchDetail{DetailStatus: batchdomain.DetailStatusRunning}). + Update(); updateErr != nil { + return gerror.Wrap(updateErr, "mark pending batch details running failed") + } + claimed = true + return nil + }) + return claimed, gerror.Wrap(err, "claim batch transaction failed") +} + +func (r *Repository) ReclaimProcessing(ctx context.Context, batchNo string, batchType int, startedAt *gtime.Time) (claimed bool, err error) { + columns := dao.BatchMain.Columns() + model := r.db.Model(dao.BatchMain.Table()).Ctx(ctx). + Where(columns.BatchNo, batchNo). + Where(columns.BatchType, batchType). + Where(columns.BatchAsyncStatus, batchdomain.AsyncStatusProcessing) + if startedAt == nil || startedAt.IsZero() { + model = model.Where(columns.StartTime, nil) + } else { + model = model.Where(columns.StartTime, startedAt) + } + + result, err := model.Data(do.BatchMain{ + StartTime: gtime.Now(), + RunResult: batchdomain.RunResultRunning, + }).Update() + if err != nil { + return false, gerror.Wrap(err, "reclaim stale batch failed") + } + affected, err := result.RowsAffected() + if err != nil { + return false, gerror.Wrap(err, "read stale batch reclaim result failed") + } + return affected == 1, nil +} + +func (r *Repository) MarkFailed(ctx context.Context, batchNo string, batchType int, reason string) (marked bool, err error) { + detailReason := truncateBatchDetailReason(reason) + err = r.db.Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + mainColumns := dao.BatchMain.Columns() + result, updateErr := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(mainColumns.BatchNo, batchNo). + Where(mainColumns.BatchType, batchType). + Where(mainColumns.BatchAsyncStatus, batchdomain.AsyncStatusProcessing). + Data(do.BatchMain{ + BatchAsyncStatus: batchdomain.AsyncStatusFailed, + RunResult: batchdomain.RunResultFailed, + FailReason: reason, + EndTime: gtime.Now(), + }). + Update() + if updateErr != nil { + return gerror.Wrap(updateErr, "mark batch main failed failed") + } + affected, affectedErr := result.RowsAffected() + if affectedErr != nil { + return gerror.Wrap(affectedErr, "read batch failure status update result failed") + } + if affected == 0 { + return nil + } + + detailColumns := dao.BatchDetail.Columns() + if _, updateErr = tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(detailColumns.BatchNo, batchNo). + Where(detailColumns.BatchType, batchType). + Where(detailColumns.DetailStatus, batchdomain.DetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: batchdomain.DetailStatusFailed, + ErrorReason: detailReason, + }). + Update(); updateErr != nil { + return gerror.Wrap(updateErr, "mark running batch details failed failed") + } + marked = true + return nil + }) + return marked, gerror.Wrap(err, "mark batch failure transaction failed") +} + +func truncateBatchDetailReason(reason string) string { + runes := []rune(strings.TrimSpace(reason)) + if len(runes) <= batchDetailErrorReasonLimit { + return string(runes) + } + return string(runes[:batchDetailErrorReasonLimit]) +} + +// MarkSuccess 将指定批次从 Processing 标记为 Success，并结算明细状态。 +func (r *Repository) MarkSuccess(ctx context.Context, batchNo string, batchType int, success, fail int, reason string) (marked bool, err error) { + err = r.db.Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + mainColumns := dao.BatchMain.Columns() + result, updateErr := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(mainColumns.BatchNo, batchNo). + Where(mainColumns.BatchType, batchType). + Where(mainColumns.BatchAsyncStatus, batchdomain.AsyncStatusProcessing). + Data(do.BatchMain{ + BatchAsyncStatus: AsyncStatusSuccess, + RunResult: RunResultSuccess, + SuccessCount: success, + FailCount: fail, + EndTime: gtime.Now(), + }). + Update() + if updateErr != nil { + return gerror.Wrap(updateErr, "mark batch main success failed") + } + affected, affectedErr := result.RowsAffected() + if affectedErr != nil { + return gerror.Wrap(affectedErr, "read batch success status update result failed") + } + if affected == 0 { + return nil + } + detailColumns := dao.BatchDetail.Columns() + if _, updateErr = tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(detailColumns.BatchNo, batchNo). + Where(detailColumns.BatchType, batchType). + Where(detailColumns.DetailStatus, batchdomain.DetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: batchdomain.DetailStatusSuccess, + ErrorReason: reason, + }). + Update(); updateErr != nil { + return gerror.Wrap(updateErr, "mark running batch details success failed") + } + marked = true + return nil + }) + return marked, gerror.Wrap(err, "mark batch success transaction failed") +} diff --git a/internal/repository/batch/repository_test.go b/internal/repository/batch/repository_test.go new file mode 100644 index 0000000..f1278ca --- /dev/null +++ b/internal/repository/batch/repository_test.go @@ -0,0 +1,204 @@ +package batch + +import ( + "context" + "strings" + "testing" + + _ "github.com/gogf/gf/contrib/drivers/sqlite/v2" + "github.com/gogf/gf/v2/database/gdb" + + batchdomain "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +func TestMarkProcessingClaimsMainAndPendingDetails(t *testing.T) { + ctx := context.Background() + repository := newTestRepository(t, ctx) + insertBatch(t, ctx, repository.db, "B-103", batchdomain.Type103) + main, err := repository.FindMain(ctx, "B-103") + if err != nil || main == nil || main.BatchType != batchdomain.Type103 { + t.Fatalf("FindMain() = (%+v, %v), want batch type %d", main, err, batchdomain.Type103) + } + missing, err := repository.FindMain(ctx, "missing") + if err != nil || missing != nil { + t.Fatalf("FindMain(missing) = (%+v, %v), want nil, nil", missing, err) + } + + claimed, err := repository.MarkProcessing(ctx, "B-103", batchdomain.Type103) + if err != nil { + t.Fatalf("MarkProcessing() error = %v", err) + } + if !claimed { + t.Fatal("MarkProcessing() claimed = false, want true") + } + assertStatus(t, ctx, repository.db, dao.BatchMain.Table(), dao.BatchMain.Columns().BatchAsyncStatus, batchdomain.AsyncStatusProcessing) + assertStatus(t, ctx, repository.db, dao.BatchMain.Table(), dao.BatchMain.Columns().RunResult, batchdomain.RunResultRunning) + main, err = repository.FindMain(ctx, "B-103") + if err != nil || main.StartTime == nil || main.StartTime.IsZero() { + t.Fatalf("processing batch start time = %v, error = %v; want current time", main.StartTime, err) + } + + var details []entity.BatchDetail + if err = repository.db.Model(dao.BatchDetail.Table()).Ctx(ctx). + Order(dao.BatchDetail.Columns().Id). + Scan(&details); err != nil { + t.Fatalf("query detail statuses: %v", err) + } + if len(details) != 2 || details[0].DetailStatus != batchdomain.DetailStatusRunning || details[1].DetailStatus != 2 { + t.Fatalf("details = %+v, want statuses [1 2]", details) + } + + claimed, err = repository.MarkProcessing(ctx, "B-103", batchdomain.Type103) + if err != nil { + t.Fatalf("second MarkProcessing() error = %v", err) + } + if claimed { + t.Fatal("second MarkProcessing() claimed = true, want false") + } +} + +func TestMarkProcessingRollsBackMainWhenDetailUpdateFails(t *testing.T) { + ctx := context.Background() + repository := newTestRepository(t, ctx) + insertBatch(t, ctx, repository.db, "B-104", batchdomain.Type104) + if _, err := repository.db.Exec(ctx, "DROP TABLE batch_detail"); err != nil { + t.Fatalf("drop detail table: %v", err) + } + + if claimed, err := repository.MarkProcessing(ctx, "B-104", batchdomain.Type104); err == nil || claimed { + t.Fatalf("MarkProcessing() = (%t, %v), want false and error", claimed, err) + } + assertStatus(t, ctx, repository.db, dao.BatchMain.Table(), dao.BatchMain.Columns().BatchAsyncStatus, batchdomain.AsyncStatusPending) + assertStatus(t, ctx, repository.db, dao.BatchMain.Table(), dao.BatchMain.Columns().RunResult, 0) + main, err := repository.FindMain(ctx, "B-104") + if err != nil || main.StartTime != nil { + t.Fatalf("rolled back batch start time = %v, error = %v; want nil", main.StartTime, err) + } +} + +func TestMarkFailedFinalizesProcessingBatchAndDetails(t *testing.T) { + ctx := context.Background() + repository := newTestRepository(t, ctx) + insertBatch(t, ctx, repository.db, "B-105", batchdomain.Type103) + if claimed, err := repository.MarkProcessing(ctx, "B-105", batchdomain.Type103); err != nil || !claimed { + t.Fatalf("MarkProcessing() = (%t, %v), want true, nil", claimed, err) + } + + marked, err := repository.MarkFailed(ctx, "B-105", batchdomain.Type103, "source file is unavailable") + if err != nil || !marked { + t.Fatalf("MarkFailed() = (%t, %v), want true, nil", marked, err) + } + assertStatus(t, ctx, repository.db, dao.BatchMain.Table(), dao.BatchMain.Columns().BatchAsyncStatus, batchdomain.AsyncStatusFailed) + assertStatus(t, ctx, repository.db, dao.BatchMain.Table(), dao.BatchMain.Columns().RunResult, batchdomain.RunResultFailed) + + var main entity.BatchMain + if err = repository.db.Model(dao.BatchMain.Table()).Ctx(ctx).Where(dao.BatchMain.Columns().BatchNo, "B-105").Scan(&main); err != nil { + t.Fatalf("query failed batch main: %v", err) + } + if main.FailReason != "source file is unavailable" || main.EndTime == nil || main.EndTime.IsZero() { + t.Fatalf("failed batch main = %+v, want failure reason and end time", main) + } + + var details []entity.BatchDetail + if err = repository.db.Model(dao.BatchDetail.Table()).Ctx(ctx).Order(dao.BatchDetail.Columns().Id).Scan(&details); err != nil { + t.Fatalf("query failed batch details: %v", err) + } + if len(details) != 2 || details[0].DetailStatus != batchdomain.DetailStatusFailed || details[0].ErrorReason != "source file is unavailable" || details[1].DetailStatus != 2 { + t.Fatalf("failed details = %+v, want statuses [3 2]", details) + } + + marked, err = repository.MarkFailed(ctx, "B-105", batchdomain.Type103, "ignored") + if err != nil || marked { + t.Fatalf("second MarkFailed() = (%t, %v), want false, nil", marked, err) + } +} + +func TestMarkFailedTruncatesDetailReason(t *testing.T) { + ctx := context.Background() + repository := newTestRepository(t, ctx) + insertBatch(t, ctx, repository.db, "B-106", batchdomain.Type103) + if claimed, err := repository.MarkProcessing(ctx, "B-106", batchdomain.Type103); err != nil || !claimed { + t.Fatalf("MarkProcessing() = (%t, %v), want true, nil", claimed, err) + } + + reason := strings.Repeat("x", batchDetailErrorReasonLimit+20) + marked, err := repository.MarkFailed(ctx, "B-106", batchdomain.Type103, reason) + if err != nil || !marked { + t.Fatalf("MarkFailed() = (%t, %v), want true, nil", marked, err) + } + + var main entity.BatchMain + if err = repository.db.Model(dao.BatchMain.Table()).Ctx(ctx).Where(dao.BatchMain.Columns().BatchNo, "B-106").Scan(&main); err != nil { + t.Fatalf("query failed batch main: %v", err) + } + if main.FailReason != reason { + t.Fatalf("main failure reason length = %d, want full length %d", len([]rune(main.FailReason)), len([]rune(reason))) + } + + var details []entity.BatchDetail + if err = repository.db.Model(dao.BatchDetail.Table()).Ctx(ctx). + Where(dao.BatchDetail.Columns().BatchNo, "B-106"). + Order(dao.BatchDetail.Columns().Id). + Scan(&details); err != nil { + t.Fatalf("query failed batch details: %v", err) + } + if len(details) != 2 || len([]rune(details[0].ErrorReason)) != batchDetailErrorReasonLimit || details[1].ErrorReason != "" { + t.Fatalf("detail failure reasons = %+v, want first truncated to %d and second unchanged", details, batchDetailErrorReasonLimit) + } +} + +func newTestRepository(t *testing.T, ctx context.Context) *Repository { + t.Helper() + db, err := gdb.New(gdb.ConfigNode{ + Type: "sqlite", + Link: "sqlite::@file(:memory:)", + }) + if err != nil { + t.Fatalf("create sqlite database: %v", err) + } + statements := []string{ + "CREATE TABLE batch_main (batch_no TEXT PRIMARY KEY, batch_type INTEGER NOT NULL, batch_async_status INTEGER NOT NULL, run_result INTEGER NOT NULL DEFAULT 0, fail_reason TEXT, start_time DATETIME, end_time DATETIME, source_channel TEXT NOT NULL DEFAULT 'legacy', execution_mode INTEGER NOT NULL DEFAULT 0)", + "CREATE TABLE batch_detail (id INTEGER PRIMARY KEY AUTOINCREMENT, batch_no TEXT NOT NULL, batch_type INTEGER NOT NULL, detail_status INTEGER NOT NULL, error_reason TEXT)", + } + for _, statement := range statements { + if _, err = db.Exec(ctx, statement); err != nil { + t.Fatalf("create test schema: %v", err) + } + } + t.Cleanup(func() { _ = db.Close(ctx) }) + return &Repository{db: db} +} + +func insertBatch(t *testing.T, ctx context.Context, db gdb.DB, batchNo string, batchType int) { + t.Helper() + if _, err := db.Model(dao.BatchMain.Table()).Ctx(ctx).Data(do.BatchMain{ + BatchNo: batchNo, + BatchType: batchType, + BatchAsyncStatus: batchdomain.AsyncStatusPending, + }).Insert(); err != nil { + t.Fatalf("insert batch main: %v", err) + } + for _, status := range []int{batchdomain.DetailStatusPending, 2} { + if _, err := db.Model(dao.BatchDetail.Table()).Ctx(ctx).Data(do.BatchDetail{ + BatchNo: batchNo, + BatchType: batchType, + DetailStatus: status, + }).Insert(); err != nil { + t.Fatalf("insert batch detail: %v", err) + } + } +} + +func assertStatus(t *testing.T, ctx context.Context, db gdb.DB, table, column string, expected int) { + t.Helper() + actual, err := db.Model(table).Ctx(ctx).Value(column) + if err != nil { + t.Fatalf("query %s status: %v", table, err) + } + if actual.Int() != expected { + t.Fatalf("%s status = %d, want %d", table, actual.Int(), expected) + } +} diff --git a/internal/repository/cancel/repository.go b/internal/repository/cancel/repository.go new file mode 100644 index 0000000..376bbc8 --- /dev/null +++ b/internal/repository/cancel/repository.go @@ -0,0 +1,1052 @@ +package cancel + +import ( + "context" + "encoding/json" + "sort" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" + cancelservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/cancel" +) + +const ( + cardInfoTable = "t_sim_card_info" + cardBindPkgTable = "t_sim_card_bind_pkg" + bindPackageLogsTable = "t_sim_bind_package_logs" + orderTable = "t_order" + orderItemTable = "t_order_item" + + bindStatusPending = 0 + bindStatusEffective = 1 + bindStatusTerminated = 3 + + orderReturnStatusPartial = 3 + orderReturnStatusSuccess = 4 +) + +type Repository struct { + db gdb.DB + resultFiles batchresult.Generator + lockEnabled bool +} + +func New() *Repository { + return &Repository{resultFiles: batchresult.New(), lockEnabled: true} +} + +func (r *Repository) database() gdb.DB { + if r.db != nil { + return r.db + } + return g.DB() +} + +func (r *Repository) applyLock(model *gdb.Model) *gdb.Model { + if r.lockEnabled { + return model.LockUpdate() + } + return model +} + +type bindPkgQuery struct { + g.Meta `orm:"table:t_sim_card_bind_pkg"` + Id int64 `orm:"id"` + SubscriptionNo string `orm:"subscription_no"` + BatchNo string `orm:"batch_no"` + Iccid string `orm:"iccid"` + Msisdn string `orm:"msisdn"` + DeptId string `orm:"dept_id"` + PackageType int `orm:"package_type"` + PackageCode string `orm:"package_code"` + PreviousPackageCode string `orm:"previous_package_code"` + BusinessType int `orm:"business_type"` + EffectiveStatus int `orm:"effective_status"` + HandleTime *gtime.Time `orm:"handle_time"` + EffectiveTime *gtime.Time `orm:"effective_time"` + ExpectedExpireTime *gtime.Time `orm:"expected_expire_time"` + ActualExpireTime *gtime.Time `orm:"actual_expire_time"` + DurationMonths int `orm:"duration_months"` + GiftDurationMonths int `orm:"gift_duration_months"` + OrderCode string `orm:"order_code"` + Remark string `orm:"remark"` + TenantId string `orm:"tenant_id"` + CreatedDept int64 `orm:"created_dept"` + CreatedBy int64 `orm:"created_by"` + CreatedAt *gtime.Time `orm:"created_at"` +} + +type bindPackageLogInsert struct { + g.Meta `orm:"table:t_sim_bind_package_logs, do:true"` + SubscriptionNo any + BatchNo any + Iccid any + Msisdn any + DeptId any + PackageType any + PackageCode any + PreviousPackageCode any + BusinessType any + EffectiveStatus any + HandleTime any + EffectiveTime any + ExpectedExpireTime any + ActualExpireTime any + DurationMonths any + GiftDurationMonths any + OrderCode any + Remark any + TenantId any + CreatedDept any + CreatedBy any + UpdatedBy any +} + +type orderUpdate struct { + g.Meta `orm:"table:t_order, do:true"` + ReturnStatus any + UpdatedBy any +} + +type orderItemUpdate struct { + g.Meta `orm:"table:t_order_item, do:true"` + ExtraSkuData any + UpdatedBy any +} + +type orderItemQuery struct { + g.Meta `orm:"table:t_order_item"` + ItemId int64 `orm:"item_id"` + OrderSn string `orm:"order_sn"` + ExtraSkuData string `orm:"extra_sku_data"` +} + +type orderItemExtra struct { + LastCancelBatchNo string `json:"lastCancelBatchNo"` + LastCancelTime string `json:"lastCancelTime"` + CancelledQuantity int `json:"cancelledQuantity"` +} + +func (r *Repository) Load(ctx context.Context, batchNo string) (*cancelservice.Task, error) { + if strings.TrimSpace(batchNo) == "" { + return nil, gerror.New("cancel batch number is empty") + } + var main entity.BatchMain + if err := r.database().Model(dao.BatchMain.Table()).Safe().Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeSimCardCancel). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Scan(&main); err != nil { + return nil, gerror.Wrap(err, "query cancel batch main") + } + if main.Id == 0 { + return nil, gerror.New("processing cancel batch does not exist") + } + + var details []*entity.BatchDetail + if err := r.database().Model(dao.BatchDetail.Table()).Safe().Ctx(ctx). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeSimCardCancel). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + OrderAsc(dao.BatchDetail.Columns().RowNo). + Scan(&details); err != nil { + return nil, gerror.Wrap(err, "query cancel running details") + } + if len(details) == 0 { + return nil, gerror.New("cancel batch has no running details") + } + + task := &cancelservice.Task{ + Batch: cancelservice.Batch{ + ID: main.Id, + BatchNo: main.BatchNo, + BatchType: main.BatchType, + TenantID: main.TenantId, + DeptID: main.DeptId, + CreatedBy: main.CreatedBy, + CreatedDept: main.CreatedDept, + StartTime: main.StartTime, + BatchInfo: main.BatchInfo, + }, + Details: make([]*cancelservice.Detail, 0, len(details)), + } + for _, raw := range details { + if raw != nil { + task.Details = append(task.Details, buildDetailFromBatch(raw)) + } + } + if len(task.Details) == 0 { + return nil, gerror.New("cancel batch has no readable running details") + } + return task, nil +} + +func buildDetailFromBatch(raw *entity.BatchDetail) *cancelservice.Detail { + detail := &cancelservice.Detail{ + ID: raw.Id, + RowNo: raw.RowNo, + ICCID: strings.TrimSpace(raw.ItemId), + Reason: strings.TrimSpace(raw.ErrorReason), + TargetStatus: consts.SimBossCardStatusCanceled, + } + if raw.Snapshot != nil { + if value := firstSnapshotString(raw.Snapshot, "ICCID", "iccid"); value != "" { + detail.ICCID = value + } + detail.MSISDN = firstSnapshotString(raw.Snapshot, "MSISDN", "msisdn") + detail.IMSI = firstSnapshotString(raw.Snapshot, "IMSI", "imsi") + detail.InputMSISDN = detail.MSISDN + detail.InputIMSI = detail.IMSI + detail.Authorized = raw.Snapshot.Get("authorized").Bool() + } + return detail +} + +func firstSnapshotString(snapshot *gjson.Json, keys ...string) string { + if snapshot == nil { + return "" + } + for _, key := range keys { + if value := strings.TrimSpace(snapshot.Get(key).String()); value != "" { + return value + } + } + return "" +} + +func (r *Repository) SaveValidationFailure(ctx context.Context, task *cancelservice.Task) error { + if task == nil { + return gerror.New("cancel task is nil") + } + reason := task.FailureReason + if reason == "" { + reason = "batch validation failed; cancellation was skipped" + } + now := gtime.Now() + return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + if err := updateDetailsFailed(ctx, tx, task, reason); err != nil { + return err + } + return updateMainFailed(ctx, tx, task, len(task.Details), reason, now) + }) +} + +func (r *Repository) CommitBusiness(ctx context.Context, task *cancelservice.Task) error { + if task == nil { + return gerror.New("cancel task is nil") + } + iccids := uniqueSortedICCID(task.Details) + now := gtime.Now() + return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + cardsByICCID, err := r.lockCardsByIccids(ctx, tx, task.Batch.TenantID, iccids) + if err != nil { + return gerror.Wrap(err, "lock cancel SIM cards") + } + cancelled, err := r.lockCancellationsByIccids(ctx, tx, iccids) + if err != nil { + return gerror.Wrap(err, "lock existing cancellation records") + } + bindsByICCID, err := r.lockCurrentActiveBindsByIccids(ctx, tx, iccids) + if err != nil { + return gerror.Wrap(err, "lock current active binds") + } + + hasFailure := false + for _, detail := range task.Details { + if detail == nil { + hasFailure = true + continue + } + card := cardsByICCID[detail.ICCID] + if reason := validateBusiness(task, detail, card, bindsByICCID[detail.ICCID], cancelled[detail.ICCID], now); reason != "" { + detail.Reason = reason + hasFailure = true + } + } + if hasFailure { + reason := "same batch contains SIM cards that cannot be cancelled; cancellation was skipped" + return finalizeFailure(ctx, tx, task, reason, now) + } + + if err = writeBindPackageLogs(ctx, tx, task, bindsByICCID, now); err != nil { + return err + } + if err = deleteCurrentBinds(ctx, tx, bindsByICCID); err != nil { + return err + } + if err = updateOrderCancelStatus(ctx, tx, task, bindsByICCID, now); err != nil { + return err + } + if err = insertCancellationCards(ctx, tx, task, cardsByICCID, now); err != nil { + return err + } + if err = deleteFormalCards(ctx, tx, task, cardsByICCID); err != nil { + return err + } + if err = updateDetailsSuccess(ctx, tx, task); err != nil { + return err + } + return updateMainSuccess(ctx, tx, task, len(iccids), now) + }) +} + +func validateBusiness(task *cancelservice.Task, detail *cancelservice.Detail, card *entity.TSimCardInfo, binds []*bindPkgQuery, existsInCancel bool, now *gtime.Time) string { + if detail.Reason != "" { + return detail.Reason + } + if !detail.Authorized { + return "ICCID is outside the current operator authorization scope" + } + if card == nil { + return "ICCID does not exist in formal SIM card records" + } + populateDetailFromCard(detail, card, binds, now) + if detail.InputMSISDN != "" && detail.InputMSISDN != strings.TrimSpace(card.Msisdn) { + return "MSISDN does not match the SIM card record" + } + if detail.InputIMSI != "" && detail.InputIMSI != strings.TrimSpace(card.Imsi) { + return "IMSI does not match the SIM card record" + } + if task.Batch.DeptID > 0 && card.DeptId != task.Batch.DeptID && card.CreatedDept != task.Batch.DeptID { + return "ICCID is outside the current department scope" + } + if card.BossCardStatus != consts.SimBossCardStatusStopped { + return "SIM card must be stopped before cancellation" + } + if existsInCancel { + return "ICCID already exists in cancellation records" + } + return "" +} + +func populateDetailFromCard(detail *cancelservice.Detail, card *entity.TSimCardInfo, binds []*bindPkgQuery, now *gtime.Time) { + detail.MSISDN = strings.TrimSpace(card.Msisdn) + detail.IMSI = strings.TrimSpace(card.Imsi) + detail.StatusBefore = card.BossCardStatus + detail.TargetStatus = consts.SimBossCardStatusCanceled + detail.CancelTime = formatTime(now) + detail.CardDeptID = card.DeptId + bind := firstCurrentBind(binds) + if bind == nil { + return + } + detail.SubscriptionNo = strings.TrimSpace(bind.SubscriptionNo) + detail.OrderCode = strings.TrimSpace(bind.OrderCode) + detail.PackageCode = strings.TrimSpace(bind.PackageCode) + detail.PackageType = bind.PackageType +} + +func (r *Repository) lockCardsByIccids(ctx context.Context, tx gdb.TX, tenantID string, iccids []string) (map[string]*entity.TSimCardInfo, error) { + result := make(map[string]*entity.TSimCardInfo) + if len(iccids) == 0 { + return result, nil + } + var rows []*entity.TSimCardInfo + model := r.applyLock(tx.Ctx(ctx).Model(dao.TSimCardInfo.Table())). + WhereIn(dao.TSimCardInfo.Columns().Iccid, iccids) + if strings.TrimSpace(tenantID) != "" { + model = model.Where(dao.TSimCardInfo.Columns().TenantId, tenantID) + } + if err := model.Scan(&rows); err != nil { + return nil, err + } + for _, row := range rows { + if row != nil && strings.TrimSpace(row.Iccid) != "" { + result[strings.TrimSpace(row.Iccid)] = row + } + } + return result, nil +} + +func (r *Repository) lockCancellationsByIccids(ctx context.Context, tx gdb.TX, iccids []string) (map[string]bool, error) { + result := make(map[string]bool) + if len(iccids) == 0 { + return result, nil + } + var rows []*entity.TSimCardCancellation + if err := r.applyLock(tx.Ctx(ctx).Model(dao.TSimCardCancellation.Table())). + WhereIn(dao.TSimCardCancellation.Columns().Iccid, iccids). + Scan(&rows); err != nil { + return nil, err + } + for _, row := range rows { + if row != nil && strings.TrimSpace(row.Iccid) != "" { + result[strings.TrimSpace(row.Iccid)] = true + } + } + return result, nil +} + +func (r *Repository) lockCurrentActiveBindsByIccids(ctx context.Context, tx gdb.TX, iccids []string) (map[string][]*bindPkgQuery, error) { + result := make(map[string][]*bindPkgQuery) + if len(iccids) == 0 { + return result, nil + } + var rows []*bindPkgQuery + if err := r.applyLock(tx.Ctx(ctx).Model(cardBindPkgTable)). + WhereIn("iccid", iccids). + WhereIn("effective_status", []int{bindStatusPending, bindStatusEffective}). + OrderAsc("id"). + Scan(&rows); err != nil { + return nil, err + } + for _, row := range rows { + if row == nil || strings.TrimSpace(row.Iccid) == "" { + continue + } + iccid := strings.TrimSpace(row.Iccid) + result[iccid] = append(result[iccid], row) + } + return result, nil +} + +func writeBindPackageLogs(ctx context.Context, tx gdb.TX, task *cancelservice.Task, binds map[string][]*bindPkgQuery, now *gtime.Time) error { + currentBinds := flattenCurrentBinds(binds) + rows := make([]bindPackageLogInsert, 0, len(currentBinds)) + for _, bind := range currentBinds { + rows = append(rows, bindPackageLogInsert{ + SubscriptionNo: bind.SubscriptionNo, + BatchNo: bind.BatchNo, + Iccid: bind.Iccid, + Msisdn: bind.Msisdn, + DeptId: bind.DeptId, + PackageType: bind.PackageType, + PackageCode: bind.PackageCode, + PreviousPackageCode: bind.PreviousPackageCode, + BusinessType: bind.BusinessType, + EffectiveStatus: bindStatusTerminated, + HandleTime: bind.HandleTime, + EffectiveTime: bind.EffectiveTime, + ExpectedExpireTime: bind.ExpectedExpireTime, + ActualExpireTime: now, + DurationMonths: bind.DurationMonths, + GiftDurationMonths: bind.GiftDurationMonths, + OrderCode: bind.OrderCode, + Remark: appendRemark(bind.Remark, "cancelBatchNo="+task.Batch.BatchNo), + TenantId: bind.TenantId, + CreatedDept: bind.CreatedDept, + CreatedBy: bind.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + }) + } + if len(rows) == 0 { + return nil + } + if _, err := tx.Ctx(ctx).Model(bindPackageLogsTable).Data(rows).Insert(); err != nil { + return gerror.Wrap(err, "insert cancel bind package logs") + } + return nil +} + +func deleteCurrentBinds(ctx context.Context, tx gdb.TX, binds map[string][]*bindPkgQuery) error { + currentBinds := flattenCurrentBinds(binds) + ids := make([]int64, 0, len(currentBinds)) + for _, bind := range currentBinds { + if bind.Id <= 0 { + return gerror.New("current subscription record has no valid ID") + } + ids = append(ids, bind.Id) + } + if len(ids) == 0 { + return nil + } + result, err := tx.Ctx(ctx).Model(cardBindPkgTable). + WhereIn("id", ids). + Delete() + if err != nil { + return gerror.Wrap(err, "delete current cancel binds") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read current cancel bind delete result") + } + if int(affected) != len(ids) { + return gerror.New("some current subscription records changed") + } + return nil +} + +func updateOrderCancelStatus(ctx context.Context, tx gdb.TX, task *cancelservice.Task, binds map[string][]*bindPkgQuery, now *gtime.Time) error { + orderICCIDs := make(map[string]map[string]struct{}) + for _, bind := range flattenCurrentBinds(binds) { + if orderCode := strings.TrimSpace(bind.OrderCode); orderCode != "" { + if orderICCIDs[orderCode] == nil { + orderICCIDs[orderCode] = make(map[string]struct{}) + } + orderICCIDs[orderCode][strings.TrimSpace(bind.Iccid)] = struct{}{} + } + } + if len(orderICCIDs) == 0 { + return nil + } + orderCodes := make([]string, 0, len(orderICCIDs)) + for orderCode := range orderICCIDs { + orderCodes = append(orderCodes, orderCode) + } + sort.Strings(orderCodes) + activeCounts, err := activeBindCountsByOrderCodes(ctx, tx, orderCodes) + if err != nil { + return gerror.Wrap(err, "count remaining active binds by order") + } + for _, orderCode := range orderCodes { + returnStatus := orderReturnStatusSuccess + if activeCounts[orderCode] > 0 { + returnStatus = orderReturnStatusPartial + } + if _, err = tx.Ctx(ctx).Model(orderTable). + Where("order_sn", orderCode). + Data(orderUpdate{ReturnStatus: returnStatus, UpdatedBy: task.Batch.CreatedBy}). + Update(); err != nil { + return gerror.Wrap(err, "update cancel order return status") + } + if err = updateOrderItemExtra(ctx, tx, orderCode, task.Batch.BatchNo, len(orderICCIDs[orderCode]), now, task.Batch.CreatedBy); err != nil { + return err + } + } + return nil +} + +func firstCurrentBind(binds []*bindPkgQuery) *bindPkgQuery { + for _, bind := range binds { + if bind != nil { + return bind + } + } + return nil +} + +func flattenCurrentBinds(bindsByICCID map[string][]*bindPkgQuery) []*bindPkgQuery { + iccids := make([]string, 0, len(bindsByICCID)) + for iccid := range bindsByICCID { + iccids = append(iccids, iccid) + } + sort.Strings(iccids) + + result := make([]*bindPkgQuery, 0) + for _, iccid := range iccids { + for _, bind := range bindsByICCID[iccid] { + if bind != nil { + result = append(result, bind) + } + } + } + return result +} + +func activeBindCountsByOrderCodes(ctx context.Context, tx gdb.TX, orderCodes []string) (map[string]int, error) { + result := make(map[string]int) + if len(orderCodes) == 0 { + return result, nil + } + var rows []struct { + OrderCode string `orm:"order_code"` + Count int `orm:"count"` + } + err := tx.Ctx(ctx).Model(cardBindPkgTable). + Fields("order_code", "COUNT(1) AS count"). + WhereIn("order_code", orderCodes). + WhereIn("effective_status", []int{bindStatusPending, bindStatusEffective}). + Group("order_code"). + Scan(&rows) + if err != nil { + return nil, err + } + for _, row := range rows { + if row.OrderCode != "" { + result[row.OrderCode] = row.Count + } + } + return result, nil +} + +func updateOrderItemExtra(ctx context.Context, tx gdb.TX, orderCode, batchNo string, quantity int, now *gtime.Time, userID int64) error { + var item orderItemQuery + if err := tx.Ctx(ctx).Model(orderItemTable). + Where("order_sn", orderCode). + Limit(1). + Scan(&item); err != nil { + return gerror.Wrap(err, "query cancel order item") + } + if item.ItemId == 0 { + return nil + } + extra := decodeOrderItemExtra(item.ExtraSkuData) + extra.LastCancelBatchNo = batchNo + extra.LastCancelTime = formatTime(now) + extra.CancelledQuantity = quantity + payload, err := json.Marshal(extra) + if err != nil { + return gerror.Wrap(err, "encode cancel order item extra") + } + if _, err = tx.Ctx(ctx).Model(orderItemTable). + Where("item_id", item.ItemId). + Data(orderItemUpdate{ExtraSkuData: string(payload), UpdatedBy: userID}). + Update(); err != nil { + return gerror.Wrap(err, "update cancel order item extra") + } + return nil +} + +func decodeOrderItemExtra(raw string) orderItemExtra { + extra := orderItemExtra{} + raw = strings.TrimSpace(raw) + if raw == "" { + return extra + } + _ = json.Unmarshal([]byte(raw), &extra) + return extra +} + +func insertCancellationCards(ctx context.Context, tx gdb.TX, task *cancelservice.Task, cards map[string]*entity.TSimCardInfo, now *gtime.Time) error { + rows := make([]do.TSimCardCancellation, 0, len(cards)) + for _, detail := range task.Details { + if detail == nil { + continue + } + card := cards[detail.ICCID] + if card == nil { + continue + } + rows = append(rows, cancellationRow(task, card, now)) + } + if len(rows) == 0 { + return gerror.New("no cancellation rows to insert") + } + if _, err := tx.Ctx(ctx).Model(dao.TSimCardCancellation.Table()).Data(rows).Insert(); err != nil { + return gerror.Wrap(err, "insert cancellation cards") + } + return nil +} + +func cancellationRow(task *cancelservice.Task, card *entity.TSimCardInfo, now *gtime.Time) do.TSimCardCancellation { + return do.TSimCardCancellation{ + Iccid: card.Iccid, + Msisdn: card.Msisdn, + Imsi: card.Imsi, + Imei: card.Imei, + RegionCode: card.RegionCode, + DeptId: card.DeptId, + CurrentPackageCode: card.CurrentPackageCode, + PackageStatus: card.PackageStatus, + AccessType: card.AccessType, + BossCardStatus: consts.SimBossCardStatusCanceled, + CancelBatchNo: task.Batch.BatchNo, + CancelledAt: now, + CardType: card.CardType, + BossOpenAt: card.BossOpenAt, + BossActivateAt: card.BossActivateAt, + IsBossActivateEnabled: card.IsBossActivateEnabled, + InboundBatchNo: card.InboundBatchNo, + OutboundBatchNo: card.OutboundBatchNo, + AllocationBatchNo: card.AllocationBatchNo, + IsFlowPool: card.IsFlowPool, + FlowPoolCode: card.FlowPoolCode, + FlowPkgBeginAt: card.FlowPkgBeginAt, + FlowPkgPresentAt: card.FlowPkgPresentAt, + FlowPkgRealPresentAt: card.FlowPkgRealPresentAt, + FlowPkgEndAt: card.FlowPkgEndAt, + DataServiceStatus: card.DataServiceStatus, + PackageTotalFlowMb: card.PackageTotalFlowMb, + UsedFlowMb: card.UsedFlowMb, + TrafficCutoffStatus: card.TrafficCutoffStatus, + TrafficCutoffThreshold: card.TrafficCutoffThreshold, + SmsServiceStatus: card.SmsServiceStatus, + PackageTotalSms: card.PackageTotalSms, + UsedSms: card.UsedSms, + VoiceServiceStatus: card.VoiceServiceStatus, + PackageTotalVoice: card.PackageTotalVoice, + UsedVoice: card.UsedVoice, + RoamServiceStatus: card.RoamServiceStatus, + PackageTotalRoamFlowMb: card.PackageTotalRoamFlowMb, + UsedRoamFlowMb: card.UsedRoamFlowMb, + StatusSyncTime: card.StatusSyncTime, + FlowSyncTime: card.FlowSyncTime, + SmsSyncTime: card.SmsSyncTime, + VoiceSyncTime: card.VoiceSyncTime, + RoamSyncTime: card.RoamSyncTime, + ExtraFeeEnabled: card.ExtraFeeEnabled, + ExtraFeeMultiple: card.ExtraFeeMultiple, + ExtraFee: card.ExtraFee, + ChanKey: card.ChanKey, + NetModel: card.NetModel, + Remark: card.Remark, + TenantId: card.TenantId, + CreatedDept: card.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + } +} + +func deleteFormalCards(ctx context.Context, tx gdb.TX, task *cancelservice.Task, cards map[string]*entity.TSimCardInfo) error { + ids := make([]int64, 0, len(cards)) + for _, detail := range task.Details { + if detail == nil { + continue + } + if card := cards[detail.ICCID]; card != nil { + ids = append(ids, card.Id) + } + } + if len(ids) == 0 { + return gerror.New("no formal SIM cards to delete") + } + result, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()). + WhereIn(dao.TSimCardInfo.Columns().Id, ids). + Where(dao.TSimCardInfo.Columns().TenantId, task.Batch.TenantID). + Where(dao.TSimCardInfo.Columns().BossCardStatus, consts.SimBossCardStatusStopped). + Delete() + if err != nil { + return gerror.Wrap(err, "delete formal cancelled cards") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read formal card delete result") + } + if int(affected) != len(ids) { + return gerror.New("some formal SIM cards changed before cancellation") + } + return nil +} + +func updateDetailsSuccess(ctx context.Context, tx gdb.TX, task *cancelservice.Task) error { + for _, detail := range task.Details { + if detail == nil { + continue + } + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: consts.BatchDetailStatusSuccess, + ErrorReason: "", + Snapshot: detail.Snapshot(), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update cancel success detail") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read cancel success detail result") + } + if affected != 1 { + return gerror.Newf("cancel detail %d is no longer running", detail.ID) + } + } + return nil +} + +func updateMainSuccess(ctx context.Context, tx gdb.TX, task *cancelservice.Task, quantity int, now *gtime.Time) error { + info := buildSuccessBatchInfo(task, quantity, now) + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeSimCardCancel). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Data(do.BatchMain{ + BatchAsyncStatus: consts.BatchAsyncStatusSuccess, + TotalCount: quantity, + ValidCount: quantity, + SuccessCount: quantity, + FailCount: 0, + RunResult: consts.BatchRunResultSuccess, + FailReason: "", + BatchInfo: info, + EndTime: now, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update cancel batch main success") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read cancel batch main success result") + } + if affected != 1 { + return gerror.New("cancel batch is no longer processing") + } + task.Batch.BatchInfo = info + return nil +} + +func finalizeFailure(ctx context.Context, tx gdb.TX, task *cancelservice.Task, reason string, now *gtime.Time) error { + if err := updateDetailsFailed(ctx, tx, task, reason); err != nil { + return err + } + return updateMainFailed(ctx, tx, task, len(task.Details), reason, now) +} + +func updateDetailsFailed(ctx context.Context, tx gdb.TX, task *cancelservice.Task, fallback string) error { + for _, detail := range task.Details { + if detail == nil { + continue + } + if strings.TrimSpace(detail.Reason) == "" { + detail.Reason = fallback + } + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: consts.BatchDetailStatusFailed, + ErrorReason: detail.Reason, + Snapshot: detail.Snapshot(), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update cancel failed detail") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read cancel failed detail result") + } + if affected != 1 { + return gerror.Newf("cancel detail %d is no longer running", detail.ID) + } + } + return nil +} + +func updateMainFailed(ctx context.Context, tx gdb.TX, task *cancelservice.Task, failCount int, reason string, now *gtime.Time) error { + info := buildFailureBatchInfo(task, reason) + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeSimCardCancel). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Data(do.BatchMain{ + BatchAsyncStatus: consts.BatchAsyncStatusFailed, + TotalCount: len(task.Details), + ValidCount: len(task.Details), + SuccessCount: 0, + FailCount: failCount, + RunResult: consts.BatchRunResultFailed, + FailReason: reason, + BatchInfo: info, + EndTime: now, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update cancel batch main failed") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read cancel batch main failed result") + } + if affected != 1 { + return gerror.New("cancel batch is no longer processing") + } + task.Batch.BatchInfo = info + task.FailureReason = reason + return nil +} + +func (r *Repository) SaveResultFile(ctx context.Context, task *cancelservice.Task, request batchresult.Request) error { + artifact, err := r.artifactGenerator().Create(ctx, request) + if err != nil { + return r.updateResultFileStatus(ctx, task, "failed", 0, 0, err.Error()) + } + now := gtime.Now() + successCount := 0 + for _, detail := range task.Details { + if detail != nil && strings.TrimSpace(detail.Reason) == "" { + successCount++ + } + } + failCount := len(task.Details) - successCount + var ( + ossID int64 + downloadID int64 + ) + txErr := r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + var insertErr error + ossID, insertErr = tx.Ctx(ctx).Model(dao.SysOss.Table()).Data(do.SysOss{ + TenantId: task.Batch.TenantID, + FileName: artifact.FileName, + OriginalName: artifact.OriginalName, + FileSuffix: artifact.FileSuffix, + Path: artifact.Path, + Url: artifact.URL, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + Service: "local", + Md5: artifact.MD5, + FileSize: artifact.Size, + }).InsertAndGetId() + if insertErr != nil { + return gerror.Wrap(insertErr, "insert cancel result OSS") + } + downloadID, insertErr = tx.Ctx(ctx).Model(dao.TResourceBatchDownload.Table()).Data(do.TResourceBatchDownload{ + BatchNo: task.Batch.BatchNo, + TaskName: batchresult.TaskName(task.Batch.BatchType), + ResourceType: consts.ResourceTypeExcel, + ExportScene: task.Batch.BatchType, + TotalCount: len(task.Details), + SuccessCount: successCount, + FailCount: failCount, + DownloadStatus: consts.DownloadStatusGenerated, + OssId: ossID, + FileName: artifact.FileName, + FileExt: artifact.FileSuffix, + FileUrl: artifact.URL, + FilePath: artifact.Path, + FileSize: artifact.Size, + FileMd5: artifact.MD5, + DownloadCount: 0, + MaxDownloadCount: 0, + DeptId: task.Batch.CreatedDept, + StartTime: task.Batch.StartTime, + FinishTime: now, + TenantId: task.Batch.TenantID, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + }).InsertAndGetId() + if insertErr != nil { + return gerror.Wrap(insertErr, "insert cancel result download") + } + return updateBatchInfoOnly(ctx, tx, task, buildResultBatchInfo(task, "success", ossID, downloadID, "")) + }) + if txErr != nil { + batchresult.RemoveLocalFile(artifact) + return r.updateResultFileStatus(ctx, task, "failed", 0, 0, txErr.Error()) + } + return nil +} + +func (r *Repository) artifactGenerator() batchresult.Generator { + if r.resultFiles != nil { + return r.resultFiles + } + return batchresult.New() +} + +func (r *Repository) updateResultFileStatus(ctx context.Context, task *cancelservice.Task, status string, ossID, downloadID int64, reason string) error { + return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + return updateBatchInfoOnly(ctx, tx, task, buildResultBatchInfo(task, status, ossID, downloadID, reason)) + }) +} + +func updateBatchInfoOnly(ctx context.Context, tx gdb.TX, task *cancelservice.Task, info *gjson.Json) error { + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeSimCardCancel). + Data(do.BatchMain{ + BatchInfo: info, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update cancel result file status") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read cancel result file update result") + } + if affected != 1 { + return gerror.New("cancel batch disappeared while saving result file") + } + task.Batch.BatchInfo = info + return nil +} + +func buildSuccessBatchInfo(task *cancelservice.Task, quantity int, now *gtime.Time) *gjson.Json { + info := cloneBatchInfo(task) + info["operation"] = "cancel" + info["mode"] = "internal" + info["requiresStopped"] = true + info["targetBossCardStatus"] = consts.SimBossCardStatusCanceled + info["quantity"] = quantity + info["cancelTime"] = formatTime(now) + return gjson.New(info) +} + +func buildFailureBatchInfo(task *cancelservice.Task, reason string) *gjson.Json { + info := cloneBatchInfo(task) + info["operation"] = "cancel" + info["mode"] = "internal" + info["requiresStopped"] = true + info["targetBossCardStatus"] = consts.SimBossCardStatusCanceled + if reason != "" { + info["failureReason"] = reason + } + return gjson.New(info) +} + +func buildResultBatchInfo(task *cancelservice.Task, status string, ossID, downloadID int64, reason string) *gjson.Json { + info := cloneBatchInfo(task) + resultFile := map[string]any{"status": status} + if ossID > 0 { + resultFile["ossId"] = ossID + } + if downloadID > 0 { + resultFile["downloadId"] = downloadID + } + if reason != "" { + resultFile["error"] = reason + } + info["resultFile"] = resultFile + return gjson.New(info) +} + +func cloneBatchInfo(task *cancelservice.Task) map[string]any { + info := make(map[string]any) + if task != nil && task.Batch.BatchInfo != nil { + for key, value := range task.Batch.BatchInfo.Map() { + info[key] = value + } + } + return info +} + +func uniqueSortedICCID(details []*cancelservice.Detail) []string { + seen := make(map[string]struct{}, len(details)) + result := make([]string, 0, len(details)) + for _, detail := range details { + if detail == nil { + continue + } + iccid := strings.TrimSpace(detail.ICCID) + if iccid == "" { + continue + } + if _, ok := seen[iccid]; ok { + continue + } + seen[iccid] = struct{}{} + result = append(result, iccid) + } + sort.Strings(result) + return result +} + +func appendRemark(base, addition string) string { + base = strings.TrimSpace(base) + addition = strings.TrimSpace(addition) + if base == "" { + return addition + } + if addition == "" { + return base + } + return base + "; " + addition +} + +func formatTime(t *gtime.Time) string { + if t == nil { + return "" + } + return t.Format("Y-m-d H:i:s") +} + +var _ cancelservice.Repository = (*Repository)(nil) diff --git a/internal/repository/data/repository.go b/internal/repository/data/repository.go new file mode 100644 index 0000000..89886da --- /dev/null +++ b/internal/repository/data/repository.go @@ -0,0 +1,514 @@ +package data + +import ( + "context" + "strconv" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" + data "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/data" +) + +type Repository struct { + db gdb.DB + resultFiles batchresult.Generator +} + +func New() *Repository { + return &Repository{resultFiles: batchresult.New()} +} + +func (r *Repository) database() gdb.DB { + if r.db != nil { + return r.db + } + return g.DB() +} + +func (r *Repository) Load(ctx context.Context, batchNo string) (*data.Task, error) { + if batchNo == "" { + return nil, gerror.New("data service batch number is empty") + } + batch := entity.BatchMain{} + if err := r.database().Model(dao.BatchMain.Table()).Safe().Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + WhereIn(dao.BatchMain.Columns().BatchType, []int{consts.BatchTypeDataCut, consts.BatchTypeDataStart, consts.BatchTypeDataStop}). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Scan(&batch); err != nil { + return nil, gerror.Wrap(err, "query data service batch") + } + if batch.Id == 0 { + return nil, gerror.New("processing data service batch does not exist") + } + operation, err := data.OperationByBatchType(batch.BatchType) + if err != nil { + return nil, err + } + + var batchDetails []*entity.BatchDetail + if err = r.database().Model(dao.BatchDetail.Table()).Safe().Ctx(ctx). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().BatchType, batch.BatchType). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + OrderAsc(dao.BatchDetail.Columns().RowNo). + Scan(&batchDetails); err != nil { + return nil, gerror.Wrap(err, "query running data service details") + } + if len(batchDetails) == 0 { + return nil, gerror.New("data service batch has no running details") + } + + task := &data.Task{ + Batch: data.Batch{ + ID: batch.Id, + BatchNo: batch.BatchNo, + BatchType: batch.BatchType, + TenantID: batch.TenantId, + CreatedBy: batch.CreatedBy, + CreatedDept: batch.CreatedDept, + StartTime: batch.StartTime, + BatchInfo: batch.BatchInfo, + }, + Operation: operation, + ApnName: apnName(batch.BatchInfo), + Details: make([]*data.Detail, 0, len(batchDetails)), + } + for _, batchDetail := range batchDetails { + if batchDetail == nil { + continue + } + detail := &data.Detail{ + ID: batchDetail.Id, + RowNo: batchDetail.RowNo, + ICCID: strings.TrimSpace(batchDetail.ItemId), + Reason: strings.TrimSpace(batchDetail.ErrorReason), + } + if batchDetail.Snapshot != nil { + if value := strings.TrimSpace(batchDetail.Snapshot.Get("ICCID").String()); value != "" { + detail.ICCID = value + } + detail.InputMSISDN = strings.TrimSpace(batchDetail.Snapshot.Get("MSISDN").String()) + detail.InputIMSI = strings.TrimSpace(batchDetail.Snapshot.Get("IMSI").String()) + detail.MSISDN = detail.InputMSISDN + detail.IMSI = detail.InputIMSI + detail.Authorized = batchDetail.Snapshot.Get("authorized").Bool() + detail.ChanKey = strings.TrimSpace(batchDetail.Snapshot.Get("chanKey").String()) + detail.TrafficCutoffCrossMonthHold = batchDetail.Snapshot.Get("trafficCutoffCrossMonthHold").Int() + } + task.Details = append(task.Details, detail) + } + if len(task.Details) == 0 { + return nil, gerror.New("data service batch has no readable running details") + } + if err = r.loadCards(ctx, task); err != nil { + return nil, err + } + return task, nil +} + +func apnName(info *gjson.Json) string { + if info != nil { + if value := strings.TrimSpace(info.Get("apnName").String()); value != "" { + return value + } + } + return consts.SimApnNameDefault +} + +func (r *Repository) loadCards(ctx context.Context, task *data.Task) error { + iccids := make([]string, 0, len(task.Details)) + for _, detail := range task.Details { + if detail != nil && detail.ICCID != "" { + iccids = append(iccids, detail.ICCID) + } + } + if len(iccids) == 0 { + return nil + } + var cards []*entity.TSimCardInfo + if err := r.database().Model(dao.TSimCardInfo.Table()).Safe().Ctx(ctx). + Where(dao.TSimCardInfo.Columns().TenantId, task.Batch.TenantID). + WhereIn(dao.TSimCardInfo.Columns().Iccid, iccids). + Scan(&cards); err != nil { + return gerror.Wrap(err, "query data service SIM cards") + } + byICCID := make(map[string]*entity.TSimCardInfo, len(cards)) + for _, card := range cards { + if card != nil { + byICCID[strings.TrimSpace(card.Iccid)] = card + } + } + for _, detail := range task.Details { + card := byICCID[detail.ICCID] + if card == nil { + continue + } + detail.Card = &data.Card{ + ID: card.Id, + ICCID: strings.TrimSpace(card.Iccid), + MSISDN: strings.TrimSpace(card.Msisdn), + IMSI: strings.TrimSpace(card.Imsi), + ChanKey: preferredDataServiceChanKey(dataAppBatchUsesSnapshotChannel(task.Batch.BatchInfo), detail.ChanKey, card.ChanKey), + DataServiceStatus: card.DataServiceStatus, + } + } + return nil +} + +func preferredDataServiceChanKey(useSnapshot bool, snapshotChanKey string, cardChanKey string) string { + if useSnapshot { + if value := strings.TrimSpace(snapshotChanKey); value != "" { + return value + } + } + return strings.TrimSpace(cardChanKey) +} + +func dataAppBatchUsesSnapshotChannel(info *gjson.Json) bool { + if info == nil { + return false + } + return strings.EqualFold(strings.TrimSpace(info.Get("source").String()), "app") +} + +func (r *Repository) RefreshDataServiceState(ctx context.Context, task *data.Task) error { + if task == nil || len(task.Details) == 0 { + return nil + } + iccids := make([]string, 0, len(task.Details)) + for _, detail := range task.Details { + if detail != nil && detail.ICCID != "" { + iccids = append(iccids, detail.ICCID) + } + } + if len(iccids) == 0 { + return nil + } + type row struct { + ICCID string `orm:"iccid"` + DataServiceStatus int `orm:"data_service_status"` + } + var cards []row + if err := r.database().Model(dao.TSimCardInfo.Table()).Safe().Ctx(ctx). + Fields(dao.TSimCardInfo.Columns().Iccid, dao.TSimCardInfo.Columns().DataServiceStatus). + Where(dao.TSimCardInfo.Columns().TenantId, task.Batch.TenantID). + WhereIn(dao.TSimCardInfo.Columns().Iccid, iccids). + Scan(&cards); err != nil { + return gerror.Wrap(err, "refresh data service card state") + } + byICCID := make(map[string]int, len(cards)) + for _, card := range cards { + byICCID[strings.TrimSpace(card.ICCID)] = card.DataServiceStatus + } + for _, detail := range task.Details { + if detail == nil || detail.Card == nil { + continue + } + if status, ok := byICCID[detail.ICCID]; ok { + detail.Card.DataServiceStatus = status + detail.StatusBefore = status + detail.StatusAfter = status + } + } + return nil +} + +func (r *Repository) SaveValidationFailure(ctx context.Context, task *data.Task) error { + return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + for _, detail := range task.Details { + if detail == nil { + continue + } + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: consts.BatchDetailStatusFailed, + ErrorReason: detail.Reason, + Snapshot: detail.Snapshot(task.Operation, task.ApnName), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return err + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.Newf("data service detail %d is no longer running", detail.ID) + } + } + return nil + }) +} + +func (r *Repository) SaveResult(ctx context.Context, task *data.Task, detail *data.Detail, outcome data.Outcome) (data.Outcome, error) { + if detail == nil { + return data.Outcome{}, gerror.New("data service detail is nil") + } + resolved := outcome + err := r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + if resolved.Success { + if detail.Card == nil { + resolved.Success = false + resolved.Message = "SIM card record is missing" + } else { + result, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()). + Where(dao.TSimCardInfo.Columns().Id, detail.Card.ID). + Where(dao.TSimCardInfo.Columns().TenantId, task.Batch.TenantID). + Data(dataServiceUpdateData(task, detail)).Update() + if err != nil { + return err + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.Newf("data service SIM card %d is missing or changed", detail.Card.ID) + } + detail.StatusAfter = task.Operation.TargetStatus + } + } + detail.ApplyOutcome(resolved) + detailStatus := consts.BatchDetailStatusFailed + if resolved.Success { + detailStatus = consts.BatchDetailStatusSuccess + } + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: detailStatus, + ErrorCode: strconv.Itoa(resolved.Code), + ErrorReason: detail.Reason, + Snapshot: detail.Snapshot(task.Operation, task.ApnName), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return err + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.Newf("data service detail %d is no longer running", detail.ID) + } + return nil + }) + return resolved, err +} + +func dataServiceUpdateData(task *data.Task, detail *data.Detail) do.TSimCardTrafficCutoffConfig { + now := gtime.Now() + result := do.TSimCardTrafficCutoffConfig{ + DataServiceStatus: task.Operation.TargetStatus, + StatusSyncTime: now, + UpdatedBy: task.Batch.CreatedBy, + } + switch task.Batch.BatchType { + case consts.BatchTypeDataCut: + if detail != nil && detail.TrafficCutoffCrossMonthHold == 1 { + result.TrafficCutoffTriggeredAt = now + } + case consts.BatchTypeDataStart: + result.TrafficCutoffTriggeredAt = gdb.Raw("NULL") + } + return result +} + +func (r *Repository) Finalize(ctx context.Context, task *data.Task) error { + successCount := 0 + for _, detail := range task.Details { + if detail != nil && detail.Reason == "" { + successCount++ + } + } + failCount := len(task.Details) - successCount + batchAsyncStatus, runResult := batchResult(successCount, failCount) + if task.FailureReason == "" && failCount > 0 { + task.FailureReason = strconv.Itoa(failCount) + " SIM cards failed" + } + validCount := 0 + if task.ValidationOK { + validCount = validDetailCount(task) + } + artifact, err := r.artifactGenerator().Create(ctx, artifactRequest(task)) + if err != nil { + return gerror.Wrap(err, "generate data service result file") + } + now := gtime.Now() + err = r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + ossID, err := tx.Ctx(ctx).Model(dao.SysOss.Table()).Data(do.SysOss{ + TenantId: task.Batch.TenantID, + FileName: artifact.FileName, + OriginalName: artifact.OriginalName, + FileSuffix: artifact.FileSuffix, + Path: artifact.Path, + Url: artifact.URL, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + Service: "local", + Md5: artifact.MD5, + FileSize: artifact.Size, + }).InsertAndGetId() + if err != nil { + return gerror.Wrap(err, "insert data service result OSS") + } + downloadID, err := tx.Ctx(ctx).Model(dao.TResourceBatchDownload.Table()).Data(do.TResourceBatchDownload{ + BatchNo: task.Batch.BatchNo, + TaskName: batchresult.TaskName(task.Batch.BatchType), + ResourceType: consts.ResourceTypeExcel, + ExportScene: task.Batch.BatchType, + TotalCount: len(task.Details), + SuccessCount: successCount, + FailCount: failCount, + DownloadStatus: consts.DownloadStatusGenerated, + OssId: ossID, + FileName: artifact.FileName, + FileExt: artifact.FileSuffix, + FileUrl: artifact.URL, + FilePath: artifact.Path, + FileSize: artifact.Size, + FileMd5: artifact.MD5, + DownloadCount: 0, + MaxDownloadCount: 0, + DeptId: task.Batch.CreatedDept, + StartTime: task.Batch.StartTime, + FinishTime: now, + TenantId: task.Batch.TenantID, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + }).InsertAndGetId() + if err != nil { + return gerror.Wrap(err, "insert data service result download") + } + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Data(do.BatchMain{ + BatchAsyncStatus: batchAsyncStatus, + ValidCount: validCount, + SuccessCount: successCount, + FailCount: failCount, + RunResult: runResult, + FailReason: task.FailureReason, + BatchInfo: buildBatchInfo(task, ossID, downloadID), + EndTime: now, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return err + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.New("data service batch is no longer processing") + } + return nil + }) + if err != nil { + batchresult.RemoveLocalFile(artifact) + return err + } + return nil +} + +func validDetailCount(task *data.Task) int { + if task == nil { + return 0 + } + count := 0 + for _, detail := range task.Details { + if detail != nil && detail.ValidationPassed { + count++ + } + } + return count +} + +func (r *Repository) artifactGenerator() batchresult.Generator { + if r.resultFiles != nil { + return r.resultFiles + } + return batchresult.New() +} + +func artifactRequest(task *data.Task) batchresult.Request { + rows := make([]batchresult.Row, 0, len(task.Details)) + for _, detail := range task.Details { + if detail == nil { + continue + } + rows = append(rows, batchresult.Row{ + RowNo: detail.RowNo, + ICCID: detail.ICCID, + MSISDN: detail.MSISDN, + IMSI: detail.IMSI, + Operation: operationLabel(task.Operation), + ApnName: task.ApnName, + Success: detail.Reason == "", + Reason: detail.Reason, + UpstreamCode: detail.UpstreamCode, + OrderNum: detail.OrderNum, + }) + } + return batchresult.Request{BatchNo: task.Batch.BatchNo, BatchType: task.Batch.BatchType, TenantID: task.Batch.TenantID, Rows: rows} +} + +func operationLabel(operation data.Operation) string { + switch operation.BatchType { + case consts.BatchTypeDataCut: + return "达量断网" + case consts.BatchTypeDataStart: + return "开数据" + case consts.BatchTypeDataStop: + return "停数据" + default: + return operation.Name + } +} + +func buildBatchInfo(task *data.Task, ossID, downloadID int64) *gjson.Json { + info := make(map[string]any) + if task.Batch.BatchInfo != nil { + for key, value := range task.Batch.BatchInfo.Map() { + info[key] = value + } + } + info["operation"] = task.Operation.Name + info["operType"] = task.Operation.ApnOperType + info["apnName"] = task.ApnName + info["resultFileOssId"] = ossID + info["resultDownloadId"] = downloadID + return gjson.New(info) +} + +func batchResult(successCount, failCount int) (int, int) { + switch { + case failCount == 0: + return consts.BatchAsyncStatusSuccess, consts.BatchRunResultSuccess + case successCount == 0: + return consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed + default: + return consts.BatchAsyncStatusPartialSuccess, consts.BatchRunResultPartialSuccess + } +} diff --git a/internal/repository/data/repository_test.go b/internal/repository/data/repository_test.go new file mode 100644 index 0000000..58967f9 --- /dev/null +++ b/internal/repository/data/repository_test.go @@ -0,0 +1,134 @@ +package data + +import ( + "context" + "testing" + + _ "github.com/gogf/gf/contrib/drivers/sqlite/v2" + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + dataService "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/data" +) + +func TestPreferredDataServiceChanKey(t *testing.T) { + tests := []struct { + name string + appBatch bool + snapshot string + card string + want string + }{ + {name: "app resolved snapshot wins", appBatch: true, snapshot: " legacy-71 ", card: "stale-1", want: "legacy-71"}, + {name: "console keeps card channel", snapshot: "legacy-71", card: " card-112 ", want: "card-112"}, + {name: "card fallback", card: " card-112 ", want: "card-112"}, + {name: "empty", want: ""}, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + if got := preferredDataServiceChanKey(test.appBatch, test.snapshot, test.card); got != test.want { + t.Fatalf("preferredDataServiceChanKey() = %q, want %q", got, test.want) + } + }) + } +} + +func TestDataAppBatchUsesSnapshotChannel(t *testing.T) { + if !dataAppBatchUsesSnapshotChannel(gjson.New(map[string]any{"source": "APP"})) { + t.Fatal("app batch should use the resolved snapshot channel") + } + if dataAppBatchUsesSnapshotChannel(gjson.New(map[string]any{"source": "console"})) { + t.Fatal("console batch must continue using the current card channel") + } +} + +func TestSaveResultDoesNotClearTrafficCutoffFields(t *testing.T) { + ctx := context.Background() + db, err := gdb.New(gdb.ConfigNode{Type: "sqlite", Link: "sqlite::@file(:memory:)"}) + if err != nil { + t.Fatalf("create sqlite database: %v", err) + } + statements := []string{ + "CREATE TABLE t_sim_card_info (id INTEGER PRIMARY KEY, tenant_id TEXT NOT NULL, data_service_status INTEGER NOT NULL DEFAULT 1, status_sync_time DATETIME, traffic_cutoff_status INTEGER NOT NULL DEFAULT 0, traffic_cutoff_threshold REAL NOT NULL DEFAULT 0, traffic_cutoff_cross_month_hold INTEGER NOT NULL DEFAULT 0, traffic_cutoff_triggered_at DATETIME, updated_by INTEGER NOT NULL DEFAULT 0, updated_at DATETIME)", + "CREATE TABLE batch_detail (id INTEGER PRIMARY KEY, batch_no TEXT NOT NULL, detail_status INTEGER NOT NULL, error_code TEXT, error_reason TEXT, snapshot TEXT, updated_by INTEGER NOT NULL DEFAULT 0, updated_at DATETIME)", + "INSERT INTO t_sim_card_info (id, tenant_id, data_service_status, traffic_cutoff_status, traffic_cutoff_threshold, traffic_cutoff_cross_month_hold, updated_by) VALUES (1, '000000', 1, 1, 512, 1, 7)", + "INSERT INTO batch_detail (id, batch_no, detail_status) VALUES (9, 'DST-TEST', 1)", + } + for _, statement := range statements { + if _, err = db.Exec(ctx, statement); err != nil { + t.Fatalf("execute setup SQL: %v", err) + } + } + + repo := &Repository{db: db} + task := &dataService.Task{ + Batch: dataService.Batch{ + BatchNo: "DST-TEST", + BatchType: consts.BatchTypeDataStop, + TenantID: "000000", + CreatedBy: 13, + }, + Operation: dataService.Operation{ + BatchType: consts.BatchTypeDataStop, + TargetStatus: consts.SimDataServiceStatusDisabled, + }, + } + detail := &dataService.Detail{ + ID: 9, + Card: &dataService.Card{ID: 1, DataServiceStatus: consts.SimDataServiceStatusEnabled}, + } + if _, err = repo.SaveResult(ctx, task, detail, dataService.Outcome{Success: true}); err != nil { + t.Fatalf("SaveResult() error = %v", err) + } + + row, err := db.GetOne(ctx, "SELECT data_service_status, traffic_cutoff_status, traffic_cutoff_threshold, traffic_cutoff_cross_month_hold FROM t_sim_card_info WHERE id = 1") + if err != nil { + t.Fatalf("query card result: %v", err) + } + if got := row["data_service_status"].Int(); got != consts.SimDataServiceStatusDisabled { + t.Fatalf("data_service_status = %d", got) + } + if got := row["traffic_cutoff_status"].Int(); got != 1 { + t.Fatalf("traffic_cutoff_status = %d, want 1", got) + } + if got := row["traffic_cutoff_threshold"].Float64(); got != 512 { + t.Fatalf("traffic_cutoff_threshold = %v, want 512", got) + } + if got := row["traffic_cutoff_cross_month_hold"].Int(); got != 1 { + t.Fatalf("traffic_cutoff_cross_month_hold = %d, want 1", got) + } +} + +func TestDataServiceUpdateDataTracksOnlyCrossMonthCutoff(t *testing.T) { + tests := []struct { + name string + batchType int + hold int + wantNil bool + }{ + {name: "cutoff with hold", batchType: consts.BatchTypeDataCut, hold: 1, wantNil: false}, + {name: "cutoff without hold", batchType: consts.BatchTypeDataCut, hold: 0, wantNil: true}, + {name: "manual stop", batchType: consts.BatchTypeDataStop, hold: 1, wantNil: true}, + {name: "data start clears trigger", batchType: consts.BatchTypeDataStart, hold: 1, wantNil: false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + task := &dataService.Task{ + Batch: dataService.Batch{BatchType: tt.batchType, CreatedBy: 7}, + Operation: dataService.Operation{TargetStatus: consts.SimDataServiceStatusDisabled}, + } + detail := &dataService.Detail{TrafficCutoffCrossMonthHold: tt.hold} + update := dataServiceUpdateData(task, detail) + if gotNil := update.TrafficCutoffTriggeredAt == nil; gotNil != tt.wantNil { + t.Fatalf("TrafficCutoffTriggeredAt nil = %v, want %v", gotNil, tt.wantNil) + } + if update.DataServiceStatus != task.Operation.TargetStatus { + t.Fatalf("DataServiceStatus = %v, want %d", update.DataServiceStatus, task.Operation.TargetStatus) + } + if update.StatusSyncTime == nil { + t.Fatal("StatusSyncTime must be set") + } + }) + } +} diff --git a/internal/repository/imei/repository.go b/internal/repository/imei/repository.go new file mode 100644 index 0000000..127aefa --- /dev/null +++ b/internal/repository/imei/repository.go @@ -0,0 +1,133 @@ +package imei + +import ( + "context" + "fmt" + "regexp" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" +) + +// imeiValuePattern bounds what may be written into t_sim_card_info.imei. +// +// The column is varchar(32) and the carrier value is digits only (15-17 for a +// real IMEI). This pattern is the primary defence for the CASE WHEN statement +// below, which is assembled as SQL text and therefore cannot rely on bound +// parameters for the value itself. +// +// The separator is intentionally absent: this service never concatenates +// multiple IMEIs (see the handler's multi-value note), so allowing ';' would +// widen the column contract for a case that cannot occur. +var imeiValuePattern = regexp.MustCompile(`^[0-9]{1,32}$`) + +// maxImeiSyncBatch bounds one ApplyBatch statement. The caller (the ZSet +// consumer) claims in batches of claimBatch (default 100), so this is a guard +// rather than a working limit: it keeps a single UPDATE statement from growing +// without bound if a future caller raises the claim size. +const maxImeiSyncBatch = 1000 + +// Result is the database projection of one IMEI sync outcome. +// +// Two update kinds are represented by one struct: +// +// - Synced: the carrier answered successfully (Code == 0). imei_sync_time is +// written because the card was covered by this polling pass. +// - ValueChanged: the carrier returned a non-empty IMEI that differs from the +// stored value. Only then are imei and imei_update_time written. +// +// A Synced-but-empty result (carrier answered, no IMEI available) is therefore +// a heartbeat-only update: it records coverage without touching the value. That +// asymmetry is what makes the whole chain "additive": an upstream outage that +// returns empty values can never clear a stored IMEI. +type Result struct { + CardID int64 + Synced bool + ValueChanged bool + Imei string +} + +type Repository struct{} + +func NewRepository() *Repository { return &Repository{} } + +// ApplyBatch writes a batch of IMEI sync results in as few statements as +// possible. +// +// Two mutually exclusive groups are emitted, because they update different +// column sets and mixing them in one CASE expression would force every card to +// write every column: +// +// 1. heartbeat only (synced, value unchanged or empty) → imei_sync_time +// 2. value changed → imei + imei_update_time + imei_sync_time +// +// A statement is only issued when its group is non-empty, so an all-empty +// upstream pass still records coverage (group 1) without writing any value. +func (r *Repository) ApplyBatch(ctx context.Context, results []Result) error { + if len(results) == 0 { + return nil + } + if len(results) > maxImeiSyncBatch { + return fmt.Errorf("imei sync batch too large: %d > %d", len(results), maxImeiSyncBatch) + } + + columns := dao.TSimCardInfo.Columns() + heartbeatIDs := make([]int64, 0, len(results)) + changedIDs := make([]int64, 0, len(results)) + var changedCase strings.Builder + changedCase.WriteString("CASE " + columns.Id + " ") + + for _, result := range results { + if result.CardID <= 0 { + return fmt.Errorf("imei result has invalid cardId %d", result.CardID) + } + if !result.Synced { + // Only successfully synchronized cards get a heartbeat; a failed + // query says nothing about coverage. + continue + } + if !result.ValueChanged { + heartbeatIDs = append(heartbeatIDs, result.CardID) + continue + } + if !imeiValuePattern.MatchString(result.Imei) { + // Reject the whole batch rather than write a value that violates + // the column contract. The caller logs the message; the card is + // retried in a later round. + return fmt.Errorf("imei result for card %d has invalid value %q", result.CardID, result.Imei) + } + fmt.Fprintf(&changedCase, "WHEN %d THEN '%s' ", result.CardID, result.Imei) + changedIDs = append(changedIDs, result.CardID) + } + + now := gtime.Now() + // Value changes first: if the second statement fails, the value and its + // change timestamp are already consistent, and the heartbeat is only a + // coverage marker that the next pass will re-write anyway. + if len(changedIDs) > 0 { + changedCase.WriteString("END") + if _, err := dao.TSimCardInfo.Ctx(ctx). + WhereIn(columns.Id, changedIDs). + Data(do.TSimCardInfo{ + Imei: gdb.Raw(changedCase.String()), + ImeiUpdateTime: now, + ImeiSyncTime: now, + }).Update(); err != nil { + return err + } + } + if len(heartbeatIDs) > 0 { + if _, err := dao.TSimCardInfo.Ctx(ctx). + WhereIn(columns.Id, heartbeatIDs). + Data(do.TSimCardInfo{ + ImeiSyncTime: now, + }).Update(); err != nil { + return err + } + } + return nil +} diff --git a/internal/repository/imei/repository_test.go b/internal/repository/imei/repository_test.go new file mode 100644 index 0000000..7ac8a96 --- /dev/null +++ b/internal/repository/imei/repository_test.go @@ -0,0 +1,281 @@ +package imei + +import ( + "context" + "strings" + "testing" + + _ "github.com/gogf/gf/contrib/drivers/sqlite/v2" + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +// setupDB builds a minimal card table mirroring the columns ApplyBatch writes. +func setupDB(t *testing.T, ctx context.Context) { + t.Helper() + if err := gdb.SetConfig(gdb.Config{"default": gdb.ConfigGroup{{Type: "sqlite", Link: "sqlite::@file(:memory:)", MaxOpenConnCount: 1}}}); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = g.DB().Close(ctx) }) + if err := g.DB().GetCore().ClearTableFieldsAll(ctx); err != nil { + t.Fatal(err) + } + if _, err := g.DB().Exec(ctx, `CREATE TABLE t_sim_card_info ( + id INTEGER PRIMARY KEY, + imei TEXT, + imei_sync_time DATETIME, + imei_update_time DATETIME + )`); err != nil { + t.Fatal(err) + } + if _, err := g.DB().Exec(ctx, `INSERT INTO t_sim_card_info (id, imei, imei_sync_time, imei_update_time) VALUES + (1,'OLD-IMEI-1',NULL,NULL), + (2,'OLD-IMEI-2',NULL,NULL), + (3,'OLD-IMEI-3',NULL,NULL), + (4,'',NULL,NULL)`); err != nil { + t.Fatal(err) + } +} + +// readCard returns the stored imei and both timestamps for one card. +func readCard(t *testing.T, ctx context.Context, id int64) (imei string, syncTime, updateTime *string) { + t.Helper() + var row struct { + Imei string `orm:"imei"` + ImeiSyncTime *string `orm:"imei_sync_time"` + ImeiUpdateTime *string `orm:"imei_update_time"` + } + if err := g.DB().Model("t_sim_card_info").Where("id", id).Scan(&row); err != nil { + t.Fatal(err) + } + return row.Imei, row.ImeiSyncTime, row.ImeiUpdateTime +} + +// TestApplyBatchValueChangeWritesAllThreeColumns 值变更时三列同时写。 +func TestApplyBatchValueChangeWritesAllThreeColumns(t *testing.T) { + ctx := context.Background() + setupDB(t, ctx) + + repo := NewRepository() + if err := repo.ApplyBatch(ctx, []Result{ + {CardID: 1, Synced: true, ValueChanged: true, Imei: "865466070817587"}, + }); err != nil { + t.Fatalf("ApplyBatch: %v", err) + } + + imei, syncTime, updateTime := readCard(t, ctx, 1) + if imei != "865466070817587" { + t.Fatalf("imei = %q", imei) + } + if syncTime == nil || updateTime == nil { + t.Fatalf("值变更应同时写两个时间戳: sync=%v update=%v", syncTime, updateTime) + } +} + +// TestApplyBatchHeartbeatDoesNotTouchValue 心跳只写 imei_sync_time， +// 绝不改动 imei 与 imei_update_time。这是 §5.2 的核心保证。 +func TestApplyBatchHeartbeatDoesNotTouchValue(t *testing.T) { + ctx := context.Background() + setupDB(t, ctx) + + repo := NewRepository() + if err := repo.ApplyBatch(ctx, []Result{ + {CardID: 1, Synced: true, ValueChanged: false}, + }); err != nil { + t.Fatalf("ApplyBatch: %v", err) + } + + imei, syncTime, updateTime := readCard(t, ctx, 1) + if imei != "OLD-IMEI-1" { + t.Fatalf("心跳不得改动 imei: %q", imei) + } + if syncTime == nil { + t.Fatal("心跳应写 imei_sync_time") + } + if updateTime != nil { + t.Fatalf("心跳不得写 imei_update_time: %v", *updateTime) + } +} + +// TestApplyBatchUnsyncedIsSkipped 未同步（查询失败）的卡完全不入库。 +func TestApplyBatchUnsyncedIsSkipped(t *testing.T) { + ctx := context.Background() + setupDB(t, ctx) + + repo := NewRepository() + if err := repo.ApplyBatch(ctx, []Result{ + {CardID: 1, Synced: false}, + {CardID: 2, Synced: false, ValueChanged: true, Imei: "999999999999999"}, + }); err != nil { + t.Fatalf("ApplyBatch: %v", err) + } + + for _, id := range []int64{1, 2} { + imei, syncTime, updateTime := readCard(t, ctx, id) + if syncTime != nil || updateTime != nil { + t.Fatalf("card %d: 未同步不应写时间戳: sync=%v update=%v", id, syncTime, updateTime) + } + _ = imei + } +} + +// TestApplyBatchMixedGroups 混合批次：值变更与心跳分组写入，互不干扰。 +func TestApplyBatchMixedGroups(t *testing.T) { + ctx := context.Background() + setupDB(t, ctx) + + repo := NewRepository() + if err := repo.ApplyBatch(ctx, []Result{ + {CardID: 1, Synced: true, ValueChanged: true, Imei: "865466070817587"}, // 值变更 + {CardID: 2, Synced: true, ValueChanged: false}, // 心跳 + {CardID: 3, Synced: false}, // 失败跳过 + {CardID: 4, Synced: true, ValueChanged: true, Imei: "111222333444555"}, // 从空到有值 + }); err != nil { + t.Fatalf("ApplyBatch: %v", err) + } + + if imei, syncTime, updateTime := readCard(t, ctx, 1); imei != "865466070817587" || syncTime == nil || updateTime == nil { + t.Fatalf("card 1 应值变更: imei=%q sync=%v update=%v", imei, syncTime, updateTime) + } + if imei, syncTime, updateTime := readCard(t, ctx, 2); imei != "OLD-IMEI-2" || syncTime == nil || updateTime != nil { + t.Fatalf("card 2 应仅心跳: imei=%q sync=%v update=%v", imei, syncTime, updateTime) + } + if imei, syncTime, updateTime := readCard(t, ctx, 3); imei != "OLD-IMEI-3" || syncTime != nil || updateTime != nil { + t.Fatalf("card 3 应完全不动: imei=%q sync=%v update=%v", imei, syncTime, updateTime) + } + if imei, syncTime, updateTime := readCard(t, ctx, 4); imei != "111222333444555" || syncTime == nil || updateTime == nil { + t.Fatalf("card 4 应从空到有值: imei=%q sync=%v update=%v", imei, syncTime, updateTime) + } +} + +// TestApplyBatchRejectsInvalidImeiValue 非数字值必须被拒绝， +// 因为 CASE WHEN 是拼接 SQL，值本身没有绑定参数保护。 +func TestApplyBatchRejectsInvalidImeiValue(t *testing.T) { + ctx := context.Background() + setupDB(t, ctx) + + repo := NewRepository() + malicious := []string{ + "865466070817587'", // 引号逃逸 + "865466070817587'; DROP TABLE x; --", // 注入尝试 + "865466070817587 OR 1=1", // 逻辑注入 + "abc", // 非数字 + "865466070817587;999999999999999", // 分号拼接（本链路不支持多值） + strings.Repeat("9", 33), // 超长 + } + for _, value := range malicious { + err := repo.ApplyBatch(ctx, []Result{ + {CardID: 1, Synced: true, ValueChanged: true, Imei: value}, + }) + if err == nil { + t.Fatalf("非法值 %q 应被拒绝", value) + } + } + + // 拒绝后数据必须保持原样 + if imei, _, _ := readCard(t, ctx, 1); imei != "OLD-IMEI-1" { + t.Fatalf("被拒绝的批次不得改动数据: %q", imei) + } +} + +// TestApplyBatchAcceptsBoundaryLengths 15/17 位正常 IMEI 与列宽边界值应通过。 +func TestApplyBatchAcceptsBoundaryLengths(t *testing.T) { + ctx := context.Background() + setupDB(t, ctx) + + repo := NewRepository() + values := []string{ + "865466070817587", // 15 位 + "86546607081758712", // 17 位 + strings.Repeat("9", 32), // varchar(32) 上限 + } + for i, value := range values { + cardID := int64(i + 1) + if err := repo.ApplyBatch(ctx, []Result{ + {CardID: cardID, Synced: true, ValueChanged: true, Imei: value}, + }); err != nil { + t.Fatalf("合法值 %q 应被接受: %v", value, err) + } + if imei, _, _ := readCard(t, ctx, cardID); imei != value { + t.Fatalf("card %d: imei = %q, want %q", cardID, imei, value) + } + } +} + +// TestApplyBatchRejectsInvalidCardID 非法卡 ID 拒绝整批。 +func TestApplyBatchRejectsInvalidCardID(t *testing.T) { + ctx := context.Background() + setupDB(t, ctx) + + repo := NewRepository() + if err := repo.ApplyBatch(ctx, []Result{ + {CardID: 1, Synced: true, ValueChanged: true, Imei: "865466070817587"}, + {CardID: 0, Synced: true, ValueChanged: true, Imei: "111222333444555"}, + }); err == nil { + t.Fatal("非法 cardId 应被拒绝") + } +} + +// TestApplyBatchRejectsOversizedBatch 超大批次拒绝，防止单条 SQL 无限增长。 +func TestApplyBatchRejectsOversizedBatch(t *testing.T) { + ctx := context.Background() + setupDB(t, ctx) + + repo := NewRepository() + results := make([]Result, 0, maxImeiSyncBatch+1) + for i := 0; i <= maxImeiSyncBatch; i++ { + results = append(results, Result{CardID: int64(i + 1), Synced: true}) + } + if err := repo.ApplyBatch(ctx, results); err == nil { + t.Fatal("超大批次应被拒绝") + } +} + +// TestApplyBatchEmptyIsNoop 空批次不发 SQL。 +func TestApplyBatchEmptyIsNoop(t *testing.T) { + ctx := context.Background() + setupDB(t, ctx) + + repo := NewRepository() + if err := repo.ApplyBatch(ctx, nil); err != nil { + t.Fatalf("空批次应无操作: %v", err) + } + if err := repo.ApplyBatch(ctx, []Result{}); err != nil { + t.Fatalf("空切片应无操作: %v", err) + } +} + +// TestApplyBatchAllUnsyncedIssuesNoSQL 全部未同步时不应发任何 UPDATE。 +func TestApplyBatchAllUnsyncedIssuesNoSQL(t *testing.T) { + ctx := context.Background() + setupDB(t, ctx) + + repo := NewRepository() + if err := repo.ApplyBatch(ctx, []Result{ + {CardID: 1, Synced: false}, + {CardID: 2, Synced: false}, + }); err != nil { + t.Fatalf("ApplyBatch: %v", err) + } + for _, id := range []int64{1, 2} { + if _, syncTime, updateTime := readCard(t, ctx, id); syncTime != nil || updateTime != nil { + t.Fatalf("card %d 不应被改动", id) + } + } +} + +// TestImeiValuePattern documents the accepted value grammar. +func TestImeiValuePattern(t *testing.T) { + valid := []string{"1", "865466070817587", strings.Repeat("9", 32)} + invalid := []string{"", "abc", "865466070817587'", "865-466", " 865466070817587", strings.Repeat("9", 33), "865466070817587;1"} + for _, value := range valid { + if !imeiValuePattern.MatchString(value) { + t.Fatalf("%q 应被接受", value) + } + } + for _, value := range invalid { + if imeiValuePattern.MatchString(value) { + t.Fatalf("%q 应被拒绝", value) + } + } +} diff --git a/internal/repository/inbound/repository.go b/internal/repository/inbound/repository.go new file mode 100644 index 0000000..3e7661a --- /dev/null +++ b/internal/repository/inbound/repository.go @@ -0,0 +1,865 @@ +package inbound + +import ( + "context" + "os" + "path/filepath" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" + inbound "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/inbound" +) + +const stockTable = "t_sim_card_stock" +const ( + inboundStockQueryBatchSizeConfigKey = "batch.inbound.stockQueryBatchSize" + inboundWriteBatchSizeConfigKey = "batch.inbound.writeBatchSize" + defaultInboundStockQueryBatchSize = 1000 + defaultInboundWriteBatchSize = 1000 +) + +type Repository struct { + db gdb.DB + resultFiles batchresult.Generator +} + +type stockDO struct { + g.Meta `orm:"table:t_sim_card_stock, do:true"` + BatchNo any `orm:"batch_no"` + ICCID any `orm:"iccid"` + MSISDN any `orm:"msisdn"` + IMSI any `orm:"imsi"` + IMEI any `orm:"imei"` + ChanKey any `orm:"chan_key"` + TenantID any `orm:"tenant_id"` + CreatedDept any `orm:"created_dept"` + CreatedBy any `orm:"created_by"` + UpdatedBy any `orm:"updated_by"` +} + +func New() *Repository { + return &Repository{db: dao.BatchMain.DB(), resultFiles: batchresult.New()} +} + +func (r *Repository) database() gdb.DB { + if r.db != nil { + return r.db + } + return g.DB() +} + +func (r *Repository) Load(ctx context.Context, batchNo string) (*inbound.Task, error) { + if batchNo == "" { + return nil, gerror.New("inbound batch number is empty") + } + var batch entity.BatchMain + if err := r.database().Model(dao.BatchMain.Table()).Safe().Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeInbound). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Scan(&batch); err != nil { + return nil, gerror.Wrap(err, "query inbound batch") + } + if batch.Id == 0 { + return nil, gerror.New("processing inbound batch does not exist") + } + task := &inbound.Task{ + Batch: inbound.Batch{ + ID: batch.Id, + BatchNo: batch.BatchNo, + BatchType: batch.BatchType, + SourceType: batch.SourceType, + TenantID: batch.TenantId, + CreatedDept: batch.CreatedDept, + CreatedBy: batch.CreatedBy, + StartTime: batch.StartTime, + BatchInfo: batch.BatchInfo, + }, + } + task.Source = parseSource(batch.BatchInfo, batch.SourceOssId, batch.SourceType) + task.Options = parseOptions(batch.BatchInfo) + return task, nil +} + +func (r *Repository) ReadOssFile(ctx context.Context, ossID int64) ([]byte, string, error) { + if ossID <= 0 { + return nil, "", gerror.New("oss id is empty") + } + var oss entity.SysOss + if err := r.database().Model(dao.SysOss.Table()).Safe().Ctx(ctx). + Where(dao.SysOss.Columns().OssId, ossID). + Scan(&oss); err != nil { + return nil, "", gerror.Wrap(err, "query OSS file") + } + if oss.OssId == 0 { + return nil, "", gerror.Newf("OSS file %d does not exist", ossID) + } + if oss.Service != "local" { + return nil, "", gerror.Newf("OSS file %d is not a local file: service=%q", ossID, oss.Service) + } + uploadRoot := g.Cfg().MustGet(ctx, "system.upload.local.path").String() + localPath, attempts, err := resolveReadableLocalOssPath(uploadRoot, oss.TenantId, oss.Path) + if err != nil { + return nil, fileDisplayName(oss, ""), gerror.Wrapf(err, "resolve OSS file %d local path from %q, attempted paths: %s", ossID, oss.Path, strings.Join(attempts, "; ")) + } + content, err := os.ReadFile(localPath) + if err != nil { + return nil, fileDisplayName(oss, localPath), gerror.Wrapf(err, "OSS file %d local path is unreadable: original path=%q, resolved path=%q", ossID, oss.Path, localPath) + } + return content, fileDisplayName(oss, localPath), nil +} + +func resolveLocalOssPath(uploadRoot, tenantID, storedPath string) (string, error) { + uploadRoot = strings.TrimSpace(uploadRoot) + if uploadRoot == "" { + return "", gerror.New("system.upload.local.path is empty") + } + tenantID = strings.TrimSpace(tenantID) + if tenantID == "" { + tenantID = "default" + } + storedPath = strings.TrimSpace(storedPath) + if storedPath == "" { + return "", gerror.New("OSS path is empty") + } + + objectSegments, err := ossObjectSegments(tenantID, storedPath) + if err != nil { + return "", err + } + objectPath := filepath.Join(objectSegments...) + localPath := filepath.Join(uploadRoot, objectPath) + relativePath, err := filepath.Rel(uploadRoot, localPath) + if err != nil { + return "", gerror.Wrap(err, "resolve local OSS path") + } + if relativePath == ".." || strings.HasPrefix(relativePath, ".."+string(filepath.Separator)) { + return "", gerror.New("resolved OSS path escapes upload root") + } + return localPath, nil +} + +func resolveReadableLocalOssPath(uploadRoot, tenantID, storedPath string) (string, []string, error) { + var attempts []string + var firstErr error + for _, root := range uploadRootCandidates(uploadRoot) { + localPath, err := resolveLocalOssPath(root, tenantID, storedPath) + if err != nil { + if firstErr == nil { + firstErr = err + } + continue + } + attempts = appendUniqueString(attempts, localPath) + if isReadableFile(localPath) { + return localPath, attempts, nil + } + } + + if _, err := ossObjectSegments(strings.TrimSpace(defaultTenant(tenantID)), storedPath); err != nil { + if firstErr == nil { + firstErr = err + } + } else { + for _, candidate := range storedPathCandidates(storedPath) { + attempts = appendUniqueString(attempts, candidate) + if isReadableFile(candidate) { + return candidate, attempts, nil + } + } + } + if firstErr != nil && len(attempts) == 0 { + return "", attempts, firstErr + } + return "", attempts, gerror.New("OSS local file does not exist") +} + +func uploadRootCandidates(uploadRoot string) []string { + var candidates []string + for _, root := range []string{ + os.Getenv("IOTLCM_WORKER_UPLOAD_ROOT"), + os.Getenv("IOTLCM_UPLOAD_ROOT"), + uploadRoot, + } { + root = strings.TrimSpace(root) + if root == "" { + continue + } + candidates = appendUploadRootCandidate(candidates, root) + } + return candidates +} + +func appendUploadRootCandidate(candidates []string, root string) []string { + for _, candidate := range uploadRootEquivalentCandidates(root) { + if filepath.IsAbs(candidate) { + candidates = appendUniqueString(candidates, candidate) + } else { + for _, base := range localBaseDirs() { + candidates = appendUniqueString(candidates, filepath.Join(base, candidate)) + } + candidates = appendUniqueString(candidates, candidate) + } + } + return candidates +} + +func uploadRootEquivalentCandidates(root string) []string { + root = strings.TrimSpace(root) + if root == "" { + return nil + } + candidates := []string{root} + normalized := filepath.ToSlash(filepath.Clean(root)) + if normalized == "resource/upload" || strings.HasSuffix(normalized, "/resource/upload") { + prefix := strings.TrimSuffix(normalized, "resource/upload") + candidates = appendUniqueString(candidates, filepath.FromSlash(prefix+"upload")) + } + return candidates +} + +func storedPathCandidates(storedPath string) []string { + storedPath = strings.TrimSpace(storedPath) + if storedPath == "" { + return nil + } + var candidates []string + candidates = appendUniqueString(candidates, storedPath) + if !filepath.IsAbs(storedPath) { + for _, base := range localBaseDirs() { + candidates = appendUniqueString(candidates, filepath.Join(base, storedPath)) + } + } + return candidates +} + +func localBaseDirs() []string { + var dirs []string + if wd, err := os.Getwd(); err == nil && strings.TrimSpace(wd) != "" { + dirs = appendUniqueString(dirs, wd) + } + if exe, err := os.Executable(); err == nil && strings.TrimSpace(exe) != "" { + dirs = appendUniqueString(dirs, filepath.Dir(exe)) + } + return dirs +} + +func ossObjectSegments(tenantID, storedPath string) ([]string, error) { + tenantID = defaultTenant(tenantID) + segments := strings.FieldsFunc(strings.ReplaceAll(storedPath, "\\", "/"), func(r rune) bool { + return r == '/' + }) + for index, segment := range segments { + if segment != tenantID { + continue + } + objectSegments := segments[index:] + for _, objectSegment := range objectSegments { + if objectSegment == "." || objectSegment == ".." { + return nil, gerror.Newf("invalid OSS object path segment %q", objectSegment) + } + } + return objectSegments, nil + } + return nil, gerror.Newf("OSS path does not contain tenant %q", tenantID) +} + +func defaultTenant(tenantID string) string { + tenantID = strings.TrimSpace(tenantID) + if tenantID == "" { + return "default" + } + return tenantID +} + +func isReadableFile(path string) bool { + info, err := os.Stat(path) + return err == nil && !info.IsDir() +} + +func appendUniqueString(values []string, value string) []string { + value = strings.TrimSpace(value) + if value == "" { + return values + } + for _, existing := range values { + if existing == value { + return values + } + } + return append(values, value) +} + +func uniqueStrings(values []string) []string { + if len(values) == 0 { + return nil + } + uniq := make([]string, 0, len(values)) + seen := make(map[string]struct{}, len(values)) + for _, value := range values { + value = strings.TrimSpace(value) + if value == "" { + continue + } + if _, ok := seen[value]; ok { + continue + } + seen[value] = struct{}{} + uniq = append(uniq, value) + } + return uniq +} + +func (r *Repository) FindExistingStocks(ctx context.Context, _ string, iccids, msisdns, imsis []string) ([]inbound.Stock, error) { + if len(iccids) == 0 && len(msisdns) == 0 && len(imsis) == 0 { + return nil, nil + } + type queryResult struct { + stocks []inbound.Stock + err error + } + results := make(chan queryResult, 3) + queries := 0 + if len(iccids) > 0 { + queries++ + go func() { + stocks, err := r.findExistingStocksByField(ctx, "iccid", "ICCID", iccids) + results <- queryResult{stocks: stocks, err: err} + }() + } + if len(msisdns) > 0 { + queries++ + go func() { + stocks, err := r.findExistingStocksByField(ctx, "msisdn", "MSISDN", msisdns) + results <- queryResult{stocks: stocks, err: err} + }() + } + if len(imsis) > 0 { + queries++ + go func() { + stocks, err := r.findExistingStocksByField(ctx, "imsi", "IMSI", imsis) + results <- queryResult{stocks: stocks, err: err} + }() + } + stocks := make([]inbound.Stock, 0) + for i := 0; i < queries; i++ { + result := <-results + if result.err != nil { + return nil, result.err + } + stocks = append(stocks, result.stocks...) + } + return stocks, nil +} + +func (r *Repository) stockQuery(ctx context.Context) *gdb.Model { + return r.database().Model(stockTable).Safe().Ctx(ctx) +} + +func (r *Repository) findExistingStocksByField(ctx context.Context, field, label string, values []string) ([]inbound.Stock, error) { + uniq := uniqueStrings(values) + if len(uniq) == 0 { + return nil, nil + } + batchSize := inboundStockQueryBatchSize(ctx) + stocks := make([]inbound.Stock, 0) + for _, chunk := range chunkStrings(uniq, batchSize) { + var chunkStocks []inbound.Stock + model := r.stockQuery(ctx).WhereIn(field, chunk) + if err := model.Fields("iccid", "msisdn", "imsi").Scan(&chunkStocks); err != nil { + return nil, gerror.Wrapf(err, "query inbound stock %s duplicates", label) + } + stocks = append(stocks, chunkStocks...) + } + return stocks, nil +} + +func (r *Repository) SaveValidationFailure(ctx context.Context, task *inbound.Task) error { + return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + batchInfo := buildInboundBatchInfo(task) + task.Batch.BatchInfo = batchInfo + details := buildBatchDetails(task, consts.BatchDetailStatusFailed) + if err := insertChunkedRows(ctx, tx, dao.BatchDetail.Table(), "inbound detail", details, inboundWriteBatchSize(ctx)); err != nil { + return err + } + return updateBatchMain(ctx, tx, task, do.BatchMain{ + BatchAsyncStatus: consts.BatchAsyncStatusFailed, + TotalCount: len(task.Rows), + ValidCount: task.ValidCount, + SuccessCount: 0, + FailCount: len(task.Rows), + RunResult: consts.BatchRunResultFailed, + FailReason: task.FailureReason, + BatchInfo: batchInfo, + EndTime: gtime.Now(), + UpdatedBy: task.Batch.CreatedBy, + }) + }) +} + +func (r *Repository) SaveSuccess(ctx context.Context, task *inbound.Task) error { + return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + batchInfo := buildInboundBatchInfo(task) + task.Batch.BatchInfo = batchInfo + stocks := make([]stockDO, 0, len(task.Rows)) + details := make([]do.BatchDetail, 0, len(task.Rows)) + for _, row := range task.Rows { + if row == nil { + continue + } + stocks = append(stocks, stockDO{ + BatchNo: task.Batch.BatchNo, + ICCID: row.ICCID, + MSISDN: nullableString(row.MSISDN), + IMSI: nullableString(row.IMSI), + IMEI: nullableString(row.IMEI), + ChanKey: row.ChanKey, + TenantID: task.Batch.TenantID, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + }) + details = append(details, batchDetailRow(task, row, consts.BatchDetailStatusSuccess)) + } + if err := insertChunkedRows(ctx, tx, stockTable, "inbound stock", stocks, inboundWriteBatchSize(ctx)); err != nil { + return err + } + if err := insertChunkedRows(ctx, tx, dao.BatchDetail.Table(), "inbound detail", details, inboundWriteBatchSize(ctx)); err != nil { + return err + } + return updateBatchMain(ctx, tx, task, do.BatchMain{ + BatchAsyncStatus: consts.BatchAsyncStatusSuccess, + TotalCount: len(task.Rows), + ValidCount: len(task.Rows), + SuccessCount: len(task.Rows), + FailCount: 0, + RunResult: consts.BatchRunResultSuccess, + FailReason: "", + BatchInfo: batchInfo, + EndTime: gtime.Now(), + UpdatedBy: task.Batch.CreatedBy, + }) + }) +} + +func (r *Repository) SaveBusinessFailure(ctx context.Context, task *inbound.Task, reason string) error { + task.ValidCount = countRowsWithoutReason(task.Rows) + for _, row := range task.Rows { + if row != nil { + row.Reason = reason + } + } + task.FailureReason = reason + return r.SaveValidationFailure(ctx, task) +} + +func buildBatchDetails(task *inbound.Task, status int) []do.BatchDetail { + details := make([]do.BatchDetail, 0, len(task.Rows)) + for _, row := range task.Rows { + if row == nil { + continue + } + details = append(details, batchDetailRow(task, row, status)) + } + return details +} + +func batchDetailRow(task *inbound.Task, row *inbound.Row, status int) do.BatchDetail { + return do.BatchDetail{ + BatchNo: task.Batch.BatchNo, + BatchType: task.Batch.BatchType, + DetailStatus: status, + BlankOutStatus: 1, + RowNo: row.RowNo, + ItemId: row.ICCID, + ErrorReason: row.Reason, + Snapshot: rowSnapshot(row), + TenantId: task.Batch.TenantID, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + } +} + +func (r *Repository) SaveResultFile(ctx context.Context, task *inbound.Task, request batchresult.Request) error { + artifact, err := r.artifactGenerator().Create(ctx, request) + if err != nil { + return r.updateResultFileStatus(ctx, task, "failed", 0, 0, err.Error()) + } + var ( + ossID int64 + downloadID int64 + ) + err = r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + var insertErr error + ossID, insertErr = tx.Ctx(ctx).Model(dao.SysOss.Table()).Data(do.SysOss{ + TenantId: task.Batch.TenantID, + FileName: artifact.FileName, + OriginalName: artifact.OriginalName, + FileSuffix: artifact.FileSuffix, + Path: artifact.Path, + Url: artifact.URL, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + Service: "local", + Md5: artifact.MD5, + FileSize: artifact.Size, + }).InsertAndGetId() + if insertErr != nil { + return gerror.Wrap(insertErr, "insert inbound result OSS") + } + downloadID, insertErr = tx.Ctx(ctx).Model(dao.TResourceBatchDownload.Table()).Data(do.TResourceBatchDownload{ + BatchNo: task.Batch.BatchNo, + TaskName: batchresult.TaskName(task.Batch.BatchType), + ResourceType: consts.ResourceTypeExcel, + ExportScene: task.Batch.BatchType, + TotalCount: len(task.Rows), + SuccessCount: successRows(task.Rows), + FailCount: len(task.Rows) - successRows(task.Rows), + DownloadStatus: consts.DownloadStatusGenerated, + OssId: ossID, + FileName: artifact.FileName, + FileExt: artifact.FileSuffix, + FileUrl: artifact.URL, + FilePath: artifact.Path, + FileSize: artifact.Size, + FileMd5: artifact.MD5, + DownloadCount: 0, + MaxDownloadCount: 0, + DeptId: task.Batch.CreatedDept, + StartTime: task.Batch.StartTime, + FinishTime: gtime.Now(), + TenantId: task.Batch.TenantID, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + }).InsertAndGetId() + if insertErr != nil { + return gerror.Wrap(insertErr, "insert inbound result download") + } + return updateBatchInfoOnly(ctx, tx, task, buildResultBatchInfo(task, "success", ossID, downloadID, ""), ossID) + }) + if err != nil { + batchresult.RemoveLocalFile(artifact) + return r.updateResultFileStatus(ctx, task, "failed", 0, 0, err.Error()) + } + return nil +} + +func (r *Repository) artifactGenerator() batchresult.Generator { + if r.resultFiles != nil { + return r.resultFiles + } + return batchresult.New() +} + +func (r *Repository) updateResultFileStatus(ctx context.Context, task *inbound.Task, status string, ossID, downloadID int64, reason string) error { + return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + return updateBatchInfoOnly(ctx, tx, task, buildResultBatchInfo(task, status, ossID, downloadID, reason), ossID) + }) +} + +func insertChunkedRows[T any](ctx context.Context, tx gdb.TX, table, label string, rows []T, batchSize int) error { + if len(rows) == 0 { + return nil + } + if batchSize <= 0 { + batchSize = defaultInboundWriteBatchSize + } + for start := 0; start < len(rows); start += batchSize { + end := start + batchSize + if end > len(rows) { + end = len(rows) + } + if _, err := tx.Ctx(ctx).Model(table).Data(rows[start:end]).Insert(); err != nil { + return gerror.Wrapf(err, "insert %s chunk", label) + } + } + return nil +} + +func chunkStrings(values []string, batchSize int) [][]string { + if batchSize <= 0 { + batchSize = defaultInboundStockQueryBatchSize + } + if len(values) == 0 { + return nil + } + chunks := make([][]string, 0, (len(values)+batchSize-1)/batchSize) + for start := 0; start < len(values); start += batchSize { + end := start + batchSize + if end > len(values) { + end = len(values) + } + chunks = append(chunks, values[start:end]) + } + return chunks +} + +func inboundStockQueryBatchSize(ctx context.Context) int { + value := g.Cfg().MustGet(ctx, inboundStockQueryBatchSizeConfigKey, defaultInboundStockQueryBatchSize).Int() + if value <= 0 { + return defaultInboundStockQueryBatchSize + } + return value +} + +func inboundWriteBatchSize(ctx context.Context) int { + value := g.Cfg().MustGet(ctx, inboundWriteBatchSizeConfigKey, defaultInboundWriteBatchSize).Int() + if value <= 0 { + return defaultInboundWriteBatchSize + } + return value +} + +func updateBatchMain(ctx context.Context, tx gdb.TX, task *inbound.Task, data do.BatchMain) error { + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchType, task.Batch.BatchType). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Data(data). + Update() + if err != nil { + return gerror.Wrap(err, "update inbound batch main") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read inbound batch main update result") + } + if affected != 1 { + return gerror.New("inbound batch is no longer processing") + } + return nil +} + +func updateBatchInfoOnly(ctx context.Context, tx gdb.TX, task *inbound.Task, info *gjson.Json, resultOssID int64) error { + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchType, task.Batch.BatchType). + Data(do.BatchMain{ + ResultOssId: workerOssIDValue(resultOssID), + BatchInfo: info, + UpdatedBy: task.Batch.CreatedBy, + }). + Update() + if err != nil { + return gerror.Wrap(err, "update inbound result file status") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read inbound result file update result") + } + if affected != 1 { + return gerror.New("inbound batch disappeared while saving result file") + } + task.Batch.BatchInfo = info + return nil +} + +func workerOssIDValue(ossID int64) any { + if ossID <= 0 { + return nil + } + return ossID +} + +func parseSource(info *gjson.Json, sourceOssID int64, sourceType int) inbound.Source { + source := inbound.Source{ + ImportFileOssID: jsonInt64(info, "source.importFileOssId", "importFileOssId"), + InputText: jsonString(info, "source.inputText", "inputText"), + InputFileOssID: jsonInt64(info, "source.inputFileOssId", "inputFileOssId"), + Range: inbound.Range{ + RangeType: jsonString(info, "source.range.rangeType", "range.rangeType", "rangeType"), + StartICCID: jsonString(info, "source.range.startIccid", "range.startIccid", "startIccid"), + EndICCID: jsonString(info, "source.range.endIccid", "range.endIccid", "endIccid"), + StartMSISDN: jsonString(info, "source.range.startMsisdn", "range.startMsisdn", "startMsisdn"), + EndMSISDN: jsonString(info, "source.range.endMsisdn", "range.endMsisdn", "endMsisdn"), + StartIMSI: jsonString(info, "source.range.startImsi", "range.startImsi", "startImsi"), + EndIMSI: jsonString(info, "source.range.endImsi", "range.endImsi", "endImsi"), + }, + } + if sourceOssID > 0 { + switch sourceType { + case consts.BatchSourceTypeBatchInput: + source.InputFileOssID = sourceOssID + default: + source.ImportFileOssID = sourceOssID + } + } + return source +} + +func parseOptions(info *gjson.Json) inbound.Options { + return inbound.Options{ + ChanKey: jsonString(info, "inboundOptions.chanKey", "chanKey"), + ValidateUpstreamBasicInfo: jsonString(info, "inboundOptions.validateUpstreamBasicInfo", "validateUpstreamBasicInfo"), + CardType: jsonString(info, "inboundOptions.cardType", "cardType"), + UpPkgCode: jsonString(info, "inboundOptions.upPkgCode", "upPkgCode"), + SuppliersCode: jsonString(info, "inboundOptions.suppliersCode", "suppliersCode"), + UpstreamActivationTime: jsonString(info, "inboundOptions.upstreamActivationTime", "upstreamActivationTime"), + InsertToPlugEnabled: jsonString(info, "inboundOptions.insertToPlugEnabled", "insertToPlugEnabled"), + CarrierTrafficCutoffEnabled: jsonString(info, "inboundOptions.carrierTrafficCutoffEnabled", "carrierTrafficCutoffEnabled"), + AgreementExpiryDate: jsonString(info, "inboundOptions.agreementExpiryDate", "agreementExpiryDate"), + HasTestPeriod: jsonString(info, "inboundOptions.hasTestPeriod", "hasTestPeriod"), + TestPeriodDays: jsonString(info, "inboundOptions.testPeriodDays", "testPeriodDays"), + TestPeriodLatestExpiryDate: jsonString(info, "inboundOptions.testPeriodLatestExpiryDate", "testPeriodLatestExpiryDate"), + HasSilentPeriod: jsonString(info, "inboundOptions.hasSilentPeriod", "hasSilentPeriod"), + SilentPeriodDays: jsonString(info, "inboundOptions.silentPeriodDays", "silentPeriodDays"), + SilentPeriodLatestExpiryDate: jsonString(info, "inboundOptions.silentPeriodLatestExpiryDate", "silentPeriodLatestExpiryDate"), + } +} + +func jsonString(info *gjson.Json, paths ...string) string { + if info == nil { + return "" + } + for _, path := range paths { + value := strings.TrimSpace(info.Get(path).String()) + if value != "" { + return value + } + } + return "" +} + +func jsonInt64(info *gjson.Json, paths ...string) int64 { + if info == nil { + return 0 + } + for _, path := range paths { + value := info.Get(path).Int64() + if value > 0 { + return value + } + } + return 0 +} + +func rowSnapshot(row *inbound.Row) *gjson.Json { + raw := map[string]any{} + for key, value := range row.Raw { + raw[key] = value + } + return gjson.New(map[string]any{ + "rowNo": row.RowNo, + "inputCode": row.InputCode, + "ICCID": row.ICCID, + "MSISDN": row.MSISDN, + "IMSI": row.IMSI, + "IMEI": row.IMEI, + "chanKey": row.ChanKey, + "cardType": row.CardType, + "upPkgCode": row.UpPkgCode, + "suppliersCode": row.SuppliersCode, + "upstreamActivationTime": row.UpstreamActivationTime, + "insertToPlugEnabled": row.InsertToPlugEnabled, + "carrierTrafficCutoffEnabled": row.CarrierTrafficCutoffEnabled, + "agreementExpiryDate": row.AgreementExpiryDate, + "hasTestPeriod": row.HasTestPeriod, + "testPeriodDays": row.TestPeriodDays, + "testPeriodLatestExpiryDate": row.TestPeriodLatestExpiryDate, + "hasSilentPeriod": row.HasSilentPeriod, + "silentPeriodDays": row.SilentPeriodDays, + "silentPeriodLatestExpiryDate": row.SilentPeriodLatestExpiryDate, + "rawFields": raw, + }) +} + +func buildInboundBatchInfo(task *inbound.Task) *gjson.Json { + if task == nil { + return gjson.New(map[string]any{}) + } + info := make(map[string]any) + if task != nil && task.Batch.BatchInfo != nil { + for key, value := range task.Batch.BatchInfo.Map() { + info[key] = value + } + } + info["inboundOptions"] = map[string]any{ + "chanKey": task.Options.ChanKey, + "validateUpstreamBasicInfo": task.Options.ValidateUpstreamBasicInfo, + "cardType": task.Options.CardType, + "upPkgCode": task.Options.UpPkgCode, + "suppliersCode": task.Options.SuppliersCode, + "upstreamActivationTime": task.Options.UpstreamActivationTime, + "insertToPlugEnabled": task.Options.InsertToPlugEnabled, + "carrierTrafficCutoffEnabled": task.Options.CarrierTrafficCutoffEnabled, + "agreementExpiryDate": task.Options.AgreementExpiryDate, + "hasTestPeriod": task.Options.HasTestPeriod, + "testPeriodDays": task.Options.TestPeriodDays, + "testPeriodLatestExpiryDate": task.Options.TestPeriodLatestExpiryDate, + "hasSilentPeriod": task.Options.HasSilentPeriod, + "silentPeriodDays": task.Options.SilentPeriodDays, + "silentPeriodLatestExpiryDate": task.Options.SilentPeriodLatestExpiryDate, + } + return gjson.New(info) +} + +func buildResultBatchInfo(task *inbound.Task, status string, ossID, downloadID int64, reason string) *gjson.Json { + info := make(map[string]any) + if task.Batch.BatchInfo != nil { + for key, value := range task.Batch.BatchInfo.Map() { + info[key] = value + } + } + resultFile := map[string]any{"status": status} + if ossID > 0 { + resultFile["ossId"] = ossID + } + if downloadID > 0 { + resultFile["downloadId"] = downloadID + } + if reason != "" { + resultFile["error"] = reason + } + info["resultFile"] = resultFile + return gjson.New(info) +} + +func fileDisplayName(oss entity.SysOss, fallback string) string { + for _, value := range []string{oss.OriginalName, oss.FileName, fallback} { + if strings.TrimSpace(value) != "" { + return strings.TrimSpace(value) + } + } + return "unknown" +} + +func nullableString(value string) any { + value = strings.TrimSpace(value) + if value == "" { + return nil + } + return value +} + +func countRowsWithoutReason(rows []*inbound.Row) int { + count := 0 + for _, row := range rows { + if row != nil && row.Reason == "" { + count++ + } + } + return count +} + +func successRows(rows []*inbound.Row) int { + count := 0 + for _, row := range rows { + if row != nil && row.Reason == "" { + count++ + } + } + return count +} diff --git a/internal/repository/inbound/repository_test.go b/internal/repository/inbound/repository_test.go new file mode 100644 index 0000000..c213006 --- /dev/null +++ b/internal/repository/inbound/repository_test.go @@ -0,0 +1,227 @@ +package inbound + +import ( + "os" + "path/filepath" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + serviceinbound "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/inbound" + + "github.com/gogf/gf/v2/encoding/gjson" +) + +func TestResolveLocalOssPath(t *testing.T) { + uploadRoot := t.TempDir() + expectedPath := filepath.Join(uploadRoot, "000000", "user", "file", "20260721", "source.xlsx") + + tests := []struct { + name string + tenantID string + storedPath string + wantPath string + wantErr bool + }{ + { + name: "relative Admin path", + tenantID: "000000", + storedPath: "resource/upload/000000/user/file/20260721/source.xlsx", + wantPath: expectedPath, + }, + { + name: "absolute Admin path", + tenantID: "000000", + storedPath: "/app/upload/000000/user/file/20260721/source.xlsx", + wantPath: expectedPath, + }, + { + name: "default tenant", + tenantID: "", + storedPath: "resource/upload/default/user/file/20260721/source.xlsx", + wantPath: filepath.Join(uploadRoot, "default", "user", "file", "20260721", "source.xlsx"), + }, + { + name: "tenant is missing", + tenantID: "000000", + storedPath: "resource/upload/000001/user/file/20260721/source.xlsx", + wantErr: true, + }, + { + name: "path traversal is rejected", + tenantID: "000000", + storedPath: "resource/upload/000000/../secret.xlsx", + wantErr: true, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + gotPath, err := resolveLocalOssPath(uploadRoot, tt.tenantID, tt.storedPath) + if tt.wantErr { + if err == nil { + t.Fatal("expected an error") + } + return + } + if err != nil { + t.Fatalf("resolveLocalOssPath() error = %v", err) + } + if gotPath != tt.wantPath { + t.Fatalf("resolveLocalOssPath() = %q, want %q", gotPath, tt.wantPath) + } + }) + } +} + +func TestResolveLocalOssPathReturnsReadableFile(t *testing.T) { + uploadRoot := t.TempDir() + storedPath := "resource/upload/000000/user/file/20260721/source.xlsx" + localPath, err := resolveLocalOssPath(uploadRoot, "000000", storedPath) + if err != nil { + t.Fatalf("resolveLocalOssPath() error = %v", err) + } + if err = os.MkdirAll(filepath.Dir(localPath), 0o755); err != nil { + t.Fatalf("create directory: %v", err) + } + if err = os.WriteFile(localPath, []byte("inbound source"), 0o600); err != nil { + t.Fatalf("write source file: %v", err) + } + content, err := os.ReadFile(localPath) + if err != nil { + t.Fatalf("read source file: %v", err) + } + if string(content) != "inbound source" { + t.Fatalf("unexpected source content: %q", content) + } +} + +func TestUploadRootCandidatesIncludeMountedUploadDirectory(t *testing.T) { + wd := t.TempDir() + oldWd, err := os.Getwd() + if err != nil { + t.Fatalf("get working directory: %v", err) + } + if err = os.Chdir(wd); err != nil { + t.Fatalf("change working directory: %v", err) + } + defer func() { + _ = os.Chdir(oldWd) + }() + + candidates := uploadRootCandidates("resource/upload") + want := filepath.Join(wd, "upload") + for _, candidate := range candidates { + if filepath.Clean(candidate) == filepath.Clean(want) { + return + } + } + t.Fatalf("uploadRootCandidates() = %#v, want %q", candidates, want) +} + +func TestResolveReadableLocalOssPathUsesMountedUploadDirectory(t *testing.T) { + wd := t.TempDir() + oldWd, err := os.Getwd() + if err != nil { + t.Fatalf("get working directory: %v", err) + } + if err = os.Chdir(wd); err != nil { + t.Fatalf("change working directory: %v", err) + } + defer func() { + _ = os.Chdir(oldWd) + }() + + storedPath := "resource/upload/000000/user/file/20260813/sim-card-inbound/import/source.xlsx" + mountedPath := filepath.Join(wd, "upload", "000000", "user", "file", "20260813", "sim-card-inbound", "import", "source.xlsx") + if err = os.MkdirAll(filepath.Dir(mountedPath), 0o755); err != nil { + t.Fatalf("create mounted upload directory: %v", err) + } + if err = os.WriteFile(mountedPath, []byte("xlsx"), 0o600); err != nil { + t.Fatalf("write mounted upload file: %v", err) + } + + gotPath, attempts, err := resolveReadableLocalOssPath("resource/upload", "000000", storedPath) + if err != nil { + t.Fatalf("resolveReadableLocalOssPath() error = %v, attempts = %#v", err, attempts) + } + if gotPath != mountedPath { + t.Fatalf("resolveReadableLocalOssPath() = %q, want %q, attempts = %#v", gotPath, mountedPath, attempts) + } +} + +func TestBuildResultBatchInfoUsesGroupedResultFile(t *testing.T) { + task := &serviceinbound.Task{ + Batch: serviceinbound.Batch{ + BatchInfo: gjson.New(map[string]any{ + "source": map[string]any{"inputText": "8986"}, + }), + }, + } + + info := buildResultBatchInfo(task, "failed", 0, 0, "disk full") + if info.Get("resultFile.status").String() != "failed" { + t.Fatalf("unexpected result file status: %s", info.Get("resultFile.status").String()) + } + if info.Get("resultFile.error").String() != "disk full" { + t.Fatalf("unexpected result file error: %s", info.Get("resultFile.error").String()) + } + if info.Get("resultFileOssId").Int64() != 0 || info.Get("resultDownloadId").Int64() != 0 { + t.Fatalf("result file metadata should stay grouped: %s", info.MustToJsonString()) + } +} + +func TestParseAndBuildInboundOptionsPreserveUpstreamValidationFlag(t *testing.T) { + info := gjson.New(map[string]any{ + "inboundOptions": map[string]any{ + "chanKey": "chan-a", + "validateUpstreamBasicInfo": 1, + }, + }) + + options := parseOptions(info) + if options.ValidateUpstreamBasicInfo != "1" { + t.Fatalf("ValidateUpstreamBasicInfo = %q, want 1", options.ValidateUpstreamBasicInfo) + } + + task := &serviceinbound.Task{ + Batch: serviceinbound.Batch{BatchInfo: info}, + Options: options, + } + built := buildInboundBatchInfo(task) + if built.Get("inboundOptions.validateUpstreamBasicInfo").String() != "1" { + t.Fatalf("validate option not preserved: %s", built.MustToJsonString()) + } + if built.Get("inboundOptions.chanKey").String() != "chan-a" { + t.Fatalf("chanKey not preserved: %s", built.MustToJsonString()) + } +} + +func TestParseSourcePrefersSourceOssId(t *testing.T) { + tests := []struct { + name string + sourceType int + wantImportID int64 + wantInputID int64 + }{ + {name: "file source", sourceType: consts.BatchSourceTypeFile, wantImportID: 88, wantInputID: 22}, + {name: "input source", sourceType: consts.BatchSourceTypeBatchInput, wantImportID: 11, wantInputID: 88}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + info := gjson.New(map[string]any{ + "source": map[string]any{ + "importFileOssId": int64(11), + "inputFileOssId": int64(22), + }, + }) + source := parseSource(info, 88, tt.sourceType) + if source.ImportFileOssID != tt.wantImportID { + t.Fatalf("ImportFileOssID = %d, want %d", source.ImportFileOssID, tt.wantImportID) + } + if source.InputFileOssID != tt.wantInputID { + t.Fatalf("InputFileOssID = %d, want %d", source.InputFileOssID, tt.wantInputID) + } + }) + } +} diff --git a/internal/repository/lifecycle/channel_test.go b/internal/repository/lifecycle/channel_test.go new file mode 100644 index 0000000..1daba23 --- /dev/null +++ b/internal/repository/lifecycle/channel_test.go @@ -0,0 +1,38 @@ +package lifecycle + +import ( + "testing" + + "github.com/gogf/gf/v2/encoding/gjson" +) + +func TestPreferredLifecycleChanKey(t *testing.T) { + tests := []struct { + name string + appBatch bool + snapshot string + card string + want string + }{ + {name: "app resolved snapshot wins", appBatch: true, snapshot: " legacy-129 ", card: "stale-1", want: "legacy-129"}, + {name: "console keeps card channel", snapshot: "legacy-129", card: " card-112 ", want: "card-112"}, + {name: "card fallback", card: " card-112 ", want: "card-112"}, + {name: "empty", want: ""}, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + if got := preferredLifecycleChanKey(test.appBatch, test.snapshot, test.card); got != test.want { + t.Fatalf("preferredLifecycleChanKey() = %q, want %q", got, test.want) + } + }) + } +} + +func TestAppBatchUsesSnapshotChannel(t *testing.T) { + if !appBatchUsesSnapshotChannel(gjson.New(map[string]any{"source": "APP"})) { + t.Fatal("app batch should use the resolved snapshot channel") + } + if appBatchUsesSnapshotChannel(gjson.New(map[string]any{"source": "console"})) { + t.Fatal("console batch must continue using the current card channel") + } +} diff --git a/internal/repository/lifecycle/repository.go b/internal/repository/lifecycle/repository.go new file mode 100644 index 0000000..f38c496 --- /dev/null +++ b/internal/repository/lifecycle/repository.go @@ -0,0 +1,432 @@ +package lifecycle + +import ( + "context" + "strconv" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" + lifecycle "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/lifecycle" +) + +type Repository struct { + db gdb.DB + resultFiles batchresult.Generator +} + +func New() *Repository { + return &Repository{resultFiles: batchresult.New()} +} + +func (r *Repository) database() gdb.DB { + if r.db != nil { + return r.db + } + return g.DB() +} + +func (r *Repository) Load(ctx context.Context, batchNo string) (*lifecycle.Task, error) { + if batchNo == "" { + return nil, gerror.New("lifecycle batch number is empty") + } + batch := entity.BatchMain{} + if err := r.database().Model(dao.BatchMain.Table()).Safe().Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + WhereIn(dao.BatchMain.Columns().BatchType, []int{consts.BatchTypeSimCardStart, consts.BatchTypeSimCardStop}). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Scan(&batch); err != nil { + return nil, gerror.Wrap(err, "query lifecycle batch") + } + if batch.Id == 0 { + return nil, gerror.New("processing lifecycle batch does not exist") + } + operation, err := lifecycle.OperationByBatchType(batch.BatchType) + if err != nil { + return nil, err + } + + var batchDetails []*entity.BatchDetail + if err = r.database().Model(dao.BatchDetail.Table()).Safe().Ctx(ctx). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().BatchType, batch.BatchType). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + OrderAsc(dao.BatchDetail.Columns().RowNo). + Scan(&batchDetails); err != nil { + return nil, gerror.Wrap(err, "query running lifecycle details") + } + if len(batchDetails) == 0 { + return nil, gerror.New("lifecycle batch has no running details") + } + + task := &lifecycle.Task{ + Batch: lifecycle.Batch{ + ID: batch.Id, + BatchNo: batch.BatchNo, + BatchType: batch.BatchType, + TenantID: batch.TenantId, + CreatedBy: batch.CreatedBy, + CreatedDept: batch.CreatedDept, + StartTime: batch.StartTime, + BatchInfo: batch.BatchInfo, + }, + Operation: operation, + Details: make([]*lifecycle.Detail, 0, len(batchDetails)), + } + for _, batchDetail := range batchDetails { + if batchDetail == nil { + continue + } + detail := &lifecycle.Detail{ + ID: batchDetail.Id, + RowNo: batchDetail.RowNo, + ICCID: strings.TrimSpace(batchDetail.ItemId), + Reason: strings.TrimSpace(batchDetail.ErrorReason), + OriginalSnapshot: batchDetail.Snapshot, + } + if batchDetail.Snapshot != nil { + if value := strings.TrimSpace(batchDetail.Snapshot.Get("ICCID").String()); value != "" { + detail.ICCID = value + } + detail.InputMSISDN = strings.TrimSpace(batchDetail.Snapshot.Get("MSISDN").String()) + detail.InputIMSI = strings.TrimSpace(batchDetail.Snapshot.Get("IMSI").String()) + detail.MSISDN = detail.InputMSISDN + detail.IMSI = detail.InputIMSI + detail.Authorized = batchDetail.Snapshot.Get("authorized").Bool() + detail.ChanKey = strings.TrimSpace(batchDetail.Snapshot.Get("chanKey").String()) + } + task.Details = append(task.Details, detail) + } + if len(task.Details) == 0 { + return nil, gerror.New("lifecycle batch has no readable running details") + } + if err = r.loadCards(ctx, task); err != nil { + return nil, err + } + return task, nil +} + +func (r *Repository) loadCards(ctx context.Context, task *lifecycle.Task) error { + iccids := make([]string, 0, len(task.Details)) + for _, detail := range task.Details { + if detail != nil && detail.ICCID != "" { + iccids = append(iccids, detail.ICCID) + } + } + if len(iccids) == 0 { + return nil + } + var cards []*entity.TSimCardInfo + if err := r.database().Model(dao.TSimCardInfo.Table()).Safe().Ctx(ctx). + Where(dao.TSimCardInfo.Columns().TenantId, task.Batch.TenantID). + WhereIn(dao.TSimCardInfo.Columns().Iccid, iccids). + Scan(&cards); err != nil { + return gerror.Wrap(err, "query lifecycle SIM cards") + } + byICCID := make(map[string]*entity.TSimCardInfo, len(cards)) + for _, card := range cards { + if card != nil { + byICCID[strings.TrimSpace(card.Iccid)] = card + } + } + for _, detail := range task.Details { + card := byICCID[detail.ICCID] + if card == nil { + continue + } + detail.Card = &lifecycle.Card{ + ID: card.Id, + ICCID: strings.TrimSpace(card.Iccid), + MSISDN: strings.TrimSpace(card.Msisdn), + IMSI: strings.TrimSpace(card.Imsi), + ChanKey: preferredLifecycleChanKey(appBatchUsesSnapshotChannel(task.Batch.BatchInfo), detail.ChanKey, card.ChanKey), + BossCardStatus: card.BossCardStatus, + } + } + return nil +} + +func preferredLifecycleChanKey(useSnapshot bool, snapshotChanKey string, cardChanKey string) string { + if useSnapshot { + if value := strings.TrimSpace(snapshotChanKey); value != "" { + return value + } + } + return strings.TrimSpace(cardChanKey) +} + +func appBatchUsesSnapshotChannel(info *gjson.Json) bool { + if info == nil { + return false + } + return strings.EqualFold(strings.TrimSpace(info.Get("source").String()), "app") +} + +func (r *Repository) SaveValidationFailure(ctx context.Context, task *lifecycle.Task) error { + return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + for _, detail := range task.Details { + if detail == nil { + continue + } + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: consts.BatchDetailStatusFailed, + ErrorReason: detail.Reason, + Snapshot: detail.Snapshot(), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return err + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.Newf("lifecycle detail %d is no longer running", detail.ID) + } + } + return nil + }) +} + +func (r *Repository) SaveResult(ctx context.Context, task *lifecycle.Task, detail *lifecycle.Detail, outcome lifecycle.Outcome) (lifecycle.Outcome, error) { + if detail == nil { + return lifecycle.Outcome{}, gerror.New("lifecycle detail is nil") + } + resolved := outcome + err := r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + if resolved.Success { + if detail.Card == nil { + resolved.Success = false + resolved.Message = "SIM card record is missing" + } else { + result, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()). + Where(dao.TSimCardInfo.Columns().Id, detail.Card.ID). + Where(dao.TSimCardInfo.Columns().TenantId, task.Batch.TenantID). + Where(dao.TSimCardInfo.Columns().BossCardStatus, detail.StatusBefore). + Data(do.TSimCardInfo{ + BossCardStatus: task.Operation.TargetStatus, + StatusSyncTime: gtime.Now(), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return err + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + resolved.Success = false + resolved.Message = "SIM card local status changed" + } + } + } + detail.ApplyOutcome(resolved) + detailStatus := consts.BatchDetailStatusFailed + if resolved.Success { + detailStatus = consts.BatchDetailStatusSuccess + } + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: detailStatus, + ErrorCode: strconv.Itoa(resolved.Code), + ErrorReason: detail.Reason, + Snapshot: detail.Snapshot(), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return err + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.Newf("lifecycle detail %d is no longer running", detail.ID) + } + return nil + }) + return resolved, err +} + +func (r *Repository) Finalize(ctx context.Context, task *lifecycle.Task) error { + successCount := 0 + for _, detail := range task.Details { + if detail != nil && detail.Reason == "" { + successCount++ + } + } + failCount := len(task.Details) - successCount + batchAsyncStatus, runResult := batchResult(successCount, failCount) + if task.FailureReason == "" && failCount > 0 { + task.FailureReason = strconv.Itoa(failCount) + " SIM cards failed" + } + validCount := 0 + if task.ValidationOK { + validCount = len(task.Details) + } + artifact, err := r.artifactGenerator().Create(ctx, artifactRequest(task)) + if err != nil { + return gerror.Wrap(err, "generate lifecycle result file") + } + now := gtime.Now() + err = r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + ossID, err := tx.Ctx(ctx).Model(dao.SysOss.Table()).Data(do.SysOss{ + TenantId: task.Batch.TenantID, + FileName: artifact.FileName, + OriginalName: artifact.OriginalName, + FileSuffix: artifact.FileSuffix, + Path: artifact.Path, + Url: artifact.URL, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + Service: "local", + Md5: artifact.MD5, + FileSize: artifact.Size, + }).InsertAndGetId() + if err != nil { + return gerror.Wrap(err, "insert lifecycle result OSS") + } + downloadID, err := tx.Ctx(ctx).Model(dao.TResourceBatchDownload.Table()).Data(do.TResourceBatchDownload{ + BatchNo: task.Batch.BatchNo, + TaskName: batchresult.TaskName(task.Batch.BatchType), + ResourceType: consts.ResourceTypeExcel, + ExportScene: task.Batch.BatchType, + TotalCount: len(task.Details), + SuccessCount: successCount, + FailCount: failCount, + DownloadStatus: consts.DownloadStatusGenerated, + OssId: ossID, + FileName: artifact.FileName, + FileExt: artifact.FileSuffix, + FileUrl: artifact.URL, + FilePath: artifact.Path, + FileSize: artifact.Size, + FileMd5: artifact.MD5, + DownloadCount: 0, + MaxDownloadCount: 0, + DeptId: task.Batch.CreatedDept, + StartTime: task.Batch.StartTime, + FinishTime: now, + TenantId: task.Batch.TenantID, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + }).InsertAndGetId() + if err != nil { + return gerror.Wrap(err, "insert lifecycle result download") + } + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Data(do.BatchMain{ + BatchAsyncStatus: batchAsyncStatus, + ValidCount: validCount, + SuccessCount: successCount, + FailCount: failCount, + RunResult: runResult, + FailReason: task.FailureReason, + ResultOssId: lifecycleOssIDValue(ossID), + BatchInfo: buildBatchInfo(task, ossID, downloadID), + EndTime: now, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return err + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.New("lifecycle batch is no longer processing") + } + return nil + }) + if err != nil { + batchresult.RemoveLocalFile(artifact) + return err + } + return nil +} + +func (r *Repository) artifactGenerator() batchresult.Generator { + if r.resultFiles != nil { + return r.resultFiles + } + return batchresult.New() +} + +func artifactRequest(task *lifecycle.Task) batchresult.Request { + rows := make([]batchresult.Row, 0, len(task.Details)) + for _, detail := range task.Details { + if detail == nil { + continue + } + rows = append(rows, batchresult.Row{ + RowNo: detail.RowNo, + ICCID: detail.ICCID, + MSISDN: detail.MSISDN, + IMSI: detail.IMSI, + Success: detail.Reason == "", + Reason: detail.Reason, + UpstreamCode: detail.UpstreamCode, + OrderNum: detail.OrderNum, + }) + } + return batchresult.Request{BatchNo: task.Batch.BatchNo, BatchType: task.Batch.BatchType, TenantID: task.Batch.TenantID, Rows: rows} +} + +func buildBatchInfo(task *lifecycle.Task, ossID, downloadID int64) *gjson.Json { + info := make(map[string]any) + if task.Batch.BatchInfo != nil { + for key, value := range task.Batch.BatchInfo.Map() { + info[key] = value + } + } + operation := "start" + if task.Batch.BatchType == consts.BatchTypeSimCardStop { + operation = "stop" + } + info["operation"] = operation + info["operType"] = task.Operation.OperType + info["resultFileOssId"] = ossID + info["resultDownloadId"] = downloadID + return gjson.New(info) +} + +func lifecycleOssIDValue(ossID int64) any { + if ossID <= 0 { + return nil + } + return ossID +} + +func batchResult(successCount, failCount int) (int, int) { + switch { + case failCount == 0: + return consts.BatchAsyncStatusSuccess, consts.BatchRunResultSuccess + case successCount == 0: + return consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed + default: + return consts.BatchAsyncStatusPartialSuccess, consts.BatchRunResultPartialSuccess + } +} diff --git a/internal/repository/lifecycle/restart.go b/internal/repository/lifecycle/restart.go new file mode 100644 index 0000000..ff59bb3 --- /dev/null +++ b/internal/repository/lifecycle/restart.go @@ -0,0 +1,610 @@ +package lifecycle + +import ( + "context" + "strconv" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" + lifecycle "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/lifecycle" +) + +func (r *Repository) FindRestartMain(ctx context.Context, batchNo string) (*entity.BatchMain, error) { + var main *entity.BatchMain + if err := r.database().Model(dao.BatchMain.Table()).Safe().Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeSimCardRestart). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Scan(&main); err != nil { + return nil, gerror.Wrap(err, "query restart batch main") + } + return main, nil +} + +func (r *Repository) LoadRestartStop(ctx context.Context, batchNo string) (*lifecycle.Task, error) { + return r.loadRestartTask(ctx, batchNo, consts.BatchDetailStatusRunning, lifecycle.Operation{ + BatchType: consts.BatchTypeSimCardRestart, + OperType: consts.SimCardStatusOperTypeStop, + ExpectedStatus: consts.SimBossCardStatusActivated, + TargetStatus: consts.SimBossCardStatusStopped, + }) +} + +func (r *Repository) LoadRestartResume(ctx context.Context, batchNo string) (*lifecycle.Task, error) { + return r.loadRestartTask(ctx, batchNo, consts.BatchDetailStatusRunning, lifecycle.Operation{ + BatchType: consts.BatchTypeSimCardRestart, + OperType: consts.SimCardStatusOperTypeStart, + ExpectedStatus: consts.SimBossCardStatusStopped, + TargetStatus: consts.SimBossCardStatusActivated, + }) +} + +func (r *Repository) loadRestartTask(ctx context.Context, batchNo string, detailStatus int, operation lifecycle.Operation) (*lifecycle.Task, error) { + if batchNo == "" { + return nil, gerror.New("restart batch number is empty") + } + batch := entity.BatchMain{} + if err := r.database().Model(dao.BatchMain.Table()).Safe().Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeSimCardRestart). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Scan(&batch); err != nil { + return nil, gerror.Wrap(err, "query restart batch") + } + if batch.Id == 0 { + return nil, gerror.New("processing restart batch does not exist") + } + + var batchDetails []*entity.BatchDetail + if err := r.database().Model(dao.BatchDetail.Table()).Safe().Ctx(ctx). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeSimCardRestart). + Where(dao.BatchDetail.Columns().DetailStatus, detailStatus). + OrderAsc(dao.BatchDetail.Columns().RowNo). + Scan(&batchDetails); err != nil { + return nil, gerror.Wrap(err, "query restart details") + } + if len(batchDetails) == 0 { + return nil, gerror.New("restart batch has no runnable details") + } + + task := &lifecycle.Task{ + Batch: lifecycle.Batch{ + ID: batch.Id, + BatchNo: batch.BatchNo, + BatchType: batch.BatchType, + TenantID: batch.TenantId, + CreatedBy: batch.CreatedBy, + CreatedDept: batch.CreatedDept, + StartTime: batch.StartTime, + BatchInfo: batch.BatchInfo, + }, + Operation: operation, + Details: make([]*lifecycle.Detail, 0, len(batchDetails)), + } + for _, batchDetail := range batchDetails { + if batchDetail == nil { + continue + } + task.Details = append(task.Details, restartDetail(batchDetail)) + } + if len(task.Details) == 0 { + return nil, gerror.New("restart batch has no readable details") + } + if err := r.loadCards(ctx, task); err != nil { + return nil, err + } + return task, nil +} + +func restartDetail(batchDetail *entity.BatchDetail) *lifecycle.Detail { + detail := &lifecycle.Detail{ + ID: batchDetail.Id, + RowNo: batchDetail.RowNo, + ICCID: strings.TrimSpace(batchDetail.ItemId), + Reason: strings.TrimSpace(batchDetail.ErrorReason), + OriginalSnapshot: batchDetail.Snapshot, + } + if batchDetail.Snapshot != nil { + if value := strings.TrimSpace(batchDetail.Snapshot.Get("ICCID").String()); value != "" { + detail.ICCID = value + } + detail.InputMSISDN = strings.TrimSpace(batchDetail.Snapshot.Get("MSISDN").String()) + detail.InputIMSI = strings.TrimSpace(batchDetail.Snapshot.Get("IMSI").String()) + detail.MSISDN = detail.InputMSISDN + detail.IMSI = detail.InputIMSI + detail.Authorized = batchDetail.Snapshot.Get("authorized").Bool() + detail.ChanKey = strings.TrimSpace(batchDetail.Snapshot.Get("chanKey").String()) + } + return detail +} + +func (r *Repository) SaveRestartValidationFailure(ctx context.Context, task *lifecycle.Task, stage string) error { + return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + for _, detail := range task.Details { + if detail == nil { + continue + } + if strings.TrimSpace(detail.Reason) == "" { + detail.Reason = "restart validation failed; upstream request was skipped" + } + outcome := lifecycle.Outcome{Message: detail.Reason} + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeSimCardRestart). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: consts.BatchDetailStatusFailed, + ErrorCode: strconv.Itoa(outcome.Code), + ErrorReason: detail.Reason, + Snapshot: restartSnapshot(task, detail, outcome, stage, gtime.Now()), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return err + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.Newf("restart detail %d is no longer running", detail.ID) + } + } + return nil + }) +} + +func (r *Repository) SaveRestartResult(ctx context.Context, task *lifecycle.Task, detail *lifecycle.Detail, outcome lifecycle.Outcome, stage string) (lifecycle.Outcome, error) { + if detail == nil { + return lifecycle.Outcome{}, gerror.New("restart detail is nil") + } + resolved := outcome + now := gtime.Now() + err := r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + if resolved.Success { + if detail.Card == nil { + resolved.Success = false + resolved.Message = "SIM card record is missing" + } else { + result, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()). + Where(dao.TSimCardInfo.Columns().Id, detail.Card.ID). + Where(dao.TSimCardInfo.Columns().TenantId, task.Batch.TenantID). + Where(dao.TSimCardInfo.Columns().BossCardStatus, detail.StatusBefore). + Data(do.TSimCardInfo{ + BossCardStatus: task.Operation.TargetStatus, + StatusSyncTime: now, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return err + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + resolved.Success = false + resolved.Message = "SIM card local status changed" + } + } + } + detail.ApplyOutcome(resolved) + detailStatus := consts.BatchDetailStatusFailed + if resolved.Success { + detailStatus = consts.BatchDetailStatusSuccess + } + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeSimCardRestart). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: detailStatus, + ErrorCode: strconv.Itoa(resolved.Code), + ErrorReason: detail.Reason, + Snapshot: restartSnapshot(task, detail, resolved, stage, now), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return err + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.Newf("restart detail %d is no longer running", detail.ID) + } + return nil + }) + return resolved, err +} + +func (r *Repository) WaitRestartResume(ctx context.Context, task *lifecycle.Task, resumeAt *gtime.Time) error { + info := buildRestartBatchInfo(task.Batch.BatchInfo) + restart := restartInfo(info) + restart["phase"] = consts.RestartPhaseWaitingResume + restart["stopCompletedAt"] = gtime.Now().String() + restart["resumeAt"] = resumeAt.String() + restart["validationOK"] = true + info = setRestartInfo(info, restart) + return r.updateRestartInfo(ctx, task.Batch.BatchNo, info, task.Batch.CreatedBy) +} + +func (r *Repository) ClaimRestartResume(ctx context.Context, batchNo string, now *gtime.Time) (claimed bool, err error) { + main, err := r.FindRestartMain(ctx, batchNo) + if err != nil { + return false, err + } + if main == nil || restartPhase(main.BatchInfo) != consts.RestartPhaseWaitingResume { + return false, nil + } + resumeAt := restartResumeAt(main.BatchInfo) + if resumeAt == nil || resumeAt.After(now) { + return false, nil + } + info := buildRestartBatchInfo(main.BatchInfo) + restart := restartInfo(info) + restart["phase"] = consts.RestartPhaseResuming + restart["resumeStartedAt"] = now.String() + info = setRestartInfo(info, restart) + + err = r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + mainColumns := dao.BatchMain.Columns() + result, updateErr := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(mainColumns.BatchNo, batchNo). + Where(mainColumns.BatchType, consts.BatchTypeSimCardRestart). + Where(mainColumns.BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Where("JSON_UNQUOTE(JSON_EXTRACT("+mainColumns.BatchInfo+", '$.restart.phase')) = ?", consts.RestartPhaseWaitingResume). + Data(do.BatchMain{ + BatchInfo: info, + UpdatedBy: main.CreatedBy, + }).Update() + if updateErr != nil { + return updateErr + } + affected, affectedErr := result.RowsAffected() + if affectedErr != nil { + return affectedErr + } + if affected == 0 { + return nil + } + if _, updateErr = tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeSimCardRestart). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusSuccess). + Data(do.BatchDetail{ + DetailStatus: consts.BatchDetailStatusRunning, + UpdatedBy: main.CreatedBy, + }).Update(); updateErr != nil { + return updateErr + } + claimed = true + return nil + }) + return claimed, err +} + +func (r *Repository) FinalizeRestart(ctx context.Context, batchNo string) error { + main, err := r.FindRestartMain(ctx, batchNo) + if err != nil { + return err + } + if main == nil { + return gerror.New("processing restart batch does not exist") + } + var details []*entity.BatchDetail + if err = r.database().Model(dao.BatchDetail.Table()).Safe().Ctx(ctx). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeSimCardRestart). + OrderAsc(dao.BatchDetail.Columns().RowNo). + Scan(&details); err != nil { + return gerror.Wrap(err, "query restart final details") + } + if len(details) == 0 { + return gerror.New("restart batch has no final details") + } + + successCount := 0 + for _, detail := range details { + if detail != nil && detail.DetailStatus == consts.BatchDetailStatusSuccess { + successCount++ + } + } + failCount := len(details) - successCount + batchAsyncStatus, runResult := batchResult(successCount, failCount) + failReason := "" + if failCount > 0 { + failReason = strconv.Itoa(failCount) + " SIM cards failed" + } + validCount := 0 + if restartValidationOK(main.BatchInfo) { + validCount = len(details) + } + + artifact, err := r.artifactGenerator().Create(ctx, restartArtifactRequest(main, details)) + if err != nil { + return gerror.Wrap(err, "generate restart result file") + } + now := gtime.Now() + err = r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + ossID, err := tx.Ctx(ctx).Model(dao.SysOss.Table()).Data(do.SysOss{ + TenantId: main.TenantId, + FileName: artifact.FileName, + OriginalName: artifact.OriginalName, + FileSuffix: artifact.FileSuffix, + Path: artifact.Path, + Url: artifact.URL, + CreatedDept: main.CreatedDept, + CreatedBy: main.CreatedBy, + UpdatedBy: main.CreatedBy, + Service: "local", + Md5: artifact.MD5, + FileSize: artifact.Size, + }).InsertAndGetId() + if err != nil { + return gerror.Wrap(err, "insert restart result OSS") + } + downloadID, err := tx.Ctx(ctx).Model(dao.TResourceBatchDownload.Table()).Data(do.TResourceBatchDownload{ + BatchNo: main.BatchNo, + TaskName: batchresult.TaskName(consts.BatchTypeSimCardRestart), + ResourceType: consts.ResourceTypeExcel, + ExportScene: consts.BatchTypeSimCardRestart, + TotalCount: len(details), + SuccessCount: successCount, + FailCount: failCount, + DownloadStatus: consts.DownloadStatusGenerated, + OssId: ossID, + FileName: artifact.FileName, + FileExt: artifact.FileSuffix, + FileUrl: artifact.URL, + FilePath: artifact.Path, + FileSize: artifact.Size, + FileMd5: artifact.MD5, + DownloadCount: 0, + MaxDownloadCount: 0, + DeptId: main.CreatedDept, + StartTime: main.StartTime, + FinishTime: now, + TenantId: main.TenantId, + CreatedDept: main.CreatedDept, + CreatedBy: main.CreatedBy, + UpdatedBy: main.CreatedBy, + }).InsertAndGetId() + if err != nil { + return gerror.Wrap(err, "insert restart result download") + } + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, main.BatchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeSimCardRestart). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Data(do.BatchMain{ + BatchAsyncStatus: batchAsyncStatus, + ValidCount: validCount, + SuccessCount: successCount, + FailCount: failCount, + RunResult: runResult, + FailReason: failReason, + ResultOssId: lifecycleOssIDValue(ossID), + BatchInfo: buildRestartFinishedInfo(main.BatchInfo, ossID, downloadID, now), + EndTime: now, + UpdatedBy: main.CreatedBy, + }).Update() + if err != nil { + return err + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.New("restart batch is no longer processing") + } + return nil + }) + if err != nil { + batchresult.RemoveLocalFile(artifact) + return err + } + return nil +} + +func (r *Repository) updateRestartInfo(ctx context.Context, batchNo string, info *gjson.Json, updatedBy int64) error { + result, err := r.database().Model(dao.BatchMain.Table()).Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeSimCardRestart). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Data(do.BatchMain{ + BatchInfo: info, + UpdatedBy: updatedBy, + }).Update() + if err != nil { + return err + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.New("restart batch info was not updated") + } + return nil +} + +func restartArtifactRequest(main *entity.BatchMain, details []*entity.BatchDetail) batchresult.Request { + rows := make([]batchresult.Row, 0, len(details)) + for _, detail := range details { + if detail == nil { + continue + } + rows = append(rows, restartArtifactRow(detail)) + } + return batchresult.Request{BatchNo: main.BatchNo, BatchType: consts.BatchTypeSimCardRestart, TenantID: main.TenantId, Rows: rows} +} + +func restartArtifactRow(detail *entity.BatchDetail) batchresult.Row { + row := batchresult.Row{ + RowNo: detail.RowNo, + ICCID: strings.TrimSpace(detail.ItemId), + Success: detail.DetailStatus == consts.BatchDetailStatusSuccess, + Reason: strings.TrimSpace(detail.ErrorReason), + } + if detail.Snapshot != nil { + if value := strings.TrimSpace(detail.Snapshot.Get("ICCID").String()); value != "" { + row.ICCID = value + } + row.MSISDN = strings.TrimSpace(detail.Snapshot.Get("MSISDN").String()) + row.IMSI = strings.TrimSpace(detail.Snapshot.Get("IMSI").String()) + row.OrderNum = strings.TrimSpace(detail.Snapshot.Get("restart.resume.orderNum").String()) + if row.OrderNum == "" { + row.OrderNum = strings.TrimSpace(detail.Snapshot.Get("restart.stop.orderNum").String()) + } + } + if code, err := strconv.Atoi(strings.TrimSpace(detail.ErrorCode)); err == nil { + row.UpstreamCode = code + } + return row +} + +func restartSnapshot(task *lifecycle.Task, detail *lifecycle.Detail, outcome lifecycle.Outcome, stage string, now *gtime.Time) *gjson.Json { + base := snapshotMap(detail) + base["ICCID"] = detail.ICCID + base["inputMsisdn"] = detail.InputMSISDN + base["inputImsi"] = detail.InputIMSI + base["MSISDN"] = detail.MSISDN + base["IMSI"] = detail.IMSI + base["authorized"] = detail.Authorized + base["chanKey"] = detail.ChanKey + base["statusBefore"] = detail.StatusBefore + base["orderNum"] = detail.OrderNum + base["upstreamCode"] = detail.UpstreamCode + base["upstreamMessage"] = detail.UpstreamReason + + restart := snapshotNestedMap(base["restart"]) + statusAfter := detail.StatusBefore + if outcome.Success { + statusAfter = task.Operation.TargetStatus + } + restart[stage] = map[string]any{ + "success": outcome.Success, + "reason": detail.Reason, + "upstreamCode": outcome.Code, + "upstreamMessage": strings.TrimSpace(outcome.Message), + "orderNum": strings.TrimSpace(outcome.OrderNum), + "operType": task.Operation.OperType, + "statusBefore": detail.StatusBefore, + "statusAfter": statusAfter, + "processedAt": now.String(), + } + base["restart"] = restart + return gjson.New(base) +} + +func snapshotMap(detail *lifecycle.Detail) map[string]any { + if detail != nil && detail.OriginalSnapshot != nil { + return copyMap(detail.OriginalSnapshot.Map()) + } + if detail != nil { + return copyMap(detail.Snapshot().Map()) + } + return map[string]any{} +} + +func snapshotNestedMap(value any) map[string]any { + if nested, ok := value.(map[string]any); ok { + return copyMap(nested) + } + return map[string]any{} +} + +func buildRestartBatchInfo(info *gjson.Json) *gjson.Json { + data := copyMap(nil) + if info != nil { + data = copyMap(info.Map()) + } + data["operation"] = "restart" + data["stopOperType"] = consts.SimCardStatusOperTypeStop + data["resumeOperType"] = consts.SimCardStatusOperTypeStart + if _, ok := data["resumeDelaySeconds"]; !ok { + data["resumeDelaySeconds"] = 300 + } + data["restart"] = restartInfo(info) + return gjson.New(data) +} + +func buildRestartFinishedInfo(info *gjson.Json, ossID, downloadID int64, finishedAt *gtime.Time) *gjson.Json { + updated := buildRestartBatchInfo(info) + restart := restartInfo(updated) + restart["phase"] = consts.RestartPhaseFinished + restart["finishedAt"] = finishedAt.String() + data := updated.Map() + data["resultFileOssId"] = ossID + data["resultDownloadId"] = downloadID + data["restart"] = restart + return gjson.New(data) +} + +func setRestartInfo(info *gjson.Json, restart map[string]any) *gjson.Json { + data := map[string]any{} + if info != nil { + data = copyMap(info.Map()) + } + data["restart"] = restart + return gjson.New(data) +} + +func restartInfo(info *gjson.Json) map[string]any { + if info == nil { + return map[string]any{} + } + return copyMap(info.Get("restart").Map()) +} + +func restartPhase(info *gjson.Json) string { + if info == nil { + return "" + } + return strings.TrimSpace(info.Get("restart.phase").String()) +} + +func restartResumeAt(info *gjson.Json) *gtime.Time { + if info == nil { + return nil + } + value := strings.TrimSpace(info.Get("restart.resumeAt").String()) + if value == "" { + return nil + } + parsed := gtime.NewFromStr(value) + if parsed == nil || parsed.IsZero() { + return nil + } + return parsed +} + +func restartValidationOK(info *gjson.Json) bool { + if info == nil { + return false + } + return info.Get("restart.validationOK").Bool() +} + +func copyMap(source map[string]any) map[string]any { + copied := make(map[string]any, len(source)) + for key, value := range source { + copied[key] = value + } + return copied +} diff --git a/internal/repository/monthlyusage/repository.go b/internal/repository/monthlyusage/repository.go new file mode 100644 index 0000000..31c9a0b --- /dev/null +++ b/internal/repository/monthlyusage/repository.go @@ -0,0 +1,51 @@ +package monthlyusage + +import ( + "context" + "fmt" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" +) + +// Repository applies the current usage result to the card information table. +// It deliberately does not write TDengine history for this full-sync flow. +type Repository struct{} + +type UsageResult struct { + CardID int64 + UsageMB float64 +} + +func NewRepository() *Repository { + return &Repository{} +} + +func (r *Repository) ApplyBatch(ctx context.Context, results []UsageResult) error { + if len(results) == 0 { + return nil + } + cardIDs := make([]int64, 0, len(results)) + var statement strings.Builder + statement.WriteString("CASE " + dao.TSimCardInfo.Columns().Id + " ") + for _, result := range results { + if result.CardID <= 0 { + return fmt.Errorf("monthly usage result has invalid cardId %d", result.CardID) + } + fmt.Fprintf(&statement, "WHEN %d THEN %f ", result.CardID, result.UsageMB) + cardIDs = append(cardIDs, result.CardID) + } + statement.WriteString("END") + _, err := dao.TSimCardInfo.Ctx(ctx). + WhereIn(dao.TSimCardInfo.Columns().Id, cardIDs). + Data(do.TSimCardInfo{ + UsedFlowMb: gdb.Raw(statement.String()), + FlowSyncTime: gtime.Now(), + }). + Update() + return err +} diff --git a/internal/repository/monthlyusage/repository_test.go b/internal/repository/monthlyusage/repository_test.go new file mode 100644 index 0000000..9d51b96 --- /dev/null +++ b/internal/repository/monthlyusage/repository_test.go @@ -0,0 +1,45 @@ +package monthlyusage + +import ( + "context" + "testing" + + _ "github.com/gogf/gf/contrib/drivers/sqlite/v2" + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +func TestApplyBatchUpdatesDifferentCardsWithOneCaseStatement(t *testing.T) { + if err := gdb.SetConfig(gdb.Config{ + "default": gdb.ConfigGroup{{Type: "sqlite", Link: "sqlite::@file(:memory:)"}}, + }); err != nil { + t.Fatal(err) + } + ctx := context.Background() + if _, err := g.DB().Exec(ctx, `CREATE TABLE t_sim_card_info ( +id INTEGER PRIMARY KEY, +used_flow_mb REAL, +flow_sync_time DATETIME +)`); err != nil { + t.Fatal(err) + } + if _, err := g.DB().Exec(ctx, `INSERT INTO t_sim_card_info(id, used_flow_mb) VALUES (1, 0), (2, 0), (3, 9)`); err != nil { + t.Fatal(err) + } + if err := NewRepository().ApplyBatch(ctx, []UsageResult{ + {CardID: 1, UsageMB: 1.5}, + {CardID: 2, UsageMB: 2.75}, + }); err != nil { + t.Fatal(err) + } + var rows []struct { + ID int64 `orm:"id"` + UsedFlowMB float64 `orm:"used_flow_mb"` + } + if err := g.DB().Model("t_sim_card_info").Fields("id,used_flow_mb").OrderAsc("id").Scan(&rows); err != nil { + t.Fatal(err) + } + if len(rows) != 3 || rows[0].UsedFlowMB != 1.5 || rows[1].UsedFlowMB != 2.75 || rows[2].UsedFlowMB != 9 { + t.Fatalf("rows = %+v", rows) + } +} diff --git a/internal/repository/renewchange/change_order_repository.go b/internal/repository/renewchange/change_order_repository.go new file mode 100644 index 0000000..89cd813 --- /dev/null +++ b/internal/repository/renewchange/change_order_repository.go @@ -0,0 +1,411 @@ +package renewchange + +import ( + "context" + "encoding/json" + "sort" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" +) + +// ChangeBindHistoryDO is the repository-local shape of the history table. +// It deliberately lives outside the generated model because Worker does not +// generate the card-package domain tables. +type ChangeBindHistoryDO struct { + g.Meta `orm:"table:t_sim_bind_package_logs, do:true"` + SubscriptionNo any `orm:"subscription_no"` + BatchNo any `orm:"batch_no"` + Iccid any `orm:"iccid"` + Msisdn any `orm:"msisdn"` + DeptId any `orm:"dept_id"` + PackageType any `orm:"package_type"` + PackageCode any `orm:"package_code"` + PreviousPackageCode any `orm:"previous_package_code"` + BusinessType any `orm:"business_type"` + EffectiveStatus any `orm:"effective_status"` + HandleTime *gtime.Time `orm:"handle_time"` + EffectiveTime *gtime.Time `orm:"effective_time"` + ExpectedExpireTime *gtime.Time `orm:"expected_expire_time"` + ActualExpireTime *gtime.Time `orm:"actual_expire_time"` + DurationMonths any `orm:"duration_months"` + GiftDurationMonths any `orm:"gift_duration_months"` + OrderCode any `orm:"order_code"` + Remark any `orm:"remark"` + TenantId any `orm:"tenant_id"` + CreatedDept any `orm:"created_dept"` + CreatedBy any `orm:"created_by"` + CreatedAt *gtime.Time `orm:"created_at"` + UpdatedBy any `orm:"updated_by"` + UpdatedAt *gtime.Time `orm:"updated_at"` +} + +type changeOrderUpdateDO struct { + g.Meta `orm:"table:t_order, do:true"` + ReturnStatus any `orm:"return_status"` + UpdatedBy any `orm:"updated_by"` + UpdatedAt *gtime.Time `orm:"updated_at"` +} + +type changeOrderItemRow struct { + ItemId int64 `orm:"item_id"` + ExtraSkuData string `orm:"extra_sku_data"` +} + +type changeOrderItemUpdateDO struct { + g.Meta `orm:"table:t_order_item, do:true"` + ExtraSkuData any `orm:"extra_sku_data"` + UpdatedBy any `orm:"updated_by"` + UpdatedAt *gtime.Time `orm:"updated_at"` +} + +type changeCardUpdateDO struct { + g.Meta `orm:"table:t_sim_card_info, do:true"` + CurrentPackageCode any `orm:"current_package_code"` + PackageStatus any `orm:"package_status"` + FlowPkgBeginAt any `orm:"flow_pkg_begin_at"` + FlowPkgPresentAt any `orm:"flow_pkg_present_at"` + FlowPkgRealPresentAt any `orm:"flow_pkg_real_present_at"` + FlowPkgEndAt any `orm:"flow_pkg_end_at"` + DataServiceStatus any `orm:"data_service_status"` + PackageTotalFlowMb any `orm:"package_total_flow_mb"` + UsedFlowMb any `orm:"used_flow_mb"` + TrafficCutoffStatus any `orm:"traffic_cutoff_status"` + TrafficCutoffThreshold any `orm:"traffic_cutoff_threshold"` + SmsServiceStatus any `orm:"sms_service_status"` + PackageTotalSms any `orm:"package_total_sms"` + UsedSms any `orm:"used_sms"` + VoiceServiceStatus any `orm:"voice_service_status"` + PackageTotalVoice any `orm:"package_total_voice"` + UsedVoice any `orm:"used_voice"` + RoamServiceStatus any `orm:"roam_service_status"` + PackageTotalRoamFlowMb any `orm:"package_total_roam_flow_mb"` + UsedRoamFlowMb any `orm:"used_roam_flow_mb"` + UpdatedBy any `orm:"updated_by"` + UpdatedAt *gtime.Time `orm:"updated_at"` +} + +type ChangeDeptRow struct { + DeptId int64 `orm:"dept_id"` + ParentId int64 `orm:"parent_id"` + TenantId string `orm:"tenant_id"` +} + +// LockChangeBatch locks the formal change batch so a duplicate delivery +// cannot create a second order after the first transaction commits. +func (r *Repository) LockChangeBatch(ctx context.Context, tx gdb.TX, batchNo string) (*BatchMainRow, error) { + var main BatchMainRow + if err := tx.Ctx(ctx).Model(TableBatchMain). + Where("batch_no", strings.TrimSpace(batchNo)). + Where("batch_type", consts.BatchTypeChange). + LockUpdate(). + Scan(&main); err != nil { + return nil, gerror.Wrap(err, "lock change batch failed") + } + if main.Id == 0 { + return nil, gerror.New("change batch does not exist") + } + return &main, nil +} + +// LockChangeDetails locks all details, allowing the executor to verify that +// the claimed batch still has exactly the expected running rows. +func (r *Repository) LockChangeDetails(ctx context.Context, tx gdb.TX, batchNo string) ([]*BatchDetailRow, error) { + var rows []*BatchDetailRow + if err := tx.Ctx(ctx).Model(TableBatchDetail). + Where("batch_no", strings.TrimSpace(batchNo)). + Where("batch_type", consts.BatchTypeChange). + OrderAsc("row_no"). + LockUpdate(). + Scan(&rows); err != nil { + return nil, gerror.Wrap(err, "lock change details failed") + } + return rows, nil +} + +// LockChangeBinds returns every pending/effective bind for the requested +// cards. The service chooses the single current bind and treats equal-priority +// duplicates as an integrity error. +func (r *Repository) LockChangeBinds(ctx context.Context, tx gdb.TX, iccids []string) ([]*SimCardBindPkg, error) { + return r.LockActiveBindRowsByIccids(ctx, tx, iccids) +} + +func (r *Repository) LoadChangePackage(ctx context.Context, tx gdb.TX, tenantID, packageCode string) (*FlowPackageInfo, error) { + var row FlowPackageInfo + if err := tx.Ctx(ctx).Model(TableSimFlowPackageInfo). + Where("tenant_id", strings.TrimSpace(tenantID)). + Where("package_code", strings.TrimSpace(packageCode)). + LockUpdate(). + Scan(&row); err != nil { + return nil, gerror.Wrap(err, "load change package failed") + } + if row.Id == 0 { + return nil, gerror.Newf("package not found: %s", packageCode) + } + return &row, nil +} + +func (r *Repository) LoadChangeDept(ctx context.Context, tx gdb.TX, tenantID string, deptID int64) (*ChangeDeptRow, error) { + var row ChangeDeptRow + if err := tx.Ctx(ctx).Model(TableSysDept). + Where("tenant_id", strings.TrimSpace(tenantID)). + Where("dept_id", deptID). + Scan(&row); err != nil { + return nil, gerror.Wrap(err, "load change customer failed") + } + if row.DeptId == 0 { + return nil, gerror.New("change customer does not exist") + } + return &row, nil +} + +func (r *Repository) ArchiveChangeBinds(ctx context.Context, tx gdb.TX, binds []*SimCardBindPkg, status int, actualExpireAt *gtime.Time, changeBatchNo string, userID int64) error { + if len(binds) == 0 { + return nil + } + rows := make([]ChangeBindHistoryDO, 0, len(binds)) + for _, bind := range binds { + if bind == nil || strings.TrimSpace(bind.SubscriptionNo) == "" { + return gerror.New("change subscription history is incomplete") + } + rows = append(rows, ChangeBindHistoryDO{ + SubscriptionNo: bind.SubscriptionNo, + BatchNo: bind.BatchNo, + Iccid: bind.Iccid, + Msisdn: bind.Msisdn, + DeptId: bind.DeptId, + PackageType: bind.PackageType, + PackageCode: bind.PackageCode, + PreviousPackageCode: bind.PreviousPackageCode, + BusinessType: bind.BusinessType, + EffectiveStatus: status, + HandleTime: bind.HandleTime, + EffectiveTime: bind.EffectiveTime, + ExpectedExpireTime: bind.ExpectedExpireTime, + ActualExpireTime: actualExpireAt, + DurationMonths: bind.DurationMonths, + GiftDurationMonths: bind.GiftDurationMonths, + OrderCode: bind.OrderCode, + Remark: appendChangeRemark(bind.Remark, changeBatchNo), + TenantId: bind.TenantId, + CreatedDept: bind.CreatedDept, + CreatedBy: bind.CreatedBy, + CreatedAt: bind.CreatedAt, + UpdatedBy: userID, + UpdatedAt: actualExpireAt, + }) + } + if _, err := tx.Ctx(ctx).Model(TableSimBindPackageLogs). + Data(rows). + OnConflict("subscription_no"). + Save(); err != nil { + return gerror.Wrap(err, "archive change subscriptions failed") + } + return nil +} + +func (r *Repository) DeleteChangeBinds(ctx context.Context, tx gdb.TX, subscriptionNos []string) (int64, error) { + unique := uniqueStrings(subscriptionNos) + if len(unique) == 0 { + return 0, nil + } + result, err := tx.Ctx(ctx).Model(TableSimCardBindPkg). + WhereIn("subscription_no", unique). + Delete() + if err != nil { + return 0, gerror.Wrap(err, "delete change subscriptions failed") + } + affected, err := result.RowsAffected() + if err != nil { + return 0, gerror.Wrap(err, "read deleted change subscriptions count failed") + } + return affected, nil +} + +func (r *Repository) UpdateChangeOrdersAfterRemoval(ctx context.Context, tx gdb.TX, orderCodes []string, batchNo, batchKey string, quantityByOrder map[string]int, eventTime *gtime.Time, userID int64) error { + orderCodes = uniqueStrings(orderCodes) + sort.Strings(orderCodes) + if len(orderCodes) == 0 { + return nil + } + if eventTime == nil { + eventTime = gtime.Now() + } + activeCounts, err := countChangeBindsByOrder(ctx, tx, orderCodes) + if err != nil { + return err + } + for _, orderCode := range orderCodes { + returnStatus := consts.OrderReturnStatusSuccess + if activeCounts[orderCode] > 0 { + returnStatus = consts.OrderReturnStatusPartial + } + result, updateErr := tx.Ctx(ctx).Model(TableOrder). + Where("order_sn", orderCode). + Data(buildChangeOrderUpdateDO(returnStatus, userID, eventTime)). + Update() + if updateErr != nil { + return gerror.Wrap(updateErr, "update changed order return status failed") + } + affected, affectedErr := result.RowsAffected() + if affectedErr != nil { + return gerror.Wrap(affectedErr, "read changed order update count failed") + } + if affected != 1 { + return gerror.Newf("changed source order does not exist: %s", orderCode) + } + if err = r.updateChangeOrderItemExtra(ctx, tx, orderCode, batchNo, batchKey, quantityByOrder[orderCode], eventTime, userID); err != nil { + return err + } + } + return nil +} + +func (r *Repository) updateChangeOrderItemExtra(ctx context.Context, tx gdb.TX, orderCode, batchNo, batchKey string, quantity int, eventTime *gtime.Time, userID int64) error { + var item changeOrderItemRow + if err := tx.Ctx(ctx).Model(TableOrderItem). + Where("order_sn", orderCode). + Limit(1). + Scan(&item); err != nil { + return gerror.Wrap(err, "load changed order item failed") + } + if item.ItemId == 0 { + return gerror.Newf("changed order item does not exist: %s", orderCode) + } + extra := map[string]any{} + if strings.TrimSpace(item.ExtraSkuData) != "" { + if err := json.Unmarshal([]byte(item.ExtraSkuData), &extra); err != nil { + return gerror.Wrap(err, "parse changed order item extra failed") + } + } + extra[batchKey+"BatchNo"] = batchNo + extra[batchKey+"Time"] = eventTime.Format("Y-m-d H:i:s") + if batchKey == "lastUnsubscribe" { + extra["unsubscribedQuantity"] = quantity + } else { + extra["voidedByChangeQuantity"] = quantity + } + data, err := json.Marshal(extra) + if err != nil { + return gerror.Wrap(err, "marshal changed order item extra failed") + } + _, err = tx.Ctx(ctx).Model(TableOrderItem). + Where("item_id", item.ItemId). + Data(buildChangeOrderItemUpdateDO(string(data), userID, eventTime)). + Update() + return gerror.Wrap(err, "update changed order item extra failed") +} + +func buildChangeOrderUpdateDO(returnStatus int, userID int64, updatedAt *gtime.Time) changeOrderUpdateDO { + return changeOrderUpdateDO{ReturnStatus: returnStatus, UpdatedBy: userID, UpdatedAt: updatedAt} +} + +func buildChangeOrderItemUpdateDO(extraSKUData string, userID int64, updatedAt *gtime.Time) changeOrderItemUpdateDO { + return changeOrderItemUpdateDO{ExtraSkuData: extraSKUData, UpdatedBy: userID, UpdatedAt: updatedAt} +} + +func countChangeBindsByOrder(ctx context.Context, tx gdb.TX, orderCodes []string) (map[string]int, error) { + type orderCountRow struct { + OrderCode string `orm:"order_code"` + Count int `orm:"count"` + } + var rows []orderCountRow + if err := tx.Ctx(ctx).Model(TableSimCardBindPkg). + Fields("order_code", "COUNT(1) AS count"). + WhereIn("order_code", orderCodes). + Group("order_code"). + Scan(&rows); err != nil { + return nil, gerror.Wrap(err, "count changed order subscriptions failed") + } + counts := make(map[string]int, len(rows)) + for _, row := range rows { + counts[strings.TrimSpace(row.OrderCode)] = row.Count + } + return counts, nil +} + +func (r *Repository) UpdateChangeCard(ctx context.Context, tx gdb.TX, iccid, packageCode string, packageStatus int, packageFlowMb float64, beginAt, endAt *gtime.Time, userID int64) error { + result, err := tx.Ctx(ctx).Model(TableSimCardInfo). + Where("iccid", strings.TrimSpace(iccid)). + Data(changeCardUpdateDO{ + CurrentPackageCode: packageCode, + PackageStatus: packageStatus, + FlowPkgBeginAt: beginAt, + FlowPkgPresentAt: gdb.Raw("NULL"), + FlowPkgRealPresentAt: gdb.Raw("NULL"), + FlowPkgEndAt: endAt, + DataServiceStatus: 1, + PackageTotalFlowMb: packageFlowMb, + UsedFlowMb: 0, + TrafficCutoffStatus: 0, + TrafficCutoffThreshold: 0, + SmsServiceStatus: 0, + PackageTotalSms: 0, + UsedSms: 0, + VoiceServiceStatus: 0, + PackageTotalVoice: 0, + UsedVoice: 0, + RoamServiceStatus: 0, + PackageTotalRoamFlowMb: 0, + UsedRoamFlowMb: 0, + UpdatedBy: userID, + UpdatedAt: gtime.Now(), + }). + Update() + if err != nil { + return gerror.Wrap(err, "update changed card package failed") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read changed card update count failed") + } + if affected != 1 { + return gerror.Newf("changed card was modified concurrently: %s", iccid) + } + return nil +} + +func (r *Repository) MarkChangeDetailsSuccess(ctx context.Context, tx gdb.TX, batchNo string, details []*BatchDetailRow, snapshots map[int]*gjson.Json, userID int64) error { + for _, detail := range details { + if detail == nil { + continue + } + data := do.BatchDetail{ + DetailStatus: consts.BatchDetailStatusSuccess, + ErrorCode: "", + ErrorReason: "", + Snapshot: snapshots[detail.RowNo], + UpdatedBy: userID, + } + result, err := tx.Ctx(ctx).Model(TableBatchDetail). + Where("batch_no", batchNo). + Where("batch_type", consts.BatchTypeChange). + Where("row_no", detail.RowNo). + Data(data). + Update() + if err != nil { + return gerror.Wrap(err, "mark change detail success failed") + } + affected, affectedErr := result.RowsAffected() + if affectedErr != nil || affected != 1 { + return gerror.Newf("change detail disappeared during success update: row %d", detail.RowNo) + } + } + return nil +} + +func appendChangeRemark(old, batchNo string) string { + addition := "changeBatchNo=" + strings.TrimSpace(batchNo) + old = strings.TrimSpace(old) + if old == "" { + return addition + } + return old + "; " + addition +} diff --git a/internal/repository/renewchange/change_order_repository_test.go b/internal/repository/renewchange/change_order_repository_test.go new file mode 100644 index 0000000..782f969 --- /dev/null +++ b/internal/repository/renewchange/change_order_repository_test.go @@ -0,0 +1,32 @@ +package renewchange + +import ( + "testing" + "time" + + "github.com/gogf/gf/v2/os/gtime" +) + +func TestChangeOrderRemovalUpdatesCarryAuditTimestamp(t *testing.T) { + updatedAt := gtime.NewFromTime(time.Date(2026, 8, 3, 10, 2, 0, 0, time.Local)) + orderUpdate := buildChangeOrderUpdateDO(2, 7, updatedAt) + itemUpdate := buildChangeOrderItemUpdateDO(`{"lastChangeBatchNo":"CHG-1"}`, 7, updatedAt) + + if orderUpdate.UpdatedAt != updatedAt || itemUpdate.UpdatedAt != updatedAt { + t.Fatalf("source order audit timestamps = %v/%v, want shared %v", orderUpdate.UpdatedAt, itemUpdate.UpdatedAt, updatedAt) + } + if orderUpdate.ReturnStatus != 2 || orderUpdate.UpdatedBy != int64(7) { + t.Fatalf("source order update = %+v", orderUpdate) + } + if itemUpdate.ExtraSkuData == nil || itemUpdate.UpdatedBy != int64(7) { + t.Fatalf("source order item update = %+v", itemUpdate) + } +} + +func TestChangeCardUpdateHasExplicitUpdatedAtField(t *testing.T) { + updatedAt := gtime.NewFromTime(time.Date(2026, 8, 3, 10, 2, 0, 0, time.Local)) + update := changeCardUpdateDO{UpdatedBy: int64(7), UpdatedAt: updatedAt} + if update.UpdatedAt != updatedAt { + t.Fatalf("card updated_at = %v, want %v", update.UpdatedAt, updatedAt) + } +} diff --git a/internal/repository/renewchange/errors.go b/internal/repository/renewchange/errors.go new file mode 100644 index 0000000..a031929 --- /dev/null +++ b/internal/repository/renewchange/errors.go @@ -0,0 +1,39 @@ +package renewchange + +import "github.com/gogf/gf/v2/errors/gerror" + +var ( + errRenewDurationEmpty = gerror.New("续费/变更套餐未配置可续费时长") + errPkgNotFound = gerror.New("套餐不存在") + errPkgDisabled = gerror.New("套餐未启用或不存在") + errDeptNotFound = gerror.New("客户不存在") + errNoChangeablePackages = gerror.New("当前套餐未配置可变更套餐范围") + errCardNotFound = gerror.New("ICCID不存在正式卡记录") + errLatestBindMissing = gerror.New("ICCID最新订购记录不存在") + errCardNotOwned = gerror.New("ICCID不属于当前机构") + errCardNoCurrentPackage = gerror.New("ICCID没有当前套餐") + errCardPackageMismatch = gerror.New("ICCID当前套餐与最新订购记录不一致") + errCardExpireMissing = gerror.New("ICCID最新订购没有预计到期时间") + errCardSourceBatchMismatch = gerror.New("ICCID不属于来源出库批次") + errDuplicateICCID = gerror.New("ICCID重复") + errPrepareIncomplete = gerror.New("预处理结果不完整") + errMultiplePackages = gerror.New("当前批次下存在多个套餐，不能统一处理") + errMultipleCustomers = gerror.New("当前批次下存在多个客户，不能统一处理") + errSameBatchInvalid = gerror.New("同批存在不满足条件的卡，本次未执行") + errSameBatchChangeInvalid = gerror.New("同批存在不满足变更条件的卡，本行未执行") + errNoPendingBinds = gerror.New("ICCID没有有效订购记录") + errActiveBindNotFound = gerror.New("ICCID当前有效订购不存在") + errActiveBindMismatch = gerror.New("ICCID当前有效订购与当前套餐不一致") + errActiveBindSourceBatch = gerror.New("ICCID当前有效订购不属于来源批次") + errMsisdnMismatch = gerror.New("MSISDN与SIM卡记录不一致") + errImsiMismatch = gerror.New("IMSI与SIM卡记录不一致") + errRenewableDurationMiss = gerror.New("续费时长不在当前套餐可续费范围内") + errPriceChanged = gerror.New("当前续费价格已变化，请重新校验") + errPackageTypeMismatch = gerror.New("目标套餐类型与当前套餐类型不一致") + errNewPackageNotInChange = gerror.New("新套餐不在当前套餐可变更范围内") + errNewPackageSameAsOld = gerror.New("新套餐不能与当前套餐相同") + errTargetPkgNotFound = gerror.New("目标套餐不存在") + errPrepareBatchNotFound = gerror.New("预处理批次不存在或已执行") + errPrepareBatchTypeWrong = gerror.New("批次不是续费/变更预处理批次") + errPrepareInfoMissing = gerror.New("预处理批次缺少关键快照") +) diff --git a/internal/repository/renewchange/ids.go b/internal/repository/renewchange/ids.go new file mode 100644 index 0000000..b980e24 --- /dev/null +++ b/internal/repository/renewchange/ids.go @@ -0,0 +1,54 @@ +package renewchange + +import ( + "crypto/rand" + "fmt" + "strings" + "time" + + "github.com/gogf/gf/v2/errors/gerror" +) + +const businessNoAlphabet = "ABCDEFGHJKLMNPQRSTUVWXYZ23456789" + +// NextOrderSN returns a production order number before any INSERT is issued. +func NextOrderSN(now time.Time) (string, error) { + suffix, err := randomBusinessNoSuffix(8) + if err != nil { + return "", err + } + return fmt.Sprintf("SO%s%s", now.Format("20060102150405"), suffix), nil +} + +func randomBusinessNoSuffix(length int) (string, error) { + if length <= 0 { + return "", gerror.New("business number suffix length must be greater than zero") + } + randomBytes := make([]byte, length) + if _, err := rand.Read(randomBytes); err != nil { + return "", gerror.Wrap(err, "generate business number suffix failed") + } + buf := make([]byte, length) + for i, value := range randomBytes { + buf[i] = businessNoAlphabet[int(value)%len(businessNoAlphabet)] + } + return string(buf), nil +} + +// GenerateOrderSn 形如 "yyyyMMddHHmmssfff"+6位随机后缀 +func GenerateOrderSn(t *time.Time) string { + now := time.Now() + if t != nil { + now = *t + } + return fmt.Sprintf("%s%06d", now.UTC().Format("20060102150405.000")[0:14], now.Nanosecond()/1000%1000000) +} + +// GenerateSubscriptionNo 形如 "SUB" + yyyyMMddHHmmss + 6位后缀 +func GenerateSubscriptionNo(prefix string) string { + now := time.Now() + if strings.TrimSpace(prefix) == "" { + prefix = "SUB" + } + return fmt.Sprintf("%s%s%06d", prefix, now.UTC().Format("20060102150405"), now.Nanosecond()/1000%1000000) +} diff --git a/internal/repository/renewchange/order_do.go b/internal/repository/renewchange/order_do.go new file mode 100644 index 0000000..f755b74 --- /dev/null +++ b/internal/repository/renewchange/order_do.go @@ -0,0 +1,195 @@ +package renewchange + +import ( + "context" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + "github.com/gogf/gf/v2/util/gconv" +) + +// OrderDO mirrors the production t_order table. It is repository-local +// because Worker does not generate DAO/DO/entity code for the order domain. +type OrderDO struct { + g.Meta `orm:"table:t_order, do:true"` + Id any `orm:"id"` + OrderSn any `orm:"order_sn"` + ParentOrderId any `orm:"parent_order_id"` + ParentOrderSn any `orm:"parent_order_sn"` + CustomerType any `orm:"customer_type"` + CustomerAgentId any `orm:"customer_agent_id"` + EndUserId any `orm:"end_user_id"` + AgentId any `orm:"agent_id"` + OrderType any `orm:"order_type"` + OrderStatus any `orm:"order_status"` + ShippingStatus any `orm:"shipping_status"` + ReturnStatus any `orm:"return_status"` + Consignee any `orm:"consignee"` + Mobile any `orm:"mobile"` + Email any `orm:"email"` + Address any `orm:"address"` + AreaCode any `orm:"area_code"` + BuyerNote any `orm:"buyer_note"` + AdminNote any `orm:"admin_note"` + Remark any `orm:"remark"` + ShippingMethod any `orm:"shipping_method"` + LogisticsId any `orm:"logistics_id"` + LogisticsName any `orm:"logistics_name"` + ShippingTypeId any `orm:"shipping_type_id"` + ShippingTypeName any `orm:"shipping_type_name"` + TrackingNo any `orm:"tracking_no"` + ShippingTime *gtime.Time `orm:"shipping_time"` + ReceivedTime *gtime.Time `orm:"received_time"` + PayType any `orm:"pay_type"` + PayStatus any `orm:"pay_status"` + PayOrderSn any `orm:"pay_order_sn"` + PayTime *gtime.Time `orm:"pay_time"` + TotalAmount any `orm:"total_amount"` + ProductAmount any `orm:"product_amount"` + ShippingFee any `orm:"shipping_fee"` + ServiceFee any `orm:"service_fee"` + InvoiceFee any `orm:"invoice_fee"` + CouponAmount any `orm:"coupon_amount"` + DiscountAmount any `orm:"discount_amount"` + Balance any `orm:"balance"` + PaidAmount any `orm:"paid_amount"` + OnlinePaidAmount any `orm:"online_paid_amount"` + OfflinePaidAmount any `orm:"offline_paid_amount"` + UnpaidAmount any `orm:"unpaid_amount"` + UnrefundAmount any `orm:"unrefund_amount"` + IsNeedCommission any `orm:"is_need_commission"` + DistributionStatus any `orm:"distribution_status"` + TenantId any `orm:"tenant_id"` + CreatedDept any `orm:"created_dept"` + CreatedBy any `orm:"created_by"` + CreatedAt *gtime.Time `orm:"created_at"` + UpdatedBy any `orm:"updated_by"` + UpdatedAt *gtime.Time `orm:"updated_at"` +} + +// OrderItemDO mirrors the production t_order_item table. +type OrderItemDO struct { + g.Meta `orm:"table:t_order_item, do:true"` + ItemId any `orm:"item_id"` + OrderId any `orm:"order_id"` + OrderSn any `orm:"order_sn"` + CustomerType any `orm:"customer_type"` + CustomerAgentId any `orm:"customer_agent_id"` + EndUserId any `orm:"end_user_id"` + Price any `orm:"price"` + Quantity any `orm:"quantity"` + ProductId any `orm:"product_id"` + ProductName any `orm:"product_name"` + ProductSn any `orm:"product_sn"` + PicThumb any `orm:"pic_thumb"` + SkuId any `orm:"sku_id"` + SkuData any `orm:"sku_data"` + DeliveryQuantity any `orm:"delivery_quantity"` + ProductType any `orm:"product_type"` + IsGift any `orm:"is_gift"` + AgentId any `orm:"agent_id"` + PrepayPrice any `orm:"prepay_price"` + Commission any `orm:"commission"` + OriginPrice any `orm:"origin_price"` + PromotionData any `orm:"promotion_data"` + ExtraSkuData any `orm:"extra_sku_data"` + PayMode any `orm:"pay_mode"` + ServiceBeginTime *gtime.Time `orm:"service_begin_time"` + ServiceEndTime *gtime.Time `orm:"service_end_time"` + TenantId any `orm:"tenant_id"` + CreatedDept any `orm:"created_dept"` + CreatedBy any `orm:"created_by"` + CreatedAt *gtime.Time `orm:"created_at"` + UpdatedBy any `orm:"updated_by"` + UpdatedAt *gtime.Time `orm:"updated_at"` +} + +// InsertContractOrder inserts a fully built order and returns its primary key. +// The order number must exist before the INSERT is issued. +func (r *Repository) InsertContractOrder(ctx context.Context, tx gdb.TX, order *OrderDO) (int64, error) { + if err := ValidateOrderDO(order); err != nil { + return 0, err + } + id, err := tx.Ctx(ctx).Model(TableOrder).Data(order).InsertAndGetId() + if err != nil { + return 0, gerror.Wrap(err, "insert order failed") + } + return id, nil +} + +// InsertContractOrderItem inserts the aggregate order item for an order. +func (r *Repository) InsertContractOrderItem(ctx context.Context, tx gdb.TX, item *OrderItemDO) error { + if err := ValidateOrderItemDO(item); err != nil { + return err + } + _, err := tx.Ctx(ctx).Model(TableOrderItem).Data(item).Insert() + return gerror.Wrap(err, "insert order item failed") +} + +func ValidateOrderDO(order *OrderDO) error { + if order == nil { + return gerror.New("order cannot be nil") + } + if strings.TrimSpace(gconv.String(order.OrderSn)) == "" { + return gerror.New("order number cannot be empty") + } + if gconv.Int64(order.CustomerAgentId) <= 0 { + return gerror.New("order customer agent id must be greater than zero") + } + if gconv.Int64(order.AgentId) <= 0 { + return gerror.New("order seller agent id must be greater than zero") + } + if strings.TrimSpace(gconv.String(order.TenantId)) == "" { + return gerror.New("order tenant id cannot be empty") + } + if gconv.Int64(order.CreatedDept) <= 0 || gconv.Int64(order.CreatedBy) <= 0 || gconv.Int64(order.UpdatedBy) <= 0 { + return gerror.New("order creator organization is incomplete") + } + if order.CreatedAt == nil || order.UpdatedAt == nil { + return gerror.New("order audit time cannot be empty") + } + return nil +} + +func ValidateOrderItemDO(item *OrderItemDO) error { + if item == nil { + return gerror.New("order item cannot be nil") + } + if gconv.Int64(item.OrderId) <= 0 { + return gerror.New("order item order id must be greater than zero") + } + if strings.TrimSpace(gconv.String(item.OrderSn)) == "" { + return gerror.New("order item order number cannot be empty") + } + if gconv.Int64(item.CustomerAgentId) <= 0 { + return gerror.New("order item customer agent id must be greater than zero") + } + if gconv.Int64(item.AgentId) <= 0 { + return gerror.New("order item seller agent id must be greater than zero") + } + if gconv.Int(item.Quantity) <= 0 { + return gerror.New("order item quantity must be greater than zero") + } + if gconv.Int64(item.ProductId) <= 0 { + return gerror.New("order item product id must be greater than zero") + } + if strings.TrimSpace(gconv.String(item.ProductName)) == "" || strings.TrimSpace(gconv.String(item.ProductSn)) == "" { + return gerror.New("order item product contract is incomplete") + } + if item.ServiceBeginTime == nil || item.ServiceEndTime == nil || !item.ServiceEndTime.After(item.ServiceBeginTime) { + return gerror.New("order item service period is invalid") + } + if strings.TrimSpace(gconv.String(item.TenantId)) == "" { + return gerror.New("order item tenant id cannot be empty") + } + if gconv.Int64(item.CreatedDept) <= 0 || gconv.Int64(item.CreatedBy) <= 0 || gconv.Int64(item.UpdatedBy) <= 0 { + return gerror.New("order item creator organization is incomplete") + } + if item.CreatedAt == nil || item.UpdatedAt == nil { + return gerror.New("order item audit time cannot be empty") + } + return nil +} diff --git a/internal/repository/renewchange/order_do_test.go b/internal/repository/renewchange/order_do_test.go new file mode 100644 index 0000000..c5a2ab5 --- /dev/null +++ b/internal/repository/renewchange/order_do_test.go @@ -0,0 +1,218 @@ +package renewchange + +import ( + "reflect" + "strings" + "testing" + "time" + + "github.com/gogf/gf/v2/os/gtime" +) + +func TestOrderDOLocalSchemaMatchesProductionContract(t *testing.T) { + want := []string{ + "Id", "OrderSn", "ParentOrderId", "ParentOrderSn", "CustomerType", + "CustomerAgentId", "EndUserId", "AgentId", "OrderType", "OrderStatus", + "ShippingStatus", "ReturnStatus", "Consignee", "Mobile", "Email", "Address", + "AreaCode", "BuyerNote", "AdminNote", "Remark", "ShippingMethod", "LogisticsId", + "LogisticsName", "ShippingTypeId", "ShippingTypeName", "TrackingNo", "ShippingTime", + "ReceivedTime", "PayType", "PayStatus", "PayOrderSn", "PayTime", "TotalAmount", + "ProductAmount", "ShippingFee", "ServiceFee", "InvoiceFee", "CouponAmount", + "DiscountAmount", "Balance", "PaidAmount", "OnlinePaidAmount", "OfflinePaidAmount", + "UnpaidAmount", "UnrefundAmount", "IsNeedCommission", "DistributionStatus", "TenantId", + "CreatedDept", "CreatedBy", "CreatedAt", "UpdatedBy", "UpdatedAt", + } + assertDOFields(t, reflect.TypeOf(OrderDO{}), want) +} + +func TestOrderItemDOLocalSchemaMatchesProductionContract(t *testing.T) { + want := []string{ + "ItemId", "OrderId", "OrderSn", "CustomerType", "CustomerAgentId", "EndUserId", + "Price", "Quantity", "ProductId", "ProductName", "ProductSn", "PicThumb", "SkuId", + "SkuData", "DeliveryQuantity", "ProductType", "IsGift", "AgentId", "PrepayPrice", + "Commission", "OriginPrice", "PromotionData", "ExtraSkuData", "PayMode", + "ServiceBeginTime", "ServiceEndTime", "TenantId", "CreatedDept", "CreatedBy", + "CreatedAt", "UpdatedBy", "UpdatedAt", + } + assertDOFields(t, reflect.TypeOf(OrderItemDO{}), want) +} + +func TestOrderDOsHaveExplicitORMColumnMappings(t *testing.T) { + for _, testCase := range []struct { + name string + typ reflect.Type + }{ + {name: "order", typ: reflect.TypeOf(OrderDO{})}, + {name: "order item", typ: reflect.TypeOf(OrderItemDO{})}, + {name: "change history", typ: reflect.TypeOf(ChangeBindHistoryDO{})}, + {name: "change order update", typ: reflect.TypeOf(changeOrderUpdateDO{})}, + {name: "change order item update", typ: reflect.TypeOf(changeOrderItemUpdateDO{})}, + {name: "change card update", typ: reflect.TypeOf(changeCardUpdateDO{})}, + } { + t.Run(testCase.name, func(t *testing.T) { + for i := 0; i < testCase.typ.NumField(); i++ { + field := testCase.typ.Field(i) + if field.Name == "Meta" { + continue + } + if got := field.Tag.Get("orm"); got == "" { + t.Errorf("%s has no explicit orm column mapping", field.Name) + } + } + }) + } +} + +func TestOrderDOTagsMatchProductionColumns(t *testing.T) { + assertDOColumnTags(t, reflect.TypeOf(OrderDO{}), map[string]string{ + "OrderSn": "order_sn", "CustomerAgentId": "customer_agent_id", "EndUserId": "end_user_id", "AgentId": "agent_id", + "OrderType": "order_type", "OrderStatus": "order_status", "ShippingStatus": "shipping_status", "ReturnStatus": "return_status", + "BuyerNote": "buyer_note", "AdminNote": "admin_note", "Remark": "remark", "PayType": "pay_type", "PayStatus": "pay_status", + "PayTime": "pay_time", "TotalAmount": "total_amount", "ProductAmount": "product_amount", "PaidAmount": "paid_amount", + "OnlinePaidAmount": "online_paid_amount", "OfflinePaidAmount": "offline_paid_amount", "TenantId": "tenant_id", + "CreatedDept": "created_dept", "CreatedBy": "created_by", "CreatedAt": "created_at", "UpdatedBy": "updated_by", "UpdatedAt": "updated_at", + }) + assertDOColumnTags(t, reflect.TypeOf(OrderItemDO{}), map[string]string{ + "ItemId": "item_id", "OrderId": "order_id", "OrderSn": "order_sn", "CustomerAgentId": "customer_agent_id", "EndUserId": "end_user_id", + "Price": "price", "Quantity": "quantity", "ProductId": "product_id", "ProductName": "product_name", "ProductSn": "product_sn", + "SkuData": "sku_data", "ProductType": "product_type", "IsGift": "is_gift", "AgentId": "agent_id", "OriginPrice": "origin_price", + "ExtraSkuData": "extra_sku_data", "PayMode": "pay_mode", "ServiceBeginTime": "service_begin_time", "ServiceEndTime": "service_end_time", + "TenantId": "tenant_id", "CreatedDept": "created_dept", "CreatedBy": "created_by", "CreatedAt": "created_at", "UpdatedBy": "updated_by", "UpdatedAt": "updated_at", + }) +} + +func TestNextOrderSNIsGeneratedBeforeInsert(t *testing.T) { + got, err := NextOrderSN(time.Date(2026, 8, 3, 10, 2, 50, 0, time.Local)) + if err != nil { + t.Fatalf("NextOrderSN returned error: %v", err) + } + if !strings.HasPrefix(got, "SO20260803100250") { + t.Fatalf("NextOrderSN = %q, want SO prefix and timestamp", got) + } + if len(got) != len("SO20260803100250")+8 { + t.Fatalf("NextOrderSN length = %d, want %d", len(got), len("SO20260803100250")+8) + } +} + +func TestValidateContractOrderRequiredFields(t *testing.T) { + now := gtime.NewFromTime(time.Date(2026, 8, 3, 10, 2, 50, 0, time.Local)) + order := &OrderDO{ + OrderSn: "SO20260803100250ABCDEFGH", + CustomerAgentId: int64(200), + AgentId: int64(100), + TenantId: "000000", + CreatedDept: int64(100), + CreatedBy: int64(1), + CreatedAt: now, + UpdatedBy: int64(1), + UpdatedAt: now, + } + if err := ValidateOrderDO(order); err != nil { + t.Fatalf("ValidateOrderDO returned error: %v", err) + } + for _, testCase := range []struct { + name string + clear func(*OrderDO) + }{ + {name: "order number", clear: func(order *OrderDO) { order.OrderSn = "" }}, + {name: "customer agent", clear: func(order *OrderDO) { order.CustomerAgentId = int64(0) }}, + {name: "seller agent", clear: func(order *OrderDO) { order.AgentId = int64(0) }}, + {name: "tenant", clear: func(order *OrderDO) { order.TenantId = "" }}, + {name: "creator department", clear: func(order *OrderDO) { order.CreatedDept = int64(0) }}, + {name: "creator", clear: func(order *OrderDO) { order.CreatedBy = int64(0) }}, + {name: "updater", clear: func(order *OrderDO) { order.UpdatedBy = int64(0) }}, + {name: "created time", clear: func(order *OrderDO) { order.CreatedAt = nil }}, + {name: "updated time", clear: func(order *OrderDO) { order.UpdatedAt = nil }}, + } { + t.Run(testCase.name, func(t *testing.T) { + invalid := *order + testCase.clear(&invalid) + if err := ValidateOrderDO(&invalid); err == nil { + t.Fatalf("ValidateOrderDO accepted missing %s", testCase.name) + } + }) + } +} + +func TestValidateContractOrderItemRequiredFields(t *testing.T) { + beginAt := gtime.NewFromTime(time.Date(2026, 8, 3, 0, 0, 0, 0, time.Local)) + endAt := gtime.NewFromTime(time.Date(2027, 8, 3, 0, 0, 0, 0, time.Local)) + item := &OrderItemDO{ + OrderId: int64(1), + OrderSn: "SO20260803100250ABCDEFGH", + CustomerAgentId: int64(200), + AgentId: int64(100), + Quantity: 2, + ProductId: int64(300), + ProductName: "package", + ProductSn: "PKG001", + ServiceBeginTime: beginAt, + ServiceEndTime: endAt, + TenantId: "000000", + CreatedDept: int64(100), + CreatedBy: int64(1), + CreatedAt: beginAt, + UpdatedBy: int64(1), + UpdatedAt: beginAt, + } + if err := ValidateOrderItemDO(item); err != nil { + t.Fatalf("ValidateOrderItemDO returned error: %v", err) + } + for _, testCase := range []struct { + name string + clear func(*OrderItemDO) + }{ + {name: "order id", clear: func(item *OrderItemDO) { item.OrderId = int64(0) }}, + {name: "order number", clear: func(item *OrderItemDO) { item.OrderSn = "" }}, + {name: "customer agent", clear: func(item *OrderItemDO) { item.CustomerAgentId = int64(0) }}, + {name: "seller agent", clear: func(item *OrderItemDO) { item.AgentId = int64(0) }}, + {name: "quantity", clear: func(item *OrderItemDO) { item.Quantity = 0 }}, + {name: "product id", clear: func(item *OrderItemDO) { item.ProductId = int64(0) }}, + {name: "product name", clear: func(item *OrderItemDO) { item.ProductName = "" }}, + {name: "product code", clear: func(item *OrderItemDO) { item.ProductSn = "" }}, + {name: "service start", clear: func(item *OrderItemDO) { item.ServiceBeginTime = nil }}, + {name: "service end", clear: func(item *OrderItemDO) { item.ServiceEndTime = nil }}, + {name: "tenant", clear: func(item *OrderItemDO) { item.TenantId = "" }}, + {name: "creator department", clear: func(item *OrderItemDO) { item.CreatedDept = int64(0) }}, + {name: "creator", clear: func(item *OrderItemDO) { item.CreatedBy = int64(0) }}, + {name: "updater", clear: func(item *OrderItemDO) { item.UpdatedBy = int64(0) }}, + {name: "created time", clear: func(item *OrderItemDO) { item.CreatedAt = nil }}, + {name: "updated time", clear: func(item *OrderItemDO) { item.UpdatedAt = nil }}, + } { + t.Run(testCase.name, func(t *testing.T) { + invalid := *item + testCase.clear(&invalid) + if err := ValidateOrderItemDO(&invalid); err == nil { + t.Fatalf("ValidateOrderItemDO accepted missing %s", testCase.name) + } + }) + } +} + +func assertDOFields(t *testing.T, typ reflect.Type, want []string) { + t.Helper() + got := make([]string, 0, typ.NumField()-1) + for i := 0; i < typ.NumField(); i++ { + field := typ.Field(i) + if field.Name == "Meta" { + continue + } + got = append(got, field.Name) + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("%s fields mismatch\n got: %v\nwant: %v", typ.Name(), got, want) + } +} + +func assertDOColumnTags(t *testing.T, typ reflect.Type, want map[string]string) { + t.Helper() + for fieldName, column := range want { + field, ok := typ.FieldByName(fieldName) + if !ok { + t.Fatalf("%s missing field %s", typ.Name(), fieldName) + } + if got := field.Tag.Get("orm"); got != column { + t.Fatalf("%s.%s orm tag = %q, want %q", typ.Name(), fieldName, got, column) + } + } +} diff --git a/internal/repository/renewchange/price_override_test.go b/internal/repository/renewchange/price_override_test.go new file mode 100644 index 0000000..8ae9e35 --- /dev/null +++ b/internal/repository/renewchange/price_override_test.go @@ -0,0 +1,39 @@ +package renewchange + +import ( + "database/sql" + "errors" + "testing" +) + +func TestNormalizePriceOverrideResultTreatsNoRowsAsMissing(t *testing.T) { + got, err := normalizePriceOverrideResult(&FlowPackagePrice{}, sql.ErrNoRows) + if err != nil { + t.Fatalf("normalizePriceOverrideResult returned error: %v", err) + } + if got != nil { + t.Fatalf("normalizePriceOverrideResult returned %+v, want nil", got) + } +} + +func TestNormalizePriceOverrideResultReturnsOtherErrors(t *testing.T) { + wantErr := errors.New("database unavailable") + got, err := normalizePriceOverrideResult(nil, wantErr) + if got != nil { + t.Fatalf("normalizePriceOverrideResult returned %+v on error", got) + } + if !errors.Is(err, wantErr) { + t.Fatalf("error = %v, want wrapped %v", err, wantErr) + } +} + +func TestNormalizePriceOverrideResultKeepsPriceRow(t *testing.T) { + want := &FlowPackagePrice{Id: 7, AgentCostProfit: 1.25} + got, err := normalizePriceOverrideResult(want, nil) + if err != nil { + t.Fatalf("normalizePriceOverrideResult returned error: %v", err) + } + if got != want { + t.Fatalf("row pointer = %p, want %p", got, want) + } +} diff --git a/internal/repository/renewchange/renew_order_repository.go b/internal/repository/renewchange/renew_order_repository.go new file mode 100644 index 0000000..e2b698f --- /dev/null +++ b/internal/repository/renewchange/renew_order_repository.go @@ -0,0 +1,90 @@ +package renewchange + +import ( + "context" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/errors/gerror" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" +) + +// LockRenewBatchMain locks the business batch before the renew transaction +// starts. This closes the gap between runtime claiming and business writes +// when a duplicate message is delivered concurrently. +func (r *Repository) LockRenewBatchMain(ctx context.Context, tx gdb.TX, batchNo string, batchType int) (*BatchMainRow, error) { + var main BatchMainRow + err := tx.Ctx(ctx).Model(TableBatchMain). + Where("batch_no", strings.TrimSpace(batchNo)). + Where("batch_type", batchType). + LockUpdate(). + Scan(&main) + if err != nil { + return nil, gerror.Wrap(err, "lock renew batch main failed") + } + if main.Id == 0 { + return nil, gerror.New("renew batch main not found") + } + return &main, nil +} + +// LockRenewDetails reloads every detail row inside the business transaction. +// The initial consumer read is only a routing hint; the transaction must lock +// the exact rows and let the service reject any row that is no longer running. +func (r *Repository) LockRenewDetails(ctx context.Context, tx gdb.TX, batchNo string, batchType int) ([]*BatchDetailRow, error) { + var rows []*BatchDetailRow + if err := tx.Ctx(ctx).Model(TableBatchDetail). + Where("batch_no", strings.TrimSpace(batchNo)). + Where("batch_type", batchType). + OrderAsc("row_no"). + LockUpdate(). + Scan(&rows); err != nil { + return nil, gerror.Wrap(err, "lock renew details failed") + } + return rows, nil +} + +// LoadRenewPackage reads the package inside the business transaction so the +// order item is built from a real package row, not a hand-written snapshot. +func (r *Repository) LoadRenewPackage(ctx context.Context, tx gdb.TX, tenantID, packageCode string) (*FlowPackageInfo, error) { + var pkg FlowPackageInfo + query := tx.Ctx(ctx).Model(TableSimFlowPackageInfo). + Where("package_code", strings.TrimSpace(packageCode)) + if strings.TrimSpace(tenantID) != "" { + query = query.Where("tenant_id", strings.TrimSpace(tenantID)) + } + query = query.LockUpdate() + if err := query.Scan(&pkg); err != nil { + return nil, gerror.Wrap(err, "load renew package failed") + } + if pkg.Id == 0 { + return nil, gerror.New("renew package not found") + } + return &pkg, nil +} + +// UpdateRenewCard writes only the fields allowed by the renew contract. +func (r *Repository) UpdateRenewCard(ctx context.Context, tx gdb.TX, iccid string, update *do.TSimCardInfo) error { + if strings.TrimSpace(iccid) == "" { + return gerror.New("renew card ICCID is empty") + } + if update == nil { + return gerror.New("renew card update is empty") + } + result, err := tx.Ctx(ctx).Model(TableSimCardInfo). + Where("iccid", strings.TrimSpace(iccid)). + Data(update). + Update() + if err != nil { + return gerror.Wrap(err, "update renew card failed") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read renew card update count failed") + } + if affected != 1 { + return gerror.Newf("renew card changed during execution: %s", iccid) + } + return nil +} diff --git a/internal/repository/renewchange/repository.go b/internal/repository/renewchange/repository.go new file mode 100644 index 0000000..a0a2a20 --- /dev/null +++ b/internal/repository/renewchange/repository.go @@ -0,0 +1,529 @@ +package renewchange + +import ( + "context" + "database/sql" + "errors" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" +) + +// Repository 提供对续费/变更相关表的 typed 访问。 +// 表名/列名都是常量（见 tables.go），与 Admin 同步链路保持一致。 +type Repository struct{} + +func New() *Repository { + return &Repository{} +} + +const simCardRenewChangeFields = "id,iccid,msisdn,imsi,dept_id,current_package_code,package_status,outbound_batch_no,flow_pkg_begin_at,flow_pkg_end_at,tenant_id,created_dept,updated_by,updated_at" + +func (r *Repository) database() gdb.DB { + return db() +} + +// DB 返回底层 gdb.DB，供 service 层发起事务。 +func (r *Repository) DB() gdb.DB { + return r.database() +} + +// LoadBatchMain 按 batchNo+batchType 加载。 +func (r *Repository) LoadBatchMain(ctx context.Context, batchNo string, batchType int) (*BatchMainRow, error) { + if batchNo == "" { + return nil, gerror.New("batch number is empty") + } + m := r.database().Model(TableBatchMain).Safe().Ctx(ctx). + Where("batch_no", batchNo). + Where("batch_type", batchType) + var out BatchMainRow + if err := m.Scan(&out); err != nil { + return nil, gerror.Wrap(err, "query batch main failed") + } + if out.Id == 0 { + return nil, nil + } + return &out, nil +} + +// LoadBatchDetails 加载指定批次下指定状态集合的明细。 +func (r *Repository) LoadBatchDetails(ctx context.Context, batchNo string, batchType int, statuses []int) ([]*BatchDetailRow, error) { + if batchNo == "" { + return nil, nil + } + m := r.database().Model(TableBatchDetail).Safe().Ctx(ctx). + Where("batch_no", batchNo). + Where("batch_type", batchType). + OrderAsc("row_no") + if len(statuses) > 0 { + m = m.WhereIn("detail_status", statuses) + } + var out []*BatchDetailRow + if err := m.Scan(&out); err != nil { + return nil, gerror.Wrap(err, "query batch details failed") + } + return out, nil +} + +func (r *Repository) LoadCardsByIdentityValues(ctx context.Context, tenantID string, values []string) ([]*SimCardInfo, error) { + uniq := uniqueStrings(values) + if len(uniq) == 0 { + return nil, nil + } + tenantID = strings.TrimSpace(tenantID) + byID := map[int64]*SimCardInfo{} + for _, column := range []string{"iccid", "msisdn", "imsi"} { + var rows []*SimCardInfo + query := r.database().Model(TableSimCardInfo).Safe().Ctx(ctx). + Fields(simCardRenewChangeFields). + WhereIn(column, uniq) + if tenantID != "" { + query = query.Where("tenant_id", tenantID) + } + if err := query.Scan(&rows); err != nil { + return nil, gerror.Wrap(err, "query sim cards by identity failed") + } + for _, row := range rows { + if row != nil && row.Id > 0 { + byID[row.Id] = row + } + } + } + out := make([]*SimCardInfo, 0, len(byID)) + for _, row := range byID { + out = append(out, row) + } + return out, nil +} + +// UpdateBatchMainStatus 在事务内更新主表。 +func (r *Repository) UpdateBatchMainStatus(ctx context.Context, tx gdb.TX, batchNo string, batchType int, data *BatchMainRow) error { + updateData := map[string]any{ + "batch_type": data.BatchType, + "batch_async_status": data.BatchAsyncStatus, + "run_result": data.RunResult, + "total_count": data.TotalCount, + "valid_count": data.ValidCount, + "success_count": data.SuccessCount, + "fail_count": data.FailCount, + "fail_reason": data.FailReason, + "batch_info": data.BatchInfo, + "end_time": data.EndTime, + "updated_by": data.UpdatedBy, + "updated_at": gtime.Now(), + "order_sn": data.OrderSn, + "target_dept_id": data.TargetDeptId, + } + if data.StartTime != nil && !data.StartTime.IsZero() { + updateData["start_time"] = data.StartTime + } + _, err := tx.Ctx(ctx).Model(TableBatchMain). + Where("batch_no", batchNo). + Where("batch_type", batchType). + Data(updateData). + Update() + return gerror.Wrap(err, "update batch main failed") +} + +// ConvertBatchDetails 将指定状态的明细转换为目标类型与状态。 +func (r *Repository) ConvertBatchDetails(ctx context.Context, tx gdb.TX, batchNo string, fromTypes []int, toType, toStatus int, data map[string]any) (int64, error) { + if data == nil { + data = map[string]any{} + } + data["batch_type"] = toType + data["detail_status"] = toStatus + data["updated_at"] = gtime.Now() + result, err := tx.Ctx(ctx).Model(TableBatchDetail). + Where("batch_no", batchNo). + WhereIn("batch_type", fromTypes). + Data(data). + Update() + if err != nil { + return 0, gerror.Wrap(err, "convert batch details failed") + } + affected, err := result.RowsAffected() + if err != nil { + return 0, err + } + return affected, nil +} + +// UpdateBatchDetailSnapshot 覆盖指定明细的 snapshot 与状态。 +func (r *Repository) UpdateBatchDetailSnapshot(ctx context.Context, tx gdb.TX, batchNo string, batchType, rowNo int, status int, snapshot *gjson.Json, reason string, userID int64) error { + data := map[string]any{ + "detail_status": status, + "snapshot": snapshot, + "updated_at": gtime.Now(), + "updated_by": userID, + } + if reason != "" { + data["error_reason"] = reason + } + _, err := tx.Ctx(ctx).Model(TableBatchDetail). + Where("batch_no", batchNo). + Where("batch_type", batchType). + Where("row_no", rowNo). + Data(data). + Update() + return gerror.Wrap(err, "update batch detail snapshot failed") +} + +// UpsertBatchDetail 按 (batch_no, batch_type, row_no) 写入明细（覆盖式）。 +func (r *Repository) UpsertBatchDetail(ctx context.Context, tx gdb.TX, detail *BatchDetailRow) error { + _, err := tx.Ctx(ctx).Model(TableBatchDetail). + Where("batch_no", detail.BatchNo). + Where("batch_type", detail.BatchType). + Where("row_no", detail.RowNo). + Data(map[string]any{ + "detail_status": detail.DetailStatus, + "blank_out_status": detail.BlankOutStatus, + "item_id": detail.ItemId, + "error_code": detail.ErrorCode, + "error_reason": detail.ErrorReason, + "snapshot": detail.Snapshot, + "updated_at": gtime.Now(), + "updated_by": detail.UpdatedBy, + }). + Update() + if err != nil { + return gerror.Wrap(err, "update batch detail failed") + } + return nil +} + +// LoadCardsByIccids 加载正式卡。 +func (r *Repository) LoadCardsByIccids(ctx context.Context, iccids []string) (map[string]*SimCardInfo, error) { + out := make(map[string]*SimCardInfo, len(iccids)) + uniq := uniqueStrings(iccids) + if len(uniq) == 0 { + return out, nil + } + var rows []*SimCardInfo + if err := r.database().Model(TableSimCardInfo).Safe().Ctx(ctx). + Fields(simCardRenewChangeFields). + WhereIn("iccid", uniq). + Scan(&rows); err != nil { + return nil, gerror.Wrap(err, "query sim cards failed") + } + for _, row := range rows { + if row == nil { + continue + } + row.Iccid = strings.TrimSpace(row.Iccid) + if row.Iccid == "" { + continue + } + out[row.Iccid] = row + } + return out, nil +} + +// LockCardsByIccids 在事务内按 ICCID 锁行加载正式卡。 +func (r *Repository) LockCardsByIccids(ctx context.Context, tx gdb.TX, iccids []string) (map[string]*SimCardInfo, error) { + out := make(map[string]*SimCardInfo, len(iccids)) + uniq := uniqueStrings(iccids) + if len(uniq) == 0 { + return out, nil + } + var rows []*SimCardInfo + if err := tx.Ctx(ctx).Model(TableSimCardInfo). + Fields(simCardRenewChangeFields). + WhereIn("iccid", uniq). + LockUpdate(). + Scan(&rows); err != nil { + return nil, gerror.Wrap(err, "lock sim cards failed") + } + for _, row := range rows { + if row == nil { + continue + } + row.Iccid = strings.TrimSpace(row.Iccid) + if row.Iccid == "" { + continue + } + out[row.Iccid] = row + } + return out, nil +} + +// LoadActiveBindRowsByIccids loads every pending/effective formal package +// subscription for prepare validation. Add-on package rows (103/104) are +// intentionally excluded from current-subscription selection. +func (r *Repository) LoadActiveBindRowsByIccids(ctx context.Context, iccids []string) ([]*SimCardBindPkg, error) { + uniq := uniqueStrings(iccids) + if len(uniq) == 0 { + return nil, nil + } + var rows []*SimCardBindPkg + if err := r.database().Ctx(ctx).Model(TableSimCardBindPkg). + WhereIn("iccid", uniq). + WhereIn("package_type", consts.FormalPackageTypes()). + WhereIn("effective_status", []int{consts.EffectiveStatusPending, consts.EffectiveStatusEffective}). + OrderDesc("effective_status"). + OrderAsc("effective_time"). + OrderAsc("id"). + Scan(&rows); err != nil { + return nil, gerror.Wrap(err, "load active binds failed") + } + return rows, nil +} + +// LoadRenewBindRowsByIccids includes expired formal-package history so renewal +// can restart after a card has naturally expired. Add-on package rows (103/104) +// never participate in renewal selection. +func (r *Repository) LoadRenewBindRowsByIccids(ctx context.Context, iccids []string) ([]*SimCardBindPkg, error) { + uniq := uniqueStrings(iccids) + if len(uniq) == 0 { + return nil, nil + } + var rows []*SimCardBindPkg + if err := r.database().Ctx(ctx).Model(TableSimCardBindPkg). + WhereIn("iccid", uniq). + WhereIn("package_type", consts.FormalPackageTypes()). + WhereIn("effective_status", []int{ + consts.EffectiveStatusPending, + consts.EffectiveStatusEffective, + consts.EffectiveStatusExpired, + }). + OrderDesc("effective_status"). + OrderAsc("effective_time"). + OrderAsc("id"). + Scan(&rows); err != nil { + return nil, gerror.Wrap(err, "load renew binds failed") + } + return rows, nil +} + +// LockActiveBindRowsByIccids is the transactional counterpart used by change +// execution before any business write. It locks formal package rows only. +func (r *Repository) LockActiveBindRowsByIccids(ctx context.Context, tx gdb.TX, iccids []string) ([]*SimCardBindPkg, error) { + uniq := uniqueStrings(iccids) + if len(uniq) == 0 { + return nil, nil + } + var rows []*SimCardBindPkg + if err := tx.Ctx(ctx).Model(TableSimCardBindPkg). + WhereIn("iccid", uniq). + WhereIn("package_type", consts.FormalPackageTypes()). + WhereIn("effective_status", []int{consts.EffectiveStatusPending, consts.EffectiveStatusEffective}). + OrderDesc("effective_status"). + OrderAsc("effective_time"). + OrderAsc("id"). + LockUpdate(). + Scan(&rows); err != nil { + return nil, gerror.Wrap(err, "lock active binds failed") + } + return rows, nil +} + +// LockRenewBindRowsByIccids includes expired formal-package history for renewal +// execution. Change execution keeps using LockActiveBindRowsByIccids. +func (r *Repository) LockRenewBindRowsByIccids(ctx context.Context, tx gdb.TX, iccids []string) ([]*SimCardBindPkg, error) { + uniq := uniqueStrings(iccids) + if len(uniq) == 0 { + return nil, nil + } + var rows []*SimCardBindPkg + if err := tx.Ctx(ctx).Model(TableSimCardBindPkg). + WhereIn("iccid", uniq). + WhereIn("package_type", consts.FormalPackageTypes()). + WhereIn("effective_status", []int{ + consts.EffectiveStatusPending, + consts.EffectiveStatusEffective, + consts.EffectiveStatusExpired, + }). + OrderDesc("effective_status"). + OrderAsc("effective_time"). + OrderAsc("id"). + LockUpdate(). + Scan(&rows); err != nil { + return nil, gerror.Wrap(err, "lock renew binds failed") + } + return rows, nil +} + +// LockPendingBindsByIccids 锁行加载 pending 状态的未来订购。 +func (r *Repository) LockPendingBindsByIccids(ctx context.Context, tx gdb.TX, iccids []string) ([]*SimCardBindPkg, error) { + uniq := uniqueStrings(iccids) + if len(uniq) == 0 { + return nil, nil + } + var rows []*SimCardBindPkg + if err := tx.Ctx(ctx).Model(TableSimCardBindPkg). + WhereIn("iccid", uniq). + WhereIn("package_type", consts.FormalPackageTypes()). + Where("effective_status", consts.EffectiveStatusPending). + OrderAsc("effective_time"). + OrderAsc("id"). + LockUpdate(). + Scan(&rows); err != nil { + return nil, gerror.Wrap(err, "lock pending binds failed") + } + return rows, nil +} + +// DeletePendingBindsBySubscriptionNos 删除（作废）一批订阅号。 +func (r *Repository) DeletePendingBindsBySubscriptionNos(ctx context.Context, tx gdb.TX, subscriptionNos []string) (int64, error) { + uniq := uniqueStrings(subscriptionNos) + if len(uniq) == 0 { + return 0, nil + } + result, err := tx.Ctx(ctx).Model(TableSimCardBindPkg). + WhereIn("subscription_no", uniq). + Delete() + if err != nil { + return 0, gerror.Wrap(err, "delete pending binds failed") + } + affected, err := result.RowsAffected() + if err != nil { + return 0, err + } + return affected, nil +} + +// LoadPackageByCode 按 packageCode 读取套餐。 +func (r *Repository) LoadPackageByCode(ctx context.Context, tenantID, packageCode string) (*FlowPackageInfo, error) { + if packageCode == "" { + return nil, errPkgNotFound + } + var row FlowPackageInfo + err := r.database().Model(TableSimFlowPackageInfo).Safe().Ctx(ctx). + Where("tenant_id", tenantID). + Where("package_code", strings.TrimSpace(packageCode)). + Scan(&row) + if err != nil { + return nil, gerror.Wrap(err, "query package failed") + } + if row.Id == 0 { + return nil, errPkgNotFound + } + return &row, nil +} + +// LoadDeptName 按 deptId 取客户名。 +func (r *Repository) LoadDeptName(ctx context.Context, tenantID string, deptID int64) (string, error) { + row, err := r.LoadDept(ctx, tenantID, deptID) + if err != nil { + return "", err + } + return strings.TrimSpace(row.DeptName), nil +} + +// LoadDept returns the current department and its parent. Renewals use the +// parent department as the price owner for the current card owner. +func (r *Repository) LoadDept(ctx context.Context, tenantID string, deptID int64) (*SysDeptRow, error) { + if deptID <= 0 { + return nil, errDeptNotFound + } + var row SysDeptRow + err := r.database().Model(TableSysDept).Safe().Ctx(ctx). + Where("tenant_id", tenantID). + Where("dept_id", deptID). + Scan(&row) + if err != nil { + return nil, gerror.Wrap(err, "query dept failed") + } + if row.DeptId == 0 { + return nil, errDeptNotFound + } + return &row, nil +} + +// LoadPriceOverride 读取代理续费/变更价格覆盖。 +func (r *Repository) LoadPriceOverride(ctx context.Context, tenantID string, deptID, targetAgentID int64, scopeType int, packageCode string) (*FlowPackagePrice, error) { + var row FlowPackagePrice + err := r.database().Model(TableSimFlowPackagePrice).Safe().Ctx(ctx). + Where("tenant_id", tenantID). + Where("dept_id", deptID). + Where("agent_scope_type", scopeType). + Where("target_agent_id", targetAgentID). + Where("package_code", strings.TrimSpace(packageCode)). + Where("enabled", 1). + Scan(&row) + return normalizePriceOverrideResult(&row, err) +} + +func normalizePriceOverrideResult(row *FlowPackagePrice, err error) (*FlowPackagePrice, error) { + if errors.Is(err, sql.ErrNoRows) { + return nil, nil + } + if err != nil { + return nil, gerror.Wrap(err, "query price override failed") + } + if row == nil || row.Id == 0 { + return nil, nil + } + return row, nil +} + +// UpdateCardPackageFields 续费/变更后更新卡的当前套餐与流量状态。 +func (r *Repository) UpdateCardPackageFields(ctx context.Context, tx gdb.TX, iccid string, fields map[string]any, userID int64) error { + if iccid == "" { + return gerror.New("iccid is empty") + } + fields["updated_by"] = userID + fields["updated_at"] = gtime.Now() + _, err := tx.Ctx(ctx).Model(TableSimCardInfo). + Where("iccid", iccid). + Data(fields). + Update() + return gerror.Wrap(err, "update sim card failed") +} + +// InsertOrder 插入订单（返回 orderSn）。 +// InsertOrderItem 插入订单项。 +// UpdateOrderReturnStatus 更新订单退订状态。 +// UpdateOrderItemExtra 合并并写入订单项 extraSkuData（JSON 字符串）。 +// ActiveBindCountByOrderCodes 统计订单当前未作废的订购数。 +func (r *Repository) ActiveBindCountByOrderCodes(ctx context.Context, tx gdb.TX, orderSNs []string) (map[string]int, error) { + out := make(map[string]int, len(orderSNs)) + uniq := uniqueStrings(orderSNs) + if len(uniq) == 0 { + return out, nil + } + type row struct { + OrderCode string `orm:"order_code"` + Count int `orm:"count"` + } + var rows []row + if err := tx.Ctx(ctx).Model(TableSimCardBindPkg). + Fields("order_code", "COUNT(1) AS count"). + WhereIn("order_code", uniq). + Group("order_code"). + Scan(&rows); err != nil { + return nil, gerror.Wrap(err, "count active binds failed") + } + for _, r := range rows { + out[strings.TrimSpace(r.OrderCode)] = r.Count + } + return out, nil +} + +// InsertBind 插入新的订购记录。 +func (r *Repository) InsertBind(ctx context.Context, tx gdb.TX, bind *SimCardBindPkg) error { + _, err := tx.Ctx(ctx).Model(TableSimCardBindPkg).Data(bind).Insert() + return gerror.Wrap(err, "insert bind failed") +} + +// InsertSysOss 插入 OSS 记录，返回 ID。 +func (r *Repository) InsertSysOss(ctx context.Context, tx gdb.TX, row *SysOssRow) (int64, error) { + id, err := tx.Ctx(ctx).Model(TableSysOss).Data(row).InsertAndGetId() + if err != nil { + return 0, gerror.Wrap(err, "insert sys oss failed") + } + return id, nil +} + +// InsertBatchDownload 插入 t_resource_batch_download 记录，返回 ID。 +func (r *Repository) InsertBatchDownload(ctx context.Context, tx gdb.TX, row *ResourceBatchDownloadRow) (int64, error) { + id, err := tx.Ctx(ctx).Model(TableTResourceBatchDl).Data(row).InsertAndGetId() + if err != nil { + return 0, gerror.Wrap(err, "insert batch download failed") + } + return id, nil +} diff --git a/internal/repository/renewchange/tables.go b/internal/repository/renewchange/tables.go new file mode 100644 index 0000000..288809d --- /dev/null +++ b/internal/repository/renewchange/tables.go @@ -0,0 +1,392 @@ +package renewchange + +import ( + "encoding/json" + "sort" + "strings" + "time" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" +) + +const ( + TableSimCardInfo = "t_sim_card_info" + TableSimCardBindPkg = "t_sim_card_bind_pkg" + TableSimPoolBindPackage = "t_sim_pool_bind_package" + TableSimBindPackageLogs = "t_sim_bind_package_logs" + TableOrder = "t_order" + TableOrderItem = "t_order_item" + TableSimFlowPackageInfo = "t_sim_flow_package_info" + TableSimFlowPackagePrice = "t_sim_flow_package_price" + TableSysDept = "sys_dept" + TableBatchMain = "batch_main" + TableBatchDetail = "batch_detail" + TableAgentBalance = "t_agent_balance" + TableBalanceDetail = "t_balance_detail" + TableSysOss = "sys_oss" + TableTResourceBatchDl = "t_resource_batch_download" +) + +// typed row structs（worker 端未生成 DAO/DO/entity，故使用本地 typed struct）。 +type SimCardInfo struct { + Id int64 `orm:"id"` + Iccid string `orm:"iccid"` + Msisdn string `orm:"msisdn"` + Imsi string `orm:"imsi"` + RegionCode string `orm:"region_code"` + DeptId int64 `orm:"dept_id"` + AllocationBatchNo string `orm:"allocation_batch_no"` + CurrentPackageCode string `orm:"current_package_code"` + PackageStatus int `orm:"package_status"` + ChanKey string `orm:"chan_key"` + OutboundBatchNo string `orm:"outbound_batch_no"` + InboundBatchNo string `orm:"inbound_batch_no"` + FlowPkgBeginAt *gtime.Time `orm:"flow_pkg_begin_at"` + FlowPkgPresentAt *gtime.Time `orm:"flow_pkg_present_at"` + FlowPkgRealPresent *gtime.Time `orm:"flow_pkg_real_present_at"` + FlowPkgEndAt *gtime.Time `orm:"flow_pkg_end_at"` + DataServiceStatus int `orm:"data_service_status"` + PackageTotalFlowMb float64 `orm:"package_total_flow_mb"` + UsedFlowMb float64 `orm:"used_flow_mb"` + TrafficCutoffEn int `orm:"traffic_cutoff_status"` + TrafficCutoffThr float64 `orm:"traffic_cutoff_threshold"` + SmsServiceStatus int `orm:"sms_service_status"` + PackageTotalSms int `orm:"package_total_sms"` + UsedSms int `orm:"used_sms"` + VoiceServiceStatus int `orm:"voice_service_status"` + PackageTotalVoice int `orm:"package_total_voice"` + UsedVoice int `orm:"used_voice"` + RoamServiceStatus int `orm:"roam_service_status"` + PkgTotalRoamFlowMb float64 `orm:"package_total_roam_flow_mb"` + UsedRoamFlowMb float64 `orm:"used_roam_flow_mb"` + TenantId string `orm:"tenant_id"` + CreatedDept int64 `orm:"created_dept"` + UpdatedBy int64 `orm:"updated_by"` + UpdatedAt *gtime.Time `orm:"updated_at"` +} + +type SimCardBindPkg struct { + Id int64 `orm:"id"` + BatchNo string `orm:"batch_no"` + Iccid string `orm:"iccid"` + Msisdn string `orm:"msisdn"` + DeptId int64 `orm:"dept_id"` + PackageType int `orm:"package_type"` + PackageCode string `orm:"package_code"` + PreviousPackageCode string `orm:"previous_package_code"` + BusinessType int `orm:"business_type"` + EffectiveStatus int `orm:"effective_status"` + HandleTime *gtime.Time `orm:"handle_time"` + EffectiveTime *gtime.Time `orm:"effective_time"` + ExpectedExpireTime *gtime.Time `orm:"expected_expire_time"` + ActualExpireTime *gtime.Time `orm:"actual_expire_time"` + DurationMonths int `orm:"duration_months"` + GiftDurationMonths int `orm:"gift_duration_months"` + OrderCode string `orm:"order_code"` + SubscriptionNo string `orm:"subscription_no"` + Remark string `orm:"remark"` + TenantId string `orm:"tenant_id"` + CreatedDept int64 `orm:"created_dept"` + CreatedBy int64 `orm:"created_by"` + CreatedAt *gtime.Time `orm:"created_at"` + UpdatedBy int64 `orm:"updated_by"` + UpdatedAt *gtime.Time `orm:"updated_at"` +} + +type SimBindPackageLogs struct { + Id int64 `orm:"id"` + SubscriptionNo string `orm:"subscription_no"` + BatchNo string `orm:"batch_no"` + Iccid string `orm:"iccid"` + Msisdn string `orm:"msisdn"` + DeptId int64 `orm:"dept_id"` + PackageType int `orm:"package_type"` + PackageCode string `orm:"package_code"` + PreviousPackageCod string `orm:"previous_package_code"` + BusinessType int `orm:"business_type"` + EffectiveStatus int `orm:"effective_status"` + HandleTime *gtime.Time `orm:"handle_time"` + EffectiveTime *gtime.Time `orm:"effective_time"` + ExpectedExpireTime *gtime.Time `orm:"expected_expire_time"` + ActualExpireTime *gtime.Time `orm:"actual_expire_time"` + DurationMonths int `orm:"duration_months"` + GiftDurationMonths int `orm:"gift_duration_months"` + OrderCode string `orm:"order_code"` + Remark string `orm:"remark"` + TenantId string `orm:"tenant_id"` + CreatedDept int64 `orm:"created_dept"` + CreatedBy int64 `orm:"created_by"` + CreatedAt *gtime.Time `orm:"created_at"` + UpdatedBy int64 `orm:"updated_by"` + UpdatedAt *gtime.Time `orm:"updated_at"` +} + +type FlowPackageInfo struct { + Id int64 `orm:"id"` + PackageCode string `orm:"package_code"` + PoolPackageCode int64 `orm:"pool_package_code"` + PackageName string `orm:"package_name"` + PackageType int `orm:"package_type"` + BillingMethod int `orm:"billing_method"` + FlowMb float64 `orm:"flow_mb"` + CostPrice float64 `orm:"cost_price"` + SysDefaultRatio float64 `orm:"sys_default_ratio"` + SysDefaultProfit float64 `orm:"sys_default_profit"` + RenewableDurationConfig *gjson.Json `orm:"renewable_duration_config"` + RenewablePackageCodes *gjson.Json `orm:"renewable_package_codes"` + ChangeablePackageCodes *gjson.Json `orm:"changeable_package_codes"` + TopupPackageCodes *gjson.Json `orm:"topup_package_codes"` + Enabled int `orm:"enabled"` + TenantId string `orm:"tenant_id"` +} + +type PoolBindPackageLogs struct { + Id int64 `orm:"id"` + SubscriptionNo string `orm:"subscription_no"` + BatchNo string `orm:"batch_no"` + PoolNo string `orm:"pool_no"` + DeptId int64 `orm:"dept_id"` + PackageType int `orm:"package_type"` + PackageCode string `orm:"package_code"` + PreviousPackageCode string `orm:"previous_package_code"` + BusinessType int `orm:"business_type"` + EffectiveStatus int `orm:"effective_status"` + HandleTime *gtime.Time `orm:"handle_time"` + EffectiveTime *gtime.Time `orm:"effective_time"` + ExpectedExpireTime *gtime.Time `orm:"expected_expire_time"` + ActualExpireTime *gtime.Time `orm:"actual_expire_time"` + DurationMonths int `orm:"duration_months"` + GiftDurationMonths int `orm:"gift_duration_months"` + OrderCode string `orm:"order_code"` + Remark string `orm:"remark"` + TenantId string `orm:"tenant_id"` + CreatedDept int64 `orm:"created_dept"` + CreatedBy int64 `orm:"created_by"` + CreatedAt *gtime.Time `orm:"created_at"` + UpdatedBy int64 `orm:"updated_by"` + UpdatedAt *gtime.Time `orm:"updated_at"` +} + +type FlowPackagePrice struct { + Id int64 `orm:"id"` + TenantId string `orm:"tenant_id"` + DeptId int64 `orm:"dept_id"` + AgentScopeType int `orm:"agent_scope_type"` + TargetAgentId int64 `orm:"target_agent_id"` + PackageCode string `orm:"package_code"` + AgentCostRatio float64 `orm:"agent_cost_ratio"` + AgentCostProfit float64 `orm:"agent_cost_profit"` +} + +type SysDeptRow struct { + DeptId int64 `orm:"dept_id"` + ParentId int64 `orm:"parent_id"` + DeptName string `orm:"dept_name"` + TenantId string `orm:"tenant_id"` +} + +type BatchMainRow struct { + Id int64 `orm:"id"` + BatchNo string `orm:"batch_no"` + OrderSn string `orm:"order_sn"` + BatchType int `orm:"batch_type"` + BatchStatus int `orm:"batch_status"` + BatchAsyncStatus int `orm:"batch_async_status"` + SourceType int `orm:"source_type"` + SourceBatchNo string `orm:"source_batch_no"` + IsCrossDept int `orm:"is_cross_dept"` + TargetDeptId int64 `orm:"target_dept_id"` + DeptId int64 `orm:"dept_id"` + TotalCount int `orm:"total_count"` + ValidCount int `orm:"valid_count"` + SuccessCount int `orm:"success_count"` + FailCount int `orm:"fail_count"` + RunResult int `orm:"run_result"` + FailReason string `orm:"fail_reason"` + BatchInfo *gjson.Json `orm:"batch_info"` + StartTime *gtime.Time `orm:"start_time"` + EndTime *gtime.Time `orm:"end_time"` + TenantId string `orm:"tenant_id"` + CreatedDept int64 `orm:"created_dept"` + CreatedBy int64 `orm:"created_by"` + CreatedAt *gtime.Time `orm:"created_at"` + UpdatedBy int64 `orm:"updated_by"` + UpdatedAt *gtime.Time `orm:"updated_at"` +} + +type BatchDetailRow struct { + Id int64 `orm:"id"` + BatchNo string `orm:"batch_no"` + BatchType int `orm:"batch_type"` + DetailStatus int `orm:"detail_status"` + BlankOutStatus int `orm:"blank_out_status"` + RowNo int `orm:"row_no"` + ItemId string `orm:"item_id"` + ErrorCode string `orm:"error_code"` + ErrorReason string `orm:"error_reason"` + Snapshot *gjson.Json `orm:"snapshot"` + TenantId string `orm:"tenant_id"` + CreatedDept int64 `orm:"created_dept"` + CreatedBy int64 `orm:"created_by"` + CreatedAt *gtime.Time `orm:"created_at"` + UpdatedBy int64 `orm:"updated_by"` + UpdatedAt *gtime.Time `orm:"updated_at"` +} + +type SysOssRow struct { + Id int64 `orm:"id"` + TenantId string `orm:"tenant_id"` + FileName string `orm:"file_name"` + OriginalName string `orm:"original_name"` + FileSuffix string `orm:"file_suffix"` + Path string `orm:"path"` + Url string `orm:"url"` + Service string `orm:"service"` + Md5 string `orm:"md5"` + FileSize int64 `orm:"file_size"` + CreatedDept int64 `orm:"created_dept"` + CreatedBy int64 `orm:"created_by"` + UpdatedBy int64 `orm:"updated_by"` +} + +type ResourceBatchDownloadRow struct { + Id int64 `orm:"id"` + BatchNo string `orm:"batch_no"` + TaskName string `orm:"task_name"` + ResourceType int `orm:"resource_type"` + ExportScene int `orm:"export_scene"` + TotalCount int `orm:"total_count"` + SuccessCount int `orm:"success_count"` + FailCount int `orm:"fail_count"` + DownloadStatus int `orm:"download_status"` + OssId int64 `orm:"oss_id"` + FileName string `orm:"file_name"` + FileExt string `orm:"file_ext"` + FileUrl string `orm:"file_url"` + FilePath string `orm:"file_path"` + FileSize int64 `orm:"file_size"` + FileMd5 string `orm:"file_md5"` + DownloadCount int `orm:"download_count"` + MaxDownloadCount int `orm:"max_download_count"` + DeptId int64 `orm:"dept_id"` + StartTime *gtime.Time `orm:"start_time"` + FinishTime *gtime.Time `orm:"finish_time"` + TenantId string `orm:"tenant_id"` + CreatedDept int64 `orm:"created_dept"` + CreatedBy int64 `orm:"created_by"` + UpdatedBy int64 `orm:"updated_by"` +} + +// DB 默认从 g.DB() 取，避免和现有 repository 紧耦合。 +func DB() gdb.DB { + return db() +} + +func db() gdb.DB { + return g.DB() +} + +func uniqueStrings(in []string) []string { + if len(in) == 0 { + return nil + } + seen := make(map[string]struct{}, len(in)) + out := make([]string, 0, len(in)) + for _, v := range in { + v = strings.TrimSpace(v) + if v == "" { + continue + } + if _, ok := seen[v]; ok { + continue + } + seen[v] = struct{}{} + out = append(out, v) + } + return out +} + +// RoundAmount 四舍五入到分（导出以便 service 层复用）。 +func RoundAmount(value float64) float64 { + return roundAmount(value) +} + +func roundAmount(value float64) float64 { + if value < 0 { + return 0 + } + return float64(int(value*100+0.5)) / 100 +} + +// RenewableDurationOptions 解析套餐的可续费时长配置。 +func RenewableDurationOptions(jsonValue *gjson.Json) ([]int, error) { + return renewableDurationOptions(jsonValue) +} + +func renewableDurationOptions(jsonValue *gjson.Json) ([]int, error) { + if jsonValue == nil || jsonValue.IsNil() { + return nil, errRenewDurationEmpty + } + values := jsonValue.Interfaces() + out := make([]int, 0, len(values)) + seen := make(map[int]struct{}, len(values)) + for _, raw := range values { + var month int + switch v := raw.(type) { + case int: + month = v + case int32: + month = int(v) + case int64: + month = int(v) + case float32: + month = int(v) + case float64: + month = int(v) + case json.Number: + n, err := v.Int64() + if err != nil { + continue + } + month = int(n) + case string: + if t, err := time.Parse("2006-01-02", strings.TrimSpace(v)); err == nil { + // tolerate date strings, fall back to ignoring + _ = t + } + continue + default: + continue + } + if month <= 0 { + continue + } + if _, ok := seen[month]; ok { + continue + } + seen[month] = struct{}{} + out = append(out, month) + } + sort.Ints(out) + if len(out) == 0 { + return nil, errRenewDurationEmpty + } + return out, nil +} + +// ContainsInt 检查切片中是否包含目标值。 +func ContainsInt(values []int, target int) bool { + return containsInt(values, target) +} + +func containsInt(values []int, target int) bool { + for _, v := range values { + if v == target { + return true + } + } + return false +} diff --git a/internal/repository/renewchange/tables_test.go b/internal/repository/renewchange/tables_test.go new file mode 100644 index 0000000..a231d7a --- /dev/null +++ b/internal/repository/renewchange/tables_test.go @@ -0,0 +1,29 @@ +package renewchange + +import ( + "encoding/json" + "testing" + + "github.com/gogf/gf/v2/encoding/gjson" +) + +func TestRenewableDurationOptionsAcceptsJsonNumber(t *testing.T) { + got, err := RenewableDurationOptions(gjson.New([]any{ + json.Number("3"), + json.Number("1"), + json.Number("2"), + })) + if err != nil { + t.Fatalf("RenewableDurationOptions returned error: %v", err) + } + + want := []int{1, 2, 3} + if len(got) != len(want) { + t.Fatalf("options = %v, want %v", got, want) + } + for i := range want { + if got[i] != want[i] { + t.Fatalf("options = %v, want %v", got, want) + } + } +} diff --git a/internal/repository/simstatus/repository.go b/internal/repository/simstatus/repository.go new file mode 100644 index 0000000..d45bc0b --- /dev/null +++ b/internal/repository/simstatus/repository.go @@ -0,0 +1,49 @@ +package simstatus + +import ( + "context" + "fmt" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" +) + +// Result is the database projection of a successful status query. +type Result struct { + CardID int64 + CardStatus int +} + +type Repository struct{} + +func NewRepository() *Repository { return &Repository{} } + +// ApplyBatch updates all successful results in one SQL statement. The status +// timestamp is written only for cards whose upstream query succeeded. +func (r *Repository) ApplyBatch(ctx context.Context, results []Result) error { + if len(results) == 0 { + return nil + } + ids := make([]int64, 0, len(results)) + var statement strings.Builder + statement.WriteString("CASE " + dao.TSimCardInfo.Columns().Id + " ") + for _, result := range results { + if result.CardID <= 0 { + return fmt.Errorf("status result has invalid cardId %d", result.CardID) + } + fmt.Fprintf(&statement, "WHEN %d THEN %d ", result.CardID, result.CardStatus) + ids = append(ids, result.CardID) + } + statement.WriteString("END") + _, err := dao.TSimCardInfo.Ctx(ctx). + WhereIn(dao.TSimCardInfo.Columns().Id, ids). + Data(do.TSimCardInfo{ + BossCardStatus: gdb.Raw(statement.String()), + StatusSyncTime: gtime.Now(), + }).Update() + return err +} diff --git a/internal/repository/trafficcutoff/repository.go b/internal/repository/trafficcutoff/repository.go new file mode 100644 index 0000000..60acdbc --- /dev/null +++ b/internal/repository/trafficcutoff/repository.go @@ -0,0 +1,749 @@ +package trafficcutoff + +import ( + "context" + "sort" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" + trafficcutoff "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/trafficcutoff" +) + +const ( + resultFileStatusSuccess = "success" + resultFileStatusFailed = "failed" + + validationFailureReason = "batch validation failed; traffic cutoff update skipped" +) + +type Repository struct { + db gdb.DB + resultFiles batchresult.Generator + lockEnabled bool +} + +func New() *Repository { + return &Repository{resultFiles: batchresult.New(), lockEnabled: true} +} + +func (r *Repository) database() gdb.DB { + if r.db != nil { + return r.db + } + return g.DB() +} + +func (r *Repository) applyLock(model *gdb.Model) *gdb.Model { + if r.lockEnabled { + return model.LockUpdate() + } + return model +} + +func (r *Repository) Load(ctx context.Context, batchNo string) (*trafficcutoff.Task, error) { + batchNo = strings.TrimSpace(batchNo) + if batchNo == "" { + return nil, gerror.New("traffic cutoff batch number is empty") + } + + var main entity.BatchMain + if err := r.database().Model(dao.BatchMain.Table()).Safe().Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeTrafficCutoff). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Scan(&main); err != nil { + return nil, gerror.Wrap(err, "query traffic cutoff batch main") + } + if main.Id == 0 { + return nil, gerror.New("processing traffic cutoff batch does not exist") + } + + var rawDetails []*entity.BatchDetail + if err := r.database().Model(dao.BatchDetail.Table()).Safe().Ctx(ctx). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeTrafficCutoff). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + OrderAsc(dao.BatchDetail.Columns().RowNo). + Scan(&rawDetails); err != nil { + return nil, gerror.Wrap(err, "query running traffic cutoff details") + } + if len(rawDetails) == 0 { + return nil, gerror.New("traffic cutoff batch has no running details") + } + trafficCutoffStatus, err := readTrafficCutoffStatus(main.BatchInfo) + if err != nil { + return nil, err + } + + task := &trafficcutoff.Task{ + Batch: trafficcutoff.Batch{ + ID: main.Id, + BatchNo: main.BatchNo, + BatchType: main.BatchType, + TenantID: main.TenantId, + DeptID: main.DeptId, + CreatedBy: main.CreatedBy, + CreatedDept: main.CreatedDept, + StartTime: main.StartTime, + BatchInfo: main.BatchInfo, + }, + TrafficCutoffStatus: trafficCutoffStatus, + TrafficCutoffThreshold: readTrafficCutoffThreshold(main.BatchInfo), + ThresholdUnit: readThresholdUnit(main.BatchInfo), + Details: make([]*trafficcutoff.Detail, 0, len(rawDetails)), + } + for _, raw := range rawDetails { + if raw == nil { + continue + } + task.Details = append(task.Details, buildDetail(raw)) + } + if len(task.Details) == 0 { + return nil, gerror.New("traffic cutoff batch has no readable running details") + } + if err := r.attachCards(ctx, task); err != nil { + return nil, err + } + return task, nil +} + +func buildDetail(raw *entity.BatchDetail) *trafficcutoff.Detail { + detail := &trafficcutoff.Detail{ + ID: raw.Id, + RowNo: raw.RowNo, + ICCID: strings.TrimSpace(raw.ItemId), + Reason: strings.TrimSpace(raw.ErrorReason), + } + if raw.Snapshot != nil { + if value := firstSnapshot(raw.Snapshot, "ICCID", "iccid"); value != "" { + detail.ICCID = value + } + detail.InputMSISDN = firstSnapshot(raw.Snapshot, "MSISDN", "msisdn", "inputMsisdn", "input_msisdn") + detail.InputIMSI = firstSnapshot(raw.Snapshot, "IMSI", "imsi", "inputImsi", "input_imsi") + detail.TrafficCutoffCrossMonthHold = raw.Snapshot.Get("trafficCutoffCrossMonthHold").Int() + detail.MSISDN = detail.InputMSISDN + detail.IMSI = detail.InputIMSI + detail.Authorized = raw.Snapshot.Get("authorized").Bool() + } + return detail +} + +func firstSnapshot(snapshot *gjson.Json, keys ...string) string { + if snapshot == nil { + return "" + } + for _, key := range keys { + if value := strings.TrimSpace(snapshot.Get(key).String()); value != "" { + return value + } + } + return "" +} + +func (r *Repository) attachCards(ctx context.Context, task *trafficcutoff.Task) error { + cards, err := r.loadCards(ctx, task.Batch.TenantID, detailICCIDList(task.Details)) + if err != nil { + return err + } + attachCards(task, cards) + return nil +} + +func (r *Repository) loadCards(ctx context.Context, tenantID string, iccids []string) (map[string]*entity.TSimCardInfo, error) { + result := make(map[string]*entity.TSimCardInfo) + iccids = uniqueSortedStrings(iccids) + if len(iccids) == 0 { + return result, nil + } + var cards []*entity.TSimCardInfo + if err := r.database().Model(dao.TSimCardInfo.Table()).Safe().Ctx(ctx). + Where(dao.TSimCardInfo.Columns().TenantId, tenantID). + WhereIn(dao.TSimCardInfo.Columns().Iccid, iccids). + Scan(&cards); err != nil { + return nil, gerror.Wrap(err, "query traffic cutoff SIM cards") + } + for _, card := range cards { + if card != nil && strings.TrimSpace(card.Iccid) != "" { + result[strings.TrimSpace(card.Iccid)] = card + } + } + return result, nil +} + +func attachCards(task *trafficcutoff.Task, cards map[string]*entity.TSimCardInfo) { + for _, detail := range task.Details { + if detail == nil { + continue + } + card := cards[strings.TrimSpace(detail.ICCID)] + if card == nil { + detail.Card = nil + continue + } + detail.Card = &trafficcutoff.Card{ + ID: card.Id, + ICCID: strings.TrimSpace(card.Iccid), + MSISDN: strings.TrimSpace(card.Msisdn), + IMSI: strings.TrimSpace(card.Imsi), + BossCardStatus: card.BossCardStatus, + TrafficCutoffStatus: card.TrafficCutoffStatus, + TrafficCutoffThreshold: card.TrafficCutoffThreshold, + } + detail.MSISDN = detail.Card.MSISDN + detail.IMSI = detail.Card.IMSI + detail.BossCardStatusBefore = detail.Card.BossCardStatus + detail.TrafficCutoffStatusBefore = detail.Card.TrafficCutoffStatus + detail.TrafficCutoffThresholdBefore = detail.Card.TrafficCutoffThreshold + } +} + +func (r *Repository) SaveValidationFailure(ctx context.Context, task *trafficcutoff.Task) error { + if task == nil { + return gerror.New("traffic cutoff task is nil") + } + reason := strings.TrimSpace(task.FailureReason) + if reason == "" { + reason = validationFailureReason + } + now := gtime.Now() + return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + if err := updateDetailsFailed(ctx, tx, task, reason); err != nil { + return err + } + return updateMainFailed(ctx, tx, task, len(task.Details), reason, now) + }) +} + +func (r *Repository) CommitBusiness(ctx context.Context, task *trafficcutoff.Task) error { + if task == nil { + return gerror.New("traffic cutoff task is nil") + } + now := gtime.Now() + iccids := uniqueSortedStrings(detailICCIDList(task.Details)) + return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + cards, err := r.lockCards(ctx, tx, task.Batch.TenantID, iccids) + if err != nil { + return gerror.Wrap(err, "lock traffic cutoff SIM cards") + } + attachCards(task, cards) + if !validateLockedTask(task) { + reason := strings.TrimSpace(task.FailureReason) + if reason == "" { + reason = validationFailureReason + } + if err = updateDetailsFailed(ctx, tx, task, reason); err != nil { + return err + } + return updateMainFailed(ctx, tx, task, len(task.Details), reason, now) + } + + ids := detailCardIDs(task.Details) + result, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()). + Where(dao.TSimCardInfo.Columns().TenantId, task.Batch.TenantID). + WhereIn(dao.TSimCardInfo.Columns().Id, ids). + Data(do.TSimCardInfo{ + TrafficCutoffStatus: task.TrafficCutoffStatus, + TrafficCutoffThreshold: task.TrafficCutoffThreshold, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update traffic cutoff fields") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read traffic cutoff card update result") + } + if int(affected) != len(ids) { + return gerror.New("some traffic cutoff cards changed before update") + } + if err = markCrossMonthCutoffTriggered(ctx, tx, task); err != nil { + return err + } + for _, detail := range task.Details { + if detail == nil { + continue + } + detail.TrafficCutoffStatusAfter = task.TrafficCutoffStatus + detail.TrafficCutoffThresholdAfter = task.TrafficCutoffThreshold + } + if err = updateDetailsSuccess(ctx, tx, task); err != nil { + return err + } + return updateMainSuccess(ctx, tx, task, len(task.Details), now) + }) +} + +func markCrossMonthCutoffTriggered(ctx context.Context, tx gdb.TX, task *trafficcutoff.Task) error { + ids := make([]int64, 0, len(task.Details)) + for _, detail := range task.Details { + if detail != nil && detail.Reason == "" && detail.TrafficCutoffCrossMonthHold == 1 && detail.Card != nil { + ids = append(ids, detail.Card.ID) + } + } + ids = uniqueSortedInt64(ids) + if len(ids) == 0 { + return nil + } + result, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()). + Where(dao.TSimCardInfo.Columns().TenantId, task.Batch.TenantID). + WhereIn(dao.TSimCardInfo.Columns().Id, ids). + Where("traffic_cutoff_cross_month_hold", 1). + Data(do.TSimCardTrafficCutoffConfig{ + TrafficCutoffTriggeredAt: gtime.Now(), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update traffic cutoff triggered time") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read traffic cutoff triggered time update result") + } + if int(affected) != len(ids) { + return gerror.New("some traffic cutoff triggered times changed before update") + } + return nil +} + +func (r *Repository) lockCards(ctx context.Context, tx gdb.TX, tenantID string, iccids []string) (map[string]*entity.TSimCardInfo, error) { + result := make(map[string]*entity.TSimCardInfo) + if len(iccids) == 0 { + return result, nil + } + var cards []*entity.TSimCardInfo + if err := r.applyLock(tx.Ctx(ctx).Model(dao.TSimCardInfo.Table())). + Where(dao.TSimCardInfo.Columns().TenantId, tenantID). + WhereIn(dao.TSimCardInfo.Columns().Iccid, iccids). + Scan(&cards); err != nil { + return nil, err + } + for _, card := range cards { + if card != nil && strings.TrimSpace(card.Iccid) != "" { + result[strings.TrimSpace(card.Iccid)] = card + } + } + return result, nil +} + +func validateLockedTask(task *trafficcutoff.Task) bool { + if task == nil || len(task.Details) == 0 { + if task != nil { + task.FailureReason = "traffic cutoff batch has no valid details" + } + return false + } + if task.TrafficCutoffStatus != 1 || task.TrafficCutoffThreshold <= 0 { + task.FailureReason = validationFailureReason + for _, detail := range task.Details { + if detail != nil { + detail.Reason = validationFailureReason + } + } + return false + } + hasFailure := false + statusSet := make(map[int]struct{}) + for _, detail := range task.Details { + if detail == nil { + hasFailure = true + continue + } + detail.Reason = "" + if detail.Card == nil { + detail.Reason = "ICCID does not exist in SIM card records" + } else if !detail.Authorized { + detail.Reason = "ICCID is outside the current operator authorization scope" + } else if detail.InputMSISDN != "" && detail.InputMSISDN != detail.Card.MSISDN { + detail.Reason = "MSISDN does not match the SIM card record" + } else if detail.InputIMSI != "" && detail.InputIMSI != detail.Card.IMSI { + detail.Reason = "IMSI does not match the SIM card record" + } else if detail.Card.BossCardStatus == trafficcutoff.BossCardStatusCanceled { + detail.Reason = "ICCID is already canceled" + } + if detail.Reason != "" { + hasFailure = true + continue + } + statusSet[detail.Card.BossCardStatus] = struct{}{} + } + if len(statusSet) > 1 { + for _, detail := range task.Details { + if detail != nil && detail.Reason == "" { + detail.Reason = "same batch contains different card statuses" + } + } + hasFailure = true + } + if !hasFailure { + task.ValidationOK = true + return true + } + for _, detail := range task.Details { + if detail != nil && detail.Reason == "" { + detail.Reason = "same batch has invalid traffic cutoff rows; update skipped" + } + } + task.FailureReason = validationFailureReason + return false +} + +func updateDetailsFailed(ctx context.Context, tx gdb.TX, task *trafficcutoff.Task, fallback string) error { + for _, detail := range task.Details { + if detail == nil { + continue + } + if strings.TrimSpace(detail.Reason) == "" { + detail.Reason = fallback + } + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeTrafficCutoff). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: consts.BatchDetailStatusFailed, + ErrorReason: detail.Reason, + Snapshot: detail.Snapshot(task), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update traffic cutoff failed detail") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read traffic cutoff failed detail result") + } + if affected != 1 { + return gerror.Newf("traffic cutoff detail %d is no longer running", detail.ID) + } + } + return nil +} + +func updateDetailsSuccess(ctx context.Context, tx gdb.TX, task *trafficcutoff.Task) error { + for _, detail := range task.Details { + if detail == nil { + continue + } + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeTrafficCutoff). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: consts.BatchDetailStatusSuccess, + ErrorReason: "", + Snapshot: detail.Snapshot(task), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update traffic cutoff success detail") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read traffic cutoff success detail result") + } + if affected != 1 { + return gerror.Newf("traffic cutoff detail %d is no longer running", detail.ID) + } + } + return nil +} + +func updateMainFailed(ctx context.Context, tx gdb.TX, task *trafficcutoff.Task, total int, reason string, now *gtime.Time) error { + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeTrafficCutoff). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Data(do.BatchMain{ + BatchAsyncStatus: consts.BatchAsyncStatusFailed, + TotalCount: total, + ValidCount: total, + SuccessCount: 0, + FailCount: total, + RunResult: consts.BatchRunResultFailed, + FailReason: reason, + BatchInfo: buildBatchInfo(task, resultFileStatusFailed, 0, 0, ""), + EndTime: now, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update traffic cutoff batch main failed") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read traffic cutoff failed main result") + } + if affected != 1 { + return gerror.New("traffic cutoff batch is no longer processing") + } + task.Batch.BatchInfo = buildBatchInfo(task, resultFileStatusFailed, 0, 0, "") + return nil +} + +func updateMainSuccess(ctx context.Context, tx gdb.TX, task *trafficcutoff.Task, total int, now *gtime.Time) error { + info := buildBatchInfo(task, "pending", 0, 0, "") + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeTrafficCutoff). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Data(do.BatchMain{ + BatchAsyncStatus: consts.BatchAsyncStatusSuccess, + TotalCount: total, + ValidCount: total, + SuccessCount: total, + FailCount: 0, + RunResult: consts.BatchRunResultSuccess, + FailReason: "", + BatchInfo: info, + EndTime: now, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update traffic cutoff batch main success") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read traffic cutoff success main result") + } + if affected != 1 { + return gerror.New("traffic cutoff batch is no longer processing") + } + task.Batch.BatchInfo = info + return nil +} + +func (r *Repository) SaveResultFile(ctx context.Context, task *trafficcutoff.Task, request batchresult.Request) error { + if task == nil { + return gerror.New("traffic cutoff task is nil") + } + artifact, err := r.artifactGenerator().Create(ctx, request) + if err != nil { + return r.updateResultFileStatus(ctx, task, resultFileStatusFailed, 0, 0, err.Error()) + } + now := gtime.Now() + var ( + ossID int64 + downloadID int64 + ) + txErr := r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + var insertErr error + ossID, insertErr = tx.Ctx(ctx).Model(dao.SysOss.Table()).Data(do.SysOss{ + TenantId: task.Batch.TenantID, + FileName: artifact.FileName, + OriginalName: artifact.OriginalName, + FileSuffix: artifact.FileSuffix, + Path: artifact.Path, + Url: artifact.URL, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + Service: "local", + Md5: artifact.MD5, + FileSize: artifact.Size, + }).InsertAndGetId() + if insertErr != nil { + return gerror.Wrap(insertErr, "insert traffic cutoff result OSS") + } + downloadID, insertErr = tx.Ctx(ctx).Model(dao.TResourceBatchDownload.Table()).Data(do.TResourceBatchDownload{ + BatchNo: task.Batch.BatchNo, + TaskName: batchresult.TaskName(task.Batch.BatchType), + ResourceType: consts.ResourceTypeExcel, + ExportScene: task.Batch.BatchType, + TotalCount: len(task.Details), + SuccessCount: task.SuccessCount(), + FailCount: task.FailCount(), + DownloadStatus: consts.DownloadStatusGenerated, + OssId: ossID, + FileName: artifact.FileName, + FileExt: artifact.FileSuffix, + FileUrl: artifact.URL, + FilePath: artifact.Path, + FileSize: artifact.Size, + FileMd5: artifact.MD5, + DownloadCount: 0, + MaxDownloadCount: 0, + DeptId: task.Batch.CreatedDept, + StartTime: task.Batch.StartTime, + FinishTime: now, + TenantId: task.Batch.TenantID, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + }).InsertAndGetId() + if insertErr != nil { + return gerror.Wrap(insertErr, "insert traffic cutoff result download") + } + return updateBatchInfoOnly(ctx, tx, task, buildBatchInfo(task, resultFileStatusSuccess, ossID, downloadID, "")) + }) + if txErr != nil { + batchresult.RemoveLocalFile(artifact) + return r.updateResultFileStatus(ctx, task, resultFileStatusFailed, 0, 0, txErr.Error()) + } + return nil +} + +func (r *Repository) artifactGenerator() batchresult.Generator { + if r.resultFiles != nil { + return r.resultFiles + } + return batchresult.New() +} + +func (r *Repository) updateResultFileStatus(ctx context.Context, task *trafficcutoff.Task, status string, ossID, downloadID int64, reason string) error { + return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + return updateBatchInfoOnly(ctx, tx, task, buildBatchInfo(task, status, ossID, downloadID, reason)) + }) +} + +func updateBatchInfoOnly(ctx context.Context, tx gdb.TX, task *trafficcutoff.Task, info *gjson.Json) error { + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeTrafficCutoff). + Data(do.BatchMain{ + BatchInfo: info, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update traffic cutoff result file status") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read traffic cutoff result file update result") + } + if affected != 1 { + return gerror.New("traffic cutoff batch disappeared while saving result file") + } + task.Batch.BatchInfo = info + return nil +} + +func buildBatchInfo(task *trafficcutoff.Task, resultStatus string, ossID, downloadID int64, reason string) *gjson.Json { + info := make(map[string]any) + if task.Batch.BatchInfo != nil { + for key, value := range task.Batch.BatchInfo.Map() { + info[key] = value + } + } + info["operation"] = "dataCut" + info["trafficCutoffStatus"] = task.TrafficCutoffStatus + info["trafficCutoffThreshold"] = task.TrafficCutoffThreshold + info["thresholdUnit"] = task.ThresholdUnit + if resultStatus != "" { + resultFile := map[string]any{"status": resultStatus} + if ossID > 0 { + resultFile["ossId"] = ossID + } + if downloadID > 0 { + resultFile["downloadId"] = downloadID + } + if strings.TrimSpace(reason) != "" { + resultFile["error"] = reason + } + info["resultFile"] = resultFile + } + return gjson.New(info) +} + +func readTrafficCutoffStatus(info *gjson.Json) (int, error) { + if info == nil || info.IsNil() { + return 0, gerror.New("traffic cutoff batch missing trafficCutoffStatus") + } + if value := info.Get("trafficCutoffStatus"); !value.IsNil() { + return value.Int(), nil + } + return 0, gerror.New("traffic cutoff batch missing trafficCutoffStatus") +} + +func readTrafficCutoffThreshold(info *gjson.Json) float64 { + if info == nil || info.IsNil() { + return 0 + } + if value := info.Get("trafficCutoffThreshold"); !value.IsNil() { + return value.Float64() + } + return info.Get("cutoffThreshold").Float64() +} + +func readThresholdUnit(info *gjson.Json) string { + if info != nil { + if value := strings.TrimSpace(info.Get("thresholdUnit").String()); value != "" { + return value + } + } + return "MB" +} + +func detailICCIDList(details []*trafficcutoff.Detail) []string { + iccids := make([]string, 0, len(details)) + for _, detail := range details { + if detail != nil && strings.TrimSpace(detail.ICCID) != "" { + iccids = append(iccids, strings.TrimSpace(detail.ICCID)) + } + } + return iccids +} + +func detailCardIDs(details []*trafficcutoff.Detail) []int64 { + ids := make([]int64, 0, len(details)) + for _, detail := range details { + if detail != nil && detail.Card != nil { + ids = append(ids, detail.Card.ID) + } + } + sort.Slice(ids, func(i, j int) bool { return ids[i] < ids[j] }) + return ids +} + +func uniqueSortedInt64(values []int64) []int64 { + if len(values) == 0 { + return nil + } + seen := make(map[int64]struct{}, len(values)) + result := make([]int64, 0, len(values)) + for _, value := range values { + if value <= 0 { + continue + } + if _, ok := seen[value]; ok { + continue + } + seen[value] = struct{}{} + result = append(result, value) + } + sort.Slice(result, func(i, j int) bool { return result[i] < result[j] }) + return result +} + +func uniqueSortedStrings(values []string) []string { + if len(values) == 0 { + return nil + } + seen := make(map[string]struct{}, len(values)) + out := make([]string, 0, len(values)) + for _, value := range values { + value = strings.TrimSpace(value) + if value == "" { + continue + } + if _, ok := seen[value]; ok { + continue + } + seen[value] = struct{}{} + out = append(out, value) + } + sort.Strings(out) + return out +} + +var _ trafficcutoff.Repository = (*Repository)(nil) diff --git a/internal/repository/trafficcutoff/repository_test.go b/internal/repository/trafficcutoff/repository_test.go new file mode 100644 index 0000000..1533af6 --- /dev/null +++ b/internal/repository/trafficcutoff/repository_test.go @@ -0,0 +1,359 @@ +package trafficcutoff + +import ( + "context" + "testing" + + _ "github.com/gogf/gf/contrib/drivers/sqlite/v2" + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + + batchdomain "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" +) + +const ( + testTenantID = "000000" + testDeptID = int64(100) +) + +func newTestRepository(t *testing.T) *Repository { + t.Helper() + ctx := context.Background() + db, err := gdb.New(gdb.ConfigNode{ + Type: "sqlite", + Link: "sqlite::@file(:memory:)", + }) + if err != nil { + t.Fatalf("create sqlite database: %v", err) + } + statements := []string{ + `CREATE TABLE batch_main ( + source_channel TEXT NOT NULL DEFAULT 'legacy', + execution_mode INTEGER NOT NULL DEFAULT 0, + id INTEGER PRIMARY KEY AUTOINCREMENT, + batch_no TEXT NOT NULL, + batch_type INTEGER NOT NULL, + batch_async_status INTEGER NOT NULL, + run_result INTEGER NOT NULL DEFAULT 0, + total_count INTEGER NOT NULL DEFAULT 0, + valid_count INTEGER NOT NULL DEFAULT 0, + success_count INTEGER NOT NULL DEFAULT 0, + fail_count INTEGER NOT NULL DEFAULT 0, + fail_reason TEXT, + start_time DATETIME, + end_time DATETIME, + dept_id INTEGER NOT NULL DEFAULT 0, + created_by INTEGER NOT NULL DEFAULT 0, + created_dept INTEGER NOT NULL DEFAULT 0, + tenant_id TEXT NOT NULL DEFAULT '', + batch_info TEXT + )`, + `CREATE TABLE batch_detail ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + batch_no TEXT NOT NULL, + batch_type INTEGER NOT NULL, + detail_status INTEGER NOT NULL, + error_reason TEXT, + snapshot TEXT, + item_id TEXT, + row_no INTEGER NOT NULL DEFAULT 0, + updated_by INTEGER NOT NULL DEFAULT 0 + )`, + `CREATE TABLE t_sim_card_info ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + iccid TEXT NOT NULL, + msisdn TEXT, + imsi TEXT, + boss_card_status INTEGER NOT NULL DEFAULT 0, + traffic_cutoff_status INTEGER NOT NULL DEFAULT 0, + traffic_cutoff_threshold REAL NOT NULL DEFAULT 0, + tenant_id TEXT NOT NULL DEFAULT '', + updated_by INTEGER NOT NULL DEFAULT 0 + )`, + `CREATE TABLE sys_oss ( + oss_id INTEGER PRIMARY KEY AUTOINCREMENT, + tenant_id TEXT, + file_name TEXT, + original_name TEXT, + file_suffix TEXT, + path TEXT, + url TEXT, + created_dept INTEGER, + created_by INTEGER, + updated_by INTEGER, + service TEXT, + md5 TEXT, + file_size INTEGER + )`, + `CREATE TABLE t_resource_batch_download ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + batch_no TEXT, + task_name TEXT, + resource_type INTEGER, + export_scene INTEGER, + total_count INTEGER, + success_count INTEGER, + fail_count INTEGER, + download_status INTEGER, + oss_id INTEGER, + file_name TEXT, + file_ext TEXT, + file_url TEXT, + file_path TEXT, + file_size INTEGER, + file_md5 TEXT, + download_count INTEGER, + max_download_count INTEGER, + dept_id INTEGER, + start_time DATETIME, + finish_time DATETIME, + tenant_id TEXT, + created_dept INTEGER, + created_by INTEGER, + updated_by INTEGER + )`, + } + for _, statement := range statements { + if _, err = db.Exec(ctx, statement); err != nil { + t.Fatalf("create test schema: %v", err) + } + } + t.Cleanup(func() { _ = db.Close(ctx) }) + return &Repository{db: db, lockEnabled: false} +} + +func seedProcessingBatch(t *testing.T, ctx context.Context, db gdb.DB, batchNo string, threshold float64) { + t.Helper() + info := gjson.New(map[string]any{ + "operation": "dataCut", + "trafficCutoffStatus": 1, + "trafficCutoffThreshold": threshold, + "thresholdUnit": "MB", + "resultFile": map[string]any{"status": "pending"}, + }) + if _, err := db.Model(dao.BatchMain.Table()).Ctx(ctx).Data(do.BatchMain{ + BatchNo: batchNo, + BatchType: consts.BatchTypeTrafficCutoff, + BatchAsyncStatus: consts.BatchAsyncStatusProcessing, + RunResult: batchdomain.RunResultRunning, + DeptId: testDeptID, + CreatedBy: 7, + CreatedDept: testDeptID, + TenantId: testTenantID, + BatchInfo: info, + }).Insert(); err != nil { + t.Fatalf("insert batch main: %v", err) + } +} + +func seedRunningDetail(t *testing.T, ctx context.Context, db gdb.DB, batchNo string, rowNo int, iccid, msisdn, imsi string, authorized bool) { + t.Helper() + snapshot := gjson.New(map[string]any{ + "ICCID": iccid, + "MSISDN": msisdn, + "IMSI": imsi, + "authorized": authorized, + }) + if _, err := db.Model(dao.BatchDetail.Table()).Ctx(ctx).Data(do.BatchDetail{ + BatchNo: batchNo, + BatchType: consts.BatchTypeTrafficCutoff, + DetailStatus: consts.BatchDetailStatusRunning, + RowNo: rowNo, + ItemId: iccid, + Snapshot: snapshot, + }).Insert(); err != nil { + t.Fatalf("insert batch detail: %v", err) + } +} + +func seedCard(t *testing.T, ctx context.Context, db gdb.DB, iccid, msisdn, imsi string, status int) { + t.Helper() + if _, err := db.Model(dao.TSimCardInfo.Table()).Ctx(ctx).Data(do.TSimCardInfo{ + Iccid: iccid, + Msisdn: msisdn, + Imsi: imsi, + BossCardStatus: status, + TrafficCutoffStatus: 0, + TrafficCutoffThreshold: 0, + TenantId: testTenantID, + }).Insert(); err != nil { + t.Fatalf("insert sim card: %v", err) + } +} + +func TestReadTrafficCutoffStatusRequiresNewField(t *testing.T) { + tests := []struct { + name string + info *gjson.Json + want int + wantErr bool + }{ + {name: "new field", info: gjson.New(map[string]any{"trafficCutoffStatus": 2}), want: 2}, + {name: "old field only", info: gjson.New(map[string]any{"trafficCutoffEnabled": 1}), wantErr: true}, + {name: "missing batch info", wantErr: true}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := readTrafficCutoffStatus(tt.info) + if (err != nil) != tt.wantErr { + t.Fatalf("readTrafficCutoffStatus() error = %v, wantErr %v", err, tt.wantErr) + } + if got != tt.want { + t.Fatalf("readTrafficCutoffStatus() = %d, want %d", got, tt.want) + } + }) + } +} + +func TestCommitBusinessHappyPathUpdatesAllCards(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "DCT-1" + seedProcessingBatch(t, ctx, repo.db, batchNo, 2048.125) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", true) + seedRunningDetail(t, ctx, repo.db, batchNo, 2, "898600000000002", "13800000002", "460001000000002", true) + seedCard(t, ctx, repo.db, "898600000000001", "13800000001", "460001000000001", 4) + seedCard(t, ctx, repo.db, "898600000000002", "13800000002", "460001000000002", 4) + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if err = repo.CommitBusiness(ctx, task); err != nil { + t.Fatalf("CommitBusiness() error = %v", err) + } + + assertBatchMainStatus(t, ctx, repo.db, batchNo, consts.BatchAsyncStatusSuccess, consts.BatchRunResultSuccess) + assertDetailStatusCount(t, ctx, repo.db, batchNo, consts.BatchDetailStatusSuccess, 2) + assertTrafficCutoff(t, ctx, repo.db, "898600000000001", 1, 2048.125) + assertTrafficCutoff(t, ctx, repo.db, "898600000000002", 1, 2048.125) +} + +func TestCommitBusinessFailsWholeBatchOnMixedStatusWithoutCardUpdate(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "DCT-2" + seedProcessingBatch(t, ctx, repo.db, batchNo, 1024) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", true) + seedRunningDetail(t, ctx, repo.db, batchNo, 2, "898600000000002", "13800000002", "460001000000002", true) + seedCard(t, ctx, repo.db, "898600000000001", "13800000001", "460001000000001", 4) + seedCard(t, ctx, repo.db, "898600000000002", "13800000002", "460001000000002", 5) + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if err = repo.CommitBusiness(ctx, task); err != nil { + t.Fatalf("CommitBusiness() error = %v", err) + } + + assertBatchMainStatus(t, ctx, repo.db, batchNo, consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed) + assertDetailStatusCount(t, ctx, repo.db, batchNo, consts.BatchDetailStatusFailed, 2) + assertTrafficCutoff(t, ctx, repo.db, "898600000000001", 0, 0) + assertTrafficCutoff(t, ctx, repo.db, "898600000000002", 0, 0) +} + +func TestCommitBusinessFailsWholeBatchOnCanceledCardWithoutCardUpdate(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "DCT-3" + seedProcessingBatch(t, ctx, repo.db, batchNo, 512) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", true) + seedCard(t, ctx, repo.db, "898600000000001", "13800000001", "460001000000001", 9) + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if err = repo.CommitBusiness(ctx, task); err != nil { + t.Fatalf("CommitBusiness() error = %v", err) + } + + assertBatchMainStatus(t, ctx, repo.db, batchNo, consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed) + assertTrafficCutoff(t, ctx, repo.db, "898600000000001", 0, 0) +} + +func TestUpdateResultFileFailureOnlyMarksBatchInfo(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "DCT-4" + seedProcessingBatch(t, ctx, repo.db, batchNo, 512) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", true) + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if err = repo.updateResultFileStatus(ctx, task, "failed", 0, 0, "disk full"); err != nil { + t.Fatalf("updateResultFileStatus() error = %v", err) + } + + var row struct { + Info string `orm:"batch_info"` + } + if err = repo.db.Model(dao.BatchMain.Table()).Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Fields(dao.BatchMain.Columns().BatchInfo). + Scan(&row); err != nil { + t.Fatalf("query batch main: %v", err) + } + info := gjson.New([]byte(row.Info)) + if info.Get("resultFile.status").String() != "failed" { + t.Fatalf("resultFile.status = %s, want failed", info.Get("resultFile.status").String()) + } + if info.Get("resultFile.error").String() != "disk full" { + t.Fatalf("resultFile.error = %s, want disk full", info.Get("resultFile.error").String()) + } + assertBatchMainStatus(t, ctx, repo.db, batchNo, consts.BatchAsyncStatusProcessing, batchdomain.RunResultRunning) +} + +func assertBatchMainStatus(t *testing.T, ctx context.Context, db gdb.DB, batchNo string, wantAsync, wantRun int) { + t.Helper() + var row struct { + Async int `orm:"batch_async_status"` + Run int `orm:"run_result"` + } + if err := db.Model(dao.BatchMain.Table()).Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Fields(dao.BatchMain.Columns().BatchAsyncStatus, dao.BatchMain.Columns().RunResult). + Scan(&row); err != nil { + t.Fatalf("query batch main: %v", err) + } + if row.Async != wantAsync || row.Run != wantRun { + t.Fatalf("batch status = (async=%d, run=%d), want (async=%d, run=%d)", row.Async, row.Run, wantAsync, wantRun) + } +} + +func assertDetailStatusCount(t *testing.T, ctx context.Context, db gdb.DB, batchNo string, status, want int) { + t.Helper() + count, err := db.Model(dao.BatchDetail.Table()).Ctx(ctx). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().DetailStatus, status). + Count() + if err != nil { + t.Fatalf("count details: %v", err) + } + if count != want { + t.Fatalf("detail status %d count = %d, want %d", status, count, want) + } +} + +func assertTrafficCutoff(t *testing.T, ctx context.Context, db gdb.DB, iccid string, enabled int, threshold float64) { + t.Helper() + var row struct { + Enabled int `orm:"traffic_cutoff_status"` + Threshold float64 `orm:"traffic_cutoff_threshold"` + } + if err := db.Model(dao.TSimCardInfo.Table()).Ctx(ctx). + Where(dao.TSimCardInfo.Columns().Iccid, iccid). + Fields(dao.TSimCardInfo.Columns().TrafficCutoffStatus, dao.TSimCardInfo.Columns().TrafficCutoffThreshold). + Scan(&row); err != nil { + t.Fatalf("query traffic cutoff fields: %v", err) + } + if row.Enabled != enabled || row.Threshold != threshold { + t.Fatalf("%s cutoff = (%d, %v), want (%d, %v)", iccid, row.Enabled, row.Threshold, enabled, threshold) + } +} diff --git a/internal/repository/trafficusage/repository.go b/internal/repository/trafficusage/repository.go new file mode 100644 index 0000000..5fab7c0 --- /dev/null +++ b/internal/repository/trafficusage/repository.go @@ -0,0 +1,159 @@ +package trafficusage + +import ( + "context" + "fmt" + "hash/fnv" + "math" + "regexp" + "strconv" + "strings" + "time" + "unicode" + + "github.com/gogf/gf/v2/errors/gerror" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/tdengine" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +var identifierPattern = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`) + +type Repository struct{} + +func New() *Repository { + return &Repository{} +} + +func (r *Repository) UpdateDaily(ctx context.Context, items []model.TrafficUsageDailyItem) error { + config := tdengine.GetConfig(ctx) + return writeUsage(ctx, config, config.DbNameFor(config.DailyStable), config.DailyStable, items, func(item model.TrafficUsageDailyItem) time.Time { + return snapshotDay(item.Date) + }) +} + +func (r *Repository) UpdateMonthly(ctx context.Context, items []model.TrafficUsageMonthlyItem) error { + config := tdengine.GetConfig(ctx) + return writeUsage(ctx, config, config.DbNameFor(config.MonthlyStable), config.MonthlyStable, items, func(item model.TrafficUsageMonthlyItem) time.Time { + return snapshotMonth(item.Month) + }) +} + +type usageItem interface { + model.TrafficUsageDailyItem | model.TrafficUsageMonthlyItem +} + +func writeUsage[T usageItem](ctx context.Context, config tdengine.Config, dbName, stable string, items []T, timestamp func(T) time.Time) error { + if len(items) == 0 { + return nil + } + if !tdengine.Enabled(ctx) { + return gerror.New("TDengine disabled; usage was not persisted") + } + if !identifierPattern.MatchString(dbName) || !identifierPattern.MatchString(stable) { + return gerror.New("invalid tdengine database or stable name") + } + batchSize := config.BatchSize + if batchSize <= 0 { + batchSize = 200 + } + for start := 0; start < len(items); start += batchSize { + end := min(start+batchSize, len(items)) + var builder strings.Builder + builder.WriteString("INSERT INTO ") + for index, item := range items[start:end] { + base := usageBase(item) + if base.Iccid == "" { + return gerror.New("traffic usage item ICCID is empty") + } + if math.IsNaN(base.UsageMb) || math.IsInf(base.UsageMb, 0) || base.UsageMb < 0 { + return gerror.Newf("invalid traffic usage value: %v", base.UsageMb) + } + if index > 0 { + builder.WriteByte(' ') + } + builder.WriteString(fmt.Sprintf("%s.%s USING %s.%s TAGS (%s) VALUES (%s, %s, %s)", + dbName, + childTableName(stable, base.Iccid), + dbName, + stable, + sqlString(base.Iccid), + sqlTime(timestamp(item)), + strconv.FormatFloat(base.UsageMb, 'f', -1, 64), + sqlString(base.PackageCode), + )) + } + if _, err := tdengine.ExecForDb(ctx, dbName, builder.String()); err != nil { + return gerror.Wrap(err, "write traffic usage failed") + } + } + return nil +} + +func usageBase[T usageItem](item T) model.TrafficUsageBase { + switch value := any(item).(type) { + case model.TrafficUsageDailyItem: + return value.TrafficUsageBase + case model.TrafficUsageMonthlyItem: + return value.TrafficUsageBase + default: + return model.TrafficUsageBase{} + } +} + +// childTableName builds the TDengine child-table name for one ICCID. +// +// The rule must stay byte-identical to Admin's childTableNameWithPrefix +// (iotlcm-admin-svr internal/logic/trafficusage/traffic_usage.go): drop every +// character that is not a letter, digit or underscore, and fall back to an FNV +// hash when the result is empty or longer than 190 characters. Admin's QueryDaily +// reads child tables by that same rule, so diverging here would silently split +// one card across two child tables and hide its data from the diagnosis page. +func childTableName(stable, iccid string) string { + prefix := "sim_card_usage_m_" + if stable == "t_sim_card_usage_daily" { + prefix = "sim_card_usage_d_" + } + var builder strings.Builder + builder.WriteString(prefix) + for _, char := range strings.ToLower(iccid) { + if unicode.IsLetter(char) || unicode.IsDigit(char) || char == '_' { + builder.WriteRune(char) + } + } + name := builder.String() + if name == prefix || len(name) > 190 { + return fmt.Sprintf("%s%x", prefix, hashString(iccid)) + } + return name +} + +// hashString mirrors Admin's FNV-1a fallback so both sides derive the same +// child-table name for identifiers that cannot be embedded verbatim. +func hashString(value string) uint32 { + hash := fnv.New32a() + _, _ = hash.Write([]byte(value)) + return hash.Sum32() +} + +func snapshotDay(value time.Time) time.Time { + if value.IsZero() { + value = time.Now() + } + return time.Date(value.Year(), value.Month(), value.Day(), 23, 59, 59, 0, value.Location()) +} + +func snapshotMonth(value time.Time) time.Time { + if value.IsZero() { + value = time.Now() + } + return time.Date(value.Year(), value.Month()+1, 0, 23, 59, 59, 0, value.Location()) +} + +func sqlString(value string) string { + return "'" + strings.ReplaceAll(value, "'", "''") + "'" +} + +func sqlTime(value time.Time) string { + return sqlString(value.Format("2006-01-02 15:04:05.000")) +} diff --git a/internal/repository/trafficusage/repository_test.go b/internal/repository/trafficusage/repository_test.go new file mode 100644 index 0000000..4e27a95 --- /dev/null +++ b/internal/repository/trafficusage/repository_test.go @@ -0,0 +1,149 @@ +package trafficusage + +import ( + "fmt" + "hash/fnv" + "strings" + "testing" + "time" + "unicode" +) + +// adminChildTableName is a verbatim copy of Admin's childTableNameWithPrefix +// (iotlcm-admin-svr internal/logic/trafficusage/traffic_usage.go). It exists so +// the worker rule is asserted against the production reader rule rather than +// against itself: Admin's QueryDaily resolves child tables with this rule, so +// any divergence here would silently hide a card's daily usage. +func adminChildTableName(prefix, value string) string { + var builder strings.Builder + builder.WriteString(prefix) + for _, r := range strings.ToLower(value) { + if unicode.IsLetter(r) || unicode.IsDigit(r) || r == '_' { + builder.WriteRune(r) + } + } + name := builder.String() + if name == prefix || len(name) > 190 { + hash := fnv.New32a() + _, _ = hash.Write([]byte(value)) + return fmt.Sprintf("%s%x", prefix, hash.Sum32()) + } + return name +} + +func adminChildTrafficTableName(stable, iccid string) string { + prefix := "sim_card_usage_m_" + if stable == "t_sim_card_usage_daily" { + prefix = "sim_card_usage_d_" + } + return adminChildTableName(prefix, iccid) +} + +func TestChildTableNameMatchesAdminRule(t *testing.T) { + longICCID := strings.Repeat("8", 191) + veryLongICCID := strings.Repeat("89", 120) + cases := []string{ + "", + "89860001123456789012", + "8986-0001", + "8986 0001", + "8986.0001", + "8986/0001", + "ICCID_8986", + "8986-0001-中文", + "中文卡号", + "____", + "8986+0001", + "8986'0001", + longICCID, + veryLongICCID, + } + for _, stable := range []string{"t_sim_card_usage_daily", "t_sim_card_usage_monthly"} { + for _, iccid := range cases { + want := adminChildTrafficTableName(stable, iccid) + got := childTableName(stable, iccid) + if got != want { + t.Errorf("childTableName(%q, %q) = %q, admin rule yields %q", stable, iccid, got, want) + } + if !strings.HasPrefix(got, "sim_card_usage_") { + t.Errorf("childTableName(%q, %q) = %q loses the table prefix", stable, iccid, got) + } + } + } +} + +// TestChildTableNameDirtyICCIDSplitsOnce guards the original defect: the old +// worker rule replaced illegal characters with "_" while Admin dropped them, so +// a dirty ICCID could land in a different child table than the reader queried. +func TestChildTableNameDirtyICCIDSplitsOnce(t *testing.T) { + for _, iccid := range []string{"8986-0001", "8986 0001", "8986.0001"} { + got := childTableName("t_sim_card_usage_daily", iccid) + if strings.Contains(got, "__") { + t.Errorf("childTableName(%q) = %q still substitutes instead of dropping", iccid, got) + } + if got != "sim_card_usage_d_89860001" { + t.Errorf("childTableName(%q) = %q, want the illegal characters dropped", iccid, got) + } + } +} + +func TestChildTableNameLongIdentifierUsesHashFallback(t *testing.T) { + iccid := strings.Repeat("8", 191) + got := childTableName("t_sim_card_usage_daily", iccid) + if len(got) > 190 { + t.Fatalf("childTableName kept an over-long name: len=%d", len(got)) + } + if got != adminChildTrafficTableName("t_sim_card_usage_daily", iccid) { + t.Fatalf("hash fallback diverged from the admin rule: %q", got) + } +} + +func TestSnapshotDayUsesEndOfDay(t *testing.T) { + // The dashboard "yesterday top 5" and the diagnosis page both select the + // 23:59:59.000 row, so the timestamp contract must not drift. + day := snapshotDay(mustParse(t, "2026-10-09T13:45:00+08:00")) + if got := day.Format("2006-01-02 15:04:05.000"); got != "2026-10-09 23:59:59.000" { + t.Fatalf("snapshotDay = %s, want 2026-10-09 23:59:59.000", got) + } +} + +func mustParse(t *testing.T, value string) time.Time { + t.Helper() + parsed, err := time.Parse(time.RFC3339, value) + if err != nil { + t.Fatalf("parse %q: %v", value, err) + } + return parsed +} + +// TestSnapshotMonthMatchesAdminRule pins the monthly timestamp contract. +// +// The diagnosis page reads monthly history by ts and groups it with +// ts.Format("2006-01"), while admin's monthStart wrote the end-of-month +// 23:59:59.000 row. A migration that changed this would leave the page reading +// a month that never gets written. +func TestSnapshotMonthMatchesAdminRule(t *testing.T) { + // adminMonthStart is a verbatim copy of Admin's monthStart + // (iotlcm-admin-svr internal/logic/trafficusage/traffic_usage.go), despite + // its name returning the end of the month. + adminMonthStart := func(tm time.Time) time.Time { + if tm.IsZero() { + tm = time.Now() + } + return time.Date(tm.Year(), tm.Month()+1, 0, 23, 59, 59, 0, tm.Location()) + } + location := time.FixedZone("UTC+8", 8*60*60) + for _, month := range []time.Time{ + time.Date(2026, 9, 1, 0, 0, 0, 0, location), + time.Date(2026, 2, 1, 0, 0, 0, 0, location), + time.Date(2026, 12, 1, 0, 0, 0, 0, location), + // Leap February must land on the 29th. + time.Date(2024, 2, 1, 0, 0, 0, 0, location), + } { + want := adminMonthStart(month).Format("2006-01-02 15:04:05.000") + got := snapshotMonth(month).Format("2006-01-02 15:04:05.000") + if got != want { + t.Errorf("snapshotMonth(%s) = %s, admin rule yields %s", month.Format("2006-01"), got, want) + } + } +} diff --git a/internal/repository/unsub/refund.go b/internal/repository/unsub/refund.go new file mode 100644 index 0000000..88eb338 --- /dev/null +++ b/internal/repository/unsub/refund.go @@ -0,0 +1,407 @@ +package unsub + +import ( + "context" + "database/sql" + "errors" + "fmt" + "math" + "sort" + "strconv" + "strings" + "time" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + unsub "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/unsub" +) + +const ( + agentBalanceTable = "t_agent_balance" + balanceDetailTable = "t_balance_detail" + + balanceAccountStatusNormal = 0 + financeIncomeExpenseIncome = 0 + financeTransactionTypeConsume = 1 + financeTransactionChannelBalance = 3 + financeTransactionStatusPaid = 1 + + refundStatusRefunded = "refunded" + refundStatusNoRefund = "no_refund" + refundStatusSkipped = "skipped" +) + +type balanceAccountQuery struct { + Id int64 `orm:"id"` + DeptId int64 `orm:"dept_id"` + Balance float64 `orm:"balance"` + Status int `orm:"status"` + TenantId string `orm:"tenant_id"` +} + +type balanceDetailQuery struct { + Id int64 `orm:"id"` + IncomeExpenseType int `orm:"income_expense_type"` + TransactionStatus int `orm:"transaction_status"` + Amount float64 `orm:"amount"` +} + +func unsubscribeTime(task *unsub.Task, fallback *gtime.Time) *gtime.Time { + if task != nil && task.Batch.StartTime != nil { + return task.Batch.StartTime + } + return fallback +} + +func (r *Repository) lockOrdersByOrderCodes(ctx context.Context, tx gdb.TX, tenantID string, orderCounts map[string]int) (map[string]*orderQuery, error) { + result := make(map[string]*orderQuery) + orderCodes := sortedOrderCodes(orderCounts) + if len(orderCodes) == 0 { + return result, nil + } + var rows []*orderQuery + model := r.applyLock(tx.Ctx(ctx).Model(orderTable)). + WhereIn("order_sn", orderCodes). + Fields("id", "order_sn", "customer_agent_id", "pay_type", "pay_status", "pay_order_sn", "total_amount", "paid_amount", "balance", "unrefund_amount", "tenant_id", "created_dept") + if strings.TrimSpace(tenantID) != "" { + model = model.Where("tenant_id", tenantID) + } + if err := model.Scan(&rows); err != nil { + return nil, err + } + for _, row := range rows { + if row != nil && strings.TrimSpace(row.OrderSn) != "" { + result[strings.TrimSpace(row.OrderSn)] = row + } + } + return result, nil +} + +func (r *Repository) lockOrderItemsByOrderCodes(ctx context.Context, tx gdb.TX, tenantID string, orderCounts map[string]int) (map[string]*orderItemQuery, error) { + result := make(map[string]*orderItemQuery) + orderCodes := sortedOrderCodes(orderCounts) + if len(orderCodes) == 0 { + return result, nil + } + var rows []*orderItemQuery + model := r.applyLock(tx.Ctx(ctx).Model(orderItemTable)). + WhereIn("order_sn", orderCodes). + Fields("item_id", "order_sn", "price", "quantity", "service_begin_time", "service_end_time", "extra_sku_data") + if strings.TrimSpace(tenantID) != "" { + model = model.Where("tenant_id", tenantID) + } + if err := model.Scan(&rows); err != nil { + return nil, err + } + for _, row := range rows { + if row == nil || strings.TrimSpace(row.OrderSn) == "" { + continue + } + orderCode := strings.TrimSpace(row.OrderSn) + if _, exists := result[orderCode]; !exists { + result[orderCode] = row + } + } + return result, nil +} + +func (r *Repository) refundOrders( + ctx context.Context, + tx gdb.TX, + task *unsub.Task, + orderCounts map[string]int, + bindsByIccid map[string]*bindPkgQuery, + orders map[string]*orderQuery, + orderItems map[string]*orderItemQuery, + activeCounts map[string]int, + cutoffTime, transactionTime *gtime.Time, +) (map[string]*unsub.OrderRefund, error) { + if cutoffTime == nil { + cutoffTime = transactionTime + } + if cutoffTime == nil { + return nil, gerror.New("unsub refund cutoff time is empty") + } + result := make(map[string]*unsub.OrderRefund, len(orderCounts)) + orderBinds := make(map[string][]*bindPkgQuery, len(orderCounts)) + for _, detail := range task.Details { + if detail == nil { + continue + } + bind := bindsByIccid[detail.ICCID] + if bind == nil { + continue + } + orderCode := strings.TrimSpace(bind.OrderCode) + if orderCode != "" { + orderBinds[orderCode] = append(orderBinds[orderCode], bind) + } + } + + for _, orderCode := range sortedOrderCodes(orderCounts) { + refund := &unsub.OrderRefund{Status: refundStatusNoRefund} + result[orderCode] = refund + order := orders[orderCode] + if order == nil { + refund.Status = refundStatusSkipped + refund.Reason = "order not found" + g.Log().Warningf(ctx, "unsub balance refund skipped, order not found, batchNo=%s orderNo=%s", task.Batch.BatchNo, orderCode) + continue + } + if order.PayType != consts.PayTypeBalance { + refund.Status = refundStatusSkipped + refund.Reason = "order is not paid by balance" + g.Log().Warningf(ctx, "unsub balance refund skipped, order is not paid by balance, batchNo=%s orderNo=%s payType=%d", task.Batch.BatchNo, orderCode, order.PayType) + continue + } + if order.PayStatus != 0 && order.PayStatus != consts.PayStatusPaid { + refund.Status = refundStatusSkipped + refund.Reason = "order is not paid" + g.Log().Warningf(ctx, "unsub balance refund skipped, order is not paid, batchNo=%s orderNo=%s payStatus=%d", task.Batch.BatchNo, orderCode, order.PayStatus) + continue + } + + binds := orderBinds[orderCode] + if len(binds) == 0 { + continue + } + item := orderItems[orderCode] + totalQuantity := resolveOrderQuantity(order, item, activeCounts[orderCode]) + if totalQuantity <= 0 { + return nil, gerror.Newf("order %s lacks a valid subscription quantity for refund", orderCode) + } + paidAmount := orderPaidAmount(order, item, totalQuantity) + if paidAmount <= 0 { + refund.Reason = "order paid amount is zero" + continue + } + unitPaidAmount := paidAmount / float64(totalQuantity) + refundAmount := 0.0 + for _, bind := range binds { + if bind == nil { + continue + } + if bind.EffectiveTime == nil || bind.DurationMonths <= 0 { + return nil, gerror.Newf("order %s has an invalid paid subscription period", orderCode) + } + refundAmount += unitPaidAmount * calculateRefundRatio(bind.EffectiveTime.Time, cutoffTime.Time, bind.DurationMonths) + } + refund.Amount = roundUnsubAmount(refundAmount) + if refund.Amount <= 0 { + refund.Reason = "paid subscription period has been fully used" + continue + } + + refund.SerialNo = unsubscribeRefundSerialNo(task.Batch.BatchNo, orderCode) + existing, err := r.lockRefundDetail(ctx, tx, task.Batch.TenantID, refund.SerialNo) + if err != nil { + return nil, gerror.Wrap(err, "query existing unsub refund detail") + } + if existing.Id != 0 { + if existing.IncomeExpenseType != financeIncomeExpenseIncome || existing.TransactionStatus != financeTransactionStatusPaid { + return nil, gerror.Newf("existing unsub refund detail is invalid, order %s", orderCode) + } + refund.Amount = roundUnsubAmount(existing.Amount) + refund.Status = refundStatusRefunded + refund.Reason = "refund detail already exists" + continue + } + + payerDeptID := refundPayerDeptID(order, binds) + if payerDeptID <= 0 { + return nil, gerror.Newf("order %s lacks a balance payer department", orderCode) + } + tenantID := strings.TrimSpace(order.TenantId) + if tenantID == "" { + tenantID = strings.TrimSpace(task.Batch.TenantID) + } + if tenantID == "" { + return nil, gerror.Newf("order %s lacks a tenant id for refund", orderCode) + } + var account balanceAccountQuery + accountModel := r.applyLock(tx.Ctx(ctx).Model(agentBalanceTable)). + Where("dept_id", payerDeptID). + Where("tenant_id", tenantID). + Fields("id", "dept_id", "balance", "status", "tenant_id") + if err := accountModel.Scan(&account); err != nil && !errors.Is(err, sql.ErrNoRows) { + return nil, gerror.Wrap(err, "query balance account for unsub refund") + } + if account.Id == 0 { + return nil, gerror.Newf("balance account for order %s does not exist", orderCode) + } + if account.Status != balanceAccountStatusNormal { + return nil, gerror.Newf("balance account for order %s is not available", orderCode) + } + before := roundUnsubAmount(account.Balance) + after := roundUnsubAmount(before + refund.Amount) + accountUpdate, err := tx.Ctx(ctx).Model(agentBalanceTable). + Where("id", account.Id). + Data(g.Map{ + "balance": after, + "updated_by": task.Batch.CreatedBy, + "updated_at": transactionTime, + }).Update() + if err != nil { + return nil, gerror.Wrap(err, "update balance account for unsub refund") + } + if affected, err := accountUpdate.RowsAffected(); err != nil { + return nil, gerror.Wrap(err, "read balance account update result") + } else if affected != 1 { + return nil, gerror.Newf("balance account for order %s changed during refund", orderCode) + } + + createdDept := order.CreatedDept + if createdDept <= 0 { + createdDept = payerDeptID + } + if _, err := tx.Ctx(ctx).Model(balanceDetailTable).Data(g.Map{ + "dept_id": payerDeptID, + "serial_no": refund.SerialNo, + "transaction_time": transactionTime, + "income_expense_type": financeIncomeExpenseIncome, + "transaction_type": financeTransactionTypeConsume, + "transaction_channel": financeTransactionChannelBalance, + "transaction_status": financeTransactionStatusPaid, + "amount": refund.Amount, + "before_balance": before, + "balance_after": after, + "order_sn": orderCode, + "remark": fmt.Sprintf("SIM card unsubscribe balance refund, batchNo=%s, orderNo=%s", task.Batch.BatchNo, orderCode), + "tenant_id": tenantID, + "created_dept": createdDept, + "created_by": task.Batch.CreatedBy, + "created_at": transactionTime, + }).Insert(); err != nil { + return nil, gerror.Wrap(err, "insert unsub refund detail") + } + refund.Status = refundStatusRefunded + } + return result, nil +} + +func (r *Repository) lockRefundDetail(ctx context.Context, tx gdb.TX, tenantID, serialNo string) (*balanceDetailQuery, error) { + var row balanceDetailQuery + model := r.applyLock(tx.Ctx(ctx).Model(balanceDetailTable)). + Where("tenant_id", tenantID). + Where("serial_no", serialNo). + Fields("id", "income_expense_type", "transaction_status", "amount") + if err := model.Scan(&row); err != nil && !errors.Is(err, sql.ErrNoRows) { + return nil, err + } + return &row, nil +} + +func sortedOrderCodes(orderCounts map[string]int) []string { + orderCodes := make([]string, 0, len(orderCounts)) + for orderCode := range orderCounts { + if strings.TrimSpace(orderCode) != "" { + orderCodes = append(orderCodes, orderCode) + } + } + sort.Strings(orderCodes) + return orderCodes +} + +func resolveOrderQuantity(order *orderQuery, item *orderItemQuery, activeCount int) int { + if item != nil && item.Quantity > 0 { + return item.Quantity + } + paidAmount := orderPaidAmount(order, nil, 0) + if item != nil && item.Price > 0 && paidAmount > 0 { + quantity := int(math.Round(paidAmount / item.Price)) + if quantity > 0 { + return quantity + } + } + return activeCount +} + +func orderPaidAmount(order *orderQuery, item *orderItemQuery, quantity int) float64 { + if order != nil { + if order.Balance > 0 { + return order.Balance + } + if order.PaidAmount > 0 { + return order.PaidAmount + } + if order.TotalAmount > 0 { + return order.TotalAmount + } + } + if item != nil && item.Price > 0 && quantity > 0 { + return item.Price * float64(quantity) + } + return 0 +} + +func refundPayerDeptID(order *orderQuery, binds []*bindPkgQuery) int64 { + if order != nil && order.CustomerAgentId > 0 { + return order.CustomerAgentId + } + for _, bind := range binds { + if bind == nil { + continue + } + if deptID, err := strconv.ParseInt(strings.TrimSpace(bind.DeptId), 10, 64); err == nil && deptID > 0 { + return deptID + } + } + if order != nil && order.CreatedDept > 0 { + return order.CreatedDept + } + return 0 +} + +func unsubscribeRefundSerialNo(batchNo, orderCode string) string { + return fmt.Sprintf("UNSUBREFUND-%s-%s", strings.TrimSpace(batchNo), strings.TrimSpace(orderCode)) +} + +func roundUnsubAmount(value float64) float64 { + if value <= 0 { + return 0 + } + return math.Round(value*100) / 100 +} + +func calculateRefundRatio(effectiveAt, cutoffAt time.Time, durationMonths int) float64 { + if effectiveAt.IsZero() || cutoffAt.IsZero() || durationMonths <= 0 { + return 0 + } + paidEndAt := addRefundMonths(effectiveAt, durationMonths) + if !cutoffAt.After(effectiveAt) { + return 1 + } + if !cutoffAt.Before(paidEndAt) { + return 0 + } + totalSeconds := paidEndAt.Sub(effectiveAt).Seconds() + remainingSeconds := paidEndAt.Sub(cutoffAt).Seconds() + if totalSeconds <= 0 || remainingSeconds <= 0 { + return 0 + } + ratio := remainingSeconds / totalSeconds + if ratio < 0 { + return 0 + } + if ratio > 1 { + return 1 + } + return ratio +} + +func addRefundMonths(value time.Time, months int) time.Time { + if months <= 0 { + return value + } + targetMonth := time.Date(value.Year(), value.Month()+time.Month(months), 1, value.Hour(), value.Minute(), value.Second(), value.Nanosecond(), value.Location()) + day := value.Day() + lastDay := time.Date(targetMonth.Year(), targetMonth.Month()+1, 0, value.Hour(), value.Minute(), value.Second(), value.Nanosecond(), value.Location()).Day() + if day > lastDay { + day = lastDay + } + return time.Date(targetMonth.Year(), targetMonth.Month(), day, value.Hour(), value.Minute(), value.Second(), value.Nanosecond(), value.Location()) +} diff --git a/internal/repository/unsub/refund_test.go b/internal/repository/unsub/refund_test.go new file mode 100644 index 0000000..888fcbc --- /dev/null +++ b/internal/repository/unsub/refund_test.go @@ -0,0 +1,37 @@ +package unsub + +import ( + "math" + "testing" + "time" +) + +func TestCalculateRefundRatio(t *testing.T) { + effective := time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC) + cases := []struct { + name string + cutoff time.Time + want float64 + }{ + {name: "before effective refunds all", cutoff: effective.Add(-time.Hour), want: 1}, + {name: "half paid period", cutoff: time.Date(2026, 7, 1, 0, 0, 0, 0, time.UTC), want: 184.0 / 365.0}, + {name: "during gift period refunds none", cutoff: time.Date(2027, 2, 1, 0, 0, 0, 0, time.UTC), want: 0}, + {name: "after paid period refunds none", cutoff: time.Date(2027, 1, 1, 0, 0, 0, 0, time.UTC), want: 0}, + } + for _, testCase := range cases { + t.Run(testCase.name, func(t *testing.T) { + got := calculateRefundRatio(effective, testCase.cutoff, 12) + if math.Abs(got-testCase.want) > 0.000001 { + t.Fatalf("calculateRefundRatio() = %v, want %v", got, testCase.want) + } + }) + } +} + +func TestAddRefundMonthsClampsMonthEnd(t *testing.T) { + got := addRefundMonths(time.Date(2026, 1, 31, 12, 0, 0, 0, time.UTC), 1) + want := time.Date(2026, 2, 28, 12, 0, 0, 0, time.UTC) + if !got.Equal(want) { + t.Fatalf("addRefundMonths() = %v, want %v", got, want) + } +} diff --git a/internal/repository/unsub/repository.go b/internal/repository/unsub/repository.go new file mode 100644 index 0000000..96cd014 --- /dev/null +++ b/internal/repository/unsub/repository.go @@ -0,0 +1,1206 @@ +package unsub + +import ( + "context" + "encoding/json" + "sort" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" + unsub "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/unsub" +) + +const ( + cardInfoTable = "t_sim_card_info" + cardBindPkgTable = "t_sim_card_bind_pkg" + bindPackageLogsTable = "t_sim_bind_package_logs" + stockTable = "t_sim_card_stock" + orderTable = "t_order" + orderItemTable = "t_order_item" + + bindStatusPending = 0 + bindStatusEffective = 1 + + bindStatusTerminated = 3 + + pkgStatusNone = 0 + pkgStatusEffective = 2 + + orderReturnStatusPartial = 3 + orderReturnStatusSuccess = 4 +) + +type Repository struct { + db gdb.DB + resultFiles batchresult.Generator + lockEnabled bool +} + +func New() *Repository { + return &Repository{resultFiles: batchresult.New(), lockEnabled: true} +} + +func (r *Repository) database() gdb.DB { + if r.db != nil { + return r.db + } + return g.DB() +} + +func (r *Repository) applyLock(model *gdb.Model) *gdb.Model { + if r.lockEnabled { + return model.LockUpdate() + } + return model +} + +type cardQuery struct { + g.Meta `orm:"table:t_sim_card_info"` + Id int64 `orm:"id"` + Iccid string `orm:"iccid"` + Msisdn string `orm:"msisdn"` + Imsi string `orm:"imsi"` + Imei string `orm:"imei"` + ChanKey string `orm:"chan_key"` + DeptId int64 `orm:"dept_id"` + CreatedDept int64 `orm:"created_dept"` + CurrentPackageCode string `orm:"current_package_code"` + PackageStatus int `orm:"package_status"` + InboundBatchNo string `orm:"inbound_batch_no"` + OutboundBatchNo string `orm:"outbound_batch_no"` + TenantId string `orm:"tenant_id"` +} + +type bindPkgQuery struct { + g.Meta `orm:"table:t_sim_card_bind_pkg"` + Id int64 `orm:"id"` + SubscriptionNo string `orm:"subscription_no"` + BatchNo string `orm:"batch_no"` + Iccid string `orm:"iccid"` + Msisdn string `orm:"msisdn"` + DeptId string `orm:"dept_id"` + PackageType int `orm:"package_type"` + PackageCode string `orm:"package_code"` + PreviousPackageCode string `orm:"previous_package_code"` + BusinessType int `orm:"business_type"` + EffectiveStatus int `orm:"effective_status"` + HandleTime *gtime.Time `orm:"handle_time"` + EffectiveTime *gtime.Time `orm:"effective_time"` + ExpectedExpireTime *gtime.Time `orm:"expected_expire_time"` + ActualExpireTime *gtime.Time `orm:"actual_expire_time"` + DurationMonths int `orm:"duration_months"` + GiftDurationMonths int `orm:"gift_duration_months"` + OrderCode string `orm:"order_code"` + Remark string `orm:"remark"` + TenantId string `orm:"tenant_id"` + CreatedDept int64 `orm:"created_dept"` + CreatedBy int64 `orm:"created_by"` + CreatedAt *gtime.Time `orm:"created_at"` +} + +type stockQuery struct { + g.Meta `orm:"table:t_sim_card_stock"` + Id int64 `orm:"id"` + Iccid string `orm:"iccid"` +} + +type bindPackageLogInsert struct { + g.Meta `orm:"table:t_sim_bind_package_logs, do:true"` + SubscriptionNo any + BatchNo any + Iccid any + Msisdn any + DeptId any + PackageType any + PackageCode any + PreviousPackageCode any + BusinessType any + EffectiveStatus any + HandleTime any + EffectiveTime any + ExpectedExpireTime any + ActualExpireTime any + DurationMonths any + GiftDurationMonths any + OrderCode any + Remark any + TenantId any + CreatedDept any + CreatedBy any + UpdatedBy any +} + +type stockInsert struct { + g.Meta `orm:"table:t_sim_card_stock, do:true"` + BatchNo any + Iccid any + Msisdn any + Imsi any + Imei any + ChanKey any + TenantId any + CreatedDept any + CreatedBy any + UpdatedBy any +} + +type orderUpdate struct { + g.Meta `orm:"table:t_order, do:true"` + ReturnStatus any + UnrefundAmount any + UpdatedBy any + UpdatedAt any +} + +type orderItemExtra struct { + LastUnsubscribeBatchNo string `json:"lastUnsubscribeBatchNo"` + LastUnsubscribeTime string `json:"lastUnsubscribeTime"` + UnsubscribedQuantity int `json:"unsubscribedQuantity"` + LastUnsubscribeRefundAmount float64 `json:"lastUnsubscribeRefundAmount"` + LastUnsubscribeRefundSerialNo string `json:"lastUnsubscribeRefundSerialNo"` + LastUnsubscribeRefundStatus string `json:"lastUnsubscribeRefundStatus"` + LastUnsubscribeRefundReason string `json:"lastUnsubscribeRefundReason"` +} + +type orderItemUpdate struct { + g.Meta `orm:"table:t_order_item, do:true"` + ExtraSkuData any + UpdatedBy any +} + +type orderItemQuery struct { + g.Meta `orm:"table:t_order_item"` + ItemId int64 `orm:"item_id"` + OrderSn string `orm:"order_sn"` + Price float64 `orm:"price"` + Quantity int `orm:"quantity"` + ServiceBeginTime *gtime.Time `orm:"service_begin_time"` + ServiceEndTime *gtime.Time `orm:"service_end_time"` + ExtraSkuData string `orm:"extra_sku_data"` +} + +type orderQuery struct { + g.Meta `orm:"table:t_order"` + Id int64 `orm:"id"` + OrderSn string `orm:"order_sn"` + CustomerAgentId int64 `orm:"customer_agent_id"` + PayType int `orm:"pay_type"` + PayStatus int `orm:"pay_status"` + PayOrderSn string `orm:"pay_order_sn"` + TotalAmount float64 `orm:"total_amount"` + PaidAmount float64 `orm:"paid_amount"` + Balance float64 `orm:"balance"` + UnrefundAmount float64 `orm:"unrefund_amount"` + TenantId string `orm:"tenant_id"` + CreatedDept int64 `orm:"created_dept"` +} + +func (r *Repository) Load(ctx context.Context, batchNo string) (*unsub.Task, error) { + if strings.TrimSpace(batchNo) == "" { + return nil, gerror.New("unsub batch number is empty") + } + var main entity.BatchMain + if err := r.database().Model(dao.BatchMain.Table()).Safe().Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeSimCardUnsub). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Scan(&main); err != nil { + return nil, gerror.Wrap(err, "query unsub batch main") + } + if main.Id == 0 { + return nil, gerror.New("processing unsub batch does not exist") + } + var details []*entity.BatchDetail + if err := r.database().Model(dao.BatchDetail.Table()).Safe().Ctx(ctx). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeSimCardUnsub). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + OrderAsc(dao.BatchDetail.Columns().RowNo). + Scan(&details); err != nil { + return nil, gerror.Wrap(err, "query unsub running details") + } + if len(details) == 0 { + return nil, gerror.New("unsub batch has no running details") + } + + batchSourceBatchNo := resolveSourceBatchNo(main, main.BatchNo) + batchRemark := "" + if main.BatchInfo != nil { + batchRemark = strings.TrimSpace(main.BatchInfo.Get("remark").String()) + } + + task := &unsub.Task{ + Batch: unsub.Batch{ + ID: main.Id, + BatchNo: main.BatchNo, + BatchType: main.BatchType, + TenantID: main.TenantId, + DeptID: main.DeptId, + CreatedBy: main.CreatedBy, + CreatedDept: main.CreatedDept, + StartTime: main.StartTime, + BatchInfo: main.BatchInfo, + SourceBatchNo: batchSourceBatchNo, + Remark: batchRemark, + }, + Details: make([]*unsub.Detail, 0, len(details)), + } + for _, raw := range details { + if raw == nil { + continue + } + detail := buildDetailFromBatch(raw, task.Batch.SourceBatchNo) + task.Details = append(task.Details, detail) + } + if len(task.Details) == 0 { + return nil, gerror.New("unsub batch has no readable running details") + } + return task, nil +} + +func buildDetailFromBatch(raw *entity.BatchDetail, fallbackSourceBatchNo string) *unsub.Detail { + detail := &unsub.Detail{ + ID: raw.Id, + RowNo: raw.RowNo, + ICCID: strings.TrimSpace(raw.ItemId), + Reason: strings.TrimSpace(raw.ErrorReason), + } + if raw.Snapshot != nil { + if value := strings.TrimSpace(raw.Snapshot.Get("ICCID").String()); value != "" { + detail.ICCID = value + } + if value := strings.TrimSpace(raw.Snapshot.Get("iccid").String()); value != "" && detail.ICCID == "" { + detail.ICCID = value + } + detail.MSISDN = firstNonEmpty(strings.TrimSpace(raw.Snapshot.Get("MSISDN").String()), strings.TrimSpace(raw.Snapshot.Get("msisdn").String())) + detail.IMSI = firstNonEmpty(strings.TrimSpace(raw.Snapshot.Get("IMSI").String()), strings.TrimSpace(raw.Snapshot.Get("imsi").String())) + detail.Authorized = raw.Snapshot.Get("authorized").Bool() + detail.Remark = strings.TrimSpace(raw.Snapshot.Get("remark").String()) + detail.SourceBatchNo = resolveSourceBatchNoFromSnapshot(raw.Snapshot, fallbackSourceBatchNo) + } + return detail +} + +func resolveSourceBatchNoFromSnapshot(snapshot *gjson.Json, fallback string) string { + if snapshot == nil { + return fallback + } + for _, key := range []string{"sourceBatchNo", "source_batch_no"} { + if value := strings.TrimSpace(snapshot.Get(key).String()); value != "" { + return value + } + } + return fallback +} + +func resolveSourceBatchNo(main entity.BatchMain, currentBatchNo string) string { + if main.BatchInfo != nil { + for _, key := range []string{"sourceBatchNo", "source_batch_no"} { + if value := strings.TrimSpace(main.BatchInfo.Get(key).String()); value != "" && value != currentBatchNo { + return value + } + } + } + if value := strings.TrimSpace(main.SourceBatchNo); value != "" && value != currentBatchNo { + return value + } + return "" +} + +func firstNonEmpty(values ...string) string { + for _, value := range values { + if strings.TrimSpace(value) != "" { + return strings.TrimSpace(value) + } + } + return "" +} + +func (r *Repository) SaveValidationFailure(ctx context.Context, task *unsub.Task) error { + if task == nil { + return gerror.New("unsub task is nil") + } + reason := task.FailureReason + if reason == "" { + reason = "batch validation failed; unsub request was skipped" + } + now := gtime.Now() + successCount := 0 + for _, detail := range task.Details { + if detail != nil && detail.Reason == "" { + successCount++ + } + } + failCount := len(task.Details) - successCount + return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + for _, detail := range task.Details { + if detail == nil { + continue + } + detailReason := detail.Reason + if detailReason == "" { + detailReason = reason + } + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: consts.BatchDetailStatusFailed, + ErrorReason: detailReason, + Snapshot: detail.Snapshot(), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update unsub validation failure detail") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read unsub detail update result") + } + if affected != 1 { + return gerror.Newf("unsub detail %d is no longer running", detail.ID) + } + } + return updateMainForValidation(ctx, tx, task, len(task.Details), failCount, reason, now) + }) +} + +func updateMainForValidation(ctx context.Context, tx gdb.TX, task *unsub.Task, total, fail int, reason string, now *gtime.Time) error { + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeSimCardUnsub). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Data(do.BatchMain{ + BatchAsyncStatus: consts.BatchAsyncStatusFailed, + TotalCount: total, + ValidCount: total, + SuccessCount: 0, + FailCount: fail, + RunResult: consts.BatchRunResultFailed, + FailReason: reason, + EndTime: now, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update unsub batch main for validation failure") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read unsub batch main validation update result") + } + if affected != 1 { + return gerror.New("unsub batch is no longer processing") + } + return nil +} + +func (r *Repository) CommitBusiness(ctx context.Context, task *unsub.Task) error { + if task == nil { + return gerror.New("unsub task is nil") + } + now := gtime.Now() + unsubTime := unsubscribeTime(task, now) + iccids := make([]string, 0, len(task.Details)) + for _, detail := range task.Details { + if detail == nil { + continue + } + iccids = append(iccids, detail.ICCID) + } + iccids = uniqueSortedStrings(iccids) + + return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + cardsByIccid, err := r.lockCardsByIccids(ctx, tx, iccids) + if err != nil { + return gerror.Wrap(err, "lock formal cards") + } + bindsByIccid, err := r.lockCurrentActiveBindsByIccids(ctx, tx, iccids) + if err != nil { + return gerror.Wrap(err, "lock current active binds") + } + stocksByIccid, err := r.lockStockByIccids(ctx, tx, iccids) + if err != nil { + return gerror.Wrap(err, "lock stock") + } + + packageCode := "" + orderCodes := make(map[string]int) + businessFailures := make(map[string]string) + for _, detail := range task.Details { + if detail == nil { + continue + } + reason := validateBusiness(ctx, tx, task, detail, cardsByIccid[detail.ICCID], bindsByIccid[detail.ICCID], stocksByIccid[detail.ICCID], packageCode) + if reason != "" { + businessFailures[detail.ICCID] = reason + continue + } + bind := bindsByIccid[detail.ICCID] + packageCode = strings.TrimSpace(bind.PackageCode) + if orderCode := strings.TrimSpace(bind.OrderCode); orderCode != "" { + orderCodes[orderCode]++ + } + } + + if len(businessFailures) > 0 { + reason := "同批存在不满足退订条件的卡，本次未执行" + for _, detail := range task.Details { + if detail == nil { + continue + } + if failureReason, ok := businessFailures[detail.ICCID]; ok { + detail.Reason = failureReason + } else if detail.Reason == "" { + detail.Reason = reason + } + } + return finalizeFailure(ctx, tx, task, reason) + } + + orders, err := r.lockOrdersByOrderCodes(ctx, tx, task.Batch.TenantID, orderCodes) + if err != nil { + return gerror.Wrap(err, "lock unsub orders") + } + orderItems, err := r.lockOrderItemsByOrderCodes(ctx, tx, task.Batch.TenantID, orderCodes) + if err != nil { + return gerror.Wrap(err, "lock unsub order items") + } + activeCountsBefore, err := activeBindCountsByOrderCodes(ctx, tx, sortedOrderCodes(orderCodes)) + if err != nil { + return gerror.Wrap(err, "count active binds before unsub") + } + refunds, err := r.refundOrders(ctx, tx, task, orderCodes, bindsByIccid, orders, orderItems, activeCountsBefore, unsubTime, now) + if err != nil { + return err + } + + if err := writeBindPackageLogs(ctx, tx, task, cardsByIccid, bindsByIccid, unsubTime); err != nil { + return err + } + if err := deleteCurrentBinds(ctx, tx, task, bindsByIccid); err != nil { + return err + } + if err := updateOrderReturnStatus(ctx, tx, task, orderCodes, orders, refunds, now, unsubTime); err != nil { + return err + } + if err := returnCardsToStock(ctx, tx, task, cardsByIccid, stocksByIccid, now); err != nil { + return err + } + if err := deleteFormalCards(ctx, tx, iccids); err != nil { + return err + } + if err := updateBatchDetailsSuccess(ctx, tx, task, cardsByIccid, bindsByIccid, unsubTime); err != nil { + return err + } + return updateBatchMainSuccess(ctx, tx, task, packageCode, orderCodes, len(iccids), now, unsubTime, refunds) + }) +} + +func (r *Repository) lockCardsByIccids(ctx context.Context, tx gdb.TX, iccids []string) (map[string]*cardQuery, error) { + result := make(map[string]*cardQuery) + if len(iccids) == 0 { + return result, nil + } + var rows []*cardQuery + if err := r.applyLock(tx.Ctx(ctx).Model(cardInfoTable)). + WhereIn("iccid", iccids). + Scan(&rows); err != nil { + return nil, err + } + for _, row := range rows { + if row != nil && row.Iccid != "" { + result[row.Iccid] = row + } + } + return result, nil +} + +func (r *Repository) lockCurrentActiveBindsByIccids(ctx context.Context, tx gdb.TX, iccids []string) (map[string]*bindPkgQuery, error) { + result := make(map[string]*bindPkgQuery) + priorities := make(map[string]int) + if len(iccids) == 0 { + return result, nil + } + var rows []*bindPkgQuery + if err := r.applyLock(tx.Ctx(ctx).Model(cardBindPkgTable)). + WhereIn("iccid", iccids). + WhereIn("effective_status", []int{bindStatusPending, bindStatusEffective}). + OrderDesc("effective_status"). + OrderAsc("effective_time"). + OrderAsc("id"). + Scan(&rows); err != nil { + return nil, err + } + for _, row := range rows { + if row == nil || row.Iccid == "" { + continue + } + priority := bindPriority(row.EffectiveStatus) + if existing, ok := priorities[row.Iccid]; ok { + if priority < existing { + continue + } + if priority == existing { + result[row.Iccid] = nil + continue + } + } + priorities[row.Iccid] = priority + result[row.Iccid] = row + } + return result, nil +} + +func (r *Repository) lockStockByIccids(ctx context.Context, tx gdb.TX, iccids []string) (map[string]*stockQuery, error) { + result := make(map[string]*stockQuery) + if len(iccids) == 0 { + return result, nil + } + var rows []*stockQuery + if err := r.applyLock(tx.Ctx(ctx).Model(stockTable)). + WhereIn("iccid", iccids). + Scan(&rows); err != nil { + return nil, err + } + for _, row := range rows { + if row != nil && row.Iccid != "" { + result[row.Iccid] = row + } + } + return result, nil +} + +func bindPriority(status int) int { + switch status { + case bindStatusEffective: + return 2 + case bindStatusPending: + return 1 + default: + return 0 + } +} + +func validateBusiness(ctx context.Context, tx gdb.TX, task *unsub.Task, detail *unsub.Detail, card *cardQuery, bind *bindPkgQuery, stock *stockQuery, currentPackage string) string { + if card == nil { + return "ICCID不存在正式卡记录，不能退订" + } + if bind == nil { + return "ICCID当前订购数据异常或不存在，不能退订" + } + if card.DeptId != task.Batch.DeptID && card.CreatedDept != task.Batch.DeptID { + return "ICCID不属于当前机构，不能退订" + } + if strings.TrimSpace(card.CurrentPackageCode) == "" || card.PackageStatus == pkgStatusNone { + return "ICCID没有可退订套餐，不能退订" + } + if currentPackage != "" && strings.TrimSpace(bind.PackageCode) != currentPackage { + return "当前批次下存在多个套餐，不能统一退订" + } + if strings.TrimSpace(card.CurrentPackageCode) != strings.TrimSpace(bind.PackageCode) { + return "ICCID当前套餐与订购记录不一致，不能退订" + } + if detail.MSISDN != "" && detail.MSISDN != strings.TrimSpace(card.Msisdn) { + return "MSISDN与正式卡记录不一致，不能退订" + } + if detail.IMSI != "" && detail.IMSI != strings.TrimSpace(card.Imsi) { + return "IMSI与正式卡记录不一致，不能退订" + } + if task.Batch.SourceBatchNo != "" && strings.TrimSpace(bind.BatchNo) != task.Batch.SourceBatchNo { + return "ICCID当前订购不属于来源出库批次，不能退订" + } + if stock != nil { + return "ICCID已存在库存记录，不能重复回库" + } + return "" +} + +func writeBindPackageLogs(ctx context.Context, tx gdb.TX, task *unsub.Task, cards map[string]*cardQuery, binds map[string]*bindPkgQuery, now *gtime.Time) error { + rows := make([]bindPackageLogInsert, 0, len(binds)) + for iccid, bind := range binds { + if bind == nil { + continue + } + remark := bind.Remark + if addition := "unsubscribeBatchNo=" + task.Batch.BatchNo; strings.TrimSpace(remark) == "" { + remark = addition + } else { + remark = strings.TrimSpace(remark) + "; " + addition + } + rows = append(rows, bindPackageLogInsert{ + SubscriptionNo: bind.SubscriptionNo, + BatchNo: bind.BatchNo, + Iccid: bind.Iccid, + Msisdn: bind.Msisdn, + DeptId: bind.DeptId, + PackageType: bind.PackageType, + PackageCode: bind.PackageCode, + PreviousPackageCode: bind.PreviousPackageCode, + BusinessType: bind.BusinessType, + EffectiveStatus: bindStatusTerminated, + HandleTime: bind.HandleTime, + EffectiveTime: bind.EffectiveTime, + ExpectedExpireTime: bind.ExpectedExpireTime, + ActualExpireTime: now, + DurationMonths: bind.DurationMonths, + GiftDurationMonths: bind.GiftDurationMonths, + OrderCode: bind.OrderCode, + Remark: remark, + TenantId: bind.TenantId, + CreatedDept: bind.CreatedDept, + CreatedBy: bind.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + }) + _ = iccid + } + if len(rows) == 0 { + return nil + } + if _, err := tx.Ctx(ctx).Model(bindPackageLogsTable).Data(rows).Insert(); err != nil { + return gerror.Wrap(err, "insert unsub bind package logs") + } + return nil +} + +func deleteCurrentBinds(ctx context.Context, tx gdb.TX, task *unsub.Task, binds map[string]*bindPkgQuery) error { + if len(binds) == 0 { + return nil + } + subscriptionNos := make([]string, 0, len(binds)) + for _, bind := range binds { + if bind != nil && bind.SubscriptionNo != "" { + subscriptionNos = append(subscriptionNos, bind.SubscriptionNo) + } + } + if len(subscriptionNos) == 0 { + return nil + } + result, err := tx.Ctx(ctx).Model(cardBindPkgTable). + WhereIn("subscription_no", subscriptionNos). + Delete() + if err != nil { + return gerror.Wrap(err, "delete current binds") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read current binds delete result") + } + if int(affected) != len(subscriptionNos) { + return gerror.New("部分当前订购已变化，请刷新后重试") + } + _ = task + return nil +} + +func updateOrderReturnStatus(ctx context.Context, tx gdb.TX, task *unsub.Task, orderCounts map[string]int, orders map[string]*orderQuery, refunds map[string]*unsub.OrderRefund, now, unsubTime *gtime.Time) error { + if len(orderCounts) == 0 { + return nil + } + orderCodes := make([]string, 0, len(orderCounts)) + for orderCode := range orderCounts { + orderCodes = append(orderCodes, orderCode) + } + sort.Strings(orderCodes) + activeCounts, err := activeBindCountsByOrderCodes(ctx, tx, orderCodes) + if err != nil { + return gerror.Wrap(err, "count active binds by order") + } + for _, orderCode := range orderCodes { + returnStatus := orderReturnStatusSuccess + if activeCounts[orderCode] > 0 { + returnStatus = orderReturnStatusPartial + } + update := orderUpdate{ + ReturnStatus: returnStatus, + UpdatedBy: task.Batch.CreatedBy, + UpdatedAt: now, + } + if order := orders[orderCode]; order != nil { + if refund := refunds[orderCode]; refund != nil && refund.Amount > 0 && order.UnrefundAmount > 0 { + remaining := order.UnrefundAmount - refund.Amount + if remaining < 0 { + remaining = 0 + } + update.UnrefundAmount = roundUnsubAmount(remaining) + } + } + _, err = tx.Ctx(ctx).Model(orderTable). + Where("order_sn", orderCode). + Data(update).Update() + if err != nil { + return gerror.Wrap(err, "update order return status") + } + if err = updateOrderItemExtra(ctx, tx, orderCode, task.Batch.BatchNo, orderCounts[orderCode], unsubTime, refunds[orderCode], task.Batch.CreatedBy); err != nil { + return err + } + } + return nil +} + +func activeBindCountsByOrderCodes(ctx context.Context, tx gdb.TX, orderCodes []string) (map[string]int, error) { + result := make(map[string]int) + if len(orderCodes) == 0 { + return result, nil + } + var rows []struct { + OrderCode string `orm:"order_code"` + Count int `orm:"count"` + } + err := tx.Ctx(ctx).Model(cardBindPkgTable). + Fields("order_code", "COUNT(1) AS count"). + WhereIn("order_code", orderCodes). + Group("order_code"). + Scan(&rows) + if err != nil { + return nil, err + } + for _, row := range rows { + if row.OrderCode != "" { + result[row.OrderCode] = row.Count + } + } + return result, nil +} + +func updateOrderItemExtra(ctx context.Context, tx gdb.TX, orderCode, batchNo string, quantity int, unsubTime *gtime.Time, refund *unsub.OrderRefund, userId int64) error { + var item orderItemQuery + if err := tx.Ctx(ctx).Model(orderItemTable). + Where("order_sn", orderCode). + Fields("item_id", "order_sn", "price", "quantity", "service_begin_time", "service_end_time", "extra_sku_data"). + Limit(1). + Scan(&item); err != nil { + return gerror.Wrap(err, "query order item") + } + if item.ItemId == 0 { + return nil + } + extra := decodeOrderItemExtra(item.ExtraSkuData) + extra.LastUnsubscribeBatchNo = batchNo + extra.LastUnsubscribeTime = formatUnsubTime(unsubTime) + extra.UnsubscribedQuantity = quantity + if refund != nil { + extra.LastUnsubscribeRefundAmount = roundUnsubAmount(refund.Amount) + extra.LastUnsubscribeRefundSerialNo = refund.SerialNo + extra.LastUnsubscribeRefundStatus = refund.Status + extra.LastUnsubscribeRefundReason = refund.Reason + } + payload, err := json.Marshal(extra) + if err != nil { + return gerror.Wrap(err, "encode order item extra") + } + _, err = tx.Ctx(ctx).Model(orderItemTable). + Where("item_id", item.ItemId). + Data(orderItemUpdate{ + ExtraSkuData: string(payload), + UpdatedBy: userId, + }).Update() + if err != nil { + return gerror.Wrap(err, "update order item extra") + } + return nil +} + +func decodeOrderItemExtra(raw string) orderItemExtra { + extra := orderItemExtra{} + raw = strings.TrimSpace(raw) + if raw == "" { + return extra + } + _ = json.Unmarshal([]byte(raw), &extra) + return extra +} + +func formatUnsubTime(t *gtime.Time) string { + if t == nil { + return "" + } + return t.Format("Y-m-d H:i:s") +} + +func returnCardsToStock(ctx context.Context, tx gdb.TX, task *unsub.Task, cards map[string]*cardQuery, stocks map[string]*stockQuery, now *gtime.Time) error { + rows := make([]stockInsert, 0, len(cards)) + for iccid, card := range cards { + if card == nil { + continue + } + if _, exists := stocks[iccid]; exists { + return gerror.Newf("ICCID=%s 已存在库存记录，不能重复回库", iccid) + } + rows = append(rows, stockInsert{ + BatchNo: card.InboundBatchNo, + Iccid: card.Iccid, + Msisdn: card.Msisdn, + Imsi: card.Imsi, + Imei: card.Imei, + ChanKey: card.ChanKey, + TenantId: card.TenantId, + CreatedDept: card.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + }) + } + if len(rows) == 0 { + return nil + } + if _, err := tx.Ctx(ctx).Model(stockTable).Data(rows).Insert(); err != nil { + return gerror.Wrap(err, "insert unsub stock") + } + _ = now + return nil +} + +func deleteFormalCards(ctx context.Context, tx gdb.TX, iccids []string) error { + if len(iccids) == 0 { + return nil + } + result, err := tx.Ctx(ctx).Model(cardInfoTable). + WhereIn("iccid", iccids). + Delete() + if err != nil { + return gerror.Wrap(err, "delete formal cards") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read formal card delete result") + } + if int(affected) != len(iccids) { + return gerror.New("部分正式卡已变化，请刷新后重试") + } + return nil +} + +func updateBatchDetailsSuccess(ctx context.Context, tx gdb.TX, task *unsub.Task, cards map[string]*cardQuery, binds map[string]*bindPkgQuery, now *gtime.Time) error { + for _, detail := range task.Details { + if detail == nil { + continue + } + card := cards[detail.ICCID] + bind := binds[detail.ICCID] + if card == nil || bind == nil { + continue + } + detail.SubscriptionNo = bind.SubscriptionNo + detail.OrderCode = bind.OrderCode + detail.PackageCode = bind.PackageCode + detail.PackageType = bind.PackageType + detail.EffectiveTime = formatUnsubTime(bind.EffectiveTime) + detail.ExpectedExpireTime = formatUnsubTime(bind.ExpectedExpireTime) + detail.ActualExpireTime = formatUnsubTime(now) + detail.ReturnedToStock = true + detail.StockDeptID = card.CreatedDept + detail.CardDeptID = card.DeptId + detail.StockBatchNo = card.InboundBatchNo + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: consts.BatchDetailStatusSuccess, + ErrorReason: "", + Snapshot: detail.Snapshot(), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update unsub success detail") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read unsub success detail result") + } + if affected != 1 { + return gerror.Newf("unsub detail %d is no longer running", detail.ID) + } + } + return nil +} + +func updateBatchMainSuccess(ctx context.Context, tx gdb.TX, task *unsub.Task, packageCode string, orderCounts map[string]int, quantity int, endTime, unsubTime *gtime.Time, refunds map[string]*unsub.OrderRefund) error { + orderCodes := make([]string, 0, len(orderCounts)) + for orderCode := range orderCounts { + orderCodes = append(orderCodes, orderCode) + } + sort.Strings(orderCodes) + info := buildSuccessBatchInfo(task, packageCode, orderCodes, quantity, unsubTime, refunds) + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Data(do.BatchMain{ + BatchAsyncStatus: consts.BatchAsyncStatusSuccess, + TotalCount: quantity, + ValidCount: quantity, + SuccessCount: quantity, + FailCount: 0, + RunResult: consts.BatchRunResultSuccess, + FailReason: "", + BatchInfo: info, + EndTime: endTime, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update unsub batch main for success") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read unsub batch main success update result") + } + if affected != 1 { + return gerror.New("unsub batch is no longer processing") + } + task.Batch.BatchInfo = info + return nil +} + +func finalizeFailure(ctx context.Context, tx gdb.TX, task *unsub.Task, reason string) error { + now := gtime.Now() + failCount := 0 + for _, detail := range task.Details { + if detail != nil { + detail.Reason = reasonFor(detail, reason) + if detail.Reason != "" { + failCount++ + } + } + } + for _, detail := range task.Details { + if detail == nil { + continue + } + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: consts.BatchDetailStatusFailed, + ErrorReason: detail.Reason, + Snapshot: detail.Snapshot(), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update unsub business failure detail") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read unsub business failure detail result") + } + if affected != 1 { + return gerror.Newf("unsub detail %d is no longer running", detail.ID) + } + } + return updateMainForValidation(ctx, tx, task, len(task.Details), failCount, reason, now) +} + +func reasonFor(detail *unsub.Detail, fallback string) string { + if detail == nil { + return fallback + } + if strings.TrimSpace(detail.Reason) == "" { + return fallback + } + return detail.Reason +} + +func buildSuccessBatchInfo(task *unsub.Task, packageCode string, orderCodes []string, quantity int, unsubTime *gtime.Time, refunds map[string]*unsub.OrderRefund) *gjson.Json { + info := make(map[string]any) + if task.Batch.BatchInfo != nil { + for key, value := range task.Batch.BatchInfo.Map() { + info[key] = value + } + } + if packageCode != "" { + info["packageCode"] = packageCode + } + info["quantity"] = quantity + info["orderCodes"] = orderCodes + info["unsubTime"] = formatUnsubTime(unsubTime) + if len(refunds) > 0 { + refundDetails := make(map[string]any, len(refunds)) + totalRefundAmount := 0.0 + for orderCode, refund := range refunds { + if refund == nil { + continue + } + amount := roundUnsubAmount(refund.Amount) + totalRefundAmount += amount + refundDetails[orderCode] = map[string]any{ + "amount": amount, + "serialNo": refund.SerialNo, + "status": refund.Status, + "reason": refund.Reason, + } + } + info["refundMode"] = "balance" + info["refundAmount"] = roundUnsubAmount(totalRefundAmount) + info["refundDetails"] = refundDetails + } + if task.Batch.SourceBatchNo != "" { + info["sourceBatchNo"] = task.Batch.SourceBatchNo + } + if task.Batch.Remark != "" { + info["remark"] = task.Batch.Remark + } + return gjson.New(info) +} + +func uniqueSortedStrings(values []string) []string { + if len(values) == 0 { + return nil + } + seen := make(map[string]struct{}, len(values)) + result := make([]string, 0, len(values)) + for _, value := range values { + value = strings.TrimSpace(value) + if value == "" { + continue + } + if _, ok := seen[value]; ok { + continue + } + seen[value] = struct{}{} + result = append(result, value) + } + sort.Strings(result) + return result +} + +func (r *Repository) SaveResultFile(ctx context.Context, task *unsub.Task, request batchresult.Request) error { + artifact, err := r.artifactGenerator().Create(ctx, request) + if err != nil { + return r.updateResultFileStatus(ctx, task, "failed", 0, 0, err.Error()) + } + now := gtime.Now() + successCount := 0 + for _, detail := range task.Details { + if detail != nil && detail.Reason == "" { + successCount++ + } + } + failCount := len(task.Details) - successCount + var ( + ossID int64 + downloadID int64 + ) + txErr := r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + var insertErr error + ossID, insertErr = tx.Ctx(ctx).Model(dao.SysOss.Table()).Data(do.SysOss{ + TenantId: task.Batch.TenantID, + FileName: artifact.FileName, + OriginalName: artifact.OriginalName, + FileSuffix: artifact.FileSuffix, + Path: artifact.Path, + Url: artifact.URL, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + Service: "local", + Md5: artifact.MD5, + FileSize: artifact.Size, + }).InsertAndGetId() + if insertErr != nil { + return gerror.Wrap(insertErr, "insert unsub result OSS") + } + downloadID, insertErr = tx.Ctx(ctx).Model(dao.TResourceBatchDownload.Table()).Data(do.TResourceBatchDownload{ + BatchNo: task.Batch.BatchNo, + TaskName: batchresult.TaskName(task.Batch.BatchType), + ResourceType: consts.ResourceTypeExcel, + ExportScene: task.Batch.BatchType, + TotalCount: len(task.Details), + SuccessCount: successCount, + FailCount: failCount, + DownloadStatus: consts.DownloadStatusGenerated, + OssId: ossID, + FileName: artifact.FileName, + FileExt: artifact.FileSuffix, + FileUrl: artifact.URL, + FilePath: artifact.Path, + FileSize: artifact.Size, + FileMd5: artifact.MD5, + DownloadCount: 0, + MaxDownloadCount: 0, + DeptId: task.Batch.CreatedDept, + StartTime: task.Batch.StartTime, + FinishTime: now, + TenantId: task.Batch.TenantID, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + }).InsertAndGetId() + if insertErr != nil { + return gerror.Wrap(insertErr, "insert unsub result download") + } + return updateBatchInfoOnly(ctx, tx, task, buildResultBatchInfo(task, "success", ossID, downloadID, "")) + }) + if txErr != nil { + batchresult.RemoveLocalFile(artifact) + return r.updateResultFileStatus(ctx, task, "failed", 0, 0, txErr.Error()) + } + return nil +} + +func (r *Repository) artifactGenerator() batchresult.Generator { + if r.resultFiles != nil { + return r.resultFiles + } + return batchresult.New() +} + +func (r *Repository) updateResultFileStatus(ctx context.Context, task *unsub.Task, status string, ossID, downloadID int64, reason string) error { + return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + return updateBatchInfoOnly(ctx, tx, task, buildResultBatchInfo(task, status, ossID, downloadID, reason)) + }) +} + +func updateBatchInfoOnly(ctx context.Context, tx gdb.TX, task *unsub.Task, info *gjson.Json) error { + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeSimCardUnsub). + Data(do.BatchMain{ + BatchInfo: info, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "update unsub result file status") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read unsub result file update result") + } + if affected != 1 { + return gerror.New("unsub batch disappeared while saving result file") + } + task.Batch.BatchInfo = info + return nil +} + +func buildResultBatchInfo(task *unsub.Task, status string, ossID, downloadID int64, reason string) *gjson.Json { + info := make(map[string]any) + if task.Batch.BatchInfo != nil { + for key, value := range task.Batch.BatchInfo.Map() { + info[key] = value + } + } + resultFile := map[string]any{"status": status} + if ossID > 0 { + resultFile["ossId"] = ossID + } + if downloadID > 0 { + resultFile["downloadId"] = downloadID + } + if reason != "" { + resultFile["error"] = reason + } + info["resultFile"] = resultFile + return gjson.New(info) +} + +var _ unsub.Repository = (*Repository)(nil) diff --git a/internal/repository/unsub/repository_test.go b/internal/repository/unsub/repository_test.go new file mode 100644 index 0000000..e989d38 --- /dev/null +++ b/internal/repository/unsub/repository_test.go @@ -0,0 +1,790 @@ +package unsub + +import ( + "context" + "math" + "testing" + + _ "github.com/gogf/gf/contrib/drivers/sqlite/v2" + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" + + batchdomain "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + unsub "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/unsub" +) + +const ( + testTenantID = "000000" + testDeptID = int64(100) + testPackage = "PKG-A" + testSource = "OUT-1" + testInbound = "IN-1" +) + +func newTestRepository(t *testing.T) *Repository { + t.Helper() + ctx := context.Background() + db, err := gdb.New(gdb.ConfigNode{ + Type: "sqlite", + Link: "sqlite::@file(:memory:)", + }) + if err != nil { + t.Fatalf("create sqlite database: %v", err) + } + statements := []string{ + `CREATE TABLE batch_main ( + source_channel TEXT NOT NULL DEFAULT 'legacy', + execution_mode INTEGER NOT NULL DEFAULT 0, + id INTEGER PRIMARY KEY AUTOINCREMENT, + batch_no TEXT NOT NULL, + batch_type INTEGER NOT NULL, + batch_async_status INTEGER NOT NULL, + run_result INTEGER NOT NULL DEFAULT 0, + fail_reason TEXT, + start_time DATETIME, + end_time DATETIME, + dept_id INTEGER NOT NULL DEFAULT 0, + source_batch_no TEXT, + created_by INTEGER NOT NULL DEFAULT 0, + created_dept INTEGER NOT NULL DEFAULT 0, + tenant_id TEXT NOT NULL DEFAULT '', + batch_info TEXT + )`, + `CREATE TABLE batch_detail ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + batch_no TEXT NOT NULL, + batch_type INTEGER NOT NULL, + detail_status INTEGER NOT NULL, + error_reason TEXT, + snapshot TEXT, + item_id TEXT, + row_no INTEGER NOT NULL DEFAULT 0 + )`, + `CREATE TABLE t_sim_card_info ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + iccid TEXT NOT NULL, + msisdn TEXT, + imsi TEXT, + imei TEXT, + chan_key TEXT, + dept_id INTEGER NOT NULL DEFAULT 0, + created_dept INTEGER NOT NULL DEFAULT 0, + current_package_code TEXT, + package_status INTEGER NOT NULL DEFAULT 0, + inbound_batch_no TEXT, + outbound_batch_no TEXT, + tenant_id TEXT NOT NULL DEFAULT '' + )`, + `CREATE TABLE t_sim_card_bind_pkg ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + subscription_no TEXT, + batch_no TEXT, + iccid TEXT, + msisdn TEXT, + dept_id TEXT, + package_type INTEGER NOT NULL DEFAULT 0, + package_code TEXT, + previous_package_code TEXT, + business_type INTEGER NOT NULL DEFAULT 0, + effective_status INTEGER NOT NULL DEFAULT 0, + handle_time DATETIME, + effective_time DATETIME, + expected_expire_time DATETIME, + actual_expire_time DATETIME, + duration_months INTEGER NOT NULL DEFAULT 0, + gift_duration_months INTEGER NOT NULL DEFAULT 0, + order_code TEXT, + remark TEXT, + tenant_id TEXT NOT NULL DEFAULT '', + created_dept INTEGER NOT NULL DEFAULT 0, + created_by INTEGER NOT NULL DEFAULT 0, + created_at DATETIME + )`, + `CREATE TABLE t_sim_bind_package_logs ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + subscription_no TEXT, + batch_no TEXT, + iccid TEXT, + msisdn TEXT, + dept_id TEXT, + package_type INTEGER, + package_code TEXT, + previous_package_code TEXT, + business_type INTEGER, + effective_status INTEGER, + handle_time DATETIME, + effective_time DATETIME, + expected_expire_time DATETIME, + actual_expire_time DATETIME, + duration_months INTEGER, + gift_duration_months INTEGER, + order_code TEXT, + remark TEXT, + tenant_id TEXT, + created_dept INTEGER, + created_by INTEGER, + updated_by INTEGER + )`, + `CREATE TABLE t_sim_card_stock ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + batch_no TEXT, + iccid TEXT, + msisdn TEXT, + imsi TEXT, + imei TEXT, + chan_key TEXT, + tenant_id TEXT NOT NULL DEFAULT '', + created_dept INTEGER NOT NULL DEFAULT 0, + created_by INTEGER NOT NULL DEFAULT 0, + updated_by INTEGER NOT NULL DEFAULT 0 + )`, + `CREATE TABLE t_order ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + order_sn TEXT, + customer_agent_id INTEGER NOT NULL DEFAULT 0, + pay_type INTEGER NOT NULL DEFAULT 0, + pay_status INTEGER NOT NULL DEFAULT 0, + pay_order_sn TEXT, + total_amount REAL NOT NULL DEFAULT 0, + paid_amount REAL NOT NULL DEFAULT 0, + balance REAL NOT NULL DEFAULT 0, + unrefund_amount REAL NOT NULL DEFAULT 0, + tenant_id TEXT NOT NULL DEFAULT '', + created_dept INTEGER NOT NULL DEFAULT 0, + return_status INTEGER NOT NULL DEFAULT 0, + updated_by INTEGER NOT NULL DEFAULT 0 + )`, + `CREATE TABLE t_order_item ( + item_id INTEGER PRIMARY KEY AUTOINCREMENT, + order_id INTEGER, + order_sn TEXT, + price REAL NOT NULL DEFAULT 0, + quantity INTEGER NOT NULL DEFAULT 0, + service_begin_time DATETIME, + service_end_time DATETIME, + tenant_id TEXT NOT NULL DEFAULT '', + extra_sku_data TEXT, + updated_by INTEGER NOT NULL DEFAULT 0 + )`, + `CREATE TABLE t_agent_balance ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + dept_id INTEGER NOT NULL DEFAULT 0, + balance REAL NOT NULL DEFAULT 0, + status INTEGER NOT NULL DEFAULT 0, + tenant_id TEXT NOT NULL DEFAULT '' + )`, + `CREATE TABLE t_balance_detail ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + dept_id INTEGER NOT NULL DEFAULT 0, + serial_no TEXT NOT NULL, + transaction_time DATETIME, + income_expense_type INTEGER NOT NULL DEFAULT 0, + transaction_type INTEGER NOT NULL DEFAULT 0, + transaction_channel INTEGER NOT NULL DEFAULT 0, + transaction_status INTEGER NOT NULL DEFAULT 0, + amount REAL NOT NULL DEFAULT 0, + before_balance REAL NOT NULL DEFAULT 0, + balance_after REAL NOT NULL DEFAULT 0, + order_sn TEXT, + remark TEXT, + tenant_id TEXT NOT NULL DEFAULT '', + created_dept INTEGER NOT NULL DEFAULT 0, + created_by INTEGER NOT NULL DEFAULT 0, + created_at DATETIME + )`, + } + for _, statement := range statements { + if _, err = db.Exec(ctx, statement); err != nil { + t.Fatalf("create test schema: %v", err) + } + } + t.Cleanup(func() { _ = db.Close(ctx) }) + return &Repository{db: db, lockEnabled: false} +} + +func seedProcessingBatch(t *testing.T, ctx context.Context, db gdb.DB, batchNo, sourceBatchNo string) { + t.Helper() + info := gjson.New(map[string]any{ + "sourceBatchNo": sourceBatchNo, + }) + if _, err := db.Model(dao.BatchMain.Table()).Ctx(ctx).Data(do.BatchMain{ + BatchNo: batchNo, + BatchType: consts.BatchTypeSimCardUnsub, + BatchAsyncStatus: consts.BatchAsyncStatusProcessing, + RunResult: batchdomain.RunResultRunning, + DeptId: testDeptID, + SourceBatchNo: sourceBatchNo, + CreatedBy: 1, + CreatedDept: 1, + TenantId: testTenantID, + BatchInfo: info, + }).Insert(); err != nil { + t.Fatalf("insert batch main: %v", err) + } +} + +func seedRunningDetail(t *testing.T, ctx context.Context, db gdb.DB, batchNo string, rowNo int, iccid, msisdn, imsi, sourceBatchNo, reason string) int64 { + t.Helper() + snap := gjson.New(map[string]any{ + "ICCID": iccid, + "MSISDN": msisdn, + "IMSI": imsi, + "authorized": true, + "sourceBatchNo": sourceBatchNo, + }) + id, err := db.Model(dao.BatchDetail.Table()).Ctx(ctx).Data(do.BatchDetail{ + BatchNo: batchNo, + BatchType: consts.BatchTypeSimCardUnsub, + DetailStatus: consts.BatchDetailStatusRunning, + RowNo: rowNo, + ItemId: iccid, + Snapshot: snap, + ErrorReason: reason, + }).InsertAndGetId() + if err != nil { + t.Fatalf("insert batch detail: %v", err) + } + return id +} + +func seedCard(t *testing.T, ctx context.Context, db gdb.DB, iccid, msisdn, imsi, packageCode string, deptID int64) { + t.Helper() + _, err := db.Model(cardInfoTable).Ctx(ctx).Data(do.TSimCardInfo{ + Iccid: iccid, + Msisdn: msisdn, + Imsi: imsi, + DeptId: deptID, + CreatedDept: deptID, + CurrentPackageCode: packageCode, + PackageStatus: pkgStatusEffective, + InboundBatchNo: testInbound, + OutboundBatchNo: testSource, + TenantId: testTenantID, + }).Insert() + if err != nil { + t.Fatalf("insert sim card: %v", err) + } +} + +func seedActiveBind(t *testing.T, ctx context.Context, db gdb.DB, iccid, packageCode, orderCode, sourceBatchNo string) { + t.Helper() + _, err := db.Model(cardBindPkgTable).Ctx(ctx).Data(map[string]any{ + "subscription_no": "SUB-" + iccid, + "batch_no": sourceBatchNo, + "iccid": iccid, + "msisdn": "13" + iccid[len(iccid)-8:], + "dept_id": "100", + "package_type": 1, + "package_code": packageCode, + "business_type": 0, + "effective_status": bindStatusEffective, + "order_code": orderCode, + "remark": "", + "tenant_id": testTenantID, + "created_dept": 1, + "created_by": 1, + }).Insert() + if err != nil { + t.Fatalf("insert bind: %v", err) + } +} + +func seedStock(t *testing.T, ctx context.Context, db gdb.DB, iccid string) { + t.Helper() + _, err := db.Model(stockTable).Ctx(ctx).Data(map[string]any{ + "iccid": iccid, + "tenant_id": testTenantID, + }).Insert() + if err != nil { + t.Fatalf("insert stock: %v", err) + } +} + +func seedOrder(t *testing.T, ctx context.Context, db gdb.DB, orderCode string) { + t.Helper() + _, err := db.Model(orderTable).Ctx(ctx).Data(map[string]any{ + "order_sn": orderCode, + "return_status": 0, + "tenant_id": testTenantID, + }).Insert() + if err != nil { + t.Fatalf("insert order: %v", err) + } +} + +func seedOrderItem(t *testing.T, ctx context.Context, db gdb.DB, orderCode, extra string) { + t.Helper() + _, err := db.Model(orderItemTable).Ctx(ctx).Data(map[string]any{ + "order_sn": orderCode, + "tenant_id": testTenantID, + "extra_sku_data": extra, + }).Insert() + if err != nil { + t.Fatalf("insert order item: %v", err) + } +} + +func assertTableCount(t *testing.T, ctx context.Context, db gdb.DB, table string, expected int, where string, args ...any) { + t.Helper() + model := db.Model(table).Ctx(ctx).Where(where, args...) + count, err := model.Count() + if err != nil { + t.Fatalf("count %s: %v", table, err) + } + if count != expected { + t.Fatalf("%s count = %d, want %d", table, count, expected) + } +} + +func assertBatchMainStatus(t *testing.T, ctx context.Context, db gdb.DB, batchNo string, wantAsync, wantRun int) { + t.Helper() + var row struct { + Async int `orm:"batch_async_status"` + Run int `orm:"run_result"` + } + if err := db.Model(dao.BatchMain.Table()).Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Fields("batch_async_status", "run_result"). + Scan(&row); err != nil { + t.Fatalf("query batch main: %v", err) + } + if row.Async != wantAsync || row.Run != wantRun { + t.Fatalf("batch %s status = (async=%d, run=%d), want (async=%d, run=%d)", batchNo, row.Async, row.Run, wantAsync, wantRun) + } +} + +func assertBatchDetailStatus(t *testing.T, ctx context.Context, db gdb.DB, batchNo string, want int) { + t.Helper() + var row struct { + Status int `orm:"detail_status"` + } + if err := db.Model(dao.BatchDetail.Table()).Ctx(ctx). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Fields("detail_status"). + Scan(&row); err != nil { + t.Fatalf("query batch detail: %v", err) + } + if row.Status != want { + t.Fatalf("batch %s detail status = %d, want %d", batchNo, row.Status, want) + } +} + +func TestLoadReadsTaskAndDetails(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "UNS-1" + seedProcessingBatch(t, ctx, repo.db, batchNo, testSource) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", testSource, "") + seedRunningDetail(t, ctx, repo.db, batchNo, 2, "898600000000002", "13800000002", "460001000000002", testSource, "") + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if task.Batch.BatchNo != batchNo { + t.Fatalf("batch no = %q, want %q", task.Batch.BatchNo, batchNo) + } + if task.Batch.SourceBatchNo != testSource { + t.Fatalf("source batch = %q, want %q", task.Batch.SourceBatchNo, testSource) + } + if len(task.Details) != 2 { + t.Fatalf("details = %d, want 2", len(task.Details)) + } + if task.Details[0].ICCID != "898600000000001" { + t.Fatalf("first detail ICCID = %q", task.Details[0].ICCID) + } +} + +func TestLoadTreatsSelfBatchNoAsEmptySourceBatch(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "UNS-1" + seedProcessingBatch(t, ctx, repo.db, batchNo, batchNo) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", "", "") + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if task.Batch.SourceBatchNo != "" { + t.Fatalf("source batch = %q, want empty when equals batchNo", task.Batch.SourceBatchNo) + } +} + +func TestSaveValidationFailureMarksAllDetailsAndMainFailed(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "UNS-1" + seedProcessingBatch(t, ctx, repo.db, batchNo, testSource) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", testSource, "snapshot has reason") + seedRunningDetail(t, ctx, repo.db, batchNo, 2, "898600000000002", "13800000002", "460001000000002", testSource, "") + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if err = repo.SaveValidationFailure(ctx, task); err != nil { + t.Fatalf("SaveValidationFailure() error = %v", err) + } + assertBatchMainStatus(t, ctx, repo.db, batchNo, consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed) + assertBatchDetailStatus(t, ctx, repo.db, batchNo, consts.BatchDetailStatusFailed) + assertTableCount(t, ctx, repo.db, cardInfoTable, 0, "iccid IN (?, ?)", "898600000000001", "898600000000002") + assertTableCount(t, ctx, repo.db, cardBindPkgTable, 0, "iccid IN (?, ?)", "898600000000001", "898600000000002") + assertTableCount(t, ctx, repo.db, stockTable, 0, "1=1") +} + +func TestCommitBusinessHappyPath(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "UNS-1" + seedProcessingBatch(t, ctx, repo.db, batchNo, testSource) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", testSource, "") + seedRunningDetail(t, ctx, repo.db, batchNo, 2, "898600000000002", "13800000002", "460001000000002", testSource, "") + seedCard(t, ctx, repo.db, "898600000000001", "13800000001", "460001000000001", testPackage, testDeptID) + seedCard(t, ctx, repo.db, "898600000000002", "13800000002", "460001000000002", testPackage, testDeptID) + seedActiveBind(t, ctx, repo.db, "898600000000001", testPackage, "ORD-1", testSource) + seedActiveBind(t, ctx, repo.db, "898600000000002", testPackage, "ORD-1", testSource) + seedOrder(t, ctx, repo.db, "ORD-1") + seedOrderItem(t, ctx, repo.db, "ORD-1", `{}`) + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if err = repo.CommitBusiness(ctx, task); err != nil { + t.Fatalf("CommitBusiness() error = %v", err) + } + + assertTableCount(t, ctx, repo.db, cardInfoTable, 0, "iccid IN (?, ?)", "898600000000001", "898600000000002") + assertTableCount(t, ctx, repo.db, cardBindPkgTable, 0, "iccid IN (?, ?)", "898600000000001", "898600000000002") + assertTableCount(t, ctx, repo.db, bindPackageLogsTable, 2, "iccid IN (?, ?)", "898600000000001", "898600000000002") + assertTableCount(t, ctx, repo.db, stockTable, 2, "iccid IN (?, ?)", "898600000000001", "898600000000002") + assertBatchMainStatus(t, ctx, repo.db, batchNo, consts.BatchAsyncStatusSuccess, consts.BatchRunResultSuccess) + assertBatchDetailStatus(t, ctx, repo.db, batchNo, consts.BatchDetailStatusSuccess) + assertOrderReturnStatus(t, ctx, repo.db, "ORD-1", orderReturnStatusSuccess) + assertOrderItemExtra(t, ctx, repo.db, "ORD-1", batchNo, 2) +} + +func TestCommitBusinessRefundsBalanceByPaidPeriodOnly(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "UNS-REFUND-1" + orderCode := "ORD-REFUND-1" + effectiveAt := gtime.New("2026-01-01 00:00:00") + cutoffAt := gtime.New("2026-07-01 00:00:00") + + seedProcessingBatch(t, ctx, repo.db, batchNo, testSource) + if _, err := repo.db.Model(dao.BatchMain.Table()).Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Data(map[string]any{"start_time": cutoffAt}).Update(); err != nil { + t.Fatalf("update batch start time: %v", err) + } + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", testSource, "") + seedCard(t, ctx, repo.db, "898600000000001", "13800000001", "460001000000001", testPackage, testDeptID) + seedActiveBind(t, ctx, repo.db, "898600000000001", testPackage, orderCode, testSource) + if _, err := repo.db.Model(cardBindPkgTable).Ctx(ctx). + Where("order_code", orderCode). + Data(map[string]any{ + "effective_time": effectiveAt, + "expected_expire_time": gtime.New("2027-04-01 23:59:59"), + "duration_months": 12, + "gift_duration_months": 3, + }).Update(); err != nil { + t.Fatalf("update bind period: %v", err) + } + if _, err := repo.db.Model(orderTable).Ctx(ctx).Data(map[string]any{ + "order_sn": orderCode, + "customer_agent_id": testDeptID, + "pay_type": consts.PayTypeBalance, + "pay_status": consts.PayStatusPaid, + "pay_order_sn": "RENPAY-REFUND-1", + "total_amount": 120, + "paid_amount": 120, + "balance": 120, + "unrefund_amount": 120, + "tenant_id": testTenantID, + "created_dept": testDeptID, + }).Insert(); err != nil { + t.Fatalf("insert refund order: %v", err) + } + if _, err := repo.db.Model(orderItemTable).Ctx(ctx).Data(map[string]any{ + "order_sn": orderCode, + "price": 120, + "quantity": 1, + "service_begin_time": effectiveAt, + "service_end_time": gtime.New("2027-04-01 23:59:59"), + "tenant_id": testTenantID, + }).Insert(); err != nil { + t.Fatalf("insert refund order item: %v", err) + } + if _, err := repo.db.Model(agentBalanceTable).Ctx(ctx).Data(map[string]any{ + "dept_id": testDeptID, + "balance": 10, + "status": 0, + "tenant_id": testTenantID, + }).Insert(); err != nil { + t.Fatalf("insert refund account: %v", err) + } + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if err = repo.CommitBusiness(ctx, task); err != nil { + t.Fatalf("CommitBusiness() error = %v", err) + } + + var account struct { + Balance float64 `orm:"balance"` + } + if err = repo.db.Model(agentBalanceTable).Ctx(ctx).Where("dept_id", testDeptID).Scan(&account); err != nil { + t.Fatalf("query refund account: %v", err) + } + if want := 70.49; math.Abs(account.Balance-want) > 0.01 { + t.Fatalf("refund balance = %.2f, want %.2f", account.Balance, want) + } + assertTableCount(t, ctx, repo.db, balanceDetailTable, 1, "serial_no = ?", unsubscribeRefundSerialNo(batchNo, orderCode)) + assertOrderReturnStatus(t, ctx, repo.db, orderCode, orderReturnStatusSuccess) +} + +func TestCommitBusinessFailsOnNoActiveBind(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "UNS-1" + seedProcessingBatch(t, ctx, repo.db, batchNo, testSource) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", testSource, "") + seedCard(t, ctx, repo.db, "898600000000001", "13800000001", "460001000000001", testPackage, testDeptID) + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if err = repo.CommitBusiness(ctx, task); err != nil { + t.Fatalf("CommitBusiness() returned error = %v, want nil (validation failure handled internally)", err) + } + assertTableCount(t, ctx, repo.db, cardInfoTable, 1, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, cardBindPkgTable, 0, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, bindPackageLogsTable, 0, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, stockTable, 0, "iccid = ?", "898600000000001") + assertBatchMainStatus(t, ctx, repo.db, batchNo, consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed) + assertBatchDetailStatus(t, ctx, repo.db, batchNo, consts.BatchDetailStatusFailed) +} + +func TestCommitBusinessFailsOnPackageMismatch(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "UNS-1" + seedProcessingBatch(t, ctx, repo.db, batchNo, testSource) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", testSource, "") + seedCard(t, ctx, repo.db, "898600000000001", "13800000001", "460001000000001", testPackage, testDeptID) + seedActiveBind(t, ctx, repo.db, "898600000000001", "PKG-OTHER", "ORD-1", testSource) + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if err = repo.CommitBusiness(ctx, task); err != nil { + t.Fatalf("CommitBusiness() returned error = %v, want nil", err) + } + assertTableCount(t, ctx, repo.db, cardInfoTable, 1, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, cardBindPkgTable, 1, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, bindPackageLogsTable, 0, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, stockTable, 0, "iccid = ?", "898600000000001") + assertBatchMainStatus(t, ctx, repo.db, batchNo, consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed) +} + +func TestCommitBusinessFailsOnDeptMismatch(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "UNS-1" + seedProcessingBatch(t, ctx, repo.db, batchNo, testSource) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", testSource, "") + seedCard(t, ctx, repo.db, "898600000000001", "13800000001", "460001000000001", testPackage, 999) + seedActiveBind(t, ctx, repo.db, "898600000000001", testPackage, "ORD-1", testSource) + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if err = repo.CommitBusiness(ctx, task); err != nil { + t.Fatalf("CommitBusiness() returned error = %v, want nil", err) + } + assertTableCount(t, ctx, repo.db, cardInfoTable, 1, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, cardBindPkgTable, 1, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, bindPackageLogsTable, 0, "iccid = ?", "898600000000001") +} + +func TestCommitBusinessFailsOnSourceBatchMismatch(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "UNS-1" + seedProcessingBatch(t, ctx, repo.db, batchNo, testSource) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", testSource, "") + seedCard(t, ctx, repo.db, "898600000000001", "13800000001", "460001000000001", testPackage, testDeptID) + seedActiveBind(t, ctx, repo.db, "898600000000001", testPackage, "ORD-1", "OUT-OTHER") + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if err = repo.CommitBusiness(ctx, task); err != nil { + t.Fatalf("CommitBusiness() returned error = %v, want nil", err) + } + assertTableCount(t, ctx, repo.db, cardInfoTable, 1, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, cardBindPkgTable, 1, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, stockTable, 0, "iccid = ?", "898600000000001") + assertBatchMainStatus(t, ctx, repo.db, batchNo, consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed) +} + +func TestCommitBusinessFailsOnExistingStock(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "UNS-1" + seedProcessingBatch(t, ctx, repo.db, batchNo, testSource) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", testSource, "") + seedCard(t, ctx, repo.db, "898600000000001", "13800000001", "460001000000001", testPackage, testDeptID) + seedActiveBind(t, ctx, repo.db, "898600000000001", testPackage, "ORD-1", testSource) + seedStock(t, ctx, repo.db, "898600000000001") + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if err = repo.CommitBusiness(ctx, task); err != nil { + t.Fatalf("CommitBusiness() returned error = %v, want nil", err) + } + assertTableCount(t, ctx, repo.db, cardInfoTable, 1, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, cardBindPkgTable, 1, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, stockTable, 1, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, bindPackageLogsTable, 0, "iccid = ?", "898600000000001") +} + +func TestCommitBusinessFailsOnMultiplePackagesInBatch(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "UNS-1" + seedProcessingBatch(t, ctx, repo.db, batchNo, testSource) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", testSource, "") + seedRunningDetail(t, ctx, repo.db, batchNo, 2, "898600000000002", "13800000002", "460001000000002", testSource, "") + seedCard(t, ctx, repo.db, "898600000000001", "13800000001", "460001000000001", "PKG-A", testDeptID) + seedCard(t, ctx, repo.db, "898600000000002", "13800000002", "460001000000002", "PKG-B", testDeptID) + seedActiveBind(t, ctx, repo.db, "898600000000001", "PKG-A", "ORD-1", testSource) + seedActiveBind(t, ctx, repo.db, "898600000000002", "PKG-B", "ORD-2", testSource) + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if err = repo.CommitBusiness(ctx, task); err != nil { + t.Fatalf("CommitBusiness() returned error = %v, want nil", err) + } + assertTableCount(t, ctx, repo.db, cardInfoTable, 2, "iccid IN (?, ?)", "898600000000001", "898600000000002") + assertTableCount(t, ctx, repo.db, cardBindPkgTable, 2, "iccid IN (?, ?)", "898600000000001", "898600000000002") + assertTableCount(t, ctx, repo.db, bindPackageLogsTable, 0, "1=1") + assertTableCount(t, ctx, repo.db, stockTable, 0, "1=1") + assertBatchMainStatus(t, ctx, repo.db, batchNo, consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed) +} + +func TestCommitBusinessRollsBackOnOrderUpdateFailure(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "UNS-1" + seedProcessingBatch(t, ctx, repo.db, batchNo, testSource) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", testSource, "") + seedCard(t, ctx, repo.db, "898600000000001", "13800000001", "460001000000001", testPackage, testDeptID) + seedActiveBind(t, ctx, repo.db, "898600000000001", testPackage, "ORD-1", testSource) + seedOrder(t, ctx, repo.db, "ORD-1") + seedOrderItem(t, ctx, repo.db, "ORD-1", `{}`) + + if _, err := repo.db.Exec(ctx, "DROP TABLE t_order"); err != nil { + t.Fatalf("drop order table: %v", err) + } + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if err = repo.CommitBusiness(ctx, task); err == nil { + t.Fatal("expected CommitBusiness to fail when order table is missing") + } + assertTableCount(t, ctx, repo.db, cardInfoTable, 1, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, cardBindPkgTable, 1, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, bindPackageLogsTable, 0, "iccid = ?", "898600000000001") + assertTableCount(t, ctx, repo.db, stockTable, 0, "iccid = ?", "898600000000001") + assertBatchMainStatus(t, ctx, repo.db, batchNo, batchdomain.AsyncStatusProcessing, batchdomain.RunResultRunning) +} + +func TestSaveResultFileFailureOnlyMarksBatchInfo(t *testing.T) { + ctx := context.Background() + repo := newTestRepository(t) + batchNo := "UNS-1" + seedProcessingBatch(t, ctx, repo.db, batchNo, testSource) + seedRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", testSource, "") + + task, err := repo.Load(ctx, batchNo) + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if err = repo.updateResultFileStatus(ctx, task, "failed", 0, 0, "disk full"); err != nil { + t.Fatalf("updateResultFileStatus() error = %v", err) + } + var row struct { + Info string `orm:"batch_info"` + } + if err = repo.db.Model(dao.BatchMain.Table()).Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Fields("batch_info"). + Scan(&row); err != nil { + t.Fatalf("query batch main: %v", err) + } + info := gjson.New([]byte(row.Info)) + if info.Get("resultFile.status").String() != "failed" { + t.Fatalf("resultFile.status = %s, want failed", info.Get("resultFile.status").String()) + } + if info.Get("resultFile.error").String() != "disk full" { + t.Fatalf("resultFile.error = %s, want disk full", info.Get("resultFile.error").String()) + } + assertBatchMainStatus(t, ctx, repo.db, batchNo, batchdomain.AsyncStatusProcessing, batchdomain.RunResultRunning) +} + +func assertOrderReturnStatus(t *testing.T, ctx context.Context, db gdb.DB, orderCode string, want int) { + t.Helper() + var row struct { + ReturnStatus int `orm:"return_status"` + } + if err := db.Model(orderTable).Ctx(ctx). + Where("order_sn", orderCode). + Fields("return_status"). + Scan(&row); err != nil { + t.Fatalf("query order return status: %v", err) + } + if row.ReturnStatus != want { + t.Fatalf("order %s return_status = %d, want %d", orderCode, row.ReturnStatus, want) + } +} + +func assertOrderItemExtra(t *testing.T, ctx context.Context, db gdb.DB, orderCode, batchNo string, wantQuantity int) { + t.Helper() + var row struct { + Extra string `orm:"extra_sku_data"` + } + if err := db.Model(orderItemTable).Ctx(ctx). + Where("order_sn", orderCode). + Fields("extra_sku_data"). + Scan(&row); err != nil { + t.Fatalf("query order item extra: %v", err) + } + info := gjson.New([]byte(row.Extra)) + if info.Get("lastUnsubscribeBatchNo").String() != batchNo { + t.Fatalf("lastUnsubscribeBatchNo = %s, want %s", info.Get("lastUnsubscribeBatchNo").String(), batchNo) + } + if int(info.Get("unsubscribedQuantity").Int()) != wantQuantity { + t.Fatalf("unsubscribedQuantity = %d, want %d", info.Get("unsubscribedQuantity").Int(), wantQuantity) + } +} + +var _ unsub.Repository = (*Repository)(nil) diff --git a/internal/service/addpackage/executor.go b/internal/service/addpackage/executor.go new file mode 100644 index 0000000..bd921f0 --- /dev/null +++ b/internal/service/addpackage/executor.go @@ -0,0 +1,1428 @@ +package addpackage + +import ( + "context" + "database/sql" + "encoding/json" + "errors" + "fmt" + "sort" + "strings" + "time" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" + renewchangeservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/renewchange" +) + +type executor struct { + repo *renewchangeservice.Repository + batchType int + poolMode bool + nowFunc NowFunc +} + +type flowPoolRow struct { + Id int64 `orm:"id"` + FlowPoolCode string `orm:"flow_pool_code"` + DeptId int64 `orm:"dept_id"` + PoolPackageCode int64 `orm:"pool_package_code"` + AddOnFlowMb float64 `orm:"add_on_flow_mb"` + TotalFlowMb float64 `orm:"total_flow_mb"` + RemainFlowMb float64 `orm:"remain_flow_mb"` + PoolStatus int `orm:"pool_status"` + TenantId string `orm:"tenant_id"` + CreatedDept int64 `orm:"created_dept"` + CreatedBy int64 `orm:"created_by"` + UpdatedBy int64 `orm:"updated_by"` + UpdatedAt *gtime.Time `orm:"updated_at"` +} + +type sysDeptRow struct { + DeptId int64 `orm:"dept_id"` + ParentId int64 `orm:"parent_id"` + TenantId string `orm:"tenant_id"` + DeletedAt *gtime.Time `orm:"deleted_at"` +} + +type agentBalanceRow struct { + Id int64 `orm:"id"` + DeptId int64 `orm:"dept_id"` + Balance float64 `orm:"balance"` + Status int `orm:"status"` + TenantId string `orm:"tenant_id"` +} + +type balanceDetailRow struct { + DeptId int64 `orm:"dept_id"` + SerialNo string `orm:"serial_no"` + TransactionTime *gtime.Time `orm:"transaction_time"` + IncomeExpenseType int `orm:"income_expense_type"` + TransactionType int `orm:"transaction_type"` + TransactionChannel int `orm:"transaction_channel"` + TransactionStatus int `orm:"transaction_status"` + Amount float64 `orm:"amount"` + BeforeBalance float64 `orm:"before_balance"` + BalanceAfter float64 `orm:"balance_after"` + ChannelSerialNo string `orm:"channel_serial_no"` + OrderSn string `orm:"order_sn"` + Remark string `orm:"remark"` + TenantId string `orm:"tenant_id"` + CreatedDept int64 `orm:"created_dept"` + CreatedBy int64 `orm:"created_by"` + CreatedAt *gtime.Time `orm:"created_at"` +} + +const ( + addPackageBizType = 3 + addPackageTransactionType = 1 + addPackageTransactionChan = 3 + addPackageTransactionPaid = 1 + addPackageAccountStatusOK = 0 +) + +const ( + addPackageSameBaseFailureReason = "同一批加油包目标当前基础套餐必须一致" + addPackageSamePackageFailureReason = "同一批加油包套餐必须一致" + addPackageSameEffectiveFailureReason = "同一批加油包生效方式必须一致" + addPackageSamePriceFailureReason = "同一批加油包单价必须一致" +) + +func NewCardExecutor(repo *renewchangeservice.Repository) batch.Executor { + if repo == nil { + repo = renewchangeservice.NewRepository() + } + return &executor{repo: repo, batchType: consts.BatchTypeCardAddPackage, nowFunc: timeNow} +} + +func NewPoolExecutor(repo *renewchangeservice.Repository) batch.Executor { + if repo == nil { + repo = renewchangeservice.NewRepository() + } + return &executor{repo: repo, batchType: consts.BatchTypePoolAddPackage, poolMode: true, nowFunc: timeNow} +} + +func (e *executor) Execute(ctx context.Context, batchNo string) error { + batchNo = strings.TrimSpace(batchNo) + if batchNo == "" { + return gerror.New("add package batch number is empty") + } + main, err := e.repo.LoadProcessingBatch(ctx, batchNo, e.batchType) + if err != nil { + return err + } + details, err := e.repo.LoadRunningDetails(ctx, batchNo, e.batchType) + if err != nil { + return gerror.Wrap(err, "load add package details failed") + } + task := &batchTask{ + main: main, + total: len(details), + plans: make([]*detailPlan, 0, len(details)), + paymentIntent: readPaymentIntent(main.BatchInfo), + paymentSnapshot: make(map[int64]*PaymentSnapshot), + } + if len(details) == 0 { + return e.finalizeFailure(ctx, task, "加油包批次没有可处理明细") + } + if err := e.buildTask(ctx, task, details); err != nil { + return e.finalizeFailure(ctx, task, err.Error()) + } + if task.valid == 0 { + if task.reason == "" { + task.reason = "加油包批次没有可处理明细" + } + return e.finalizeFailure(ctx, task, task.reason) + } + orderSN, err := e.applyBusiness(ctx, task) + if err != nil { + return e.finalizeFailure(ctx, task, err.Error()) + } + if err = e.finalizeSuccess(ctx, task, orderSN); err != nil { + return err + } + return nil +} + +func (e *executor) buildTask(ctx context.Context, task *batchTask, details []*rc.BatchDetailRow) error { + seen := make(map[string]struct{}, len(details)) + now := e.nowTime() + for _, detail := range details { + if detail == nil { + continue + } + plan := &detailPlan{detail: detail} + snapshot := readDetailSnapshot(detail.Snapshot) + plan.targetCode = strings.TrimSpace(detail.ItemId) + if snapshot != nil { + if v := strings.TrimSpace(snapshot.TargetCode); v != "" { + plan.targetCode = v + } + plan.packageCode = strings.TrimSpace(snapshot.PackageCode) + plan.remark = strings.TrimSpace(snapshot.Remark) + } + if plan.packageCode == "" && task.main.BatchInfo != nil { + plan.packageCode = strings.TrimSpace(task.main.BatchInfo.Get("packageCode").String()) + } + mode, modeErr := effectiveModeFrom(task.main.BatchInfo, snapshot) + if modeErr != nil { + plan.reason = modeErr.Error() + task.plans = append(task.plans, plan) + continue + } + plan.effectiveMode = mode + if plan.targetCode == "" { + plan.reason = "目标编码不能为空" + task.plans = append(task.plans, plan) + continue + } + hasPackageCode := plan.packageCode != "" + if !hasPackageCode && !task.prepareOnly { + plan.reason = "加油包套餐编码不能为空" + task.plans = append(task.plans, plan) + continue + } + dupKey := strings.Join([]string{plan.targetCode, plan.packageCode}, "|") + if _, ok := seen[dupKey]; ok { + plan.reason = "批次内重复的目标编码 + 套餐编码" + task.plans = append(task.plans, plan) + continue + } + seen[dupKey] = struct{}{} + + if e.poolMode { + pool, err := e.loadPool(ctx, task.main.TenantId, plan.targetCode) + if err != nil { + plan.reason = err.Error() + task.plans = append(task.plans, plan) + continue + } + if pool == nil || pool.Id == 0 { + plan.reason = "流量池不存在" + task.plans = append(task.plans, plan) + continue + } + plan.pool = pool + plan.customerDeptID = pool.DeptId + plan.basePackageKey = fmt.Sprintf("%d", pool.PoolPackageCode) + if pool.PoolStatus != 0 { + plan.reason = "流量池状态异常，不能订购加油包" + task.plans = append(task.plans, plan) + continue + } + if err := e.ensureNoActiveTopup(ctx, nil, task.main.TenantId, plan.targetCode, now); err != nil { + plan.reason = err.Error() + task.plans = append(task.plans, plan) + continue + } + if !hasPackageCode { + task.valid++ + task.plans = append(task.plans, plan) + continue + } + pkg, price, err := e.resolvePrice(ctx, task.main.TenantId, pool.DeptId, plan.packageCode) + if err != nil { + plan.reason = err.Error() + task.plans = append(task.plans, plan) + continue + } + if pkg.PackageType != 104 { + plan.reason = "目标套餐类型不是流量池加油包" + task.plans = append(task.plans, plan) + continue + } + if pkg.PoolPackageCode != pool.PoolPackageCode { + plan.reason = "加油包不属于当前流量池套餐" + task.plans = append(task.plans, plan) + continue + } + plan.price = price + plan.unitAmount = price.UnitPrice + plan.amount = roundAmount(price.UnitPrice) + plan.effectiveTime, plan.expireTime, plan.effectiveNow, err = calcMonthWindow(plan.effectiveMode, now) + if err != nil { + plan.reason = err.Error() + task.plans = append(task.plans, plan) + continue + } + task.valid++ + task.plans = append(task.plans, plan) + continue + } + + cardMap, err := e.repo.Tables().LoadCardsByIccids(ctx, []string{plan.targetCode}) + if err != nil { + plan.reason = err.Error() + task.plans = append(task.plans, plan) + continue + } + card := cardMap[plan.targetCode] + if card == nil || card.Id == 0 { + plan.reason = "ICCID不存在" + task.plans = append(task.plans, plan) + continue + } + plan.card = card + plan.customerDeptID = card.DeptId + plan.basePackageKey = strings.TrimSpace(card.CurrentPackageCode) + if card.PackageStatus != consts.PackageStatusEffective { + plan.reason = "当前卡套餐未生效，不能订购加油包" + task.plans = append(task.plans, plan) + continue + } + if err := e.ensureNoActiveTopup(ctx, nil, task.main.TenantId, plan.targetCode, now); err != nil { + plan.reason = err.Error() + task.plans = append(task.plans, plan) + continue + } + basePkg, err := e.repo.Tables().LoadPackageByCode(ctx, task.main.TenantId, card.CurrentPackageCode) + if err != nil || basePkg == nil { + plan.reason = "当前卡基础套餐不存在" + task.plans = append(task.plans, plan) + continue + } + if !containsString(packageCodesFromJSON(basePkg.TopupPackageCodes), plan.packageCode) { + if hasPackageCode { + plan.reason = "当前套餐不允许订购该加油包" + task.plans = append(task.plans, plan) + continue + } + } + if !hasPackageCode { + task.valid++ + task.plans = append(task.plans, plan) + continue + } + pkg, price, err := e.resolvePrice(ctx, task.main.TenantId, card.DeptId, plan.packageCode) + if err != nil { + plan.reason = err.Error() + task.plans = append(task.plans, plan) + continue + } + if pkg.PackageType != 103 { + plan.reason = "目标套餐类型不是单卡加油包" + task.plans = append(task.plans, plan) + continue + } + plan.price = price + plan.unitAmount = price.UnitPrice + plan.amount = roundAmount(price.UnitPrice) + plan.effectiveTime, plan.expireTime, plan.effectiveNow, err = calcMonthWindow(plan.effectiveMode, now) + if err != nil { + plan.reason = err.Error() + task.plans = append(task.plans, plan) + continue + } + task.valid++ + task.plans = append(task.plans, plan) + } + enforceAddPackageBatchUniformity(task) + task.valid = 0 + task.success = 0 + task.fail = 0 + task.reason = "" + for _, plan := range task.plans { + if plan == nil { + continue + } + if plan.reason != "" { + task.fail++ + if task.reason == "" { + task.reason = plan.reason + } + continue + } + task.success++ + task.valid++ + } + if task.reason == "" && task.valid == 0 { + task.reason = "加油包批次没有可处理明细" + } + return nil +} + +func enforceAddPackageBatchUniformity(task *batchTask) { + if task == nil { + return + } + var first *detailPlan + for _, plan := range task.plans { + if plan == nil || plan.reason != "" { + continue + } + if first == nil { + first = plan + continue + } + switch { + case strings.TrimSpace(plan.basePackageKey) != strings.TrimSpace(first.basePackageKey): + plan.reason = addPackageSameBaseFailureReason + case strings.TrimSpace(plan.packageCode) != strings.TrimSpace(first.packageCode): + plan.reason = addPackageSamePackageFailureReason + case strings.TrimSpace(plan.effectiveMode) != strings.TrimSpace(first.effectiveMode): + plan.reason = addPackageSameEffectiveFailureReason + case !sameAddPackageAmount(plan.unitAmount, first.unitAmount): + plan.reason = addPackageSamePriceFailureReason + } + } +} + +func sameAddPackageAmount(left, right float64) bool { + return amountCents(left) == amountCents(right) +} + +func amountCents(value float64) int64 { + if value < 0 { + value = 0 + } + return int64(value*100 + 0.5) +} + +func (e *executor) applyBusiness(ctx context.Context, task *batchTask) (string, error) { + now := e.nowTime() + tenantID := task.main.TenantId + userID := task.main.CreatedBy + deptID := task.main.CreatedDept + isTopDept, err := e.isTopDept(ctx, tenantID, deptID) + if err != nil { + return "", err + } + intent, err := normalizePaymentIntent(task.paymentIntent, isTopDept) + if err != nil { + return "", err + } + task.paymentIntent = intent + if orderSN, recovered, err := e.recoverAppliedBusiness(ctx, task); err != nil { + return "", err + } else if recovered { + return orderSN, nil + } + planGroups := make(map[int64][]*detailPlan) + orderedDeptIDs := make([]int64, 0, len(task.plans)) + for _, plan := range task.plans { + if plan == nil || plan.reason != "" { + continue + } + payerDeptID := plan.customerDeptID + if payerDeptID <= 0 { + return "", gerror.New("加油包归属部门不能为空") + } + if _, ok := planGroups[payerDeptID]; !ok { + orderedDeptIDs = append(orderedDeptIDs, payerDeptID) + } + planGroups[payerDeptID] = append(planGroups[payerDeptID], plan) + } + if len(planGroups) == 0 { + return "", gerror.New("加油包订单金额不能为空") + } + sort.Slice(orderedDeptIDs, func(i, j int) bool { return orderedDeptIDs[i] < orderedDeptIDs[j] }) + orderSNs := make([]string, 0, len(orderedDeptIDs)) + if err := e.repo.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + for _, plan := range task.plans { + if plan == nil || plan.reason != "" { + continue + } + if err := e.ensureNoActiveTopup(ctx, tx, tenantID, plan.targetCode, now); err != nil { + return err + } + } + for _, payerDeptID := range orderedDeptIDs { + groupPlans := planGroups[payerDeptID] + if len(groupPlans) == 0 { + continue + } + groupAmount := 0.0 + for _, plan := range groupPlans { + groupAmount = roundAmount(groupAmount + plan.amount) + } + if groupAmount <= 0 { + return gerror.New("加油包订单金额不能为空") + } + groupOrderSN := rc.GenerateOrderSn(&now) + payment, err := e.executePayment( + ctx, + tx, + task.main, + intent, + payerDeptID, + groupAmount, + groupOrderSN, + now, + isTopDept, + ) + if err != nil { + return err + } + task.paymentSnapshot[payerDeptID] = payment + orderRow := buildAddPackageOrderDO(groupOrderSN, task.main, payerDeptID, groupAmount, payment, now) + orderID, err := e.repo.Tables().InsertContractOrder(ctx, tx, orderRow) + if err != nil { + return err + } + for _, plan := range groupPlans { + subscriptionNo := rc.GenerateSubscriptionNo("ADP") + plan.orderSN = groupOrderSN + if e.poolMode { + if err := e.insertPoolBind(ctx, tx, plan, task.main, groupOrderSN, subscriptionNo, now); err != nil { + return err + } + } else { + if err := e.insertCardBind(ctx, tx, plan, task.main, groupOrderSN, subscriptionNo, now); err != nil { + return err + } + } + item := buildAddPackageOrderItemDO(orderID, groupOrderSN, task.main, payerDeptID, deptID, plan, now) + if err := e.repo.Tables().InsertContractOrderItem(ctx, tx, item); err != nil { + return err + } + } + orderSNs = append(orderSNs, groupOrderSN) + } + if e.poolMode { + if err := e.updateFlowPool(ctx, tx, task.plans, tenantID, userID, now); err != nil { + return err + } + } else { + if err := e.updateCardTopupFlow(ctx, tx, task.plans, tenantID, userID, now); err != nil { + return err + } + } + return nil + }); err != nil { + return "", err + } + if len(orderSNs) == 0 { + return "", nil + } + return orderSNs[0], nil +} + +func (e *executor) finalizeSuccess(ctx context.Context, task *batchTask, orderSN string) error { + now := e.nowTime() + tenantID := task.main.TenantId + userID := task.main.CreatedBy + deptID := task.main.CreatedDept + successCount := task.valid + failCount := task.total - task.valid + runResult := consts.BatchRunResultSuccess + asyncStatus := consts.BatchAsyncStatusSuccess + failReason := "" + if failCount > 0 && successCount > 0 { + runResult = consts.BatchRunResultPartialSuccess + asyncStatus = consts.BatchAsyncStatusPartialSuccess + failReason = firstFailureReason(task.plans) + } + if failCount > 0 && successCount == 0 { + runResult = consts.BatchRunResultFailed + asyncStatus = consts.BatchAsyncStatusFailed + failReason = firstFailureReason(task.plans) + } + if err := e.repo.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + for _, plan := range task.plans { + if plan == nil || plan.detail == nil { + continue + } + snapshot := buildDetailSnapshot(plan, plan.orderSN) + status := consts.BatchDetailStatusSuccess + reason := "" + if plan.reason != "" { + status = consts.BatchDetailStatusFailed + reason = plan.reason + } + if err := e.repo.Tables().UpdateBatchDetailSnapshot(ctx, tx, task.main.BatchNo, task.main.BatchType, plan.detail.RowNo, status, snapshot, reason, userID); err != nil { + return err + } + } + info := mergeBatchInfo(task.main.BatchInfo, map[string]any{ + "orderSn": orderSN, + "orderSnList": collectOrderSNs(task.plans), + "priceSnapshot": buildBatchPriceSnapshot(task), + "paymentMode": task.paymentIntent.PaymentMode, + "offlineContractCompleted": task.paymentIntent.OfflineContractCompleted, + "paymentSnapshotByPayerDept": task.paymentSnapshot, + "batchMode": map[bool]string{true: "pool", false: "card"}[e.poolMode], + }) + return e.repo.UpdateBatchMainInfo(ctx, tx, task.main.BatchNo, task.main.BatchType, &rc.BatchMainRow{ + BatchType: task.main.BatchType, + BatchAsyncStatus: asyncStatus, + RunResult: runResult, + TotalCount: task.total, + ValidCount: task.valid, + SuccessCount: successCount, + FailCount: failCount, + FailReason: failReason, + BatchInfo: gjson.New(info), + OrderSn: orderSN, + EndTime: gtime.NewFromTime(now), + UpdatedBy: userID, + }) + }); err != nil { + return gerror.Wrap(err, "finalize add package success failed") + } + rows, _ := e.repo.ResolveResultSnapshot(ctx, task.main.BatchNo, task.main.BatchType) + _ = e.repo.WriteResultFileSafe(ctx, task.main.BatchNo, task.main.BatchType, tenantID, deptID, userID, rows) + g.Log().Infof(ctx, "add package batch success, batchNo:%s, orderSn:%s, batchType:%d", task.main.BatchNo, orderSN, task.main.BatchType) + return nil +} + +func (e *executor) finalizeFailure(ctx context.Context, task *batchTask, reason string) error { + now := e.nowTime() + tenantID := task.main.TenantId + userID := task.main.CreatedBy + deptID := task.main.CreatedDept + if err := e.repo.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + for _, plan := range task.plans { + if plan == nil || plan.detail == nil { + continue + } + snapshot := buildDetailSnapshot(plan, "") + rowReason := plan.reason + if rowReason == "" { + rowReason = reason + } + if err := e.repo.Tables().UpdateBatchDetailSnapshot(ctx, tx, task.main.BatchNo, task.main.BatchType, plan.detail.RowNo, consts.BatchDetailStatusFailed, snapshot, rowReason, userID); err != nil { + return err + } + } + return e.repo.FinalizeBatchMain(ctx, tx, task.main.BatchNo, task.main.BatchType, &rc.BatchMainRow{ + BatchAsyncStatus: consts.BatchAsyncStatusFailed, + RunResult: consts.BatchRunResultFailed, + FailReason: reason, + TotalCount: task.total, + ValidCount: task.valid, + SuccessCount: 0, + FailCount: task.total, + EndTime: gtime.NewFromTime(now), + UpdatedBy: userID, + }) + }); err != nil { + return gerror.Wrap(err, "finalize add package failure failed") + } + rows, _ := e.repo.ResolveResultSnapshot(ctx, task.main.BatchNo, task.main.BatchType) + _ = e.repo.WriteResultFileSafe(ctx, task.main.BatchNo, task.main.BatchType, tenantID, deptID, userID, rows) + g.Log().Warningf(ctx, "add package batch failed, batchNo:%s, batchType:%d, reason:%s", task.main.BatchNo, task.main.BatchType, reason) + return nil +} + +func buildAddPackageOrderDO(orderSN string, main *rc.BatchMainRow, payerDeptID int64, totalAmount float64, payment *PaymentSnapshot, now time.Time) *rc.OrderDO { + payTime := gtime.NewFromTime(now) + createdDept := main.CreatedDept + if createdDept <= 0 { + createdDept = payerDeptID + } + payType := consts.PayTypeBalance + balanceAmount := totalAmount + offlinePaidAmount := 0.0 + payOrderSN := orderSN + if payment != nil { + payType = payment.PayType + balanceAmount = payment.BalanceAmount + offlinePaidAmount = payment.OfflinePaidAmount + if payment.BalanceDetailSerialNo != "" { + payOrderSN = payment.BalanceDetailSerialNo + } + if !payment.PaymentTime.IsZero() { + payTime = gtime.NewFromTime(payment.PaymentTime) + } + } + return &rc.OrderDO{ + OrderSn: orderSN, + ParentOrderId: int64(0), + ParentOrderSn: "", + CustomerType: consts.CustomerTypeAgent, + CustomerAgentId: payerDeptID, + EndUserId: int64(0), + AgentId: payerDeptID, + OrderType: consts.OrderTypeCardSubscribe, + OrderStatus: consts.OrderStatusCompleted, + ShippingStatus: 0, + ReturnStatus: consts.OrderReturnStatusNone, + Consignee: "", + Mobile: "", + Email: "", + Address: "", + AreaCode: "", + BuyerNote: "加油包自动生成订单", + AdminNote: "", + Remark: "加油包自动生成订单", + ShippingMethod: consts.ShippingMethodNone, + LogisticsId: int64(0), + LogisticsName: "", + ShippingTypeId: int64(0), + ShippingTypeName: "", + TrackingNo: "", + PayType: payType, + PayStatus: consts.PayStatusPaid, + PayOrderSn: payOrderSN, + PayTime: payTime, + TotalAmount: totalAmount, + ProductAmount: totalAmount, + ShippingFee: 0, + ServiceFee: 0, + InvoiceFee: 0, + CouponAmount: 0, + DiscountAmount: 0, + Balance: balanceAmount, + PaidAmount: totalAmount, + OnlinePaidAmount: 0, + OfflinePaidAmount: offlinePaidAmount, + UnpaidAmount: 0, + UnrefundAmount: 0, + IsNeedCommission: 0, + DistributionStatus: 0, + TenantId: main.TenantId, + CreatedDept: createdDept, + CreatedBy: main.CreatedBy, + CreatedAt: payTime, + UpdatedBy: main.CreatedBy, + UpdatedAt: payTime, + } +} + +func buildAddPackageOrderItemDO(orderID int64, orderSN string, main *rc.BatchMainRow, payerDeptID, createdDeptID int64, plan *detailPlan, now time.Time) *rc.OrderItemDO { + if plan == nil || plan.price == nil { + return nil + } + createdDept := createdDeptID + if createdDept <= 0 { + createdDept = payerDeptID + } + skuData, _ := json.Marshal([]map[string]any{ + {"skuName": "targetCode", "skuValue": plan.targetCode}, + {"skuName": "package", "skuValue": plan.price.PackageName, "packageCode": plan.price.PackageCode, "packageType": plan.price.PackageType}, + {"skuName": "effectiveMode", "skuValue": effectiveModeText(plan.effectiveMode)}, + }) + extraSKUData, _ := json.Marshal(map[string]any{ + "businessBatchNo": main.BatchNo, + "targetCode": plan.targetCode, + "packageCode": plan.price.PackageCode, + "effectiveMode": plan.effectiveMode, + "effectiveModeText": effectiveModeText(plan.effectiveMode), + "remark": plan.remark, + }) + return &rc.OrderItemDO{ + OrderId: orderID, + OrderSn: orderSN, + CustomerType: consts.CustomerTypeAgent, + CustomerAgentId: payerDeptID, + EndUserId: int64(0), + Price: plan.amount, + Quantity: 1, + ProductId: plan.price.PackageID, + ProductName: plan.price.PackageName, + ProductSn: plan.price.PackageCode, + PicThumb: "", + SkuId: int64(0), + SkuData: string(skuData), + DeliveryQuantity: 0, + ProductType: consts.ProductTypeCard, + IsGift: 0, + AgentId: payerDeptID, + PrepayPrice: 0, + Commission: "", + OriginPrice: plan.amount, + PromotionData: "", + ExtraSkuData: string(extraSKUData), + PayMode: consts.PayModeYearMonth, + ServiceBeginTime: gtime.NewFromTime(plan.effectiveTime), + ServiceEndTime: gtime.NewFromTime(plan.expireTime), + TenantId: main.TenantId, + CreatedDept: createdDept, + CreatedBy: main.CreatedBy, + CreatedAt: gtime.NewFromTime(now), + UpdatedBy: main.CreatedBy, + UpdatedAt: gtime.NewFromTime(now), + } +} + +func collectOrderSNs(plans []*detailPlan) []string { + if len(plans) == 0 { + return nil + } + seen := make(map[string]struct{}, len(plans)) + out := make([]string, 0, len(plans)) + for _, plan := range plans { + if plan == nil || strings.TrimSpace(plan.orderSN) == "" { + continue + } + orderSN := strings.TrimSpace(plan.orderSN) + if _, ok := seen[orderSN]; ok { + continue + } + seen[orderSN] = struct{}{} + out = append(out, orderSN) + } + sort.Strings(out) + return out +} + +func (e *executor) insertCardBind(ctx context.Context, tx gdb.TX, plan *detailPlan, main *rc.BatchMainRow, orderSN, subscriptionNo string, now time.Time) error { + nowGT := gtime.NewFromTime(now) + bind := &rc.SimCardBindPkg{ + BatchNo: main.BatchNo, + Iccid: plan.targetCode, + Msisdn: "", + DeptId: plan.customerDeptID, + PackageType: plan.price.PackageType, + PackageCode: plan.packageCode, + PreviousPackageCode: "", + BusinessType: addPackageBizType, + EffectiveStatus: map[bool]int{true: consts.EffectiveStatusEffective, false: consts.EffectiveStatusPending}[plan.effectiveNow], + HandleTime: nowGT, + EffectiveTime: gtime.NewFromTime(plan.effectiveTime), + ExpectedExpireTime: gtime.NewFromTime(plan.expireTime), + ActualExpireTime: nil, + DurationMonths: 1, + GiftDurationMonths: 0, + OrderCode: orderSN, + SubscriptionNo: subscriptionNo, + Remark: plan.remark, + TenantId: main.TenantId, + CreatedDept: main.CreatedDept, + CreatedBy: main.CreatedBy, + CreatedAt: nowGT, + } + if err := e.repo.Tables().InsertBind(ctx, tx, bind); err != nil { + return err + } + return nil +} + +func (e *executor) insertPoolBind(ctx context.Context, tx gdb.TX, plan *detailPlan, main *rc.BatchMainRow, orderSN, subscriptionNo string, now time.Time) error { + nowGT := gtime.NewFromTime(now) + _, err := tx.Ctx(ctx).Model("t_sim_pool_bind_package").Data(map[string]any{ + "subscription_no": subscriptionNo, + "batch_no": main.BatchNo, + "pool_no": plan.targetCode, + "dept_id": plan.customerDeptID, + "package_type": plan.price.PackageType, + "package_code": plan.packageCode, + "previous_package_code": "", + "business_type": addPackageBizType, + "effective_status": map[bool]int{true: consts.EffectiveStatusEffective, false: consts.EffectiveStatusPending}[plan.effectiveNow], + "handle_time": nowGT, + "effective_time": gtime.NewFromTime(plan.effectiveTime), + "expected_expire_time": gtime.NewFromTime(plan.expireTime), + "actual_expire_time": nil, + "duration_months": 1, + "gift_duration_months": 0, + "order_code": orderSN, + "remark": plan.remark, + "tenant_id": main.TenantId, + "created_dept": main.CreatedDept, + "created_by": main.CreatedBy, + "created_at": nowGT, + }).Insert() + if err != nil { + return gerror.Wrap(err, "insert flow pool add package failed") + } + return nil +} + +func (e *executor) updateFlowPool(ctx context.Context, tx gdb.TX, plans []*detailPlan, tenantID string, userID int64, now time.Time) error { + type poolDelta struct { + pool *flowPoolRow + flow float64 + } + pools := make(map[int64]*poolDelta) + for _, plan := range plans { + if plan == nil || !plan.effectiveNow || plan.price == nil || plan.pool == nil { + continue + } + delta, ok := pools[plan.pool.Id] + if !ok { + delta = &poolDelta{pool: plan.pool} + pools[plan.pool.Id] = delta + } + delta.flow = roundAmount(delta.flow + plan.price.FlowMb) + } + if len(pools) == 0 { + return nil + } + for _, delta := range pools { + if delta == nil || delta.pool == nil || delta.flow <= 0 { + continue + } + _, err := tx.Ctx(ctx).Model("t_sim_flow_pool"). + Where("id", delta.pool.Id). + Data(map[string]any{ + "add_on_flow_mb": roundAmount(delta.pool.AddOnFlowMb + delta.flow), + "total_flow_mb": roundAmount(delta.pool.TotalFlowMb + delta.flow), + "remain_flow_mb": roundAmount(delta.pool.RemainFlowMb + delta.flow), + "updated_by": userID, + "updated_at": gtime.NewFromTime(now), + }). + Update() + if err != nil { + return gerror.Wrap(err, "update flow pool add-on flow failed") + } + } + return nil +} + +func (e *executor) updateCardTopupFlow(ctx context.Context, tx gdb.TX, plans []*detailPlan, tenantID string, userID int64, now time.Time) error { + type cardDelta struct { + card *rc.SimCardInfo + flow float64 + } + cards := make(map[int64]*cardDelta) + for _, plan := range plans { + if plan == nil || !plan.effectiveNow || plan.price == nil || plan.card == nil { + continue + } + delta, ok := cards[plan.card.Id] + if !ok { + delta = &cardDelta{card: plan.card} + cards[plan.card.Id] = delta + } + delta.flow = roundAmount(delta.flow + plan.price.FlowMb) + } + if len(cards) == 0 { + return nil + } + for _, delta := range cards { + if delta == nil || delta.card == nil || delta.card.Id <= 0 || delta.flow <= 0 { + continue + } + update := map[string]any{ + "package_total_flow_mb": roundAmount(delta.card.PackageTotalFlowMb + delta.flow), + "updated_by": userID, + "updated_at": gtime.NewFromTime(now), + } + mod := tx.Ctx(ctx).Model("t_sim_card_info").Where("id", delta.card.Id) + if strings.TrimSpace(tenantID) != "" { + mod = mod.Where("tenant_id", tenantID) + } + if _, err := mod.Data(update).Update(); err != nil { + return gerror.Wrap(err, "update card add-on flow failed") + } + } + return nil +} + +func normalizePaymentIntent(intent PaymentIntent, isTopDept bool) (PaymentIntent, error) { + intent.PaymentMode = strings.TrimSpace(intent.PaymentMode) + if intent.PaymentMode == "" { + intent.PaymentMode = "balance" + } + switch intent.PaymentMode { + case "balance": + intent.OfflineContractCompleted = false + return intent, nil + case "offline": + if !isTopDept { + return PaymentIntent{}, gerror.New("当前组织加油包必须使用余额支付") + } + if !intent.OfflineContractCompleted { + return PaymentIntent{}, gerror.New("请确认线下合同、销售和走账已完成") + } + return intent, nil + default: + return PaymentIntent{}, gerror.New("加油包付款方式不正确") + } +} + +func (e *executor) executePayment( + ctx context.Context, + tx gdb.TX, + main *rc.BatchMainRow, + intent PaymentIntent, + payerDeptID int64, + amount float64, + orderSN string, + now time.Time, + isTopDept bool, +) (*PaymentSnapshot, error) { + if main == nil { + return nil, gerror.New("加油包批次不能为空") + } + snapshot := &PaymentSnapshot{ + PaymentMode: intent.PaymentMode, + PayStatus: consts.PayStatusPaid, + PayAmount: amount, + PaymentTime: now, + PayerDeptID: payerDeptID, + PayerUserID: main.CreatedBy, + IsTopDept: isTopDept, + } + if intent.PaymentMode == "offline" { + snapshot.PayType = consts.PayTypeOffline + snapshot.OfflinePaidAmount = amount + return snapshot, nil + } + serialNo := addPackageBalanceSerialNo(main.BatchNo, payerDeptID) + before, after, err := e.debitBalance( + ctx, + tx, + main.TenantId, + payerDeptID, + amount, + main.CreatedBy, + orderSN, + serialNo, + now, + main.CreatedDept, + ) + if err != nil { + return nil, err + } + snapshot.PayType = consts.PayTypeBalance + snapshot.BalanceAmount = amount + snapshot.BalanceBefore = before + snapshot.BalanceAfter = after + snapshot.BalanceDetailSerialNo = serialNo + return snapshot, nil +} + +func addPackageBalanceSerialNo(batchNo string, payerDeptID int64) string { + batchNo = strings.TrimSpace(batchNo) + if batchNo == "" || payerDeptID <= 0 { + return "" + } + return fmt.Sprintf("ADPPAY%sD%d", batchNo, payerDeptID) +} + +func (e *executor) isTopDept(ctx context.Context, tenantID string, deptID int64) (bool, error) { + if strings.TrimSpace(tenantID) == "" || deptID <= 0 { + return false, gerror.New("当前用户未绑定组织或租户") + } + var dept sysDeptRow + if err := e.repo.DB().Model("sys_dept").Ctx(ctx). + Where("tenant_id", tenantID). + Where("dept_id", deptID). + WhereNull("deleted_at"). + Scan(&dept); err != nil { + return false, gerror.Wrap(err, "query current department failed") + } + if dept.DeptId == 0 { + return false, gerror.New("当前组织不存在") + } + return dept.ParentId == 0, nil +} + +func (e *executor) recoverAppliedBusiness(ctx context.Context, task *batchTask) (string, bool, error) { + if task == nil || task.main == nil { + return "", false, nil + } + type existingOrderItem struct { + OrderSN string `orm:"order_sn"` + ExtraSKUData string `orm:"extra_sku_data"` + } + var existingItems []existingOrderItem + if err := e.repo.DB().Model("t_order_item oi").Ctx(ctx). + Fields("oi.order_sn, oi.extra_sku_data"). + LeftJoin("t_order o", "o.id = oi.order_id"). + Where("o.tenant_id", task.main.TenantId). + Where("oi.extra_sku_data LIKE ?", "%\"businessBatchNo\":\""+task.main.BatchNo+"\"%"). + OrderAsc("oi.item_id"). + Scan(&existingItems); err != nil { + return "", false, gerror.Wrap(err, "query existing add package orders failed") + } + if len(existingItems) == 0 { + return "", false, nil + } + + orderByPlan := make(map[string]string, len(existingItems)) + for _, item := range existingItems { + orderSN := strings.TrimSpace(item.OrderSN) + if orderSN == "" || strings.TrimSpace(item.ExtraSKUData) == "" { + continue + } + var extra struct { + BusinessBatchNo string `json:"businessBatchNo"` + TargetCode string `json:"targetCode"` + PackageCode string `json:"packageCode"` + } + if err := json.Unmarshal([]byte(item.ExtraSKUData), &extra); err != nil { + return "", false, gerror.Wrap(err, "decode existing add package order item failed") + } + if extra.BusinessBatchNo != task.main.BatchNo { + continue + } + key := addPackageRecoveryKey(extra.TargetCode, extra.PackageCode) + if key == "" { + return "", false, gerror.New("existing add package order item is missing target or package") + } + if previous, ok := orderByPlan[key]; ok && previous != orderSN { + return "", false, gerror.New("existing add package orders contain duplicate target records") + } + orderByPlan[key] = orderSN + } + + matched := 0 + firstOrderSN := "" + for _, plan := range task.plans { + if plan == nil || plan.reason != "" { + continue + } + key := addPackageRecoveryKey(plan.targetCode, plan.packageCode) + orderSN, ok := orderByPlan[key] + if !ok { + return "", false, gerror.New("existing add package orders are incomplete; refusing to retry business writes") + } + plan.orderSN = orderSN + if firstOrderSN == "" { + firstOrderSN = orderSN + } + matched++ + } + if matched == 0 { + return "", false, nil + } + return firstOrderSN, true, nil +} + +func addPackageRecoveryKey(targetCode, packageCode string) string { + targetCode = strings.TrimSpace(targetCode) + packageCode = strings.TrimSpace(packageCode) + if targetCode == "" || packageCode == "" { + return "" + } + return targetCode + "\\x00" + packageCode +} + +func (e *executor) ensureNoActiveTopup(ctx context.Context, tx gdb.TX, tenantID, targetCode string, now time.Time) error { + targetCode = strings.TrimSpace(targetCode) + tenantID = strings.TrimSpace(tenantID) + if targetCode == "" || tenantID == "" { + return nil + } + if e.poolMode { + return ensureNoActiveTopupRecord(ctx, tx, rc.TableSimPoolBindPackage, "pool_no", consts.PackageTypePoolAddPackage, tenantID, targetCode, now, "当前流量池存在未过期加油包，不能重复订购") + } + return ensureNoActiveTopupRecord(ctx, tx, rc.TableSimCardBindPkg, "iccid", consts.PackageTypeCardAddPackage, tenantID, targetCode, now, "当前卡存在未过期加油包，不能重复订购") +} + +func ensureNoActiveTopupRecord(ctx context.Context, tx gdb.TX, table, targetColumn string, packageType int, tenantID, targetCode string, now time.Time, message string) error { + model := g.DB().Model(table).Ctx(ctx) + if tx != nil { + model = tx.Ctx(ctx).Model(table) + } + model = model. + Where("tenant_id", tenantID). + Where(targetColumn, targetCode). + Where("package_type", packageType). + WhereIn("effective_status", []int{consts.EffectiveStatusPending, consts.EffectiveStatusEffective}). + Where("(expected_expire_time IS NULL OR expected_expire_time > ?)", gtime.NewFromTime(now)) + if tx != nil { + model = model.LockUpdate() + } + var row struct { + Id int64 `orm:"id"` + } + err := model.Fields("id").Limit(1).Scan(&row) + if err != nil && !errors.Is(err, sql.ErrNoRows) { + return gerror.Wrap(err, "查询未过期加油包失败") + } + if row.Id > 0 { + return gerror.New(message) + } + return nil +} + +func (e *executor) debitBalance(ctx context.Context, tx gdb.TX, tenantID string, deptID int64, amount float64, userID int64, orderSN, serialNo string, now time.Time, createdDept int64) (float64, float64, error) { + serialNo = strings.TrimSpace(serialNo) + if serialNo == "" { + return 0, 0, gerror.New("加油包余额流水号不能为空") + } + var existing balanceDetailRow + if err := tx.Ctx(ctx).Model("t_balance_detail"). + Where("tenant_id", tenantID). + Where("serial_no", serialNo). + LockUpdate(). + Scan(&existing); err != nil && !errors.Is(err, sql.ErrNoRows) { + return 0, 0, gerror.Wrap(err, "query add package balance detail failed") + } + if strings.TrimSpace(existing.SerialNo) != "" { + return 0, 0, gerror.New("当前加油包批次已扣款，请勿重复提交") + } + var account agentBalanceRow + if err := tx.Ctx(ctx).Model("t_agent_balance"). + Where("dept_id", deptID). + Where("tenant_id", tenantID). + LockUpdate(). + Scan(&account); err != nil { + return 0, 0, gerror.Wrap(err, "lock balance account failed") + } + if account.Id == 0 { + return 0, 0, gerror.New("账户余额不存在") + } + if account.Status != addPackageAccountStatusOK { + return 0, 0, gerror.New("账户余额已冻结，不能扣费") + } + before := roundAmount(account.Balance) + if before+0.0001 < amount { + return 0, 0, gerror.Newf("账户余额不足，需支付%.2f元，当前余额%.2f元", amount, before) + } + after := roundAmount(before - amount) + nowGT := gtime.NewFromTime(now) + if _, err := tx.Ctx(ctx).Model("t_agent_balance"). + Where("id", account.Id). + Data(map[string]any{ + "balance": after, + "updated_by": userID, + "updated_at": nowGT, + }). + Update(); err != nil { + return 0, 0, gerror.Wrap(err, "update agent balance failed") + } + if _, err := tx.Ctx(ctx).Model("t_balance_detail").Data(balanceDetailRow{ + DeptId: deptID, + SerialNo: serialNo, + TransactionTime: nowGT, + IncomeExpenseType: 1, + TransactionType: addPackageTransactionType, + TransactionChannel: addPackageTransactionChan, + TransactionStatus: addPackageTransactionPaid, + Amount: amount, + BeforeBalance: before, + BalanceAfter: after, + ChannelSerialNo: serialNo, + OrderSn: orderSN, + Remark: fmt.Sprintf("加油包余额支付，批次号：%s", serialNo), + TenantId: tenantID, + CreatedDept: createdDept, + CreatedBy: userID, + CreatedAt: nowGT, + }).Insert(); err != nil { + return 0, 0, gerror.Wrap(err, "insert balance detail failed") + } + return before, after, nil +} + +func (e *executor) resolvePrice(ctx context.Context, tenantID string, customerDeptID int64, packageCode string) (*rc.FlowPackageInfo, *PriceSnapshot, error) { + pkg, err := e.repo.Tables().LoadPackageByCode(ctx, tenantID, packageCode) + if err != nil || pkg == nil { + return nil, nil, gerror.New("加油包套餐不存在") + } + if pkg.Enabled != 1 { + return nil, nil, gerror.New("加油包套餐未启用") + } + parentDeptID, err := e.loadParentDeptID(ctx, tenantID, customerDeptID) + if err != nil { + return nil, nil, err + } + snapshot := &PriceSnapshot{ + PackageID: pkg.Id, + PackageCode: packageCode, + PackageName: pkg.PackageName, + PackageType: pkg.PackageType, + PriceSource: "system", + FlowMb: pkg.FlowMb, + } + salePrice, err := renewchangeservice.ResolveEffectiveSalePrice(ctx, e.repo.Tables(), tenantID, parentDeptID, customerDeptID, pkg) + if err != nil { + return nil, nil, gerror.Wrap(err, "calculate add package effective price failed") + } + snapshot.PriceSource = salePrice.Rule.PriceSource + snapshot.AgentDefaultPriceID = salePrice.Rule.AgentDefaultPriceId + snapshot.TargetPriceID = salePrice.Rule.TargetPriceId + snapshot.UnitPrice = salePrice.AgentShipmentPrice + return pkg, snapshot, nil +} + +func (e *executor) loadParentDeptID(ctx context.Context, tenantID string, deptID int64) (int64, error) { + if deptID <= 0 { + return 0, nil + } + var dept sysDeptRow + if err := e.repo.DB().Model("sys_dept").Ctx(ctx). + Where("tenant_id", tenantID). + Where("dept_id", deptID). + WhereNull("deleted_at"). + Scan(&dept); err != nil { + return 0, gerror.Wrap(err, "query parent dept failed") + } + if dept.DeptId == 0 { + return 0, nil + } + return dept.ParentId, nil +} + +func (e *executor) loadPool(ctx context.Context, tenantID, flowPoolCode string) (*flowPoolRow, error) { + var row flowPoolRow + if err := e.repo.DB().Model("t_sim_flow_pool").Ctx(ctx). + Where("tenant_id", tenantID). + Where("flow_pool_code", flowPoolCode). + Scan(&row); err != nil { + return nil, gerror.Wrap(err, "query flow pool failed") + } + if row.Id == 0 { + return nil, nil + } + return &row, nil +} + +func (e *executor) nowTime() time.Time { + if e.nowFunc == nil { + return time.Now() + } + return e.nowFunc() +} + +func effectiveModeFrom(info *gjson.Json, snapshot *DetailSnapshot) (string, error) { + if snapshot != nil && strings.TrimSpace(snapshot.EffectiveMode) != "" { + return normalizeEffectiveMode(snapshot.EffectiveMode) + } + if info != nil && !info.IsNil() { + if mode := strings.TrimSpace(info.Get("effectiveMode").String()); mode != "" { + return normalizeEffectiveMode(mode) + } + } + return effectiveModeCurrentMonth, nil +} + +func normalizeEffectiveMode(mode string) (string, error) { + switch strings.TrimSpace(mode) { + case "", "immediate", "current_month", "this_month", "本月生效", "本月", "当月", "当月生效": + return effectiveModeCurrentMonth, nil + case "next_month", "nextMonth", "次月生效", "次月", "下月", "下月生效": + return "", gerror.New("加油包只支持本月生效") + default: + return "", gerror.New("加油包生效方式不正确") + } +} + +func calcMonthWindow(mode string, now time.Time) (time.Time, time.Time, bool, error) { + if _, err := normalizeEffectiveMode(mode); err != nil { + return time.Time{}, time.Time{}, false, err + } + start := time.Date(now.Year(), now.Month(), now.Day(), now.Hour(), now.Minute(), now.Second(), 0, now.Location()) + next := time.Date(start.Year(), start.Month()+1, 1, 0, 0, 0, 0, start.Location()) + return start, next.Add(-time.Second), !start.After(now), nil +} + +func effectiveModeText(mode string) string { + _, _ = normalizeEffectiveMode(mode) + return effectiveModeCurrentText +} + +func buildDetailSnapshot(plan *detailPlan, orderSN string) *gjson.Json { + if plan == nil { + return nil + } + data := map[string]any{ + "targetCode": plan.targetCode, + "packageCode": plan.packageCode, + "basePackageKey": plan.basePackageKey, + "effectiveMode": plan.effectiveMode, + "effectiveModeText": effectiveModeText(plan.effectiveMode), + "remark": plan.remark, + "orderSn": orderSN, + } + if plan.price != nil { + data["priceSnapshot"] = plan.price + data["packageName"] = plan.price.PackageName + data["packageType"] = plan.price.PackageType + data["unitPrice"] = plan.price.UnitPrice + data["flowMb"] = plan.price.FlowMb + } + if !plan.effectiveTime.IsZero() { + data["effectiveTime"] = gtime.NewFromTime(plan.effectiveTime).Format("Y-m-d H:i:s") + } + if !plan.expireTime.IsZero() { + data["expectedExpireTime"] = gtime.NewFromTime(plan.expireTime).Format("Y-m-d H:i:s") + } + if plan.effectiveNow { + data["effectiveStatus"] = consts.EffectiveStatusEffective + } else { + data["effectiveStatus"] = consts.EffectiveStatusPending + } + return gjson.New(data) +} + +func buildBatchPriceSnapshot(task *batchTask) map[string]any { + if task == nil { + return nil + } + rows := make([]map[string]any, 0, len(task.plans)) + for _, plan := range task.plans { + if plan == nil || plan.price == nil { + continue + } + rows = append(rows, map[string]any{ + "targetCode": plan.targetCode, + "basePackageKey": plan.basePackageKey, + "packageCode": plan.packageCode, + "effectiveMode": plan.effectiveMode, + "effectiveModeText": effectiveModeText(plan.effectiveMode), + "priceSource": plan.price.PriceSource, + "unitPrice": plan.price.UnitPrice, + }) + } + return map[string]any{"rows": rows} +} + +func mergeBatchInfo(prev *gjson.Json, addition map[string]any) map[string]any { + out := map[string]any{} + if prev != nil { + for k, v := range prev.Map() { + out[k] = v + } + } + for k, v := range addition { + out[k] = v + } + return out +} + +func firstFailureReason(plans []*detailPlan) string { + for _, plan := range plans { + if plan != nil && plan.reason != "" { + return plan.reason + } + } + return "" +} + +func packageCodesFromJSON(jsonValue *gjson.Json) []string { + if jsonValue == nil || jsonValue.IsNil() { + return nil + } + values := jsonValue.Interfaces() + out := make([]string, 0, len(values)) + seen := make(map[string]struct{}, len(values)) + for _, raw := range values { + code := strings.TrimSpace(fmt.Sprint(raw)) + if code == "" { + continue + } + if _, ok := seen[code]; ok { + continue + } + seen[code] = struct{}{} + out = append(out, code) + } + return out +} + +func containsString(values []string, target string) bool { + target = strings.TrimSpace(target) + for _, value := range values { + if strings.TrimSpace(value) == target { + return true + } + } + return false +} + +func normalizePriceRatio(ratio float64) float64 { + if ratio > 10 { + return ratio / 100 + } + return ratio +} + +func roundAmount(value float64) float64 { + if value < 0 { + return 0 + } + return float64(int(value*100+0.5)) / 100 +} diff --git a/internal/service/addpackage/prepare.go b/internal/service/addpackage/prepare.go new file mode 100644 index 0000000..b406149 --- /dev/null +++ b/internal/service/addpackage/prepare.go @@ -0,0 +1,831 @@ +package addpackage + +import ( + "context" + "database/sql" + "errors" + "sort" + "strings" + "time" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" + renewchangeservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/renewchange" +) + +const addPackagePrepareProcessingRetryAfter = 2 * time.Minute + +type PrepareExecutor struct { + repo *renewchangeservice.Repository + inner *executor + targetBatchType int + nowFunc NowFunc +} + +func NewCardPrepareExecutor(repo *renewchangeservice.Repository) *PrepareExecutor { + if repo == nil { + repo = renewchangeservice.NewRepository() + } + return &PrepareExecutor{ + repo: repo, + inner: &executor{repo: repo, batchType: consts.BatchTypePrepare, nowFunc: timeNow}, + targetBatchType: consts.BatchTypeCardAddPackage, + nowFunc: timeNow, + } +} + +func NewPoolPrepareExecutor(repo *renewchangeservice.Repository) *PrepareExecutor { + if repo == nil { + repo = renewchangeservice.NewRepository() + } + return &PrepareExecutor{ + repo: repo, + inner: &executor{repo: repo, batchType: consts.BatchTypePrepare, poolMode: true, nowFunc: timeNow}, + targetBatchType: consts.BatchTypePoolAddPackage, + nowFunc: timeNow, + } +} + +func (p *PrepareExecutor) CanExecuteProcessing(_ context.Context, main *entity.BatchMain) (bool, error) { + if main == nil || main.BatchType != consts.BatchTypePrepare || main.BatchAsyncStatus != consts.BatchAsyncStatusProcessing { + return false, nil + } + if targetBatchTypeFromInfo(main.BatchInfo) != p.targetBatchType { + return false, nil + } + if main.StartTime == nil || main.StartTime.IsZero() { + return true, nil + } + return p.now().Sub(main.StartTime.Time) >= addPackagePrepareProcessingRetryAfter, nil +} + +func (p *PrepareExecutor) Execute(ctx context.Context, batchNo string) error { + batchNo = strings.TrimSpace(batchNo) + if batchNo == "" { + return gerror.New("add package prepare batch number is empty") + } + main, err := p.repo.LoadProcessingBatch(ctx, batchNo, consts.BatchTypePrepare) + if err != nil { + return err + } + if targetBatchTypeFromInfo(main.BatchInfo) != p.targetBatchType { + return p.finalizePrepareFailure(ctx, main, "加油包预处理批次类型不匹配", nil) + } + details, err := p.repo.LoadRunningDetails(ctx, batchNo, consts.BatchTypePrepare) + if err != nil { + return gerror.Wrap(err, "load add package prepare details failed") + } + task := &batchTask{ + main: main, + total: len(details), + plans: make([]*detailPlan, 0, len(details)), + paymentIntent: readPaymentIntent(main.BatchInfo), + paymentSnapshot: make(map[int64]*PaymentSnapshot), + prepareOnly: true, + } + if len(details) == 0 { + return p.finalizePrepareFailure(ctx, main, "加油包预处理批次没有可处理明细", task) + } + p.inner.nowFunc = p.now + if err := p.inner.buildTask(ctx, task, details); err != nil { + return p.finalizePrepareFailure(ctx, main, err.Error(), task) + } + if task.fail > 0 || task.valid != len(details) { + reason := strings.TrimSpace(task.reason) + if reason == "" { + reason = "加油包预处理存在不满足条件的明细" + } + return p.finalizePrepareFailure(ctx, main, reason, task) + } + return p.finalizePrepareSuccess(ctx, task) +} + +func (p *PrepareExecutor) finalizePrepareFailure(ctx context.Context, main *rc.BatchMainRow, reason string, task *batchTask) error { + if main == nil { + return gerror.New("add package prepare batch is empty") + } + if isAutoConfirmPrepare(main) { + return p.finalizeAutoPrepareFailure(ctx, main, reason, task) + } + reason = strings.TrimSpace(reason) + if reason == "" { + reason = "加油包预处理失败" + } + now := p.now() + failCount := main.TotalCount + err := p.repo.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + if task != nil { + for _, plan := range task.plans { + if plan == nil || plan.detail == nil { + continue + } + rowReason := strings.TrimSpace(plan.reason) + if rowReason == "" { + rowReason = reason + } + if err := p.repo.Tables().UpdateBatchDetailSnapshot(ctx, tx, main.BatchNo, main.BatchType, plan.detail.RowNo, consts.BatchDetailStatusFailed, buildDetailSnapshot(plan, ""), rowReason, main.CreatedBy); err != nil { + return err + } + } + } + if err := p.repo.MarkAllDetailsFailed(ctx, tx, main.BatchNo, main.BatchType, reason, main.CreatedBy); err != nil { + return err + } + return p.repo.FinalizeBatchMain(ctx, tx, main.BatchNo, main.BatchType, &rc.BatchMainRow{ + BatchAsyncStatus: consts.BatchAsyncStatusFailed, + RunResult: consts.BatchRunResultFailed, + FailReason: reason, + TotalCount: main.TotalCount, + ValidCount: 0, + SuccessCount: 0, + FailCount: failCount, + EndTime: gtime.NewFromTime(now), + UpdatedBy: main.CreatedBy, + }) + }) + return gerror.Wrap(err, "finalize add package prepare failure failed") +} + +func (p *PrepareExecutor) finalizePrepareSuccess(ctx context.Context, task *batchTask) error { + if task == nil || task.main == nil { + return gerror.New("add package prepare task is empty") + } + now := p.now() + info, err := p.buildPrepareInfo(ctx, task) + if err != nil { + if isAutoConfirmPrepare(task.main) { + return p.finalizeAutoPrepareFailure(ctx, task.main, err.Error(), task) + } + return err + } + if isAutoConfirmPrepare(task.main) { + return p.finalizeAutoPrepareSuccess(ctx, task, info, now) + } + err = p.repo.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + for _, plan := range task.plans { + if plan == nil || plan.detail == nil { + continue + } + if err := p.repo.Tables().UpdateBatchDetailSnapshot(ctx, tx, task.main.BatchNo, task.main.BatchType, plan.detail.RowNo, consts.BatchDetailStatusSuccess, buildDetailSnapshot(plan, ""), "", task.main.CreatedBy); err != nil { + return err + } + } + if err := p.repo.MarkAllDetailsSuccess(ctx, tx, task.main.BatchNo, task.main.BatchType, task.main.CreatedBy, nil); err != nil { + return err + } + return p.repo.UpdateBatchMainInfo(ctx, tx, task.main.BatchNo, task.main.BatchType, &rc.BatchMainRow{ + BatchType: task.main.BatchType, + BatchAsyncStatus: consts.BatchAsyncStatusSuccess, + RunResult: consts.BatchRunResultSuccess, + TotalCount: task.total, + ValidCount: task.valid, + SuccessCount: task.valid, + FailCount: 0, + FailReason: "", + BatchInfo: gjson.New(info), + EndTime: gtime.NewFromTime(now), + UpdatedBy: task.main.CreatedBy, + }) + }) + return gerror.Wrap(err, "finalize add package prepare success failed") +} + +func (p *PrepareExecutor) finalizeAutoPrepareSuccess(ctx context.Context, task *batchTask, info map[string]any, now time.Time) error { + if task == nil || task.main == nil { + return gerror.New("add package auto prepare task is empty") + } + targetBatchType := p.targetBatchType + info["autoConfirmed"] = true + info["prepareBatchType"] = consts.BatchTypePrepare + info["status"] = "pending_execute" + err := p.repo.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + for _, plan := range task.plans { + if plan == nil || plan.detail == nil { + continue + } + if err := p.repo.Tables().UpdateBatchDetailSnapshot(ctx, tx, task.main.BatchNo, task.main.BatchType, plan.detail.RowNo, consts.BatchDetailStatusSuccess, buildDetailSnapshot(plan, ""), "", task.main.CreatedBy); err != nil { + return err + } + } + if err := convertAddPackagePrepareDetails(ctx, tx, task.main.BatchNo, targetBatchType, consts.BatchDetailStatusPending, "", task.main.CreatedBy, now); err != nil { + return err + } + return convertAddPackagePrepareMain(ctx, tx, task.main.BatchNo, targetBatchType, &rc.BatchMainRow{ + BatchAsyncStatus: batch.AsyncStatusPending, + RunResult: 0, + TotalCount: task.total, + ValidCount: 0, + SuccessCount: 0, + FailCount: 0, + FailReason: "", + BatchInfo: gjson.New(info), + UpdatedBy: task.main.CreatedBy, + }) + }) + if err != nil { + return gerror.Wrap(err, "auto confirm add package prepare failed") + } + p.publishExecution(ctx, task.main.BatchNo, targetBatchType) + g.Log().Infof(ctx, "add package prepare auto confirmed, batchNo:%s, targetType:%d", task.main.BatchNo, targetBatchType) + return nil +} + +func (p *PrepareExecutor) finalizeAutoPrepareFailure(ctx context.Context, main *rc.BatchMainRow, reason string, task *batchTask) error { + reason = strings.TrimSpace(reason) + if reason == "" { + reason = "加油包预处理失败" + } + now := p.now() + targetBatchType := p.targetBatchType + info := mergeBatchInfo(main.BatchInfo, map[string]any{ + "targetBatchType": targetBatchType, + "prepareBatchType": consts.BatchTypePrepare, + "autoConfirm": true, + "autoConfirmed": false, + "status": "failed", + }) + err := p.repo.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + if task != nil { + for _, plan := range task.plans { + if plan == nil || plan.detail == nil { + continue + } + rowReason := strings.TrimSpace(plan.reason) + if rowReason == "" { + rowReason = reason + } + if err := p.repo.Tables().UpdateBatchDetailSnapshot(ctx, tx, main.BatchNo, main.BatchType, plan.detail.RowNo, consts.BatchDetailStatusFailed, buildDetailSnapshot(plan, ""), rowReason, main.CreatedBy); err != nil { + return err + } + } + } + if err := p.repo.MarkAllDetailsFailed(ctx, tx, main.BatchNo, main.BatchType, reason, main.CreatedBy); err != nil { + return err + } + detailReason := reason + if task != nil && len(task.plans) > 0 { + detailReason = "" + } + if err := convertAddPackagePrepareDetails(ctx, tx, main.BatchNo, targetBatchType, consts.BatchDetailStatusFailed, detailReason, main.CreatedBy, now); err != nil { + return err + } + return convertAddPackagePrepareMain(ctx, tx, main.BatchNo, targetBatchType, &rc.BatchMainRow{ + BatchAsyncStatus: consts.BatchAsyncStatusFailed, + RunResult: consts.BatchRunResultFailed, + FailReason: reason, + TotalCount: main.TotalCount, + ValidCount: 0, + SuccessCount: 0, + FailCount: main.TotalCount, + BatchInfo: gjson.New(info), + EndTime: gtime.NewFromTime(now), + UpdatedBy: main.CreatedBy, + }) + }) + if err != nil { + return gerror.Wrap(err, "finalize auto add package prepare failure failed") + } + rows, _ := p.repo.ResolveResultSnapshot(ctx, main.BatchNo, targetBatchType) + _ = p.repo.WriteResultFileSafe(ctx, main.BatchNo, targetBatchType, main.TenantId, main.CreatedDept, main.CreatedBy, rows) + g.Log().Warningf(ctx, "add package prepare auto failed, batchNo:%s, targetType:%d, reason:%s", main.BatchNo, targetBatchType, reason) + return nil +} + +func (p *PrepareExecutor) publishExecution(ctx context.Context, batchNo string, batchType int) { + if err := rabbitmq.Publish(ctx, consts.QueueBatchExecute, batch.Message{BatchNo: batchNo, BatchType: batchType}); err != nil { + g.Log().Errorf(ctx, "publish add package execution after prepare failed, batchNo:%s, batchType:%d, err:%+v", batchNo, batchType, err) + } +} + +func (p *PrepareExecutor) buildPrepareInfo(ctx context.Context, task *batchTask) (map[string]any, error) { + info := mergeBatchInfo(task.main.BatchInfo, map[string]any{ + "targetBatchType": p.targetBatchType, + "batchMode": map[bool]string{true: "pool", false: "card"}[p.inner.poolMode], + }) + quantity := 0 + totalAmount := 0.0 + var sample *detailPlan + for _, plan := range task.plans { + if plan == nil || plan.reason != "" { + continue + } + if sample == nil { + sample = plan + } + quantity++ + if plan.price == nil { + continue + } + totalAmount = roundAmount(totalAmount + plan.amount) + } + if task.prepareOnly && sample != nil && strings.TrimSpace(sample.packageCode) == "" { + options, err := p.prepareAvailablePackages(ctx, task) + if err != nil { + return nil, err + } + if len(options) == 0 { + return nil, gerror.New("当前没有共同可购的加油包套餐") + } + info["quantity"] = quantity + info["basePackageKey"] = sample.basePackageKey + info["availablePackages"] = options + info["packageOptions"] = options + if _, err := p.fillPreparePaymentView(ctx, task, info); err != nil { + return nil, err + } + info["prepareTime"] = p.now().Format("2006-01-02 15:04:05") + info["status"] = "success" + return info, nil + } + if sample == nil || sample.price == nil { + return nil, gerror.New("加油包预处理没有可确认的明细") + } + info["quantity"] = quantity + info["basePackageKey"] = sample.basePackageKey + info["packageCode"] = sample.packageCode + info["packageName"] = sample.price.PackageName + info["flowMb"] = sample.price.FlowMb + info["unitPrice"] = sample.price.UnitPrice + info["totalAmount"] = totalAmount + info["effectiveMode"] = sample.effectiveMode + info["effectiveModeText"] = effectiveModeText(sample.effectiveMode) + info["effectiveTime"] = gtime.NewFromTime(sample.effectiveTime).Format("Y-m-d H:i:s") + info["expectedExpireTime"] = gtime.NewFromTime(sample.expireTime).Format("Y-m-d H:i:s") + info["priceSnapshot"] = sample.price + accounts, err := p.preparePaymentAccounts(ctx, task) + if err != nil { + return nil, err + } + info["paymentAccounts"] = accounts + isTop, err := p.fillPreparePaymentView(ctx, task, info) + if isAutoConfirmPrepare(task.main) { + if err != nil { + return nil, err + } + intent, err := normalizePaymentIntent(task.paymentIntent, isTop) + if err != nil { + return nil, err + } + task.paymentIntent = intent + info["paymentMode"] = intent.PaymentMode + info["offlineContractCompleted"] = intent.OfflineContractCompleted + if intent.PaymentMode == "balance" { + if err := validatePrepareBalanceEnough(accounts); err != nil { + return nil, err + } + } + } + info["prepareTime"] = p.now().Format("2006-01-02 15:04:05") + info["status"] = "success" + return info, nil +} + +func (p *PrepareExecutor) fillPreparePaymentView(ctx context.Context, task *batchTask, info map[string]any) (bool, error) { + isTop, err := p.inner.isTopDept(ctx, task.main.TenantId, task.main.CreatedDept) + if err == nil { + info["isTopDept"] = isTop + info["defaultPaymentMode"] = "balance" + } + balance, balanceErr := p.availableBalance(ctx, task.main.TenantId, task.main.CreatedDept) + if balanceErr != nil { + return isTop, balanceErr + } + info["availableBalance"] = balance + return isTop, err +} + +func (p *PrepareExecutor) prepareAvailablePackages(ctx context.Context, task *batchTask) ([]map[string]any, error) { + sample := firstValidAddPackagePlan(task) + if sample == nil { + return nil, gerror.New("加油包预处理没有可确认的明细") + } + codes, err := p.availablePackageCodes(ctx, task.main.TenantId, sample) + if err != nil { + return nil, err + } + options := make([]map[string]any, 0, len(codes)) + for _, code := range codes { + option, err := p.buildAvailablePackageOption(ctx, task, code) + if err != nil { + return nil, err + } + if option != nil { + options = append(options, option) + } + } + return options, nil +} + +func firstValidAddPackagePlan(task *batchTask) *detailPlan { + if task == nil { + return nil + } + for _, plan := range task.plans { + if plan != nil && plan.reason == "" { + return plan + } + } + return nil +} + +func (p *PrepareExecutor) availablePackageCodes(ctx context.Context, tenantID string, sample *detailPlan) ([]string, error) { + if p.inner.poolMode { + return p.availablePoolPackageCodes(ctx, tenantID, sample) + } + basePkg, err := p.repo.Tables().LoadPackageByCode(ctx, tenantID, sample.basePackageKey) + if err != nil || basePkg == nil { + return nil, gerror.New("当前卡基础套餐不存在") + } + codes := packageCodesFromJSON(basePkg.TopupPackageCodes) + if len(codes) == 0 { + return nil, gerror.New("当前卡基础套餐未配置可购加油包") + } + return codes, nil +} + +func (p *PrepareExecutor) availablePoolPackageCodes(ctx context.Context, tenantID string, sample *detailPlan) ([]string, error) { + if sample.pool == nil || sample.pool.PoolPackageCode <= 0 { + return nil, gerror.New("流量池未配置基础套餐") + } + var packages []*rc.FlowPackageInfo + if err := p.repo.DB().Model(rc.TableSimFlowPackageInfo).Safe().Ctx(ctx). + Where("tenant_id", tenantID). + Where("pool_package_code", sample.pool.PoolPackageCode). + Where("package_type", consts.PackageTypePoolAddPackage). + Where("enabled", 1). + OrderAsc("sort_no"). + Scan(&packages); err != nil { + return nil, gerror.Wrap(err, "查询流量池加油包套餐失败") + } + codes := make([]string, 0, len(packages)) + for _, pkg := range packages { + if pkg == nil { + continue + } + if code := strings.TrimSpace(pkg.PackageCode); code != "" { + codes = append(codes, code) + } + } + if len(codes) == 0 { + return nil, gerror.New("所选流量池没有共同可购的加油包套餐") + } + return codes, nil +} + +func (p *PrepareExecutor) buildAvailablePackageOption(ctx context.Context, task *batchTask, packageCode string) (map[string]any, error) { + packageCode = strings.TrimSpace(packageCode) + if packageCode == "" { + return nil, nil + } + var ( + samplePrice *PriceSnapshot + totalAmount float64 + unitPrice float64 + priceAvailable = true + priceReason string + accounts = make(map[int64]*prepareAccount) + deptIDs = make([]int64, 0) + ) + for _, plan := range task.plans { + if plan == nil || plan.reason != "" { + continue + } + pkg, price, err := p.inner.resolvePrice(ctx, task.main.TenantId, plan.customerDeptID, packageCode) + if err != nil { + priceAvailable = false + if priceReason == "" { + priceReason = err.Error() + } + continue + } + if p.inner.poolMode { + if pkg.PackageType != consts.PackageTypePoolAddPackage || plan.pool == nil || pkg.PoolPackageCode != plan.pool.PoolPackageCode { + priceAvailable = false + priceReason = "加油包不属于当前流量池套餐" + continue + } + } else if pkg.PackageType != consts.PackageTypeCardAddPackage { + priceAvailable = false + priceReason = "目标套餐类型不是单卡加油包" + continue + } + if samplePrice == nil { + samplePrice = price + unitPrice = price.UnitPrice + } else if !sameAddPackageAmount(unitPrice, price.UnitPrice) { + priceAvailable = false + priceReason = addPackageSamePriceFailureReason + } + totalAmount = roundAmount(totalAmount + price.UnitPrice) + account := accounts[plan.customerDeptID] + if account == nil { + account = &prepareAccount{payerDeptID: plan.customerDeptID} + accounts[plan.customerDeptID] = account + deptIDs = append(deptIDs, plan.customerDeptID) + } + account.count++ + account.required = roundAmount(account.required + price.UnitPrice) + } + if samplePrice == nil { + return nil, nil + } + paymentAccounts, err := p.prepareAccountItems(ctx, task.main.TenantId, accounts, deptIDs) + if err != nil { + return nil, err + } + option := map[string]any{ + "packageCode": samplePrice.PackageCode, + "packageName": samplePrice.PackageName, + "flowMb": samplePrice.FlowMb, + "unitPrice": unitPrice, + "totalAmount": totalAmount, + "priceAvailable": priceAvailable, + "paymentAccounts": paymentAccounts, + } + if priceReason != "" { + option["priceReason"] = priceReason + } + return option, nil +} + +type prepareAccount struct { + payerDeptID int64 + count int + required float64 +} + +func (p *PrepareExecutor) prepareAccountItems(ctx context.Context, tenantID string, accounts map[int64]*prepareAccount, deptIDs []int64) ([]map[string]any, error) { + sort.Slice(deptIDs, func(i, j int) bool { return deptIDs[i] < deptIDs[j] }) + out := make([]map[string]any, 0, len(deptIDs)) + for _, deptID := range deptIDs { + item := accounts[deptID] + if item == nil { + continue + } + balance, err := p.availableBalance(ctx, tenantID, deptID) + if err != nil { + return nil, err + } + out = append(out, map[string]any{ + "payerDeptId": item.payerDeptID, + "targetCount": item.count, + "requiredAmount": item.required, + "availableBalance": balance, + "balanceEnough": balance+0.0001 >= item.required, + }) + } + return out, nil +} + +func validatePrepareBalanceEnough(accounts []map[string]any) error { + for _, account := range accounts { + if account == nil { + continue + } + if enough, _ := account["balanceEnough"].(bool); enough { + continue + } + return gerror.Newf( + "账户余额不足，付款部门%v需支付%.2f元，当前余额%.2f元", + account["payerDeptId"], + floatFromAny(account["requiredAmount"]), + floatFromAny(account["availableBalance"]), + ) + } + return nil +} + +func floatFromAny(value any) float64 { + switch v := value.(type) { + case float64: + return v + case float32: + return float64(v) + case int: + return float64(v) + case int64: + return float64(v) + case int32: + return float64(v) + case uint: + return float64(v) + case uint64: + return float64(v) + case uint32: + return float64(v) + default: + return 0 + } +} + +func (p *PrepareExecutor) preparePaymentAccounts(ctx context.Context, task *batchTask) ([]map[string]any, error) { + type account struct { + payerDeptID int64 + count int + required float64 + } + byDept := make(map[int64]*account) + deptIDs := make([]int64, 0) + for _, plan := range task.plans { + if plan == nil || plan.reason != "" { + continue + } + payerDeptID := plan.customerDeptID + if payerDeptID <= 0 { + return nil, gerror.New("加油包付款部门不能为空") + } + item := byDept[payerDeptID] + if item == nil { + item = &account{payerDeptID: payerDeptID} + byDept[payerDeptID] = item + deptIDs = append(deptIDs, payerDeptID) + } + item.count++ + item.required = roundAmount(item.required + plan.amount) + } + sort.Slice(deptIDs, func(i, j int) bool { return deptIDs[i] < deptIDs[j] }) + out := make([]map[string]any, 0, len(deptIDs)) + for _, deptID := range deptIDs { + item := byDept[deptID] + balance, err := p.availableBalance(ctx, task.main.TenantId, deptID) + if err != nil { + return nil, err + } + out = append(out, map[string]any{ + "payerDeptId": item.payerDeptID, + "targetCount": item.count, + "requiredAmount": item.required, + "availableBalance": balance, + "balanceEnough": balance+0.0001 >= item.required, + }) + } + return out, nil +} + +func (p *PrepareExecutor) availableBalance(ctx context.Context, tenantID string, deptID int64) (float64, error) { + if deptID <= 0 { + return 0, nil + } + var row agentBalanceRow + if err := p.repo.DB().Model("t_agent_balance").Ctx(ctx). + Where("tenant_id", tenantID). + Where("dept_id", deptID). + Scan(&row); err != nil { + if errors.Is(err, sql.ErrNoRows) { + return 0, nil + } + return 0, gerror.Wrap(err, "query add package balance failed") + } + if row.Id == 0 { + return 0, nil + } + return roundAmount(row.Balance), nil +} + +func (p *PrepareExecutor) now() time.Time { + if p == nil || p.nowFunc == nil { + return time.Now() + } + return p.nowFunc() +} + +type PrepareMux struct { + repo *renewchangeservice.Repository + renew batch.Executor + card *PrepareExecutor + pool *PrepareExecutor +} + +func NewPrepareMux(repo *renewchangeservice.Repository, renew batch.Executor) *PrepareMux { + if repo == nil { + repo = renewchangeservice.NewRepository() + } + return &PrepareMux{ + repo: repo, + renew: renew, + card: NewCardPrepareExecutor(repo), + pool: NewPoolPrepareExecutor(repo), + } +} + +func (m *PrepareMux) Execute(ctx context.Context, batchNo string) error { + main, err := m.repo.LoadProcessingBatch(ctx, strings.TrimSpace(batchNo), consts.BatchTypePrepare) + if err != nil { + return err + } + switch targetBatchTypeFromInfo(main.BatchInfo) { + case consts.BatchTypeCardAddPackage: + return m.card.Execute(ctx, batchNo) + case consts.BatchTypePoolAddPackage: + return m.pool.Execute(ctx, batchNo) + default: + if m.renew != nil { + return m.renew.Execute(ctx, batchNo) + } + return gerror.Newf("unsupported prepare target batch type: %d", targetBatchTypeFromInfo(main.BatchInfo)) + } +} + +func (m *PrepareMux) CanExecuteProcessing(ctx context.Context, main *entity.BatchMain) (bool, error) { + if main == nil || main.BatchType != consts.BatchTypePrepare || main.BatchAsyncStatus != consts.BatchAsyncStatusProcessing { + return false, nil + } + switch targetBatchTypeFromInfo(main.BatchInfo) { + case consts.BatchTypeCardAddPackage: + return m.card.CanExecuteProcessing(ctx, main) + case consts.BatchTypePoolAddPackage: + return m.pool.CanExecuteProcessing(ctx, main) + default: + if processing, ok := m.renew.(batch.ProcessingExecutor); ok { + return processing.CanExecuteProcessing(ctx, main) + } + return false, nil + } +} + +func targetBatchTypeFromInfo(info *gjson.Json) int { + if info == nil || info.IsNil() { + return 0 + } + if v := info.Get("targetBatchType"); v != nil && !v.IsNil() { + return v.Int() + } + if v := info.Get("target_batch_type"); v != nil && !v.IsNil() { + return v.Int() + } + return 0 +} + +func isAutoConfirmPrepare(main *rc.BatchMainRow) bool { + if main == nil || main.SourceType != consts.BatchSourceTypeFile || main.BatchInfo == nil || main.BatchInfo.IsNil() { + return false + } + return main.BatchInfo.Get("autoConfirm").Bool() +} + +func convertAddPackagePrepareDetails(ctx context.Context, tx gdb.TX, batchNo string, targetBatchType int, detailStatus int, reason string, userID int64, now time.Time) error { + data := map[string]any{ + "batch_type": targetBatchType, + "detail_status": detailStatus, + "updated_by": userID, + "updated_at": gtime.NewFromTime(now), + } + if strings.TrimSpace(reason) != "" { + data["error_reason"] = reason + } + _, err := tx.Ctx(ctx).Model(rc.TableBatchDetail). + Where("batch_no", batchNo). + Where("batch_type", consts.BatchTypePrepare). + Data(data). + Update() + return gerror.Wrap(err, "convert add package prepare details failed") +} + +func convertAddPackagePrepareMain(ctx context.Context, tx gdb.TX, batchNo string, targetBatchType int, data *rc.BatchMainRow) error { + if data == nil { + return gerror.New("add package batch conversion data is empty") + } + update := map[string]any{ + "batch_type": targetBatchType, + "batch_async_status": data.BatchAsyncStatus, + "run_result": data.RunResult, + "total_count": data.TotalCount, + "valid_count": data.ValidCount, + "success_count": data.SuccessCount, + "fail_count": data.FailCount, + "fail_reason": data.FailReason, + "batch_info": data.BatchInfo, + "start_time": nil, + "end_time": data.EndTime, + "updated_by": data.UpdatedBy, + "updated_at": gtime.Now(), + } + if data.EndTime == nil { + update["end_time"] = nil + } + result, err := tx.Ctx(ctx).Model(rc.TableBatchMain). + Where("batch_no", batchNo). + Where("batch_type", consts.BatchTypePrepare). + Data(update). + Update() + if err != nil { + return gerror.Wrap(err, "convert add package prepare main failed") + } + affected, err := result.RowsAffected() + if err != nil { + return gerror.Wrap(err, "read add package prepare main conversion result failed") + } + if affected == 0 { + return gerror.New("加油包预处理批次已变化，请重试") + } + return nil +} diff --git a/internal/service/addpackage/types.go b/internal/service/addpackage/types.go new file mode 100644 index 0000000..60332c9 --- /dev/null +++ b/internal/service/addpackage/types.go @@ -0,0 +1,129 @@ +package addpackage + +import ( + "strings" + "time" + + "github.com/gogf/gf/v2/encoding/gjson" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" +) + +const ( + effectiveModeCurrentMonth = "current_month" + effectiveModeCurrentText = "本月生效" +) + +type NowFunc func() time.Time + +type PriceSnapshot struct { + PackageID int64 `json:"packageId"` + PackageCode string `json:"packageCode"` + PackageName string `json:"packageName"` + PackageType int `json:"packageType"` + PriceSource string `json:"priceSource"` + AgentDefaultPriceID int64 `json:"agentDefaultPriceId"` + TargetPriceID int64 `json:"targetPriceId"` + UnitPrice float64 `json:"unitPrice"` + FlowMb float64 `json:"flowMb"` +} + +type DetailSnapshot struct { + TargetCode string `json:"targetCode"` + PackageCode string `json:"packageCode"` + Remark string `json:"remark"` + EffectiveMode string `json:"effectiveMode"` +} + +type PaymentIntent struct { + PaymentMode string `json:"paymentMode"` + OfflineContractCompleted bool `json:"offlineContractCompleted"` +} + +type PaymentSnapshot struct { + PaymentMode string `json:"paymentMode"` + PayType int `json:"payType"` + PayStatus int `json:"payStatus"` + PayAmount float64 `json:"payAmount"` + BalanceAmount float64 `json:"balanceAmount"` + OfflinePaidAmount float64 `json:"offlinePaidAmount"` + BalanceBefore float64 `json:"balanceBefore"` + BalanceAfter float64 `json:"balanceAfter"` + BalanceDetailSerialNo string `json:"balanceDetailSerialNo"` + PaymentTime time.Time `json:"paymentTime"` + PayerDeptID int64 `json:"payerDeptId"` + PayerUserID int64 `json:"payerUserId"` + IsTopDept bool `json:"isTopDept"` +} + +type detailPlan struct { + detail *renewchange.BatchDetailRow + targetCode string + packageCode string + basePackageKey string + effectiveMode string + remark string + customerDeptID int64 + orderSN string + price *PriceSnapshot + unitAmount float64 + amount float64 + effectiveTime time.Time + expireTime time.Time + effectiveNow bool + reason string + card *renewchange.SimCardInfo + pool *flowPoolRow +} + +type batchTask struct { + main *renewchange.BatchMainRow + plans []*detailPlan + paymentIntent PaymentIntent + paymentSnapshot map[int64]*PaymentSnapshot + prepareOnly bool + reason string + total int + valid int + success int + fail int +} + +func timeNow() time.Time { + return time.Now() +} + +func readPaymentIntent(info *gjson.Json) PaymentIntent { + intent := PaymentIntent{PaymentMode: "balance"} + if info == nil || info.IsNil() { + return intent + } + if mode := strings.TrimSpace(info.Get("paymentMode").String()); mode != "" { + intent.PaymentMode = mode + } + if intent.PaymentMode == "" { + intent.PaymentMode = "balance" + } + intent.OfflineContractCompleted = info.Get("offlineContractCompleted").Bool() + return intent +} + +func readDetailSnapshot(info *gjson.Json) *DetailSnapshot { + if info == nil { + return nil + } + raw := info.Map() + if len(raw) == 0 { + return nil + } + snapshot := &DetailSnapshot{ + TargetCode: strings.TrimSpace(info.Get("targetCode").String()), + PackageCode: strings.TrimSpace(info.Get("packageCode").String()), + Remark: strings.TrimSpace(info.Get("remark").String()), + EffectiveMode: strings.TrimSpace(info.Get("effectiveMode").String()), + } + if snapshot.TargetCode == "" && snapshot.PackageCode == "" && snapshot.Remark == "" && snapshot.EffectiveMode == "" { + return nil + } + return snapshot +} diff --git a/internal/service/allocate/atomic_parent.go b/internal/service/allocate/atomic_parent.go new file mode 100644 index 0000000..ceb8e15 --- /dev/null +++ b/internal/service/allocate/atomic_parent.go @@ -0,0 +1,147 @@ +package allocate + +import ( + "context" + "sort" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/errors/gerror" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +func (r *Repository) loadParentTaskTx(ctx context.Context, tx gdb.TX, parent *entity.BatchMain) (*Task, error) { + var batchDetails []*entity.BatchDetail + if err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where("batch_no IN (SELECT batch_no FROM batch_main WHERE source_batch_no = ? AND batch_type = ? AND tenant_id = ?)", parent.BatchNo, consts.BatchTypeAllocate, parent.TenantId). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeAllocate). + OrderAsc(dao.BatchDetail.Columns().RowNo). + OrderAsc(dao.BatchDetail.Columns().BatchNo). + OrderAsc(dao.BatchDetail.Columns().Id). + Scan(&batchDetails); err != nil { + return nil, gerror.Wrap(err, "load allocate parent details failed") + } + if len(batchDetails) == 0 { + return nil, gerror.New("allocate parent has no child details") + } + + task := &Task{ + Batch: Batch{ + ID: parent.Id, BatchNo: parent.BatchNo, BatchType: consts.BatchTypeAllocate, + TenantID: parent.TenantId, TargetDeptID: targetDeptIDFromBatch(parent), + CreatedDept: parent.CreatedDept, CreatedBy: parent.CreatedBy, + StartTime: parent.StartTime, BatchInfo: parent.BatchInfo, + }, + Details: make([]*Detail, 0, len(batchDetails)), + TargetsByID: make(map[int64]*Target), TargetsByKey: make(map[string]*Target), TargetsByName: make(map[string][]*Target), + } + cardIDs := make([]int64, 0, len(batchDetails)) + targetIDs := make([]int64, 0, len(batchDetails)) + cardIDByDetailID := make(map[int64]int64, len(batchDetails)) + for _, batchDetail := range batchDetails { + if batchDetail == nil { + continue + } + detail := &Detail{OriginalSnapshot: batchDetail.Snapshot, ID: batchDetail.Id, BatchNo: batchDetail.BatchNo, RowNo: batchDetail.RowNo, DetailStatus: batchDetail.DetailStatus, ICCID: strings.TrimSpace(batchDetail.ItemId), Reason: strings.TrimSpace(batchDetail.ErrorReason)} + if detail.DetailStatus != consts.BatchDetailStatusSuccess && detail.Reason == "" { + detail.Reason = "子批次明细尚未全部验证成功" + } + if snapshot := batchDetail.Snapshot; snapshot != nil { + detail.Remark = firstSnapshot(snapshot, "remark") + detail.MSISDN = firstSnapshot(snapshot, "MSISDN", "msisdn") + detail.IMSI = firstSnapshot(snapshot, "IMSI", "imsi") + detail.TargetDeptID = snapshot.Get("targetDeptId").Int64() + detail.TargetDeptKey = firstSnapshot(snapshot, "targetDeptKey", "target_dept_key") + detail.TargetDeptName = firstSnapshot(snapshot, "targetDeptName", "target_dept_name") + if cardID := snapshot.Get("cardId").Int64(); cardID > 0 { + cardIDs = append(cardIDs, cardID) + cardIDByDetailID[batchDetail.Id] = cardID + } else if detail.Reason == "" { + detail.Reason = "卡片快照缺少有效卡片ID" + } + } + if detail.TargetDeptID > 0 { + targetIDs = append(targetIDs, detail.TargetDeptID) + } + task.Details = append(task.Details, detail) + } + + cardsByID := make(map[int64]*entity.TSimCardInfo, len(cardIDs)) + cardIDs = uniqueInt64s(cardIDs) + sort.Slice(cardIDs, func(i, j int) bool { return cardIDs[i] < cardIDs[j] }) + for start := 0; start < len(cardIDs); start += allocateCardQueryBatchSize { + end := min(start+allocateCardQueryBatchSize, len(cardIDs)) + var cards []*entity.TSimCardInfo + if err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()).Safe(). + Where(dao.TSimCardInfo.Columns().TenantId, parent.TenantId). + WhereIn(dao.TSimCardInfo.Columns().Id, cardIDs[start:end]). + LockUpdate().Scan(&cards); err != nil { + return nil, gerror.Wrap(err, "lock allocate parent cards failed") + } + for _, card := range cards { + if card != nil { + cardsByID[card.Id] = card + } + } + } + + targetsByID := make(map[int64]*Target, len(targetIDs)) + targetIDs = uniqueInt64s(targetIDs) + sort.Slice(targetIDs, func(i, j int) bool { return targetIDs[i] < targetIDs[j] }) + for start := 0; start < len(targetIDs); start += allocateCardQueryBatchSize { + end := min(start+allocateCardQueryBatchSize, len(targetIDs)) + var depts []*entity.SysDept + if err := tx.Ctx(ctx).Model(dao.SysDept.Table()). + Where(dao.SysDept.Columns().TenantId, parent.TenantId). + Where(dao.SysDept.Columns().ParentId, parent.CreatedDept). + Where(dao.SysDept.Columns().Status, "0"). + WhereNull(dao.SysDept.Columns().DeletedAt). + WhereIn(dao.SysDept.Columns().DeptId, targetIDs[start:end]). + LockUpdate().Scan(&depts); err != nil { + return nil, gerror.Wrap(err, "lock allocate parent targets failed") + } + for _, dept := range depts { + if dept == nil { + continue + } + target := &Target{DeptID: dept.DeptId, ParentID: dept.ParentId, DeptKey: strings.TrimSpace(dept.DeptKey), DeptName: strings.TrimSpace(dept.DeptName)} + targetsByID[target.DeptID] = target + task.TargetsByID[target.DeptID] = target + if target.DeptKey != "" { + task.TargetsByKey[target.DeptKey] = target + } + if target.DeptName != "" { + task.TargetsByName[target.DeptName] = append(task.TargetsByName[target.DeptName], target) + } + } + } + for _, detail := range task.Details { + if detail == nil { + continue + } + if detail.Target == nil { + detail.Target = targetsByID[detail.TargetDeptID] + } + cardID := cardIDByDetailID[detail.ID] + if card := cardsByID[cardID]; card != nil { + detail.Card = &Card{ID: card.Id, ICCID: strings.TrimSpace(card.Iccid), MSISDN: strings.TrimSpace(card.Msisdn), IMSI: strings.TrimSpace(card.Imsi), DeptID: card.DeptId, AllocationBatchNo: strings.TrimSpace(card.AllocationBatchNo), PackageStatus: card.PackageStatus, BossCardStatus: card.BossCardStatus, FlowPkgEndAt: card.FlowPkgEndAt} + } + } + return task, nil +} + +func uniqueInt64s(values []int64) []int64 { + seen := make(map[int64]struct{}, len(values)) + unique := make([]int64, 0, len(values)) + for _, value := range values { + if _, exists := seen[value]; exists { + continue + } + seen[value] = struct{}{} + unique = append(unique, value) + } + return unique +} diff --git a/internal/service/allocate/deadlock.go b/internal/service/allocate/deadlock.go new file mode 100644 index 0000000..ded49f9 --- /dev/null +++ b/internal/service/allocate/deadlock.go @@ -0,0 +1,88 @@ +package allocate + +import ( + "context" + "strings" + "sync" + "time" + + "github.com/gogf/gf/v2/frame/g" +) + +const ( + defaultAllocateDeadlockMaxRetries = 3 + defaultAllocateDeadlockRetryDelay = 200 * time.Millisecond +) + +type allocateParentLock struct { + mu sync.Mutex + refs int +} + +var allocateParentLocks = struct { + mu sync.Mutex + values map[string]*allocateParentLock +}{values: make(map[string]*allocateParentLock)} + +func acquireAllocateParentLock(batchNo string) func() { + allocateParentLocks.mu.Lock() + lock := allocateParentLocks.values[batchNo] + if lock == nil { + lock = &allocateParentLock{} + allocateParentLocks.values[batchNo] = lock + } + lock.refs++ + allocateParentLocks.mu.Unlock() + + lock.mu.Lock() + return func() { + lock.mu.Unlock() + allocateParentLocks.mu.Lock() + lock.refs-- + if lock.refs == 0 { + delete(allocateParentLocks.values, batchNo) + } + allocateParentLocks.mu.Unlock() + } +} + +func allocateDeadlockMaxRetries(ctx context.Context) int { + retries := g.Cfg().MustGet(ctx, "batch.allocate.deadlockMaxRetries", defaultAllocateDeadlockMaxRetries).Int() + if retries < 0 { + return 0 + } + return retries +} + +func allocateDeadlockRetryDelay(ctx context.Context, attempt int) time.Duration { + delay := g.Cfg().MustGet(ctx, "batch.allocate.deadlockRetryDelay", defaultAllocateDeadlockRetryDelay).Duration() + if delay <= 0 { + delay = defaultAllocateDeadlockRetryDelay + } + for i := 0; i < attempt; i++ { + if delay >= time.Second { + return time.Second + } + delay *= 2 + } + return delay +} + +func waitAllocateDeadlockRetry(ctx context.Context, attempt int) error { + timer := time.NewTimer(allocateDeadlockRetryDelay(ctx, attempt)) + defer timer.Stop() + select { + case <-ctx.Done(): + return ctx.Err() + case <-timer.C: + return nil + } +} + +func isAllocateDeadlock(err error) bool { + if err == nil { + return false + } + message := strings.ToLower(err.Error()) + return strings.Contains(message, "deadlock found") || strings.Contains(message, "error 1213") || strings.Contains(message, "1213 (40001)") +} diff --git a/internal/service/allocate/detail_update_test.go b/internal/service/allocate/detail_update_test.go new file mode 100644 index 0000000..52ab8c8 --- /dev/null +++ b/internal/service/allocate/detail_update_test.go @@ -0,0 +1,119 @@ +package allocate + +import ( + "context" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + _ "github.com/gogf/gf/contrib/drivers/sqlite/v2" + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" + "testing" +) + +func TestPersistedAllocateDetailsRequireExactBusinessResult(t *testing.T) { + task := &Task{Batch: Batch{BatchNo: "ALC-child", TenantID: "000000", CreatedBy: 133, CreatedDept: 175}} + detail := &Detail{ID: 1, ICCID: "89860401790000277308", TargetDeptID: 185} + good := entity.BatchDetail{Id: 1, BatchNo: task.Batch.BatchNo, BatchType: consts.BatchTypeAllocate, TenantId: task.Batch.TenantID, DetailStatus: consts.BatchDetailStatusSuccess, UpdatedBy: 133, Snapshot: gjson.New(detail.Snapshot(task.Batch).String())} + if err := validatePersistedDetailResults(task, []*Detail{detail}, []*entity.BatchDetail{&good}); err != nil { + t.Fatal(err) + } + for _, name := range []string{"missing", "tenant", "batch", "type", "status", "reason", "snapshot", "operator", "duplicate"} { + t.Run(name, func(t *testing.T) { + row := good + rows := []*entity.BatchDetail{&row} + details := []*Detail{detail} + switch name { + case "missing": + rows = nil + case "tenant": + row.TenantId = "other" + case "batch": + row.BatchNo = "other" + case "type": + row.BatchType = 4 + case "status": + row.DetailStatus = consts.BatchDetailStatusRunning + case "reason": + row.ErrorReason = "unexpected" + case "snapshot": + row.Snapshot = gjson.New(map[string]any{"targetDeptId": 999}) + case "operator": + row.UpdatedBy = 999 + case "duplicate": + details = append(details, detail) + rows = append(rows, &row) + } + if validatePersistedDetailResults(task, details, rows) == nil { + t.Fatal("inconsistent detail accepted") + } + }) + } + detail.Reason = "整批校验失败" + good.DetailStatus = consts.BatchDetailStatusFailed + good.ErrorReason = detail.Reason + good.Snapshot = gjson.New(detail.Snapshot(task.Batch).String()) + if err := validatePersistedDetailResults(task, []*Detail{detail}, []*entity.BatchDetail{&good}); err != nil { + t.Fatal(err) + } +} + +func TestDetailChunkZeroAffectedChecksPersistedResult(t *testing.T) { + ctx := context.Background() + db, err := gdb.New(gdb.ConfigNode{Type: "sqlite", Link: "sqlite::@file(:memory:)"}) + if err != nil { + t.Fatal(err) + } + defer db.Close(ctx) + for _, sql := range []string{ + "CREATE TABLE batch_detail (id INTEGER PRIMARY KEY, batch_no TEXT, batch_type INTEGER, tenant_id TEXT, detail_status INTEGER, error_reason TEXT, snapshot TEXT, updated_by INTEGER, updated_at DATETIME)", + "CREATE TABLE card_test (id INTEGER PRIMARY KEY, dept_id INTEGER)", + "INSERT INTO card_test VALUES (1,175)", + } { + if _, err = db.Exec(ctx, sql); err != nil { + t.Fatal(err) + } + } + task := &Task{Batch: Batch{BatchNo: "ALC-child", TenantID: "000000", CreatedBy: 133, CreatedDept: 175}} + detail := &Detail{ID: 1, ICCID: "CARD", TargetDeptID: 185, Remark: "保留备注"} + if _, err = db.Model("batch_detail").Data(do.BatchDetail{Id: 1, BatchNo: task.Batch.BatchNo, BatchType: 3, TenantId: task.Batch.TenantID, DetailStatus: 2, ErrorReason: "", Snapshot: detail.Snapshot(task.Batch), UpdatedBy: 133}).Insert(); err != nil { + t.Fatal(err) + } + // 强制模拟驱动返回零修改行，不依赖 SQLite 与 MySQL 行数语义差异。 + if _, err = db.Exec(ctx, "CREATE TRIGGER ignore_update BEFORE UPDATE ON batch_detail BEGIN SELECT RAISE(IGNORE); END"); err != nil { + t.Fatal(err) + } + repo := &Repository{db: db} + if err = db.Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + return repo.updateDetailChunk(ctx, tx, task, []*Detail{detail}, gtime.Now()) + }); err != nil { + t.Fatalf("identical persisted result rejected: %v", err) + } + for _, name := range []string{"mismatch", "missing", "tenant"} { + t.Run(name, func(t *testing.T) { + changed := *detail + changed.Reason = "unexpected" + if name == "missing" { + changed.ID = 999 + } + scopedTask := *task + if name == "tenant" { + scopedTask.Batch.TenantID = "other" + } + err := db.Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + if _, err := tx.Exec("UPDATE card_test SET dept_id=185 WHERE id=1"); err != nil { + return err + } + return repo.updateDetailChunk(ctx, tx, &scopedTask, []*Detail{&changed}, gtime.Now()) + }) + if err == nil { + t.Fatal("unsafe update accepted") + } + dept, err := db.Model("card_test").Value("dept_id") + if err != nil || dept.Int() != 175 { + t.Fatalf("transaction failed to rollback ownership: %v %v", dept, err) + } + }) + } +} diff --git a/internal/service/allocate/executor.go b/internal/service/allocate/executor.go new file mode 100644 index 0000000..b7cf66f --- /dev/null +++ b/internal/service/allocate/executor.go @@ -0,0 +1,294 @@ +package allocate + +import ( + "context" + "strings" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/packagecustomer" +) + +const defaultAllocateProcessingRetryAfter = 30 * time.Minute + +const allocateParentJobRole = "allocate_parent" + +type executor struct { + repository *Repository +} + +func NewExecutor() *executor { + return &executor{repository: NewRepository()} +} + +func (e *executor) CanExecuteProcessing(ctx context.Context, main *entity.BatchMain) (bool, error) { + if main == nil || main.BatchAsyncStatus != consts.BatchAsyncStatusProcessing { + return false, nil + } + if isAllocateParent(main) { + return true, nil + } + if main.BatchType != consts.BatchTypeAllocate { + return false, nil + } + if main.StartTime == nil || main.StartTime.IsZero() { + return true, nil + } + return gtime.Now().Sub(main.StartTime) >= allocateProcessingRetryAfter(ctx), nil +} + +func (e *executor) HandleExecutionFailure(ctx context.Context, batchNo string, batchType int, _ string) error { + if batchType != consts.BatchTypeAllocate { + return nil + } + main, err := e.repository.LoadMain(ctx, batchNo) + if err != nil { + return err + } + if main == nil { + return nil + } + if strings.TrimSpace(main.SourceBatchNo) == "" { + return nil + } + return e.repository.AggregateParent(ctx, main.SourceBatchNo) +} + +func allocateProcessingRetryAfter(ctx context.Context) time.Duration { + duration := g.Cfg().MustGet(ctx, "batch.allocate.processingRetryAfter", defaultAllocateProcessingRetryAfter).Duration() + if duration <= 0 { + return defaultAllocateProcessingRetryAfter + } + return duration +} + +func (e *executor) Execute(ctx context.Context, batchNo string) error { + main, err := e.repository.LoadMain(ctx, strings.TrimSpace(batchNo)) + if err != nil { + return gerror.Wrap(err, "load allocate batch main failed") + } + if isAllocateParent(main) { + return e.repository.AggregateParent(ctx, main.BatchNo) + } + task, err := e.repository.Load(ctx, strings.TrimSpace(batchNo)) + if err != nil { + return gerror.Wrap(err, "load allocate batch failed") + } + validateTask(task) + if err = e.repository.Commit(ctx, task); err != nil { + return gerror.Wrap(err, "commit allocate batch failed") + } + if strings.TrimSpace(task.Batch.ParentBatchNo) == "" { + publishPackageCustomerSync(ctx, task) + } + return nil +} + +func isAllocateParent(main *entity.BatchMain) bool { + if main == nil || main.BatchType != consts.BatchTypeAllocate || strings.TrimSpace(main.SourceBatchNo) != "" { + return false + } + return main.BatchInfo != nil && main.BatchInfo.Get("jobRole").String() == allocateParentJobRole +} + +func validateTask(task *Task) { + if task == nil || len(task.Details) == 0 { + if task != nil { + task.FailureReason = "划拨批次没有可处理明细" + } + return + } + + iccidCounts := make(map[string]int, len(task.Details)) + for _, detail := range task.Details { + if detail == nil { + continue + } + detail.ICCID = strings.TrimSpace(detail.ICCID) + detail.MSISDN = strings.TrimSpace(detail.MSISDN) + detail.IMSI = strings.TrimSpace(detail.IMSI) + detail.TargetDeptKey = strings.TrimSpace(detail.TargetDeptKey) + detail.TargetDeptName = strings.TrimSpace(detail.TargetDeptName) + if detail.ICCID != "" { + iccidCounts[detail.ICCID]++ + } + } + + for _, detail := range task.Details { + if detail == nil { + continue + } + if detail.Reason == "" && iccidCounts[detail.ICCID] > 1 { + detail.Reason = "ICCID在批次内重复" + } + validateDetail(task, detail) + } + _, failCount := countResult(task) + if failCount > 0 { + markAtomicFailure(task, "划拨批次存在失败明细，整批未执行") + } +} + +func publishPackageCustomerSync(ctx context.Context, task *Task) { + if task == nil { + return + } + deptIds := []int64{task.Batch.CreatedDept} + hasSuccess := false + for _, detail := range task.Details { + if detail == nil || !detail.Success() { + continue + } + hasSuccess = true + if detail.TargetDeptID > 0 { + deptIds = append(deptIds, detail.TargetDeptID) + } + } + if !hasSuccess { + return + } + packagecustomer.PublishSyncForDeptsSafe(ctx, task.Batch.TenantID, task.Batch.CreatedBy, deptIds, "allocate", task.Batch.BatchNo) +} + +func markAtomicFailure(task *Task, reason string) { + if task == nil { + return + } + reason = strings.TrimSpace(reason) + if reason == "" { + reason = "本批次存在失败明细，整批未执行" + } + task.FailureReason = reason + for _, detail := range task.Details { + if detail != nil && strings.TrimSpace(detail.Reason) == "" { + detail.Reason = "本批次存在失败明细，整批未执行" + } + } +} + +func validateDetail(task *Task, detail *Detail) { + if detail == nil || strings.TrimSpace(detail.Reason) != "" { + return + } + if detail.ICCID == "" { + detail.Reason = "ICCID不能为空" + return + } + target := resolveTarget(task, detail) + if target == nil { + return + } + detail.Target = target + detail.TargetDeptID = target.DeptID + if detail.TargetDeptKey == "" { + detail.TargetDeptKey = strings.TrimSpace(target.DeptKey) + } + if detail.TargetDeptName == "" { + detail.TargetDeptName = strings.TrimSpace(target.DeptName) + } + if detail.Card == nil { + detail.Reason = "ICCID不存在或不属于当前租户" + return + } + if strings.TrimSpace(detail.ICCID) != strings.TrimSpace(detail.Card.ICCID) { + detail.Reason = "ICCID与卡档案不一致" + return + } + detail.MSISDN = coalesce(detail.MSISDN, detail.Card.MSISDN) + detail.IMSI = coalesce(detail.IMSI, detail.Card.IMSI) + if detail.Card.DeptID != task.Batch.CreatedDept { + detail.Reason = "ICCID不在当前操作员授权范围内" + return + } + if strings.TrimSpace(detail.MSISDN) != "" && strings.TrimSpace(detail.MSISDN) != detail.Card.MSISDN { + detail.Reason = "MSISDN与卡档案不一致" + return + } + if strings.TrimSpace(detail.IMSI) != "" && strings.TrimSpace(detail.IMSI) != detail.Card.IMSI { + detail.Reason = "IMSI与卡档案不一致" + return + } + if detail.Card.BossCardStatus == consts.SimBossCardStatusStopped { + detail.Reason = "卡已停机，不能划拨" + return + } + if detail.Card.BossCardStatus == consts.SimBossCardStatusCanceled { + detail.Reason = "卡已销户，不能划拨" + return + } + if detail.Card.PackageStatus == consts.PackageStatusExpired || + (detail.Card.FlowPkgEndAt != nil && !detail.Card.FlowPkgEndAt.IsZero() && detail.Card.FlowPkgEndAt.Before(gtime.Now())) { + detail.Reason = "套餐已过期，不能划拨" + return + } + if detail.Card.DeptID == target.DeptID { + detail.Reason = "卡已属于目标客户，无需重复划拨" + } +} + +func resolveTarget(task *Task, detail *Detail) *Target { + if task == nil || detail == nil { + return nil + } + targetDeptID := detail.TargetDeptID + if targetDeptID <= 0 { + targetDeptID = task.Batch.TargetDeptID + } + if targetDeptID > 0 { + target := task.TargetsByID[targetDeptID] + if target == nil { + detail.Reason = "划拨目标客户不存在或不在当前部门的下级范围内" + return nil + } + return target + } + if detail.TargetDeptKey != "" { + target := task.TargetsByKey[detail.TargetDeptKey] + if target == nil { + detail.Reason = "划拨目标客户编码不存在或不在当前部门的下级范围内" + return nil + } + return target + } + if detail.TargetDeptName != "" { + targets := task.TargetsByName[detail.TargetDeptName] + if len(targets) == 0 { + detail.Reason = "划拨目标客户名称不存在或不在当前部门的下级范围内" + return nil + } + if len(targets) > 1 { + detail.Reason = "划拨目标客户名称不唯一，请使用目标客户编码" + return nil + } + return targets[0] + } + detail.Reason = "目标客户不能为空" + return nil +} + +func countResult(task *Task) (successCount, failCount int) { + if task == nil { + return 0, 0 + } + for _, detail := range task.Details { + if detail != nil && detail.Success() { + successCount++ + } else { + failCount++ + } + } + return successCount, failCount +} + +func coalesce(value, fallback string) string { + value = strings.TrimSpace(value) + if value != "" { + return value + } + return strings.TrimSpace(fallback) +} diff --git a/internal/service/allocate/executor_test.go b/internal/service/allocate/executor_test.go new file mode 100644 index 0000000..bedfea9 --- /dev/null +++ b/internal/service/allocate/executor_test.go @@ -0,0 +1,197 @@ +package allocate + +import ( + "context" + "errors" + "strings" + "testing" + "time" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +func TestAllocateProcessingRetryAfterHasLargeBatchDefault(t *testing.T) { + if got := allocateProcessingRetryAfter(context.Background()); got < 30*time.Minute { + t.Fatalf("allocateProcessingRetryAfter() = %s, want at least 30m", got) + } +} + +func TestIsAllocateParentRequiresTypeAndRole(t *testing.T) { + parent := &entity.BatchMain{ + BatchType: consts.BatchTypeAllocate, + BatchInfo: gjson.New(`{"jobRole":"allocate_parent"}`), + } + if !isAllocateParent(parent) { + t.Fatal("expected allocate parent metadata to be recognized") + } + parent.SourceBatchNo = "ALC-CHILD" + if isAllocateParent(parent) { + t.Fatal("child batch must not be recognized as allocate parent") + } +} + +func TestIsAllocateDeadlock(t *testing.T) { + if !isAllocateDeadlock(errors.New("Error 1213 (40001): Deadlock found when trying to get lock")) { + t.Fatal("expected MySQL deadlock to be retryable") + } + if isAllocateDeadlock(errors.New("validation failed")) { + t.Fatal("validation errors must not be treated as deadlocks") + } +} + +func TestResolveAllocateParentStartTime(t *testing.T) { + parentStart := gtime.New("2026-09-29 12:59:00") + childStart := gtime.New("2026-09-29 12:59:32") + fallback := gtime.New("2026-09-29 13:00:21") + + if got := resolveAllocateParentStartTime(parentStart, childStart, fallback); got != parentStart { + t.Fatalf("parent start time = %v, want existing parent time %v", got, parentStart) + } + if got := resolveAllocateParentStartTime(nil, childStart, fallback); got != childStart { + t.Fatalf("parent start time = %v, want earliest child time %v", got, childStart) + } + if got := resolveAllocateParentStartTime(nil, nil, fallback); got != fallback { + t.Fatalf("parent start time = %v, want fallback time %v", got, fallback) + } +} + +func TestAllocateBatchSizesRemainBounded(t *testing.T) { + if allocateCardQueryBatchSize <= 0 || allocateCardQueryBatchSize > 1000 { + t.Fatalf("allocateCardQueryBatchSize = %d", allocateCardQueryBatchSize) + } + if allocateCardUpdateBatchSize <= 0 || allocateCardUpdateBatchSize > 1000 { + t.Fatalf("allocateCardUpdateBatchSize = %d", allocateCardUpdateBatchSize) + } + if allocateDetailUpdateBatchSize <= 0 || allocateDetailUpdateBatchSize > 1000 { + t.Fatalf("allocateDetailUpdateBatchSize = %d", allocateDetailUpdateBatchSize) + } +} + +func TestBuildDetailUpdateStatementUsesBoundParameters(t *testing.T) { + task := &Task{Batch: Batch{BatchNo: "ALC-1", CreatedBy: 7}} + details := []*Detail{ + {ID: 11, DetailStatus: consts.BatchDetailStatusPending, Reason: "unsafe ' reason"}, + {ID: 12, DetailStatus: consts.BatchDetailStatusRunning, Reason: "second"}, + } + statement, args := buildDetailUpdateStatement(task, details, gtime.Now()) + if strings.Contains(statement, details[0].Reason) { + t.Fatal("detail reason was interpolated into SQL") + } + if !strings.Contains(statement, "CASE") || !strings.Contains(statement, "IN (?,?)") { + t.Fatalf("unexpected statement: %s", statement) + } + if len(args) != len(details)*7+9 { + t.Fatalf("argument count = %d, want %d", len(args), len(details)*7+9) + } +} + +func TestNormalizeAtomicParentResultRejectsPartialSuccess(t *testing.T) { + task := &Task{Details: []*Detail{ + {ICCID: "8986000000000000001"}, + {ICCID: "8986000000000000002", Reason: "目标客户不存在"}, + }} + + successCount, failCount := normalizeAtomicParentResult(task) + if successCount != 0 || failCount != len(task.Details) { + t.Fatalf("normalizeAtomicParentResult() = %d, %d; want 0, %d", successCount, failCount, len(task.Details)) + } + for _, detail := range task.Details { + if detail.Success() { + t.Fatalf("detail %s remained successful", detail.ICCID) + } + } + status, result, _ := parentBatchResult(successCount, failCount) + if status != consts.BatchAsyncStatusFailed || result != consts.BatchRunResultFailed { + t.Fatalf("parentBatchResult() = %d, %d; want failed, failed", status, result) + } +} + +func TestValidateDetailAllowsCurrentHolderReallocation(t *testing.T) { + task := &Task{ + Batch: Batch{ + CreatedDept: 100, + }, + TargetsByID: map[int64]*Target{ + 200: {DeptID: 200, ParentID: 100}, + }, + } + detail := &Detail{ + ICCID: "8986000000000000000", + TargetDeptID: 200, + Card: &Card{ + ICCID: "8986000000000000000", + DeptID: 100, + AllocationBatchNo: "ALC202608050001", + PackageStatus: consts.PackageStatusEffective, + }, + } + + validateDetail(task, detail) + if detail.Reason != "" { + t.Fatalf("expected current holder reallocation to pass, got %q", detail.Reason) + } +} + +func TestValidateDetailRejectsAncestorReallocationAfterCardMoved(t *testing.T) { + task := &Task{ + Batch: Batch{ + CreatedDept: 100, + }, + TargetsByID: map[int64]*Target{ + 300: {DeptID: 300, ParentID: 100}, + }, + } + detail := &Detail{ + ICCID: "8986000000000000000", + TargetDeptID: 300, + Card: &Card{ + ICCID: "8986000000000000000", + DeptID: 200, + AllocationBatchNo: "ALC202608050001", + PackageStatus: consts.PackageStatusEffective, + }, + } + + validateDetail(task, detail) + if detail.Reason == "" { + t.Fatal("expected ancestor reallocation to be rejected after card moved") + } +} + +func TestValidateTaskMarksWholeBatchFailed(t *testing.T) { + task := &Task{ + Details: []*Detail{ + {ICCID: "8986000000000000001"}, + {ICCID: "8986000000000000002", Reason: "目标客户不存在"}, + }, + } + + validateTask(task) + + for _, detail := range task.Details { + if detail.Success() { + t.Fatalf("detail %s remained successful after atomic failure", detail.ICCID) + } + } + if task.FailureReason == "" { + t.Fatal("expected batch failure reason") + } + status, result := batchResult(countResult(task)) + if status != consts.BatchAsyncStatusFailed || result != consts.BatchRunResultFailed { + t.Fatalf("batchResult() = %d, %d; want failed, failed", status, result) + } +} + +func TestValidateDetailRejectsChangedCardICCID(t *testing.T) { + task := &Task{Batch: Batch{CreatedDept: 100}, TargetsByID: map[int64]*Target{200: {DeptID: 200, ParentID: 100}}} + detail := &Detail{ICCID: "8986000000000000001", TargetDeptID: 200, Card: &Card{ICCID: "8986000000000000002", DeptID: 100, PackageStatus: consts.PackageStatusEffective}} + + validateDetail(task, detail) + if detail.Reason == "" { + t.Fatal("expected changed card ICCID to be rejected") + } +} diff --git a/internal/service/allocate/repository.go b/internal/service/allocate/repository.go new file mode 100644 index 0000000..d97bcc7 --- /dev/null +++ b/internal/service/allocate/repository.go @@ -0,0 +1,1016 @@ +package allocate + +import ( + "context" + "encoding/json" + "fmt" + "sort" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +type Repository struct { + db gdb.DB + results batchresult.Generator +} + +const ( + allocateCardQueryBatchSize = 500 + allocateCardUpdateBatchSize = 500 + allocateDetailUpdateBatchSize = 500 +) + +type allocateParentAggregate struct { + ChildCount int `orm:"child_count"` + FinishedCount int `orm:"finished_count"` + TotalCount int `orm:"total_count"` + ProcessedCount int `orm:"processed_count"` + SuccessCount int `orm:"success_count"` + FailCount int `orm:"fail_count"` + EarliestStartTime *gtime.Time `orm:"earliest_start_time"` +} + +func NewRepository() *Repository { + return &Repository{results: batchresult.New()} +} + +func (r *Repository) database() gdb.DB { + if r.db != nil { + return r.db + } + return g.DB() +} + +func (r *Repository) LoadMain(ctx context.Context, batchNo string) (*entity.BatchMain, error) { + batchNo = strings.TrimSpace(batchNo) + if batchNo == "" { + return nil, gerror.New("allocate batch number is required") + } + main := &entity.BatchMain{} + if err := r.database().Model(dao.BatchMain.Table()).Safe().Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Scan(main); err != nil { + return nil, gerror.Wrap(err, "load allocate batch main failed") + } + if main.Id == 0 { + return nil, gerror.New("allocate batch does not exist") + } + return main, nil +} + +func (r *Repository) Load(ctx context.Context, batchNo string) (*Task, error) { + batchNo = strings.TrimSpace(batchNo) + if batchNo == "" { + return nil, gerror.New("划拨批次号不能为空") + } + var main *entity.BatchMain + if err := r.database().Model(dao.BatchMain.Table()).Safe().Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeAllocate). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Scan(&main); err != nil { + return nil, gerror.Wrap(err, "查询划拨批次主表失败") + } + if main == nil || main.Id == 0 { + return nil, gerror.New("处理中划拨批次不存在") + } + + var batchDetails []*entity.BatchDetail + if err := r.database().Model(dao.BatchDetail.Table()).Safe().Ctx(ctx). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeAllocate). + OrderAsc(dao.BatchDetail.Columns().RowNo). + Scan(&batchDetails); err != nil { + return nil, gerror.Wrap(err, "查询划拨批次明细失败") + } + if len(batchDetails) == 0 { + return nil, gerror.New("划拨批次没有运行中明细") + } + + task := &Task{ + Batch: Batch{ + ID: main.Id, + BatchNo: main.BatchNo, + BatchType: main.BatchType, + ParentBatchNo: strings.TrimSpace(main.SourceBatchNo), + TenantID: main.TenantId, + TargetDeptID: targetDeptIDFromBatch(main), + CreatedDept: main.CreatedDept, + CreatedBy: main.CreatedBy, + StartTime: main.StartTime, + BatchInfo: main.BatchInfo, + }, + Details: make([]*Detail, 0, len(batchDetails)), + } + for _, batchDetail := range batchDetails { + if batchDetail == nil { + continue + } + detail := &Detail{OriginalSnapshot: batchDetail.Snapshot, + ID: batchDetail.Id, + RowNo: batchDetail.RowNo, + DetailStatus: batchDetail.DetailStatus, + ICCID: strings.TrimSpace(batchDetail.ItemId), + Reason: strings.TrimSpace(batchDetail.ErrorReason), + } + if detail.DetailStatus != consts.BatchDetailStatusPending && + detail.DetailStatus != consts.BatchDetailStatusRunning && + detail.Reason == "" { + detail.Reason = "批次明细已处理，不能重复执行" + } + if batchDetail.Snapshot != nil { + if value := firstSnapshot(batchDetail.Snapshot, "ICCID", "iccid"); value != "" { + detail.ICCID = value + } + detail.MSISDN = firstSnapshot(batchDetail.Snapshot, "MSISDN", "msisdn") + detail.IMSI = firstSnapshot(batchDetail.Snapshot, "IMSI", "imsi") + detail.TargetDeptID = batchDetail.Snapshot.Get("targetDeptId").Int64() + detail.TargetDeptKey = firstSnapshot(batchDetail.Snapshot, "targetDeptKey", "target_dept_key") + detail.TargetDeptName = firstSnapshot(batchDetail.Snapshot, "targetDeptName", "target_dept_name") + detail.Remark = firstSnapshot(batchDetail.Snapshot, "remark") + } + if detail.TargetDeptID <= 0 { + detail.TargetDeptID = task.Batch.TargetDeptID + } + task.Details = append(task.Details, detail) + } + if len(task.Details) == 0 { + return nil, gerror.New("划拨批次没有可读取明细") + } + if err := r.loadCards(ctx, task); err != nil { + return nil, err + } + if err := r.loadTargets(ctx, task); err != nil { + return nil, err + } + return task, nil +} + +func targetDeptIDFromBatch(main *entity.BatchMain) int64 { + if main == nil { + return 0 + } + if main.TargetDeptId > 0 { + return main.TargetDeptId + } + if main.BatchInfo != nil { + return main.BatchInfo.Get("targetDeptId").Int64() + } + return 0 +} + +func firstSnapshot(snapshot *gjson.Json, keys ...string) string { + if snapshot == nil { + return "" + } + for _, key := range keys { + if value := strings.TrimSpace(snapshot.Get(key).String()); value != "" { + return value + } + } + return "" +} + +func (r *Repository) loadCards(ctx context.Context, task *Task) error { + iccids := make([]string, 0, len(task.Details)) + for _, detail := range task.Details { + if detail != nil && strings.TrimSpace(detail.ICCID) != "" { + iccids = append(iccids, strings.TrimSpace(detail.ICCID)) + } + } + iccids = uniqueStrings(iccids) + if len(iccids) == 0 { + return nil + } + columns := dao.TSimCardInfo.Columns() + cards := make([]*entity.TSimCardInfo, 0, len(iccids)) + for start := 0; start < len(iccids); start += allocateCardQueryBatchSize { + end := min(start+allocateCardQueryBatchSize, len(iccids)) + var chunk []*entity.TSimCardInfo + if err := r.database().Model(dao.TSimCardInfo.Table()).Safe().Ctx(ctx). + Where(columns.TenantId, task.Batch.TenantID). + WhereIn(columns.Iccid, iccids[start:end]). + Scan(&chunk); err != nil { + return gerror.Wrapf(err, "查询SIM卡档案失败，范围:%d-%d", start+1, end) + } + cards = append(cards, chunk...) + } + byICCID := make(map[string]*entity.TSimCardInfo, len(cards)) + for _, card := range cards { + if card != nil { + byICCID[strings.TrimSpace(card.Iccid)] = card + } + } + for _, detail := range task.Details { + if detail == nil { + continue + } + card := byICCID[strings.TrimSpace(detail.ICCID)] + if card == nil { + continue + } + detail.Card = &Card{ + ID: card.Id, + ICCID: strings.TrimSpace(card.Iccid), + MSISDN: strings.TrimSpace(card.Msisdn), + IMSI: strings.TrimSpace(card.Imsi), + DeptID: card.DeptId, + AllocationBatchNo: strings.TrimSpace(card.AllocationBatchNo), + PackageStatus: card.PackageStatus, + BossCardStatus: card.BossCardStatus, + FlowPkgEndAt: card.FlowPkgEndAt, + } + } + return nil +} + +func (r *Repository) loadTargets(ctx context.Context, task *Task) error { + task.TargetsByID = make(map[int64]*Target) + task.TargetsByKey = make(map[string]*Target) + task.TargetsByName = make(map[string][]*Target) + if task.Batch.CreatedDept <= 0 { + return nil + } + var depts []*entity.SysDept + columns := dao.SysDept.Columns() + if err := r.database().Model(dao.SysDept.Table()).Safe().Ctx(ctx). + Fields(columns.DeptId, columns.ParentId, columns.DeptKey, columns.DeptName). + Where(columns.TenantId, task.Batch.TenantID). + Where(columns.ParentId, task.Batch.CreatedDept). + Where(columns.Status, "0"). + Where(columns.DeletedAt, nil). + Scan(&depts); err != nil { + return gerror.Wrap(err, "查询划拨目标客户失败") + } + for _, dept := range depts { + if dept == nil || dept.DeptId <= 0 { + continue + } + target := &Target{ + DeptID: dept.DeptId, + ParentID: dept.ParentId, + DeptKey: strings.TrimSpace(dept.DeptKey), + DeptName: strings.TrimSpace(dept.DeptName), + } + task.TargetsByID[target.DeptID] = target + if target.DeptKey != "" { + task.TargetsByKey[target.DeptKey] = target + } + if target.DeptName != "" { + task.TargetsByName[target.DeptName] = append(task.TargetsByName[target.DeptName], target) + } + } + return nil +} + +func (r *Repository) Commit(ctx context.Context, task *Task) error { + if task == nil { + return gerror.New("allocate task is nil") + } + + if _, failCount := countResult(task); failCount > 0 { + markAtomicFailure(task, "划拨批次存在失败明细，整批未执行") + } + successCount, failCount := countResult(task) + autoSplitChild := strings.TrimSpace(task.Batch.ParentBatchNo) != "" + var artifact batchresult.Artifact + var artifactErr error + if !autoSplitChild { + artifact, artifactErr = r.createResultFile(ctx, task) + } + if artifactErr != nil && !autoSplitChild { + g.Log().Warningf(ctx, "allocate result file generation failed, batchNo:%s, err:%+v", task.Batch.BatchNo, artifactErr) + } + + now := gtime.Now() + err := r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + if successCount > 0 && !autoSplitChild { + if err := r.updateCardOwners(ctx, tx, task, now); err != nil { + return err + } + } + if err := r.updateDetails(ctx, tx, task, now); err != nil { + return err + } + + var ossID, downloadID int64 + if !autoSplitChild && artifactErr == nil { + var err error + ossID, downloadID, err = r.persistArtifact(ctx, tx, task, artifact, successCount, failCount, now) + if err != nil { + return err + } + } + return r.finalizeBatch(ctx, tx, task, successCount, failCount, ossID, downloadID, now, !autoSplitChild && artifactErr != nil) + }) + if err != nil && artifactErr == nil && artifact.Path != "" { + batchresult.RemoveLocalFile(artifact) + } + if err != nil { + return gerror.Wrap(err, "commit allocate batch failed") + } + if autoSplitChild { + if err := r.AggregateParent(ctx, task.Batch.ParentBatchNo); err != nil { + return gerror.Wrap(err, "aggregate allocate parent batch failed") + } + } + return nil +} + +func (r *Repository) AggregateParent(ctx context.Context, parentBatchNo string) error { + parentBatchNo = strings.TrimSpace(parentBatchNo) + if parentBatchNo == "" { + return gerror.New("allocate parent batch number is required") + } + release := acquireAllocateParentLock(parentBatchNo) + defer release() + + maxRetries := allocateDeadlockMaxRetries(ctx) + for attempt := 0; ; attempt++ { + err := r.aggregateParentOnce(ctx, parentBatchNo) + if err == nil || !isAllocateDeadlock(err) || attempt >= maxRetries { + return err + } + if err := waitAllocateDeadlockRetry(ctx, attempt); err != nil { + return err + } + g.Log().Warningf(ctx, "allocate parent aggregation deadlock, retrying batchNo:%s attempt:%d/%d", parentBatchNo, attempt+1, maxRetries) + } +} + +func (r *Repository) aggregateParentOnce(ctx context.Context, parentBatchNo string) error { + parentBatchNo = strings.TrimSpace(parentBatchNo) + if parentBatchNo == "" { + return gerror.New("allocate parent batch number is required") + } + now := gtime.Now() + var committedTask *Task + var parentArtifact batchresult.Artifact + parentArtifactReady := false + err := r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + parent := &entity.BatchMain{} + mainColumns := dao.BatchMain.Columns() + if err := tx.Ctx(ctx).Model(dao.BatchMain.Table()).Where(mainColumns.BatchNo, parentBatchNo).Where(mainColumns.BatchType, consts.BatchTypeAllocate).LockUpdate().Scan(parent); err != nil { + return gerror.Wrap(err, "load allocate parent batch failed") + } + if parent.Id == 0 { + return gerror.New("allocate parent batch does not exist") + } + if !isAllocateParent(parent) { + return gerror.New("allocate parent batch metadata is invalid") + } + if parent.BatchAsyncStatus == consts.BatchAsyncStatusSuccess || parent.BatchAsyncStatus == consts.BatchAsyncStatusFailed || parent.BatchAsyncStatus == consts.BatchAsyncStatusPartialSuccess { + return nil + } + childColumns := dao.BatchMain.Columns() + aggregate := &allocateParentAggregate{} + finishedStatus := fmt.Sprintf("%s IN (%d, %d, %d)", childColumns.BatchAsyncStatus, consts.BatchAsyncStatusSuccess, consts.BatchAsyncStatusFailed, consts.BatchAsyncStatusPartialSuccess) + fields := fmt.Sprintf("COUNT(*) AS child_count, COALESCE(SUM(CASE WHEN %s THEN 1 ELSE 0 END), 0) AS finished_count, COALESCE(SUM(%s), 0) AS total_count, COALESCE(SUM(CASE WHEN %s THEN %s ELSE 0 END), 0) AS processed_count, COALESCE(SUM(%s), 0) AS success_count, COALESCE(SUM(CASE WHEN %s THEN %s - %s ELSE 0 END), 0) AS fail_count, MIN(%s) AS earliest_start_time", finishedStatus, childColumns.TotalCount, finishedStatus, childColumns.TotalCount, childColumns.SuccessCount, finishedStatus, childColumns.TotalCount, childColumns.SuccessCount, childColumns.StartTime) + if err := tx.Ctx(ctx).Model(dao.BatchMain.Table()).Fields(fields).Where(childColumns.BatchType, consts.BatchTypeAllocate).Where(childColumns.SourceBatchNo, parentBatchNo).Scan(aggregate); err != nil { + return gerror.Wrap(err, "aggregate allocate child batches failed") + } + info := mergeAllocateParentInfo(parent.BatchInfo, aggregate) + splitCompleted := parent.BatchInfo != nil && parent.BatchInfo.Get("splitCompleted").Bool() + allChildrenFinished := splitCompleted && aggregate.ChildCount > 0 && aggregate.ChildCount == aggregate.FinishedCount + if !allChildrenFinished { + startTime := resolveAllocateParentStartTime(parent.StartTime, aggregate.EarliestStartTime, now) + infoData := info.Map() + infoData["successCount"] = 0 + infoData["failCount"] = 0 + data := do.BatchMain{StartTime: startTime, TotalCount: aggregate.TotalCount, ValidCount: 0, SuccessCount: 0, FailCount: 0, BatchInfo: gjson.New(infoData), BatchAsyncStatus: consts.BatchAsyncStatusProcessing, RunResult: consts.BatchRunResultRunning, UpdatedBy: parent.CreatedBy, UpdatedAt: now} + _, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()).Where(mainColumns.Id, parent.Id).WhereIn(mainColumns.BatchAsyncStatus, []int{consts.BatchAsyncStatusPending, consts.BatchAsyncStatusProcessing}).Data(data).Update() + if err != nil { + return gerror.Wrap(err, "update allocate parent progress failed") + } + return nil + } + task, err := r.loadParentTaskTx(ctx, tx, parent) + if err != nil { + return err + } + if len(task.Details) != aggregate.TotalCount { + return gerror.Newf("allocate parent detail count mismatch, expected:%d actual:%d", aggregate.TotalCount, len(task.Details)) + } + if aggregate.FailCount > 0 { + markAtomicFailure(task, "至少一个划拨子批次校验失败，整批未执行") + } else { + validateTask(task) + if _, failures := countResult(task); failures > 0 { + markAtomicFailure(task, "划拨明细复核失败，整批未执行") + } + } + successCount, failCount := normalizeAtomicParentResult(task) + if successCount > 0 { + if err := r.updateCardOwners(ctx, tx, task, now); err != nil { + return err + } + } + if err := r.updateParentDetails(ctx, tx, task, now); err != nil { + return err + } + artifact, artifactErr := r.createResultFile(ctx, task) + if artifactErr == nil { + parentArtifact, parentArtifactReady = artifact, true + } + if artifactErr != nil { + g.Log().Warningf(ctx, "allocate parent result file generation failed, batchNo:%s, err:%+v", parentBatchNo, artifactErr) + } + var ossID, downloadID int64 + if artifactErr == nil { + ossID, downloadID, err = r.persistArtifact(ctx, tx, task, artifact, successCount, failCount, now) + if err != nil { + batchresult.RemoveLocalFile(artifact) + return err + } + } + status, runResult, failReason := parentBatchResult(successCount, failCount) + if failCount > 0 { + failReason = coalesce(task.FailureReason, failReason) + } + finalInfo := info.Map() + finalInfo["processedCount"] = len(task.Details) + finalInfo["successCount"] = successCount + finalInfo["failCount"] = failCount + startTime := resolveAllocateParentStartTime(parent.StartTime, aggregate.EarliestStartTime, now) + data := do.BatchMain{StartTime: startTime, BatchAsyncStatus: status, RunResult: runResult, TotalCount: len(task.Details), ValidCount: successCount, SuccessCount: successCount, FailCount: failCount, FailReason: failReason, BatchInfo: mergeBatchInfo(gjson.New(finalInfo), ossID, downloadID, artifactErr != nil), EndTime: now, UpdatedBy: parent.CreatedBy, UpdatedAt: now} + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()).Where(mainColumns.Id, parent.Id).WhereIn(mainColumns.BatchAsyncStatus, []int{consts.BatchAsyncStatusPending, consts.BatchAsyncStatusProcessing}).Data(data).Update() + if err != nil { + if artifactErr == nil { + batchresult.RemoveLocalFile(artifact) + } + return gerror.Wrap(err, "finalize allocate parent batch failed") + } + affected, err := result.RowsAffected() + if err != nil { + if artifactErr == nil { + batchresult.RemoveLocalFile(artifact) + } + return err + } + if affected != 1 { + if artifactErr == nil { + batchresult.RemoveLocalFile(artifact) + } + return gerror.New("allocate parent batch is no longer processing") + } + if successCount > 0 { + committedTask = task + } + return nil + }) + if err != nil { + if parentArtifactReady { + batchresult.RemoveLocalFile(parentArtifact) + } + return err + } + if committedTask != nil { + publishPackageCustomerSync(ctx, committedTask) + } + return nil +} + +func resolveAllocateParentStartTime(parentStartTime, earliestChildStartTime, fallback *gtime.Time) *gtime.Time { + if parentStartTime != nil && !parentStartTime.IsZero() { + return parentStartTime + } + if earliestChildStartTime != nil && !earliestChildStartTime.IsZero() { + return earliestChildStartTime + } + return fallback +} + +func mergeAllocateParentInfo(info *gjson.Json, aggregate *allocateParentAggregate) *gjson.Json { + data := make(map[string]any) + if info != nil { + for key, value := range info.Map() { + data[key] = value + } + } + completed := 0 + totalChunks := 0 + if aggregate != nil { + completed = aggregate.FinishedCount + } + if info != nil { + totalChunks = info.Get("chunkCount").Int() + } + if totalChunks == 0 && aggregate != nil { + totalChunks = aggregate.ChildCount + } + data["completedChunkCount"] = completed + data["chunkCount"] = totalChunks + if aggregate != nil { + data["processedCount"] = aggregate.ProcessedCount + data["successCount"] = aggregate.SuccessCount + data["failCount"] = aggregate.FailCount + } + return gjson.New(data) +} + +func parentBatchResult(successCount, failCount int) (int, int, string) { + if failCount > 0 || successCount == 0 { + return consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed, "allocate child batches failed" + } + return consts.BatchAsyncStatusSuccess, consts.BatchRunResultSuccess, "" +} + +func normalizeAtomicParentResult(task *Task) (int, int) { + successCount, failCount := countResult(task) + if failCount == 0 { + return successCount, 0 + } + if task == nil { + return 0, 0 + } + task.FailureReason = coalesce(task.FailureReason, "划拨批次存在失败明细，整批未执行") + markAtomicFailure(task, task.FailureReason) + return 0, len(task.Details) +} + +func (r *Repository) updateDetails(ctx context.Context, tx gdb.TX, task *Task, now *gtime.Time) error { + details := make([]*Detail, 0, len(task.Details)) + for _, detail := range task.Details { + if detail == nil || + (detail.DetailStatus != consts.BatchDetailStatusPending && + detail.DetailStatus != consts.BatchDetailStatusRunning) { + continue + } + details = append(details, detail) + } + for start := 0; start < len(details); start += allocateDetailUpdateBatchSize { + end := min(start+allocateDetailUpdateBatchSize, len(details)) + if err := r.updateDetailChunk(ctx, tx, task, details[start:end], now); err != nil { + return gerror.Wrapf(err, "批量更新划拨明细失败，范围:%d-%d", start+1, end) + } + } + return nil +} + +func (r *Repository) updateParentDetails(ctx context.Context, tx gdb.TX, task *Task, now *gtime.Time) error { + detailsByBatch := make(map[string][]*Detail) + for _, detail := range task.Details { + if detail == nil { + continue + } + batchNo := strings.TrimSpace(detail.BatchNo) + if batchNo == "" { + return gerror.New("allocate parent detail is missing child batch number") + } + detailsByBatch[batchNo] = append(detailsByBatch[batchNo], detail) + } + batchNos := make([]string, 0, len(detailsByBatch)) + for batchNo := range detailsByBatch { + batchNos = append(batchNos, batchNo) + } + sort.Strings(batchNos) + for _, batchNo := range batchNos { + details := detailsByBatch[batchNo] + sort.SliceStable(details, func(i, j int) bool { + if details[i].ID != details[j].ID { + return details[i].ID < details[j].ID + } + return details[i].RowNo < details[j].RowNo + }) + batchTask := *task + batchTask.Batch.BatchNo = batchNo + for start := 0; start < len(details); start += allocateDetailUpdateBatchSize { + end := min(start+allocateDetailUpdateBatchSize, len(details)) + if err := r.updateDetailChunk(ctx, tx, &batchTask, details[start:end], now); err != nil { + return gerror.Wrapf(err, "update allocate child details failed, batchNo:%s, range:%d-%d", batchNo, start+1, end) + } + } + } + return nil +} + +func (r *Repository) updateDetailChunk(ctx context.Context, tx gdb.TX, task *Task, details []*Detail, now *gtime.Time) error { + if len(details) == 0 { + return nil + } + statement, args := buildDetailUpdateStatement(task, details, now) + result, err := tx.Ctx(ctx).Exec(statement, args...) + if err != nil { + return err + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != int64(len(details)) { + // MySQL 默认返回实际修改行数。子批次已写入相同结果时可能为零， + // 必须在同一事务核对最终结果，不能据此放过缺失或越界明细。 + ids := make([]int64, 0, len(details)) + for _, detail := range details { + ids = append(ids, detail.ID) + } + var persisted []*entity.BatchDetail + columns := dao.BatchDetail.Columns() + if err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(columns.BatchNo, task.Batch.BatchNo). + Where(columns.BatchType, consts.BatchTypeAllocate). + Where(columns.TenantId, task.Batch.TenantID). + WhereIn(columns.Id, ids).Scan(&persisted); err != nil { + return gerror.Wrap(err, "核对划拨明细更新结果失败") + } + if err := validatePersistedDetailResults(task, details, persisted); err != nil { + return gerror.Wrapf(err, "划拨明细更新未确认，预期:%d，实际修改:%d", len(details), affected) + } + } + return nil +} + +func validatePersistedDetailResults(task *Task, details []*Detail, persisted []*entity.BatchDetail) error { + if len(persisted) != len(details) { + return gerror.New("划拨明细缺失或不在当前批次租户范围内") + } + byID := make(map[int64]*entity.BatchDetail, len(persisted)) + for _, row := range persisted { + if row == nil || row.BatchNo != task.Batch.BatchNo || row.BatchType != consts.BatchTypeAllocate || row.TenantId != task.Batch.TenantID { + return gerror.New("划拨明细不在当前批次租户范围内") + } + byID[row.Id] = row + } + for _, detail := range details { + row := byID[detail.ID] + status := consts.BatchDetailStatusFailed + if detail.Success() { + status = consts.BatchDetailStatusSuccess + } + if row == nil || row.DetailStatus != status || row.ErrorReason != detail.Reason || row.UpdatedBy != task.Batch.CreatedBy || row.Snapshot == nil || !sameDetailSnapshot(row.Snapshot, detail.Snapshot(task.Batch)) { + return gerror.Newf("划拨明细最终结果不一致，明细ID:%d", detail.ID) + } + delete(byID, detail.ID) + } + return nil +} + +func sameDetailSnapshot(actual, expected *gjson.Json) bool { + if actual == nil || expected == nil { + return actual == expected + } + actualJSON, actualErr := json.Marshal(actual.Map()) + expectedJSON, expectedErr := json.Marshal(expected.Map()) + return actualErr == nil && expectedErr == nil && string(actualJSON) == string(expectedJSON) +} + +func buildDetailUpdateStatement(task *Task, details []*Detail, now *gtime.Time) (string, []any) { + columns := dao.BatchDetail.Columns() + var builder strings.Builder + fmt.Fprintf(&builder, "UPDATE %s SET %s = CASE %s ", dao.BatchDetail.Table(), columns.DetailStatus, columns.Id) + args := make([]any, 0, len(details)*7+8) + for _, detail := range details { + status := consts.BatchDetailStatusFailed + if detail.Success() { + status = consts.BatchDetailStatusSuccess + } + builder.WriteString("WHEN ? THEN ? ") + args = append(args, detail.ID, status) + } + fmt.Fprintf(&builder, "ELSE %s END, %s = CASE %s ", columns.DetailStatus, columns.ErrorReason, columns.Id) + for _, detail := range details { + builder.WriteString("WHEN ? THEN ? ") + args = append(args, detail.ID, detail.Reason) + } + fmt.Fprintf(&builder, "ELSE %s END, %s = CASE %s ", columns.ErrorReason, columns.Snapshot, columns.Id) + for _, detail := range details { + builder.WriteString("WHEN ? THEN ? ") + args = append(args, detail.ID, detail.Snapshot(task.Batch).String()) + } + fmt.Fprintf(&builder, "ELSE %s END, %s = ?, %s = ? ", columns.Snapshot, columns.UpdatedBy, columns.UpdatedAt) + args = append(args, task.Batch.CreatedBy, now) + fmt.Fprintf(&builder, "WHERE %s = ? AND %s = ? AND %s = ? AND %s IN (?, ?, ?, ?) AND %s IN (%s)", + columns.BatchNo, + columns.BatchType, + columns.TenantId, + columns.DetailStatus, + columns.Id, + strings.TrimSuffix(strings.Repeat("?,", len(details)), ","), + ) + args = append(args, + task.Batch.BatchNo, + consts.BatchTypeAllocate, + task.Batch.TenantID, + consts.BatchDetailStatusPending, + consts.BatchDetailStatusRunning, + consts.BatchDetailStatusSuccess, + consts.BatchDetailStatusFailed, + ) + for _, detail := range details { + args = append(args, detail.ID) + } + return builder.String(), args +} + +func (r *Repository) updateCardOwners(ctx context.Context, tx gdb.TX, task *Task, now *gtime.Time) error { + groups := make(map[int64][]int64) + totalCards := 0 + for _, detail := range task.Details { + if detail == nil || !detail.Success() || detail.Card == nil || detail.Target == nil { + continue + } + groups[detail.Target.DeptID] = append(groups[detail.Target.DeptID], detail.Card.ID) + totalCards++ + } + successCount, _ := countResult(task) + if totalCards != successCount { + return gerror.Newf("划拨成功明细缺少卡或目标客户快照，预期:%d，实际:%d", successCount, totalCards) + } + for _, cardIDs := range groups { + sort.Slice(cardIDs, func(i, j int) bool { return cardIDs[i] < cardIDs[j] }) + } + targetDeptIDs := make([]int64, 0, len(groups)) + for targetDeptID := range groups { + targetDeptIDs = append(targetDeptIDs, targetDeptID) + } + sort.Slice(targetDeptIDs, func(i, j int) bool { return targetDeptIDs[i] < targetDeptIDs[j] }) + for _, targetDeptID := range targetDeptIDs { + cardIDs := groups[targetDeptID] + for start := 0; start < len(cardIDs); start += allocateCardUpdateBatchSize { + end := min(start+allocateCardUpdateBatchSize, len(cardIDs)) + ids := cardIDs[start:end] + result, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()).Safe(). + Where(dao.TSimCardInfo.Columns().TenantId, task.Batch.TenantID). + Where(dao.TSimCardInfo.Columns().DeptId, task.Batch.CreatedDept). + WhereIn(dao.TSimCardInfo.Columns().Id, ids). + Data(do.TSimCardInfo{ + DeptId: targetDeptID, + OutboundBatchNo: task.Batch.BatchNo, + AllocationBatchNo: task.Batch.BatchNo, + UpdatedBy: task.Batch.CreatedBy, + UpdatedAt: now, + }).Update() + if err != nil { + return gerror.Wrapf(err, "批量更新SIM卡划拨归属失败，目标客户:%d，范围:%d-%d", targetDeptID, start+1, end) + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != int64(len(ids)) { + return gerror.Newf("卡归属已变化，划拨批次整批回滚，目标客户:%d，预期:%d，实际:%d", targetDeptID, len(ids), affected) + } + } + } + return nil +} + +func (r *Repository) commitLegacy(ctx context.Context, task *Task) error { + if task == nil { + return gerror.New("划拨任务不能为空") + } + now := gtime.Now() + if err := r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + for _, detail := range task.Details { + if detail == nil { + continue + } + if detail.Success() { + if err := r.updateCardOwner(ctx, tx, task, detail, now); err != nil { + return err + } + } + if err := r.updateDetail(ctx, tx, task, detail, now); err != nil { + return err + } + } + return nil + }); err != nil { + return gerror.Wrap(err, "保存划拨明细结果失败") + } + + successCount, failCount := countResult(task) + artifact, artifactErr := r.createResultFile(ctx, task) + if artifactErr != nil { + g.Log().Warningf(ctx, "allocate result file generation failed, batchNo:%s, err:%+v", task.Batch.BatchNo, artifactErr) + } + + err := r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + var ossID, downloadID int64 + if artifactErr == nil { + var err error + ossID, downloadID, err = r.persistArtifact(ctx, tx, task, artifact, successCount, failCount, now) + if err != nil { + batchresult.RemoveLocalFile(artifact) + return err + } + } + return r.finalizeBatch(ctx, tx, task, successCount, failCount, ossID, downloadID, now, artifactErr != nil) + }) + return gerror.Wrap(err, "汇总划拨批次失败") +} + +func (r *Repository) updateCardOwner(ctx context.Context, tx gdb.TX, task *Task, detail *Detail, now *gtime.Time) error { + if detail.Card == nil || detail.Target == nil { + return gerror.New("划拨明细缺少卡或目标客户档案") + } + result, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()).Safe(). + Where(dao.TSimCardInfo.Columns().Id, detail.Card.ID). + Where(dao.TSimCardInfo.Columns().TenantId, task.Batch.TenantID). + Where(dao.TSimCardInfo.Columns().DeptId, task.Batch.CreatedDept). + Data(do.TSimCardInfo{ + DeptId: detail.Target.DeptID, + OutboundBatchNo: task.Batch.BatchNo, + AllocationBatchNo: task.Batch.BatchNo, + UpdatedBy: task.Batch.CreatedBy, + UpdatedAt: now, + }).Update() + if err != nil { + return gerror.Wrap(err, "更新SIM卡划拨归属失败") + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.New("卡归属已变化，划拨批次整批回滚") + } + return nil +} + +func (r *Repository) updateDetail(ctx context.Context, tx gdb.TX, task *Task, detail *Detail, now *gtime.Time) error { + if detail == nil || + (detail.DetailStatus != consts.BatchDetailStatusPending && + detail.DetailStatus != consts.BatchDetailStatusRunning) { + return nil + } + status := consts.BatchDetailStatusFailed + if detail.Success() { + status = consts.BatchDetailStatusSuccess + } + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeAllocate). + WhereIn(dao.BatchDetail.Columns().DetailStatus, []int{ + consts.BatchDetailStatusPending, + consts.BatchDetailStatusRunning, + }). + Data(do.BatchDetail{ + DetailStatus: status, + ErrorReason: detail.Reason, + Snapshot: detail.Snapshot(task.Batch), + UpdatedBy: task.Batch.CreatedBy, + UpdatedAt: now, + }).Update() + if err != nil { + return gerror.Wrap(err, "更新划拨批次明细失败") + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.Newf("划拨批次明细%d已不是运行中状态", detail.ID) + } + return nil +} + +func (r *Repository) createResultFile(ctx context.Context, task *Task) (batchresult.Artifact, error) { + if r.results == nil { + return batchresult.Artifact{}, gerror.New("result generator is not configured") + } + return r.results.Create(ctx, batchresult.Request{ + BatchNo: task.Batch.BatchNo, + BatchType: task.Batch.BatchType, + TenantID: task.Batch.TenantID, + Rows: task.ResultRows(), + }) +} + +func (r *Repository) persistArtifact(ctx context.Context, tx gdb.TX, task *Task, artifact batchresult.Artifact, successCount, failCount int, now *gtime.Time) (int64, int64, error) { + ossID, err := tx.Ctx(ctx).Model(dao.SysOss.Table()).Data(do.SysOss{ + TenantId: task.Batch.TenantID, + FileName: artifact.FileName, + OriginalName: artifact.OriginalName, + FileSuffix: artifact.FileSuffix, + Path: artifact.Path, + Url: artifact.URL, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + Service: "local", + Md5: artifact.MD5, + FileSize: artifact.Size, + }).InsertAndGetId() + if err != nil { + return 0, 0, gerror.Wrap(err, "写入划拨结果文件OSS失败") + } + downloadID, err := tx.Ctx(ctx).Model(dao.TResourceBatchDownload.Table()).Data(do.TResourceBatchDownload{ + BatchNo: task.Batch.BatchNo, + TaskName: batchresult.TaskName(task.Batch.BatchType), + ResourceType: consts.ResourceTypeExcel, + ExportScene: consts.BatchTypeAllocate, + TotalCount: len(task.Details), + SuccessCount: successCount, + FailCount: failCount, + DownloadStatus: consts.DownloadStatusGenerated, + OssId: ossID, + FileName: artifact.FileName, + FileExt: artifact.FileSuffix, + FileUrl: artifact.URL, + FilePath: artifact.Path, + FileSize: artifact.Size, + FileMd5: artifact.MD5, + DownloadCount: 0, + MaxDownloadCount: 0, + DeptId: task.Batch.CreatedDept, + StartTime: task.Batch.StartTime, + FinishTime: now, + TenantId: task.Batch.TenantID, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + }).InsertAndGetId() + if err != nil { + return 0, 0, gerror.Wrap(err, "写入划拨结果下载记录失败") + } + return ossID, downloadID, nil +} + +func (r *Repository) finalizeBatch(ctx context.Context, tx gdb.TX, task *Task, successCount, failCount int, ossID, downloadID int64, now *gtime.Time, resultFileFailed bool) error { + batchAsyncStatus, runResult := batchResult(successCount, failCount) + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeAllocate). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Data(do.BatchMain{ + BatchAsyncStatus: batchAsyncStatus, + RunResult: runResult, + TotalCount: len(task.Details), + ValidCount: successCount, + SuccessCount: successCount, + FailCount: failCount, + FailReason: task.FailureReason, + BatchInfo: mergeBatchInfo(task.Batch.BatchInfo, ossID, downloadID, resultFileFailed), + EndTime: now, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "更新划拨批次主表失败") + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.New("划拨批次已不是处理中状态") + } + return nil +} + +func mergeBatchInfo(info *gjson.Json, ossID, downloadID int64, resultFileFailed bool) *gjson.Json { + data := make(map[string]any) + if info != nil { + for key, value := range info.Map() { + data[key] = value + } + } + if ossID > 0 { + data["resultFileOssId"] = ossID + } + if resultFileFailed { + data["resultFile"] = map[string]any{"status": "failed"} + } else if downloadID > 0 { + data["resultDownloadId"] = downloadID + data["resultFile"] = map[string]any{"status": "generated"} + } else { + data["resultFile"] = map[string]any{"status": "skipped"} + } + return gjson.New(data) +} + +func batchResult(successCount, failCount int) (int, int) { + if failCount > 0 || successCount == 0 { + return consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed + } + return consts.BatchAsyncStatusSuccess, consts.BatchRunResultSuccess +} + +func uniqueStrings(values []string) []string { + seen := make(map[string]struct{}, len(values)) + result := make([]string, 0, len(values)) + for _, value := range values { + value = strings.TrimSpace(value) + if value == "" { + continue + } + if _, exists := seen[value]; exists { + continue + } + seen[value] = struct{}{} + result = append(result, value) + } + return result +} diff --git a/internal/service/allocate/types.go b/internal/service/allocate/types.go new file mode 100644 index 0000000..cb423fa --- /dev/null +++ b/internal/service/allocate/types.go @@ -0,0 +1,162 @@ +package allocate + +import ( + "strings" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +type Batch struct { + ID int64 + BatchNo string + BatchType int + ParentBatchNo string + TenantID string + TargetDeptID int64 + CreatedDept int64 + CreatedBy int64 + StartTime *gtime.Time + BatchInfo *gjson.Json +} + +type Card struct { + ID int64 + ICCID string + MSISDN string + IMSI string + DeptID int64 + AllocationBatchNo string + PackageStatus int + BossCardStatus int + FlowPkgEndAt *gtime.Time +} + +type Target struct { + DeptID int64 + ParentID int64 + DeptKey string + DeptName string +} + +type Detail struct { + OriginalSnapshot *gjson.Json + ID int64 + BatchNo string + RowNo int + DetailStatus int + ICCID string + MSISDN string + IMSI string + TargetDeptID int64 + TargetDeptKey string + TargetDeptName string + Remark string + Card *Card + Target *Target + Reason string +} + +type Task struct { + Batch Batch + Details []*Detail + TargetsByID map[int64]*Target + TargetsByKey map[string]*Target + TargetsByName map[string][]*Target + FailureReason string +} + +func (d *Detail) Success() bool { + return d != nil && strings.TrimSpace(d.Reason) == "" +} + +func (d *Detail) DisplayCode() string { + if d == nil { + return "" + } + if d.Card != nil && strings.TrimSpace(d.Card.ICCID) != "" { + return strings.TrimSpace(d.Card.ICCID) + } + return strings.TrimSpace(d.ICCID) +} + +func (d *Detail) Snapshot(batch Batch) *gjson.Json { + if d == nil { + return gjson.New(nil) + } + data := map[string]any{} + if d.OriginalSnapshot != nil { + for k, v := range d.OriginalSnapshot.Map() { + data[k] = v + } + } + sourceDeptID := batch.CreatedDept + if d.OriginalSnapshot != nil && d.OriginalSnapshot.Get("sourceDeptId").Int64() > 0 { + sourceDeptID = d.OriginalSnapshot.Get("sourceDeptId").Int64() + } + current := map[string]any{ + "rowNo": d.RowNo, + "ICCID": strings.TrimSpace(d.ICCID), + "MSISDN": strings.TrimSpace(d.MSISDN), + "IMSI": strings.TrimSpace(d.IMSI), + "sourceDeptId": sourceDeptID, + "targetDeptId": d.TargetDeptID, + "targetDeptKey": strings.TrimSpace(d.TargetDeptKey), + "targetDeptName": strings.TrimSpace(d.TargetDeptName), + "remark": strings.TrimSpace(d.Remark), + "errorReason": strings.TrimSpace(d.Reason), + } + for k, v := range current { + data[k] = v + } + if d.Card != nil { + data["cardId"] = d.Card.ID + data["cardIccid"] = d.Card.ICCID + data["cardMsisdn"] = d.Card.MSISDN + data["cardImsi"] = d.Card.IMSI + data["cardDeptIdBefore"] = d.Card.DeptID + data["allocationBatchNoBefore"] = strings.TrimSpace(d.Card.AllocationBatchNo) + data["packageStatus"] = d.Card.PackageStatus + data["bossCardStatus"] = d.Card.BossCardStatus + if d.Card.FlowPkgEndAt != nil { + data["flowPkgEndAt"] = d.Card.FlowPkgEndAt.String() + } + } + if d.Target != nil { + data["targetDeptId"] = d.Target.DeptID + if strings.TrimSpace(d.TargetDeptKey) == "" { + data["targetDeptKey"] = d.Target.DeptKey + } + if strings.TrimSpace(d.TargetDeptName) == "" { + data["targetDeptName"] = d.Target.DeptName + } + } + return gjson.New(data) +} + +func (t *Task) ResultRows() []batchresult.Row { + rows := make([]batchresult.Row, 0, len(t.Details)) + for _, detail := range t.Details { + if detail == nil { + continue + } + targetName := strings.TrimSpace(detail.TargetDeptName) + if targetName == "" && detail.Target != nil && strings.TrimSpace(detail.Target.DeptName) != "" { + targetName = strings.TrimSpace(detail.Target.DeptName) + } + rows = append(rows, batchresult.Row{ + RowNo: detail.RowNo, + ObjectCode: detail.DisplayCode(), + ICCID: strings.TrimSpace(detail.ICCID), + MSISDN: strings.TrimSpace(detail.MSISDN), + IMSI: strings.TrimSpace(detail.IMSI), + TargetDeptName: targetName, + Operation: "卡划拨", + Success: detail.Success(), + Reason: strings.TrimSpace(detail.Reason), + }) + } + return rows +} diff --git a/internal/service/allocate/unit_snapshot_test.go b/internal/service/allocate/unit_snapshot_test.go new file mode 100644 index 0000000..55830d4 --- /dev/null +++ b/internal/service/allocate/unit_snapshot_test.go @@ -0,0 +1,34 @@ +package allocate + +import ( + "github.com/gogf/gf/v2/encoding/gjson" + "testing" +) + +func TestUnitSnapshotSurvivesRenameAndWriteback(t *testing.T) { + detail := &Detail{OriginalSnapshot: gjson.New(map[string]any{"sourceDeptName": "划拨旧名称", "sourceDeptKey": "source", "auditMarker": "keep"}), TargetDeptID: 2, TargetDeptName: "接收旧名称", TargetDeptKey: "target", Target: &Target{DeptID: 2, DeptName: "接收新名称", DeptKey: "new"}} + info := detail.Snapshot(Batch{CreatedDept: 1}) + if info.Get("sourceDeptName").String() != "划拨旧名称" || info.Get("targetDeptName").String() != "接收旧名称" || info.Get("targetDeptKey").String() != "target" || info.Get("auditMarker").String() != "keep" { + t.Fatal(info) + } + task := &Task{Details: []*Detail{detail}} + if task.ResultRows()[0].TargetDeptName != "接收旧名称" { + t.Fatal("结果文件覆盖历史名称") + } + parent := mergeAllocateParentInfo(gjson.New(map[string]any{"sourceDeptName": "划拨旧名称", "targetDeptName": "接收旧名称"}), nil) + if parent.Get("sourceDeptName").String() != "划拨旧名称" { + t.Fatal(parent) + } +} + +func TestUnitTargetUsesStableIDAfterRename(t *testing.T) { + target := &Target{DeptID: 2, DeptName: "接收单位新名称", DeptKey: "new-key"} + detail := &Detail{OriginalSnapshot: gjson.New(map[string]any{"sourceDeptId": 9}), TargetDeptID: 2, TargetDeptName: "接收单位旧名称"} + task := &Task{TargetsByID: map[int64]*Target{2: target}} + if resolveTarget(task, detail) != target { + t.Fatal("单位改名后未优先使用稳定 ID") + } + if detail.Snapshot(Batch{CreatedDept: 1}).Get("sourceDeptId").Int64() != 9 { + t.Fatal("执行写回覆盖原划拨单位 ID") + } +} diff --git a/internal/service/balance/alipay.go b/internal/service/balance/alipay.go new file mode 100644 index 0000000..ac35324 --- /dev/null +++ b/internal/service/balance/alipay.go @@ -0,0 +1,137 @@ +package balance + +import ( + "context" + "errors" + "net/http" + "strings" + "sync" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/smartwalle/alipay/v3" +) + +type alipayConfig struct { + AppID string `yaml:"appId"` + PrivateKey string `yaml:"privateKey"` + AlipayPublicKey string `yaml:"alipayPublicKey"` + IsProduction bool `yaml:"isProduction"` +} + +type alipayTradeQueryer struct { + once sync.Once + client *alipay.Client + clientErr error +} + +func newAlipayTradeQueryer() *alipayTradeQueryer { + return &alipayTradeQueryer{} +} + +func (q *alipayTradeQueryer) Query(ctx context.Context, outTradeNo string) (*TradeQueryResult, error) { + client, err := q.getClient(ctx) + if err != nil { + return nil, err + } + rsp, err := q.tradeQueryWithRetry(ctx, client, alipay.TradeQuery{OutTradeNo: outTradeNo}) + if err != nil { + if isAlipayTradeNotFound(err.Error()) { + return &TradeQueryResult{OutTradeNo: outTradeNo, TradeStatus: string(alipay.TradeStatusWaitBuyerPay)}, nil + } + return nil, gerror.Wrap(err, "query Alipay trade failed") + } + if rsp == nil { + return nil, gerror.New("Alipay trade query returned no result") + } + if rsp.IsFailure() { + if isAlipayTradeNotFound(string(rsp.Code), rsp.SubCode, rsp.SubMsg) { + return &TradeQueryResult{OutTradeNo: outTradeNo, TradeStatus: string(alipay.TradeStatusWaitBuyerPay)}, nil + } + return nil, gerror.Newf("Alipay trade query failed: code=%s, subCode=%s, subMsg=%s", rsp.Code, rsp.SubCode, rsp.SubMsg) + } + return &TradeQueryResult{ + OutTradeNo: rsp.OutTradeNo, + TradeNo: rsp.TradeNo, + TradeStatus: string(rsp.TradeStatus), + TotalAmount: rsp.TotalAmount, + BuyerLogonID: rsp.BuyerLogonId, + }, nil +} + +func (q *alipayTradeQueryer) getClient(ctx context.Context) (*alipay.Client, error) { + q.once.Do(func() { + cfg := alipayConfig{} + if err := g.Cfg().MustGet(ctx, "alipay").Scan(&cfg); err != nil { + q.clientErr = gerror.Wrap(err, "read Alipay config failed") + return + } + if strings.TrimSpace(cfg.AppID) == "" || strings.TrimSpace(cfg.PrivateKey) == "" || strings.TrimSpace(cfg.AlipayPublicKey) == "" { + q.clientErr = gerror.New("Alipay credentials are not configured") + return + } + q.client, q.clientErr = alipay.New(cfg.AppID, cfg.PrivateKey, cfg.IsProduction, alipay.WithHTTPClient(&http.Client{Timeout: 10 * time.Second})) + if q.clientErr != nil { + q.clientErr = gerror.Wrap(q.clientErr, "initialize Alipay client failed") + return + } + if err := q.client.LoadAliPayPublicKey(cfg.AlipayPublicKey); err != nil { + q.clientErr = gerror.Wrap(err, "load Alipay public key failed") + } + }) + if q.clientErr != nil { + return nil, q.clientErr + } + return q.client, nil +} + +func (q *alipayTradeQueryer) tradeQueryWithRetry(ctx context.Context, client *alipay.Client, query alipay.TradeQuery) (*alipay.TradeQueryRsp, error) { + var err error + for attempt := 1; attempt <= 3; attempt++ { + rsp, queryErr := client.TradeQuery(ctx, query) + if queryErr == nil { + return rsp, nil + } + err = queryErr + if !isRetryableAlipayError(err) || attempt == 3 { + return nil, err + } + if err = sleepWithContext(ctx, time.Duration(attempt)*300*time.Millisecond); err != nil { + return nil, err + } + } + return nil, err +} + +func isRetryableAlipayError(err error) bool { + if err == nil { + return false + } + if errors.Is(err, context.DeadlineExceeded) { + return true + } + msg := strings.ToLower(err.Error()) + for _, marker := range []string{"invalid character '<'", "unexpected end of json", "bad response", "eof", "timeout", "connection reset", "connection refused", "tls handshake", "temporary", "too many requests", "429", "500", "502", "503", "504"} { + if strings.Contains(msg, marker) { + return true + } + } + return false +} + +func isAlipayTradeNotFound(parts ...string) bool { + msg := strings.ToLower(strings.Join(parts, " ")) + return strings.Contains(msg, "acq.trade_not_exist") || strings.Contains(msg, "trade_not_exist") || strings.Contains(msg, "transaction not exist") || strings.Contains(msg, "交易不存在") || strings.Contains(msg, "订单不存在") +} + +func sleepWithContext(ctx context.Context, delay time.Duration) error { + timer := time.NewTimer(delay) + defer timer.Stop() + select { + case <-ctx.Done(): + return ctx.Err() + case <-timer.C: + return nil + } +} diff --git a/internal/service/balance/service.go b/internal/service/balance/service.go new file mode 100644 index 0000000..b096432 --- /dev/null +++ b/internal/service/balance/service.go @@ -0,0 +1,192 @@ +package balance + +import ( + "context" + "database/sql" + "errors" + "fmt" + "math" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + "github.com/gogf/gf/v2/util/gconv" +) + +const ( + rechargeStatusPending = iota + rechargeStatusPaid + rechargeStatusExpired + + rechargeChannelAlipay = 1 +) + +type TradeQueryResult struct { + OutTradeNo string + TradeNo string + TradeStatus string + TotalAmount string + BuyerLogonID string +} + +type TradeQueryer interface { + Query(context.Context, string) (*TradeQueryResult, error) +} + +type Service struct { + queryer TradeQueryer +} + +func New() *Service { + return NewWithQuery(newAlipayTradeQueryer()) +} + +func NewWithQuery(queryer TradeQueryer) *Service { + return &Service{queryer: queryer} +} + +func (s *Service) CheckRechargeTimeout(ctx context.Context, outTradeNo string) error { + outTradeNo = strings.TrimSpace(outTradeNo) + if outTradeNo == "" { + return nil + } + + detail := rechargeDetail{} + if err := g.DB().Model("t_balance_detail").Ctx(ctx). + Where("order_sn", outTradeNo). + Where("transaction_channel", rechargeChannelAlipay). + Scan(&detail); err != nil { + if errors.Is(err, sql.ErrNoRows) { + return nil + } + return gerror.Wrap(err, "query recharge detail failed") + } + if detail.ID == 0 || detail.TransactionStatus != rechargeStatusPending { + return nil + } + if s.queryer == nil { + return gerror.New("alipay trade queryer is not configured") + } + + result, err := s.queryer.Query(ctx, outTradeNo) + if err != nil { + return err + } + if result != nil && (result.TradeStatus == "TRADE_SUCCESS" || result.TradeStatus == "TRADE_FINISHED") { + return s.creditRecharge(ctx, detail, result) + } + + _, err = g.DB().Model("t_balance_detail").Ctx(ctx). + Where("id", detail.ID). + Where("transaction_status", rechargeStatusPending). + Data(g.Map{ + "transaction_status": rechargeStatusExpired, + "remark": "Alipay recharge timed out without payment", + "updated_at": gtime.Now(), + }). + Update() + return gerror.Wrap(err, "expire recharge detail failed") +} + +func (s *Service) creditRecharge(ctx context.Context, initial rechargeDetail, result *TradeQueryResult) error { + amount := gconv.Float64(strings.TrimSpace(result.TotalAmount)) + if amount <= 0 { + return gerror.New("paid recharge has invalid amount") + } + + return g.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + detail := rechargeDetail{} + detailModel := lockForUpdate(tx.Ctx(ctx).Model("t_balance_detail")) + if err := detailModel. + Where("id", initial.ID). + Scan(&detail); err != nil { + return gerror.Wrap(err, "lock recharge detail failed") + } + if detail.ID == 0 || detail.TransactionStatus == rechargeStatusPaid { + return nil + } + if detail.TransactionStatus != rechargeStatusPending { + return gerror.New("recharge order is closed") + } + if math.Abs(detail.Amount-amount) > 0.001 { + return gerror.New("paid amount does not match recharge detail") + } + + account := agentBalance{} + accountModel := lockForUpdate(tx.Ctx(ctx).Model("t_agent_balance")) + if err := accountModel. + Where("dept_id", detail.DeptID). + Where("tenant_id", detail.TenantID). + Scan(&account); err != nil { + return gerror.Wrap(err, "lock agent balance failed") + } + now := gtime.Now() + if account.ID == 0 { + id, err := tx.Ctx(ctx).Model("t_agent_balance").Data(g.Map{ + "dept_id": detail.DeptID, + "balance": 0, + "withdraw_amount": 0, + "withdraw_amount_status": 1, + "invoice_amount": 0, + "can_invoice_amount": 0, + "credit_limit": 0, + "used_credit": 0, + "pending_repayment": 0, + "status": 0, + "tenant_id": detail.TenantID, + "created_dept": detail.CreatedDept, + "created_by": detail.CreatedBy, + "created_at": now, + }).InsertAndGetId() + if err != nil { + return gerror.Wrap(err, "initialize agent balance failed") + } + account = agentBalance{ID: id} + } + + before := account.Balance + after := before + detail.Amount + if _, err := tx.Ctx(ctx).Model("t_agent_balance").Where("id", account.ID).Data(g.Map{ + "balance": after, + "updated_by": detail.CreatedBy, + "updated_at": now, + }).Update(); err != nil { + return gerror.Wrap(err, "update agent balance failed") + } + _, err := tx.Ctx(ctx).Model("t_balance_detail").Where("id", detail.ID).Data(g.Map{ + "transaction_status": rechargeStatusPaid, + "transaction_time": now, + "before_balance": before, + "balance_after": after, + "channel_serial_no": result.TradeNo, + "remark": fmt.Sprintf("Alipay recharge, buyer: %s", result.BuyerLogonID), + "updated_by": detail.CreatedBy, + "updated_at": now, + }).Update() + return gerror.Wrap(err, "mark recharge paid failed") + }) +} + +type rechargeDetail struct { + ID int64 `orm:"id"` + DeptID int64 `orm:"dept_id"` + TransactionStatus int `orm:"transaction_status"` + Amount float64 `orm:"amount"` + TenantID string `orm:"tenant_id"` + CreatedDept int64 `orm:"created_dept"` + CreatedBy int64 `orm:"created_by"` +} + +type agentBalance struct { + ID int64 `orm:"id"` + Balance float64 `orm:"balance"` +} + +func lockForUpdate(model *gdb.Model) *gdb.Model { + if g.DB().GetConfig().Type == "sqlite" { + return model + } + return model.LockUpdate() +} diff --git a/internal/service/balance/service_test.go b/internal/service/balance/service_test.go new file mode 100644 index 0000000..94a89ab --- /dev/null +++ b/internal/service/balance/service_test.go @@ -0,0 +1,198 @@ +package balance + +import ( + "context" + "errors" + "io" + "path/filepath" + "testing" + + _ "github.com/gogf/gf/contrib/drivers/sqlite/v2" + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" +) + +type tradeQueryerStub struct { + result *TradeQueryResult + err error + calls int +} + +func (s *tradeQueryerStub) Query(_ context.Context, _ string) (*TradeQueryResult, error) { + s.calls++ + return s.result, s.err +} + +func TestCheckRechargeTimeout(t *testing.T) { + g.Log().SetWriter(io.Discard) + ctx := context.Background() + if err := gdb.SetConfig(gdb.Config{ + "default": gdb.ConfigGroup{{Type: "sqlite", Name: filepath.Join(t.TempDir(), "balance.db")}}, + }); err != nil { + t.Fatalf("configure test database: %v", err) + } + defer g.DB().Close(ctx) + createBalanceTables(t, ctx) + + t.Run("paid recharge is credited exactly once", func(t *testing.T) { + insertRechargeDetail(t, ctx, "BAL-PAID", 10) + insertAgentBalance(t, ctx, 11, "tenant-1", 5) + queryer := &tradeQueryerStub{result: &TradeQueryResult{TradeStatus: "TRADE_SUCCESS", TotalAmount: "10.00", TradeNo: "ALI-1"}} + service := NewWithQuery(queryer) + + if err := service.CheckRechargeTimeout(ctx, "BAL-PAID"); err != nil { + t.Fatalf("first check failed: %v", err) + } + if status := rechargeStatus(t, ctx, "BAL-PAID"); status != rechargeStatusPaid { + t.Fatalf("expected paid status, got %d", status) + } + if balance := agentBalanceValue(t, ctx, 11, "tenant-1"); balance != 15 { + t.Fatalf("expected balance 15, got %v", balance) + } + if err := service.CheckRechargeTimeout(ctx, "BAL-PAID"); err != nil { + t.Fatalf("duplicate check failed: %v", err) + } + if queryer.calls != 1 { + t.Fatalf("expected one Alipay query, got %d", queryer.calls) + } + if balance := agentBalanceValue(t, ctx, 11, "tenant-1"); balance != 15 { + t.Fatalf("duplicate message changed balance to %v", balance) + } + }) + + t.Run("unpaid recharge is expired", func(t *testing.T) { + insertRechargeDetail(t, ctx, "BAL-EXPIRED", 10) + service := NewWithQuery(&tradeQueryerStub{result: &TradeQueryResult{TradeStatus: "WAIT_BUYER_PAY"}}) + if err := service.CheckRechargeTimeout(ctx, "BAL-EXPIRED"); err != nil { + t.Fatalf("timeout check failed: %v", err) + } + if status := rechargeStatus(t, ctx, "BAL-EXPIRED"); status != rechargeStatusExpired { + t.Fatalf("expected expired status, got %d", status) + } + }) + + t.Run("missing recharge is ignored", func(t *testing.T) { + queryer := &tradeQueryerStub{err: errors.New("must not be called")} + if err := NewWithQuery(queryer).CheckRechargeTimeout(ctx, "BAL-MISSING"); err != nil { + t.Fatalf("missing recharge returned error: %v", err) + } + if queryer.calls != 0 { + t.Fatalf("missing recharge queried Alipay %d times", queryer.calls) + } + }) + + t.Run("amount mismatch returns an error without crediting", func(t *testing.T) { + insertRechargeDetail(t, ctx, "BAL-MISMATCH", 10) + insertAgentBalance(t, ctx, 11, "tenant-1", 15) + service := NewWithQuery(&tradeQueryerStub{result: &TradeQueryResult{TradeStatus: "TRADE_SUCCESS", TotalAmount: "9.00"}}) + if err := service.CheckRechargeTimeout(ctx, "BAL-MISMATCH"); err == nil { + t.Fatal("expected amount mismatch error") + } + if status := rechargeStatus(t, ctx, "BAL-MISMATCH"); status != rechargeStatusPending { + t.Fatalf("mismatch changed status to %d", status) + } + if balance := agentBalanceValue(t, ctx, 11, "tenant-1"); balance != 15 { + t.Fatalf("mismatch changed balance to %v", balance) + } + }) + + t.Run("Alipay failure is returned for delayed retry", func(t *testing.T) { + insertRechargeDetail(t, ctx, "BAL-RETRY", 10) + service := NewWithQuery(&tradeQueryerStub{err: errors.New("Alipay unavailable")}) + if err := service.CheckRechargeTimeout(ctx, "BAL-RETRY"); err == nil { + t.Fatal("expected query error") + } + if status := rechargeStatus(t, ctx, "BAL-RETRY"); status != rechargeStatusPending { + t.Fatalf("retryable failure changed status to %d", status) + } + }) +} + +func createBalanceTables(t *testing.T, ctx context.Context) { + t.Helper() + statements := []string{ + `CREATE TABLE t_balance_detail ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + dept_id INTEGER NOT NULL, + transaction_status INTEGER NOT NULL, + transaction_channel INTEGER NOT NULL, + amount REAL NOT NULL, + order_sn TEXT NOT NULL, + tenant_id TEXT NOT NULL, + created_dept INTEGER NOT NULL, + created_by INTEGER NOT NULL, + transaction_time DATETIME, + before_balance REAL, + balance_after REAL, + channel_serial_no TEXT, + remark TEXT, + updated_by INTEGER, + updated_at DATETIME + )`, + `CREATE TABLE t_agent_balance ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + dept_id INTEGER NOT NULL, + balance REAL NOT NULL, + withdraw_amount REAL, + withdraw_amount_status INTEGER, + invoice_amount REAL, + can_invoice_amount REAL, + credit_limit REAL, + used_credit REAL, + pending_repayment REAL, + status INTEGER, + tenant_id TEXT NOT NULL, + created_dept INTEGER, + created_by INTEGER, + created_at DATETIME, + updated_by INTEGER, + updated_at DATETIME + )`, + } + for _, statement := range statements { + if _, err := g.DB().Exec(ctx, statement); err != nil { + t.Fatalf("create table: %v", err) + } + } +} + +func insertRechargeDetail(t *testing.T, ctx context.Context, orderSN string, amount float64) { + t.Helper() + if _, err := g.DB().Model("t_balance_detail").Ctx(ctx).Data(g.Map{ + "dept_id": 11, + "transaction_status": rechargeStatusPending, + "transaction_channel": rechargeChannelAlipay, + "amount": amount, + "order_sn": orderSN, + "tenant_id": "tenant-1", + "created_dept": 11, + "created_by": 7, + }).Insert(); err != nil { + t.Fatalf("insert recharge detail: %v", err) + } +} + +func insertAgentBalance(t *testing.T, ctx context.Context, deptID int64, tenantID string, balance float64) { + t.Helper() + if _, err := g.DB().Model("t_agent_balance").Ctx(ctx).Data(g.Map{"dept_id": deptID, "tenant_id": tenantID, "balance": balance}).Insert(); err != nil { + t.Fatalf("insert agent balance: %v", err) + } +} + +func rechargeStatus(t *testing.T, ctx context.Context, orderSN string) int { + t.Helper() + value, err := g.DB().Model("t_balance_detail").Ctx(ctx).Where("order_sn", orderSN).Value("transaction_status") + if err != nil { + t.Fatalf("get recharge status: %v", err) + } + return value.Int() +} + +func agentBalanceValue(t *testing.T, ctx context.Context, deptID int64, tenantID string) float64 { + t.Helper() + value, err := g.DB().Model("t_agent_balance").Ctx(ctx).Where("dept_id", deptID).Where("tenant_id", tenantID).Value("balance") + if err != nil { + t.Fatalf("get agent balance: %v", err) + } + return value.Float64() +} diff --git a/internal/service/batchresult/generator.go b/internal/service/batchresult/generator.go new file mode 100644 index 0000000..2068eed --- /dev/null +++ b/internal/service/batchresult/generator.go @@ -0,0 +1,693 @@ +package batchresult + +import ( + "context" + "crypto/md5" + "encoding/hex" + "fmt" + "io" + "os" + "path/filepath" + "strings" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/xuri/excelize/v2" +) + +const configOssURLPathKey = "sys.oss.urlPath" + +type Row struct { + RowNo int + ObjectCode string + TargetCode string + PackageCode string + EffectiveMode string + ICCID string + MSISDN string + IMSI string + TargetDeptName string + StatusBefore int + TargetStatus int + Operation string + ApnName string + Success bool + Reason string + UpstreamCode int + OrderNum string +} + +type Request struct { + BatchNo string + BatchType int + TenantID string + Rows []Row +} + +type Artifact struct { + FileName string + OriginalName string + FileSuffix string + Path string + LocalPath string + URL string + Size int64 + MD5 string +} + +type Generator interface { + Create(ctx context.Context, request Request) (Artifact, error) +} + +type generator struct{} + +func New() Generator { + return &generator{} +} + +func (gtr *generator) Create(ctx context.Context, request Request) (Artifact, error) { + if request.BatchNo == "" || filepath.Base(request.BatchNo) != request.BatchNo { + return Artifact{}, gerror.New("invalid batch number") + } + uploadPath := g.Cfg().MustGet(ctx, "system.upload.local.path").String() + if uploadPath == "" { + return Artifact{}, gerror.New("system.upload.local.path is empty") + } + tenantID := request.TenantID + if tenantID == "" { + tenantID = "default" + } + resultDir, err := resultDir(request.BatchType) + if err != nil { + return Artifact{}, err + } + objectDir := filepath.Join(tenantID, "user", "file", time.Now().Format("20060102"), resultDir) + physicalUploadPath := resultPhysicalUploadRoot(uploadPath) + directory := filepath.Join(physicalUploadPath, objectDir) + if err = os.MkdirAll(directory, 0o755); err != nil { + return Artifact{}, gerror.Wrap(err, "create batch result directory") + } + fileName := request.BatchNo + "-result.xlsx" + filePath := filepath.Join(directory, fileName) + logicalPath := filepath.Join(uploadPath, objectDir, fileName) + if err = writeWorkbook(filePath, request); err != nil { + return Artifact{}, err + } + artifact, err := inspectArtifact(ctx, filePath, logicalPath, fileName, tenantID) + if err != nil { + _ = os.Remove(filePath) + return Artifact{}, err + } + return artifact, nil +} + +func resultPhysicalUploadRoot(uploadPath string) string { + uploadPath = strings.TrimSpace(uploadPath) + for _, envName := range []string{"IOTLCM_WORKER_UPLOAD_ROOT", "IOTLCM_UPLOAD_ROOT"} { + if value := strings.TrimSpace(os.Getenv(envName)); value != "" { + return value + } + } + for _, candidate := range uploadRootEquivalentCandidates(uploadPath) { + if samePath(candidate, uploadPath) { + continue + } + for _, resolved := range resolvedRootCandidates(candidate) { + if info, err := os.Stat(resolved); err == nil && info.IsDir() { + return resolved + } + } + } + return uploadPath +} + +func uploadRootEquivalentCandidates(root string) []string { + root = strings.TrimSpace(root) + if root == "" { + return nil + } + candidates := []string{root} + normalized := filepath.ToSlash(filepath.Clean(root)) + if normalized == "resource/upload" || strings.HasSuffix(normalized, "/resource/upload") { + prefix := strings.TrimSuffix(normalized, "resource/upload") + candidates = appendUniqueString(candidates, filepath.FromSlash(prefix+"upload")) + } + return candidates +} + +func resolvedRootCandidates(root string) []string { + root = strings.TrimSpace(root) + if root == "" { + return nil + } + if filepath.IsAbs(root) { + return []string{root} + } + candidates := []string{root} + for _, base := range localBaseDirs() { + candidates = appendUniqueString(candidates, filepath.Join(base, root)) + } + return candidates +} + +func localBaseDirs() []string { + var dirs []string + if wd, err := os.Getwd(); err == nil && strings.TrimSpace(wd) != "" { + dirs = appendUniqueString(dirs, wd) + } + if exe, err := os.Executable(); err == nil && strings.TrimSpace(exe) != "" { + dirs = appendUniqueString(dirs, filepath.Dir(exe)) + } + return dirs +} + +func samePath(a, b string) bool { + return filepath.Clean(strings.TrimSpace(a)) == filepath.Clean(strings.TrimSpace(b)) +} + +func appendUniqueString(values []string, value string) []string { + value = strings.TrimSpace(value) + if value == "" { + return values + } + for _, existing := range values { + if samePath(existing, value) { + return values + } + } + return append(values, value) +} + +func resultDir(batchType int) (string, error) { + if batchType == 11 { + return filepath.Join("sim-card-add-package", "result"), nil + } + if batchType == 12 { + return filepath.Join("sim-pool-add-package", "result"), nil + } + switch batchType { + case 1: + return filepath.Join("sim-card-inbound", "result"), nil + case 3: + return filepath.Join("sim-card-allocate", "result"), nil + case 4: + return filepath.Join("sim-card-renew-change-prepare", "result"), nil + case 101: + return filepath.Join("sim-card-renew", "result"), nil + case 102: + return filepath.Join("sim-card-change", "result"), nil + case 103: + return filepath.Join("sim-card-start", "result"), nil + case 104: + return filepath.Join("sim-card-stop", "result"), nil + case 105: + return filepath.Join("sim-card-restart", "result"), nil + case 106: + return filepath.Join("sim-card-cancel", "result"), nil + case 107: + return filepath.Join("sim-card-data-cut", "result"), nil + case 108: + return filepath.Join("sim-card-unsub", "result"), nil + case 109: + return filepath.Join("sim-card-data-start", "result"), nil + case 110: + return filepath.Join("sim-card-data-stop", "result"), nil + case 40: + return filepath.Join("sim-warning-config", "result"), nil + case 41: + return filepath.Join("sim-card-traffic-cutoff-config", "result"), nil + default: + return "", gerror.Newf("unsupported batch type %d", batchType) + } +} + +func writeWorkbook(path string, request Request) error { + file := excelize.NewFile() + defer func() { _ = file.Close() }() + + const resultSheet = "处理结果" + defaultSheet := file.GetSheetName(0) + if err := file.SetSheetName(defaultSheet, resultSheet); err != nil { + return gerror.Wrap(err, "name result sheet") + } + if request.BatchType == 1 { + return writeInboundWorkbook(file, resultSheet, path, request) + } + if request.BatchType == 3 { + return writeAllocateWorkbook(file, resultSheet, path, request) + } + if request.BatchType == 11 || request.BatchType == 12 { + return writeAddPackageWorkbook(file, resultSheet, path, request) + } + if request.BatchType == 106 { + return writeCancelWorkbook(file, resultSheet, path, request) + } + if request.BatchType == 40 || request.BatchType == 41 { + return writeWarningConfigWorkbook(file, resultSheet, path, request) + } + if isDataServiceBatch(request.BatchType) { + return writeDataServiceWorkbook(file, resultSheet, path, request) + } + return writeLifecycleWorkbook(file, resultSheet, path, request) +} + +func writeAllocateWorkbook(file *excelize.File, sheet, path string, request Request) error { + headers := []interface{}{"行号", "ICCID", "MSISDN", "IMSI", "目标客户", "执行结果", "失败原因"} + if err := file.SetSheetRow(sheet, "A1", &headers); err != nil { + return gerror.Wrap(err, "write allocate result headers") + } + for index, row := range request.Rows { + status := "失败" + if row.Success { + status = "成功" + } + values := []interface{}{row.RowNo, row.ICCID, row.MSISDN, row.IMSI, row.TargetDeptName, status, row.Reason} + cell, err := excelize.CoordinatesToCellName(1, index+2) + if err != nil { + return gerror.Wrap(err, "locate allocate result row") + } + if err = file.SetSheetRow(sheet, cell, &values); err != nil { + return gerror.Wrap(err, "write allocate result row") + } + } + if err := file.SetColWidth(sheet, "A", "A", 10); err != nil { + return gerror.Wrap(err, "format allocate result workbook") + } + if err := file.SetColWidth(sheet, "B", "E", 24); err != nil { + return gerror.Wrap(err, "format allocate result workbook") + } + if err := file.SetColWidth(sheet, "F", "G", 24); err != nil { + return gerror.Wrap(err, "format allocate result workbook") + } + if err := file.SaveAs(path); err != nil { + return gerror.Wrap(err, "save allocate result workbook") + } + return nil +} + +func writeWarningConfigWorkbook(file *excelize.File, sheet, path string, request Request) error { + headers := []interface{}{"行号", "对象编码", "执行结果", "失败原因"} + if err := file.SetSheetRow(sheet, "A1", &headers); err != nil { + return gerror.Wrap(err, "write warning config result headers") + } + for index, row := range request.Rows { + status := "失败" + if row.Success { + status = "成功" + } + objectCode := row.ObjectCode + if objectCode == "" { + objectCode = row.ICCID + } + values := []interface{}{row.RowNo, objectCode, status, row.Reason} + cell, err := excelize.CoordinatesToCellName(1, index+2) + if err != nil { + return gerror.Wrap(err, "locate warning config result row") + } + if err = file.SetSheetRow(sheet, cell, &values); err != nil { + return gerror.Wrap(err, "write warning config result row") + } + } + if err := file.SetColWidth(sheet, "A", "A", 10); err != nil { + return gerror.Wrap(err, "format warning config result workbook") + } + if err := file.SetColWidth(sheet, "B", "B", 28); err != nil { + return gerror.Wrap(err, "format warning config result workbook") + } + if err := file.SetColWidth(sheet, "C", "D", 24); err != nil { + return gerror.Wrap(err, "format warning config result workbook") + } + if err := file.SaveAs(path); err != nil { + return gerror.Wrap(err, "save warning config result workbook") + } + return nil +} + +func writeAddPackageWorkbook(file *excelize.File, sheet, path string, request Request) error { + headers := []interface{}{"行号", "目标编码", "加油包编码", "生效方式", "执行结果", "失败原因", "订单号"} + if err := file.SetSheetRow(sheet, "A1", &headers); err != nil { + return gerror.Wrap(err, "write add package result headers") + } + for index, row := range request.Rows { + status := "失败" + if row.Success { + status = "成功" + } + targetCode := row.ObjectCode + if targetCode == "" { + targetCode = row.TargetCode + } + values := []interface{}{row.RowNo, targetCode, row.PackageCode, addPackageEffectiveModeText(row.EffectiveMode), status, row.Reason, row.OrderNum} + cell, err := excelize.CoordinatesToCellName(1, index+2) + if err != nil { + return gerror.Wrap(err, "locate add package result row") + } + if err = file.SetSheetRow(sheet, cell, &values); err != nil { + return gerror.Wrap(err, "write add package result row") + } + } + if err := file.SetColWidth(sheet, "A", "A", 10); err != nil { + return gerror.Wrap(err, "format add package result workbook") + } + if err := file.SetColWidth(sheet, "B", "D", 24); err != nil { + return gerror.Wrap(err, "format add package result workbook") + } + if err := file.SetColWidth(sheet, "E", "G", 22); err != nil { + return gerror.Wrap(err, "format add package result workbook") + } + if err := file.SaveAs(path); err != nil { + return gerror.Wrap(err, "save add package result workbook") + } + return nil +} + +func addPackageEffectiveModeText(mode string) string { + switch strings.TrimSpace(mode) { + case "", "current_month", "immediate", "this_month", "本月生效", "本月", "当月": + return "本月生效" + case "next_month", "nextMonth", "次月生效", "次月", "下月", "下月生效": + return "次月生效" + default: + return strings.TrimSpace(mode) + } +} + +func isDataServiceBatch(batchType int) bool { + return batchType == 107 || batchType == 109 || batchType == 110 +} + +func writeInboundWorkbook(file *excelize.File, sheet, path string, request Request) error { + headers := []interface{}{"行号", "ICCID", "执行结果", "失败原因"} + if err := file.SetSheetRow(sheet, "A1", &headers); err != nil { + return gerror.Wrap(err, "write inbound result headers") + } + for index, row := range request.Rows { + status := "失败" + if row.Success { + status = "成功" + } + values := []interface{}{row.RowNo, row.ICCID, status, row.Reason} + cell, err := excelize.CoordinatesToCellName(1, index+2) + if err != nil { + return gerror.Wrap(err, "locate inbound result row") + } + if err = file.SetSheetRow(sheet, cell, &values); err != nil { + return gerror.Wrap(err, "write inbound result row") + } + } + if err := file.SetColWidth(sheet, "A", "A", 10); err != nil { + return gerror.Wrap(err, "format inbound result workbook") + } + if err := file.SetColWidth(sheet, "B", "B", 24); err != nil { + return gerror.Wrap(err, "format inbound result workbook") + } + if err := file.SetColWidth(sheet, "C", "D", 24); err != nil { + return gerror.Wrap(err, "format inbound result workbook") + } + if err := file.SaveAs(path); err != nil { + return gerror.Wrap(err, "save inbound result workbook") + } + return nil +} + +func writeLifecycleWorkbook(file *excelize.File, sheet, path string, request Request) error { + headers := []interface{}{"行号", "ICCID", "MSISDN", "IMSI", "执行结果", "失败原因", "上游状态码", "上游订单号"} + if err := file.SetSheetRow(sheet, "A1", &headers); err != nil { + return gerror.Wrap(err, "write lifecycle result headers") + } + for index, row := range request.Rows { + status := "失败" + if row.Success { + status = "成功" + } + values := []interface{}{row.RowNo, row.ICCID, row.MSISDN, row.IMSI, status, row.Reason, row.UpstreamCode, row.OrderNum} + cell, err := excelize.CoordinatesToCellName(1, index+2) + if err != nil { + return gerror.Wrap(err, "locate lifecycle result row") + } + if err = file.SetSheetRow(sheet, cell, &values); err != nil { + return gerror.Wrap(err, "write lifecycle result row") + } + } + if err := file.SetColWidth(sheet, "A", "A", 10); err != nil { + return gerror.Wrap(err, "format lifecycle result workbook") + } + if err := file.SetColWidth(sheet, "B", "D", 24); err != nil { + return gerror.Wrap(err, "format lifecycle result workbook") + } + if err := file.SetColWidth(sheet, "E", "H", 20); err != nil { + return gerror.Wrap(err, "format lifecycle result workbook") + } + if err := file.SaveAs(path); err != nil { + return gerror.Wrap(err, "save lifecycle result workbook") + } + return nil +} + +func writeCancelWorkbook(file *excelize.File, sheet, path string, request Request) error { + headers := []interface{}{"行号", "ICCID", "MSISDN", "IMSI", "原卡状态", "目标卡状态", "执行结果", "失败原因"} + if err := file.SetSheetRow(sheet, "A1", &headers); err != nil { + return gerror.Wrap(err, "write cancel result headers") + } + for index, row := range request.Rows { + status := "失败" + if row.Success { + status = "成功" + } + values := []interface{}{row.RowNo, row.ICCID, row.MSISDN, row.IMSI, row.StatusBefore, row.TargetStatus, status, row.Reason} + cell, err := excelize.CoordinatesToCellName(1, index+2) + if err != nil { + return gerror.Wrap(err, "locate cancel result row") + } + if err = file.SetSheetRow(sheet, cell, &values); err != nil { + return gerror.Wrap(err, "write cancel result row") + } + } + if err := file.SetColWidth(sheet, "A", "A", 10); err != nil { + return gerror.Wrap(err, "format cancel result workbook") + } + if err := file.SetColWidth(sheet, "B", "D", 24); err != nil { + return gerror.Wrap(err, "format cancel result workbook") + } + if err := file.SetColWidth(sheet, "E", "H", 20); err != nil { + return gerror.Wrap(err, "format cancel result workbook") + } + if err := file.SaveAs(path); err != nil { + return gerror.Wrap(err, "save cancel result workbook") + } + return nil +} + +func writeDataServiceWorkbook(file *excelize.File, sheet, path string, request Request) error { + headers := []interface{}{"行号", "ICCID", "MSISDN", "IMSI", "操作", "APN", "执行结果", "失败原因", "上游状态码", "上游订单号"} + if err := file.SetSheetRow(sheet, "A1", &headers); err != nil { + return gerror.Wrap(err, "write data service result headers") + } + for index, row := range request.Rows { + status := "失败" + if row.Success { + status = "成功" + } + values := []interface{}{row.RowNo, row.ICCID, row.MSISDN, row.IMSI, row.Operation, row.ApnName, status, row.Reason, row.UpstreamCode, row.OrderNum} + cell, err := excelize.CoordinatesToCellName(1, index+2) + if err != nil { + return gerror.Wrap(err, "locate data service result row") + } + if err = file.SetSheetRow(sheet, cell, &values); err != nil { + return gerror.Wrap(err, "write data service result row") + } + } + if err := file.SetColWidth(sheet, "A", "A", 10); err != nil { + return gerror.Wrap(err, "format data service result workbook") + } + if err := file.SetColWidth(sheet, "B", "D", 24); err != nil { + return gerror.Wrap(err, "format data service result workbook") + } + if err := file.SetColWidth(sheet, "E", "J", 20); err != nil { + return gerror.Wrap(err, "format data service result workbook") + } + if err := file.SaveAs(path); err != nil { + return gerror.Wrap(err, "save data service result workbook") + } + return nil +} + +func inspectArtifact(ctx context.Context, localPath, logicalPath, fileName string, tenantID string) (Artifact, error) { + file, err := os.Open(localPath) + if err != nil { + return Artifact{}, gerror.Wrap(err, "open result file") + } + defer file.Close() + hash := md5.New() + size, err := io.Copy(hash, file) + if err != nil { + return Artifact{}, gerror.Wrap(err, "hash result file") + } + return Artifact{ + FileName: fileName, + OriginalName: fileName, + FileSuffix: "xlsx", + Path: logicalPath, + LocalPath: localPath, + URL: localUploadURL(ctx, logicalPath, tenantID), + Size: size, + MD5: hex.EncodeToString(hash.Sum(nil)), + }, nil +} + +func RemoveLocalFile(artifact Artifact) { + path := strings.TrimSpace(artifact.LocalPath) + if path == "" { + path = strings.TrimSpace(artifact.Path) + } + if path != "" { + _ = os.Remove(path) + } +} + +func localUploadURL(ctx context.Context, filePath string, tenantID string) string { + filePath = strings.TrimSpace(filepath.ToSlash(filepath.Clean(filePath))) + if filePath == "" { + return "" + } + + if configValue := loadUploadURLPathConfig(ctx, tenantID); configValue != "" { + if strings.Contains(configValue, "|") { + split := strings.SplitN(configValue, "|", 2) + from := strings.TrimSpace(filepath.ToSlash(split[0])) + to := strings.TrimSpace(filepath.ToSlash(split[1])) + if suffix, ok := localUploadMappedSuffix(filePath, from); ok { + return joinUploadURL(to, suffix) + } + } else { + if suffix := localUploadRelativePath(filePath); suffix != "" { + return joinUploadURL(configValue, suffix) + } + return joinUploadURL(configValue, filePath) + } + } + + if suffix := localUploadRelativePath(filePath); suffix != "" { + return joinUploadURL("/upload", suffix) + } + return filePath +} + +func loadUploadURLPathConfig(ctx context.Context, tenantID string) (configValue string) { + defer func() { + if recover() != nil { + configValue = "" + } + }() + for _, tenant := range []string{strings.TrimSpace(tenantID), "000000"} { + if tenant == "" { + continue + } + var row struct { + ConfigValue string `orm:"config_value"` + } + err := g.DB().Model("sys_config").Safe().Ctx(ctx). + Fields("config_value"). + Where("tenant_id", tenant). + Where("config_key", configOssURLPathKey). + Limit(1). + Scan(&row) + if err == nil && strings.TrimSpace(row.ConfigValue) != "" { + return strings.TrimSpace(row.ConfigValue) + } + } + return "" +} + +func localUploadMappedSuffix(filePath string, from string) (string, bool) { + from = strings.Trim(strings.TrimSpace(filepath.ToSlash(from)), "/") + if from == "" { + return "", false + } + normalized := strings.TrimSpace(filepath.ToSlash(filePath)) + marker := "/" + from + "/" + if idx := strings.Index(normalized, marker); idx >= 0 { + return strings.TrimPrefix(normalized[idx+len(marker):], "/"), true + } + prefix := from + "/" + if strings.HasPrefix(normalized, prefix) { + return strings.TrimPrefix(normalized[len(prefix):], "/"), true + } + if strings.Trim(normalized, "/") == from { + return "", true + } + return "", false +} + +func localUploadRelativePath(filePath string) string { + normalized := filepath.ToSlash(filepath.Clean(strings.TrimSpace(filePath))) + for _, marker := range []string{ + "resource/upload/", + "/resource/upload/", + "/app/resource/upload/", + "/app/upload/", + "upload/", + "/upload/", + } { + if idx := strings.Index(normalized, marker); idx >= 0 { + return strings.TrimPrefix(normalized[idx+len(marker):], "/") + } + } + return "" +} + +func joinUploadURL(base string, suffix string) string { + base = strings.TrimRight(strings.TrimSpace(filepath.ToSlash(base)), "/") + suffix = strings.TrimLeft(strings.TrimSpace(filepath.ToSlash(suffix)), "/") + if base == "" { + base = "/upload" + } + if suffix == "" { + return base + } + return base + "/" + suffix +} + +func TaskName(batchType int) string { + if batchType == 11 { + return "单卡加油包结果" + } + if batchType == 12 { + return "流量池加油包结果" + } + switch batchType { + case 1: + return "SIM卡批量入库结果" + case 3: + return "卡划拨结果" + case 4: + return "卡续费/变更预处理结果" + case 101: + return "卡续费结果" + case 102: + return "卡变更套餐结果" + case 103: + return "SIM卡批量复机结果" + case 104: + return "SIM卡批量停机结果" + case 105: + return "SIM卡批量重启结果" + case 106: + return "卡销户结果" + case 108: + return "SIM卡批量退订结果" + case 107: + return "SIM卡达量断网结果" + case 109: + return "卡开数据结果" + case 110: + return "卡停数据结果" + case 40: + return "SIM卡预警配置结果" + case 41: + return "达量断网配置结果" + default: + return fmt.Sprintf("SIM卡批量任务%d结果", batchType) + } +} diff --git a/internal/service/batchresult/generator_test.go b/internal/service/batchresult/generator_test.go new file mode 100644 index 0000000..09d8bc6 --- /dev/null +++ b/internal/service/batchresult/generator_test.go @@ -0,0 +1,71 @@ +package batchresult + +import ( + "context" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gcfg" +) + +func TestCreateWritesToMountedUploadAndKeepsLogicalPath(t *testing.T) { + wd := t.TempDir() + oldWd, err := os.Getwd() + if err != nil { + t.Fatalf("get working directory: %v", err) + } + if err = os.Chdir(wd); err != nil { + t.Fatalf("change working directory: %v", err) + } + defer func() { + _ = os.Chdir(oldWd) + }() + + if err = os.MkdirAll(filepath.Join(wd, "upload"), 0o755); err != nil { + t.Fatalf("create mounted upload root: %v", err) + } + oldAdapter := g.Cfg().GetAdapter() + adapter, err := gcfg.NewAdapterContent(` +system: + upload: + local: + path: "resource/upload" +`) + if err != nil { + t.Fatalf("create config adapter: %v", err) + } + g.Cfg().SetAdapter(adapter) + defer g.Cfg().SetAdapter(oldAdapter) + + artifact, err := New().Create(context.Background(), Request{ + BatchNo: "INB202608140001001", + BatchType: 1, + TenantID: "000000", + Rows: []Row{{RowNo: 1, ICCID: "89860401780000567308", Success: true}}, + }) + if err != nil { + t.Fatalf("Create() error = %v", err) + } + defer RemoveLocalFile(artifact) + + logicalPath := filepath.ToSlash(artifact.Path) + if !strings.HasPrefix(logicalPath, "resource/upload/000000/user/file/") { + t.Fatalf("logical path = %q, want resource/upload path", artifact.Path) + } + localPath := filepath.ToSlash(artifact.LocalPath) + if !strings.HasPrefix(localPath, "upload/000000/user/file/") { + t.Fatalf("local path = %q, want mounted upload path", artifact.LocalPath) + } + if _, err = os.Stat(artifact.LocalPath); err != nil { + t.Fatalf("result file was not written at local path %q: %v", artifact.LocalPath, err) + } + if _, err = os.Stat(artifact.Path); !os.IsNotExist(err) { + t.Fatalf("result file should not be written to logical-only path %q, stat error=%v", artifact.Path, err) + } + if !strings.HasPrefix(artifact.URL, "/upload/000000/user/file/") { + t.Fatalf("url = %q, want /upload URL", artifact.URL) + } +} diff --git a/internal/service/cancel/executor.go b/internal/service/cancel/executor.go new file mode 100644 index 0000000..c8be145 --- /dev/null +++ b/internal/service/cancel/executor.go @@ -0,0 +1,122 @@ +package cancel + +import ( + "context" + "strings" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/packagecustomer" +) + +const ( + cascadeFailureReason = "same batch has invalid cancellation rows; this row was skipped" + validationSkipReason = "batch validation failed; cancellation was skipped" + emptyICCIDReason = "ICCID cannot be empty" + duplicateICCIDReason = "duplicate ICCID in batch" +) + +type executor struct { + repository Repository +} + +func NewExecutor(repository Repository) Executor { + return &executor{repository: repository} +} + +func (e *executor) Execute(ctx context.Context, batchNo string) error { + task, err := e.repository.Load(ctx, strings.TrimSpace(batchNo)) + if err != nil { + return gerror.Wrap(err, "load cancel batch failed") + } + if !validateTask(task) { + if err = e.repository.SaveValidationFailure(ctx, task); err != nil { + return gerror.Wrap(err, "save cancel validation failure failed") + } + e.saveResultFile(ctx, task) + return nil + } + if err = e.repository.CommitBusiness(ctx, task); err != nil { + return gerror.Wrap(err, "commit cancel business failed") + } + packagecustomer.PublishSyncForDeptsSafe(ctx, task.Batch.TenantID, task.Batch.CreatedBy, cancelPackageSyncDeptIDs(task), "cancel", task.Batch.BatchNo) + e.saveResultFile(ctx, task) + return nil +} + +func cancelPackageSyncDeptIDs(task *Task) []int64 { + if task == nil { + return nil + } + deptIds := []int64{task.Batch.DeptID, task.Batch.CreatedDept} + for _, detail := range task.Details { + if detail == nil || strings.TrimSpace(detail.Reason) != "" { + continue + } + if detail.CardDeptID > 0 { + deptIds = append(deptIds, detail.CardDeptID) + } + } + return deptIds +} + +func (e *executor) saveResultFile(ctx context.Context, task *Task) { + request := batchresult.Request{ + BatchNo: task.Batch.BatchNo, + BatchType: task.Batch.BatchType, + TenantID: task.Batch.TenantID, + Rows: task.ResultRows(), + } + if err := e.repository.SaveResultFile(ctx, task, request); err != nil { + g.Log().Errorf(ctx, "save cancel result file status failed, batchNo:%s, err:%+v", task.Batch.BatchNo, err) + } +} + +func validateTask(task *Task) bool { + if task == nil || len(task.Details) == 0 { + if task != nil { + task.FailureReason = "cancel batch has no valid details" + } + return false + } + hasFailure := false + iccids := make(map[string]int, len(task.Details)) + for _, detail := range task.Details { + if detail == nil { + hasFailure = true + continue + } + if detail.Reason != "" { + hasFailure = true + continue + } + iccid := strings.TrimSpace(detail.ICCID) + if iccid == "" { + detail.Reason = emptyICCIDReason + hasFailure = true + continue + } + iccids[iccid]++ + } + for _, detail := range task.Details { + if detail == nil || detail.Reason != "" { + continue + } + if iccids[strings.TrimSpace(detail.ICCID)] > 1 { + detail.Reason = duplicateICCIDReason + hasFailure = true + } + } + if !hasFailure { + return true + } + for _, detail := range task.Details { + if detail != nil && detail.Reason == "" { + detail.Reason = cascadeFailureReason + } + } + task.FailureReason = validationSkipReason + return false +} diff --git a/internal/service/cancel/types.go b/internal/service/cancel/types.go new file mode 100644 index 0000000..b7f975e --- /dev/null +++ b/internal/service/cancel/types.go @@ -0,0 +1,117 @@ +package cancel + +import ( + "context" + "strings" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +type Batch struct { + ID int64 + BatchNo string + BatchType int + TenantID string + DeptID int64 + CreatedBy int64 + CreatedDept int64 + StartTime *gtime.Time + BatchInfo *gjson.Json +} + +type Detail struct { + ID int64 + RowNo int + ICCID string + MSISDN string + IMSI string + Reason string + + Authorized bool + InputMSISDN string + InputIMSI string + + StatusBefore int + TargetStatus int + CancelTime string + + SubscriptionNo string + OrderCode string + PackageCode string + PackageType int + CardDeptID int64 +} + +type Task struct { + Batch Batch + Details []*Detail + FailureReason string +} + +type Repository interface { + Load(ctx context.Context, batchNo string) (*Task, error) + SaveValidationFailure(ctx context.Context, task *Task) error + CommitBusiness(ctx context.Context, task *Task) error + SaveResultFile(ctx context.Context, task *Task, request batchresult.Request) error +} + +type Executor interface { + Execute(ctx context.Context, batchNo string) error +} + +func (d *Detail) Snapshot() *gjson.Json { + if d == nil { + return gjson.New(nil) + } + return gjson.New(map[string]any{ + "rowNo": d.RowNo, + "ICCID": d.ICCID, + "MSISDN": d.MSISDN, + "IMSI": d.IMSI, + "inputMsisdn": d.InputMSISDN, + "inputImsi": d.InputIMSI, + "authorized": d.Authorized, + "statusBefore": d.StatusBefore, + "targetStatus": d.TargetStatus, + "cancelTime": d.CancelTime, + "subscriptionNo": d.SubscriptionNo, + "orderCode": d.OrderCode, + "packageCode": d.PackageCode, + "packageType": d.PackageType, + "cardDeptId": d.CardDeptID, + "errorReason": d.Reason, + }) +} + +func (t *Task) ResultRows() []batchresult.Row { + rows := make([]batchresult.Row, 0, len(t.Details)) + for _, detail := range t.Details { + if detail == nil { + continue + } + rows = append(rows, batchresult.Row{ + RowNo: detail.RowNo, + ICCID: detail.ICCID, + MSISDN: detail.MSISDN, + IMSI: detail.IMSI, + StatusBefore: detail.StatusBefore, + TargetStatus: detail.TargetStatus, + Success: detail.Reason == "", + Reason: detail.Reason, + }) + } + return rows +} + +func firstNonEmpty(values ...string) string { + for _, value := range values { + value = strings.TrimSpace(value) + if value != "" { + return value + } + } + return "" +} diff --git a/internal/service/dailyusage/handler.go b/internal/service/dailyusage/handler.go new file mode 100644 index 0000000..92b1b6c --- /dev/null +++ b/internal/service/dailyusage/handler.go @@ -0,0 +1,252 @@ +// Package dailyusage implements the zset.Handler for previous-day usage. +// +// It queries one card per task through the upstream gateway (the same path the +// status sync uses), validates the returned period and identity, and writes the +// usage to the TDengine daily table. +// +// The failure classifier is intentionally separate from the monthly-usage one +// and deliberately conservative: only errors that are known to be transient are +// retried. An unknown carrier code is parked as a terminal business failure +// instead of being retried, because a retry storm against a rate-limited carrier +// is worse than a card that needs manual review (see the 2026-09-16 disk-full +// incident). Like the monthly classifier, it never inspects the error text. +package dailyusage + +import ( + "context" + "fmt" + "strings" + "time" + + zsetconsumer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consumer/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +const ( + // errCodeTelecomChannelLimited is the carrier error code that means the + // whole channel is throttled or disabled. + errCodeTelecomChannelLimited = "S00004" // chanGatewayInternalError is the gateway code for transport, HTTP and + // response-decoding failures. An empty errCode distinguishes those from a + // carrier business result that happens to use the same number. + chanGatewayInternalError = 9999 + // tokenExpiredCode is the carrier code for an expired upstream token. The + // gateway refreshes the token once; a later round can then succeed. + tokenExpiredCode = 12021 + + mobileBusyFirstCode = 13000 + mobileBusyLastCode = 13007 + mobileBusyClass = "mobile_busy" +) + +// mobileUsageTemplates identifies the China Mobile OneLink implementations. +// Carrier identity is resolved from the channel template rather than from the +// channel name or the error text, both of which are unstable. +// +// The set mirrors the OneLink registration table in the upstream proxy and is +// kept in sync with the monthly-usage classifier +// (service/monthlyusage/handler.go). Unknown templates stay "generic" and are +// never classified as mobile. +var mobileUsageTemplates = map[string]struct{}{ + "sim_yd_v5": {}, + "sim_yd_mock_v5": {}, + "sim_sdtywl_v1": {}, + "sim_onelink_tengyu_v1": {}, + "sim_onelink_xuzhou_mobile_v1": {}, + "sim_onelink_lelian_v1": {}, + "sim_onelink_qianxun_v1": {}, + "sim_onelink_henan_xinjing_v1": {}, + "sim_henan_xinjing_direct_v1": {}, + "sim_onelink_zibo_v1": {}, + "sim_tibiot_v1": {}, + "sim_qianxun_v3": {}, + "sim_yunda_v1": {}, + "sim_mingqian_v3": {}, + "sim_daihe_v1": {}, + "sim_onelink_daihe_v1": {}, + "sim_onelink_daliuliang_v1": {}, + "sim__daliuliang_v1": {}, + "sim_onelink_mobile_public_v1": {}, + "sim_onelink_kedang_v1": {}, + "sim_onelink_beixin_v1": {}, + "lanshang_8000": {}, + "lanshang_8003": {}, +} + +// usageCarrierForTemplate resolves a channel template to its carrier. +// +// Telecom is detected through model.IsTelecomTemplate so every worker path +// shares one definition (it covers both sim_dx_v1 and sim_dx_v3). Hard-coding +// the v1 template here would silently classify a v3 channel as generic, which +// in turn would drop its S00004 channel-blocking behaviour. +func usageCarrierForTemplate(tempKey string) string { + tempKey = strings.ToLower(strings.TrimSpace(tempKey)) + if model.IsTelecomTemplate(tempKey) { + return "telecom" + } + if _, ok := mobileUsageTemplates[tempKey]; ok { + return "mobile" + } + return "generic" +} + +// classifyDailyUsageFailure maps one upstream item to the queue semantics owned +// by this service. +// +// message is accepted only so callers can log it; it never influences the +// decision, because text matching was the root cause of the 2026-09-14 channel +// retry loop. The default is a non-retryable business failure: a wrong retry +// multiplies carrier throttling, while a missing retry costs one manual replay. +func classifyDailyUsageFailure(tempKey string, code int, errCode string) (errorClass string, retryable, blockChannel bool) { + carrier := usageCarrierForTemplate(tempKey) + switch { + case carrier == "telecom" && strings.EqualFold(strings.TrimSpace(errCode), errCodeTelecomChannelLimited): + return "channel_limited", false, true + case carrier == "mobile" && code >= mobileBusyFirstCode && code <= mobileBusyLastCode: + return mobileBusyClass, true, false + case code == tokenExpiredCode: + return "token_expired", true, false + case code == chanGatewayInternalError && strings.TrimSpace(errCode) == "": + return "network", true, false + default: + return "business", false, false + } +} + +// UsageClient is the upstream surface needed by the handler. +type UsageClient interface { + QuerySimDatausageDaily(context.Context, *model.SimDataUsageDailyInput) ([]model.SimBaseRes[model.SimDataUsageDailyOutputItem], error) +} + +// UsageRepository persists the daily usage rows. +type UsageRepository interface { + UpdateDaily(context.Context, []model.TrafficUsageDailyItem) error +} + +type Handler struct { + client UsageClient + repo UsageRepository +} + +func NewHandler(client UsageClient, repo UsageRepository) (*Handler, error) { + if client == nil || repo == nil { + return nil, fmt.Errorf("daily usage handler dependencies are incomplete") + } + return &Handler{client: client, repo: repo}, nil +} + +// Query performs one upstream daily-usage request for a single card. +// +// task.QueryMonth carries the run's period identifier: for a daily run that is +// yyyyMMdd, and it is sent upstream verbatim as queryDate. +func (h *Handler) Query(ctx context.Context, task queue.Task) zsetconsumer.Result { + result := zsetconsumer.Result{Task: task} + queryDate := strings.TrimSpace(task.QueryMonth) + if queryDate == "" { + result.ErrorClass = "protocol" + result.Message = "daily usage task is missing its query date" + return result + } + output, err := h.client.QuerySimDatausageDaily(ctx, &model.SimDataUsageDailyInput{ + SimKeys: []*model.SimKey{{ + ChanKey: task.ChanKey, + ICCID: task.ICCID, + IMSI: task.IMSI, + MSISDN: task.MSISDN, + }}, + ReqData: model.SimDataUsageDailyInputItem{QueryDate: queryDate}, + }) + if err != nil { + result.ErrorClass = "network" + result.Retryable = true + result.Message = err.Error() + return result + } + if len(output) == 0 || output[0].SimKey == nil { + result.ErrorClass = "protocol" + result.Retryable = true + result.Message = "chan returned empty daily usage result" + return result + } + item := output[0] + if task.ICCID != "" && item.SimKey.ICCID != task.ICCID { + result.ErrorClass = "protocol" + result.Retryable = true + result.Message = "chan returned mismatched card identifier" + return result + } + result.Code = item.Code + result.Message = item.Message + result.ErrCode = item.ErrCode + if item.Code != 0 { + result.ErrorClass, result.Retryable, result.BlockChannel = classifyDailyUsageFailure(task.TempKey, item.Code, item.ErrCode) + return result + } + // A successful code with a different (or missing) period would write the + // wrong day, so it is treated as a protocol failure rather than silently + // accepted. This mirrors the legacy admin reconcile requirement. + if period, ok := normalizeDailyQueryDate(item.ResData.QueryDate); !ok || period != queryDate { + result.ErrorClass = "protocol" + result.Retryable = true + result.Message = "chan returned mismatched daily usage period" + return result + } + result.Success = true + result.UsageMB = item.ResData.DataAmount + return result +} + +// Apply writes every successful result to the TDengine daily table. +// +// The timestamp is derived from the run period rather than from the response, so +// every card in a run lands on the same 23:59:59.000 row that the dashboard and +// the diagnosis page read. +func (h *Handler) Apply(ctx context.Context, results []zsetconsumer.Result) error { + items := make([]model.TrafficUsageDailyItem, 0, len(results)) + for _, result := range results { + if !result.Success { + continue + } + date, ok := parseDailyQueryDate(strings.TrimSpace(result.Task.QueryMonth)) + if !ok { + return fmt.Errorf("daily usage task %s has an invalid period %q", result.Task.TaskID, result.Task.QueryMonth) + } + items = append(items, model.TrafficUsageDailyItem{ + TrafficUsageBase: model.TrafficUsageBase{ + Iccid: result.Task.ICCID, + UsageMb: result.UsageMB, + PackageCode: result.Task.PackageCode, + }, + Date: date, + }) + } + return h.repo.UpdateDaily(ctx, items) +} + +// ShouldRetryNextRound is the daily-usage retry policy. It is passed to the +// run-aware ZSet consumer only; it keeps the classifier's verdict authoritative. +func ShouldRetryNextRound(result zsetconsumer.Result) bool { + return !result.Success && result.Retryable +} + +var dailyUsageLocation = time.FixedZone("UTC+8", 8*60*60) + +// normalizeDailyQueryDate canonicalizes an upstream period to yyyyMMdd and +// reports whether the value was interpretable at all. +func normalizeDailyQueryDate(value string) (string, bool) { + for _, layout := range []string{"20060102", "2006-01-02"} { + parsed, err := time.ParseInLocation(layout, strings.TrimSpace(value), dailyUsageLocation) + if err == nil { + return parsed.Format("20060102"), true + } + } + return "", false +} + +func parseDailyQueryDate(value string) (time.Time, bool) { + parsed, err := time.ParseInLocation("20060102", value, dailyUsageLocation) + if err != nil { + return time.Time{}, false + } + return parsed, true +} diff --git a/internal/service/dailyusage/handler_test.go b/internal/service/dailyusage/handler_test.go new file mode 100644 index 0000000..6c92658 --- /dev/null +++ b/internal/service/dailyusage/handler_test.go @@ -0,0 +1,298 @@ +package dailyusage + +import ( + "context" + "errors" + "testing" + + zsetconsumer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consumer/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +type fakeClient struct { + output []model.SimBaseRes[model.SimDataUsageDailyOutputItem] + err error + // gotQueryDate records the period the handler sent upstream so a test can + // prove the task period is forwarded verbatim. + gotQueryDate string +} + +func (c *fakeClient) QuerySimDatausageDaily(_ context.Context, input *model.SimDataUsageDailyInput) ([]model.SimBaseRes[model.SimDataUsageDailyOutputItem], error) { + c.gotQueryDate = input.ReqData.QueryDate + return c.output, c.err +} + +type fakeRepo struct { + items []model.TrafficUsageDailyItem + err error +} + +func (r *fakeRepo) UpdateDaily(_ context.Context, items []model.TrafficUsageDailyItem) error { + r.items = append(r.items, items...) + return r.err +} + +func dailyTask() queue.Task { + return queue.Task{ + TaskID: "run:op:42", + RunID: "run", + Operation: model.OperationPreviousDayUsageFullSync, + QueryMonth: "20261009", + ChanKey: "217", + TempKey: "sim_yd_v5", + CardID: 42, + ICCID: "89860042", + PackageCode: "PKG-A", + } +} + +func TestQueryForwardsTaskPeriodUpstream(t *testing.T) { + client := &fakeClient{output: []model.SimBaseRes[model.SimDataUsageDailyOutputItem]{{ + Code: 0, + SimKey: &model.SimKey{ICCID: "89860042"}, + ResData: model.SimDataUsageDailyOutputItem{ + QueryDate: "20261009", + DataAmount: 12.5, + }, + }}} + handler, err := NewHandler(client, &fakeRepo{}) + if err != nil { + t.Fatal(err) + } + result := handler.Query(context.Background(), dailyTask()) + if !result.Success { + t.Fatalf("expected success, got class=%s message=%s", result.ErrorClass, result.Message) + } + if client.gotQueryDate != "20261009" { + t.Fatalf("upstream queryDate = %q, want the task period 20261009", client.gotQueryDate) + } + if result.UsageMB != 12.5 { + t.Fatalf("usage = %v, want 12.5", result.UsageMB) + } +} + +func TestQueryAcceptsEquivalentPeriodFormats(t *testing.T) { + for _, period := range []string{"20261009", "2026-10-09"} { + client := &fakeClient{output: []model.SimBaseRes[model.SimDataUsageDailyOutputItem]{{ + Code: 0, + SimKey: &model.SimKey{ICCID: "89860042"}, + ResData: model.SimDataUsageDailyOutputItem{QueryDate: period, DataAmount: 1}, + }}} + handler, _ := NewHandler(client, &fakeRepo{}) + if result := handler.Query(context.Background(), dailyTask()); !result.Success { + t.Fatalf("period %q was rejected: class=%s message=%s", period, result.ErrorClass, result.Message) + } + } +} + +func TestQueryRejectsMismatchedPeriod(t *testing.T) { + client := &fakeClient{output: []model.SimBaseRes[model.SimDataUsageDailyOutputItem]{{ + Code: 0, + SimKey: &model.SimKey{ICCID: "89860042"}, + ResData: model.SimDataUsageDailyOutputItem{QueryDate: "20261008", DataAmount: 1}, + }}} + handler, _ := NewHandler(client, &fakeRepo{}) + result := handler.Query(context.Background(), dailyTask()) + if result.Success { + t.Fatal("a mismatched period must not be accepted") + } + if result.ErrorClass != "protocol" || !result.Retryable { + t.Fatalf("class=%s retryable=%v, want protocol/retryable", result.ErrorClass, result.Retryable) + } +} + +func TestQueryRejectsMissingPeriodEcho(t *testing.T) { + // A channel that omits QueryDate from a successful response must fail too: + // the task period would be unverifiable, so writing the row risks wrong-day + // data. This mirrors the admin reconcile requirement. + client := &fakeClient{output: []model.SimBaseRes[model.SimDataUsageDailyOutputItem]{{ + Code: 0, + SimKey: &model.SimKey{ICCID: "89860042"}, + ResData: model.SimDataUsageDailyOutputItem{DataAmount: 1}, + }}} + handler, _ := NewHandler(client, &fakeRepo{}) + result := handler.Query(context.Background(), dailyTask()) + if result.Success { + t.Fatal("a missing period echo must not be accepted") + } + if result.ErrorClass != "protocol" || !result.Retryable { + t.Fatalf("class=%s retryable=%v, want protocol/retryable", result.ErrorClass, result.Retryable) + } +} + +func TestQueryRejectsMismatchedICCID(t *testing.T) { + client := &fakeClient{output: []model.SimBaseRes[model.SimDataUsageDailyOutputItem]{{ + Code: 0, + SimKey: &model.SimKey{ICCID: "89869999"}, + ResData: model.SimDataUsageDailyOutputItem{QueryDate: "20261009"}, + }}} + handler, _ := NewHandler(client, &fakeRepo{}) + result := handler.Query(context.Background(), dailyTask()) + if result.Success || result.ErrorClass != "protocol" || !result.Retryable { + t.Fatalf("class=%s retryable=%v, want protocol/retryable", result.ErrorClass, result.Retryable) + } +} + +func TestQueryRejectsEmptyResult(t *testing.T) { + handler, _ := NewHandler(&fakeClient{}, &fakeRepo{}) + result := handler.Query(context.Background(), dailyTask()) + if result.Success || result.ErrorClass != "protocol" || !result.Retryable { + t.Fatalf("class=%s retryable=%v, want protocol/retryable", result.ErrorClass, result.Retryable) + } +} + +func TestQueryTreatsTransportErrorAsRetryableNetwork(t *testing.T) { + handler, _ := NewHandler(&fakeClient{err: errors.New("connection refused")}, &fakeRepo{}) + result := handler.Query(context.Background(), dailyTask()) + if result.ErrorClass != "network" || !result.Retryable { + t.Fatalf("class=%s retryable=%v, want network/retryable", result.ErrorClass, result.Retryable) + } +} + +func TestQueryRejectsTaskWithoutPeriod(t *testing.T) { + handler, _ := NewHandler(&fakeClient{}, &fakeRepo{}) + task := dailyTask() + task.QueryMonth = "" + result := handler.Query(context.Background(), task) + if result.Success || result.Retryable { + t.Fatalf("a task without a period must fail without retrying: %+v", result) + } +} + +func TestApplyWritesPackageCodeAndEndOfDayTimestamp(t *testing.T) { + repo := &fakeRepo{} + handler, _ := NewHandler(&fakeClient{}, repo) + err := handler.Apply(context.Background(), []zsetconsumer.Result{ + {Task: dailyTask(), Success: true, UsageMB: 7.25}, + {Task: dailyTask(), Success: false}, + }) + if err != nil { + t.Fatal(err) + } + if len(repo.items) != 1 { + t.Fatalf("wrote %d items, want only the successful one", len(repo.items)) + } + item := repo.items[0] + if item.Iccid != "89860042" || item.UsageMb != 7.25 || item.PackageCode != "PKG-A" { + t.Fatalf("item = %+v, want iccid/usage/packageCode carried from the task", item) + } + // The handler supplies the day; repository.snapshotDay turns it into the + // 23:59:59.000 row (covered by repository_test.go). Assert the day here so a + // period mix-up cannot slip through. + if got := item.Date.Format("2006-01-02"); got != "2026-10-09" { + t.Fatalf("date = %s, want the run period 2026-10-09", got) + } +} + +func TestApplyRejectsInvalidPeriod(t *testing.T) { + handler, _ := NewHandler(&fakeClient{}, &fakeRepo{}) + task := dailyTask() + task.QueryMonth = "2026-10-09" + if err := handler.Apply(context.Background(), []zsetconsumer.Result{{Task: task, Success: true}}); err == nil { + t.Fatal("expected an unparseable period to fail the write") + } +} + +func TestClassifyDailyUsageFailure(t *testing.T) { + tests := []struct { + name string + tempKey string + code int + errCode string + wantClass string + wantRetry bool + wantBlocked bool + }{ + {name: "telecom channel limited", tempKey: "sim_dx_v1", code: 500, errCode: "S00004", wantClass: "channel_limited", wantBlocked: true}, + {name: "telecom channel limited lowercase", tempKey: "sim_dx_v1", code: 500, errCode: "s00004", wantClass: "channel_limited", wantBlocked: true}, + // sim_dx_v3 is a telecom template too. It is resolved through + // model.IsTelecomTemplate so a v3 channel keeps S00004 blocking instead + // of falling through to generic and retrying every card to exhaustion. + {name: "telecom v3 channel limited", tempKey: "sim_dx_v3", code: 500, errCode: "S00004", wantClass: "channel_limited", wantBlocked: true}, + {name: "telecom v3 template is not treated as mobile", tempKey: "sim_dx_v3", code: 13000, wantClass: "business"}, + {name: "mobile busy first code", tempKey: "sim_yd_v5", code: 13000, wantClass: "mobile_busy", wantRetry: true}, + {name: "mobile busy last code", tempKey: "sim_yd_v5", code: 13007, wantClass: "mobile_busy", wantRetry: true}, + {name: "mobile one past the busy range", tempKey: "sim_yd_v5", code: 13008, wantClass: "business"}, + {name: "mobile one below the busy range", tempKey: "sim_yd_v5", code: 12999, wantClass: "business"}, + {name: "busy code on a non-mobile template", tempKey: "sim_qianxun_v3_bogus", code: 13000, wantClass: "business"}, + {name: "token expired", tempKey: "sim_yd_v5", code: 12021, wantClass: "token_expired", wantRetry: true}, + {name: "gateway internal without errCode", tempKey: "sim_yd_v5", code: 9999, errCode: "", wantClass: "network", wantRetry: true}, + {name: "gateway code with a carrier errCode stays business", tempKey: "sim_yd_v5", code: 9999, errCode: "E42", wantClass: "business"}, + {name: "telecom S00004 without the telecom template is not channel limited", tempKey: "sim_yd_v5", code: 500, errCode: "S00004", wantClass: "business"}, + {name: "unknown code is not retried", tempKey: "sim_yd_v5", code: 4242, wantClass: "business"}, + {name: "unknown code on an unknown template is not retried", tempKey: "some_new_channel_v9", code: 13000, wantClass: "business"}, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + class, retry, blocked := classifyDailyUsageFailure(test.tempKey, test.code, test.errCode) + if class != test.wantClass || retry != test.wantRetry || blocked != test.wantBlocked { + t.Fatalf("classify(%q, %d, %q) = (%s, %v, %v), want (%s, %v, %v)", + test.tempKey, test.code, test.errCode, class, retry, blocked, test.wantClass, test.wantRetry, test.wantBlocked) + } + }) + } +} + +// TestClassifyIgnoresMessageText pins the 2026-09-14 discipline: classification +// must never depend on carrier text, because text matching caused the channel +// retry loop that the rule was introduced to prevent. +func TestClassifyIgnoresMessageText(t *testing.T) { + // A "busy"-looking message on an unknown code must stay non-retryable. + if _, retry, _ := classifyDailyUsageFailure("sim_yd_v5", 4242, ""); retry { + t.Fatal("an unknown code must not become retryable because its message looked transient") + } + // And a busy code must stay retryable regardless of what the text says. + if _, retry, _ := classifyDailyUsageFailure("sim_yd_v5", 13000, ""); !retry { + t.Fatal("a mobile busy code must stay retryable") + } +} + +func TestShouldRetryNextRoundFollowsClassifier(t *testing.T) { + if ShouldRetryNextRound(zsetconsumer.Result{Success: true}) { + t.Fatal("a successful result must not be retried") + } + if !ShouldRetryNextRound(zsetconsumer.Result{Retryable: true}) { + t.Fatal("a retryable failure must be deferred to the next round") + } + if ShouldRetryNextRound(zsetconsumer.Result{Retryable: false}) { + t.Fatal("a terminal failure must not be retried") + } +} + +func TestUsageCarrierForTemplateMatchesSharedDefinition(t *testing.T) { + // The daily classifier must agree with model.IsTelecomTemplate for every + // telecom template the rest of the worker recognises. A drift here means a + // channel silently loses its S00004 blocking behaviour. + for _, tempKey := range []string{"sim_dx_v1", "sim_dx_v3", "SIM_DX_V1", " sim_dx_v3 "} { + if got := usageCarrierForTemplate(tempKey); got != "telecom" { + t.Errorf("usageCarrierForTemplate(%q) = %q, want telecom", tempKey, got) + } + } + for _, tempKey := range []string{"sim_yd_v5", "sim_onelink_daihe_v1"} { + if got := usageCarrierForTemplate(tempKey); got != "mobile" { + t.Errorf("usageCarrierForTemplate(%q) = %q, want mobile", tempKey, got) + } + } + for _, tempKey := range []string{"", "some_new_channel_v9", "sim_dx_v2"} { + if got := usageCarrierForTemplate(tempKey); got != "generic" { + t.Errorf("usageCarrierForTemplate(%q) = %q, want generic", tempKey, got) + } + } +} + +func TestQueryBlockChannelPropagatesForTelecomS00004(t *testing.T) { + client := &fakeClient{output: []model.SimBaseRes[model.SimDataUsageDailyOutputItem]{{ + Code: 500, + ErrCode: "S00004", + Message: "channel disabled", + SimKey: &model.SimKey{ICCID: "89860042"}, + }}} + handler, _ := NewHandler(client, &fakeRepo{}) + task := dailyTask() + task.TempKey = "sim_dx_v1" + result := handler.Query(context.Background(), task) + if result.ErrorClass != "channel_limited" || !result.BlockChannel || result.Retryable { + t.Fatalf("telecom S00004 must block the channel without retrying: %+v", result) + } +} diff --git a/internal/service/data/executor.go b/internal/service/data/executor.go new file mode 100644 index 0000000..57b578e --- /dev/null +++ b/internal/service/data/executor.go @@ -0,0 +1,398 @@ +package data + +import ( + "context" + "sort" + "strconv" + "strings" + "sync" + + "github.com/gogf/gf/v2/errors/gerror" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +type executor struct { + repository Repository + upstream Upstream +} + +func NewExecutor(repository Repository, upstream Upstream) Executor { + return &executor{repository: repository, upstream: upstream} +} + +func (e *executor) Execute(ctx context.Context, batchNo string) error { + task, err := e.repository.Load(ctx, strings.TrimSpace(batchNo)) + if err != nil { + return gerror.Wrap(err, "load data service batch") + } + if task.Operation.BatchType == consts.BatchTypeDataCut { + return e.executeTrafficCutoff(ctx, task) + } + if !validateTask(task) { + if err = e.repository.SaveValidationFailure(ctx, task); err != nil { + return gerror.Wrap(err, "save data service validation failure") + } + return gerror.Wrap(e.repository.Finalize(ctx, task), "finalize data service validation failure") + } + + return e.executeValidated(ctx, task) +} + +func (e *executor) executeTrafficCutoff(ctx context.Context, task *Task) error { + if !validateTrafficCutoffTask(task) { + if err := e.repository.SaveValidationFailure(ctx, task); err != nil { + return gerror.Wrap(err, "save traffic cutoff validation failure") + } + return gerror.Wrap(e.repository.Finalize(ctx, task), "finalize traffic cutoff validation failure") + } + if err := e.refreshDataServiceState(ctx, task); err != nil { + return gerror.Wrap(err, "refresh data service state") + } + if err := e.saveInvalidTrafficCutoffRows(ctx, task); err != nil { + return err + } + if err := e.saveIdempotentCutoffSkips(ctx, task); err != nil { + return err + } + return e.executeValidated(ctx, task) +} + +func (e *executor) executeValidated(ctx context.Context, task *Task) error { + groups := groupByChanKey(task.Operation, task.Details) + keys := make([]string, 0, len(groups)) + for chanKey := range groups { + keys = append(keys, chanKey) + } + sort.Strings(keys) + for _, chanKey := range keys { + if err := e.changeGroup(ctx, task, groups[chanKey]); err != nil { + return err + } + } + return gerror.Wrap(e.repository.Finalize(ctx, task), "finalize data service batch") +} + +type dataServiceStateRefresher interface { + RefreshDataServiceState(ctx context.Context, task *Task) error +} + +func (e *executor) refreshDataServiceState(ctx context.Context, task *Task) error { + refresher, ok := e.repository.(dataServiceStateRefresher) + if !ok { + return nil + } + return refresher.RefreshDataServiceState(ctx, task) +} + +func (e *executor) saveIdempotentCutoffSkips(ctx context.Context, task *Task) error { + if task == nil || task.Operation.BatchType != consts.BatchTypeDataCut { + return nil + } + for _, detail := range task.Details { + if !skipUpstreamForIdempotentCutoff(task.Operation, detail) { + continue + } + resolved, saveErr := e.repository.SaveResult(ctx, task, detail, Outcome{ + Success: true, + Message: "data service is already disabled; upstream request was skipped", + }) + if saveErr != nil { + return gerror.Wrapf(saveErr, "save data service detail %d idempotent skip result", detail.ID) + } + detail.ApplyOutcome(resolved) + } + return nil +} + +func validateTask(task *Task) bool { + if task == nil || len(task.Details) == 0 { + return false + } + hasFailure := false + for _, detail := range task.Details { + if detail == nil { + hasFailure = true + continue + } + validateDetail(task.Operation, detail) + if detail.Reason != "" { + hasFailure = true + } + } + if !hasFailure { + task.ValidationOK = true + for _, detail := range task.Details { + if detail != nil { + detail.ValidationPassed = true + } + } + return true + } + for _, detail := range task.Details { + if detail != nil && detail.Reason == "" { + detail.Reason = "same batch contains validation failure; upstream request was skipped" + } + } + task.FailureReason = "batch validation failed; upstream request was skipped" + return false +} + +func validateTrafficCutoffTask(task *Task) bool { + if task == nil || len(task.Details) == 0 { + if task != nil { + task.FailureReason = "traffic cutoff batch has no valid details" + } + return false + } + hasSuccess := false + for _, detail := range task.Details { + if detail == nil { + continue + } + validateDetail(task.Operation, detail) + detail.ValidationPassed = detail.Reason == "" + if detail.ValidationPassed { + hasSuccess = true + } + } + if hasSuccess { + task.ValidationOK = true + if failed := countFailedDetails(task); failed > 0 { + task.FailureReason = strconv.Itoa(failed) + " SIM cards failed validation" + } else { + task.FailureReason = "" + } + return true + } + task.FailureReason = "traffic cutoff batch validation failed; upstream request was skipped" + return false +} + +func countFailedDetails(task *Task) int { + if task == nil { + return 0 + } + count := 0 + for _, detail := range task.Details { + if detail == nil || detail.Reason != "" { + count++ + } + } + return count +} + +func validateDetail(_ Operation, detail *Detail) { + if detail.Reason != "" { + return + } + if detail.Card == nil { + detail.Reason = "ICCID does not exist in SIM card records" + return + } + detail.MSISDN = strings.TrimSpace(detail.Card.MSISDN) + detail.IMSI = strings.TrimSpace(detail.Card.IMSI) + detail.ChanKey = strings.TrimSpace(detail.Card.ChanKey) + detail.StatusBefore = detail.Card.DataServiceStatus + detail.StatusAfter = detail.Card.DataServiceStatus + + if !detail.Authorized { + detail.Reason = "ICCID is outside the current operator authorization scope" + return + } + if detail.ChanKey == "" { + detail.Reason = "ICCID is missing chanKey" + return + } + if detail.InputMSISDN != "" && detail.InputMSISDN != detail.MSISDN { + detail.Reason = "MSISDN does not match the SIM card record" + return + } + if detail.InputIMSI != "" && detail.InputIMSI != detail.IMSI { + detail.Reason = "IMSI does not match the SIM card record" + return + } +} + +func groupByChanKey(operation Operation, details []*Detail) map[string][]*Detail { + groups := make(map[string][]*Detail) + for _, detail := range details { + if operation.BatchType == consts.BatchTypeDataCut && (detail == nil || !detail.ValidationPassed || detail.Reason != "") { + continue + } + if skipUpstreamForIdempotentCutoff(operation, detail) { + continue + } + groups[detail.ChanKey] = append(groups[detail.ChanKey], detail) + } + return groups +} + +func (e *executor) saveInvalidTrafficCutoffRows(ctx context.Context, task *Task) error { + if task == nil { + return nil + } + for _, detail := range task.Details { + if detail == nil || detail.ValidationPassed || detail.Reason == "" { + continue + } + resolved, saveErr := e.repository.SaveResult(ctx, task, detail, Outcome{ + Message: detail.Reason, + }) + if saveErr != nil { + return gerror.Wrapf(saveErr, "save traffic cutoff detail %d validation result", detail.ID) + } + detail.ApplyOutcome(resolved) + } + return nil +} + +func skipUpstreamForIdempotentCutoff(operation Operation, detail *Detail) bool { + return operation.BatchType == consts.BatchTypeDataCut && + detail != nil && + detail.ValidationPassed && + detail.StatusBefore == consts.SimDataServiceStatusDisabled && + detail.StatusAfter == consts.SimDataServiceStatusDisabled +} + +// changeGroup 把同一通道下的明细切成子批次并发调用上游，并在**每个子批次拿到结果后立即落库该子批的明细**。 +// +// 不再等所有子批次跑完才统一写库：大批量（如 2 万张 = 200 个子批）执行时，执行日志可以按子批 +// 实时反映进度，而不是在整个批次结束前始终全部停留在"执行中"。子批之间相互隔离——某个子批 +// 落库失败只记录错误、不影响其他子批继续办理与落库。 +func (e *executor) changeGroup(ctx context.Context, task *Task, details []*Detail) error { + subBatches := splitDetails(details, consts.BatchSubBatchSize) + + sem := make(chan struct{}, consts.BatchSubBatchConcurrency) + var wg sync.WaitGroup + var mu sync.Mutex + var firstErr error + + for _, sub := range subBatches { + wg.Add(1) + sem <- struct{}{} + go func(batch []*Detail) { + defer wg.Done() + defer func() { <-sem }() + if err := e.changeAndSaveSubBatch(ctx, task, batch); err != nil { + mu.Lock() + if firstErr == nil { + firstErr = err + } + mu.Unlock() + } + }(sub) + } + wg.Wait() + + return firstErr +} + +// changeAndSaveSubBatch 处理单个子批次：构建请求、调用上游，然后逐条落库本子批的明细。 +// 明细结果只在所属子批次落库，天然做到"查到一个子批就落库一个子批"。 +func (e *executor) changeAndSaveSubBatch(ctx context.Context, task *Task, details []*Detail) error { + for index, outcome := range e.changeSubBatch(ctx, task, details) { + detail := details[index] + resolved, saveErr := e.repository.SaveResult(ctx, task, detail, outcome) + if saveErr != nil { + return gerror.Wrapf(saveErr, "save data service detail %d result", detail.ID) + } + detail.ApplyOutcome(resolved) + } + return nil +} + +// changeSubBatch 处理单个子批次：构建请求并调用上游，返回与该子批次明细等长的 Outcome 切片。 +func (e *executor) changeSubBatch(ctx context.Context, task *Task, details []*Detail) []Outcome { + input := &model.SimCardApnChangeInput{ + SimKeys: make([]*model.SimKey, 0, len(details)), + ReqData: model.SimCardApnChangeInputItem{OperType: task.Operation.ApnOperType, ApnName: task.ApnName}, + } + for _, detail := range details { + input.SimKeys = append(input.SimKeys, &model.SimKey{ChanKey: detail.ChanKey, ICCID: detail.ICCID, MSISDN: detail.MSISDN, IMSI: detail.IMSI}) + } + response, err := e.upstream.ChangeSimApnFunction(ctx, input) + if err != nil { + outcomes := make([]Outcome, len(details)) + for index := range outcomes { + outcomes[index].Message = "upstream request failed: " + err.Error() + } + return outcomes + } + return mapOutcomes(details, response) +} + +// splitDetails 按固定大小把明细切成有序的子批次；空输入返回空切片。 +func splitDetails(details []*Detail, size int) [][]*Detail { + if size <= 0 { + size = 1 + } + batches := make([][]*Detail, 0, (len(details)+size-1)/size) + for i := 0; i < len(details); i += size { + end := i + size + if end > len(details) { + end = len(details) + } + batches = append(batches, details[i:end]) + } + return batches +} + +func mapOutcomes(details []*Detail, response []model.SimBaseRes[model.SimCardApnChangeOutputItem]) []Outcome { + outcomes := make([]Outcome, len(details)) + used := make([]bool, len(response)) + for detailIndex, detail := range details { + matched := -1 + for responseIndex, item := range response { + if !used[responseIndex] && sameSim(detail, item.SimKey) { + matched = responseIndex + break + } + } + if matched < 0 && detailIndex < len(response) && !used[detailIndex] && response[detailIndex].SimKey == nil { + matched = detailIndex + } + if matched < 0 { + outcomes[detailIndex] = Outcome{Message: "upstream did not return a result for this SIM card"} + continue + } + used[matched] = true + item := response[matched] + message := strings.TrimSpace(item.Message) + if item.Code != 0 && message == "" { + message = "upstream processing failed" + } + outcomes[detailIndex] = Outcome{Success: item.Code == 0, Code: item.Code, Message: message, OrderNum: strings.TrimSpace(item.ResData.OrderNum)} + } + return outcomes +} + +func sameSim(detail *Detail, key *model.SimKey) bool { + if detail == nil || key == nil { + return false + } + if value := strings.TrimSpace(key.ICCID); value != "" { + return value == detail.ICCID + } + if value := strings.TrimSpace(key.MSISDN); value != "" { + return value == detail.MSISDN + } + if value := strings.TrimSpace(key.IMSI); value != "" { + return value == detail.IMSI + } + return false +} + +func OperationByBatchType(batchType int) (Operation, error) { + switch batchType { + case consts.BatchTypeDataStart: + return Operation{BatchType: batchType, Name: "dataStart", ApnOperType: consts.SimApnOperTypeOpen, TargetStatus: consts.SimDataServiceStatusEnabled}, nil + case consts.BatchTypeDataCut: + return Operation{BatchType: batchType, Name: "trafficCutoff", ApnOperType: consts.SimApnOperTypeStop, TargetStatus: consts.SimDataServiceStatusDisabled}, nil + case consts.BatchTypeDataStop: + return Operation{BatchType: batchType, Name: "dataStop", ApnOperType: consts.SimApnOperTypeStop, TargetStatus: consts.SimDataServiceStatusDisabled}, nil + default: + return Operation{}, gerror.Newf("unsupported data service batch type %d", batchType) + } +} diff --git a/internal/service/data/executor_test.go b/internal/service/data/executor_test.go new file mode 100644 index 0000000..4a3a1fd --- /dev/null +++ b/internal/service/data/executor_test.go @@ -0,0 +1,524 @@ +package data + +import ( + "context" + "errors" + "fmt" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +func TestExecuteDataStartSuccess(t *testing.T) { + operation, err := OperationByBatchType(consts.BatchTypeDataStart) + if err != nil { + t.Fatal(err) + } + task := &Task{ + Batch: Batch{BatchNo: "D-1"}, + Operation: operation, + ApnName: consts.SimApnNameDefault, + Details: []*Detail{ + {ID: 1, ICCID: "iccid-1", Authorized: true, Card: &Card{ID: 11, ChanKey: "channel", DataServiceStatus: consts.SimDataServiceStatusDisabled}}, + }, + } + repo := &fakeRepository{task: task} + upstream := &fakeUpstream{response: []model.SimBaseRes[model.SimCardApnChangeOutputItem]{ + {Code: 0, Message: "ok", SimKey: &model.SimKey{ICCID: "iccid-1"}, ResData: model.SimCardApnChangeOutputItem{OrderNum: "order-1"}}, + }} + + if err = NewExecutor(repo, upstream).Execute(context.Background(), "D-1"); err != nil { + t.Fatal(err) + } + if upstream.calls != 1 { + t.Fatalf("upstream calls = %d, want 1", upstream.calls) + } + if upstream.input.ReqData.OperType != consts.SimApnOperTypeOpen || upstream.input.ReqData.ApnName != consts.SimApnNameDefault { + t.Fatalf("reqData = %+v", upstream.input.ReqData) + } + if len(repo.saved) != 1 || !repo.saved[0].Success || repo.saved[0].OrderNum != "order-1" { + t.Fatalf("saved = %+v", repo.saved) + } + if task.Details[0].StatusBefore != consts.SimDataServiceStatusDisabled { + t.Fatalf("StatusBefore = %d", task.Details[0].StatusBefore) + } + if !repo.finalized { + t.Fatal("batch was not finalized") + } +} + +func TestExecuteDataStopSuccess(t *testing.T) { + operation, err := OperationByBatchType(consts.BatchTypeDataStop) + if err != nil { + t.Fatal(err) + } + task := &Task{ + Batch: Batch{BatchNo: "D-2"}, + Operation: operation, + ApnName: consts.SimApnNameDefault, + Details: []*Detail{ + {ID: 1, ICCID: "iccid-1", Authorized: true, Card: &Card{ID: 11, ChanKey: "channel", DataServiceStatus: consts.SimDataServiceStatusEnabled}}, + }, + } + repo := &fakeRepository{task: task} + upstream := &fakeUpstream{response: []model.SimBaseRes[model.SimCardApnChangeOutputItem]{ + {Code: 0, Message: "ok", SimKey: &model.SimKey{ICCID: "iccid-1"}}, + }} + + if err = NewExecutor(repo, upstream).Execute(context.Background(), "D-2"); err != nil { + t.Fatal(err) + } + if upstream.calls != 1 || upstream.input.ReqData.OperType != consts.SimApnOperTypeStop { + t.Fatalf("upstream calls = %d, operType = %s", upstream.calls, upstream.input.ReqData.OperType) + } + if len(repo.saved) != 1 || !repo.saved[0].Success { + t.Fatalf("saved = %+v", repo.saved) + } +} + +func TestExecuteTrafficCutoffSuccess(t *testing.T) { + operation, err := OperationByBatchType(consts.BatchTypeDataCut) + if err != nil { + t.Fatal(err) + } + task := &Task{ + Batch: Batch{BatchNo: "DCT-1"}, + Operation: operation, + ApnName: consts.SimApnNameDefault, + Details: []*Detail{ + {ID: 1, ICCID: "iccid-1", Authorized: true, Card: &Card{ID: 11, ChanKey: "channel", DataServiceStatus: consts.SimDataServiceStatusEnabled}}, + }, + } + repo := &fakeRepository{task: task} + upstream := &fakeUpstream{response: []model.SimBaseRes[model.SimCardApnChangeOutputItem]{ + {Code: 0, Message: "ok", SimKey: &model.SimKey{ICCID: "iccid-1"}}, + }} + + if err = NewExecutor(repo, upstream).Execute(context.Background(), "DCT-1"); err != nil { + t.Fatal(err) + } + if upstream.calls != 1 || upstream.input.ReqData.OperType != consts.SimApnOperTypeStop { + t.Fatalf("upstream calls = %d, operType = %s", upstream.calls, upstream.input.ReqData.OperType) + } + if len(repo.saved) != 1 || !repo.saved[0].Success { + t.Fatalf("saved = %+v", repo.saved) + } +} + +func TestExecuteTrafficCutoffRefreshesCurrentStatusBeforeUpstream(t *testing.T) { + operation, err := OperationByBatchType(consts.BatchTypeDataCut) + if err != nil { + t.Fatal(err) + } + task := &Task{ + Batch: Batch{BatchNo: "DCT-3"}, + Operation: operation, + ApnName: consts.SimApnNameDefault, + Details: []*Detail{ + {ID: 1, ICCID: "iccid-1", Authorized: true, Card: &Card{ID: 11, ChanKey: "channel", DataServiceStatus: consts.SimDataServiceStatusEnabled}}, + }, + } + repo := &fakeRepository{ + task: task, + refresh: func(task *Task) { + if len(task.Details) == 0 || task.Details[0] == nil || task.Details[0].Card == nil { + return + } + task.Details[0].Card.DataServiceStatus = consts.SimDataServiceStatusDisabled + task.Details[0].StatusBefore = consts.SimDataServiceStatusDisabled + task.Details[0].StatusAfter = consts.SimDataServiceStatusDisabled + }, + } + upstream := &fakeUpstream{responseError: errors.New("must not be called")} + + if err = NewExecutor(repo, upstream).Execute(context.Background(), "DCT-3"); err != nil { + t.Fatal(err) + } + if upstream.calls != 0 { + t.Fatalf("upstream calls = %d, want 0", upstream.calls) + } + if len(repo.saved) != 1 || !repo.saved[0].Success { + t.Fatalf("saved = %+v", repo.saved) + } +} + +func TestExecuteTrafficCutoffSkipsAlreadyDisabledCard(t *testing.T) { + operation, err := OperationByBatchType(consts.BatchTypeDataCut) + if err != nil { + t.Fatal(err) + } + task := &Task{ + Batch: Batch{BatchNo: "DCT-2"}, + Operation: operation, + ApnName: consts.SimApnNameDefault, + Details: []*Detail{ + {ID: 1, ICCID: "iccid-1", Authorized: true, Card: &Card{ID: 11, ChanKey: "channel", DataServiceStatus: consts.SimDataServiceStatusDisabled}}, + }, + } + repo := &fakeRepository{task: task} + upstream := &fakeUpstream{responseError: errors.New("must not be called")} + + if err = NewExecutor(repo, upstream).Execute(context.Background(), "DCT-2"); err != nil { + t.Fatal(err) + } + if upstream.calls != 0 { + t.Fatalf("upstream calls = %d, want 0", upstream.calls) + } + if len(repo.saved) != 1 || !repo.saved[0].Success { + t.Fatalf("saved = %+v", repo.saved) + } + if task.Details[0].Reason != "" { + t.Fatalf("already disabled cutoff should be settled as success, reason = %q", task.Details[0].Reason) + } + if !repo.finalized { + t.Fatal("batch was not finalized") + } +} + +func TestExecuteTrafficCutoffValidationFailureAllowsValidRows(t *testing.T) { + operation, err := OperationByBatchType(consts.BatchTypeDataCut) + if err != nil { + t.Fatal(err) + } + task := &Task{ + Batch: Batch{BatchNo: "DCT-4"}, + Operation: operation, + ApnName: consts.SimApnNameDefault, + Details: []*Detail{ + {ID: 1, ICCID: "iccid-1", Authorized: false, Card: &Card{ID: 11, ChanKey: "channel", DataServiceStatus: consts.SimDataServiceStatusEnabled}}, + {ID: 2, ICCID: "iccid-2", Authorized: true, Card: &Card{ID: 12, ChanKey: "channel", DataServiceStatus: consts.SimDataServiceStatusEnabled}}, + }, + } + repo := &fakeRepository{task: task} + upstream := &fakeUpstream{response: []model.SimBaseRes[model.SimCardApnChangeOutputItem]{ + {Code: 0, Message: "ok", SimKey: &model.SimKey{ICCID: "iccid-2"}}, + }} + + if err = NewExecutor(repo, upstream).Execute(context.Background(), "DCT-4"); err != nil { + t.Fatal(err) + } + if upstream.calls != 1 { + t.Fatalf("upstream calls = %d, want 1", upstream.calls) + } + if len(upstream.input.SimKeys) != 1 || upstream.input.SimKeys[0].ICCID != "iccid-2" { + t.Fatalf("upstream input = %+v", upstream.input.SimKeys) + } + if len(repo.saved) != 2 { + t.Fatalf("saved count = %d, want 2; saved = %+v", len(repo.saved), repo.saved) + } + if repo.saved[0].Success || repo.saved[0].Message != "ICCID is outside the current operator authorization scope" { + t.Fatalf("invalid row result = %+v", repo.saved[0]) + } + if !repo.saved[1].Success { + t.Fatalf("valid row result = %+v", repo.saved[1]) + } + if task.Details[0].ValidationPassed || !task.Details[1].ValidationPassed { + t.Fatalf("validation flags = (%v, %v)", task.Details[0].ValidationPassed, task.Details[1].ValidationPassed) + } + if !repo.finalized { + t.Fatal("batch was not finalized") + } +} + +func TestExecuteSameLocalStatusStillCallsUpstream(t *testing.T) { + operation, err := OperationByBatchType(consts.BatchTypeDataStart) + if err != nil { + t.Fatal(err) + } + task := &Task{ + Batch: Batch{BatchNo: "D-3"}, + Operation: operation, + ApnName: consts.SimApnNameDefault, + Details: []*Detail{ + {ID: 1, ICCID: "iccid-1", Authorized: true, Card: &Card{ID: 11, ChanKey: "channel", DataServiceStatus: consts.SimDataServiceStatusEnabled}}, + }, + } + repo := &fakeRepository{task: task} + upstream := &fakeUpstream{response: []model.SimBaseRes[model.SimCardApnChangeOutputItem]{ + {Code: 0, Message: "ok", SimKey: &model.SimKey{ICCID: "iccid-1"}}, + }} + + if err = NewExecutor(repo, upstream).Execute(context.Background(), "D-3"); err != nil { + t.Fatal(err) + } + if upstream.calls != 1 { + t.Fatalf("upstream calls = %d, want 1 even when local status already matches", upstream.calls) + } + if task.Details[0].Reason != "" { + t.Fatalf("same local status must not be a validation blocker, reason = %q", task.Details[0].Reason) + } + if len(repo.saved) != 1 || !repo.saved[0].Success { + t.Fatalf("saved = %+v", repo.saved) + } +} + +func TestExecuteValidationFailureSkipsUpstream(t *testing.T) { + operation, err := OperationByBatchType(consts.BatchTypeDataStop) + if err != nil { + t.Fatal(err) + } + task := &Task{ + Batch: Batch{BatchNo: "D-4"}, + Operation: operation, + ApnName: consts.SimApnNameDefault, + Details: []*Detail{ + {ID: 1, ICCID: "iccid-1", Authorized: true, Card: &Card{ID: 11, ChanKey: "channel", DataServiceStatus: consts.SimDataServiceStatusEnabled}}, + {ID: 2, ICCID: "iccid-2", Authorized: false, Card: &Card{ID: 12, ChanKey: "channel", DataServiceStatus: consts.SimDataServiceStatusEnabled}}, + }, + } + repo := &fakeRepository{task: task} + upstream := &fakeUpstream{responseError: errors.New("must not be called")} + + if err = NewExecutor(repo, upstream).Execute(context.Background(), "D-4"); err != nil { + t.Fatal(err) + } + if upstream.calls != 0 { + t.Fatalf("upstream calls = %d, want 0", upstream.calls) + } + if !repo.validationSaved || !repo.finalized { + t.Fatal("validation failure was not persisted and finalized") + } + if task.Details[0].Reason == "" || task.Details[1].Reason == "" { + t.Fatal("every detail must receive a validation failure reason") + } +} + +func TestExecuteUpstreamPartialSuccess(t *testing.T) { + operation, err := OperationByBatchType(consts.BatchTypeDataStart) + if err != nil { + t.Fatal(err) + } + task := &Task{ + Batch: Batch{BatchNo: "D-5"}, + Operation: operation, + ApnName: consts.SimApnNameDefault, + Details: []*Detail{ + {ID: 1, ICCID: "iccid-1", MSISDN: "msisdn-1", IMSI: "imsi-1", Authorized: true, Card: &Card{ID: 11, ChanKey: "channel", MSISDN: "msisdn-1", IMSI: "imsi-1", DataServiceStatus: consts.SimDataServiceStatusDisabled}}, + {ID: 2, ICCID: "iccid-2", MSISDN: "msisdn-2", IMSI: "imsi-2", Authorized: true, Card: &Card{ID: 12, ChanKey: "channel", MSISDN: "msisdn-2", IMSI: "imsi-2", DataServiceStatus: consts.SimDataServiceStatusDisabled}}, + }, + } + repo := &fakeRepository{task: task} + upstream := &fakeUpstream{response: []model.SimBaseRes[model.SimCardApnChangeOutputItem]{ + {Code: 0, Message: "ok", SimKey: nil, ResData: model.SimCardApnChangeOutputItem{OrderNum: "order-1"}}, + {Code: 9001, Message: "denied", SimKey: &model.SimKey{ICCID: "iccid-2"}, ResData: model.SimCardApnChangeOutputItem{OrderNum: "order-2"}}, + }} + + if err = NewExecutor(repo, upstream).Execute(context.Background(), "D-5"); err != nil { + t.Fatal(err) + } + if len(repo.saved) != 2 || !repo.saved[0].Success || repo.saved[0].OrderNum != "order-1" { + t.Fatalf("first result = %+v", repo.saved) + } + if repo.saved[1].Success || repo.saved[1].Code != 9001 || repo.saved[1].Message != "denied" || repo.saved[1].OrderNum != "order-2" { + t.Fatalf("second result = %+v", repo.saved[1]) + } + if !repo.finalized { + t.Fatal("batch was not finalized") + } +} + +// 大批量场景下，每个子批次一拿到结果就立即落库，不必等所有子批次跑完。 +func TestExecuteSavesEachSubBatchAsSoonAsItSettles(t *testing.T) { + operation, err := OperationByBatchType(consts.BatchTypeDataStart) + if err != nil { + t.Fatal(err) + } + // 201 张 = 3 个子批（100 / 100 / 1）；并发度为 5，三个子批会同时发起。 + details := make([]*Detail, 0, 201) + for i := 0; i < 201; i++ { + details = append(details, &Detail{ + ID: int64(i + 1), + ICCID: fmt.Sprintf("iccid-%03d", i), + Authorized: true, + Card: &Card{ + ID: int64(1000 + i), + ChanKey: "channel", + DataServiceStatus: consts.SimDataServiceStatusDisabled, + }, + }) + } + task := &Task{Batch: Batch{BatchNo: "D-6"}, Operation: operation, ApnName: consts.SimApnNameDefault, Details: details} + + repo := &fakeRepository{task: task} + release := make(chan struct{}) + upstream := &subBatchUpstream{blockICCID: "iccid-100", block: release} + + done := make(chan error, 1) + go func() { done <- NewExecutor(repo, upstream).Execute(context.Background(), "D-6") }() + + // 第 1、3 个子批共 101 条应先落库，此时第 2 个子批仍阻塞在上游。 + waitForSavedCount(t, repo, 101) + if repo.isFinalized() { + t.Fatal("batch must not be finalized while one sub batch is still running") + } + + close(release) + if err = <-done; err != nil { + t.Fatal(err) + } + if got := repo.savedCount(); got != 201 { + t.Fatalf("saved count = %d, want 201", got) + } + if !repo.isFinalized() { + t.Fatal("batch was not finalized") + } + if calls := atomic.LoadInt32(&upstream.calls); calls != 3 { + t.Fatalf("upstream calls = %d, want 3", calls) + } +} + +func waitForSavedCount(t *testing.T, repo *fakeRepository, want int) { + t.Helper() + deadline := time.Now().Add(5 * time.Second) + for time.Now().Before(deadline) { + if repo.savedCount() >= want { + return + } + time.Sleep(5 * time.Millisecond) + } + t.Fatalf("saved count = %d, want at least %d", repo.savedCount(), want) +} + +func TestOperationByBatchTypeMapping(t *testing.T) { + start, err := OperationByBatchType(consts.BatchTypeDataStart) + if err != nil || start.Name != "dataStart" || start.ApnOperType != consts.SimApnOperTypeOpen || start.TargetStatus != consts.SimDataServiceStatusEnabled { + t.Fatalf("start operation = %+v, err = %v", start, err) + } + stop, err := OperationByBatchType(consts.BatchTypeDataStop) + if err != nil || stop.Name != "dataStop" || stop.ApnOperType != consts.SimApnOperTypeStop || stop.TargetStatus != consts.SimDataServiceStatusDisabled { + t.Fatalf("stop operation = %+v, err = %v", stop, err) + } + cut, err := OperationByBatchType(consts.BatchTypeDataCut) + if err != nil || cut.Name != "trafficCutoff" || cut.ApnOperType != consts.SimApnOperTypeStop || cut.TargetStatus != consts.SimDataServiceStatusDisabled { + t.Fatalf("cut operation = %+v, err = %v", cut, err) + } + if _, err = OperationByBatchType(consts.BatchTypeSimCardStart); err == nil { + t.Fatal("lifecycle batch type must not map to a data service operation") + } +} + +func TestDetailSnapshotIncludesOperationContext(t *testing.T) { + operation, err := OperationByBatchType(consts.BatchTypeDataStart) + if err != nil { + t.Fatal(err) + } + detail := &Detail{ + ID: 1, + ICCID: "iccid-1", + ChanKey: "channel", + StatusBefore: consts.SimDataServiceStatusDisabled, + StatusAfter: consts.SimDataServiceStatusEnabled, + OrderNum: "order-1", + UpstreamCode: 0, + UpstreamReason: "ok", + } + snapshot := detail.Snapshot(operation, consts.SimApnNameDefault) + if snapshot.Get("operation").String() != "dataStart" || + snapshot.Get("operType").String() != consts.SimApnOperTypeOpen || + snapshot.Get("apnName").String() != consts.SimApnNameDefault || + snapshot.Get("statusBefore").Int() != consts.SimDataServiceStatusDisabled || + snapshot.Get("statusAfter").Int() != consts.SimDataServiceStatusEnabled || + snapshot.Get("orderNum").String() != "order-1" { + t.Fatalf("snapshot = %s", snapshot.MustToJsonString()) + } +} + +// fakeRepository 会被多个子批次 goroutine 并发写入，因此内部状态统一用锁保护。 +type fakeRepository struct { + task *Task + refresh func(*Task) + + mu sync.Mutex + validationSaved bool + finalized bool + saved []Outcome +} + +func (r *fakeRepository) Load(context.Context, string) (*Task, error) { return r.task, nil } + +func (r *fakeRepository) RefreshDataServiceState(_ context.Context, task *Task) error { + if r.refresh != nil { + r.refresh(task) + } + return nil +} + +func (r *fakeRepository) SaveValidationFailure(context.Context, *Task) error { + r.mu.Lock() + defer r.mu.Unlock() + r.validationSaved = true + return nil +} + +func (r *fakeRepository) SaveResult(_ context.Context, _ *Task, detail *Detail, outcome Outcome) (Outcome, error) { + detail.ApplyOutcome(outcome) + r.mu.Lock() + defer r.mu.Unlock() + r.saved = append(r.saved, outcome) + return outcome, nil +} + +func (r *fakeRepository) Finalize(context.Context, *Task) error { + r.mu.Lock() + defer r.mu.Unlock() + r.finalized = true + return nil +} + +func (r *fakeRepository) savedCount() int { + r.mu.Lock() + defer r.mu.Unlock() + return len(r.saved) +} + +func (r *fakeRepository) isFinalized() bool { + r.mu.Lock() + defer r.mu.Unlock() + return r.finalized +} + +// subBatchUpstream 按子批次返回成功结果，并可在指定子批（以其首张 ICCID 标识）上阻塞， +// 用于验证"某个子批还没结束时，其他子批的结果已经落库"。 +type subBatchUpstream struct { + calls int32 + blockICCID string + block chan struct{} +} + +func (u *subBatchUpstream) ChangeSimApnFunction(ctx context.Context, input *model.SimCardApnChangeInput) ([]model.SimBaseRes[model.SimCardApnChangeOutputItem], error) { + atomic.AddInt32(&u.calls, 1) + if u.block != nil && len(input.SimKeys) > 0 && strings.TrimSpace(input.SimKeys[0].ICCID) == u.blockICCID { + select { + case <-u.block: + case <-ctx.Done(): + return nil, ctx.Err() + } + } + response := make([]model.SimBaseRes[model.SimCardApnChangeOutputItem], 0, len(input.SimKeys)) + for _, key := range input.SimKeys { + response = append(response, model.SimBaseRes[model.SimCardApnChangeOutputItem]{ + Code: 0, + Message: "ok", + SimKey: &model.SimKey{ICCID: key.ICCID}, + }) + } + return response, nil +} + +type fakeUpstream struct { + calls int + input *model.SimCardApnChangeInput + response []model.SimBaseRes[model.SimCardApnChangeOutputItem] + responseError error +} + +func (u *fakeUpstream) ChangeSimApnFunction(_ context.Context, input *model.SimCardApnChangeInput) ([]model.SimBaseRes[model.SimCardApnChangeOutputItem], error) { + u.calls++ + u.input = input + return u.response, u.responseError +} diff --git a/internal/service/data/types.go b/internal/service/data/types.go new file mode 100644 index 0000000..1ff38b5 --- /dev/null +++ b/internal/service/data/types.go @@ -0,0 +1,147 @@ +package data + +import ( + "context" + "strings" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +type Operation struct { + BatchType int + Name string + ApnOperType string + TargetStatus int +} + +type Batch struct { + ID int64 + BatchNo string + BatchType int + TenantID string + CreatedBy int64 + CreatedDept int64 + StartTime *gtime.Time + BatchInfo *gjson.Json +} + +type Card struct { + ID int64 + ICCID string + MSISDN string + IMSI string + ChanKey string + DataServiceStatus int +} + +type Detail struct { + ID int64 + RowNo int + ICCID string + InputMSISDN string + InputIMSI string + MSISDN string + IMSI string + Authorized bool + ChanKey string + StatusBefore int + StatusAfter int + TrafficCutoffCrossMonthHold int + Reason string + ValidationPassed bool + OrderNum string + UpstreamCode int + UpstreamReason string + Card *Card +} + +type Outcome struct { + Success bool + Code int + Message string + OrderNum string +} + +type Task struct { + Batch Batch + Operation Operation + ApnName string + Details []*Detail + ValidationOK bool + FailureReason string +} + +func (d *Detail) ApplyOutcome(outcome Outcome) { + d.OrderNum = strings.TrimSpace(outcome.OrderNum) + d.UpstreamCode = outcome.Code + d.UpstreamReason = strings.TrimSpace(outcome.Message) + if outcome.Success { + d.Reason = "" + return + } + if d.UpstreamReason == "" { + d.UpstreamReason = "upstream processing failed" + } + d.Reason = d.UpstreamReason +} + +func (d *Detail) Snapshot(operation Operation, apnName string) *gjson.Json { + if d == nil { + return gjson.New(nil) + } + return gjson.New(detailSnapshot{ + ICCID: d.ICCID, + InputMSISDN: d.InputMSISDN, + InputIMSI: d.InputIMSI, + MSISDN: d.MSISDN, + IMSI: d.IMSI, + Authorized: d.Authorized, + ChanKey: d.ChanKey, + Operation: operation.Name, + OperType: operation.ApnOperType, + ApnName: apnName, + StatusBefore: d.StatusBefore, + StatusAfter: d.StatusAfter, + TrafficCutoffCrossMonthHold: d.TrafficCutoffCrossMonthHold, + OrderNum: d.OrderNum, + UpstreamCode: d.UpstreamCode, + UpstreamMessage: d.UpstreamReason, + }) +} + +type detailSnapshot struct { + ICCID string `json:"ICCID"` + InputMSISDN string `json:"inputMsisdn"` + InputIMSI string `json:"inputImsi"` + MSISDN string `json:"MSISDN"` + IMSI string `json:"IMSI"` + Authorized bool `json:"authorized"` + ChanKey string `json:"chanKey"` + Operation string `json:"operation"` + OperType string `json:"operType"` + ApnName string `json:"apnName"` + StatusBefore int `json:"statusBefore"` + StatusAfter int `json:"statusAfter"` + TrafficCutoffCrossMonthHold int `json:"trafficCutoffCrossMonthHold,omitempty"` + OrderNum string `json:"orderNum"` + UpstreamCode int `json:"upstreamCode"` + UpstreamMessage string `json:"upstreamMessage"` +} + +type Repository interface { + Load(ctx context.Context, batchNo string) (*Task, error) + SaveValidationFailure(ctx context.Context, task *Task) error + SaveResult(ctx context.Context, task *Task, detail *Detail, outcome Outcome) (Outcome, error) + Finalize(ctx context.Context, task *Task) error +} + +type Upstream interface { + ChangeSimApnFunction(ctx context.Context, input *model.SimCardApnChangeInput) ([]model.SimBaseRes[model.SimCardApnChangeOutputItem], error) +} + +type Executor interface { + Execute(ctx context.Context, batchNo string) error +} diff --git a/internal/service/imei/handler.go b/internal/service/imei/handler.go new file mode 100644 index 0000000..eaae057 --- /dev/null +++ b/internal/service/imei/handler.go @@ -0,0 +1,360 @@ +package imei + +import ( + "context" + "fmt" + "strings" + + zsetconsumer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consumer/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + imeirepo "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/imei" +) + +const ( + // chanGatewayInternalErrorCode is emitted by the gateway for an internal + // fault. An empty errCode distinguishes it from a carrier business code. + chanGatewayInternalErrorCode = 9999 + + // tokenExpiredCode is the carrier code for an expired/invalid upstream + // token. The gateway already invalidates and refreshes the token once on + // this code; when it still reaches the worker, retrying in a later round + // lets a refreshed token succeed instead of parking the card as a terminal + // business failure. + tokenExpiredCode = 12021 + + // Mobile OneLink business codes 13000-13007 mean "system busy". Per the + // QPS field test this is an HTTP 200 business rejection that still occurs + // on roughly 70% of requests at 1 QPS, so it must be retried across rounds + // rather than in place. + mobileBusyFirstCode = 13000 + mobileBusyLastCode = 13007 + + // chanNoRecordCode is the carrier answer for a card it holds no record of + // ("系统无记录，请检查输入参数是否正确"). An IMEI is reported by the device, so + // a card that was never inserted into one is a permanent, expected answer + // rather than a failure: it is covered by the pass, never retried and never + // parked in the failure pool. Measured on 2026-10-20 across channels + // 220/250/251: all 4900 recorded failures were this code. + chanNoRecordCode = 14000 + + // chanNoDeviceInfoCode is the Telecom 5G CMP answer for a card with no bound + // device: {"result":"-1","resultMsg":"查询失败，查询不到指定卡的设备信息"}. + // The chan layer passes the upstream string through gconv.Int, so it arrives + // here as -1. Measured on 2026-10-22 run imei-20261022000000: 11,240 such + // answers across channels 142/111/187, all with that exact message. + chanNoDeviceInfoCode = -1 + + // chanTibiotNoInfoCode is Tibiot's "查询不到信息" business rejection, which + // arrives as HTTP status 500 with a JSON body status=500. It is the same + // semantic as chanNoDeviceInfoCode: the card has no device record. Measured + // on the same run: 1,612 answers, of which 1,494 were 查询不到信息 and 118 + // were 系统无记录. Tibiot answers with the same shape for genuine upstream + // faults, so the code alone is not enough — see isNoDeviceMessage. + chanTibiotNoInfoCode = 500 + + // mobileNoRecordMessage and tibiotNoInfoMessage are the carrier texts that + // mean "this card has no device, so there is no IMEI". They gate + // chanTibiotNoInfoCode because 500 is also used for real faults; matching + // text here is safer than treating every 500 as a definitive answer. + // chanNoDeviceInfoCode does not need this gate: -1 is not used for faults by + // the Telecom endpoint. + // + // Matching on text is normally fragile, so these strings are intentionally + // narrow and come from captured production traffic (2026-10-22). + mobileNoRecordMessage = "系统无记录" + tibiotNoInfoMessage = "查询不到信息" + + mobileBusyClass = "mobile_busy" + unsupportedClass = "unsupported" + dataInvalidClass = "data_invalid" + noRecordClass = "no_record" + networkClass = "network" + protocolClass = "protocol" + tokenExpiredClass = "token_expired" + gatewayClass = "gateway" + businessClass = "business" +) + +// imeiUnsupportedTemplates lists the templates whose upstream implementation +// has no IMEI endpoint at all. Such a template is a deterministic terminal +// state: retrying it every round would flood the failure pool with tens of +// thousands of meaningless records and distort the run failure rate. +// +// Template keys are used rather than error text because the text changes while +// the template does not. Keep this list aligned with the chan registry +// (iotlcm-upstream-chan/internal/logic/upstreamimpl): +// +// - sim_mingqian_v3 → mingqian.Adapter (per-card IMEI endpoint unavailable) +// - sim_daihe_v1 → sSimDaihe.QuerySimImei returns daiheUnsupported +// - sim_qianxun_v3 → sSimQianxun.QuerySimImei returns qianxunUnsupported +// +// Note that the shared-token templates (sim_onelink_*) are NOT listed here: +// they are registered to sSimYD, which does implement QuerySimImei. Verified +// against production data on 2026-10-12 — every sim_onelink_* template with +// cards is served by sSimYD. +var imeiUnsupportedTemplates = map[string]struct{}{ + "sim_mingqian_v3": {}, + "sim_daihe_v1": {}, + "sim_qianxun_v3": {}, +} + +// mobileTemplates lists the templates served by the China Mobile OneLink +// implementation, so numeric codes 13000-13007 can be classified as +// "system busy" for them. Derived from the chan registration table; keep in +// sync with that table. Unknown templates stay generic and are never treated +// as mobile. +var mobileTemplates = map[string]struct{}{ + "sim_yd_v5": {}, + "sim_yd_mock_v5": {}, + "sim_sdtywl_v1": {}, + "sim_onelink_tengyu_v1": {}, + "sim_onelink_xuzhou_mobile_v1": {}, + "sim_onelink_lelian_v1": {}, + "sim_onelink_qianxun_v1": {}, + "sim_onelink_henan_xinjing_v1": {}, + "sim_henan_xinjing_direct_v1": {}, + "sim_onelink_zibo_v1": {}, + "sim_tibiot_v1": {}, + "sim_yunda_v1": {}, + "sim_onelink_daihe_v1": {}, + "sim_onelink_daliuliang_v1": {}, + "sim__daliuliang_v1": {}, + "sim_onelink_mobile_public_v1": {}, + "sim_onelink_kedang_v1": {}, + "sim_onelink_beixin_v1": {}, + "lanshang_8000": {}, + "lanshang_8003": {}, +} + +func isMobileTemplate(tempKey string) bool { + _, ok := mobileTemplates[strings.ToLower(strings.TrimSpace(tempKey))] + return ok +} + +func isImeiUnsupportedTemplate(tempKey string) bool { + _, ok := imeiUnsupportedTemplates[strings.ToLower(strings.TrimSpace(tempKey))] + return ok +} + +// isNoDeviceMessage reports whether a carrier message means "this card has no +// device bound, so no IMEI exists". +// +// This is only consulted for codes that carriers reuse for genuine faults (the +// Tibiot 500). Codes that are unambiguous on their own (14000, -1) do not need +// it, and deliberately do not depend on the text: the message is carrier +// wording and may be reworded at any time. +func isNoDeviceMessage(message string) bool { + message = strings.TrimSpace(message) + if message == "" { + return false + } + return strings.Contains(message, tibiotNoInfoMessage) || + strings.Contains(message, mobileNoRecordMessage) +} + +// ImeiClient is the boundary to the chan gateway. +type ImeiClient interface { + QuerySimImei(context.Context, *model.SimImeiInput) ([]model.SimBaseRes[model.SimImeiItem], error) +} + +// ImeiRepository persists one batch of sync results. +type ImeiRepository interface { + ApplyBatch(context.Context, []imeirepo.Result) error +} + +type Handler struct { + client ImeiClient + repo ImeiRepository + networkRetries int +} + +func NewHandler(client ImeiClient, repo ImeiRepository, networkRetries ...int) (*Handler, error) { + if client == nil || repo == nil { + return nil, fmt.Errorf("imei handler dependencies are incomplete") + } + retries := 0 + if len(networkRetries) > 0 && networkRetries[0] > 0 { + retries = networkRetries[0] + } + return &Handler{client: client, repo: repo, networkRetries: retries}, nil +} + +// Query performs one carrier IMEI lookup and maps the outcome onto the queue's +// retry semantics. +// +// The classification matrix is deliberately explicit, because the difference +// between "retry forever" and "park as terminal" decides whether the failure +// pool stays meaningful: +// +// network error → network retryable +// empty result slice → protocol retryable +// mismatched card identifier → protocol retryable +// Code == 0, non-empty IMEI → success, value written +// Code == 0, empty IMEI → data_invalid terminal, covered, heartbeat only +// 14000 → no_record terminal, covered, heartbeat only +// -1 (no device info) → no_record terminal, covered, heartbeat only +// 500 + "查询不到信息" (Tibiot) → no_record terminal, covered, heartbeat only +// 13000-13007 on a mobile template → mobile_busy retryable +// unsupported template → unsupported terminal failure +// 9999 with empty errCode → gateway retryable +// 12021 → token_expired retryable +// any other non-zero Code → business terminal failure +// +// "Covered" marks a terminal answer that says something definitive about the +// card but produces no value. It is deliberately not a failure: it records +// coverage (ImeiSynced) so the heartbeat is written, while the queue keeps it +// out of the failure pool and out of the run's failure count. +func (h *Handler) Query(ctx context.Context, task queue.Task) zsetconsumer.Result { + result := zsetconsumer.Result{Task: task} + + // An unsupported template is known before any network call: skip the + // request entirely and park the card immediately. + if isImeiUnsupportedTemplate(task.TempKey) { + result.ErrorClass = unsupportedClass + result.Message = "template does not support imei query" + return result + } + + input := &model.SimImeiInput{SimKeys: []*model.SimKey{{ + ChanKey: task.ChanKey, + ICCID: task.ICCID, + IMSI: task.IMSI, + MSISDN: task.MSISDN, + }}} + + var ( + output []model.SimBaseRes[model.SimImeiItem] + err error + ) + for attempt := 0; attempt <= h.networkRetries; attempt++ { + output, err = h.client.QuerySimImei(ctx, input) + if err == nil { + break + } + if ctx.Err() != nil { + break + } + } + if err != nil { + result.ErrorClass = networkClass + result.Retryable = true + result.Message = err.Error() + return result + } + if len(output) == 0 { + result.ErrorClass = protocolClass + result.Retryable = true + result.Message = "chan returned empty imei result" + return result + } + + item := output[0] + if item.SimKey == nil { + item.SimKey = &model.SimKey{ChanKey: task.ChanKey, ICCID: task.ICCID, IMSI: task.IMSI, MSISDN: task.MSISDN} + } + if task.ICCID != "" && item.SimKey.ICCID != task.ICCID { + result.ErrorClass = protocolClass + result.Retryable = true + result.Message = "chan returned mismatched card identifier" + return result + } + + result.Code = item.Code + result.Message = item.Message + result.ErrCode = item.ErrCode + + if item.Code != 0 { + switch { + case isMobileTemplate(task.TempKey) && item.Code >= mobileBusyFirstCode && item.Code <= mobileBusyLastCode: + result.ErrorClass = mobileBusyClass + result.Retryable = true + case item.Code == chanGatewayInternalErrorCode && strings.TrimSpace(item.ErrCode) == "": + result.ErrorClass = gatewayClass + result.Retryable = true + case item.Code == tokenExpiredCode: + result.ErrorClass = tokenExpiredClass + result.Retryable = true + case item.Code == chanNoRecordCode: + // The carrier answered definitively: this card has no IMEI on record + // and never will until a device reports one. Retrying cannot change + // that, and counting it as a failure reported a 21% success rate for + // a run that had in fact queried every card successfully. + result.ErrorClass = noRecordClass + result.ImeiSynced = true + result.Covered = true + case item.Code == chanNoDeviceInfoCode: + // Telecom 5G CMP: "查询失败，查询不到指定卡的设备信息". The card is + // valid and the query succeeded; there is simply no device bound to + // it, so no IMEI exists to report. Same terminal semantics as 14000. + result.ErrorClass = noRecordClass + result.ImeiSynced = true + result.Covered = true + case item.Code == chanTibiotNoInfoCode && isNoDeviceMessage(item.Message): + // Tibiot uses 500 both for "查询不到信息" and for genuine faults, + // so the text decides. Only the no-info wording is a terminal answer; + // any other 500 keeps falling through to business below. + result.ErrorClass = noRecordClass + result.ImeiSynced = true + result.Covered = true + default: + result.ErrorClass = businessClass + } + return result + } + + // Code == 0: the carrier answered. Coverage is recorded either way; the + // value is only written when the carrier actually returned one. + result.Success = true + result.ImeiSynced = true + imei := strings.TrimSpace(item.ResData.Imei) + if imei == "" { + // A successful query with no value is a legitimate terminal outcome + // (device never reported an IMEI, or a multi-IMEI answer that the + // chan layer drops). It is NOT a failure to retry, and it must never + // clear the stored value. Like code 14000 it only records coverage. + result.Success = false + result.ErrorClass = dataInvalidClass + result.Covered = true + result.Message = "carrier returned no imei value" + return result + } + result.Imei = imei + return result +} + +// Apply persists one batch. Only successfully synchronized cards produce an +// update; a value is written only when the carrier returned a non-empty one. +// +// The extra `result.Imei != ""` guard is intentional duplication of the +// handler check: it makes "never clear a stored IMEI" a property of the write +// path itself, so a future handler change cannot silently start writing empty +// values over good data. +func (h *Handler) Apply(ctx context.Context, results []zsetconsumer.Result) error { + updates := make([]imeirepo.Result, 0, len(results)) + for _, result := range results { + if !result.ImeiSynced { + continue + } + update := imeirepo.Result{ + CardID: result.Task.CardID, + Synced: true, + ValueChanged: result.Success && result.Imei != "", + Imei: result.Imei, + } + if !update.ValueChanged { + // Clear the value so a heartbeat-only result cannot carry a stale + // IMEI into the CASE expression. + update.Imei = "" + } + updates = append(updates, update) + } + return h.repo.ApplyBatch(ctx, updates) +} + +// ShouldRetryNextRound is the full-sync retry policy: retryable failures are +// deferred into the next run round. Terminal failures go straight to the +// failure pool; covered results are terminal but are not failures at all. +func ShouldRetryNextRound(result zsetconsumer.Result) bool { + return !result.Success && result.Retryable +} diff --git a/internal/service/imei/handler_test.go b/internal/service/imei/handler_test.go new file mode 100644 index 0000000..4fec9a4 --- /dev/null +++ b/internal/service/imei/handler_test.go @@ -0,0 +1,548 @@ +package imei + +import ( + "context" + "errors" + "testing" + + zsetconsumer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consumer/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + imeirepo "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/imei" +) + +// fakeClient returns a canned response or error and records the call count. +type fakeClient struct { + output []model.SimBaseRes[model.SimImeiItem] + err error + calls int +} + +func (f *fakeClient) QuerySimImei(context.Context, *model.SimImeiInput) ([]model.SimBaseRes[model.SimImeiItem], error) { + f.calls++ + return f.output, f.err +} + +// fakeRepo captures what the handler asked to persist. +type fakeRepo struct { + results []imeirepo.Result + calls int + err error +} + +func (f *fakeRepo) ApplyBatch(_ context.Context, results []imeirepo.Result) error { + f.calls++ + f.results = append(f.results, results...) + return f.err +} + +func task(tempKey string) queue.Task { + return queue.Task{ + TaskID: "run:imei_full_sync:1", + RunID: "run", + ChanKey: "173", + TempKey: tempKey, + CardID: 1, + ICCID: "898604F4152000015474", + MSISDN: "1064000000001", + IMSI: "460000000000001", + Operator: 1, + } +} + +func item(code int, errCode, imei string) model.SimBaseRes[model.SimImeiItem] { + return model.SimBaseRes[model.SimImeiItem]{ + Code: code, + ErrCode: errCode, + SimKey: &model.SimKey{ChanKey: "173", ICCID: "898604F4152000015474"}, + ResData: model.SimImeiItem{Imei: imei}, + } +} + +func newTestHandler(t *testing.T, client ImeiClient, repo ImeiRepository, retries ...int) *Handler { + t.Helper() + h, err := NewHandler(client, repo, retries...) + if err != nil { + t.Fatalf("NewHandler: %v", err) + } + return h +} + +// TestQuerySuccessWithValue 覆盖 §7 的正常命中路径。 +func TestQuerySuccessWithValue(t *testing.T) { + client := &fakeClient{output: []model.SimBaseRes[model.SimImeiItem]{item(0, "", "865466070817587")}} + handler := newTestHandler(t, client, &fakeRepo{}) + + result := handler.Query(context.Background(), task("sim_yd_v5")) + if !result.Success { + t.Fatalf("应成功: %+v", result) + } + if !result.ImeiSynced { + t.Fatal("ImeiSynced 应为 true") + } + if result.Imei != "865466070817587" { + t.Fatalf("imei = %q", result.Imei) + } + if result.Retryable { + t.Fatal("成功不应可重试") + } +} + +// TestQueryEmptyValueIsTerminalAndSynced 覆盖 §7 T5/T6：上游成功但无值。 +// 这是最关键的语义——必须记录覆盖（ImeiSynced）但不得视为失败重试。 +func TestQueryEmptyValueIsTerminalAndSynced(t *testing.T) { + client := &fakeClient{output: []model.SimBaseRes[model.SimImeiItem]{item(0, "", "")}} + handler := newTestHandler(t, client, &fakeRepo{}) + + result := handler.Query(context.Background(), task("sim_yd_v5")) + if result.Success { + t.Fatal("空值不应标记为 Success") + } + if result.Retryable { + t.Fatal("空值是终态，不应重试") + } + if !result.ImeiSynced { + t.Fatal("空值仍应记录为已同步（心跳）") + } + if !result.Covered { + t.Fatal("空值与 14000 同族，不应进失败池") + } + if result.ErrorClass != dataInvalidClass { + t.Fatalf("errorClass = %q, want %q", result.ErrorClass, dataInvalidClass) + } + if result.Imei != "" { + t.Fatalf("imei 应为空: %q", result.Imei) + } +} + +// TestQueryNoRecordIsCoveredNotFailed 覆盖生产实测的主路径（2026-10-20 chan +// 220/250/251 的 4900 条“失败”全部是 code=14000）。运营商持有卡但没有 IMEI +// 记录，这是终态正常结果：记录覆盖、绝不重试、不进失败池。 +func TestQueryNoRecordIsCoveredNotFailed(t *testing.T) { + for _, tempKey := range []string{"sim_yd_v5", "sim_dx_v1"} { + client := &fakeClient{output: []model.SimBaseRes[model.SimImeiItem]{ + item(chanNoRecordCode, "", ""), + }} + handler := newTestHandler(t, client, &fakeRepo{}) + // 14000 的 message 是运营商原文，处理逻辑不得依赖它。 + client.output[0].Message = "系统无记录，请检查输入参数是否正确" + + result := handler.Query(context.Background(), task(tempKey)) + if result.Success { + t.Fatalf("%s: 无记录不得标记为 Success", tempKey) + } + if result.Retryable { + t.Fatalf("%s: 无记录是终态，不应重试", tempKey) + } + if !result.ImeiSynced { + t.Fatalf("%s: 无记录仍应记录覆盖（心跳）", tempKey) + } + if !result.Covered { + t.Fatalf("%s: 无记录不得计入失败池", tempKey) + } + if result.ErrorClass != noRecordClass { + t.Fatalf("%s: errorClass = %q, want %q", tempKey, result.ErrorClass, noRecordClass) + } + if result.Imei != "" { + t.Fatalf("%s: 无记录不得携带值: %q", tempKey, result.Imei) + } + } +} + +// TestQueryNoRecordBeatsMobileBusyRange 14000 不在 13000-13007 内， +// 但仍要求移动模板走同一条“无记录”分类——不能被 busy 分支或业务错误吞掉。 +func TestQueryNoRecordDoesNotRetryOnMobileTemplate(t *testing.T) { + client := &fakeClient{output: []model.SimBaseRes[model.SimImeiItem]{item(chanNoRecordCode, "", "")}} + handler := newTestHandler(t, client, &fakeRepo{}) + + result := handler.Query(context.Background(), task("sim_yd_v5")) + if result.Retryable || result.ErrorClass == mobileBusyClass || result.ErrorClass == businessClass { + t.Fatalf("14000 应为 no_record 终态: %+v", result) + } +} + +// TestQueryUnsupportedTemplateSkipsUpstream 覆盖 §7 T3： +// 不支持的模板必须在不发请求的前提下直接终态，避免失败池爆炸。 +func TestQueryUnsupportedTemplateSkipsUpstream(t *testing.T) { + for _, tempKey := range []string{"sim_mingqian_v3", "sim_daihe_v1", "sim_qianxun_v3"} { + client := &fakeClient{output: []model.SimBaseRes[model.SimImeiItem]{item(0, "", "865466070817587")}} + handler := newTestHandler(t, client, &fakeRepo{}) + + result := handler.Query(context.Background(), task(tempKey)) + if result.Retryable { + t.Fatalf("%s: 不支持应为终态，不应重试", tempKey) + } + if result.ErrorClass != unsupportedClass { + t.Fatalf("%s: errorClass = %q, want %q", tempKey, result.ErrorClass, unsupportedClass) + } + if client.calls != 0 { + t.Fatalf("%s: 不支持模板不应发起上游请求，实际调用 %d 次", tempKey, client.calls) + } + if result.ImeiSynced { + t.Fatalf("%s: 未查询不应记录覆盖", tempKey) + } + } +} + +// TestQueryUnsupportedTemplatesAreCaseInsensitive 模板键大小写不应影响判定。 +func TestQueryUnsupportedTemplatesAreCaseInsensitive(t *testing.T) { + client := &fakeClient{} + handler := newTestHandler(t, client, &fakeRepo{}) + result := handler.Query(context.Background(), task("SIM_MingQian_V3")) + if result.ErrorClass != unsupportedClass || client.calls != 0 { + t.Fatalf("大小写变体未被识别: %+v, calls=%d", result, client.calls) + } +} + +// TestQuerySharedTokenTemplatesAreSupported 回归保护： +// sim_onelink_* 走 shared_token 模型 → sSimYD（支持 IMEI）， +// 不得被误列入不支持白名单（这是 2026-10-12 生产基线纠正过的判断）。 +func TestQuerySharedTokenTemplatesAreSupported(t *testing.T) { + for _, tempKey := range []string{ + "sim_onelink_tengyu_v1", + "sim_onelink_daihe_v1", + "sim_onelink_qianxun_v1", + "sim_onelink_kedang_v1", + "sim_onelink_daliuliang_v1", + "sim_onelink_mengwang_v1", + } { + if isImeiUnsupportedTemplate(tempKey) { + t.Fatalf("%s 走 sSimYD 支持 IMEI，不应在不支持白名单", tempKey) + } + } +} + +// TestQueryMobileBusyIsRetryable 覆盖 §7 T4：移动 13000 区间跨轮重试。 +func TestQueryMobileBusyIsRetryable(t *testing.T) { + for _, code := range []int{13000, 13003, 13007} { + client := &fakeClient{output: []model.SimBaseRes[model.SimImeiItem]{item(code, "", "")}} + handler := newTestHandler(t, client, &fakeRepo{}) + + result := handler.Query(context.Background(), task("sim_yd_v5")) + if result.ErrorClass != mobileBusyClass || !result.Retryable { + t.Fatalf("code %d: errorClass=%q retryable=%v, want %q/true", code, result.ErrorClass, result.Retryable, mobileBusyClass) + } + if result.Covered { + t.Fatalf("code %d: 系统忙必须计入失败", code) + } + if result.ImeiSynced { + t.Fatalf("code %d: 系统忙不应记录覆盖", code) + } + } +} + +// TestQueryMobileBusyOnlyForMobileTemplates 13000 只对移动族判定为 busy。 +func TestQueryMobileBusyOnlyForMobileTemplates(t *testing.T) { + client := &fakeClient{output: []model.SimBaseRes[model.SimImeiItem]{item(13000, "", "")}} + handler := newTestHandler(t, client, &fakeRepo{}) + + // 电信模板收到 13000 不是移动系统忙语义，应按普通业务错误处理 + result := handler.Query(context.Background(), task("sim_dx_v1")) + if result.ErrorClass == mobileBusyClass { + t.Fatalf("非移动模板不应判定为 mobile_busy: %+v", result) + } + if result.Retryable { + t.Fatalf("非移动模板的 13000 应为终态: %+v", result) + } +} + +// TestQueryTelecomRetryableCodeIsNotTreatedAsRetryableForImei 回归保护： +// S00004 是电信「渠道限流」业务码，属于状态/流量链路的可重试语义； +// IMEI 链路没有对应的限流处理，必须按普通业务错误终态处理， +// 否则会无限重试同一张卡。 +func TestQueryTelecomRetryableCodeIsNotTreatedAsRetryableForImei(t *testing.T) { + client := &fakeClient{output: []model.SimBaseRes[model.SimImeiItem]{item(1, "S00004", "")}} + handler := newTestHandler(t, client, &fakeRepo{}) + + result := handler.Query(context.Background(), task("sim_dx_v1")) + if result.Retryable { + t.Fatalf("IMEI 链路不应把 S00004 当作可重试: %+v", result) + } + if result.ErrorClass != businessClass { + t.Fatalf("errorClass = %q, want %q", result.ErrorClass, businessClass) + } +} + +// TestQueryGatewayAndTokenErrorsAreRetryable 覆盖 §7 T8。 +func TestQueryGatewayAndTokenErrorsAreRetryable(t *testing.T) { + cases := []struct { + name string + code int + errCode string + wantClass string + }{ + {"网关内部错误", 9999, "", gatewayClass}, + {"token 过期", tokenExpiredCode, "", tokenExpiredClass}, + } + for _, tc := range cases { + client := &fakeClient{output: []model.SimBaseRes[model.SimImeiItem]{item(tc.code, tc.errCode, "")}} + handler := newTestHandler(t, client, &fakeRepo{}) + + result := handler.Query(context.Background(), task("sim_yd_v5")) + if result.ErrorClass != tc.wantClass || !result.Retryable { + t.Fatalf("%s: errorClass=%q retryable=%v, want %q/true", tc.name, result.ErrorClass, result.Retryable, tc.wantClass) + } + if result.Covered { + t.Fatalf("%s: 可重试错误必须计入失败", tc.name) + } + } +} + +// TestQueryBusinessErrorIsTerminal 其他业务码为终态，避免无意义重试。 +func TestQueryBusinessErrorIsTerminal(t *testing.T) { + client := &fakeClient{output: []model.SimBaseRes[model.SimImeiItem]{item(11011, "", "")}} + handler := newTestHandler(t, client, &fakeRepo{}) + + result := handler.Query(context.Background(), task("sim_yd_v5")) + if result.Retryable { + t.Fatal("11011 码号信息错误是确定性终态，不应重试") + } + if result.ErrorClass != businessClass { + t.Fatalf("errorClass = %q, want %q", result.ErrorClass, businessClass) + } + if result.Covered { + t.Fatal("11011 是真失败，必须进失败池") + } + if result.ImeiSynced { + t.Fatal("11011 未拿到运营商结果，不应记录覆盖") + } +} + +// TestQueryNetworkErrorIsRetryable 网络错误跨轮重试，且受 networkRetries 控制。 +func TestQueryNetworkErrorIsRetryable(t *testing.T) { + client := &fakeClient{err: errors.New("connection refused")} + handler := newTestHandler(t, client, &fakeRepo{}, 2) + + result := handler.Query(context.Background(), task("sim_yd_v5")) + if result.ErrorClass != networkClass || !result.Retryable { + t.Fatalf("errorClass=%q retryable=%v, want network/true", result.ErrorClass, result.Retryable) + } + // 1 次初始 + 2 次重试 + if client.calls != 3 { + t.Fatalf("calls = %d, want 3（1 次初始 + 2 次重试）", client.calls) + } +} + +// TestQueryEmptyResultSliceIsProtocolError 空切片是协议异常，应重试。 +func TestQueryEmptyResultSliceIsProtocolError(t *testing.T) { + client := &fakeClient{output: nil} + handler := newTestHandler(t, client, &fakeRepo{}) + + result := handler.Query(context.Background(), task("sim_yd_v5")) + if result.ErrorClass != protocolClass || !result.Retryable { + t.Fatalf("errorClass=%q retryable=%v, want protocol/true", result.ErrorClass, result.Retryable) + } +} + +// TestQueryMismatchedIdentifierIsProtocolError 防止上游串卡。 +func TestQueryMismatchedIdentifierIsProtocolError(t *testing.T) { + mismatched := item(0, "", "865466070817587") + mismatched.SimKey = &model.SimKey{ChanKey: "173", ICCID: "OTHER-CARD"} + client := &fakeClient{output: []model.SimBaseRes[model.SimImeiItem]{mismatched}} + handler := newTestHandler(t, client, &fakeRepo{}) + + result := handler.Query(context.Background(), task("sim_yd_v5")) + if result.ErrorClass != protocolClass || !result.Retryable { + t.Fatalf("串卡应判为协议错误: %+v", result) + } + if result.ImeiSynced { + t.Fatal("串卡不应记录覆盖") + } +} + +// TestApplyWritesValueOnlyWhenChanged 覆盖 §7 T9/T10 与 §5.2 守卫： +// - 有值 → ValueChanged=true，带值写入 +// - 空值 → ValueChanged=false，且 Imei 必须被清空（不得覆盖已有值） +// - 失败 → 完全不入库 +func TestApplyWritesValueOnlyWhenChanged(t *testing.T) { + repo := &fakeRepo{} + handler := newTestHandler(t, &fakeClient{}, repo) + + successTask := task("sim_yd_v5") + emptyTask := task("sim_yd_v5") + emptyTask.CardID = 2 + failedTask := task("sim_yd_v5") + failedTask.CardID = 3 + + err := handler.Apply(context.Background(), []zsetconsumer.Result{ + {Task: successTask, Success: true, ImeiSynced: true, Imei: "865466070817587"}, + {Task: emptyTask, Success: false, ImeiSynced: true, ErrorClass: dataInvalidClass}, + {Task: failedTask, Success: false, Retryable: true, ErrorClass: networkClass, ImeiSynced: false}, + }) + if err != nil { + t.Fatalf("Apply: %v", err) + } + if len(repo.results) != 2 { + t.Fatalf("应只写入 2 条（成功的 + 心跳的），实际 %d 条: %+v", len(repo.results), repo.results) + } + + // 第 1 条：值变更 + first := repo.results[0] + if first.CardID != 1 || !first.ValueChanged || first.Imei != "865466070817587" { + t.Fatalf("第一条应为值变更: %+v", first) + } + // 第 2 条：心跳，值必须为空（防止覆盖） + second := repo.results[1] + if second.CardID != 2 || second.ValueChanged { + t.Fatalf("第二条应为心跳: %+v", second) + } + if second.Imei != "" { + t.Fatalf("心跳结果的 Imei 必须为空，否则会覆盖已有值: %q", second.Imei) + } + // 第 3 条：失败完全不入库 + for _, r := range repo.results { + if r.CardID == 3 { + t.Fatal("失败结果不应入库") + } + } +} + +// TestApplyNeverWritesEmptyOverValue 是 §5.2 的直接回归测试： +// 即使上游大面积返回空值，写路径也不得产生任何带值的更新。 +func TestApplyNeverWritesEmptyOverValue(t *testing.T) { + repo := &fakeRepo{} + handler := newTestHandler(t, &fakeClient{}, repo) + + results := make([]zsetconsumer.Result, 0, 50) + for i := int64(1); i <= 50; i++ { + tk := task("sim_yd_v5") + tk.CardID = i + results = append(results, zsetconsumer.Result{ + Task: tk, Success: false, ImeiSynced: true, ErrorClass: dataInvalidClass, + }) + } + if err := handler.Apply(context.Background(), results); err != nil { + t.Fatalf("Apply: %v", err) + } + if len(repo.results) != 50 { + t.Fatalf("50 张卡都应记录心跳，实际 %d", len(repo.results)) + } + for _, r := range repo.results { + if r.ValueChanged || r.Imei != "" { + t.Fatalf("空值结果不得携带值: %+v", r) + } + } +} + +// TestApplyNoRecordWritesHeartbeatOnly 14000 只能写心跳，绝不能清掉已有 IMEI。 +func TestApplyNoRecordWritesHeartbeatOnly(t *testing.T) { + repo := &fakeRepo{} + handler := newTestHandler(t, &fakeClient{}, repo) + + noRecord := task("sim_yd_v5") + if err := handler.Apply(context.Background(), []zsetconsumer.Result{ + {Task: noRecord, Success: false, ImeiSynced: true, Covered: true, ErrorClass: noRecordClass, Code: chanNoRecordCode}, + }); err != nil { + t.Fatalf("Apply: %v", err) + } + if len(repo.results) != 1 { + t.Fatalf("无记录仍应写入 1 条心跳，实际 %d: %+v", len(repo.results), repo.results) + } + got := repo.results[0] + if !got.Synced || got.ValueChanged || got.Imei != "" { + t.Fatalf("无记录应只写心跳: %+v", got) + } +} + +// TestShouldRetryNextRound 覆盖重试策略。 +func TestShouldRetryNextRound(t *testing.T) { + cases := []struct { + name string + result zsetconsumer.Result + want bool + }{ + {"成功不重试", zsetconsumer.Result{Success: true}, false}, + {"可重试失败", zsetconsumer.Result{Success: false, Retryable: true}, true}, + {"终态失败不重试", zsetconsumer.Result{Success: false, Retryable: false}, false}, + {"无记录不重试", zsetconsumer.Result{Success: false, Covered: true, ErrorClass: noRecordClass}, false}, + } + for _, tc := range cases { + if got := ShouldRetryNextRound(tc.result); got != tc.want { + t.Fatalf("%s: got %v, want %v", tc.name, got, tc.want) + } + } +} + +// TestQueryNoDeviceIsCoveredNotFailed 覆盖 2026-10-22 生产 run +// imei-20261022000000 的 12,865 条终态失败：全部是「卡上没有绑定设备， +// 所以没有 IMEI」，不是真失败。 +// +// 证据（生产日志实采）： +// - 电信 5G CMP 11,240 条：result="-1"，message=查询失败，查询不到指定卡的设备信息 +// （通道 142/111/187，template sim_dx_v1） +// - Tibiot 1,612 条：HTTP 500 + body status=500，message=查询不到信息（1,494） +// 或 系统无记录（118） +// +// 三者都是终态：记录覆盖（写心跳）、不重试、不进失败池。 +func TestQueryNoDeviceIsCoveredNotFailed(t *testing.T) { + cases := []struct { + name string + tempKey string + code int + message string + }{ + {"电信无绑定设备", "sim_dx_v1", chanNoDeviceInfoCode, "查询失败，查询不到指定卡的设备信息"}, + {"Tibiot 查询不到信息", "sim_tibiot_v1", chanTibiotNoInfoCode, "查询不到信息"}, + {"Tibiot 系统无记录", "sim_tibiot_v1", chanTibiotNoInfoCode, "系统无记录，请检查输入参数是否正确"}, + } + for _, tc := range cases { + it := item(tc.code, "", "") + it.Message = tc.message + client := &fakeClient{output: []model.SimBaseRes[model.SimImeiItem]{it}} + handler := newTestHandler(t, client, &fakeRepo{}) + + result := handler.Query(context.Background(), task(tc.tempKey)) + if result.Success { + t.Fatalf("%s: 无设备不得标记为 Success", tc.name) + } + if result.Retryable { + t.Fatalf("%s: 无设备是终态，不应重试", tc.name) + } + if !result.ImeiSynced { + t.Fatalf("%s: 无设备仍应记录覆盖（心跳）", tc.name) + } + if !result.Covered { + t.Fatalf("%s: 无设备不得计入失败池", tc.name) + } + if result.ErrorClass != noRecordClass { + t.Fatalf("%s: errorClass = %q, want %q", tc.name, result.ErrorClass, noRecordClass) + } + } +} + +// TestQueryTibiotNonNoInfo500StaysBusiness 锁住 500 的边界：Tibiot 也用 500 +// 表达真实故障，只有「查询不到信息/系统无记录」的文案才是终态无记录。 +// 不加这道闸门会把真故障当成功覆盖掉，那是比多算失败更严重的错误。 +func TestQueryTibiotNonNoInfo500StaysBusiness(t *testing.T) { + for _, message := range []string{"系统繁忙，请稍后重试", "服务异常", "internal error", ""} { + it := item(chanTibiotNoInfoCode, "", "") + it.Message = message + client := &fakeClient{output: []model.SimBaseRes[model.SimImeiItem]{it}} + handler := newTestHandler(t, client, &fakeRepo{}) + + result := handler.Query(context.Background(), task("sim_tibiot_v1")) + if result.Covered { + t.Fatalf("message=%q: 非无记录文案的 500 不得计入覆盖", message) + } + if result.ImeiSynced { + t.Fatalf("message=%q: 非无记录文案的 500 不应记录覆盖", message) + } + if result.ErrorClass != businessClass { + t.Fatalf("message=%q: errorClass = %q, want %q", message, result.ErrorClass, businessClass) + } + } +} + +// TestQueryNoDeviceCodeIgnoresMessage 14000 与 -1 语义明确，不得依赖运营商文案 +// （文案随时可能改写），即使 message 为空也要判为无记录。 +func TestQueryNoDeviceCodeIgnoresMessage(t *testing.T) { + for _, code := range []int{chanNoRecordCode, chanNoDeviceInfoCode} { + client := &fakeClient{output: []model.SimBaseRes[model.SimImeiItem]{item(code, "", "")}} + handler := newTestHandler(t, client, &fakeRepo{}) + + result := handler.Query(context.Background(), task("sim_dx_v1")) + if !result.Covered || result.ErrorClass != noRecordClass { + t.Fatalf("code=%d 空 message 仍应判 no_record: %+v", code, result) + } + } +} diff --git a/internal/service/inbound/executor.go b/internal/service/inbound/executor.go new file mode 100644 index 0000000..8b4e9fb --- /dev/null +++ b/internal/service/inbound/executor.go @@ -0,0 +1,1552 @@ +package inbound + +import ( + "bytes" + "context" + "fmt" + "math/big" + "path/filepath" + "regexp" + "strings" + "sync" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/xuri/excelize/v2" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +const ( + cascadeFailureReason = "同批存在校验失败，本次整批未入库" + maxIdentifierLength = 32 + minICCIDLength = 19 + minIMSILength = 15 + minMSISDNLength = 11 +) + +const ( + inboundUpstreamBatchSizeConfigKey = "batch.inbound.upstreamBatchSize" + inboundUpstreamConcurrencyKey = "batch.inbound.upstreamConcurrency" + defaultInboundUpstreamBatchSize = 1 + defaultInboundUpstreamConcurrency = 8 +) + +var tokenSplitter = regexp.MustCompile(`[,\t，\s]+`) + +type executor struct { + repository Repository + upstream Upstream + rateLimiter upstreamBasicInfoRateLimiter +} + +func NewExecutor(repository Repository) *executor { + return &executor{repository: repository} +} + +func NewExecutorWithUpstream(repository Repository, upstream Upstream) *executor { + return &executor{ + repository: repository, + upstream: upstream, + rateLimiter: newUpstreamBasicInfoRateLimiter(), + } +} + +func (e *executor) Execute(ctx context.Context, batchNo string) (err error) { + totalStarted := time.Now() + var task *Task + defer func() { + logInboundTotal(ctx, batchNo, task, totalStarted, err) + }() + + stageStarted := time.Now() + task, err = e.repository.Load(ctx, batchNo) + if err != nil { + logInboundStageByBatchNo(ctx, batchNo, "load_batch", stageStarted, 0, 0, 0, err) + return gerror.Wrap(err, "load inbound batch failed") + } + logInboundStage(ctx, task, "load_batch", stageStarted, nil) + + stageStarted = time.Now() + if err = e.populateRows(ctx, task); err != nil { + logInboundStage(ctx, task, "populate_rows", stageStarted, err) + return err + } + logInboundStage(ctx, task, "populate_rows", stageStarted, nil) + + stageStarted = time.Now() + if err = e.validate(ctx, task); err != nil { + logInboundStage(ctx, task, "validate", stageStarted, err) + return err + } + logInboundStage(ctx, task, "validate", stageStarted, nil) + + if len(task.Rows) == 0 { + stageStarted = time.Now() + if err = e.repository.SaveValidationFailure(ctx, task); err != nil { + logInboundStage(ctx, task, "save_validation_failure", stageStarted, err) + return gerror.Wrap(err, "save empty inbound failure failed") + } + logInboundStage(ctx, task, "save_validation_failure", stageStarted, nil) + _ = e.saveResultFile(ctx, task) + return nil + } + if hasFailedRows(task.Rows) { + applyCascadeFailure(task) + stageStarted = time.Now() + if err = e.repository.SaveValidationFailure(ctx, task); err != nil { + logInboundStage(ctx, task, "save_validation_failure", stageStarted, err) + return gerror.Wrap(err, "save inbound validation failure failed") + } + logInboundStage(ctx, task, "save_validation_failure", stageStarted, nil) + _ = e.saveResultFile(ctx, task) + return nil + } + + stageStarted = time.Now() + if err = e.repository.SaveSuccess(ctx, task); err != nil { + logInboundStage(ctx, task, "save_success", stageStarted, err) + g.Log().Errorf(ctx, "inbound batch DB write failed, batchNo:%s, err:%+v", task.Batch.BatchNo, err) + stageStarted = time.Now() + if saveErr := e.repository.SaveBusinessFailure(ctx, task, "数据库写入失败，本次整批未入库"); saveErr != nil { + logInboundStage(ctx, task, "save_business_failure", stageStarted, saveErr) + return gerror.Wrap(saveErr, "save inbound DB failure failed") + } + logInboundStage(ctx, task, "save_business_failure", stageStarted, nil) + _ = e.saveResultFile(ctx, task) + return nil + } + logInboundStage(ctx, task, "save_success", stageStarted, nil) + _ = e.saveResultFile(ctx, task) + return nil +} + +func (e *executor) saveResultFile(ctx context.Context, task *Task) error { + stageStarted := time.Now() + if err := e.repository.SaveResultFile(ctx, task, resultRequest(task)); err != nil { + g.Log().Errorf(ctx, "save inbound result file status failed, batchNo:%s, err:%+v", task.Batch.BatchNo, err) + logInboundStage(ctx, task, "result_file", stageStarted, err) + return err + } + logInboundStage(ctx, task, "result_file", stageStarted, nil) + return nil +} + +func logInboundStage(ctx context.Context, task *Task, stage string, started time.Time, err error) { + batchNo := "" + rows, valid, failed := 0, 0, 0 + if task != nil { + batchNo = task.Batch.BatchNo + rows = len(task.Rows) + valid = countValidRows(task.Rows) + failed = countFailedRows(task.Rows) + } + logInboundStageByBatchNo(ctx, batchNo, stage, started, rows, valid, failed, err) +} + +func logInboundStageByBatchNo(ctx context.Context, batchNo, stage string, started time.Time, rows, valid, failed int, err error) { + durationMs := time.Since(started).Milliseconds() + if err != nil { + g.Log().Warningf(ctx, "inbound batch stage failed, batchNo:%s, stage:%s, rows:%d, valid:%d, failed:%d, durationMs:%d, err:%+v", + batchNo, stage, rows, valid, failed, durationMs, err) + return + } + g.Log().Infof(ctx, "inbound batch stage completed, batchNo:%s, stage:%s, rows:%d, valid:%d, failed:%d, durationMs:%d", + batchNo, stage, rows, valid, failed, durationMs) +} + +func logInboundTotal(ctx context.Context, batchNo string, task *Task, started time.Time, err error) { + rows, valid, failed := 0, 0, 0 + outcome := "success" + if task != nil { + if task.Batch.BatchNo != "" { + batchNo = task.Batch.BatchNo + } + rows = len(task.Rows) + valid = countValidRows(task.Rows) + failed = countFailedRows(task.Rows) + if task.FailureReason != "" || failed > 0 { + outcome = "business_failed" + } + } + if err != nil { + outcome = "error" + } + duration := time.Since(started) + rowsPerSecond := 0.0 + if duration > 0 && rows > 0 { + rowsPerSecond = float64(rows) / duration.Seconds() + } + if err != nil { + g.Log().Warningf(ctx, "inbound batch finished, batchNo:%s, outcome:%s, rows:%d, valid:%d, failed:%d, durationMs:%d, rowsPerSecond:%.2f, err:%+v", + batchNo, outcome, rows, valid, failed, duration.Milliseconds(), rowsPerSecond, err) + return + } + g.Log().Infof(ctx, "inbound batch finished, batchNo:%s, outcome:%s, rows:%d, valid:%d, failed:%d, durationMs:%d, rowsPerSecond:%.2f", + batchNo, outcome, rows, valid, failed, duration.Milliseconds(), rowsPerSecond) +} + +func (e *executor) populateRows(ctx context.Context, task *Task) error { + switch task.Batch.SourceType { + case consts.BatchSourceTypeFile: + return e.populateExcelRows(ctx, task) + case consts.BatchSourceTypeBatchInput: + return e.populateInputRows(ctx, task) + case consts.BatchSourceTypeRange: + return e.populateRangeRows(task) + default: + return e.populateRowsBySourceShape(ctx, task) + } +} + +func (e *executor) populateRowsBySourceShape(ctx context.Context, task *Task) error { + switch { + case task.Source.ImportFileOssID > 0: + return e.populateExcelRows(ctx, task) + case task.Source.InputText != "" || task.Source.InputFileOssID > 0: + return e.populateInputRows(ctx, task) + case task.Source.Range.StartICCID != "" || task.Source.Range.EndICCID != "": + return e.populateRangeRows(task) + default: + return gerror.Newf("unsupported inbound source type %d", task.Batch.SourceType) + } +} + +func (e *executor) populateExcelRows(ctx context.Context, task *Task) error { + if task.Source.ImportFileOssID <= 0 { + return gerror.New("inbound Excel source importFileOssId is empty") + } + content, name, err := e.repository.ReadOssFile(ctx, task.Source.ImportFileOssID) + if err != nil { + return batch.Retryable(gerror.Wrap(err, "read inbound Excel source failed")) + } + workbook, err := excelize.OpenReader(bytes.NewReader(content)) + if err != nil { + return gerror.Wrapf(err, "open inbound Excel source %s failed", name) + } + defer func() { _ = workbook.Close() }() + sheets := workbook.GetSheetList() + if len(sheets) == 0 { + return gerror.New("inbound Excel source has no sheets") + } + lines, err := workbook.GetRows(sheets[0]) + if err != nil { + return gerror.Wrap(err, "read inbound Excel rows failed") + } + task.Rows = parseTableRows(lines) + return nil +} + +func (e *executor) populateInputRows(ctx context.Context, task *Task) error { + text := task.Source.InputText + if strings.TrimSpace(text) == "" && task.Source.InputFileOssID > 0 { + content, name, err := e.repository.ReadOssFile(ctx, task.Source.InputFileOssID) + if err != nil { + return batch.Retryable(gerror.Wrap(err, "read inbound input source failed")) + } + if strings.ToLower(filepath.Ext(name)) != ".txt" { + g.Log().Warningf(ctx, "inbound input source file is not .txt, batchNo:%s, file:%s", task.Batch.BatchNo, name) + } + text = string(content) + } + task.Rows = parseInputText(text) + return nil +} + +func (e *executor) populateRangeRows(task *Task) error { + rows, err := expandRange(task.Source.Range) + if err != nil { + return err + } + task.Rows = rows + return nil +} + +func (e *executor) validate(ctx context.Context, task *Task) error { + if len(task.Rows) == 0 { + task.FailureReason = "未解析到有效入库数据" + return nil + } + useOptionFallback := shouldUseInboundOptionFallback(task) + if !useOptionFallback { + task.Options = Options{ValidateUpstreamBasicInfo: task.Options.ValidateUpstreamBasicInfo} + } + for _, row := range task.Rows { + normalizeRow(row, task.Options, useOptionFallback) + validateRow(row, task.Source.Range.RangeType) + } + syncInboundOptionsFromRows(task) + markSameBatchDuplicates(task.Rows) + if err := e.markStockDuplicates(ctx, task); err != nil { + return err + } + task.Options.ValidateUpstreamBasicInfo = normalizeUpstreamBasicInfoValidationOption(task.Options.ValidateUpstreamBasicInfo) + if e.upstream != nil && task.Options.ValidateUpstreamBasicInfo == "1" { + if err := e.validateUpstreamBasicInfo(ctx, task); err != nil { + return err + } + } + markIncompleteRows(task.Rows) + markSameBatchDuplicates(task.Rows) + if err := e.markStockDuplicates(ctx, task); err != nil { + return err + } + task.ValidCount = countValidRows(task.Rows) + return nil +} + +func (e *executor) markStockDuplicates(ctx context.Context, task *Task) error { + iccids, msisdns, imsis := rowKeys(task.Rows) + stocks, err := e.repository.FindExistingStocks(ctx, task.Batch.TenantID, iccids, msisdns, imsis) + if err != nil { + return gerror.Wrap(err, "query inbound stock duplicates failed") + } + markStockDuplicates(task.Rows, stocks) + return nil +} + +func parseInputText(text string) []*Row { + lines := strings.Split(strings.ReplaceAll(text, "\r\n", "\n"), "\n") + rows := make([]*Row, 0, len(lines)) + for index, line := range lines { + fields := splitFields(line) + if len(fields) == 0 { + continue + } + if len(fields) == 1 { + rows = append(rows, rowFromSingleIdentifier(index+1, fields[0])) + continue + } + rows = append(rows, rowFromFields(index+1, fields, nil)) + } + return rows +} + +func parseTableRows(lines [][]string) []*Row { + if len(lines) == 0 { + return nil + } + header := headerIndex(lines[0]) + start := 0 + if len(header) > 0 { + start = 1 + } + rows := make([]*Row, 0, len(lines)-start) + for index := start; index < len(lines); index++ { + fields := trimRow(lines[index]) + if rowEmpty(fields) { + continue + } + if len(header) > 0 { + rows = append(rows, rowFromHeader(index+1, fields, header)) + continue + } + rows = append(rows, rowFromFields(index+1, fields, nil)) + } + return rows +} + +func rowFromHeader(rowNo int, fields []string, header map[string]int) *Row { + raw := make(map[string]string, len(header)) + for key, index := range header { + if index >= 0 && index < len(fields) { + raw[key] = fields[index] + } + } + return &Row{ + RowNo: rowNo, + ICCID: valueForHeader(fields, header, "iccid"), + MSISDN: valueForHeader(fields, header, "msisdn"), + IMSI: valueForHeader(fields, header, "imsi"), + IMEI: valueForHeader(fields, header, "imei"), + ChanKey: firstNonEmpty( + valueForHeader(fields, header, "chanKey"), + valueForHeader(fields, header, "chan_key"), + ), + CardType: valueForHeader(fields, header, "cardType"), + UpPkgCode: valueForHeader(fields, header, "upPkgCode"), + SuppliersCode: valueForHeader(fields, header, "suppliersCode"), + UpstreamActivationTime: valueForHeader(fields, header, "upstreamActivationTime"), + InsertToPlugEnabled: valueForHeader(fields, header, "insertToPlugEnabled"), + CarrierTrafficCutoffEnabled: valueForHeader(fields, header, "carrierTrafficCutoffEnabled"), + AgreementExpiryDate: valueForHeader(fields, header, "agreementExpiryDate"), + HasTestPeriod: valueForHeader(fields, header, "hasTestPeriod"), + TestPeriodDays: valueForHeader(fields, header, "testPeriodDays"), + TestPeriodLatestExpiryDate: valueForHeader(fields, header, "testPeriodLatestExpiryDate"), + HasSilentPeriod: valueForHeader(fields, header, "hasSilentPeriod"), + SilentPeriodDays: valueForHeader(fields, header, "silentPeriodDays"), + SilentPeriodLatestExpiryDate: valueForHeader(fields, header, "silentPeriodLatestExpiryDate"), + Raw: raw, + } +} + +func rowFromFields(rowNo int, fields []string, raw map[string]string) *Row { + return &Row{ + RowNo: rowNo, + InputCode: valueAt(fields, 0), + ICCID: valueAt(fields, 0), + MSISDN: valueAt(fields, 1), + IMSI: valueAt(fields, 2), + IMEI: valueAt(fields, 3), + ChanKey: valueAt(fields, 4), + Raw: raw, + } +} + +func rowFromSingleIdentifier(rowNo int, code string) *Row { + code = strings.TrimSpace(code) + row := &Row{ + RowNo: rowNo, + InputCode: code, + Raw: map[string]string{"inputCode": code}, + } + switch inferIdentifierType(code) { + case "iccid": + row.ICCID = code + case "imsi": + row.IMSI = code + case "msisdn": + row.MSISDN = code + } + return row +} + +func inferIdentifierType(code string) string { + code = strings.TrimSpace(code) + switch { + case code == "": + return "" + case len(code) >= minICCIDLength && len(code) <= maxIdentifierLength && allAlphaNumeric(code): + return "iccid" + case len(code) >= minIMSILength && len(code) < minICCIDLength && allDigits(code): + return "imsi" + case len(code) >= minMSISDNLength && len(code) < minIMSILength && allDigits(code): + return "msisdn" + default: + return "" + } +} + +func headerIndex(fields []string) map[string]int { + indexes := make(map[string]int) + for index, field := range fields { + key := normalizeHeader(field) + switch key { + case "iccid", "msisdn", "imsi", "imei", "chankey", "chan_key", + "cardType", "upPkgCode", "suppliersCode", "upstreamActivationTime", + "insertToPlugEnabled", "carrierTrafficCutoffEnabled", "agreementExpiryDate", + "hasTestPeriod", "testPeriodDays", "testPeriodLatestExpiryDate", + "hasSilentPeriod", "silentPeriodDays", "silentPeriodLatestExpiryDate": + if key == "chankey" { + key = "chanKey" + } + indexes[key] = index + } + } + if _, ok := indexes["iccid"]; !ok { + return nil + } + return indexes +} + +func normalizeHeader(value string) string { + value = strings.TrimSpace(value) + value = strings.Trim(value, "\ufeff") + value = strings.TrimRight(value, "*＊") + value = strings.TrimSpace(value) + value = strings.ToLower(value) + value = strings.NewReplacer( + " ", "", + "_", "", + "-", "", + "/", "", + "（", "", + "）", "", + "(", "", + ")", "", + ":", "", + "：", "", + ).Replace(value) + switch value { + case "iccid", "iccid号", "集成电路卡识别码": + return "iccid" + case "msisdn", "卡号", "手机号", "接入号", "号码": + return "msisdn" + case "imsi", "国际移动用户识别码": + return "imsi" + case "imei", "国际移动设备识别码": + return "imei" + case "chankey", "通道", "通道编号", "通道编码", "通道标识": + return "chankey" + case "cardtype", "卡类型", "卡片类型": + return "cardType" + case "uppkgcode", "上游套餐", "上游套餐编码", "上游套餐编号": + return "upPkgCode" + case "supplierscode", "suppliercode", "供应商", "供应商编码", "供应商编号": + return "suppliersCode" + case "upstreamactivationtime", "上游激活时间", "上游给出激活时间", "上游开户时间": + return "upstreamActivationTime" + case "inserttoplugenabled", "是否贴片转插拔", "贴片转插拔": + return "insertToPlugEnabled" + case "carriertrafficcutoffenabled", "是否运营商达量断网", "运营商达量断网": + return "carrierTrafficCutoffEnabled" + case "agreementexpirydate", "协议到期时间", "协议期到期时间": + return "agreementExpiryDate" + case "hastestperiod", "是否有测试期", "有测试期": + return "hasTestPeriod" + case "testperioddays", "测试期天数": + return "testPeriodDays" + case "testperiodlatestexpirydate", "测试期最晚过期日期", "测试期最晚到期日期": + return "testPeriodLatestExpiryDate" + case "hassilentperiod", "是否有沉默期", "有沉默期": + return "hasSilentPeriod" + case "silentperioddays", "沉默期天数": + return "silentPeriodDays" + case "silentperiodlatestexpirydate", "沉默期最晚过期日期", "沉默期最晚到期日期": + return "silentPeriodLatestExpiryDate" + default: + return value + } +} + +func splitFields(line string) []string { + line = strings.TrimSpace(line) + if line == "" { + return nil + } + parts := tokenSplitter.Split(line, -1) + fields := make([]string, 0, len(parts)) + for _, part := range parts { + if value := strings.TrimSpace(part); value != "" { + fields = append(fields, value) + } + } + return fields +} + +func trimRow(values []string) []string { + fields := make([]string, len(values)) + for index, value := range values { + fields[index] = strings.TrimSpace(value) + } + return fields +} + +func rowEmpty(fields []string) bool { + for _, field := range fields { + if strings.TrimSpace(field) != "" { + return false + } + } + return true +} + +func expandRange(input Range) ([]*Row, error) { + rangeType := strings.ToLower(strings.TrimSpace(input.RangeType)) + if rangeType == "" { + rangeType = "iccid" + } + if rangeType != "iccid" && rangeType != "msisdn" && rangeType != "imsi" { + return nil, gerror.New("rangeType must be iccid, msisdn or imsi") + } + + if rangeType == "msisdn" { + return expandSelectedRange(input.StartMSISDN, input.EndMSISDN, "msisdn") + } + if rangeType == "imsi" { + return expandSelectedRange(input.StartIMSI, input.EndIMSI, "imsi") + } + + iccidValues, err := iccidRange(input.StartICCID, input.EndICCID, true) + if err != nil { + return nil, gerror.Wrap(err, "invalid ICCID range") + } + msisdnValues, err := numericRange(input.StartMSISDN, input.EndMSISDN, false) + if err != nil { + return nil, gerror.Wrap(err, "invalid MSISDN range") + } + imsiValues, err := numericRange(input.StartIMSI, input.EndIMSI, false) + if err != nil { + return nil, gerror.Wrap(err, "invalid IMSI range") + } + if err = ensureRangeLength(len(iccidValues), len(msisdnValues), "MSISDN"); err != nil { + return nil, err + } + if err = ensureRangeLength(len(iccidValues), len(imsiValues), "IMSI"); err != nil { + return nil, err + } + rows := make([]*Row, 0, len(iccidValues)) + for index, iccid := range iccidValues { + row := &Row{RowNo: index + 1, ICCID: iccid} + if len(msisdnValues) > 0 { + row.MSISDN = msisdnValues[index] + } + if len(imsiValues) > 0 { + row.IMSI = imsiValues[index] + } + rows = append(rows, row) + } + return rows, nil +} + +func expandSelectedRange(start, end, rangeType string) ([]*Row, error) { + values, err := numericRange(start, end, true) + if err != nil { + return nil, gerror.Wrapf(err, "invalid %s range", strings.ToUpper(rangeType)) + } + rows := make([]*Row, 0, len(values)) + for index, value := range values { + row := &Row{RowNo: index + 1} + switch rangeType { + case "msisdn": + row.MSISDN = value + case "imsi": + row.IMSI = value + } + rows = append(rows, row) + } + return rows, nil +} + +func iccidRange(start, end string, required bool) ([]string, error) { + start = strings.TrimSpace(start) + end = strings.TrimSpace(end) + if start == "" && end == "" && !required { + return nil, nil + } + if start == "" || end == "" { + return nil, gerror.New("start and end are required together") + } + if !allAlphaNumeric(start) || !allAlphaNumeric(end) { + return nil, gerror.New("range values must be alphanumeric") + } + if len(start) != len(end) { + return nil, gerror.New("range values must have the same length") + } + if start == end { + return []string{start}, nil + } + startPrefix, startTail := splitTrailingDigits(start) + endPrefix, endTail := splitTrailingDigits(end) + if startTail == "" || endTail == "" { + return nil, gerror.New("range values must contain incrementable numeric suffixes") + } + if startPrefix != endPrefix { + return nil, gerror.New("range value prefixes must match") + } + tails, err := numericRange(startTail, endTail, true) + if err != nil { + return nil, err + } + values := make([]string, 0, len(tails)) + for _, tail := range tails { + values = append(values, startPrefix+tail) + } + return values, nil +} + +func numericRange(start, end string, required bool) ([]string, error) { + start = strings.TrimSpace(start) + end = strings.TrimSpace(end) + if start == "" && end == "" && !required { + return nil, nil + } + if start == "" || end == "" { + return nil, gerror.New("start and end are required together") + } + if !allDigits(start) || !allDigits(end) { + return nil, gerror.New("range values must be numeric") + } + startInt, ok := new(big.Int).SetString(start, 10) + if !ok { + return nil, gerror.New("invalid start value") + } + endInt, ok := new(big.Int).SetString(end, 10) + if !ok { + return nil, gerror.New("invalid end value") + } + if startInt.Cmp(endInt) > 0 { + return nil, gerror.New("start must be less than or equal to end") + } + count := new(big.Int).Sub(endInt, startInt) + count.Add(count, big.NewInt(1)) + if !count.IsInt64() || count.Int64() > 100000 { + return nil, gerror.New("range is too large") + } + width := len(start) + if len(end) > width { + width = len(end) + } + values := make([]string, 0, int(count.Int64())) + current := new(big.Int).Set(startInt) + for current.Cmp(endInt) <= 0 { + value := current.String() + if len(value) < width { + value = strings.Repeat("0", width-len(value)) + value + } + values = append(values, value) + current.Add(current, big.NewInt(1)) + } + return values, nil +} + +func ensureRangeLength(iccidLength, otherLength int, name string) error { + if otherLength > 0 && otherLength != iccidLength { + return gerror.Newf("%s range count must match ICCID range count", name) + } + return nil +} + +func normalizeRow(row *Row, options Options, useOptionFallback bool) { + if row == nil { + return + } + row.ICCID = strings.TrimSpace(row.ICCID) + row.MSISDN = strings.TrimSpace(row.MSISDN) + row.IMSI = strings.TrimSpace(row.IMSI) + row.IMEI = strings.TrimSpace(row.IMEI) + row.ChanKey = strings.TrimSpace(firstNonEmpty(row.ChanKey, fallbackOption(useOptionFallback, options.ChanKey))) + row.CardType = normalizeCardType(firstNonEmpty(row.CardType, fallbackOption(useOptionFallback, options.CardType))) + row.UpPkgCode = strings.TrimSpace(firstNonEmpty(row.UpPkgCode, fallbackOption(useOptionFallback, options.UpPkgCode))) + row.SuppliersCode = strings.TrimSpace(firstNonEmpty(row.SuppliersCode, fallbackOption(useOptionFallback, options.SuppliersCode))) + row.UpstreamActivationTime = strings.TrimSpace(firstNonEmpty(row.UpstreamActivationTime, fallbackOption(useOptionFallback, options.UpstreamActivationTime))) + row.InsertToPlugEnabled = normalizeYesNo(firstNonEmpty(row.InsertToPlugEnabled, fallbackOption(useOptionFallback, options.InsertToPlugEnabled))) + row.CarrierTrafficCutoffEnabled = normalizeYesNo(firstNonEmpty(row.CarrierTrafficCutoffEnabled, fallbackOption(useOptionFallback, options.CarrierTrafficCutoffEnabled))) + row.AgreementExpiryDate = strings.TrimSpace(firstNonEmpty(row.AgreementExpiryDate, fallbackOption(useOptionFallback, options.AgreementExpiryDate))) + row.HasTestPeriod = normalizeYesNo(firstNonEmpty(row.HasTestPeriod, fallbackOption(useOptionFallback, options.HasTestPeriod))) + row.TestPeriodDays = strings.TrimSpace(firstNonEmpty(row.TestPeriodDays, fallbackOption(useOptionFallback, options.TestPeriodDays))) + row.TestPeriodLatestExpiryDate = strings.TrimSpace(firstNonEmpty(row.TestPeriodLatestExpiryDate, fallbackOption(useOptionFallback, options.TestPeriodLatestExpiryDate))) + row.HasSilentPeriod = normalizeYesNo(firstNonEmpty(row.HasSilentPeriod, fallbackOption(useOptionFallback, options.HasSilentPeriod))) + row.SilentPeriodDays = strings.TrimSpace(firstNonEmpty(row.SilentPeriodDays, fallbackOption(useOptionFallback, options.SilentPeriodDays))) + row.SilentPeriodLatestExpiryDate = strings.TrimSpace(firstNonEmpty(row.SilentPeriodLatestExpiryDate, fallbackOption(useOptionFallback, options.SilentPeriodLatestExpiryDate))) +} + +func fallbackOption(enabled bool, value string) string { + if !enabled { + return "" + } + return value +} + +func shouldUseInboundOptionFallback(task *Task) bool { + if task == nil { + return false + } + return task.Batch.SourceType != consts.BatchSourceTypeFile && task.Source.ImportFileOssID <= 0 +} + +func validateRow(row *Row, rangeType string) { + if row == nil { + return + } + var reasons []string + if row.ICCID == "" && row.MSISDN == "" && row.IMSI == "" { + reasons = append(reasons, "ICCID/MSISDN/IMSI至少填写一个") + } + if row.ICCID != "" && (len(row.ICCID) < minICCIDLength || len(row.ICCID) > maxIdentifierLength || !allAlphaNumeric(row.ICCID)) { + reasons = append(reasons, "ICCID格式不正确") + } + if row.MSISDN != "" && (len(row.MSISDN) < minMSISDNLength || len(row.MSISDN) > maxIdentifierLength || !allDigits(row.MSISDN)) { + reasons = append(reasons, "MSISDN格式不正确") + } + if row.IMSI != "" && (len(row.IMSI) < minIMSILength || len(row.IMSI) > maxIdentifierLength || !allDigits(row.IMSI)) { + reasons = append(reasons, "IMSI格式不正确") + } + if row.IMEI != "" && (len(row.IMEI) > maxIdentifierLength || !allDigits(row.IMEI)) { + reasons = append(reasons, "IMEI格式不正确") + } + if row.ChanKey == "" { + reasons = append(reasons, "chanKey不能为空") + } + if len(reasons) > 0 { + row.Reason = strings.Join(reasons, "；") + } +} + +func markSameBatchDuplicates(rows []*Row) { + seenICCID := make(map[string]struct{}, len(rows)) + seenMSISDN := make(map[string]struct{}, len(rows)) + seenIMSI := make(map[string]struct{}, len(rows)) + for _, row := range rows { + if row == nil || row.Reason != "" { + continue + } + if row.ICCID != "" { + if _, ok := seenICCID[row.ICCID]; ok { + appendReason(row, "同批重复ICCID") + } else { + seenICCID[row.ICCID] = struct{}{} + } + } + if row.MSISDN != "" { + if _, ok := seenMSISDN[row.MSISDN]; ok { + appendReason(row, "同批重复MSISDN") + } else { + seenMSISDN[row.MSISDN] = struct{}{} + } + } + if row.IMSI != "" { + if _, ok := seenIMSI[row.IMSI]; ok { + appendReason(row, "同批重复IMSI") + } else { + seenIMSI[row.IMSI] = struct{}{} + } + } + } +} + +func markStockDuplicates(rows []*Row, stocks []Stock) { + existingICCID := make(map[string]struct{}, len(stocks)) + existingMSISDN := make(map[string]struct{}, len(stocks)) + existingIMSI := make(map[string]struct{}, len(stocks)) + for _, stock := range stocks { + if stock.ICCID != "" { + existingICCID[strings.TrimSpace(stock.ICCID)] = struct{}{} + } + if stock.MSISDN != "" { + existingMSISDN[strings.TrimSpace(stock.MSISDN)] = struct{}{} + } + if stock.IMSI != "" { + existingIMSI[strings.TrimSpace(stock.IMSI)] = struct{}{} + } + } + for _, row := range rows { + if row == nil || row.Reason != "" { + continue + } + if _, ok := existingICCID[row.ICCID]; ok && row.ICCID != "" { + appendReason(row, "ICCID已存在库存") + } + if _, ok := existingMSISDN[row.MSISDN]; ok && row.MSISDN != "" { + appendReason(row, "MSISDN已存在库存") + } + if _, ok := existingIMSI[row.IMSI]; ok && row.IMSI != "" { + appendReason(row, "IMSI已存在库存") + } + } +} + +func markIncompleteRows(rows []*Row) { + for _, row := range rows { + if row == nil || row.Reason != "" { + continue + } + missing := make([]string, 0, 3) + if strings.TrimSpace(row.ICCID) == "" { + missing = append(missing, "ICCID") + } + if strings.TrimSpace(row.MSISDN) == "" { + missing = append(missing, "MSISDN") + } + if strings.TrimSpace(row.IMSI) == "" { + missing = append(missing, "IMSI") + } + if len(missing) > 0 { + appendReason(row, "三码信息不完整: "+strings.Join(missing, ",")) + } + } +} + +func normalizeUpstreamBasicInfoValidationOption(value string) string { + normalized := normalizeYesNo(value) + if normalized == "1" { + return "1" + } + return "0" +} + +func syncInboundOptionsFromRows(task *Task) { + if task == nil { + return + } + checks := []struct { + name string + get func(*Row) string + assign func(string) + }{ + {name: "通道编号", get: func(row *Row) string { return row.ChanKey }, assign: func(value string) { task.Options.ChanKey = value }}, + {name: "卡类型", get: func(row *Row) string { return row.CardType }, assign: func(value string) { task.Options.CardType = value }}, + {name: "上游套餐编码", get: func(row *Row) string { return row.UpPkgCode }, assign: func(value string) { task.Options.UpPkgCode = value }}, + {name: "供应商编码", get: func(row *Row) string { return row.SuppliersCode }, assign: func(value string) { task.Options.SuppliersCode = value }}, + {name: "上游激活时间", get: func(row *Row) string { return row.UpstreamActivationTime }, assign: func(value string) { task.Options.UpstreamActivationTime = value }}, + {name: "是否贴片转插拔", get: func(row *Row) string { return row.InsertToPlugEnabled }, assign: func(value string) { task.Options.InsertToPlugEnabled = value }}, + {name: "是否运营商达量断网", get: func(row *Row) string { return row.CarrierTrafficCutoffEnabled }, assign: func(value string) { task.Options.CarrierTrafficCutoffEnabled = value }}, + {name: "协议到期时间", get: func(row *Row) string { return row.AgreementExpiryDate }, assign: func(value string) { task.Options.AgreementExpiryDate = value }}, + {name: "是否有测试期", get: func(row *Row) string { return row.HasTestPeriod }, assign: func(value string) { task.Options.HasTestPeriod = value }}, + {name: "测试期天数", get: func(row *Row) string { return row.TestPeriodDays }, assign: func(value string) { task.Options.TestPeriodDays = value }}, + {name: "测试期最晚过期日期", get: func(row *Row) string { return row.TestPeriodLatestExpiryDate }, assign: func(value string) { task.Options.TestPeriodLatestExpiryDate = value }}, + {name: "是否有沉默期", get: func(row *Row) string { return row.HasSilentPeriod }, assign: func(value string) { task.Options.HasSilentPeriod = value }}, + {name: "沉默期天数", get: func(row *Row) string { return row.SilentPeriodDays }, assign: func(value string) { task.Options.SilentPeriodDays = value }}, + {name: "沉默期最晚过期日期", get: func(row *Row) string { return row.SilentPeriodLatestExpiryDate }, assign: func(value string) { task.Options.SilentPeriodLatestExpiryDate = value }}, + } + for _, check := range checks { + value, ok := sameBatchOptionValue(task.Rows, check.get) + if !ok { + markRowsWithReason(task.Rows, check.name+"必须同批一致") + if task.FailureReason == "" { + task.FailureReason = "同批入库配置必须一致" + } + continue + } + if value != "" { + check.assign(value) + } + } + fillRowOptionsFromTask(task.Rows, task.Options) +} + +func sameBatchOptionValue(rows []*Row, get func(*Row) string) (string, bool) { + value := "" + for _, row := range rows { + if row == nil { + continue + } + current := strings.TrimSpace(get(row)) + if current == "" { + continue + } + if value == "" { + value = current + continue + } + if value != current { + return "", false + } + } + return value, true +} + +func markRowsWithReason(rows []*Row, reason string) { + for _, row := range rows { + if row != nil { + appendReason(row, reason) + } + } +} + +func fillRowOptionsFromTask(rows []*Row, options Options) { + for _, row := range rows { + if row == nil { + continue + } + row.ChanKey = firstNonEmpty(row.ChanKey, options.ChanKey) + row.CardType = firstNonEmpty(row.CardType, options.CardType) + row.UpPkgCode = firstNonEmpty(row.UpPkgCode, options.UpPkgCode) + row.SuppliersCode = firstNonEmpty(row.SuppliersCode, options.SuppliersCode) + row.UpstreamActivationTime = firstNonEmpty(row.UpstreamActivationTime, options.UpstreamActivationTime) + row.InsertToPlugEnabled = firstNonEmpty(row.InsertToPlugEnabled, options.InsertToPlugEnabled) + row.CarrierTrafficCutoffEnabled = firstNonEmpty(row.CarrierTrafficCutoffEnabled, options.CarrierTrafficCutoffEnabled) + row.AgreementExpiryDate = firstNonEmpty(row.AgreementExpiryDate, options.AgreementExpiryDate) + row.HasTestPeriod = firstNonEmpty(row.HasTestPeriod, options.HasTestPeriod) + row.TestPeriodDays = firstNonEmpty(row.TestPeriodDays, options.TestPeriodDays) + row.TestPeriodLatestExpiryDate = firstNonEmpty(row.TestPeriodLatestExpiryDate, options.TestPeriodLatestExpiryDate) + row.HasSilentPeriod = firstNonEmpty(row.HasSilentPeriod, options.HasSilentPeriod) + row.SilentPeriodDays = firstNonEmpty(row.SilentPeriodDays, options.SilentPeriodDays) + row.SilentPeriodLatestExpiryDate = firstNonEmpty(row.SilentPeriodLatestExpiryDate, options.SilentPeriodLatestExpiryDate) + } +} + +func (e *executor) validateUpstreamBasicInfo(ctx context.Context, task *Task) error { + started := time.Now() + groups := groupRowsForUpstream(task.Rows) + batchSize := inboundUpstreamBatchSize(ctx) + jobs := make([]upstreamBasicInfoJob, 0, len(groups)) + upstreamRows := 0 + for chanKey, rows := range groups { + upstreamRows += len(rows) + for _, chunk := range chunkRows(rows, batchSize) { + if len(chunk) == 0 { + continue + } + jobs = append(jobs, upstreamBasicInfoJob{ + chanKey: chanKey, + rows: chunk, + }) + } + } + if len(jobs) == 0 { + return nil + } + workers := inboundUpstreamConcurrency(ctx) + if workers < 1 { + workers = 1 + } + if workers > len(jobs) { + workers = len(jobs) + } + g.Log().Infof(ctx, "inbound upstream basic-info started, batchNo:%s, rows:%d, groups:%d, jobs:%d, batchSize:%d, concurrency:%d", + task.Batch.BatchNo, upstreamRows, len(groups), len(jobs), batchSize, workers) + jobCh := make(chan upstreamBasicInfoJob) + var wg sync.WaitGroup + for i := 0; i < workers; i++ { + wg.Add(1) + go func() { + defer wg.Done() + for job := range jobCh { + e.validateUpstreamBasicInfoChunk(ctx, task.Batch.BatchNo, job.chanKey, job.rows) + } + }() + } + for _, job := range jobs { + jobCh <- job + } + close(jobCh) + wg.Wait() + g.Log().Infof(ctx, "inbound upstream basic-info completed, batchNo:%s, rows:%d, groups:%d, jobs:%d, batchSize:%d, concurrency:%d, failed:%d, durationMs:%d", + task.Batch.BatchNo, upstreamRows, len(groups), len(jobs), batchSize, workers, countFailedRows(task.Rows), time.Since(started).Milliseconds()) + return nil +} + +func (e *executor) validateUpstreamBasicInfoChunk(ctx context.Context, batchNo, chanKey string, rows []*Row) { + if len(rows) == 0 { + return + } + started := time.Now() + simKeys := make([]*model.SimKey, 0, len(rows)) + for _, row := range rows { + simKeys = append(simKeys, simKeyFromRow(row, chanKey)) + } + if e.rateLimiter != nil { + // The limiter controls upstream HTTP requests. One chunk produces one + // request, regardless of how many SIM keys it contains. + if err := e.rateLimiter.Wait(ctx, chanKey, 1); err != nil { + for _, row := range rows { + appendReason(row, "上游三码校验限流失败: "+err.Error()) + } + g.Log().Warningf(ctx, "inbound upstream basic-info rate limit failed, batchNo:%s, chanKey:%s, rows:%d, durationMs:%d, err:%+v", + batchNo, chanKey, len(rows), time.Since(started).Milliseconds(), err) + return + } + } + response, err := e.upstream.QuerySimBasicInfo(ctx, &model.SimBasicInfoInput{ + SimKeys: simKeys, + ReqData: model.SimInputReqNullDate{}, + }) + if err != nil { + for _, row := range rows { + appendReason(row, "上游三码校验失败: "+err.Error()) + } + g.Log().Warningf(ctx, "inbound upstream basic-info chunk failed, batchNo:%s, chanKey:%s, rows:%d, durationMs:%d, err:%+v", + batchNo, chanKey, len(rows), time.Since(started).Milliseconds(), err) + return + } + applyUpstreamBasicInfoResponse(rows, response) + if duration := time.Since(started); duration >= time.Second { + g.Log().Infof(ctx, "inbound upstream basic-info chunk completed, batchNo:%s, chanKey:%s, rows:%d, response:%d, failed:%d, durationMs:%d", + batchNo, chanKey, len(rows), len(response), countFailedRows(rows), duration.Milliseconds()) + } +} + +func simKeyFromRow(row *Row, chanKey string) *model.SimKey { + if row == nil { + return &model.SimKey{ChanKey: chanKey} + } + if row.ICCID != "" { + return &model.SimKey{ + ChanKey: chanKey, + ICCID: row.ICCID, + } + } + return &model.SimKey{ + ChanKey: chanKey, + MSISDN: row.MSISDN, + IMSI: row.IMSI, + } +} + +func groupRowsForUpstream(rows []*Row) map[string][]*Row { + groups := make(map[string][]*Row) + for _, row := range rows { + if row == nil || row.Reason != "" || row.ChanKey == "" { + continue + } + groups[row.ChanKey] = append(groups[row.ChanKey], row) + } + return groups +} + +type upstreamBasicInfoJob struct { + chanKey string + rows []*Row +} + +func chunkRows(rows []*Row, size int) [][]*Row { + if size <= 0 { + size = defaultInboundUpstreamBatchSize + } + if len(rows) == 0 { + return nil + } + chunks := make([][]*Row, 0, (len(rows)+size-1)/size) + for start := 0; start < len(rows); start += size { + end := start + size + if end > len(rows) { + end = len(rows) + } + chunks = append(chunks, rows[start:end]) + } + return chunks +} + +func inboundUpstreamBatchSize(ctx context.Context) int { + value := g.Cfg().MustGet(ctx, inboundUpstreamBatchSizeConfigKey, defaultInboundUpstreamBatchSize).Int() + if value <= 0 { + return defaultInboundUpstreamBatchSize + } + return value +} + +func inboundUpstreamConcurrency(ctx context.Context) int { + value := g.Cfg().MustGet(ctx, inboundUpstreamConcurrencyKey, defaultInboundUpstreamConcurrency).Int() + if value <= 0 { + return defaultInboundUpstreamConcurrency + } + return value +} + +func applyUpstreamBasicInfoResponse(rows []*Row, response []model.SimBaseRes[model.SimBasicInfoItem]) { + if len(response) == 0 { + for _, row := range rows { + appendReason(row, "上游未返回SIM卡信息") + } + return + } + index := newBasicInfoResponseIndex(response) + for _, row := range rows { + item, ok := index.take(row) + if !ok { + appendReason(row, "上游未返回SIM卡信息") + continue + } + if item.Code != 0 { + reason := strings.TrimSpace(item.Message) + if reason == "" { + reason = "上游返回失败" + } + appendReason(row, "上游三码校验失败: "+reason) + continue + } + info := normalizeBasicInfo(item.ResData) + if reason := upstreamBasicInfoEmptyReason(item, info); reason != "" { + appendReason(row, reason) + continue + } + if reason := compareBasicInfo(row, info); reason != "" { + appendReason(row, reason) + continue + } + if reason := completeBasicInfoReason(info); reason != "" { + appendReason(row, reason) + continue + } + applyBasicInfo(row, info) + } +} + +type basicInfoResponseIndex struct { + response []model.SimBaseRes[model.SimBasicInfoItem] + infos []model.SimBasicInfoItem + used []bool + byICCID map[string][]int + byMSISDN map[string][]int + byIMSI map[string][]int +} + +func newBasicInfoResponseIndex(response []model.SimBaseRes[model.SimBasicInfoItem]) *basicInfoResponseIndex { + index := &basicInfoResponseIndex{ + response: response, + infos: make([]model.SimBasicInfoItem, len(response)), + used: make([]bool, len(response)), + byICCID: make(map[string][]int, len(response)), + byMSISDN: make(map[string][]int, len(response)), + byIMSI: make(map[string][]int, len(response)), + } + for i := range response { + info := normalizeBasicInfo(response[i].ResData) + index.infos[i] = info + index.addInfoKeys(i, info) + if response[i].SimKey != nil { + index.addSimKeyKeys(i, response[i].SimKey) + } + } + return index +} + +func (i *basicInfoResponseIndex) addInfoKeys(index int, info model.SimBasicInfoItem) { + appendResponseIndex(i.byICCID, iccidIndexKey(info.ICCID), index) + appendResponseIndex(i.byMSISDN, identifierIndexKey(info.MSISDN), index) + appendResponseIndex(i.byIMSI, identifierIndexKey(info.IMSI), index) +} + +func (i *basicInfoResponseIndex) addSimKeyKeys(index int, simKey *model.SimKey) { + if simKey == nil { + return + } + appendResponseIndex(i.byICCID, iccidIndexKey(simKey.ICCID), index) + appendResponseIndex(i.byMSISDN, identifierIndexKey(simKey.MSISDN), index) + appendResponseIndex(i.byIMSI, identifierIndexKey(simKey.IMSI), index) +} + +func appendResponseIndex(values map[string][]int, key string, index int) { + if key == "" { + return + } + values[key] = append(values[key], index) +} + +func (i *basicInfoResponseIndex) take(row *Row) (model.SimBaseRes[model.SimBasicInfoItem], bool) { + if i == nil || row == nil { + return model.SimBaseRes[model.SimBasicInfoItem]{}, false + } + for _, candidates := range [][]int{ + i.byICCID[iccidIndexKey(row.ICCID)], + i.byMSISDN[identifierIndexKey(row.MSISDN)], + i.byIMSI[identifierIndexKey(row.IMSI)], + } { + for _, candidate := range candidates { + if candidate < 0 || candidate >= len(i.response) || i.used[candidate] { + continue + } + if rowMatchesBasicInfo(row, i.infos[candidate]) || rowMatchesSimKey(row, i.response[candidate].SimKey) { + i.used[candidate] = true + return i.response[candidate], true + } + } + } + return model.SimBaseRes[model.SimBasicInfoItem]{}, false +} + +func iccidIndexKey(value string) string { + return strings.ToUpper(strings.TrimSpace(value)) +} + +func identifierIndexKey(value string) string { + return strings.TrimSpace(value) +} + +func compareBasicInfo(row *Row, info model.SimBasicInfoItem) string { + missing := make([]string, 0, 3) + mismatches := make([]string, 0, 3) + if row.ICCID != "" { + actual := strings.TrimSpace(info.ICCID) + if actual == "" { + missing = append(missing, "ICCID") + } else if !sameICCID(actual, row.ICCID) { + mismatches = append(mismatches, "ICCID") + } + } + if row.MSISDN != "" { + actual := strings.TrimSpace(info.MSISDN) + if actual == "" { + missing = append(missing, "MSISDN") + } else if actual != row.MSISDN { + mismatches = append(mismatches, "MSISDN") + } + } + if row.IMSI != "" { + actual := strings.TrimSpace(info.IMSI) + if actual == "" { + missing = append(missing, "IMSI") + } else if actual != row.IMSI { + mismatches = append(mismatches, "IMSI") + } + } + if len(missing) > 0 { + return "上游未返回三码信息: " + strings.Join(missing, ",") + } + if len(mismatches) == 0 { + return "" + } + return "上游三码校验不一致: " + strings.Join(mismatches, ",") +} + +func normalizeBasicInfo(info model.SimBasicInfoItem) model.SimBasicInfoItem { + info.ICCID = strings.TrimSpace(info.ICCID) + info.MSISDN = strings.TrimSpace(info.MSISDN) + info.IMSI = strings.TrimSpace(info.IMSI) + info.ActiveDate = strings.TrimSpace(info.ActiveDate) + info.OpenDate = strings.TrimSpace(info.OpenDate) + info.Remark = strings.TrimSpace(info.Remark) + return info +} + +func upstreamBasicInfoEmptyReason(item model.SimBaseRes[model.SimBasicInfoItem], info model.SimBasicInfoItem) string { + if info.ICCID != "" || info.MSISDN != "" || info.IMSI != "" { + return "" + } + message := strings.TrimSpace(item.Message) + if message == "" { + return "" + } + if strings.Contains(message, "未查询") || strings.Contains(message, "不存在") { + return "上游三码校验失败: " + message + } + return "" +} + +func completeBasicInfoReason(info model.SimBasicInfoItem) string { + missing := make([]string, 0, 3) + if strings.TrimSpace(info.ICCID) == "" { + missing = append(missing, "ICCID") + } + if strings.TrimSpace(info.MSISDN) == "" { + missing = append(missing, "MSISDN") + } + if strings.TrimSpace(info.IMSI) == "" { + missing = append(missing, "IMSI") + } + if len(missing) == 0 { + return "" + } + return "上游未返回完整三码信息: " + strings.Join(missing, ",") +} + +func applyBasicInfo(row *Row, info model.SimBasicInfoItem) { + if row == nil { + return + } + row.ICCID = strings.TrimSpace(info.ICCID) + row.MSISDN = strings.TrimSpace(info.MSISDN) + row.IMSI = strings.TrimSpace(info.IMSI) +} + +func rowMatchesBasicInfo(row *Row, info model.SimBasicInfoItem) bool { + if row == nil { + return false + } + if row.ICCID != "" && info.ICCID != "" && sameICCID(info.ICCID, row.ICCID) { + return true + } + if row.MSISDN != "" && info.MSISDN != "" && sameIdentifier(info.MSISDN, row.MSISDN) { + return true + } + if row.IMSI != "" && info.IMSI != "" && sameIdentifier(info.IMSI, row.IMSI) { + return true + } + return false +} + +func rowMatchesSimKey(row *Row, simKey *model.SimKey) bool { + if row == nil || simKey == nil { + return false + } + return rowMatchesBasicInfo(row, model.SimBasicInfoItem{ + ICCID: simKey.ICCID, + MSISDN: simKey.MSISDN, + IMSI: simKey.IMSI, + }) +} + +func sameICCID(a, b string) bool { + return strings.EqualFold(strings.TrimSpace(a), strings.TrimSpace(b)) +} + +func sameIdentifier(a, b string) bool { + return strings.TrimSpace(a) == strings.TrimSpace(b) +} + +func applyCascadeFailure(task *Task) { + for _, row := range task.Rows { + if row != nil && row.Reason == "" { + row.Reason = cascadeFailureReason + } + } + if task.FailureReason == "" { + task.FailureReason = "同批存在校验失败，本次整批未入库" + } +} + +func hasFailedRows(rows []*Row) bool { + for _, row := range rows { + if row != nil && row.Reason != "" { + return true + } + } + return false +} + +func countValidRows(rows []*Row) int { + count := 0 + for _, row := range rows { + if row != nil && row.Reason == "" { + count++ + } + } + return count +} + +func countFailedRows(rows []*Row) int { + count := 0 + for _, row := range rows { + if row != nil && row.Reason != "" { + count++ + } + } + return count +} + +func rowKeys(rows []*Row) ([]string, []string, []string) { + iccidSet := make(map[string]struct{}, len(rows)) + msisdnSet := make(map[string]struct{}, len(rows)) + imsiSet := make(map[string]struct{}, len(rows)) + for _, row := range rows { + if row == nil || row.Reason != "" { + continue + } + if row.ICCID != "" { + iccidSet[row.ICCID] = struct{}{} + } + if row.MSISDN != "" { + msisdnSet[row.MSISDN] = struct{}{} + } + if row.IMSI != "" { + imsiSet[row.IMSI] = struct{}{} + } + } + return setKeys(iccidSet), setKeys(msisdnSet), setKeys(imsiSet) +} + +func setKeys(values map[string]struct{}) []string { + keys := make([]string, 0, len(values)) + for key := range values { + keys = append(keys, key) + } + return keys +} + +func appendReason(row *Row, reason string) { + if row.Reason == "" { + row.Reason = reason + return + } + row.Reason += "；" + reason +} + +func allDigits(value string) bool { + if value == "" { + return false + } + for _, r := range value { + if r < '0' || r > '9' { + return false + } + } + return true +} + +func splitTrailingDigits(value string) (prefix, digits string) { + index := len(value) + for index > 0 { + ch := value[index-1] + if ch < '0' || ch > '9' { + break + } + index-- + } + return value[:index], value[index:] +} + +func allAlphaNumeric(value string) bool { + if value == "" { + return false + } + for _, r := range value { + if (r >= '0' && r <= '9') || (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') { + continue + } + return false + } + return true +} + +func firstNonEmpty(values ...string) string { + for _, value := range values { + if strings.TrimSpace(value) != "" { + return strings.TrimSpace(value) + } + } + return "" +} + +func normalizeYesNo(value string) string { + value = strings.TrimSpace(strings.ToLower(value)) + if value == "" { + return "" + } + value = strings.NewReplacer(" ", "", "　", "").Replace(value) + switch value { + case "0", "是", "true", "y", "yes", "启用", "开启", "有": + return "0" + case "1", "否", "false", "n", "no", "禁用", "关闭", "无": + return "1" + default: + return value + } +} + +func normalizeCardType(value string) string { + value = strings.TrimSpace(value) + if value == "" { + return "" + } + compact := strings.NewReplacer(" ", "", "　", "", "-", "", "_", "", "‑", "", "－", "", "×", "x", "X", "x").Replace(strings.ToLower(value)) + switch compact { + case "1", "通用插拔sim卡", "通用插拔卡", "插拔卡": + return "1" + case "2", "贴片m2m卡5x6mm", "m2m5x6", "贴片卡5x6": + return "2" + case "3", "贴片m2m卡2x2mm", "m2m2x2", "贴片卡2x2": + return "3" + case "4", "mp1消费级插拔物联卡", "mp1": + return "4" + case "5", "mp2工业级插拔物联卡", "mp2": + return "5" + case "6", "ms0消费级贴片物联卡", "ms0": + return "6" + case "7", "ms1工业级贴片物联卡", "ms1": + return "7" + default: + return value + } +} + +func valueAt(values []string, index int) string { + if index < 0 || index >= len(values) { + return "" + } + return strings.TrimSpace(values[index]) +} + +func valueForHeader(values []string, header map[string]int, key string) string { + index, ok := header[key] + if !ok { + return "" + } + return valueAt(values, index) +} + +func resultRequest(task *Task) batchresult.Request { + rows := make([]batchresult.Row, 0, len(task.Rows)) + for _, row := range task.Rows { + if row == nil { + continue + } + rows = append(rows, batchresult.Row{ + RowNo: row.RowNo, + ICCID: row.ICCID, + MSISDN: row.MSISDN, + IMSI: row.IMSI, + Success: row.Reason == "", + Reason: row.Reason, + }) + } + return batchresult.Request{BatchNo: task.Batch.BatchNo, BatchType: task.Batch.BatchType, TenantID: task.Batch.TenantID, Rows: rows} +} + +func (r Range) String() string { + return fmt.Sprintf("%s-%s", r.StartICCID, r.EndICCID) +} diff --git a/internal/service/inbound/executor_test.go b/internal/service/inbound/executor_test.go new file mode 100644 index 0000000..7ce7ff0 --- /dev/null +++ b/internal/service/inbound/executor_test.go @@ -0,0 +1,682 @@ +package inbound + +import ( + "context" + "errors" + "strings" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +func TestAdminSourceTypeContract(t *testing.T) { + if consts.BatchSourceTypeRange != 1 || consts.BatchSourceTypeBatchInput != 2 || consts.BatchSourceTypeFile != 3 { + t.Fatalf("source_type contract mismatch: range=%d input=%d file=%d", + consts.BatchSourceTypeRange, consts.BatchSourceTypeBatchInput, consts.BatchSourceTypeFile) + } +} + +func TestValidateRowRequiresAtLeastOneCardIdentifierAndChanKey(t *testing.T) { + row := &Row{} + validateRow(row, "") + + for _, want := range []string{"ICCID/MSISDN/IMSI至少填写一个", "chanKey不能为空"} { + if !containsReason(row.Reason, want) { + t.Fatalf("expected reason %q in %q", want, row.Reason) + } + } +} + +func TestParseInputTextAcceptsOneIdentifierPerLine(t *testing.T) { + rows := parseInputText(strings.Join([]string{ + "89860401780000567308", + "14100007308", + "460046527567308", + }, "\n")) + + if len(rows) != 3 { + t.Fatalf("row count = %d, want 3", len(rows)) + } + if rows[0].ICCID != "89860401780000567308" || rows[1].MSISDN != "14100007308" || rows[2].IMSI != "460046527567308" { + t.Fatalf("unexpected parsed rows: %+v", rows) + } + for _, row := range rows { + if row.InputCode == "" { + t.Fatalf("expected input code to be recorded: %+v", row) + } + } +} + +func TestValidateRowRejectsTooShortIdentifiers(t *testing.T) { + row := &Row{ICCID: "4", MSISDN: "4", IMSI: "4", ChanKey: "chan-a"} + validateRow(row, "") + + for _, want := range []string{"ICCID格式不正确", "MSISDN格式不正确", "IMSI格式不正确"} { + if !containsReason(row.Reason, want) { + t.Fatalf("expected reason %q in %q", want, row.Reason) + } + } +} + +func TestExpandRangeUsesSelectedMSISDN(t *testing.T) { + rows, err := expandRange(Range{RangeType: "msisdn", StartMSISDN: "13800008000", EndMSISDN: "13800008001"}) + if err != nil { + t.Fatal(err) + } + if len(rows) != 2 || rows[0].MSISDN != "13800008000" || rows[1].MSISDN != "13800008001" { + t.Fatalf("unexpected MSISDN rows: %+v", rows) + } + if rows[0].ICCID != "" || rows[0].IMSI != "" { + t.Fatalf("unselected identifiers must remain empty: %+v", rows[0]) + } +} + +func TestExpandRangeAllowsAlphaNumericICCIDRange(t *testing.T) { + rows, err := expandRange(Range{RangeType: "iccid", StartICCID: "898604F000000000001", EndICCID: "898604F000000000003"}) + if err != nil { + t.Fatal(err) + } + + want := []string{ + "898604F000000000001", + "898604F000000000002", + "898604F000000000003", + } + if len(rows) != len(want) { + t.Fatalf("row count = %d, want %d", len(rows), len(want)) + } + for i, value := range want { + if rows[i].ICCID != value { + t.Fatalf("row %d ICCID = %q, want %q", i, rows[i].ICCID, value) + } + } +} + +func TestExpandRangeRejectsAlphaNumericMSISDNRange(t *testing.T) { + _, err := expandRange(Range{RangeType: "msisdn", StartMSISDN: "1380013800A", EndMSISDN: "1380013800B"}) + if err == nil || !strings.Contains(err.Error(), "range values must be numeric") { + t.Fatalf("expected numeric range error, got %v", err) + } +} + +func TestValidateRowSelectedMSISDNDoesNotRequireOtherIdentifiers(t *testing.T) { + row := &Row{MSISDN: "13800008000", ChanKey: "chan-a"} + validateRow(row, "msisdn") + if row.Reason != "" { + t.Fatalf("unexpected validation reason: %q", row.Reason) + } +} + +func TestValidateRowAllowsAlphaNumericICCID(t *testing.T) { + row := &Row{ICCID: "898604F4152000015473", MSISDN: "13800008000", IMSI: "460001234567890", ChanKey: "chan-a"} + validateRow(row, "") + if row.Reason != "" { + t.Fatalf("unexpected validation reason: %q", row.Reason) + } +} + +func TestMarkSameBatchDuplicatesChecksICCIDAndMSISDN(t *testing.T) { + rows := []*Row{ + {RowNo: 1, ICCID: "8986", MSISDN: "13800008000"}, + {RowNo: 2, ICCID: "8986", MSISDN: "13800008001"}, + {RowNo: 3, ICCID: "8987", MSISDN: "13800008000"}, + } + + markSameBatchDuplicates(rows) + + if !containsReason(rows[1].Reason, "同批重复ICCID") { + t.Fatalf("expected row 2 duplicate ICCID, got %q", rows[1].Reason) + } + if !containsReason(rows[2].Reason, "同批重复MSISDN") { + t.Fatalf("expected row 3 duplicate MSISDN, got %q", rows[2].Reason) + } +} + +func TestMarkSameBatchDuplicatesChecksIMSI(t *testing.T) { + rows := []*Row{ + {RowNo: 1, IMSI: "460001234567890"}, + {RowNo: 2, IMSI: "460001234567890"}, + } + + markSameBatchDuplicates(rows) + + if !containsReason(rows[1].Reason, "同批重复IMSI") { + t.Fatalf("expected duplicate IMSI, got %q", rows[1].Reason) + } +} + +func TestValidateUpstreamBasicInfoAcceptsMatchingIdentifiers(t *testing.T) { + rows := []*Row{{RowNo: 1, ICCID: "898604F4152000015473", MSISDN: "1440000615473", IMSI: "460087436015473", ChanKey: "chan-a"}} + upstream := &basicInfoUpstreamStub{response: []model.SimBaseRes[model.SimBasicInfoItem]{ + { + Code: 0, + SimKey: &model.SimKey{ChanKey: "chan-a", ICCID: "898604F4152000015473"}, + ResData: model.SimBasicInfoItem{ + ICCID: "898604F4152000015473", + MSISDN: "1440000615473", + IMSI: "460087436015473", + }, + }, + }} + + err := (&executor{upstream: upstream}).validateUpstreamBasicInfo(context.Background(), &Task{Rows: rows}) + if err != nil { + t.Fatal(err) + } + if rows[0].Reason != "" { + t.Fatalf("unexpected reason: %q", rows[0].Reason) + } + if upstream.calls != 1 { + t.Fatalf("upstream calls = %d, want 1", upstream.calls) + } + if len(upstream.inputs) != 1 || len(upstream.inputs[0].SimKeys) != 1 { + t.Fatalf("unexpected upstream inputs: %+v", upstream.inputs) + } + requestKey := upstream.inputs[0].SimKeys[0] + if requestKey.ChanKey != "chan-a" || requestKey.ICCID != "898604F4152000015473" { + t.Fatalf("unexpected upstream request key: %+v", requestKey) + } + if requestKey.MSISDN != "" || requestKey.IMSI != "" { + t.Fatalf("upstream basic-info request must only carry chanKey and ICCID: %+v", requestKey) + } +} + +func TestValidateUpstreamBasicInfoRequiresCompleteResData(t *testing.T) { + rows := []*Row{{RowNo: 1, ICCID: "898604F4152000015473", MSISDN: "1440000615473", IMSI: "460087436015473", ChanKey: "chan-a"}} + upstream := &basicInfoUpstreamStub{response: []model.SimBaseRes[model.SimBasicInfoItem]{ + { + Code: 0, + SimKey: &model.SimKey{ChanKey: "chan-a", ICCID: "898604f4152000015473", MSISDN: "1440000615473", IMSI: "460087436015473"}, + ResData: model.SimBasicInfoItem{ + ActiveDate: "2026-08-12 00:00:00", + OpenDate: "2026-08-12 00:00:00", + }, + }, + }} + + err := (&executor{upstream: upstream}).validateUpstreamBasicInfo(context.Background(), &Task{Rows: rows}) + if err != nil { + t.Fatal(err) + } + if !containsReason(rows[0].Reason, "上游未返回三码信息") { + t.Fatalf("expected missing upstream data failure, got %q", rows[0].Reason) + } +} + +func TestValidateUpstreamBasicInfoPreservesNotFoundMessage(t *testing.T) { + rows := []*Row{{RowNo: 1, ICCID: "8986112223600594371", ChanKey: "chan-a"}} + upstream := &basicInfoUpstreamStub{response: []model.SimBaseRes[model.SimBasicInfoItem]{ + { + Code: 0, + Message: "未查询到对应SIM卡信息", + SimKey: &model.SimKey{ChanKey: "chan-a", ICCID: "8986112223600594371"}, + }, + }} + + err := (&executor{upstream: upstream}).validateUpstreamBasicInfo(context.Background(), &Task{Rows: rows}) + if err != nil { + t.Fatal(err) + } + if rows[0].Reason != "上游三码校验失败: 未查询到对应SIM卡信息" { + t.Fatalf("unexpected reason: %q", rows[0].Reason) + } +} + +func TestValidateUpstreamBasicInfoMatchesByMSISDNAndBackfillsCodes(t *testing.T) { + rows := []*Row{{RowNo: 1, MSISDN: "14100007308", ChanKey: "chan-a"}} + upstream := &basicInfoUpstreamStub{response: []model.SimBaseRes[model.SimBasicInfoItem]{ + { + Code: 0, + SimKey: &model.SimKey{ChanKey: "chan-a", MSISDN: "14100007308"}, + ResData: model.SimBasicInfoItem{ + ICCID: "89860401780000567308", + MSISDN: "14100007308", + IMSI: "460046527567308", + }, + }, + }} + + err := (&executor{upstream: upstream}).validateUpstreamBasicInfo(context.Background(), &Task{Rows: rows}) + if err != nil { + t.Fatal(err) + } + if rows[0].Reason != "" { + t.Fatalf("unexpected reason: %q", rows[0].Reason) + } + if rows[0].ICCID != "89860401780000567308" || rows[0].IMSI != "460046527567308" { + t.Fatalf("expected upstream codes to be backfilled, got %+v", rows[0]) + } + requestKey := upstream.inputs[0].SimKeys[0] + if requestKey.ICCID != "" || requestKey.MSISDN != "14100007308" || requestKey.IMSI != "" { + t.Fatalf("unexpected upstream request key: %+v", requestKey) + } +} + +func TestValidateBatchInputSingleMSISDNBackfillsBeforeCompletenessCheck(t *testing.T) { + rows := parseInputText("14100007308") + repo := &inboundRepositoryStub{} + upstream := &basicInfoUpstreamStub{response: []model.SimBaseRes[model.SimBasicInfoItem]{ + { + Code: 0, + SimKey: &model.SimKey{ChanKey: "chan-a", MSISDN: "14100007308"}, + ResData: model.SimBasicInfoItem{ + ICCID: "89860401780000567308", + MSISDN: "14100007308", + IMSI: "460046527567308", + }, + }, + }} + task := &Task{ + Batch: Batch{SourceType: consts.BatchSourceTypeBatchInput}, + Options: Options{ChanKey: "chan-a", ValidateUpstreamBasicInfo: "1"}, + Rows: rows, + } + + err := (&executor{repository: repo, upstream: upstream}).validate(context.Background(), task) + if err != nil { + t.Fatal(err) + } + if task.ValidCount != 1 { + t.Fatalf("valid count = %d, want 1, reason=%q", task.ValidCount, rows[0].Reason) + } + if rows[0].ICCID != "89860401780000567308" || rows[0].IMSI != "460046527567308" { + t.Fatalf("expected upstream codes to be backfilled, got %+v", rows[0]) + } + if upstream.calls != 1 { + t.Fatalf("upstream calls = %d, want 1", upstream.calls) + } +} + +func TestValidateSkipsUpstreamBasicInfoWhenDisabled(t *testing.T) { + rows := []*Row{{ + RowNo: 1, + ICCID: "89860401780000567308", + MSISDN: "14100007308", + IMSI: "460046527567308", + ChanKey: "chan-a", + }} + upstream := &basicInfoUpstreamStub{err: errors.New("gateway unavailable")} + task := &Task{ + Batch: Batch{SourceType: consts.BatchSourceTypeFile}, + Options: Options{ValidateUpstreamBasicInfo: "0"}, + Rows: rows, + } + + err := (&executor{repository: &inboundRepositoryStub{}, upstream: upstream}).validate(context.Background(), task) + if err != nil { + t.Fatal(err) + } + if upstream.calls != 0 { + t.Fatalf("upstream calls = %d, want 0", upstream.calls) + } + if task.ValidCount != 1 { + t.Fatalf("valid count = %d, want 1, reason=%q", task.ValidCount, rows[0].Reason) + } + if task.Options.ValidateUpstreamBasicInfo != "0" { + t.Fatalf("validate option = %q, want 0", task.Options.ValidateUpstreamBasicInfo) + } +} + +func TestValidateRunsUpstreamBasicInfoWhenEnabled(t *testing.T) { + rows := []*Row{{ + RowNo: 1, + ICCID: "89860401780000567308", + MSISDN: "14100007308", + IMSI: "460046527567308", + ChanKey: "chan-a", + }} + upstream := &basicInfoUpstreamStub{err: errors.New("gateway unavailable")} + task := &Task{ + Batch: Batch{SourceType: consts.BatchSourceTypeFile}, + Options: Options{ValidateUpstreamBasicInfo: "1"}, + Rows: rows, + } + + err := (&executor{repository: &inboundRepositoryStub{}, upstream: upstream}).validate(context.Background(), task) + if err != nil { + t.Fatal(err) + } + if upstream.calls != 1 { + t.Fatalf("upstream calls = %d, want 1", upstream.calls) + } + if task.ValidCount != 0 { + t.Fatalf("valid count = %d, want 0, reason=%q", task.ValidCount, rows[0].Reason) + } + if task.Options.ValidateUpstreamBasicInfo != "1" { + t.Fatalf("validate option = %q, want 1", task.Options.ValidateUpstreamBasicInfo) + } +} + +func TestPopulateExcelRowsReturnsRetryableErrorWhenSourceIsUnreadable(t *testing.T) { + sourceErr := errors.New("OSS local file does not exist") + task := &Task{Source: Source{ImportFileOssID: 1270}} + repository := &inboundRepositoryStub{readOssErr: sourceErr} + + err := (&executor{repository: repository}).populateExcelRows(context.Background(), task) + if !errors.Is(err, sourceErr) || !batch.IsRetryable(err) { + t.Fatalf("populateExcelRows() error = %v, want retryable wrapper of %v", err, sourceErr) + } +} + +func TestValidateUpstreamBasicInfoWaitsForRateLimitBeforeQuery(t *testing.T) { + var order []string + rows := []*Row{{RowNo: 1, ICCID: "89860401780000567308", MSISDN: "14100007308", IMSI: "460046527567308", ChanKey: "chan-a"}} + limiter := &recordingRateLimiter{ + onWait: func(context.Context, string, int) error { + order = append(order, "limit") + return nil + }, + } + upstream := &basicInfoUpstreamStub{ + onQuery: func() { + order = append(order, "query") + }, + response: []model.SimBaseRes[model.SimBasicInfoItem]{ + { + Code: 0, + SimKey: &model.SimKey{ChanKey: "chan-a", ICCID: "89860401780000567308"}, + ResData: model.SimBasicInfoItem{ + ICCID: "89860401780000567308", + MSISDN: "14100007308", + IMSI: "460046527567308", + }, + }, + }, + } + + err := (&executor{upstream: upstream, rateLimiter: limiter}).validateUpstreamBasicInfo(context.Background(), &Task{ + Batch: Batch{BatchNo: "B-1"}, + Rows: rows, + }) + if err != nil { + t.Fatal(err) + } + if limiter.calls != 1 || limiter.chanKey != "chan-a" || limiter.itemCount != 1 { + t.Fatalf("unexpected limiter call: %+v", limiter) + } + if upstream.calls != 1 { + t.Fatalf("upstream calls = %d, want 1", upstream.calls) + } + if len(order) != 2 || order[0] != "limit" || order[1] != "query" { + t.Fatalf("unexpected call order: %+v", order) + } +} + +func TestValidateUpstreamBasicInfoStopsWhenRateLimitFails(t *testing.T) { + rows := []*Row{{RowNo: 1, ICCID: "89860401780000567308", ChanKey: "chan-a"}} + upstream := &basicInfoUpstreamStub{} + limiter := &recordingRateLimiter{err: errors.New("redis down")} + + err := (&executor{upstream: upstream, rateLimiter: limiter}).validateUpstreamBasicInfo(context.Background(), &Task{ + Batch: Batch{BatchNo: "B-1"}, + Rows: rows, + }) + if err != nil { + t.Fatal(err) + } + if upstream.calls != 0 { + t.Fatalf("upstream calls = %d, want 0", upstream.calls) + } + if !strings.Contains(rows[0].Reason, "redis down") { + t.Fatalf("expected rate limit failure reason, got %q", rows[0].Reason) + } +} + +func TestValidateUpstreamBasicInfoRejectsMismatch(t *testing.T) { + rows := []*Row{{RowNo: 1, ICCID: "89860001", MSISDN: "13800008000", IMSI: "460001", ChanKey: "chan-a"}} + upstream := &basicInfoUpstreamStub{response: []model.SimBaseRes[model.SimBasicInfoItem]{ + { + Code: 0, + SimKey: &model.SimKey{ChanKey: "chan-a", ICCID: "89860001"}, + ResData: model.SimBasicInfoItem{ICCID: "89860001", MSISDN: "13800008099", IMSI: "460001"}, + }, + }} + + err := (&executor{upstream: upstream}).validateUpstreamBasicInfo(context.Background(), &Task{Rows: rows}) + if err != nil { + t.Fatal(err) + } + if !strings.Contains(rows[0].Reason, "MSISDN") { + t.Fatalf("expected MSISDN mismatch reason, got %q", rows[0].Reason) + } +} + +func TestValidateUpstreamBasicInfoRejectsUpstreamError(t *testing.T) { + rows := []*Row{{RowNo: 1, ICCID: "89860001", ChanKey: "chan-a"}} + upstream := &basicInfoUpstreamStub{err: errors.New("gateway unavailable")} + + err := (&executor{upstream: upstream}).validateUpstreamBasicInfo(context.Background(), &Task{Rows: rows}) + if err != nil { + t.Fatal(err) + } + if !strings.Contains(rows[0].Reason, "gateway unavailable") { + t.Fatalf("expected upstream error reason, got %q", rows[0].Reason) + } +} + +func TestParseTableRowsAcceptsChineseInboundHeaders(t *testing.T) { + rows := parseTableRows([][]string{ + {"ICCID号", "手机号", "国际移动用户识别码", "国际移动设备识别码", "通道编码", "卡类型", "上游套餐编码", "供应商编码"}, + {"898604F4152000015473", "1440000615473", "460087436015473", "865111111111111", "channel_003", "通用插拔 SIM 卡", "UP001", "SUP001"}, + }) + + if len(rows) != 1 { + t.Fatalf("row count = %d, want 1", len(rows)) + } + row := rows[0] + if row.ICCID != "898604F4152000015473" || row.MSISDN != "1440000615473" || + row.IMSI != "460087436015473" || row.IMEI != "865111111111111" || row.ChanKey != "channel_003" { + t.Fatalf("unexpected parsed row: %+v", row) + } + if row.CardType != "通用插拔 SIM 卡" || row.UpPkgCode != "UP001" || row.SuppliersCode != "SUP001" { + t.Fatalf("unexpected inbound options: %+v", row) + } +} + +func TestParseTableRowsAcceptsRequiredMarkerHeaders(t *testing.T) { + rows := parseTableRows([][]string{ + {"ICCID*", "MSISDN*", "IMSI*", "通道编号*"}, + {"89860401780000567308", "14100007308", "460046527567308", "channel_003"}, + }) + + if len(rows) != 1 { + t.Fatalf("row count = %d, want 1", len(rows)) + } + row := rows[0] + if row.ICCID != "89860401780000567308" || row.MSISDN != "14100007308" || + row.IMSI != "460046527567308" || row.ChanKey != "channel_003" { + t.Fatalf("unexpected parsed row: %+v", row) + } +} + +func TestValidateSkipsUpstreamWhenLocalIdentifierFormatFails(t *testing.T) { + rows := []*Row{{RowNo: 1, ICCID: "4", MSISDN: "4", IMSI: "4", ChanKey: "chan-a"}} + upstream := &basicInfoUpstreamStub{} + repo := &inboundRepositoryStub{} + task := &Task{ + Rows: rows, + Options: Options{ChanKey: "chan-a"}, + } + + err := (&executor{repository: repo, upstream: upstream}).validate(context.Background(), task) + if err != nil { + t.Fatal(err) + } + if upstream.calls != 0 { + t.Fatalf("upstream calls = %d, want 0", upstream.calls) + } + if !containsReason(rows[0].Reason, "ICCID格式不正确") { + t.Fatalf("expected local validation failure, got %q", rows[0].Reason) + } + if task.ValidCount != 0 { + t.Fatalf("valid count = %d, want 0", task.ValidCount) + } +} + +func TestValidateRequiresCompleteIdentifiersAfterUpstreamStep(t *testing.T) { + rows := []*Row{{RowNo: 1, MSISDN: "14100007308", ChanKey: "chan-a"}} + repo := &inboundRepositoryStub{} + task := &Task{ + Rows: rows, + Source: Source{Range: Range{RangeType: "msisdn"}}, + } + + err := (&executor{repository: repo}).validate(context.Background(), task) + if err != nil { + t.Fatal(err) + } + if !containsReason(rows[0].Reason, "三码信息不完整") { + t.Fatalf("expected incomplete identifiers failure, got %q", rows[0].Reason) + } + if task.ValidCount != 0 { + t.Fatalf("valid count = %d, want 0", task.ValidCount) + } +} + +func TestValidateNormalizesExcelInboundOptionsAndBackfillsBatchInfo(t *testing.T) { + rows := parseTableRows([][]string{ + {"ICCID*", "MSISDN*", "IMSI*", "通道编号*", "卡类型", "上游套餐编码", "供应商编码", "是否贴片转插拔", "是否运营商达量断网"}, + {"89860401780000567308", "14100007308", "460046527567308", "channel_003", "MP1-消费级插拔物联卡", "UP001", "SUP001", "是", "否"}, + {"89860401780000567309", "14100007309", "460046527567309", "channel_003", "", "UP001", "SUP001", "", ""}, + }) + task := &Task{Rows: rows, Batch: Batch{SourceType: consts.BatchSourceTypeFile}} + + err := (&executor{repository: &inboundRepositoryStub{}}).validate(context.Background(), task) + if err != nil { + t.Fatal(err) + } + if task.ValidCount != 2 { + t.Fatalf("valid count = %d, want 2", task.ValidCount) + } + if task.Options.ChanKey != "channel_003" || task.Options.CardType != "4" || + task.Options.UpPkgCode != "UP001" || task.Options.SuppliersCode != "SUP001" { + t.Fatalf("unexpected task options: %+v", task.Options) + } + if rows[1].CardType != "4" || rows[1].InsertToPlugEnabled != "0" || rows[1].CarrierTrafficCutoffEnabled != "1" { + t.Fatalf("expected blank row options to be backfilled from batch, got %+v", rows[1]) + } +} + +func TestValidateRejectsMixedInboundOptionsInOneExcelBatch(t *testing.T) { + rows := parseTableRows([][]string{ + {"ICCID*", "MSISDN*", "IMSI*", "通道编号*", "卡类型"}, + {"89860401780000567308", "14100007308", "460046527567308", "channel_003", "1"}, + {"89860401780000567309", "14100007309", "460046527567309", "channel_003", "2"}, + }) + task := &Task{Rows: rows, Batch: Batch{SourceType: consts.BatchSourceTypeFile}} + + err := (&executor{repository: &inboundRepositoryStub{}}).validate(context.Background(), task) + if err != nil { + t.Fatal(err) + } + if task.ValidCount != 0 { + t.Fatalf("valid count = %d, want 0", task.ValidCount) + } + if task.FailureReason != "同批入库配置必须一致" { + t.Fatalf("unexpected task failure reason: %q", task.FailureReason) + } + if !containsReason(rows[0].Reason, "卡类型必须同批一致") || !containsReason(rows[1].Reason, "卡类型必须同批一致") { + t.Fatalf("expected consistency errors, got row1=%q row2=%q", rows[0].Reason, rows[1].Reason) + } +} + +func containsReason(actual, want string) bool { + return strings.Contains(actual, want) +} + +type basicInfoUpstreamStub struct { + calls int + inputs []*model.SimBasicInfoInput + response []model.SimBaseRes[model.SimBasicInfoItem] + err error + onQuery func() +} + +func (s *basicInfoUpstreamStub) QuerySimBasicInfo(_ context.Context, input *model.SimBasicInfoInput) ([]model.SimBaseRes[model.SimBasicInfoItem], error) { + if s.onQuery != nil { + s.onQuery() + } + s.calls++ + if input == nil || len(input.SimKeys) == 0 { + return nil, errors.New("missing sim keys") + } + s.inputs = append(s.inputs, cloneBasicInfoInput(input)) + if s.err != nil { + return nil, s.err + } + return s.response, nil +} + +func cloneBasicInfoInput(input *model.SimBasicInfoInput) *model.SimBasicInfoInput { + clone := &model.SimBasicInfoInput{ + ReqData: input.ReqData, + SimKeys: make([]*model.SimKey, 0, len(input.SimKeys)), + } + for _, item := range input.SimKeys { + if item == nil { + clone.SimKeys = append(clone.SimKeys, nil) + continue + } + copied := *item + clone.SimKeys = append(clone.SimKeys, &copied) + } + return clone +} + +type recordingRateLimiter struct { + calls int + chanKey string + itemCount int + err error + onWait func(context.Context, string, int) error +} + +func (l *recordingRateLimiter) Wait(ctx context.Context, chanKey string, itemCount int) error { + l.calls++ + l.chanKey = chanKey + l.itemCount = itemCount + if l.onWait != nil { + return l.onWait(ctx, chanKey, itemCount) + } + return l.err +} + +type inboundRepositoryStub struct { + readOssContent []byte + readOssName string + readOssErr error +} + +func (s *inboundRepositoryStub) Load(context.Context, string) (*Task, error) { + return nil, errors.New("not implemented") +} + +func (s *inboundRepositoryStub) ReadOssFile(context.Context, int64) ([]byte, string, error) { + if s.readOssErr != nil { + return nil, "", s.readOssErr + } + return s.readOssContent, s.readOssName, nil +} + +func (s *inboundRepositoryStub) FindExistingStocks(context.Context, string, []string, []string, []string) ([]Stock, error) { + return nil, nil +} + +func (s *inboundRepositoryStub) SaveValidationFailure(context.Context, *Task) error { + return errors.New("not implemented") +} + +func (s *inboundRepositoryStub) SaveSuccess(context.Context, *Task) error { + return errors.New("not implemented") +} + +func (s *inboundRepositoryStub) SaveBusinessFailure(context.Context, *Task, string) error { + return errors.New("not implemented") +} + +func (s *inboundRepositoryStub) SaveResultFile(context.Context, *Task, batchresult.Request) error { + return errors.New("not implemented") +} diff --git a/internal/service/inbound/rate_limiter.go b/internal/service/inbound/rate_limiter.go new file mode 100644 index 0000000..f277f37 --- /dev/null +++ b/internal/service/inbound/rate_limiter.go @@ -0,0 +1,738 @@ +package inbound + +import ( + "context" + "fmt" + "math" + "strconv" + "strings" + "sync" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "github.com/gogf/gf/v2/container/gvar" + "github.com/gogf/gf/v2/frame/g" +) + +const ( + inboundUpstreamRateLimitConfigKey = "batch.inbound.upstreamRateLimit" + + defaultInboundUpstreamRateLimitMode = "redis" + defaultInboundUpstreamRateLimitRedisGroup = "default" + defaultInboundUpstreamRateLimitKeyPrefix = "iotlcm:worker:inbound:upstream_basic_info" + defaultInboundUpstreamQPS = 20 + defaultInboundUpstreamBurst = 1 + defaultInboundUpstreamRequestCost = 1 + defaultInboundChannelTempKeyCacheTTL = time.Minute + + inboundRateLimitFallbackLocal = "local" + inboundRateLimitFallbackError = "error" + + inboundCarrierTelecom = "telecom" + inboundCarrierMobile = "mobile" +) + +type upstreamBasicInfoRateLimiter interface { + Wait(ctx context.Context, chanKey string, requestCount int) error +} + +type redisEvaler interface { + Eval(ctx context.Context, script string, numKeys int64, keys []string, args []any) (*gvar.Var, error) +} + +type inboundUpstreamRateLimitConfig struct { + Enabled *bool `json:"enabled"` + Mode string `json:"mode"` + RedisGroup string `json:"redisGroup"` + KeyPrefix string `json:"keyPrefix"` + RedisFailureFallback string `json:"redisFailureFallback"` + QPS float64 `json:"qps"` + Burst int `json:"burst"` + RequestCost int `json:"requestCost"` + Default inboundUpstreamRateLimitRule `json:"default"` + Carriers map[string]inboundUpstreamRateLimitRule `json:"carriers"` + Channels map[string]inboundUpstreamRateLimitRule `json:"channels"` +} + +type inboundUpstreamRateLimitRule struct { + Enabled *bool `json:"enabled"` + QPS float64 `json:"qps"` + Burst int `json:"burst"` + RequestCost int `json:"requestCost"` +} + +type inboundUpstreamRateLimitSetting struct { + enabled bool + mode string + redisGroup string + keyPrefix string + redisFailureFallback string + qps float64 + burst int + requestCost int + carrier string + tempKey string + skipReason string +} + +type inboundChannelMetadata struct { + TempKey string `orm:"temp_key"` + Operator int `orm:"operator"` +} + +type inboundChannelTempKeyCacheEntry struct { + metadata inboundChannelMetadata + expiresAt time.Time +} + +type configUpstreamBasicInfoRateLimiter struct { + redis func(string) redisEvaler + channelMetadata func(context.Context, string) (inboundChannelMetadata, error) + channelTempKey func(context.Context, string) (string, error) + local *localTokenBucketLimiter + channelCache sync.Map + redisLogTimes sync.Map + skipLogTimes sync.Map +} + +func newUpstreamBasicInfoRateLimiter() *configUpstreamBasicInfoRateLimiter { + return &configUpstreamBasicInfoRateLimiter{ + redis: func(name string) redisEvaler { + return g.Redis(name) + }, + channelMetadata: queryInboundChannelMetadata, + local: newLocalTokenBucketLimiter(), + } +} + +func (l *configUpstreamBasicInfoRateLimiter) Wait(ctx context.Context, chanKey string, requestCount int) error { + if l == nil { + return nil + } + cfg := loadInboundUpstreamRateLimitConfig(ctx) + if !inboundUpstreamRateLimitGloballyEnabled(cfg) { + return nil + } + metadata, err := l.lookupChannelMetadata(ctx, chanKey) + if err != nil { + return err + } + setting := resolveInboundUpstreamRateLimitFromChannelMetadata(cfg, chanKey, metadata.Operator, metadata.TempKey) + if !setting.enabled || setting.qps <= 0 || setting.mode == "off" { + if setting.skipReason != "" { + l.logSkipped(ctx, chanKey, setting) + } + return nil + } + if requestCount <= 0 { + requestCount = 1 + } + permits := requestCount * setting.requestCost + key := inboundUpstreamRateLimitKey(setting.keyPrefix, chanKey, setting.carrier) + for i := 0; i < permits; i++ { + if err := l.waitOne(ctx, setting, key); err != nil { + return err + } + } + return nil +} + +func (l *configUpstreamBasicInfoRateLimiter) waitOne(ctx context.Context, setting inboundUpstreamRateLimitSetting, key string) error { + if setting.mode == "local" { + if l.local == nil { + return fmt.Errorf("local rate limiter is nil") + } + return l.local.Wait(ctx, key, setting.qps, setting.burst) + } + if l.redis == nil { + return l.handleRedisError(ctx, setting, key, fmt.Errorf("redis client resolver is nil")) + } + redis := l.redis(setting.redisGroup) + if redis == nil { + return l.handleRedisError(ctx, setting, key, fmt.Errorf("redis client is nil")) + } + if err := (&redisTokenBucketLimiter{redis: redis}).Wait(ctx, key, setting.qps, setting.burst); err != nil { + return l.handleRedisError(ctx, setting, key, err) + } + return nil +} + +func (l *configUpstreamBasicInfoRateLimiter) handleRedisError(ctx context.Context, setting inboundUpstreamRateLimitSetting, key string, err error) error { + if setting.redisFailureFallback == inboundRateLimitFallbackError { + return err + } + if l.local == nil { + return err + } + l.logRedisFallback(ctx, key, err) + return l.local.Wait(ctx, key, setting.qps, setting.burst) +} + +func (l *configUpstreamBasicInfoRateLimiter) logRedisFallback(ctx context.Context, key string, err error) { + now := time.Now() + if value, ok := l.redisLogTimes.Load(key); ok { + if last, ok := value.(time.Time); ok && now.Sub(last) < 10*time.Second { + return + } + } + l.redisLogTimes.Store(key, now) + g.Log().Errorf(ctx, "inbound upstream basic-info redis rate limiter failed, fallback local, key:%s, err:%+v", key, err) +} + +func (l *configUpstreamBasicInfoRateLimiter) lookupChannelMetadata(ctx context.Context, chanKey string) (inboundChannelMetadata, error) { + chanKey = strings.TrimSpace(chanKey) + if chanKey == "" { + return inboundChannelMetadata{}, nil + } + now := time.Now() + if value, ok := l.channelCache.Load(chanKey); ok { + if entry, ok := value.(inboundChannelTempKeyCacheEntry); ok && now.Before(entry.expiresAt) { + return entry.metadata, nil + } + } + if l.channelMetadata != nil { + metadata, err := l.channelMetadata(ctx, chanKey) + if err != nil { + return inboundChannelMetadata{}, fmt.Errorf("query channel metadata failed, chanKey:%s: %w", chanKey, err) + } + metadata.TempKey = strings.TrimSpace(metadata.TempKey) + l.channelCache.Store(chanKey, inboundChannelTempKeyCacheEntry{ + metadata: metadata, + expiresAt: now.Add(defaultInboundChannelTempKeyCacheTTL), + }) + return metadata, nil + } + if l.channelTempKey == nil { + return inboundChannelMetadata{}, fmt.Errorf("channel metadata resolver is nil") + } + tempKey, err := l.channelTempKey(ctx, chanKey) + if err != nil { + return inboundChannelMetadata{}, fmt.Errorf("query channel template failed, chanKey:%s: %w", chanKey, err) + } + metadata := inboundChannelMetadata{TempKey: strings.TrimSpace(tempKey)} + l.channelCache.Store(chanKey, inboundChannelTempKeyCacheEntry{ + metadata: metadata, + expiresAt: now.Add(defaultInboundChannelTempKeyCacheTTL), + }) + return metadata, nil +} + +func (l *configUpstreamBasicInfoRateLimiter) logSkipped(ctx context.Context, chanKey string, setting inboundUpstreamRateLimitSetting) { + key := strings.TrimSpace(chanKey) + ":" + setting.skipReason + now := time.Now() + if value, ok := l.skipLogTimes.Load(key); ok { + if last, ok := value.(time.Time); ok && now.Sub(last) < 10*time.Second { + return + } + } + l.skipLogTimes.Store(key, now) + g.Log().Warningf(ctx, "inbound upstream basic-info rate limit skipped, chanKey:%s, tempKey:%s, reason:%s", + chanKey, setting.tempKey, setting.skipReason) +} + +func queryInboundChannelMetadata(ctx context.Context, chanKey string) (inboundChannelMetadata, error) { + var row inboundChannelMetadata + if err := g.DB().Model("t_sim_channels").Ctx(ctx). + Fields("temp_key,operator"). + Where("chan_key", strings.TrimSpace(chanKey)). + Limit(1). + Scan(&row); err != nil { + return inboundChannelMetadata{}, err + } + row.TempKey = strings.TrimSpace(row.TempKey) + return row, nil +} + +func queryInboundChannelTempKey(ctx context.Context, chanKey string) (string, error) { + metadata, err := queryInboundChannelMetadata(ctx, chanKey) + if err != nil { + return "", err + } + return metadata.TempKey, nil +} + +func resolveInboundUpstreamRateLimit(ctx context.Context, chanKey string) inboundUpstreamRateLimitSetting { + return resolveInboundUpstreamRateLimitWithTempKey(ctx, chanKey, "") +} + +func resolveInboundUpstreamRateLimitWithTempKey(ctx context.Context, chanKey, tempKey string) inboundUpstreamRateLimitSetting { + cfg := loadInboundUpstreamRateLimitConfig(ctx) + return resolveInboundUpstreamRateLimitFromConfig(cfg, chanKey, tempKey) +} + +func resolveInboundUpstreamRateLimitWithChannelMetadata(ctx context.Context, chanKey string, operator int, tempKey string) inboundUpstreamRateLimitSetting { + cfg := loadInboundUpstreamRateLimitConfig(ctx) + return resolveInboundUpstreamRateLimitFromChannelMetadata(cfg, chanKey, operator, tempKey) +} + +func loadInboundUpstreamRateLimitConfig(ctx context.Context) inboundUpstreamRateLimitConfig { + cfg := defaultInboundUpstreamRateLimitConfig() + value, err := g.Cfg().Get(ctx, inboundUpstreamRateLimitConfigKey) + if err != nil { + g.Log().Warningf(ctx, "load inbound upstream rate limit config failed: %+v", err) + return normalizeInboundUpstreamRateLimitConfig(cfg) + } + if value != nil && !value.IsNil() { + if err = value.Scan(&cfg); err != nil { + g.Log().Warningf(ctx, "parse inbound upstream rate limit config failed: %+v", err) + cfg = defaultInboundUpstreamRateLimitConfig() + } + } + return normalizeInboundUpstreamRateLimitConfig(cfg) +} + +func defaultInboundUpstreamRateLimitConfig() inboundUpstreamRateLimitConfig { + return inboundUpstreamRateLimitConfig{ + Mode: defaultInboundUpstreamRateLimitMode, + RedisGroup: defaultInboundUpstreamRateLimitRedisGroup, + KeyPrefix: defaultInboundUpstreamRateLimitKeyPrefix, + RedisFailureFallback: inboundRateLimitFallbackError, + Default: inboundUpstreamRateLimitRule{ + Enabled: inboundBool(false), + }, + Carriers: map[string]inboundUpstreamRateLimitRule{ + inboundCarrierTelecom: { + Enabled: inboundBool(true), + QPS: defaultInboundUpstreamQPS, + Burst: defaultInboundUpstreamBurst, + RequestCost: defaultInboundUpstreamRequestCost, + }, + inboundCarrierMobile: { + Enabled: inboundBool(false), + }, + }, + } +} + +func normalizeInboundUpstreamRateLimitConfig(cfg inboundUpstreamRateLimitConfig) inboundUpstreamRateLimitConfig { + cfg.Mode = normalizeInboundRateLimitMode(cfg.Mode) + cfg.RedisGroup = firstNonEmpty(strings.TrimSpace(cfg.RedisGroup), defaultInboundUpstreamRateLimitRedisGroup) + cfg.KeyPrefix = firstNonEmpty(strings.TrimSpace(cfg.KeyPrefix), defaultInboundUpstreamRateLimitKeyPrefix) + cfg.RedisFailureFallback = normalizeInboundRateLimitFallback(cfg.RedisFailureFallback) + if cfg.Default.Enabled == nil { + cfg.Default.Enabled = inboundBool(false) + } + if len(cfg.Carriers) == 0 { + cfg.Carriers = make(map[string]inboundUpstreamRateLimitRule, 2) + } else { + normalized := make(map[string]inboundUpstreamRateLimitRule, len(cfg.Carriers)) + for key, rule := range cfg.Carriers { + normalized[normalizeInboundCarrier(key)] = rule + } + cfg.Carriers = normalized + } + telecomRule, ok := cfg.Carriers[inboundCarrierTelecom] + if !ok { + telecomRule = inboundUpstreamRateLimitRule{} + } + if telecomRule.Enabled == nil { + telecomRule.Enabled = inboundBool(true) + } + if telecomRule.QPS <= 0 { + telecomRule.QPS = defaultInboundUpstreamQPS + } + if telecomRule.Burst <= 0 { + telecomRule.Burst = defaultInboundUpstreamBurst + } + if telecomRule.RequestCost <= 0 { + telecomRule.RequestCost = defaultInboundUpstreamRequestCost + } + cfg.Carriers[inboundCarrierTelecom] = telecomRule + + mobileRule, ok := cfg.Carriers[inboundCarrierMobile] + if !ok { + mobileRule = inboundUpstreamRateLimitRule{} + } + if mobileRule.Enabled == nil { + mobileRule.Enabled = inboundBool(false) + } + cfg.Carriers[inboundCarrierMobile] = mobileRule + + return cfg +} + +func resolveInboundUpstreamRateLimitFromConfig(cfg inboundUpstreamRateLimitConfig, chanKey, tempKey string) inboundUpstreamRateLimitSetting { + return resolveInboundUpstreamRateLimitFromChannelMetadata(cfg, chanKey, 0, tempKey) +} + +func resolveInboundUpstreamRateLimitFromChannelMetadata(cfg inboundUpstreamRateLimitConfig, chanKey string, operator int, tempKey string) inboundUpstreamRateLimitSetting { + cfg = normalizeInboundUpstreamRateLimitConfig(cfg) + setting := inboundUpstreamRateLimitSetting{ + enabled: true, + mode: cfg.Mode, + redisGroup: cfg.RedisGroup, + keyPrefix: cfg.KeyPrefix, + redisFailureFallback: cfg.RedisFailureFallback, + qps: defaultInboundUpstreamQPS, + burst: defaultInboundUpstreamBurst, + requestCost: defaultInboundUpstreamRequestCost, + tempKey: strings.TrimSpace(tempKey), + } + if cfg.Enabled != nil { + setting.enabled = *cfg.Enabled + } + applyInboundRateLimitRule(&setting, inboundUpstreamRateLimitRule{ + QPS: cfg.QPS, + Burst: cfg.Burst, + RequestCost: cfg.RequestCost, + }) + applyInboundRateLimitRule(&setting, cfg.Default) + carrier, known := inboundCarrierForOperator(operator) + if model.IsTelecomTemplate(tempKey) { + carrier, known = inboundCarrierTelecom, true + } + if known { + setting.carrier = carrier + if rule, ok := inboundCarrierRateLimitRule(cfg.Carriers, carrier); ok { + applyInboundRateLimitRule(&setting, rule) + } + } else if operator == 0 { + setting.skipReason = "missing_channel_operator" + } else { + setting.skipReason = "unsupported_channel_operator" + } + if rule, ok := inboundChannelRateLimitRule(cfg.Channels, chanKey); ok { + applyInboundRateLimitRule(&setting, rule) + } + return normalizeInboundRateLimitSetting(setting) +} + +func inboundUpstreamRateLimitGloballyEnabled(cfg inboundUpstreamRateLimitConfig) bool { + cfg = normalizeInboundUpstreamRateLimitConfig(cfg) + if cfg.Mode == "off" { + return false + } + if cfg.Enabled == nil { + return true + } + return *cfg.Enabled +} + +func applyInboundRateLimitRule(setting *inboundUpstreamRateLimitSetting, rule inboundUpstreamRateLimitRule) { + if setting == nil { + return + } + if rule.Enabled != nil { + setting.enabled = *rule.Enabled + } + if rule.QPS > 0 { + setting.qps = rule.QPS + } + if rule.Burst > 0 { + setting.burst = rule.Burst + } + if rule.RequestCost > 0 { + setting.requestCost = rule.RequestCost + } +} + +func inboundChannelRateLimitRule(configs map[string]inboundUpstreamRateLimitRule, chanKey string) (inboundUpstreamRateLimitRule, bool) { + chanKey = strings.TrimSpace(chanKey) + if chanKey == "" || len(configs) == 0 { + return inboundUpstreamRateLimitRule{}, false + } + if cfg, ok := configs[chanKey]; ok { + return cfg, true + } + lower := strings.ToLower(chanKey) + for key, cfg := range configs { + if strings.ToLower(strings.TrimSpace(key)) == lower { + return cfg, true + } + } + return inboundUpstreamRateLimitRule{}, false +} + +func inboundCarrierRateLimitRule(configs map[string]inboundUpstreamRateLimitRule, carrier string) (inboundUpstreamRateLimitRule, bool) { + carrier = normalizeInboundCarrier(carrier) + if carrier == "" || len(configs) == 0 { + return inboundUpstreamRateLimitRule{}, false + } + if cfg, ok := configs[carrier]; ok { + return cfg, true + } + for key, cfg := range configs { + if normalizeInboundCarrier(key) == carrier { + return cfg, true + } + } + return inboundUpstreamRateLimitRule{}, false +} + +func inboundCarrierForOperator(operator int) (string, bool) { + switch operator { + case 1: + return inboundCarrierMobile, true + case 2: + return inboundCarrierTelecom, true + default: + return "", false + } +} + +func normalizeInboundCarrier(value string) string { + switch strings.ToLower(strings.TrimSpace(value)) { + case "", "unknown": + return "" + case inboundCarrierTelecom, "dx", "ctcc", "china_telecom", "chinatelecom": + return inboundCarrierTelecom + case inboundCarrierMobile, "yd", "cmcc", "china_mobile", "chinamobile": + return inboundCarrierMobile + default: + return strings.ToLower(strings.TrimSpace(value)) + } +} + +func normalizeInboundRateLimitSetting(setting inboundUpstreamRateLimitSetting) inboundUpstreamRateLimitSetting { + if setting.qps <= 0 || math.IsNaN(setting.qps) || math.IsInf(setting.qps, 0) { + setting.qps = defaultInboundUpstreamQPS + } + if setting.burst <= 0 { + setting.burst = defaultInboundUpstreamBurst + } + if setting.requestCost <= 0 { + setting.requestCost = defaultInboundUpstreamRequestCost + } + return setting +} + +func normalizeInboundRateLimitMode(mode string) string { + switch strings.ToLower(strings.TrimSpace(mode)) { + case "", "redis", "distributed": + return defaultInboundUpstreamRateLimitMode + case "local", "memory": + return "local" + case "off", "disable", "disabled", "none": + return "off" + default: + return defaultInboundUpstreamRateLimitMode + } +} + +func normalizeInboundRateLimitFallback(value string) string { + switch strings.ToLower(strings.TrimSpace(value)) { + case "", inboundRateLimitFallbackError: + return inboundRateLimitFallbackError + case inboundRateLimitFallbackLocal: + return inboundRateLimitFallbackLocal + default: + return inboundRateLimitFallbackError + } +} + +func inboundUpstreamRateLimitKey(prefix, chanKey, carrier string) string { + prefix = strings.TrimRight(strings.TrimSpace(prefix), ":") + if prefix == "" { + prefix = defaultInboundUpstreamRateLimitKeyPrefix + } + chanKey = strings.TrimLeft(strings.TrimSpace(chanKey), ":") + carrier = strings.TrimLeft(normalizeInboundCarrier(carrier), ":") + if carrier != "" { + return prefix + ":carrier:" + carrier + ":chan:" + chanKey + } + return prefix + ":chan:" + chanKey +} + +func inboundBool(value bool) *bool { + return &value +} + +type localTokenBucketLimiter struct { + mu sync.Mutex + buckets map[string]*tokenBucket +} + +func newLocalTokenBucketLimiter() *localTokenBucketLimiter { + return &localTokenBucketLimiter{buckets: make(map[string]*tokenBucket)} +} + +func (l *localTokenBucketLimiter) Wait(ctx context.Context, key string, qps float64, burst int) error { + if qps <= 0 { + return nil + } + bucket := l.bucket(key) + delay := bucket.reserve(time.Now(), qps, burst) + if delay <= 0 { + return nil + } + timer := time.NewTimer(delay) + defer timer.Stop() + select { + case <-ctx.Done(): + return ctx.Err() + case <-timer.C: + return nil + } +} + +func (l *localTokenBucketLimiter) bucket(key string) *tokenBucket { + l.mu.Lock() + defer l.mu.Unlock() + bucket := l.buckets[key] + if bucket == nil { + bucket = &tokenBucket{} + l.buckets[key] = bucket + } + return bucket +} + +type tokenBucket struct { + mu sync.Mutex + tokens float64 + last time.Time +} + +func (b *tokenBucket) reserve(now time.Time, qps float64, burst int) time.Duration { + if qps <= 0 { + return 0 + } + capacity := float64(burst) + if capacity <= 0 { + capacity = defaultInboundUpstreamBurst + } + b.mu.Lock() + defer b.mu.Unlock() + if b.last.IsZero() { + b.last = now + b.tokens = capacity + } + if now.After(b.last) { + elapsed := now.Sub(b.last).Seconds() + b.tokens += elapsed * qps + if b.tokens > capacity { + b.tokens = capacity + } + b.last = now + } + if b.tokens >= 1 { + b.tokens-- + return 0 + } + waitSeconds := (1 - b.tokens) / qps + wait := time.Duration(waitSeconds * float64(time.Second)) + if wait <= 0 { + wait = time.Nanosecond + } + b.tokens = 0 + b.last = b.last.Add(wait) + return b.last.Sub(now) +} + +type redisTokenBucketLimiter struct { + redis redisEvaler +} + +const redisTokenBucketScript = ` +local now = tonumber(ARGV[1]) +local qps = tonumber(ARGV[2]) +local burst = tonumber(ARGV[3]) +local ttl = tonumber(ARGV[4]) + +if qps == nil or qps <= 0 then + return {1, 0} +end +if burst == nil or burst <= 0 then + burst = 1 +end +if ttl == nil or ttl <= 0 then + ttl = 60000 +end + +local values = redis.call("HMGET", KEYS[1], "tokens", "updated_at") +local tokens = tonumber(values[1]) +local updated_at = tonumber(values[2]) +local capacity = burst + +if tokens == nil or updated_at == nil then + tokens = capacity + updated_at = now +end +if tokens > capacity then + tokens = capacity +end +if now > updated_at then + local elapsed = (now - updated_at) / 1000 + tokens = math.min(capacity, tokens + elapsed * qps) + updated_at = now +end + +local wait = 0 +if tokens >= 1 then + tokens = tokens - 1 +else + wait = math.ceil((1 - tokens) / qps * 1000) + if wait < 1 then + wait = 1 + end + tokens = 0 + updated_at = updated_at + wait + wait = updated_at - now + if wait < 0 then + wait = 0 + end +end + +redis.call("HSET", KEYS[1], "tokens", tostring(tokens), "updated_at", tostring(updated_at)) +redis.call("PEXPIRE", KEYS[1], ttl) +return {1, wait} +` + +func (l *redisTokenBucketLimiter) Wait(ctx context.Context, key string, qps float64, burst int) error { + wait, err := l.reserve(ctx, key, qps, burst) + if err != nil { + return err + } + if wait <= 0 { + return nil + } + timer := time.NewTimer(wait) + defer timer.Stop() + select { + case <-ctx.Done(): + return ctx.Err() + case <-timer.C: + return nil + } +} + +func (l *redisTokenBucketLimiter) reserve(ctx context.Context, key string, qps float64, burst int) (time.Duration, error) { + if l == nil || l.redis == nil || qps <= 0 { + return 0, nil + } + if burst <= 0 { + burst = defaultInboundUpstreamBurst + } + reply, err := l.redis.Eval(ctx, redisTokenBucketScript, 1, []string{key}, []any{ + time.Now().UnixMilli(), + strconv.FormatFloat(qps, 'f', -1, 64), + burst, + rateLimitRedisKeyTTLMillis(qps, burst), + }) + if err != nil { + return 0, err + } + values := reply.Int64s() + if len(values) < 2 || values[0] != 1 { + return 0, fmt.Errorf("unexpected redis rate limiter reply: %v", reply.Val()) + } + waitMillis := values[1] + if waitMillis <= 0 { + return 0, nil + } + return time.Duration(waitMillis) * time.Millisecond, nil +} + +func rateLimitRedisKeyTTLMillis(qps float64, burst int) int64 { + if qps <= 0 { + return int64(time.Minute / time.Millisecond) + } + if burst <= 0 { + burst = defaultInboundUpstreamBurst + } + idleMillis := int64(math.Ceil(float64(burst) / qps * 1000 * 2)) + minMillis := int64(time.Minute / time.Millisecond) + if idleMillis < minMillis { + return minMillis + } + return idleMillis +} diff --git a/internal/service/inbound/rate_limiter_test.go b/internal/service/inbound/rate_limiter_test.go new file mode 100644 index 0000000..32d57ad --- /dev/null +++ b/internal/service/inbound/rate_limiter_test.go @@ -0,0 +1,60 @@ +package inbound + +import "testing" + +func TestInboundCarrierForOperatorUsesDictionaryValues(t *testing.T) { + tests := []struct { + name string + operator int + carrier string + ok bool + }{ + {name: "mobile", operator: 1, carrier: inboundCarrierMobile, ok: true}, + {name: "telecom", operator: 2, carrier: inboundCarrierTelecom, ok: true}, + {name: "legacy telecom value is unsupported", operator: 3, carrier: "", ok: false}, + {name: "missing", operator: 0, carrier: "", ok: false}, + } + + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + carrier, ok := inboundCarrierForOperator(test.operator) + if carrier != test.carrier || ok != test.ok { + t.Fatalf("inboundCarrierForOperator(%d) = %q,%v, want %q,%v", test.operator, carrier, ok, test.carrier, test.ok) + } + }) + } +} + +func TestResolveInboundUpstreamRateLimitFromChannelMetadataUsesCarrierPolicy(t *testing.T) { + cfg := defaultInboundUpstreamRateLimitConfig() + + mobile := resolveInboundUpstreamRateLimitFromChannelMetadata(cfg, "chan-mobile", 1, "sim_yd_v5") + if mobile.carrier != inboundCarrierMobile { + t.Fatalf("mobile carrier = %q, want %q", mobile.carrier, inboundCarrierMobile) + } + if mobile.enabled { + t.Fatalf("mobile enabled = %v, want false", mobile.enabled) + } + + telecom := resolveInboundUpstreamRateLimitFromChannelMetadata(cfg, "chan-telecom", 2, "sim_dx_v1") + if telecom.carrier != inboundCarrierTelecom { + t.Fatalf("telecom carrier = %q, want %q", telecom.carrier, inboundCarrierTelecom) + } + if !telecom.enabled { + t.Fatalf("telecom enabled = %v, want true", telecom.enabled) + } + if telecom.qps != defaultInboundUpstreamQPS { + t.Fatalf("telecom qps = %v, want %v", telecom.qps, defaultInboundUpstreamQPS) + } +} + +func TestTelecomTemplateOverridesInconsistentOperatorMetadata(t *testing.T) { + for _, key := range []string{"sim_dx_v1", "sim_dx_v3"} { + for _, operator := range []int{0, 1, 2, 3} { + setting := resolveInboundUpstreamRateLimitFromChannelMetadata(defaultInboundUpstreamRateLimitConfig(), "telecom", operator, key) + if setting.carrier != inboundCarrierTelecom || !setting.enabled || setting.skipReason != "" { + t.Fatalf("template %q operator %d: %+v", key, operator, setting) + } + } + } +} diff --git a/internal/service/inbound/types.go b/internal/service/inbound/types.go new file mode 100644 index 0000000..4cbe94d --- /dev/null +++ b/internal/service/inbound/types.go @@ -0,0 +1,112 @@ +package inbound + +import ( + "context" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +type Repository interface { + Load(ctx context.Context, batchNo string) (*Task, error) + ReadOssFile(ctx context.Context, ossID int64) ([]byte, string, error) + FindExistingStocks(ctx context.Context, tenantID string, iccids, msisdns, imsis []string) ([]Stock, error) + SaveValidationFailure(ctx context.Context, task *Task) error + SaveSuccess(ctx context.Context, task *Task) error + SaveBusinessFailure(ctx context.Context, task *Task, reason string) error + SaveResultFile(ctx context.Context, task *Task, request batchresult.Request) error +} + +type Upstream interface { + QuerySimBasicInfo(ctx context.Context, input *model.SimBasicInfoInput) ([]model.SimBaseRes[model.SimBasicInfoItem], error) +} + +type Task struct { + Batch Batch + Source Source + Options Options + Rows []*Row + ValidCount int + FailureReason string +} + +type Batch struct { + ID int64 + BatchNo string + BatchType int + SourceType int + TenantID string + CreatedDept int64 + CreatedBy int64 + StartTime *gtime.Time + BatchInfo *gjson.Json +} + +type Source struct { + ImportFileOssID int64 + InputText string + InputFileOssID int64 + Range Range +} + +type Range struct { + RangeType string + StartICCID string + EndICCID string + StartMSISDN string + EndMSISDN string + StartIMSI string + EndIMSI string +} + +type Options struct { + ChanKey string + ValidateUpstreamBasicInfo string + CardType string + UpPkgCode string + SuppliersCode string + UpstreamActivationTime string + InsertToPlugEnabled string + CarrierTrafficCutoffEnabled string + AgreementExpiryDate string + HasTestPeriod string + TestPeriodDays string + TestPeriodLatestExpiryDate string + HasSilentPeriod string + SilentPeriodDays string + SilentPeriodLatestExpiryDate string +} + +type Row struct { + RowNo int + InputCode string + ICCID string + MSISDN string + IMSI string + IMEI string + ChanKey string + CardType string + UpPkgCode string + SuppliersCode string + UpstreamActivationTime string + InsertToPlugEnabled string + CarrierTrafficCutoffEnabled string + AgreementExpiryDate string + HasTestPeriod string + TestPeriodDays string + TestPeriodLatestExpiryDate string + HasSilentPeriod string + SilentPeriodDays string + SilentPeriodLatestExpiryDate string + Reason string + Raw map[string]string +} + +type Stock struct { + ICCID string `orm:"iccid"` + MSISDN string `orm:"msisdn"` + IMSI string `orm:"imsi"` +} diff --git a/internal/service/lifecycle/executor.go b/internal/service/lifecycle/executor.go new file mode 100644 index 0000000..2aabf3f --- /dev/null +++ b/internal/service/lifecycle/executor.go @@ -0,0 +1,242 @@ +package lifecycle + +import ( + "context" + "sort" + "strings" + "sync" + + "github.com/gogf/gf/v2/errors/gerror" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +type executor struct { + repository Repository + upstream Upstream +} + +func NewExecutor(repository Repository, upstream Upstream) Executor { + return &executor{repository: repository, upstream: upstream} +} + +func (e *executor) Execute(ctx context.Context, batchNo string) error { + task, err := e.repository.Load(ctx, strings.TrimSpace(batchNo)) + if err != nil { + return gerror.Wrap(err, "load lifecycle batch") + } + if !validateTask(task) { + if err = e.repository.SaveValidationFailure(ctx, task); err != nil { + return gerror.Wrap(err, "save lifecycle validation failure") + } + return gerror.Wrap(e.repository.Finalize(ctx, task), "finalize lifecycle validation failure") + } + + groups := groupByChanKey(task.Details) + keys := make([]string, 0, len(groups)) + for chanKey := range groups { + keys = append(keys, chanKey) + } + sort.Strings(keys) + for _, chanKey := range keys { + for index, outcome := range e.changeGroup(ctx, task, groups[chanKey]) { + detail := groups[chanKey][index] + resolved, saveErr := e.repository.SaveResult(ctx, task, detail, outcome) + if saveErr != nil { + return gerror.Wrapf(saveErr, "save lifecycle detail %d result", detail.ID) + } + detail.ApplyOutcome(resolved) + } + } + return gerror.Wrap(e.repository.Finalize(ctx, task), "finalize lifecycle batch") +} + +func validateTask(task *Task) bool { + if task == nil || len(task.Details) == 0 { + return false + } + hasFailure := false + for _, detail := range task.Details { + if detail == nil { + hasFailure = true + continue + } + validateDetail(task.Operation, detail) + if detail.Reason != "" { + hasFailure = true + } + } + if !hasFailure { + task.ValidationOK = true + return true + } + for _, detail := range task.Details { + if detail != nil && detail.Reason == "" { + detail.Reason = "same batch contains validation failure; upstream request was skipped" + } + } + task.FailureReason = "batch validation failed; upstream request was skipped" + return false +} + +func validateDetail(operation Operation, detail *Detail) { + if detail.Reason != "" { + return + } + if detail.Card == nil { + detail.Reason = "ICCID does not exist in SIM card records" + return + } + detail.MSISDN = strings.TrimSpace(detail.Card.MSISDN) + detail.IMSI = strings.TrimSpace(detail.Card.IMSI) + detail.ChanKey = strings.TrimSpace(detail.Card.ChanKey) + detail.StatusBefore = detail.Card.BossCardStatus + + if !detail.Authorized { + detail.Reason = "ICCID is outside the current operator authorization scope" + return + } + if detail.ChanKey == "" { + detail.Reason = "ICCID is missing chanKey" + return + } + if detail.InputMSISDN != "" && detail.InputMSISDN != detail.MSISDN { + detail.Reason = "MSISDN does not match the SIM card record" + return + } + if detail.InputIMSI != "" && detail.InputIMSI != detail.IMSI { + detail.Reason = "IMSI does not match the SIM card record" + return + } + // TODO: 正常系统开发完成后恢复卡状态限制 + // 注释原因：开发环境经常出现卡状态不在预期可操作范围内的情况，先放行 + // if detail.Card.BossCardStatus != operation.ExpectedStatus { + // detail.Reason = "SIM card status does not allow this lifecycle operation" + // return + // } +} + +func groupByChanKey(details []*Detail) map[string][]*Detail { + groups := make(map[string][]*Detail) + for _, detail := range details { + groups[detail.ChanKey] = append(groups[detail.ChanKey], detail) + } + return groups +} + +func (e *executor) changeGroup(ctx context.Context, task *Task, details []*Detail) []Outcome { + subBatches := splitDetails(details, consts.BatchSubBatchSize) + subOutcomes := make([][]Outcome, len(subBatches)) + + sem := make(chan struct{}, consts.BatchSubBatchConcurrency) + var wg sync.WaitGroup + for i, sub := range subBatches { + wg.Add(1) + sem <- struct{}{} + go func(index int, batch []*Detail) { + defer wg.Done() + defer func() { <-sem }() + subOutcomes[index] = e.changeSubBatch(ctx, task, batch) + }(i, sub) + } + wg.Wait() + + outcomes := make([]Outcome, 0, len(details)) + for _, subOutcome := range subOutcomes { + outcomes = append(outcomes, subOutcome...) + } + return outcomes +} + +// changeSubBatch 处理单个子批次：构建请求并调用上游，返回与该子批次明细等长的 Outcome 切片。 +func (e *executor) changeSubBatch(ctx context.Context, task *Task, details []*Detail) []Outcome { + input := &model.SimCardStatusChangeInput{ + SimKeys: make([]*model.SimKey, 0, len(details)), + ReqData: model.SimCardStatusChangeInputItem{OperType: task.Operation.OperType}, + } + for _, detail := range details { + input.SimKeys = append(input.SimKeys, &model.SimKey{ChanKey: detail.ChanKey, ICCID: detail.ICCID, MSISDN: detail.MSISDN, IMSI: detail.IMSI}) + } + response, err := e.upstream.ChangeSimStatus(ctx, input) + if err != nil { + outcomes := make([]Outcome, len(details)) + for index := range outcomes { + outcomes[index].Message = "upstream request failed: " + err.Error() + } + return outcomes + } + return mapOutcomes(details, response) +} + +// splitDetails 按固定大小把明细切成有序的子批次；空输入返回空切片。 +func splitDetails(details []*Detail, size int) [][]*Detail { + if size <= 0 { + size = 1 + } + batches := make([][]*Detail, 0, (len(details)+size-1)/size) + for i := 0; i < len(details); i += size { + end := i + size + if end > len(details) { + end = len(details) + } + batches = append(batches, details[i:end]) + } + return batches +} + +func mapOutcomes(details []*Detail, response []model.SimBaseRes[model.SimCardStatusChangeOutputItem]) []Outcome { + outcomes := make([]Outcome, len(details)) + used := make([]bool, len(response)) + for detailIndex, detail := range details { + matched := -1 + for responseIndex, item := range response { + if !used[responseIndex] && sameSim(detail, item.SimKey) { + matched = responseIndex + break + } + } + if matched < 0 && detailIndex < len(response) && !used[detailIndex] && response[detailIndex].SimKey == nil { + matched = detailIndex + } + if matched < 0 { + outcomes[detailIndex] = Outcome{Message: "upstream did not return a result for this SIM card"} + continue + } + used[matched] = true + item := response[matched] + message := strings.TrimSpace(item.Message) + if item.Code != 0 && message == "" { + message = "upstream processing failed" + } + outcomes[detailIndex] = Outcome{Success: item.Code == 0, Code: item.Code, Message: message, OrderNum: strings.TrimSpace(item.ResData.OrderNum)} + } + return outcomes +} + +func sameSim(detail *Detail, key *model.SimKey) bool { + if detail == nil || key == nil { + return false + } + if value := strings.TrimSpace(key.ICCID); value != "" { + return value == detail.ICCID + } + if value := strings.TrimSpace(key.MSISDN); value != "" { + return value == detail.MSISDN + } + if value := strings.TrimSpace(key.IMSI); value != "" { + return value == detail.IMSI + } + return false +} + +func OperationByBatchType(batchType int) (Operation, error) { + switch batchType { + case consts.BatchTypeSimCardStart: + return Operation{BatchType: batchType, OperType: consts.SimCardStatusOperTypeStart, ExpectedStatus: consts.SimBossCardStatusStopped, TargetStatus: consts.SimBossCardStatusActivated}, nil + case consts.BatchTypeSimCardStop: + return Operation{BatchType: batchType, OperType: consts.SimCardStatusOperTypeStop, ExpectedStatus: consts.SimBossCardStatusActivated, TargetStatus: consts.SimBossCardStatusStopped}, nil + default: + return Operation{}, gerror.Newf("unsupported lifecycle batch type %d", batchType) + } +} diff --git a/internal/service/lifecycle/executor_test.go b/internal/service/lifecycle/executor_test.go new file mode 100644 index 0000000..372fb25 --- /dev/null +++ b/internal/service/lifecycle/executor_test.go @@ -0,0 +1,192 @@ +package lifecycle + +import ( + "context" + "errors" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +func TestExecuteValidationFailureSkipsUpstream(t *testing.T) { + operation, err := OperationByBatchType(consts.BatchTypeSimCardStop) + if err != nil { + t.Fatal(err) + } + task := &Task{ + Batch: Batch{BatchNo: "B-1"}, + Operation: operation, + Details: []*Detail{ + {ID: 1, ICCID: "iccid-1", Authorized: true, Card: &Card{ID: 11, ChanKey: "channel", BossCardStatus: consts.SimBossCardStatusActivated}}, + {ID: 2, ICCID: "iccid-2", Authorized: false, Card: &Card{ID: 12, ChanKey: "channel", BossCardStatus: consts.SimBossCardStatusActivated}}, + }, + } + repo := &fakeRepository{task: task} + upstream := &fakeUpstream{responseError: errors.New("must not be called")} + + if err = NewExecutor(repo, upstream).Execute(context.Background(), "B-1"); err != nil { + t.Fatal(err) + } + if upstream.calls != 0 { + t.Fatalf("upstream calls = %d, want 0", upstream.calls) + } + if !repo.validationSaved || !repo.finalized { + t.Fatal("validation failure was not persisted and finalized") + } + if task.Details[0].Reason == "" || task.Details[1].Reason == "" { + t.Fatal("every detail must receive a validation failure reason") + } +} + +func TestExecuteMapsMixedResponseAndPreservesOrderNumber(t *testing.T) { + operation, err := OperationByBatchType(consts.BatchTypeSimCardStart) + if err != nil { + t.Fatal(err) + } + task := &Task{ + Batch: Batch{BatchNo: "B-2"}, + Operation: operation, + Details: []*Detail{ + {ID: 1, ICCID: "iccid-1", MSISDN: "msisdn-1", IMSI: "imsi-1", Authorized: true, Card: &Card{ID: 11, ChanKey: "channel", MSISDN: "msisdn-1", IMSI: "imsi-1", BossCardStatus: consts.SimBossCardStatusStopped}}, + {ID: 2, ICCID: "iccid-2", MSISDN: "msisdn-2", IMSI: "imsi-2", Authorized: true, Card: &Card{ID: 12, ChanKey: "channel", MSISDN: "msisdn-2", IMSI: "imsi-2", BossCardStatus: consts.SimBossCardStatusStopped}}, + }, + } + repo := &fakeRepository{task: task} + upstream := &fakeUpstream{response: []model.SimBaseRes[model.SimCardStatusChangeOutputItem]{ + {Code: 0, Message: "ok", SimKey: nil, ResData: model.SimCardStatusChangeOutputItem{OrderNum: "order-1"}}, + {Code: 9001, Message: "denied", SimKey: &model.SimKey{ICCID: "iccid-2"}, ResData: model.SimCardStatusChangeOutputItem{OrderNum: "order-2"}}, + }} + + if err = NewExecutor(repo, upstream).Execute(context.Background(), "B-2"); err != nil { + t.Fatal(err) + } + if upstream.input.ReqData.OperType != consts.SimCardStatusOperTypeStart { + t.Fatalf("operType = %q", upstream.input.ReqData.OperType) + } + if len(repo.saved) != 2 || !repo.saved[0].Success || repo.saved[0].OrderNum != "order-1" { + t.Fatalf("first result = %+v", repo.saved) + } + if repo.saved[1].Success || repo.saved[1].Code != 9001 || repo.saved[1].Message != "denied" || repo.saved[1].OrderNum != "order-2" { + t.Fatalf("second result = %+v", repo.saved[1]) + } + if !repo.finalized { + t.Fatal("batch was not finalized") + } +} + +func TestOperationByBatchTypeOperType(t *testing.T) { + start, err := OperationByBatchType(consts.BatchTypeSimCardStart) + if err != nil || start.OperType != "1" || start.TargetStatus != consts.SimBossCardStatusActivated { + t.Fatalf("start operation = %+v, err = %v", start, err) + } + stop, err := OperationByBatchType(consts.BatchTypeSimCardStop) + if err != nil || stop.OperType != "0" || stop.TargetStatus != consts.SimBossCardStatusStopped { + t.Fatalf("stop operation = %+v, err = %v", stop, err) + } +} + +func TestValidateDetailPopulatesCardFieldsOnStatusFailure(t *testing.T) { + t.Skip("TODO: 恢复卡状态限制后启用此测试") + operation, err := OperationByBatchType(consts.BatchTypeSimCardStart) + if err != nil { + t.Fatal(err) + } + detail := &Detail{ + ID: 1, + RowNo: 1, + ICCID: "iccid-1", + Card: &Card{ + ID: 11, + ICCID: "iccid-1", + MSISDN: "msisdn-1", + IMSI: "imsi-1", + ChanKey: "channel", + BossCardStatus: consts.SimBossCardStatusActivated, + }, + } + validateDetail(operation, detail) + if detail.Reason == "" { + t.Fatal("expected validation failure reason") + } + if detail.MSISDN != "msisdn-1" || detail.IMSI != "imsi-1" || detail.ChanKey != "channel" { + t.Fatalf("card fields not populated on status failure: %+v", detail) + } + if detail.StatusBefore != consts.SimBossCardStatusActivated { + t.Fatalf("StatusBefore = %d, want %d", detail.StatusBefore, consts.SimBossCardStatusActivated) + } + snapshot := detail.Snapshot() + if snapshot.Get("MSISDN").String() != "msisdn-1" || snapshot.Get("IMSI").String() != "imsi-1" || snapshot.Get("chanKey").String() != "channel" { + t.Fatalf("snapshot missing card fields: %s", snapshot.MustToJsonString()) + } +} + +func TestValidateDetailDetectsMSISDNMismatch(t *testing.T) { + operation, err := OperationByBatchType(consts.BatchTypeSimCardStart) + if err != nil { + t.Fatal(err) + } + detail := &Detail{ + ID: 1, + ICCID: "iccid-1", + Authorized: true, + InputMSISDN: "msisdn-input", + Card: &Card{ + ID: 11, + ICCID: "iccid-1", + MSISDN: "msisdn-card", + IMSI: "imsi-1", + ChanKey: "channel", + BossCardStatus: consts.SimBossCardStatusStopped, + }, + } + validateDetail(operation, detail) + if detail.Reason != "MSISDN does not match the SIM card record" { + t.Fatalf("unexpected reason: %q", detail.Reason) + } + if detail.MSISDN != "msisdn-card" || detail.InputMSISDN != "msisdn-input" { + t.Fatalf("mismatch not preserved: MSISDN=%q InputMSISDN=%q", detail.MSISDN, detail.InputMSISDN) + } + snapshot := detail.Snapshot() + if snapshot.Get("inputMsisdn").String() != "msisdn-input" || snapshot.Get("MSISDN").String() != "msisdn-card" { + t.Fatalf("snapshot mismatch fields wrong: %s", snapshot.MustToJsonString()) + } +} + +type fakeRepository struct { + task *Task + validationSaved bool + finalized bool + saved []Outcome +} + +func (r *fakeRepository) Load(context.Context, string) (*Task, error) { return r.task, nil } + +func (r *fakeRepository) SaveValidationFailure(_ context.Context, task *Task) error { + r.validationSaved = true + return nil +} + +func (r *fakeRepository) SaveResult(_ context.Context, _ *Task, detail *Detail, outcome Outcome) (Outcome, error) { + detail.ApplyOutcome(outcome) + r.saved = append(r.saved, outcome) + return outcome, nil +} + +func (r *fakeRepository) Finalize(context.Context, *Task) error { + r.finalized = true + return nil +} + +type fakeUpstream struct { + calls int + input *model.SimCardStatusChangeInput + response []model.SimBaseRes[model.SimCardStatusChangeOutputItem] + responseError error +} + +func (u *fakeUpstream) ChangeSimStatus(_ context.Context, input *model.SimCardStatusChangeInput) ([]model.SimBaseRes[model.SimCardStatusChangeOutputItem], error) { + u.calls++ + u.input = input + return u.response, u.responseError +} diff --git a/internal/service/lifecycle/restart_executor.go b/internal/service/lifecycle/restart_executor.go new file mode 100644 index 0000000..0a77afe --- /dev/null +++ b/internal/service/lifecycle/restart_executor.go @@ -0,0 +1,242 @@ +package lifecycle + +import ( + "context" + "encoding/json" + "sort" + "strings" + "time" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +const ( + restartStageStop = "stop" + restartStageResume = "resume" +) + +type RestartRepository interface { + FindRestartMain(ctx context.Context, batchNo string) (*entity.BatchMain, error) + LoadRestartStop(ctx context.Context, batchNo string) (*Task, error) + LoadRestartResume(ctx context.Context, batchNo string) (*Task, error) + SaveRestartValidationFailure(ctx context.Context, task *Task, stage string) error + SaveRestartResult(ctx context.Context, task *Task, detail *Detail, outcome Outcome, stage string) (Outcome, error) + WaitRestartResume(ctx context.Context, task *Task, resumeAt *gtime.Time) error + ClaimRestartResume(ctx context.Context, batchNo string, now *gtime.Time) (bool, error) + FinalizeRestart(ctx context.Context, batchNo string) error +} + +type RestartPublisher func(ctx context.Context, delayQueue, targetQueue string, delay time.Duration, data any) error + +type restartExecutor struct { + repository RestartRepository + upstream Upstream + publishDelayed RestartPublisher + now func() *gtime.Time +} + +func NewRestartExecutor(repository RestartRepository, upstream Upstream) Executor { + return NewRestartExecutorWithPublisher(repository, upstream, nil) +} + +func NewRestartExecutorWithPublisher(repository RestartRepository, upstream Upstream, publisher RestartPublisher) Executor { + if publisher == nil { + publisher = publishRestartDelayed + } + return &restartExecutor{ + repository: repository, + upstream: upstream, + publishDelayed: publisher, + now: gtime.Now, + } +} + +func publishRestartDelayed(ctx context.Context, delayQueue, targetQueue string, delay time.Duration, data any) error { + body, err := json.Marshal(data) + if err != nil { + return gerror.Wrap(err, "marshal restart resume message") + } + publishCtx, cancel := context.WithTimeout(ctx, 10*time.Second) + defer cancel() + return mq.PublishDelayedBytesWithHeaders(publishCtx, delayQueue, targetQueue, delay, body, nil) +} + +func (e *restartExecutor) CanExecuteProcessing(ctx context.Context, main *entity.BatchMain) (bool, error) { + if main == nil || main.BatchType != consts.BatchTypeSimCardRestart || main.BatchAsyncStatus != consts.BatchAsyncStatusProcessing { + return false, nil + } + if restartPhase(main.BatchInfo) != consts.RestartPhaseWaitingResume { + return false, nil + } + resumeAt := restartResumeAt(main.BatchInfo) + if resumeAt == nil { + g.Log().Warningf(ctx, "restart batch waiting without resumeAt, batchNo:%s", main.BatchNo) + return false, nil + } + now := e.currentTime() + if resumeAt.After(now) { + delay := resumeAt.Sub(now) + if delay < time.Millisecond { + delay = time.Millisecond + } + message := batch.Message{BatchNo: main.BatchNo, BatchType: main.BatchType} + if err := e.publishDelayed(ctx, consts.QueueBatchRestartResumeDelay, consts.QueueBatchExecute, delay, message); err != nil { + return false, gerror.Wrap(err, "reschedule early restart resume message") + } + g.Log().Infof(ctx, "restart batch resume message rescheduled before resumeAt, batchNo:%s, resumeAt:%s, delay:%s", main.BatchNo, resumeAt.String(), delay) + return false, nil + } + return true, nil +} + +func (e *restartExecutor) ClaimsProcessingReentry() bool { + return true +} + +func (e *restartExecutor) Execute(ctx context.Context, batchNo string) error { + batchNo = strings.TrimSpace(batchNo) + if batchNo == "" { + return gerror.New("restart batch number is empty") + } + main, err := e.repository.FindRestartMain(ctx, batchNo) + if err != nil { + return gerror.Wrap(err, "load restart batch main") + } + if main == nil { + return gerror.New("restart batch does not exist") + } + if restartPhase(main.BatchInfo) == consts.RestartPhaseWaitingResume { + claimed, claimErr := e.repository.ClaimRestartResume(ctx, batchNo, e.currentTime()) + if claimErr != nil { + return gerror.Wrap(claimErr, "claim restart resume phase") + } + if !claimed { + g.Log().Infof(ctx, "restart resume message ignored after CAS lost, batchNo:%s", batchNo) + return nil + } + return e.executeResume(ctx, batchNo) + } + return e.executeStop(ctx, batchNo) +} + +func (e *restartExecutor) executeStop(ctx context.Context, batchNo string) error { + task, err := e.repository.LoadRestartStop(ctx, batchNo) + if err != nil { + return gerror.Wrap(err, "load restart stop phase") + } + if !validateTask(task) { + if err = e.repository.SaveRestartValidationFailure(ctx, task, restartStageStop); err != nil { + return gerror.Wrap(err, "save restart stop validation failure") + } + return gerror.Wrap(e.repository.FinalizeRestart(ctx, batchNo), "finalize restart stop validation failure") + } + + successCount, err := e.changeRestartGroups(ctx, task, restartStageStop) + if err != nil { + return err + } + if successCount == 0 { + return gerror.Wrap(e.repository.FinalizeRestart(ctx, batchNo), "finalize restart with no stop success") + } + + delay := restartDelay(task.Batch.BatchInfo) + resumeAt := e.currentTime().Add(delay) + if err = e.repository.WaitRestartResume(ctx, task, resumeAt); err != nil { + return gerror.Wrap(err, "mark restart waiting resume") + } + message := batch.Message{BatchNo: task.Batch.BatchNo, BatchType: consts.BatchTypeSimCardRestart} + if err = e.publishDelayed(ctx, consts.QueueBatchRestartResumeDelay, consts.QueueBatchExecute, delay, message); err != nil { + return gerror.Wrap(err, "publish restart resume delayed message") + } + return nil +} + +func (e *restartExecutor) executeResume(ctx context.Context, batchNo string) error { + task, err := e.repository.LoadRestartResume(ctx, batchNo) + if err != nil { + return gerror.Wrap(err, "load restart resume phase") + } + if !validateTask(task) { + if err = e.repository.SaveRestartValidationFailure(ctx, task, restartStageResume); err != nil { + return gerror.Wrap(err, "save restart resume validation failure") + } + return gerror.Wrap(e.repository.FinalizeRestart(ctx, batchNo), "finalize restart resume validation failure") + } + if _, err = e.changeRestartGroups(ctx, task, restartStageResume); err != nil { + return err + } + return gerror.Wrap(e.repository.FinalizeRestart(ctx, batchNo), "finalize restart batch") +} + +func (e *restartExecutor) changeRestartGroups(ctx context.Context, task *Task, stage string) (int, error) { + groups := groupByChanKey(task.Details) + keys := make([]string, 0, len(groups)) + for chanKey := range groups { + keys = append(keys, chanKey) + } + sort.Strings(keys) + + successCount := 0 + for _, chanKey := range keys { + outcomes := (&executor{upstream: e.upstream}).changeGroup(ctx, task, groups[chanKey]) + for index, outcome := range outcomes { + detail := groups[chanKey][index] + resolved, saveErr := e.repository.SaveRestartResult(ctx, task, detail, outcome, stage) + if saveErr != nil { + return successCount, gerror.Wrapf(saveErr, "save restart %s detail %d result", stage, detail.ID) + } + detail.ApplyOutcome(resolved) + if resolved.Success { + successCount++ + } + } + } + return successCount, nil +} + +func (e *restartExecutor) currentTime() *gtime.Time { + if e.now != nil { + return e.now() + } + return gtime.Now() +} + +func restartDelay(info *gjson.Json) time.Duration { + seconds := 300 + if info != nil { + if configured := info.Get("resumeDelaySeconds").Int(); configured > 0 { + seconds = configured + } + } + return time.Duration(seconds) * time.Second +} + +func restartPhase(info *gjson.Json) string { + if info == nil { + return "" + } + return strings.TrimSpace(info.Get("restart.phase").String()) +} + +func restartResumeAt(info *gjson.Json) *gtime.Time { + if info == nil { + return nil + } + value := strings.TrimSpace(info.Get("restart.resumeAt").String()) + if value == "" { + return nil + } + parsed := gtime.NewFromStr(value) + if parsed == nil || parsed.IsZero() { + return nil + } + return parsed +} diff --git a/internal/service/lifecycle/restart_executor_test.go b/internal/service/lifecycle/restart_executor_test.go new file mode 100644 index 0000000..194db36 --- /dev/null +++ b/internal/service/lifecycle/restart_executor_test.go @@ -0,0 +1,303 @@ +package lifecycle + +import ( + "context" + "errors" + "testing" + "time" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +func TestRestartResumeBeforeScheduledTimeIsRescheduled(t *testing.T) { + now := gtime.New("2026-10-10 13:46:42") + repo := &fakeRestartRepository{ + main: restartMain(consts.RestartPhaseWaitingResume, gjson.New(map[string]any{ + "restart": map[string]any{"resumeAt": "2026-10-10 13:46:44"}, + })), + } + publisher := &recordingRestartPublisher{} + executor := NewRestartExecutorWithPublisher(repo, &fakeUpstream{}, publisher.PublishDelayed).(*restartExecutor) + executor.now = func() *gtime.Time { return now } + + ok, err := executor.CanExecuteProcessing(context.Background(), repo.main) + if ok || err != nil { + t.Fatalf("CanExecuteProcessing() = %t, %v; want false, nil", ok, err) + } + if publisher.calls != 1 || publisher.delay != 2*time.Second { + t.Fatalf("publisher calls=%d delay=%s; want one call with 2s delay", publisher.calls, publisher.delay) + } + if publisher.message != (batch.Message{BatchNo: repo.main.BatchNo, BatchType: repo.main.BatchType}) { + t.Fatalf("rescheduled message = %+v", publisher.message) + } + if repo.claimed || repo.finalized { + t.Fatal("early message must not claim or finalize the batch") + } +} + +func TestRestartResumeRescheduleFailureIsReturned(t *testing.T) { + now := gtime.New("2026-10-10 13:46:42") + main := restartMain(consts.RestartPhaseWaitingResume, gjson.New(map[string]any{ + "restart": map[string]any{"resumeAt": "2026-10-10 13:46:44"}, + })) + publishErr := errors.New("publish failed") + publisher := func(context.Context, string, string, time.Duration, any) error { return publishErr } + executor := NewRestartExecutorWithPublisher(&fakeRestartRepository{main: main}, &fakeUpstream{}, publisher).(*restartExecutor) + executor.now = func() *gtime.Time { return now } + ok, err := executor.CanExecuteProcessing(context.Background(), main) + if ok || !errors.Is(err, publishErr) || batch.IsRetryable(err) { + t.Fatalf("CanExecuteProcessing() = %t, %v; want publish error without execution retry flag", ok, err) + } +} + +func TestRestartStopValidationFailureSkipsUpstream(t *testing.T) { + task := restartStopTask() + task.Details = append(task.Details, &Detail{ + ID: 2, + ICCID: "iccid-2", + Authorized: false, + Card: &Card{ID: 12, ChanKey: "channel", BossCardStatus: consts.SimBossCardStatusActivated}, + }) + repo := &fakeRestartRepository{ + main: restartMain(consts.RestartPhaseCreated, nil), + stopTask: task, + } + upstream := &fakeUpstream{} + executor := NewRestartExecutorWithPublisher(repo, upstream, fakeRestartPublisher(t)).(*restartExecutor) + + if err := executor.Execute(context.Background(), "RST-1"); err != nil { + t.Fatal(err) + } + if upstream.calls != 0 { + t.Fatalf("upstream calls = %d, want 0", upstream.calls) + } + if repo.validationStage != restartStageStop || !repo.finalized { + t.Fatalf("validation/finalize state = stage:%q finalized:%t", repo.validationStage, repo.finalized) + } + if len(repo.saved) != 0 || repo.waited { + t.Fatalf("unexpected stop result or wait: saved=%d waited=%t", len(repo.saved), repo.waited) + } +} + +func TestRestartStopSuccessSchedulesDelayedResume(t *testing.T) { + now := gtime.NewFromStr("2026-07-20 10:00:00") + task := restartStopTask() + task.Batch.BatchInfo = gjson.New(map[string]any{ + "resumeDelaySeconds": 1, + "restart": map[string]any{"phase": consts.RestartPhaseCreated}, + }) + repo := &fakeRestartRepository{ + main: restartMain(consts.RestartPhaseCreated, gjson.New(map[string]any{"resumeDelaySeconds": 1})), + stopTask: task, + } + upstream := &fakeUpstream{response: []model.SimBaseRes[model.SimCardStatusChangeOutputItem]{ + {Code: 0, Message: "ok", ResData: model.SimCardStatusChangeOutputItem{OrderNum: "stop-order"}}, + }} + publisher := &recordingRestartPublisher{} + executor := NewRestartExecutorWithPublisher(repo, upstream, publisher.PublishDelayed).(*restartExecutor) + executor.now = func() *gtime.Time { return now } + + if err := executor.Execute(context.Background(), "RST-2"); err != nil { + t.Fatal(err) + } + if upstream.calls != 1 || upstream.input.ReqData.OperType != consts.SimCardStatusOperTypeStop { + t.Fatalf("upstream stop input = %+v calls=%d", upstream.input, upstream.calls) + } + if len(repo.saved) != 1 || repo.saved[0].stage != restartStageStop || !repo.saved[0].outcome.Success { + t.Fatalf("saved stop results = %+v", repo.saved) + } + if !repo.waited || repo.resumeAt.String() != now.Add(time.Second).String() { + t.Fatalf("resume wait = waited:%t resumeAt:%v", repo.waited, repo.resumeAt) + } + message, ok := publisher.message.(batch.Message) + if !ok || publisher.calls != 1 || publisher.delay != time.Second || message.BatchType != consts.BatchTypeSimCardRestart { + t.Fatalf("publisher = %+v", publisher) + } + if repo.finalized { + t.Fatal("batch must not finalize before resume when stop succeeded") + } +} + +func TestRestartResumeClaimLostSkipsDuplicateUpstream(t *testing.T) { + now := gtime.NewFromStr("2026-07-20 10:05:00") + repo := &fakeRestartRepository{ + main: restartMain(consts.RestartPhaseWaitingResume, gjson.New(map[string]any{"restart": map[string]any{"resumeAt": now.String()}})), + claimResult: false, + resumeTask: restartResumeTask(), + } + upstream := &fakeUpstream{} + executor := NewRestartExecutorWithPublisher(repo, upstream, fakeRestartPublisher(t)).(*restartExecutor) + executor.now = func() *gtime.Time { return now } + + if err := executor.Execute(context.Background(), "RST-3"); err != nil { + t.Fatal(err) + } + if !repo.claimed { + t.Fatal("resume CAS was not attempted") + } + if upstream.calls != 0 || repo.loadedResume || repo.finalized { + t.Fatalf("duplicate was not skipped: upstream=%d loadedResume=%t finalized=%t", upstream.calls, repo.loadedResume, repo.finalized) + } +} + +func TestRestartResumeExecutesAfterClaim(t *testing.T) { + now := gtime.NewFromStr("2026-07-20 10:05:00") + repo := &fakeRestartRepository{ + main: restartMain(consts.RestartPhaseWaitingResume, gjson.New(map[string]any{"restart": map[string]any{"resumeAt": now.String()}})), + claimResult: true, + resumeTask: restartResumeTask(), + } + upstream := &fakeUpstream{response: []model.SimBaseRes[model.SimCardStatusChangeOutputItem]{ + {Code: 0, Message: "ok", ResData: model.SimCardStatusChangeOutputItem{OrderNum: "resume-order"}}, + }} + executor := NewRestartExecutorWithPublisher(repo, upstream, fakeRestartPublisher(t)).(*restartExecutor) + executor.now = func() *gtime.Time { return now } + + if ok, err := executor.CanExecuteProcessing(context.Background(), repo.main); err != nil || !ok { + t.Fatalf("CanExecuteProcessing() = %t, %v; want true, nil", ok, err) + } + if err := executor.Execute(context.Background(), "RST-4"); err != nil { + t.Fatal(err) + } + if upstream.calls != 1 || upstream.input.ReqData.OperType != consts.SimCardStatusOperTypeStart { + t.Fatalf("upstream resume input = %+v calls=%d", upstream.input, upstream.calls) + } + if len(repo.saved) != 1 || repo.saved[0].stage != restartStageResume || !repo.saved[0].outcome.Success { + t.Fatalf("saved resume results = %+v", repo.saved) + } + if !repo.finalized { + t.Fatal("resume phase did not finalize the RST batch") + } +} + +func restartMain(phase string, info *gjson.Json) *entity.BatchMain { + data := map[string]any{"restart": map[string]any{"phase": phase}} + if info != nil { + for key, value := range info.Map() { + if key == "restart" { + restart := data["restart"].(map[string]any) + for restartKey, restartValue := range gjson.New(value).Map() { + restart[restartKey] = restartValue + } + continue + } + data[key] = value + } + } + return &entity.BatchMain{ + BatchNo: "RST", + BatchType: consts.BatchTypeSimCardRestart, + BatchAsyncStatus: consts.BatchAsyncStatusProcessing, + BatchInfo: gjson.New(data), + } +} + +func restartStopTask() *Task { + return &Task{ + Batch: Batch{ + BatchNo: "RST", + BatchType: consts.BatchTypeSimCardRestart, + BatchInfo: gjson.New(map[string]any{"restart": map[string]any{"phase": consts.RestartPhaseCreated}}), + }, + Operation: Operation{OperType: consts.SimCardStatusOperTypeStop, ExpectedStatus: consts.SimBossCardStatusActivated, TargetStatus: consts.SimBossCardStatusStopped}, + Details: []*Detail{ + {ID: 1, ICCID: "iccid-1", Authorized: true, Card: &Card{ID: 11, ChanKey: "channel", BossCardStatus: consts.SimBossCardStatusActivated}}, + }, + } +} + +func restartResumeTask() *Task { + return &Task{ + Batch: Batch{BatchNo: "RST", BatchType: consts.BatchTypeSimCardRestart}, + Operation: Operation{OperType: consts.SimCardStatusOperTypeStart, ExpectedStatus: consts.SimBossCardStatusStopped, TargetStatus: consts.SimBossCardStatusActivated}, + Details: []*Detail{ + {ID: 1, ICCID: "iccid-1", Authorized: true, Card: &Card{ID: 11, ChanKey: "channel", BossCardStatus: consts.SimBossCardStatusStopped}}, + }, + } +} + +type fakeRestartRepository struct { + main *entity.BatchMain + stopTask *Task + resumeTask *Task + validationStage string + saved []restartSavedResult + waited bool + resumeAt *gtime.Time + claimed bool + claimResult bool + loadedResume bool + finalized bool +} + +type restartSavedResult struct { + stage string + outcome Outcome +} + +func (r *fakeRestartRepository) FindRestartMain(context.Context, string) (*entity.BatchMain, error) { + return r.main, nil +} + +func (r *fakeRestartRepository) LoadRestartStop(context.Context, string) (*Task, error) { + return r.stopTask, nil +} + +func (r *fakeRestartRepository) LoadRestartResume(context.Context, string) (*Task, error) { + r.loadedResume = true + return r.resumeTask, nil +} + +func (r *fakeRestartRepository) SaveRestartValidationFailure(_ context.Context, _ *Task, stage string) error { + r.validationStage = stage + return nil +} + +func (r *fakeRestartRepository) SaveRestartResult(_ context.Context, _ *Task, detail *Detail, outcome Outcome, stage string) (Outcome, error) { + detail.ApplyOutcome(outcome) + r.saved = append(r.saved, restartSavedResult{stage: stage, outcome: outcome}) + return outcome, nil +} + +func (r *fakeRestartRepository) WaitRestartResume(_ context.Context, _ *Task, resumeAt *gtime.Time) error { + r.waited = true + r.resumeAt = resumeAt + return nil +} + +func (r *fakeRestartRepository) ClaimRestartResume(context.Context, string, *gtime.Time) (bool, error) { + r.claimed = true + return r.claimResult, nil +} + +func (r *fakeRestartRepository) FinalizeRestart(context.Context, string) error { + r.finalized = true + return nil +} + +type recordingRestartPublisher struct { + calls int + delay time.Duration + message any +} + +func (p *recordingRestartPublisher) PublishDelayed(_ context.Context, _, _ string, delay time.Duration, data any) error { + p.calls++ + p.delay = delay + p.message = data + return nil +} + +func fakeRestartPublisher(t *testing.T) RestartPublisher { + t.Helper() + return func(context.Context, string, string, time.Duration, any) error { + t.Fatal("publisher must not be called") + return nil + } +} diff --git a/internal/service/lifecycle/types.go b/internal/service/lifecycle/types.go new file mode 100644 index 0000000..fc9c5da --- /dev/null +++ b/internal/service/lifecycle/types.go @@ -0,0 +1,134 @@ +package lifecycle + +import ( + "context" + "strings" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +type Operation struct { + BatchType int + OperType string + ExpectedStatus int + TargetStatus int +} + +type Batch struct { + ID int64 + BatchNo string + BatchType int + TenantID string + CreatedBy int64 + CreatedDept int64 + StartTime *gtime.Time + BatchInfo *gjson.Json +} + +type Card struct { + ID int64 + ICCID string + MSISDN string + IMSI string + ChanKey string + BossCardStatus int +} + +type Detail struct { + ID int64 + RowNo int + ICCID string + InputMSISDN string + InputIMSI string + MSISDN string + IMSI string + Authorized bool + ChanKey string + StatusBefore int + Reason string + OrderNum string + UpstreamCode int + UpstreamReason string + Card *Card + OriginalSnapshot *gjson.Json +} + +type Outcome struct { + Success bool + Code int + Message string + OrderNum string +} + +type Task struct { + Batch Batch + Operation Operation + Details []*Detail + ValidationOK bool + FailureReason string +} + +func (d *Detail) ApplyOutcome(outcome Outcome) { + d.OrderNum = strings.TrimSpace(outcome.OrderNum) + d.UpstreamCode = outcome.Code + d.UpstreamReason = strings.TrimSpace(outcome.Message) + if outcome.Success { + d.Reason = "" + return + } + if d.UpstreamReason == "" { + d.UpstreamReason = "upstream processing failed" + } + d.Reason = d.UpstreamReason +} + +func (d *Detail) Snapshot() *gjson.Json { + if d == nil { + return gjson.New(nil) + } + return gjson.New(detailSnapshot{ + ICCID: d.ICCID, + InputMSISDN: d.InputMSISDN, + InputIMSI: d.InputIMSI, + MSISDN: d.MSISDN, + IMSI: d.IMSI, + Authorized: d.Authorized, + ChanKey: d.ChanKey, + StatusBefore: d.StatusBefore, + OrderNum: d.OrderNum, + UpstreamCode: d.UpstreamCode, + UpstreamMessage: d.UpstreamReason, + }) +} + +type detailSnapshot struct { + ICCID string `json:"ICCID"` + InputMSISDN string `json:"inputMsisdn"` + InputIMSI string `json:"inputImsi"` + MSISDN string `json:"MSISDN"` + IMSI string `json:"IMSI"` + Authorized bool `json:"authorized"` + ChanKey string `json:"chanKey"` + StatusBefore int `json:"statusBefore"` + OrderNum string `json:"orderNum"` + UpstreamCode int `json:"upstreamCode"` + UpstreamMessage string `json:"upstreamMessage"` +} + +type Repository interface { + Load(ctx context.Context, batchNo string) (*Task, error) + SaveValidationFailure(ctx context.Context, task *Task) error + SaveResult(ctx context.Context, task *Task, detail *Detail, outcome Outcome) (Outcome, error) + Finalize(ctx context.Context, task *Task) error +} + +type Upstream interface { + ChangeSimStatus(ctx context.Context, input *model.SimCardStatusChangeInput) ([]model.SimBaseRes[model.SimCardStatusChangeOutputItem], error) +} + +type Executor interface { + Execute(ctx context.Context, batchNo string) error +} diff --git a/internal/service/monthlyusage/handler.go b/internal/service/monthlyusage/handler.go new file mode 100644 index 0000000..7d96872 --- /dev/null +++ b/internal/service/monthlyusage/handler.go @@ -0,0 +1,179 @@ +package monthlyusage + +import ( + "context" + "fmt" + "strings" + + zsetconsumer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consumer/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + monthlyusagerepository "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/monthlyusage" +) + +// errCodeTelecomChannelLimited is the carrier error code that means the whole +// channel is throttled or disabled and must stop for this run. +const errCodeTelecomChannelLimited = "S00004" + +// The gateway uses code 9999 without an errCode for transport, HTTP, and +// response-decoding failures. These failures are safe to retry from the task +// lease; a populated errCode is treated as a business result. +const chanGatewayInternalErrorCode = 9999 + +// tokenExpiredCode is the carrier code for an expired/invalid upstream token. +// The gateway already invalidates and refreshes the token once on this code; +// when it still reaches the worker, retrying in a later round lets a refreshed +// token succeed instead of parking the card as a terminal business failure. +const tokenExpiredCode = 12021 + +const ( + mobileBusyFirstCode = 13000 + mobileBusyLastCode = 13007 + mobileBusyClass = "mobile_busy" +) + +// Mobile usage templates are identified by the template returned with the +// channel metadata. The channel name and error message are not stable +// carrier identifiers. +// +// The set is derived from the OneLink registration table in the upstream +// proxy (iotlcm-upstream-proxy/internal/logic/upstreamimpl/sim_yd.go +// init()): every key below is registered to the China Mobile OneLink +// sSimYD implementation, so numeric codes 13000-13007 emitted by any of +// them carry OneLink semantics and are safe to retry across rounds. +// Keep this list in sync with that registration table; templates that are +// not registered there (including unknown future keys) stay "generic" and +// are never classified as mobile. +var mobileUsageTemplates = map[string]struct{}{ + "sim_yd_v5": {}, + "sim_yd_mock_v5": {}, + "sim_sdtywl_v1": {}, + "sim_onelink_tengyu_v1": {}, + "sim_onelink_xuzhou_mobile_v1": {}, + "sim_onelink_lelian_v1": {}, + "sim_onelink_qianxun_v1": {}, + "sim_onelink_henan_xinjing_v1": {}, + "sim_henan_xinjing_direct_v1": {}, + "sim_onelink_zibo_v1": {}, + "sim_tibiot_v1": {}, + "sim_qianxun_v3": {}, + "sim_yunda_v1": {}, + "sim_mingqian_v3": {}, + "sim_daihe_v1": {}, + "sim_onelink_daihe_v1": {}, + "sim_onelink_daliuliang_v1": {}, + "sim__daliuliang_v1": {}, + "sim_onelink_mobile_public_v1": {}, + "sim_onelink_kedang_v1": {}, + "sim_onelink_beixin_v1": {}, + "lanshang_8000": {}, + "lanshang_8003": {}, +} + +const telecomUsageTemplate = "sim_dx_v1" + +func usageCarrierForTemplate(tempKey string) string { + tempKey = strings.ToLower(strings.TrimSpace(tempKey)) + if model.IsTelecomTemplate(tempKey) { + return "telecom" + } + if _, ok := mobileUsageTemplates[tempKey]; ok { + return "mobile" + } + return "generic" +} + +// classifyUsageFailure maps a gateway item to the queue semantics owned by +// this service. Carrier identity is resolved from the template first; error +// text and channel names are deliberately excluded from the decision. +func classifyUsageFailure(tempKey string, code int, errCode string) (errorClass string, retryable, blockChannel bool) { + errorClass = "business" + carrier := usageCarrierForTemplate(tempKey) + switch { + case carrier == "telecom" && strings.EqualFold(strings.TrimSpace(errCode), errCodeTelecomChannelLimited): + return "channel_limited", false, true + case carrier == "mobile" && code >= mobileBusyFirstCode && code <= mobileBusyLastCode: + return mobileBusyClass, true, false + case code == chanGatewayInternalErrorCode && strings.TrimSpace(errCode) == "": + return "network", true, false + case code == tokenExpiredCode: + return "token_expired", true, false + default: + return errorClass, false, false + } +} + +type UsageClient interface { + QuerySimDataUsage(context.Context, *model.SimDataUsageInput) ([]model.SimBaseRes[model.SimDataUsageItem], error) +} + +type Handler struct { + client UsageClient + repo UsageRepository +} + +type UsageRepository interface { + ApplyBatch(context.Context, []monthlyusagerepository.UsageResult) error +} + +func NewHandler(client UsageClient, repo UsageRepository) (*Handler, error) { + if client == nil || repo == nil { + return nil, fmt.Errorf("monthly usage handler dependencies are incomplete") + } + return &Handler{client: client, repo: repo}, nil +} + +func (h *Handler) Query(ctx context.Context, task queue.Task) zsetconsumer.Result { + result := zsetconsumer.Result{Task: task} + output, err := h.client.QuerySimDataUsage(ctx, &model.SimDataUsageInput{ + SimKeys: []*model.SimKey{{ + ChanKey: task.ChanKey, + ICCID: task.ICCID, + IMSI: task.IMSI, + MSISDN: task.MSISDN, + }}, + }) + if err != nil { + result.ErrorClass = "network" + result.Retryable = true + result.Message = err.Error() + return result + } + if len(output) == 0 || output[0].SimKey == nil { + result.ErrorClass = "protocol" + result.Retryable = true + result.Message = "chan returned empty current usage result" + return result + } + item := output[0] + if task.ICCID != "" && item.SimKey.ICCID != task.ICCID { + result.ErrorClass = "protocol" + result.Retryable = true + result.Message = "chan returned mismatched card identifier" + return result + } + result.Code = item.Code + result.Message = item.Message + result.ErrCode = item.ErrCode + if item.Code != 0 { + result.ErrorClass, result.Retryable, result.BlockChannel = classifyUsageFailure(task.TempKey, item.Code, item.ErrCode) + return result + } + result.Success = true + result.UsageMB = item.ResData.DataUsage + return result +} + +func (h *Handler) Apply(ctx context.Context, results []zsetconsumer.Result) error { + updates := make([]monthlyusagerepository.UsageResult, 0, len(results)) + for _, result := range results { + if !result.Success { + continue + } + updates = append(updates, monthlyusagerepository.UsageResult{ + CardID: result.Task.CardID, + UsageMB: result.UsageMB, + }) + } + return h.repo.ApplyBatch(ctx, updates) +} diff --git a/internal/service/monthlyusage/handler_test.go b/internal/service/monthlyusage/handler_test.go new file mode 100644 index 0000000..2668df1 --- /dev/null +++ b/internal/service/monthlyusage/handler_test.go @@ -0,0 +1,252 @@ +package monthlyusage + +import ( + "context" + "errors" + "fmt" + "strings" + "testing" + + zsetconsumer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consumer/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + monthlyusagerepository "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/monthlyusage" +) + +type usageClientStub struct { + output []model.SimBaseRes[model.SimDataUsageItem] + err error +} + +func (s *usageClientStub) QuerySimDataUsage(context.Context, *model.SimDataUsageInput) ([]model.SimBaseRes[model.SimDataUsageItem], error) { + return s.output, s.err +} + +type usageRepositoryStub struct { + calls int + updates []monthlyusagerepository.UsageResult + err error +} + +func (s *usageRepositoryStub) ApplyBatch(_ context.Context, updates []monthlyusagerepository.UsageResult) error { + s.calls++ + s.updates = append(s.updates, updates...) + return s.err +} + +func TestQueryClassifiesCurrentUsageResults(t *testing.T) { + task := queue.Task{TaskID: "1", ChanKey: "telecom-a", TempKey: telecomUsageTemplate, CardID: 1, ICCID: "89860001"} + tests := []struct { + name string + client *usageClientStub + success bool + errorClass string + retryable bool + blockChannel bool + }{ + { + name: "success", + client: &usageClientStub{output: []model.SimBaseRes[model.SimDataUsageItem]{{ + SimKey: &model.SimKey{ICCID: task.ICCID}, + ResData: model.SimDataUsageItem{DataUsage: 12.5}, + }}}, + success: true, + }, + { + name: "mobile busy is terminal business failure", + client: &usageClientStub{output: []model.SimBaseRes[model.SimDataUsageItem]{{SimKey: &model.SimKey{ICCID: task.ICCID}, Code: 13000, Message: "busy"}}}, + errorClass: "business", + }, + { + name: "gateway internal failure is retryable", + client: &usageClientStub{output: []model.SimBaseRes[model.SimDataUsageItem]{{SimKey: &model.SimKey{ICCID: task.ICCID}, Code: 9999, Message: "upstream timeout"}}}, + errorClass: "network", + retryable: true, + }, + { + name: "gateway business error code remains terminal", + client: &usageClientStub{output: []model.SimBaseRes[model.SimDataUsageItem]{{SimKey: &model.SimKey{ICCID: task.ICCID}, Code: 9999, ErrCode: "9999", Message: "invalid request"}}}, + errorClass: "business", + }, + { + name: "telecom threshold blocks channel via errCode", + client: &usageClientStub{output: []model.SimBaseRes[model.SimDataUsageItem]{{SimKey: &model.SimKey{ICCID: task.ICCID}, Code: 9999, Message: "流量查询失败", ErrCode: "S00004"}}}, + errorClass: "channel_limited", + blockChannel: true, + }, + { + name: "message text alone does not block channel", + client: &usageClientStub{output: []model.SimBaseRes[model.SimDataUsageItem]{{SimKey: &model.SimKey{ICCID: task.ICCID}, Code: 13000, Message: "carrier S00004 mentioned in text"}}}, + errorClass: "business", + }, + { + name: "network failure is terminal for run", + client: &usageClientStub{err: errors.New("connection reset")}, + errorClass: "network", + retryable: true, + }, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + handler, err := NewHandler(test.client, &usageRepositoryStub{}) + if err != nil { + t.Fatal(err) + } + result := handler.Query(context.Background(), task) + if result.Success != test.success || result.ErrorClass != test.errorClass || result.Retryable != test.retryable || result.BlockChannel != test.blockChannel { + t.Fatalf("result = %+v", result) + } + }) + } +} + +func TestMobileTemplatesClassifyBusyForRoundRetry(t *testing.T) { + for template := range mobileUsageTemplates { + for code := mobileBusyFirstCode; code <= mobileBusyLastCode; code++ { + t.Run(fmt.Sprintf("%s/code-%d", template, code), func(t *testing.T) { + handler, err := NewHandler(&usageClientStub{output: []model.SimBaseRes[model.SimDataUsageItem]{{ + SimKey: &model.SimKey{ICCID: "89860001"}, Code: code, Message: "system busy", + }}}, &usageRepositoryStub{}) + if err != nil { + t.Fatal(err) + } + result := handler.Query(context.Background(), queue.Task{TaskID: "busy", TempKey: template, ICCID: "89860001"}) + if result.ErrorClass != mobileBusyClass || !result.Retryable || result.BlockChannel { + t.Fatalf("template %q code %d result = %+v", template, code, result) + } + }) + } + } +} + +func TestLanshangSharedTokenTemplatesRetryAcrossRounds(t *testing.T) { + for _, template := range []string{"lanshang_8000", "lanshang_8003"} { + for _, code := range []int{13000, 13001} { + t.Run(fmt.Sprintf("%s/code-%d", template, code), func(t *testing.T) { + errorClass, retryable, blockChannel := classifyUsageFailure(template, code, "") + if errorClass != mobileBusyClass || !retryable || blockChannel { + t.Fatalf("template %q code %d = (%q, %t, %t)", template, code, errorClass, retryable, blockChannel) + } + }) + } + } +} + +func TestMobileBusyRangeBoundariesAndPermanentCodes(t *testing.T) { + template := "sim_onelink_kedang_v1" + tests := []struct { + name string + code int + errCode string + mobileBusy bool + retryable bool + }{ + {name: "13007 is last busy code", code: 13007, mobileBusy: true, retryable: true}, + {name: "13008 is outside busy range", code: 13008}, + {name: "12021 token expiry is retryable but not busy", code: 12021, retryable: true}, + {name: "11011 permanent failure is not busy", code: 11011}, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + errorClass, retryable, _ := classifyUsageFailure(template, test.code, test.errCode) + if test.mobileBusy { + if errorClass != mobileBusyClass || !retryable { + t.Fatalf("code %d = (%q, %t)", test.code, errorClass, retryable) + } + return + } + if errorClass == mobileBusyClass { + t.Fatalf("code %d = (%q, %t)", test.code, errorClass, retryable) + } + if retryable != test.retryable { + t.Fatalf("code %d retryable = %t, want %t (errorClass=%q)", test.code, retryable, test.retryable, errorClass) + } + }) + } +} + +func TestTelecomTemplateIsNeverMobileBusy(t *testing.T) { + for code := mobileBusyFirstCode; code <= mobileBusyLastCode; code++ { + errorClass, retryable, blockChannel := classifyUsageFailure(telecomUsageTemplate, code, "") + if errorClass == mobileBusyClass || retryable || blockChannel { + t.Fatalf("telecom code %d = (%q, %t, %t)", code, errorClass, retryable, blockChannel) + } + } +} + +func TestUnknownTemplateDefaultsToGenericNotMobile(t *testing.T) { + if carrier := usageCarrierForTemplate("sim_dx_v2_future"); carrier != "generic" { + t.Fatalf("unknown template carrier = %q", carrier) + } + errorClass, retryable, _ := classifyUsageFailure("sim_dx_v2_future", 13000, "") + if errorClass == mobileBusyClass || retryable { + t.Fatalf("unknown template 13000 = (%q, %t)", errorClass, retryable) + } +} + +func TestTelecomV3ChannelLimitClassification(t *testing.T) { + errorClass, retryable, blockChannel := classifyUsageFailure(" SIM_DX_V3 ", 9999, "S00004") + if errorClass != "channel_limited" || retryable || !blockChannel { + t.Fatalf("v3 channel limit = (%q, %t, %t)", errorClass, retryable, blockChannel) + } +} + +func TestNonTelecomS00004DoesNotBlockChannel(t *testing.T) { + handler, err := NewHandler(&usageClientStub{output: []model.SimBaseRes[model.SimDataUsageItem]{{ + SimKey: &model.SimKey{ICCID: "89860001"}, Code: 9999, ErrCode: errCodeTelecomChannelLimited, + }}}, &usageRepositoryStub{}) + if err != nil { + t.Fatal(err) + } + result := handler.Query(context.Background(), queue.Task{TaskID: "mobile", TempKey: "sim_yd_v5", ICCID: "89860001"}) + if result.ErrorClass != "business" || result.BlockChannel { + t.Fatalf("result = %+v", result) + } +} + +func TestApplyWritesSuccessfulResultsInOneBatch(t *testing.T) { + repository := &usageRepositoryStub{} + handler, err := NewHandler(&usageClientStub{}, repository) + if err != nil { + t.Fatal(err) + } + results := []zsetconsumer.Result{ + {Task: queue.Task{CardID: 1}, Success: true, UsageMB: 1.5}, + {Task: queue.Task{CardID: 2}, ErrorClass: "business", Code: 13000}, + {Task: queue.Task{CardID: 3}, Success: true, UsageMB: 3.5}, + } + if err = handler.Apply(context.Background(), results); err != nil { + t.Fatal(err) + } + if repository.calls != 1 || len(repository.updates) != 2 { + t.Fatalf("repository calls = %d, updates = %+v", repository.calls, repository.updates) + } + if repository.updates[0].CardID != 1 || repository.updates[1].CardID != 3 { + t.Fatalf("updates = %+v", repository.updates) + } +} + +func TestQueryPreservesNetworkFailureMessage(t *testing.T) { + handler, err := NewHandler(&usageClientStub{err: errors.New("connection refused")}, &usageRepositoryStub{}) + if err != nil { + t.Fatal(err) + } + result := handler.Query(context.Background(), queue.Task{ICCID: "89860001"}) + if result.ErrorClass != "network" || !strings.Contains(result.Message, "connection refused") { + t.Fatalf("result = %+v", result) + } +} + +func TestQuerySkipsIccidCheckWhenTaskHasNoIccid(t *testing.T) { + handler, err := NewHandler(&usageClientStub{output: []model.SimBaseRes[model.SimDataUsageItem]{{ + SimKey: &model.SimKey{ICCID: ""}, + ResData: model.SimDataUsageItem{DataUsage: 7.5}, + }}}, &usageRepositoryStub{}) + if err != nil { + t.Fatal(err) + } + result := handler.Query(context.Background(), queue.Task{TaskID: "1", ChanKey: "telecom-a", CardID: 1, MSISDN: "14900000001"}) + if !result.Success || result.UsageMB != 7.5 { + t.Fatalf("result = %+v", result) + } +} diff --git a/internal/service/monthlyusage/run_integration_test.go b/internal/service/monthlyusage/run_integration_test.go new file mode 100644 index 0000000..ceeae45 --- /dev/null +++ b/internal/service/monthlyusage/run_integration_test.go @@ -0,0 +1,160 @@ +package monthlyusage + +import ( + "context" + "encoding/json" + "errors" + "os" + "path/filepath" + "testing" + "time" + + zsetconsumer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consumer/zset" + fullsyncarchive "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/archive" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract" + fullsyncruntime "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/runtime" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil" +) + +type runUsageClient struct{} + +func (runUsageClient) QuerySimDataUsage(_ context.Context, input *model.SimDataUsageInput) ([]model.SimBaseRes[model.SimDataUsageItem], error) { + iccid := input.SimKeys[0].ICCID + switch iccid { + case "retry": + return nil, errors.New("temporary network failure") + case "business": + return []model.SimBaseRes[model.SimDataUsageItem]{{SimKey: &model.SimKey{ICCID: iccid}, Code: 11011, Message: "business failure"}}, nil + default: + return []model.SimBaseRes[model.SimDataUsageItem]{{SimKey: &model.SimKey{ICCID: iccid}, ResData: model.SimDataUsageItem{DataUsage: 12.5}}}, nil + } +} + +func TestMonthlyRunRetryConvergesAndArchives(t *testing.T) { + redis := testutil.Redis(t) + ctx := context.Background() + store, err := zset.New(redis, "flow:") + if err != nil { + t.Fatal(err) + } + store.EnableArchiveIndex(true) + const runID = "monthly-retry-archive" + if err := store.CreateRun(ctx, zset.RunMeta{RunID: runID, Operation: "current_month_usage", QueryMonth: "2026-09"}); err != nil { + t.Fatal(err) + } + tasks := []zset.Task{ + {RunID: runID, TaskID: "retry", CardID: 1, ChanKey: "channel", TempKey: "sim_yd_v5", ICCID: "retry"}, + {RunID: runID, TaskID: "business", CardID: 2, ChanKey: "channel", TempKey: "sim_yd_v5", ICCID: "business"}, + {RunID: runID, TaskID: "success", CardID: 3, ChanKey: "channel", TempKey: "sim_yd_v5", ICCID: "success"}, + } + if n, err := store.EnqueueRunBatch(ctx, runID, "channel", tasks, time.Now()); err != nil || n != len(tasks) { + t.Fatalf("enqueue = %d, err = %v", n, err) + } + if err := store.MarkProducerDone(ctx, runID); err != nil { + t.Fatal(err) + } + repo := &usageRepositoryStub{} + handler, err := NewHandler(runUsageClient{}, repo) + if err != nil { + t.Fatal(err) + } + consumer, err := zsetconsumer.New(store, handler, zsetconsumer.Config{ + RunQueueEnabled: true, + RoundRetryEnabled: true, + MaxRounds: 2, + ShouldRetryNextRound: func(result zsetconsumer.Result) bool { + return !result.Success && result.Retryable + }, + PollInterval: 500 * time.Millisecond, + ClaimBatch: 3, + TemplateQPS: map[string]float64{"sim_yd_v5": 1000000}, + LeaseTTL: time.Minute, + TaskTimeout: 30 * time.Second, + }) + if err != nil { + t.Fatal(err) + } + runCtx, cancel := context.WithCancel(ctx) + done := make(chan error, 1) + go func() { done <- consumer.Run(runCtx) }() + defer func() { + cancel() + if err := <-done; err != nil { + t.Errorf("consumer stopped: %v", err) + } + }() + + deadline := time.Now().Add(10 * time.Second) + var meta zset.RunMeta + for time.Now().Before(deadline) { + meta, err = store.GetRunMeta(ctx, runID) + if err != nil { + t.Fatal(err) + } + if meta.NextRoundCount == 1 { + break + } + time.Sleep(10 * time.Millisecond) + } + if meta.CurrentRound != 1 || meta.ActiveCount != 0 || meta.InflightCount != 0 || meta.NextRoundCount != 1 || meta.SuccessCount != 1 || meta.FailedCount != 1 || meta.Status != "running" { + t.Fatalf("first round did not defer only retryable failure: %+v", meta) + } + failedKey, _ := store.FailedKey(runID) + if count, err := redis.Do(ctx, "HLEN", failedKey); err != nil || count.Int() != 1 { + t.Fatalf("first round final failures = %v, err = %v", count, err) + } + for time.Now().Before(deadline) { + meta, err = store.GetRunMeta(ctx, runID) + if err != nil { + t.Fatal(err) + } + if meta.Status == "completed_with_failures" { + break + } + time.Sleep(10 * time.Millisecond) + } + if meta.Status != "completed_with_failures" || meta.CurrentRound != 2 || meta.SuccessCount != 1 || meta.FailedCount != 2 || meta.ActiveCount != 0 || meta.NextRoundCount != 0 || meta.CompletedAt.IsZero() { + t.Fatalf("run did not converge at max rounds: %+v", meta) + } + if len(repo.updates) != 1 || repo.updates[0].CardID != 3 { + t.Fatalf("successful usage update changed: %+v", repo.updates) + } + failed, err := redis.Do(ctx, "HGET", failedKey, "retry") + if err != nil { + t.Fatal(err) + } + var record zset.FailureRecord + if err := json.Unmarshal([]byte(failed.String()), &record); err != nil || record.Round != 2 || record.FailureType != "network" { + t.Fatalf("exhausted failure = %+v, err = %v", record, err) + } + + root := t.TempDir() + runtimeRepo, err := fullsyncruntime.New(store, root) + if err != nil { + t.Fatal(err) + } + page, err := runtimeRepo.ListArchiveWork(ctx, contract.PageRequest{Limit: 10}) + if err != nil || len(page.Items) != 1 || page.Items[0].RunID != runID { + t.Fatalf("pending archive = %+v, err = %v", page, err) + } + service := fullsyncarchive.New(root, runtimeRepo, runtimeRepo, runtimeRepo) + if err := service.RunOnce(ctx); err != nil { + t.Fatal(err) + } + data, err := os.ReadFile(filepath.Join(root, "run-"+runID+".json")) + if err != nil { + t.Fatal(err) + } + var archive contract.RunArchive + if err := json.Unmarshal(data, &archive); err != nil { + t.Fatal(err) + } + if archive.Run.Status != contract.RunCompletedWithFailures || archive.Run.SuccessCount != 1 || archive.Run.FailedCount != 2 || len(archive.Rounds) != 2 { + t.Fatalf("archived run summary = %+v", archive) + } + if first, last := archive.Rounds[0], archive.Rounds[1]; first.Round != 1 || first.EnqueuedCount != 3 || first.SuccessCount != 1 || first.FailedCount != 1 || first.NextRoundCount != 1 || last.Round != 2 || last.EnqueuedCount != 1 || last.FailedCount != 1 || last.NextRoundCount != 0 { + t.Fatalf("archived round stats = %+v", archive.Rounds) + } +} diff --git a/internal/service/monthlyusagehistory/handler.go b/internal/service/monthlyusagehistory/handler.go new file mode 100644 index 0000000..2406bb9 --- /dev/null +++ b/internal/service/monthlyusagehistory/handler.go @@ -0,0 +1,259 @@ +// Package monthlyusagehistory implements the zset.Handler for previous-month +// usage history. +// +// It queries one card per task through the upstream gateway (the same path the +// status sync uses), validates the returned period and identity, and writes the +// usage to the TDengine monthly table. +// +// The failure classifier is intentionally separate from the monthly-usage one +// and deliberately conservative: only errors that are known to be transient are +// retried. An unknown carrier code is parked as a terminal business failure +// instead of being retried, because a retry storm against a rate-limited carrier +// is worse than a card that needs manual review (see the 2026-09-16 disk-full +// incident). Like the monthly classifier, it never inspects the error text. +package monthlyusagehistory + +import ( + "context" + "fmt" + "strings" + "time" + + zsetconsumer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consumer/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +const ( + // errCodeTelecomChannelLimited is the carrier error code that means the + // whole channel is throttled or disabled. + errCodeTelecomChannelLimited = "S00004" // chanGatewayInternalError is the gateway code for transport, HTTP and + // response-decoding failures. An empty errCode distinguishes those from a + // carrier business result that happens to use the same number. + chanGatewayInternalError = 9999 + // tokenExpiredCode is the carrier code for an expired upstream token. The + // gateway refreshes the token once; a later round can then succeed. + tokenExpiredCode = 12021 + + mobileBusyFirstCode = 13000 + mobileBusyLastCode = 13007 + mobileBusyClass = "mobile_busy" +) + +// mobileUsageTemplates identifies the China Mobile OneLink implementations. +// Carrier identity is resolved from the channel template rather than from the +// channel name or the error text, both of which are unstable. +// +// The set mirrors the OneLink registration table in the upstream proxy and is +// kept in sync with the monthly-usage classifier +// (service/monthlyusage/handler.go). Unknown templates stay "generic" and are +// never classified as mobile. +var mobileUsageTemplates = map[string]struct{}{ + "sim_yd_v5": {}, + "sim_yd_mock_v5": {}, + "sim_sdtywl_v1": {}, + "sim_onelink_tengyu_v1": {}, + "sim_onelink_xuzhou_mobile_v1": {}, + "sim_onelink_lelian_v1": {}, + "sim_onelink_qianxun_v1": {}, + "sim_onelink_henan_xinjing_v1": {}, + "sim_henan_xinjing_direct_v1": {}, + "sim_onelink_zibo_v1": {}, + "sim_tibiot_v1": {}, + "sim_qianxun_v3": {}, + "sim_yunda_v1": {}, + "sim_mingqian_v3": {}, + "sim_daihe_v1": {}, + "sim_onelink_daihe_v1": {}, + "sim_onelink_daliuliang_v1": {}, + "sim__daliuliang_v1": {}, + "sim_onelink_mobile_public_v1": {}, + "sim_onelink_kedang_v1": {}, + "sim_onelink_beixin_v1": {}, + "lanshang_8000": {}, + "lanshang_8003": {}, +} + +// usageCarrierForTemplate resolves a channel template to its carrier. +// +// Telecom is detected through model.IsTelecomTemplate so every worker path +// shares one definition (it covers both sim_dx_v1 and sim_dx_v3). Hard-coding +// the v1 template here would silently classify a v3 channel as generic, which +// in turn would drop its S00004 channel-blocking behaviour. +func usageCarrierForTemplate(tempKey string) string { + tempKey = strings.ToLower(strings.TrimSpace(tempKey)) + if model.IsTelecomTemplate(tempKey) { + return "telecom" + } + if _, ok := mobileUsageTemplates[tempKey]; ok { + return "mobile" + } + return "generic" +} + +// classifyMonthlyHistoryFailure maps one upstream item to the queue semantics owned +// by this service. +// +// message is accepted only so callers can log it; it never influences the +// decision, because text matching was the root cause of the 2026-09-14 channel +// retry loop. The default is a non-retryable business failure: a wrong retry +// multiplies carrier throttling, while a missing retry costs one manual replay. +func classifyMonthlyHistoryFailure(tempKey string, code int, errCode string) (errorClass string, retryable, blockChannel bool) { + carrier := usageCarrierForTemplate(tempKey) + switch { + case carrier == "telecom" && strings.EqualFold(strings.TrimSpace(errCode), errCodeTelecomChannelLimited): + return "channel_limited", false, true + case carrier == "mobile" && code >= mobileBusyFirstCode && code <= mobileBusyLastCode: + return mobileBusyClass, true, false + case code == tokenExpiredCode: + return "token_expired", true, false + case code == chanGatewayInternalError && strings.TrimSpace(errCode) == "": + return "network", true, false + default: + return "business", false, false + } +} + +// UsageClient is the upstream surface needed by the handler. +type UsageClient interface { + QuerySimDatausageMonthly(context.Context, *model.SimDataUsageMonthlyInput) ([]model.SimBaseRes[model.SimDataUsageMonthlyOutputItem], error) +} + +// UsageRepository persists the monthly usage history rows. +type UsageRepository interface { + UpdateMonthly(context.Context, []model.TrafficUsageMonthlyItem) error +} + +type Handler struct { + client UsageClient + repo UsageRepository +} + +func NewHandler(client UsageClient, repo UsageRepository) (*Handler, error) { + if client == nil || repo == nil { + return nil, fmt.Errorf("monthly history handler dependencies are incomplete") + } + return &Handler{client: client, repo: repo}, nil +} + +// Query performs one upstream monthly-usage request for a single card. +// +// task.QueryMonth carries the run's period identifier: for a monthly history run +// that is yyyyMM, and it is sent upstream verbatim as queryDate. +func (h *Handler) Query(ctx context.Context, task queue.Task) zsetconsumer.Result { + result := zsetconsumer.Result{Task: task} + queryMonth := strings.TrimSpace(task.QueryMonth) + if queryMonth == "" { + result.ErrorClass = "protocol" + result.Message = "monthly history task is missing its query month" + return result + } + output, err := h.client.QuerySimDatausageMonthly(ctx, &model.SimDataUsageMonthlyInput{ + SimKeys: []*model.SimKey{{ + ChanKey: task.ChanKey, + ICCID: task.ICCID, + IMSI: task.IMSI, + MSISDN: task.MSISDN, + }}, + ReqData: model.SimDataUsageMonthlyInputItem{QueryDate: queryMonth}, + }) + if err != nil { + result.ErrorClass = "network" + result.Retryable = true + result.Message = err.Error() + return result + } + if len(output) == 0 || output[0].SimKey == nil { + result.ErrorClass = "protocol" + result.Retryable = true + result.Message = "chan returned empty monthly usage result" + return result + } + item := output[0] + if task.ICCID != "" && item.SimKey.ICCID != task.ICCID { + result.ErrorClass = "protocol" + result.Retryable = true + result.Message = "chan returned mismatched card identifier" + return result + } + result.Code = item.Code + result.Message = item.Message + result.ErrCode = item.ErrCode + if item.Code != 0 { + result.ErrorClass, result.Retryable, result.BlockChannel = classifyMonthlyHistoryFailure(task.TempKey, item.Code, item.ErrCode) + return result + } + // A successful code with a different (or missing) period would write the + // wrong month, so it is treated as a protocol failure rather than silently + // accepted. This mirrors the legacy admin reconcile requirement. + if period, ok := normalizeMonthlyHistoryQueryMonth(item.ResData.DeadLine); !ok || period != queryMonth { + result.ErrorClass = "protocol" + result.Retryable = true + result.Message = "chan returned mismatched monthly usage period" + return result + } + result.Success = true + result.UsageMB = item.ResData.DataAmount + return result +} + +// Apply writes every successful result to the TDengine monthly table. +// +// The timestamp is derived from the run period rather than from the response, so +// every card in a run lands on the same 23:59:59.000 row that the dashboard and +// the diagnosis page read. +func (h *Handler) Apply(ctx context.Context, results []zsetconsumer.Result) error { + items := make([]model.TrafficUsageMonthlyItem, 0, len(results)) + for _, result := range results { + if !result.Success { + continue + } + month, ok := parseMonthlyHistoryQueryMonth(strings.TrimSpace(result.Task.QueryMonth)) + if !ok { + return fmt.Errorf("monthly history task %s has an invalid period %q", result.Task.TaskID, result.Task.QueryMonth) + } + items = append(items, model.TrafficUsageMonthlyItem{ + TrafficUsageBase: model.TrafficUsageBase{ + Iccid: result.Task.ICCID, + UsageMb: result.UsageMB, + PackageCode: result.Task.PackageCode, + }, + Month: month, + }) + } + return h.repo.UpdateMonthly(ctx, items) +} + +// ShouldRetryNextRound is the monthly-history retry policy. It is passed to the +// run-aware ZSet consumer only; it keeps the classifier's verdict authoritative. +func ShouldRetryNextRound(result zsetconsumer.Result) bool { + return !result.Success && result.Retryable +} + +var monthlyHistoryLocation = time.FixedZone("UTC+8", 8*60*60) + +// normalizeMonthlyHistoryQueryMonth canonicalizes an upstream period to yyyyMM +// and reports whether the value was interpretable at all. +// +// The monthly usage endpoint answers with DeadLine, which may be a month, a +// month-first date, or a full date; all of those collapse to the same yyyyMM so +// the run period can be compared against the response. +func normalizeMonthlyHistoryQueryMonth(value string) (string, bool) { + for _, layout := range []string{"200601", "2006-01", "20060102", "2006-01-02"} { + parsed, err := time.ParseInLocation(layout, strings.TrimSpace(value), monthlyHistoryLocation) + if err == nil { + return parsed.Format("200601"), true + } + } + return "", false +} + +// parseMonthlyHistoryQueryMonth parses a run period. The month-first day is +// used because the repository derives the end-of-month timestamp from it. +func parseMonthlyHistoryQueryMonth(value string) (time.Time, bool) { + parsed, err := time.ParseInLocation("200601", value, monthlyHistoryLocation) + if err != nil { + return time.Time{}, false + } + return parsed, true +} diff --git a/internal/service/monthlyusagehistory/handler_test.go b/internal/service/monthlyusagehistory/handler_test.go new file mode 100644 index 0000000..3707f89 --- /dev/null +++ b/internal/service/monthlyusagehistory/handler_test.go @@ -0,0 +1,298 @@ +package monthlyusagehistory + +import ( + "context" + "errors" + "testing" + + zsetconsumer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consumer/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" +) + +type fakeClient struct { + output []model.SimBaseRes[model.SimDataUsageMonthlyOutputItem] + err error + // gotQueryDate records the period the handler sent upstream so a test can + // prove the task period is forwarded verbatim. + gotQueryDate string +} + +func (c *fakeClient) QuerySimDatausageMonthly(_ context.Context, input *model.SimDataUsageMonthlyInput) ([]model.SimBaseRes[model.SimDataUsageMonthlyOutputItem], error) { + c.gotQueryDate = input.ReqData.QueryDate + return c.output, c.err +} + +type fakeRepo struct { + items []model.TrafficUsageMonthlyItem + err error +} + +func (r *fakeRepo) UpdateMonthly(_ context.Context, items []model.TrafficUsageMonthlyItem) error { + r.items = append(r.items, items...) + return r.err +} + +func monthlyTask() queue.Task { + return queue.Task{ + TaskID: "run:op:42", + RunID: "run", + Operation: model.OperationPreviousDayUsageFullSync, + QueryMonth: "202609", + ChanKey: "217", + TempKey: "sim_yd_v5", + CardID: 42, + ICCID: "89860042", + PackageCode: "PKG-A", + } +} + +func TestQueryForwardsTaskMonthUpstream(t *testing.T) { + client := &fakeClient{output: []model.SimBaseRes[model.SimDataUsageMonthlyOutputItem]{{ + Code: 0, + SimKey: &model.SimKey{ICCID: "89860042"}, + ResData: model.SimDataUsageMonthlyOutputItem{ + DeadLine: "202609", + DataAmount: 12.5, + }, + }}} + handler, err := NewHandler(client, &fakeRepo{}) + if err != nil { + t.Fatal(err) + } + result := handler.Query(context.Background(), monthlyTask()) + if !result.Success { + t.Fatalf("expected success, got class=%s message=%s", result.ErrorClass, result.Message) + } + if client.gotQueryDate != "202609" { + t.Fatalf("upstream queryDate = %q, want the task period 202609", client.gotQueryDate) + } + if result.UsageMB != 12.5 { + t.Fatalf("usage = %v, want 12.5", result.UsageMB) + } +} + +func TestQueryAcceptsEquivalentPeriodFormats(t *testing.T) { + for _, period := range []string{"202609", "2026-09", "20260901", "2026-09-01"} { + client := &fakeClient{output: []model.SimBaseRes[model.SimDataUsageMonthlyOutputItem]{{ + Code: 0, + SimKey: &model.SimKey{ICCID: "89860042"}, + ResData: model.SimDataUsageMonthlyOutputItem{DeadLine: period, DataAmount: 1}, + }}} + handler, _ := NewHandler(client, &fakeRepo{}) + if result := handler.Query(context.Background(), monthlyTask()); !result.Success { + t.Fatalf("period %q was rejected: class=%s message=%s", period, result.ErrorClass, result.Message) + } + } +} + +func TestQueryRejectsMismatchedMonth(t *testing.T) { + client := &fakeClient{output: []model.SimBaseRes[model.SimDataUsageMonthlyOutputItem]{{ + Code: 0, + SimKey: &model.SimKey{ICCID: "89860042"}, + ResData: model.SimDataUsageMonthlyOutputItem{DeadLine: "202608", DataAmount: 1}, + }}} + handler, _ := NewHandler(client, &fakeRepo{}) + result := handler.Query(context.Background(), monthlyTask()) + if result.Success { + t.Fatal("a mismatched period must not be accepted") + } + if result.ErrorClass != "protocol" || !result.Retryable { + t.Fatalf("class=%s retryable=%v, want protocol/retryable", result.ErrorClass, result.Retryable) + } +} + +func TestQueryRejectsMissingMonthEcho(t *testing.T) { + // A channel that omits DeadLine from a successful response must fail too: + // the task period would be unverifiable, so writing the row risks + // wrong-month data. This mirrors the admin reconcile requirement. + client := &fakeClient{output: []model.SimBaseRes[model.SimDataUsageMonthlyOutputItem]{{ + Code: 0, + SimKey: &model.SimKey{ICCID: "89860042"}, + ResData: model.SimDataUsageMonthlyOutputItem{DataAmount: 1}, + }}} + handler, _ := NewHandler(client, &fakeRepo{}) + result := handler.Query(context.Background(), monthlyTask()) + if result.Success { + t.Fatal("a missing period echo must not be accepted") + } + if result.ErrorClass != "protocol" || !result.Retryable { + t.Fatalf("class=%s retryable=%v, want protocol/retryable", result.ErrorClass, result.Retryable) + } +} + +func TestQueryRejectsMismatchedICCID(t *testing.T) { + client := &fakeClient{output: []model.SimBaseRes[model.SimDataUsageMonthlyOutputItem]{{ + Code: 0, + SimKey: &model.SimKey{ICCID: "89869999"}, + ResData: model.SimDataUsageMonthlyOutputItem{DeadLine: "202609"}, + }}} + handler, _ := NewHandler(client, &fakeRepo{}) + result := handler.Query(context.Background(), monthlyTask()) + if result.Success || result.ErrorClass != "protocol" || !result.Retryable { + t.Fatalf("class=%s retryable=%v, want protocol/retryable", result.ErrorClass, result.Retryable) + } +} + +func TestQueryRejectsEmptyResult(t *testing.T) { + handler, _ := NewHandler(&fakeClient{}, &fakeRepo{}) + result := handler.Query(context.Background(), monthlyTask()) + if result.Success || result.ErrorClass != "protocol" || !result.Retryable { + t.Fatalf("class=%s retryable=%v, want protocol/retryable", result.ErrorClass, result.Retryable) + } +} + +func TestQueryTreatsTransportErrorAsRetryableNetwork(t *testing.T) { + handler, _ := NewHandler(&fakeClient{err: errors.New("connection refused")}, &fakeRepo{}) + result := handler.Query(context.Background(), monthlyTask()) + if result.ErrorClass != "network" || !result.Retryable { + t.Fatalf("class=%s retryable=%v, want network/retryable", result.ErrorClass, result.Retryable) + } +} + +func TestQueryRejectsTaskWithoutMonth(t *testing.T) { + handler, _ := NewHandler(&fakeClient{}, &fakeRepo{}) + task := monthlyTask() + task.QueryMonth = "" + result := handler.Query(context.Background(), task) + if result.Success || result.Retryable { + t.Fatalf("a task without a period must fail without retrying: %+v", result) + } +} + +func TestApplyWritesPackageCodeAndEndOfMonthTimestamp(t *testing.T) { + repo := &fakeRepo{} + handler, _ := NewHandler(&fakeClient{}, repo) + err := handler.Apply(context.Background(), []zsetconsumer.Result{ + {Task: monthlyTask(), Success: true, UsageMB: 7.25}, + {Task: monthlyTask(), Success: false}, + }) + if err != nil { + t.Fatal(err) + } + if len(repo.items) != 1 { + t.Fatalf("wrote %d items, want only the successful one", len(repo.items)) + } + item := repo.items[0] + if item.Iccid != "89860042" || item.UsageMb != 7.25 || item.PackageCode != "PKG-A" { + t.Fatalf("item = %+v, want iccid/usage/packageCode carried from the task", item) + } + // The handler supplies the month; repository.snapshotMonth turns it into the + // end-of-month 23:59:59.000 row (covered by repository_test.go). Assert the + // month here so a period mix-up cannot slip through. + if got := item.Month.Format("2006-01"); got != "2026-09" { + t.Fatalf("month = %s, want the run period 2026-09", got) + } +} + +func TestApplyRejectsInvalidMonth(t *testing.T) { + handler, _ := NewHandler(&fakeClient{}, &fakeRepo{}) + task := monthlyTask() + task.QueryMonth = "2026-10-09" + if err := handler.Apply(context.Background(), []zsetconsumer.Result{{Task: task, Success: true}}); err == nil { + t.Fatal("expected an unparseable period to fail the write") + } +} + +func TestClassifyMonthlyHistoryFailure(t *testing.T) { + tests := []struct { + name string + tempKey string + code int + errCode string + wantClass string + wantRetry bool + wantBlocked bool + }{ + {name: "telecom channel limited", tempKey: "sim_dx_v1", code: 500, errCode: "S00004", wantClass: "channel_limited", wantBlocked: true}, + {name: "telecom channel limited lowercase", tempKey: "sim_dx_v1", code: 500, errCode: "s00004", wantClass: "channel_limited", wantBlocked: true}, + // sim_dx_v3 is a telecom template too. It is resolved through + // model.IsTelecomTemplate so a v3 channel keeps S00004 blocking instead + // of falling through to generic and retrying every card to exhaustion. + {name: "telecom v3 channel limited", tempKey: "sim_dx_v3", code: 500, errCode: "S00004", wantClass: "channel_limited", wantBlocked: true}, + {name: "telecom v3 template is not treated as mobile", tempKey: "sim_dx_v3", code: 13000, wantClass: "business"}, + {name: "mobile busy first code", tempKey: "sim_yd_v5", code: 13000, wantClass: "mobile_busy", wantRetry: true}, + {name: "mobile busy last code", tempKey: "sim_yd_v5", code: 13007, wantClass: "mobile_busy", wantRetry: true}, + {name: "mobile one past the busy range", tempKey: "sim_yd_v5", code: 13008, wantClass: "business"}, + {name: "mobile one below the busy range", tempKey: "sim_yd_v5", code: 12999, wantClass: "business"}, + {name: "busy code on a non-mobile template", tempKey: "sim_qianxun_v3_bogus", code: 13000, wantClass: "business"}, + {name: "token expired", tempKey: "sim_yd_v5", code: 12021, wantClass: "token_expired", wantRetry: true}, + {name: "gateway internal without errCode", tempKey: "sim_yd_v5", code: 9999, errCode: "", wantClass: "network", wantRetry: true}, + {name: "gateway code with a carrier errCode stays business", tempKey: "sim_yd_v5", code: 9999, errCode: "E42", wantClass: "business"}, + {name: "telecom S00004 without the telecom template is not channel limited", tempKey: "sim_yd_v5", code: 500, errCode: "S00004", wantClass: "business"}, + {name: "unknown code is not retried", tempKey: "sim_yd_v5", code: 4242, wantClass: "business"}, + {name: "unknown code on an unknown template is not retried", tempKey: "some_new_channel_v9", code: 13000, wantClass: "business"}, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + class, retry, blocked := classifyMonthlyHistoryFailure(test.tempKey, test.code, test.errCode) + if class != test.wantClass || retry != test.wantRetry || blocked != test.wantBlocked { + t.Fatalf("classify(%q, %d, %q) = (%s, %v, %v), want (%s, %v, %v)", + test.tempKey, test.code, test.errCode, class, retry, blocked, test.wantClass, test.wantRetry, test.wantBlocked) + } + }) + } +} + +// TestClassifyIgnoresMessageText pins the 2026-09-14 discipline: classification +// must never depend on carrier text, because text matching caused the channel +// retry loop that the rule was introduced to prevent. +func TestClassifyIgnoresMessageText(t *testing.T) { + // A "busy"-looking message on an unknown code must stay non-retryable. + if _, retry, _ := classifyMonthlyHistoryFailure("sim_yd_v5", 4242, ""); retry { + t.Fatal("an unknown code must not become retryable because its message looked transient") + } + // And a busy code must stay retryable regardless of what the text says. + if _, retry, _ := classifyMonthlyHistoryFailure("sim_yd_v5", 13000, ""); !retry { + t.Fatal("a mobile busy code must stay retryable") + } +} + +func TestShouldRetryNextRoundFollowsClassifier(t *testing.T) { + if ShouldRetryNextRound(zsetconsumer.Result{Success: true}) { + t.Fatal("a successful result must not be retried") + } + if !ShouldRetryNextRound(zsetconsumer.Result{Retryable: true}) { + t.Fatal("a retryable failure must be deferred to the next round") + } + if ShouldRetryNextRound(zsetconsumer.Result{Retryable: false}) { + t.Fatal("a terminal failure must not be retried") + } +} + +func TestUsageCarrierForTemplateMatchesSharedDefinition(t *testing.T) { + // The monthly-history classifier must agree with model.IsTelecomTemplate for every + // telecom template the rest of the worker recognises. A drift here means a + // channel silently loses its S00004 blocking behaviour. + for _, tempKey := range []string{"sim_dx_v1", "sim_dx_v3", "SIM_DX_V1", " sim_dx_v3 "} { + if got := usageCarrierForTemplate(tempKey); got != "telecom" { + t.Errorf("usageCarrierForTemplate(%q) = %q, want telecom", tempKey, got) + } + } + for _, tempKey := range []string{"sim_yd_v5", "sim_onelink_daihe_v1"} { + if got := usageCarrierForTemplate(tempKey); got != "mobile" { + t.Errorf("usageCarrierForTemplate(%q) = %q, want mobile", tempKey, got) + } + } + for _, tempKey := range []string{"", "some_new_channel_v9", "sim_dx_v2"} { + if got := usageCarrierForTemplate(tempKey); got != "generic" { + t.Errorf("usageCarrierForTemplate(%q) = %q, want generic", tempKey, got) + } + } +} + +func TestQueryBlockChannelPropagatesForTelecomS00004(t *testing.T) { + client := &fakeClient{output: []model.SimBaseRes[model.SimDataUsageMonthlyOutputItem]{{ + Code: 500, + ErrCode: "S00004", + Message: "channel disabled", + SimKey: &model.SimKey{ICCID: "89860042"}, + }}} + handler, _ := NewHandler(client, &fakeRepo{}) + task := monthlyTask() + task.TempKey = "sim_dx_v1" + result := handler.Query(context.Background(), task) + if result.ErrorClass != "channel_limited" || !result.BlockChannel || result.Retryable { + t.Fatalf("telecom S00004 must block the channel without retrying: %+v", result) + } +} diff --git a/internal/service/packagecustomer/publisher.go b/internal/service/packagecustomer/publisher.go new file mode 100644 index 0000000..af65241 --- /dev/null +++ b/internal/service/packagecustomer/publisher.go @@ -0,0 +1,61 @@ +package packagecustomer + +import ( + "context" + "strings" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +func PublishSync(ctx context.Context, message model.PackageCustomerSyncMessage) error { + message.TenantId = strings.TrimSpace(message.TenantId) + message.Trigger = strings.TrimSpace(message.Trigger) + message.SourceBatchNo = strings.TrimSpace(message.SourceBatchNo) + if message.TenantId == "" { + return gerror.New("package customer sync tenantId is empty") + } + if message.DeptId <= 0 { + return gerror.New("package customer sync deptId is empty") + } + if message.Trigger == "" { + message.Trigger = "worker" + } + if err := mq.Publish(ctx, consts.QueuePackageCustomerSync, message); err != nil { + return gerror.Wrapf(err, "publish package customer sync failed, queue:%s", consts.QueuePackageCustomerSync) + } + g.Log().Infof(ctx, "package customer sync published, tenantId:%s, deptId:%d, trigger:%s, sourceBatchNo:%s", + message.TenantId, message.DeptId, message.Trigger, message.SourceBatchNo) + return nil +} + +func PublishSyncSafe(ctx context.Context, message model.PackageCustomerSyncMessage) { + if err := PublishSync(ctx, message); err != nil { + g.Log().Warningf(ctx, "package customer sync publish skipped, tenantId:%s, deptId:%d, trigger:%s, sourceBatchNo:%s, err:%+v", + message.TenantId, message.DeptId, message.Trigger, message.SourceBatchNo, err) + } +} + +func PublishSyncForDeptsSafe(ctx context.Context, tenantId string, userId int64, deptIds []int64, trigger, sourceBatchNo string) { + seen := make(map[int64]struct{}, len(deptIds)) + for _, deptId := range deptIds { + if deptId <= 0 { + continue + } + if _, ok := seen[deptId]; ok { + continue + } + seen[deptId] = struct{}{} + PublishSyncSafe(ctx, model.PackageCustomerSyncMessage{ + TenantId: tenantId, + DeptId: deptId, + UserId: userId, + Trigger: trigger, + SourceBatchNo: sourceBatchNo, + }) + } +} diff --git a/internal/service/packagecustomer/service.go b/internal/service/packagecustomer/service.go new file mode 100644 index 0000000..9dde592 --- /dev/null +++ b/internal/service/packagecustomer/service.go @@ -0,0 +1,177 @@ +package packagecustomer + +import ( + "context" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +const ( + sysDeptTable = "sys_dept" + cardInfoTable = "t_sim_card_info" + flowPackageTable = "t_sim_flow_package_info" + packageCustomerTable = "t_package_customer" +) + +var ErrDeptNotFound = gerror.New("package customer sync dept not found") + +type Service struct { + db gdb.DB +} + +type deptRow struct { + DeptId int64 `orm:"dept_id"` + ParentId int64 `orm:"parent_id"` + Ancestors string `orm:"ancestors"` + DeptName string `orm:"dept_name"` +} + +type packageRow struct { + PackageCode string `orm:"package_code"` + PackageName string `orm:"package_name"` +} + +func New() *Service { + return &Service{} +} + +func (s *Service) SyncDeptPackages(ctx context.Context, message model.PackageCustomerSyncMessage) error { + message.TenantId = strings.TrimSpace(message.TenantId) + message.Trigger = strings.TrimSpace(message.Trigger) + if message.TenantId == "" { + return gerror.New("package customer sync tenantId is empty") + } + if message.DeptId <= 0 { + return gerror.New("package customer sync deptId is empty") + } + var packageCount int + err := s.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + dept, err := s.lockDept(ctx, tx, message.TenantId, message.DeptId) + if err != nil { + return err + } + packages, err := s.loadPackages(ctx, tx, message.TenantId, dept) + if err != nil { + return err + } + packageCount = len(packages) + return s.rebuild(ctx, tx, message.TenantId, dept, message.UserId, packages) + }) + if err != nil { + return err + } + g.Log().Infof(ctx, "package customer sync finished, tenantId:%s, deptId:%d, packages:%d, trigger:%s, sourceBatchNo:%s", + message.TenantId, message.DeptId, packageCount, message.Trigger, message.SourceBatchNo) + return nil +} + +func (s *Service) database() gdb.DB { + if s.db != nil { + return s.db + } + return g.DB() +} + +func (s *Service) lockDept(ctx context.Context, tx gdb.TX, tenantId string, deptId int64) (*deptRow, error) { + var dept *deptRow + err := tx.Ctx(ctx).Model(sysDeptTable).Safe(). + Fields("dept_id", "parent_id", "ancestors", "dept_name"). + Where("tenant_id", tenantId). + Where("dept_id", deptId). + WhereNull("deleted_at"). + LockUpdate(). + Limit(1). + Scan(&dept) + if err != nil { + return nil, gerror.Wrap(err, "load package customer sync dept failed") + } + if dept == nil || dept.DeptId <= 0 { + return nil, ErrDeptNotFound + } + return dept, nil +} + +func (s *Service) loadPackages(ctx context.Context, tx gdb.TX, tenantId string, dept *deptRow) ([]packageRow, error) { + if deptIsTop(dept) { + var packages []packageRow + err := tx.Ctx(ctx).Model(flowPackageTable).Safe(). + Fields("package_code", "package_name"). + Where("tenant_id", tenantId). + Scan(&packages) + if err != nil { + return nil, gerror.Wrap(err, "load top dept packages failed") + } + return packages, nil + } + + var packages []packageRow + err := tx.Ctx(ctx).Model(cardInfoTable+" c").Safe(). + LeftJoin(flowPackageTable+" p", "c.current_package_code = p.package_code"). + Fields("DISTINCT c.current_package_code AS package_code", "p.package_name AS package_name"). + Where("c.tenant_id", tenantId). + Where("c.dept_id", dept.DeptId). + Where("p.tenant_id", tenantId). + Where("c.current_package_code IS NOT NULL"). + Where("c.current_package_code <> ?", ""). + Where("p.package_code IS NOT NULL"). + Scan(&packages) + if err != nil { + return nil, gerror.Wrap(err, "load dept packages failed") + } + return packages, nil +} + +func (s *Service) rebuild(ctx context.Context, tx gdb.TX, tenantId string, dept *deptRow, userId int64, packages []packageRow) error { + if _, err := tx.Ctx(ctx).Model(packageCustomerTable).Safe(). + Where("tenant_id", tenantId). + Where("dept_id", dept.DeptId). + Delete(); err != nil { + return gerror.Wrap(err, "clear dept package customer relation failed") + } + + now := gtime.Now() + rows := make([]g.Map, 0, len(packages)) + seen := make(map[string]struct{}, len(packages)) + for _, item := range packages { + packageCode := strings.TrimSpace(item.PackageCode) + if packageCode == "" { + continue + } + if _, ok := seen[packageCode]; ok { + continue + } + seen[packageCode] = struct{}{} + rows = append(rows, g.Map{ + "package_code": packageCode, + "package_name": strings.TrimSpace(item.PackageName), + "dept_id": dept.DeptId, + "dept_name": strings.TrimSpace(dept.DeptName), + "tenant_id": tenantId, + "created_dept": dept.DeptId, + "created_by": userId, + "created_at": now, + "updated_by": userId, + "updated_at": now, + }) + } + if len(rows) == 0 { + return nil + } + if _, err := tx.Ctx(ctx).Model(packageCustomerTable).Safe().Data(rows).Insert(); err != nil { + return gerror.Wrap(err, "insert dept package customer relation failed") + } + return nil +} + +func deptIsTop(dept *deptRow) bool { + if dept == nil { + return false + } + return dept.ParentId == 0 || strings.Trim(dept.Ancestors, ",") == "0" +} diff --git a/internal/service/renewchange/change.go b/internal/service/renewchange/change.go new file mode 100644 index 0000000..8899ad5 --- /dev/null +++ b/internal/service/renewchange/change.go @@ -0,0 +1,505 @@ +package renewchange + +import ( + "context" + "fmt" + "strings" + "time" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/packagecustomer" +) + +// ChangeExecutor executes a confirmed batch_type=102 change as one business +// transaction. The transaction includes old subscription cleanup, the new +// order/subscriptions, card reset, detail success, and batch success. +type ChangeExecutor struct { + repo *Repository + nowFunc NowFunc +} + +func NewChangeExecutor(repo *Repository) *ChangeExecutor { + if repo == nil { + repo = NewRepository() + } + return &ChangeExecutor{repo: repo, nowFunc: timeNow} +} + +func (c *ChangeExecutor) SetNowFunc(now NowFunc) { c.nowFunc = now } + +func (c *ChangeExecutor) Execute(ctx context.Context, batchNo string) error { + batchNo = strings.TrimSpace(batchNo) + if batchNo == "" { + return gerror.New("change batch number is empty") + } + main, err := c.repo.LoadProcessingBatch(ctx, batchNo, consts.BatchTypeChange) + if err != nil { + return err + } + details, err := c.repo.LoadRunningDetails(ctx, batchNo, consts.BatchTypeChange) + if err != nil { + return gerror.Wrap(err, "load change details failed") + } + if len(details) == 0 { + return c.finalizeFailure(ctx, main, "change batch has no runnable details") + } + contract, err := ParseExecutionContract(main) + if err != nil { + return c.finalizeFailure(ctx, main, err.Error()) + } + if contract.BatchType != consts.BatchTypeChange { + return c.finalizeFailure(ctx, main, "change execution contract has invalid batch type") + } + + now := pnow(c.nowFunc) + orderSN, skipped, err := c.apply(ctx, main, contract, details, now) + if err != nil { + return c.finalizeFailure(ctx, main, err.Error()) + } + if skipped { + return nil + } + packagecustomer.PublishSyncForDeptsSafe(ctx, main.TenantId, main.CreatedBy, []int64{contract.CustomerAgentID}, "change", main.BatchNo) + + rows, snapshotErr := c.repo.ResolveResultSnapshot(ctx, main.BatchNo, main.BatchType) + if snapshotErr == nil { + _ = c.repo.WriteResultFileSafe(ctx, main.BatchNo, main.BatchType, main.TenantId, main.CreatedDept, main.CreatedBy, rows) + } + g.Log().Infof(ctx, "change batch success, batchNo:%s, orderSn:%s, tenant:%s", main.BatchNo, orderSN, main.TenantId) + return nil +} + +// apply is deliberately the only success transaction. A duplicate delivery +// that reaches this method after a completed transaction is safely ignored. +func (c *ChangeExecutor) apply(ctx context.Context, main *rc.BatchMainRow, contract *ExecutionContract, details []*rc.BatchDetailRow, now time.Time) (orderSN string, skipped bool, err error) { + iccids := uniqueIccidsFromDetails(details) + if len(iccids) != len(details) { + return "", false, gerror.New("change batch contains duplicate or empty ICCID") + } + if contract.Quantity != len(iccids) { + return "", false, gerror.Newf("change contract quantity mismatch: contract=%d actual=%d", contract.Quantity, len(iccids)) + } + + err = c.repo.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + lockedMain, lockErr := c.repo.Tables().LockChangeBatch(ctx, tx, main.BatchNo) + if lockErr != nil { + return lockErr + } + if lockedMain.BatchAsyncStatus != consts.BatchAsyncStatusProcessing { + skipped = true + return nil + } + lockedDetails, lockErr := c.repo.Tables().LockChangeDetails(ctx, tx, main.BatchNo) + if lockErr != nil { + return lockErr + } + if err = validateRunningChangeDetails(lockedDetails, details); err != nil { + return err + } + + cards, lockErr := c.repo.Tables().LockCardsByIccids(ctx, tx, iccids) + if lockErr != nil { + return lockErr + } + if len(cards) != len(iccids) { + return gerror.New("some change cards no longer exist") + } + bindRows, lockErr := c.repo.Tables().LockChangeBinds(ctx, tx, iccids) + if lockErr != nil { + return lockErr + } + currentBinds, futureBinds, lockErr := resolveCurrentSubscriptions(bindRows, iccids) + if lockErr != nil { + return lockErr + } + + oldPackageCode := readChangeOldPackageCode(main.BatchInfo) + if oldPackageCode == "" { + return gerror.New("change contract old package is missing") + } + newPackage, loadErr := c.repo.Tables().LoadChangePackage(ctx, tx, contract.TenantID, contract.PackageCode) + if loadErr != nil { + return loadErr + } + oldPackage, loadErr := c.repo.Tables().LoadChangePackage(ctx, tx, contract.TenantID, oldPackageCode) + if loadErr != nil { + return loadErr + } + if newPackage.Enabled != 1 || oldPackage.Enabled != 1 { + return gerror.New("change package is disabled") + } + if newPackage.PackageType != oldPackage.PackageType { + return gerror.New("new package type differs from current package") + } + if !containsPackageCode(oldPackage.ChangeablePackageCodes, contract.PackageCode) { + return gerror.New("new package is not in current package changeable range") + } + durationOptions, durationErr := changeDurationOptions(newPackage.RenewableDurationConfig) + if durationErr != nil { + return gerror.Wrap(durationErr, "load target package change duration options failed") + } + if !rc.ContainsInt(durationOptions, contract.DurationMonths) { + return gerror.Newf("change duration %d months is not allowed for target package", contract.DurationMonths) + } + customer, loadErr := c.repo.Tables().LoadChangeDept(ctx, tx, contract.TenantID, contract.CustomerAgentID) + if loadErr != nil { + return loadErr + } + if customer.DeptId != contract.SellerAgentID && customer.ParentId != contract.SellerAgentID { + return gerror.New("change customer is outside the operator scope") + } + + sourceBatchNo := contract.SourceBatchNo + for _, iccid := range iccids { + card := cards[iccid] + current := currentBinds[iccid] + var expectedMSISDN, expectedIMSI string + if detail := findDetailByICCID(lockedDetails, iccid); detail != nil && detail.Snapshot != nil { + expectedMSISDN = strings.TrimSpace(detail.Snapshot.Get("msisdn").String()) + expectedIMSI = strings.TrimSpace(detail.Snapshot.Get("imsi").String()) + } + if err = validateChangeCard(card, current, contract, iccid, oldPackageCode, sourceBatchNo, expectedMSISDN, expectedIMSI); err != nil { + return gerror.Wrapf(err, "validate change card failed, iccid:%s", iccid) + } + for _, future := range futureBinds[iccid] { + if future == nil || future.TenantId != contract.TenantID || future.DeptId != contract.CustomerAgentID { + return gerror.Newf("future change subscription is inconsistent: %s", iccid) + } + } + } + + currentList := make([]*rc.SimCardBindPkg, 0, len(iccids)) + currentOrderCodes := make([]string, 0, len(iccids)) + currentOrderQuantities := make(map[string]int) + for _, iccid := range iccids { + currentList = append(currentList, currentBinds[iccid]) + if orderCode := strings.TrimSpace(currentBinds[iccid].OrderCode); orderCode != "" { + currentOrderCodes = append(currentOrderCodes, orderCode) + currentOrderQuantities[orderCode]++ + } + } + if err = c.repo.Tables().ArchiveChangeBinds(ctx, tx, currentList, consts.EffectiveStatusTerminated, gtime.NewFromTime(now), main.BatchNo, contract.OperatorUserID); err != nil { + return err + } + currentSubscriptionNos := subscriptionNos(currentList) + if affected, deleteErr := c.repo.Tables().DeleteChangeBinds(ctx, tx, currentSubscriptionNos); deleteErr != nil { + return deleteErr + } else if int(affected) != len(currentSubscriptionNos) { + return gerror.New("current change subscriptions changed concurrently") + } + if err = c.repo.Tables().UpdateChangeOrdersAfterRemoval(ctx, tx, currentOrderCodes, main.BatchNo, "lastUnsubscribe", currentOrderQuantities, gtime.NewFromTime(now), contract.OperatorUserID); err != nil { + return err + } + + futureList := flattenFutureBinds(futureBinds, iccids) + futureOrderCodes := make([]string, 0, len(futureList)) + futureOrderQuantities := make(map[string]int) + for _, bind := range futureList { + if orderCode := strings.TrimSpace(bind.OrderCode); orderCode != "" { + futureOrderCodes = append(futureOrderCodes, orderCode) + futureOrderQuantities[orderCode]++ + } + } + if err = c.repo.Tables().ArchiveChangeBinds(ctx, tx, futureList, consts.EffectiveStatusVoided, gtime.NewFromTime(now), main.BatchNo, contract.OperatorUserID); err != nil { + return err + } + futureSubscriptionNos := subscriptionNos(futureList) + if affected, deleteErr := c.repo.Tables().DeleteChangeBinds(ctx, tx, futureSubscriptionNos); deleteErr != nil { + return deleteErr + } else if int(affected) != len(futureSubscriptionNos) { + return gerror.New("future change subscriptions changed concurrently") + } + if err = c.repo.Tables().UpdateChangeOrdersAfterRemoval(ctx, tx, futureOrderCodes, main.BatchNo, "lastChange", futureOrderQuantities, gtime.NewFromTime(now), contract.OperatorUserID); err != nil { + return err + } + + beginAt, resolveErr := resolveChangeBeginAt(contract, now) + if resolveErr != nil { + return resolveErr + } + endAt := contract.ServiceEnd(beginAt) + packageStatus := calculateChangePackageStatus(now, beginAt, endAt) + effectiveStatus := calculateChangeEffectiveStatus(now, beginAt, endAt) + orderSN, err = rc.NextOrderSN(now) + if err != nil { + return err + } + order, buildErr := BuildOrderDO(contract, orderSN, now) + if buildErr != nil { + return buildErr + } + orderID, insertErr := c.repo.Tables().InsertContractOrder(ctx, tx, order) + if insertErr != nil { + return gerror.Wrap(insertErr, "insert change order failed") + } + item, buildErr := BuildOrderItemDO(contract, OrderProductContract{ID: newPackage.Id, Code: newPackage.PackageCode, Name: newPackage.PackageName, Type: newPackage.PackageType}, orderID, orderSN, beginAt, endAt, now) + if buildErr != nil { + return buildErr + } + if err = c.repo.Tables().InsertContractOrderItem(ctx, tx, item); err != nil { + return gerror.Wrap(err, "insert change order item failed") + } + + snapshots := make(map[int]*gjson.Json, len(lockedDetails)) + for _, detail := range lockedDetails { + iccid := strings.TrimSpace(detail.ItemId) + card := cards[iccid] + newSubscriptionNo := rc.GenerateSubscriptionNo("SUB") + newBind := buildChangeBind(contract, newPackage, card, main.BatchNo, oldPackageCode, orderSN, newSubscriptionNo, effectiveStatus, beginAt, endAt, now) + if err = c.repo.Tables().InsertBind(ctx, tx, newBind); err != nil { + return gerror.Wrap(err, "insert change subscription failed") + } + if err = c.repo.Tables().UpdateChangeCard(ctx, tx, iccid, newPackage.PackageCode, packageStatus, newPackage.FlowMb, gtime.NewFromTime(beginAt), gtime.NewFromTime(endAt), contract.OperatorUserID); err != nil { + return err + } + snapshots[detail.RowNo] = buildChangeDetailSnapshot(detail, card, currentBinds[iccid], newSubscriptionNo, futureBinds[iccid], contract, oldPackageCode, orderSN, beginAt, endAt) + } + if err = c.repo.Tables().MarkChangeDetailsSuccess(ctx, tx, main.BatchNo, lockedDetails, snapshots, contract.OperatorUserID); err != nil { + return err + } + info := mergeBatchInfo(main.BatchInfo, map[string]any{ + "orderSn": orderSN, + "oldPackageCode": oldPackageCode, + "newPackageCode": newPackage.PackageCode, + "newPackageName": newPackage.PackageName, + "durationMonths": contract.DurationMonths, + "giftDurationMonths": contract.GiftDurationMonths, + "effectiveMode": contract.EffectiveMode, + "packageBeginTime": formatTime(gtime.NewFromTime(beginAt)), + "packageEndTime": formatTime(gtime.NewFromTime(endAt)), + "amount": contract.TotalAmount(), + "changeTime": formatTime(gtime.NewFromTime(now)), + }) + return c.repo.UpdateBatchMainInfo(ctx, tx, main.BatchNo, main.BatchType, buildChangeSuccessBatchMain(main, contract, gjson.New(info), orderSN, lockedMain.StartTime, now, len(lockedDetails))) + }) + return orderSN, skipped, err +} + +func buildChangeBind(contract *ExecutionContract, pkg *rc.FlowPackageInfo, card *rc.SimCardInfo, batchNo, oldPackageCode, orderSN, subscriptionNo string, effectiveStatus int, beginAt, endAt, now time.Time) *rc.SimCardBindPkg { + return &rc.SimCardBindPkg{ + BatchNo: batchNo, + Iccid: card.Iccid, + Msisdn: card.Msisdn, + DeptId: contract.CustomerAgentID, + PackageType: pkg.PackageType, + PackageCode: pkg.PackageCode, + PreviousPackageCode: oldPackageCode, + BusinessType: consts.PackageBusinessTypeChange, + EffectiveStatus: effectiveStatus, + HandleTime: gtime.NewFromTime(now), + EffectiveTime: gtime.NewFromTime(beginAt), + ExpectedExpireTime: gtime.NewFromTime(endAt), + DurationMonths: contract.DurationMonths, + GiftDurationMonths: contract.GiftDurationMonths, + OrderCode: orderSN, + SubscriptionNo: subscriptionNo, + Remark: "卡变更套餐自动生成", + TenantId: contract.TenantID, + CreatedDept: contract.CreatedDeptID, + CreatedBy: contract.OperatorUserID, + CreatedAt: gtime.NewFromTime(now), + UpdatedBy: contract.OperatorUserID, + UpdatedAt: gtime.NewFromTime(now), + } +} + +func buildChangeSuccessBatchMain(main *rc.BatchMainRow, contract *ExecutionContract, info *gjson.Json, orderSN string, startTime *gtime.Time, now time.Time, validCount int) *rc.BatchMainRow { + return &rc.BatchMainRow{ + BatchType: main.BatchType, + BatchAsyncStatus: consts.BatchAsyncStatusSuccess, + RunResult: consts.BatchRunResultSuccess, + TotalCount: main.TotalCount, + ValidCount: validCount, + SuccessCount: validCount, + FailCount: 0, + BatchInfo: info, + OrderSn: orderSN, + StartTime: startTime, + EndTime: gtime.NewFromTime(now), + TargetDeptId: contract.CustomerAgentID, + UpdatedBy: contract.OperatorUserID, + } +} + +func (c *ChangeExecutor) finalizeFailure(ctx context.Context, main *rc.BatchMainRow, reason string) error { + now := pnow(c.nowFunc) + err := c.repo.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + if err := c.repo.MarkAllDetailsFailed(ctx, tx, main.BatchNo, main.BatchType, reason, main.CreatedBy); err != nil { + return err + } + return c.repo.FinalizeBatchMain(ctx, tx, main.BatchNo, main.BatchType, &rc.BatchMainRow{ + BatchAsyncStatus: consts.BatchAsyncStatusFailed, + RunResult: consts.BatchRunResultFailed, + FailReason: reason, + TotalCount: main.TotalCount, + FailCount: main.TotalCount, + EndTime: gtime.NewFromTime(now), + UpdatedBy: main.CreatedBy, + }) + }) + if err != nil { + return gerror.Wrap(err, "finalize change failure failed") + } + g.Log().Warningf(ctx, "change batch failed, batchNo:%s, reason:%s, tenant:%s", main.BatchNo, reason, main.TenantId) + return nil +} + +func validateRunningChangeDetails(locked, loaded []*rc.BatchDetailRow) error { + if len(locked) != len(loaded) { + return gerror.New("change detail count changed during execution") + } + for _, detail := range locked { + if detail == nil || detail.DetailStatus != consts.BatchDetailStatusRunning { + return gerror.New("change detail is no longer running") + } + } + return nil +} + +func validateChangeCard(card *rc.SimCardInfo, bind *rc.SimCardBindPkg, contract *ExecutionContract, iccid, oldPackageCode, sourceBatchNo, expectedMSISDN, expectedIMSI string) error { + if contract == nil { + return gerror.New("change execution contract is missing") + } + return validateCurrentSubscription(card, bind, subscriptionValidation{ + ICCID: iccid, + MSISDN: expectedMSISDN, + IMSI: expectedIMSI, + TenantID: contract.TenantID, + CustomerDeptID: contract.CustomerAgentID, + PackageCode: oldPackageCode, + SourceBatchNo: sourceBatchNo, + }) +} + +func findDetailByICCID(details []*rc.BatchDetailRow, iccid string) *rc.BatchDetailRow { + for _, detail := range details { + if detail != nil && strings.TrimSpace(detail.ItemId) == strings.TrimSpace(iccid) { + return detail + } + } + return nil +} + +func readChangeOldPackageCode(info *gjson.Json) string { + return firstString(info, "oldPackageCode", "changePrepareSnapshot.oldPackageCode") +} + +func containsPackageCode(values *gjson.Json, target string) bool { + if values == nil || values.IsNil() { + return false + } + for _, value := range values.Interfaces() { + if strings.TrimSpace(gconvString(value)) == strings.TrimSpace(target) { + return true + } + } + return false +} + +func resolveChangeBeginAt(contract *ExecutionContract, now time.Time) (time.Time, error) { + if contract == nil || contract.PackageBeginTime == nil { + return time.Time{}, gerror.New("change package begin time is missing") + } + switch strings.TrimSpace(contract.EffectiveMode) { + case "immediate": + return now, nil + case "next_month": + begin := contract.PackageBeginTime.Time + if begin.Day() != 1 || begin.Hour() != 0 || begin.Minute() != 0 || begin.Second() != 0 { + return time.Time{}, gerror.New("change next month begin time is invalid") + } + return begin, nil + default: + return time.Time{}, gerror.New("change effective mode is unsupported") + } +} + +func calculateChangePackageStatus(now, begin, end time.Time) int { + if begin.After(now) { + return consts.PackageStatusPending + } + if !end.After(now) { + return consts.PackageStatusExpired + } + return consts.PackageStatusEffective +} + +func calculateChangeEffectiveStatus(now, begin, end time.Time) int { + if begin.After(now) { + return consts.EffectiveStatusPending + } + if !end.After(now) { + return consts.EffectiveStatusExpired + } + return consts.EffectiveStatusEffective +} + +func subscriptionNos(rows []*rc.SimCardBindPkg) []string { + result := make([]string, 0, len(rows)) + for _, row := range rows { + if row != nil { + result = append(result, strings.TrimSpace(row.SubscriptionNo)) + } + } + return result +} + +func flattenFutureBinds(future map[string][]*rc.SimCardBindPkg, iccids []string) []*rc.SimCardBindPkg { + result := make([]*rc.SimCardBindPkg, 0) + for _, iccid := range iccids { + result = append(result, future[iccid]...) + } + return result +} + +func buildChangeDetailSnapshot(detail *rc.BatchDetailRow, card *rc.SimCardInfo, oldBind *rc.SimCardBindPkg, newSubscriptionNo string, voided []*rc.SimCardBindPkg, contract *ExecutionContract, oldPackageCode, orderSN string, beginAt, endAt time.Time) *gjson.Json { + data := map[string]any{} + if detail != nil && detail.Snapshot != nil { + for key, value := range detail.Snapshot.Map() { + data[key] = value + } + } + data["ICCID"] = card.Iccid + data["MSISDN"] = card.Msisdn + data["IMSI"] = card.Imsi + data["iccid"] = card.Iccid + data["msisdn"] = card.Msisdn + data["imsi"] = card.Imsi + data["oldPackageCode"] = oldPackageCode + data["newPackageCode"] = contract.PackageCode + data["newPackageName"] = contract.PackageName + data["oldSubscriptionNo"] = oldBind.SubscriptionNo + data["oldOrderSn"] = oldBind.OrderCode + data["newSubscriptionNo"] = newSubscriptionNo + data["newOrderSn"] = orderSN + data["durationMonths"] = contract.DurationMonths + data["giftDurationMonths"] = contract.GiftDurationMonths + data["effectiveMode"] = contract.EffectiveMode + data["changeUnitPrice"] = contract.UnitPrice + data["amount"] = roundAmount(contract.UnitPrice * float64(contract.DurationMonths)) + data["newEffectiveTime"] = formatTime(gtime.NewFromTime(beginAt)) + data["newExpectedExpireTime"] = formatTime(gtime.NewFromTime(endAt)) + data["oldActualExpireTime"] = formatTime(gtime.Now()) + voidedNos := make([]string, 0, len(voided)) + for _, row := range voided { + if row != nil { + voidedNos = append(voidedNos, row.SubscriptionNo) + } + } + data["voidedSubscriptionNos"] = voidedNos + return gjson.New(data) +} + +func gconvString(value any) string { + if value == nil { + return "" + } + return strings.TrimSpace(fmt.Sprint(value)) +} diff --git a/internal/service/renewchange/change_test.go b/internal/service/renewchange/change_test.go new file mode 100644 index 0000000..a6a5b4e --- /dev/null +++ b/internal/service/renewchange/change_test.go @@ -0,0 +1,115 @@ +package renewchange + +import ( + "testing" + "time" + + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" +) + +func TestResolveCurrentSubscriptionsSelectsEffectiveAndFuturePending(t *testing.T) { + rows := []*rc.SimCardBindPkg{ + {Iccid: "8986001", SubscriptionNo: "SUB-E", EffectiveStatus: consts.EffectiveStatusEffective}, + {Iccid: "8986001", SubscriptionNo: "SUB-P", EffectiveStatus: consts.EffectiveStatusPending}, + } + current, future, err := resolveCurrentSubscriptions(rows, []string{"8986001"}) + if err != nil { + t.Fatalf("resolveCurrentSubscriptions() error = %v", err) + } + if current["8986001"].SubscriptionNo != "SUB-E" { + t.Fatalf("current subscription = %q, want SUB-E", current["8986001"].SubscriptionNo) + } + if len(future["8986001"]) != 1 || future["8986001"][0].SubscriptionNo != "SUB-P" { + t.Fatalf("future subscriptions = %+v, want SUB-P", future["8986001"]) + } +} + +func TestResolveCurrentSubscriptionsRejectsDuplicateCurrentSubscriptions(t *testing.T) { + rows := []*rc.SimCardBindPkg{ + {Iccid: "8986001", EffectiveStatus: consts.EffectiveStatusEffective}, + {Iccid: "8986001", EffectiveStatus: consts.EffectiveStatusEffective}, + } + if _, _, err := resolveCurrentSubscriptions(rows, []string{"8986001"}); err == nil { + t.Fatal("resolveCurrentSubscriptions() error = nil, want duplicate-current error") + } +} + +func TestResolveChangeBeginAtUsesConfirmedNextMonthContract(t *testing.T) { + now := time.Date(2026, 8, 3, 10, 2, 0, 0, time.Local) + confirmed := time.Date(2026, 9, 1, 0, 0, 0, 0, time.Local) + contract := &ExecutionContract{ + EffectiveMode: "next_month", + PackageBeginTime: gtime.NewFromTime(confirmed), + } + got, err := resolveChangeBeginAt(contract, now) + if err != nil { + t.Fatalf("resolveChangeBeginAt() error = %v", err) + } + if !got.Equal(confirmed) { + t.Fatalf("begin time = %v, want %v", got, confirmed) + } +} + +func TestResolveChangeBeginAtUsesExecutionTimeForImmediate(t *testing.T) { + now := time.Date(2026, 8, 3, 10, 2, 0, 0, time.Local) + contract := &ExecutionContract{ + EffectiveMode: "immediate", + PackageBeginTime: gtime.NewFromTime(now.Add(-time.Minute)), + } + got, err := resolveChangeBeginAt(contract, now) + if err != nil { + t.Fatalf("resolveChangeBeginAt() error = %v", err) + } + if !got.Equal(now) { + t.Fatalf("begin time = %v, want %v", got, now) + } +} + +func TestChangeAmountAndServiceEndUseConfirmedDurationAndGift(t *testing.T) { + contract := &ExecutionContract{UnitPrice: 8.5, Quantity: 2, DurationMonths: 12, GiftDurationMonths: 2} + if got := contract.TotalAmount(); got != 204 { + t.Fatalf("TotalAmount() = %v, want 204", got) + } + begin := time.Date(2026, 8, 3, 10, 2, 0, 0, time.Local) + wantEnd := begin.AddDate(0, 14, 0) + if got := contract.ServiceEnd(begin); !got.Equal(wantEnd) { + t.Fatalf("ServiceEnd() = %v, want %v", got, wantEnd) + } +} + +func TestBuildChangeBindAndSuccessBatchKeepConfirmedContractAndAuditFields(t *testing.T) { + contract := validExecutionContract() + contract.BatchType = consts.BatchTypeChange + contract.UnitPrice = 8.5 + contract.Quantity = 2 + contract.DurationMonths = 12 + contract.GiftDurationMonths = 2 + now := time.Date(2026, 8, 3, 10, 2, 0, 0, time.Local) + begin := now.AddDate(0, 1, 0) + end := contract.ServiceEnd(begin) + pkg := &rc.FlowPackageInfo{Id: 300, PackageCode: contract.PackageCode, PackageName: "Changed package", PackageType: 1} + card := &rc.SimCardInfo{Iccid: "8986001", Msisdn: "13800000000"} + + bind := buildChangeBind(contract, pkg, card, contract.BatchNo, "PKG-OLD", "SO-NEW", "SUB-NEW", consts.EffectiveStatusPending, begin, end, now) + if bind.CreatedAt == nil || bind.UpdatedAt == nil || !bind.CreatedAt.Time.Equal(now) || !bind.UpdatedAt.Time.Equal(now) { + t.Fatalf("subscription audit fields = %v/%v, want %v", bind.CreatedAt, bind.UpdatedAt, now) + } + if bind.DurationMonths != 12 || bind.GiftDurationMonths != 2 || bind.PackageCode != contract.PackageCode { + t.Fatalf("subscription contract = package %q, duration %d+%d", bind.PackageCode, bind.DurationMonths, bind.GiftDurationMonths) + } + if bind.BusinessType != consts.PackageBusinessTypeChange { + t.Fatalf("subscription business type = %d, want %d", bind.BusinessType, consts.PackageBusinessTypeChange) + } + if !bind.EffectiveTime.Time.Equal(begin) || !bind.ExpectedExpireTime.Time.Equal(end) { + t.Fatalf("subscription service period = %v/%v, want %v/%v", bind.EffectiveTime, bind.ExpectedExpireTime, begin, end) + } + + main := &rc.BatchMainRow{BatchType: consts.BatchTypeChange, TotalCount: 2, StartTime: gtime.NewFromTime(now.Add(-time.Minute))} + batch := buildChangeSuccessBatchMain(main, contract, nil, "SO-NEW", main.StartTime, now, 2) + if batch.TargetDeptId != contract.CustomerAgentID { + t.Fatalf("target_dept_id = %d, want %d", batch.TargetDeptId, contract.CustomerAgentID) + } +} diff --git a/internal/service/renewchange/helpers.go b/internal/service/renewchange/helpers.go new file mode 100644 index 0000000..ed5299f --- /dev/null +++ b/internal/service/renewchange/helpers.go @@ -0,0 +1,9 @@ +package renewchange + +// roundAmount 四舍五入到分。 +func roundAmount(value float64) float64 { + if value < 0 { + return 0 + } + return float64(int(value*100+0.5)) / 100 +} diff --git a/internal/service/renewchange/order_contract.go b/internal/service/renewchange/order_contract.go new file mode 100644 index 0000000..4feeb83 --- /dev/null +++ b/internal/service/renewchange/order_contract.go @@ -0,0 +1,660 @@ +package renewchange + +import ( + "encoding/json" + "fmt" + "strings" + "time" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" +) + +// ExecutionContract is the confirmed renew/change contract persisted in +// batch_main.batch_info. DurationMonths is the actual persisted service +// duration in months; DurationDisplayValue is the value selected by the user. +type ExecutionContract struct { + BatchNo string + BatchType int + SourceBatchNo string + TenantID string + CustomerAgentID int64 + SellerAgentID int64 + CreatedDeptID int64 + OperatorUserID int64 + PackageCode string + PackageName string + PriceSource string + AgentDefaultPriceID int64 + TargetPriceID int64 + UnitPrice float64 + Quantity int + BillingMethod int + RenewDurationUnit string + RenewDurationUnitText string + DurationDisplayValue int + DurationMonths int + GiftDurationMonths int + OfflineContractCompleted bool + PaymentMode string + PayType int + PayStatus int + PayAmount float64 + BalanceAmount float64 + OfflinePaidAmount float64 + BalanceDetailSerialNo string + PaymentTime string + PayerDeptID int64 + PayerUserID int64 + BuyerNote string + AdminNote string + Remark string + EffectiveMode string + RenewDate string + PackageBeginTime *gtime.Time +} + +type OrderProductContract struct { + ID int64 + Code string + Name string + Type int +} + +// ParseExecutionContract reads the canonical top-level contract and falls +// back to the prepare snapshots only for batches created before the flat +// contract migration. +func ParseExecutionContract(main *rc.BatchMainRow) (*ExecutionContract, error) { + if main == nil { + return nil, gerror.New("batch main cannot be nil") + } + if main.BatchType != consts.BatchTypeRenew && main.BatchType != consts.BatchTypeChange { + return nil, gerror.Newf("unsupported renew/change batch type: %d", main.BatchType) + } + if main.BatchInfo == nil || main.BatchInfo.IsNil() { + return nil, gerror.New("confirmed batch contract is missing") + } + + var snapshotPath, packageCodeField, packageNameField, unitPriceField string + if main.BatchType == consts.BatchTypeRenew { + snapshotPath = "priceSnapshot" + packageCodeField = "packageCode" + packageNameField = "packageName" + unitPriceField = "renewUnitPrice" + } else { + snapshotPath = "changePriceSnapshot" + packageCodeField = "newPackageCode" + packageNameField = "newPackageName" + unitPriceField = "changeUnitPrice" + } + + info := main.BatchInfo + duration := executionDurationFromInfo(info, snapshotPath) + if main.BatchType == consts.BatchTypeRenew { + duration = normalizeRenewExecutionDuration(info, duration) + } + contract := &ExecutionContract{ + BatchNo: strings.TrimSpace(main.BatchNo), + BatchType: main.BatchType, + SourceBatchNo: executionSourceBatchNo(main, info), + TenantID: strings.TrimSpace(main.TenantId), + CustomerAgentID: firstInt64(info, "customerAgentId", snapshotPath+".customerAgentId"), + SellerAgentID: main.DeptId, + CreatedDeptID: main.CreatedDept, + OperatorUserID: main.CreatedBy, + PackageCode: firstString(info, packageCodeField, snapshotPath+"."+packageCodeField), + PackageName: firstString(info, packageNameField, snapshotPath+"."+packageNameField), + PriceSource: firstString(info, "priceSource", snapshotPath+".priceSource"), + AgentDefaultPriceID: firstInt64(info, "agentDefaultPriceId", snapshotPath+".agentDefaultPriceId"), + TargetPriceID: firstInt64(info, "targetPriceId", snapshotPath+".targetPriceId"), + Quantity: firstInt(info, "quantity"), + BillingMethod: duration.billingMethod, + RenewDurationUnit: duration.unit, + RenewDurationUnitText: duration.unitText, + DurationDisplayValue: duration.displayValue, + DurationMonths: duration.actualMonths, + GiftDurationMonths: duration.giftMonths, + OfflineContractCompleted: info.Get("offlineContractCompleted").Bool(), + PaymentMode: firstString(info, "paymentMode", "paymentSnapshot.paymentMode"), + PayType: firstInt(info, "payType", "paymentSnapshot.payType"), + PayStatus: firstInt(info, "payStatus", "paymentSnapshot.payStatus"), + PayAmount: firstAmount(info, "payAmount", "paymentSnapshot.payAmount"), + BalanceAmount: firstAmount(info, "balanceAmount", "paymentSnapshot.balanceAmount"), + OfflinePaidAmount: firstAmount(info, "offlinePaidAmount", "paymentSnapshot.offlinePaidAmount"), + BalanceDetailSerialNo: firstString(info, "balanceDetailSerialNo", "paymentSnapshot.balanceDetailSerialNo"), + PaymentTime: firstString(info, "paymentTime", "paymentSnapshot.paymentTime"), + PayerDeptID: firstInt64(info, "payerDeptId", "paymentSnapshot.payerDeptId"), + PayerUserID: firstInt64(info, "payerUserId", "paymentSnapshot.payerUserId"), + BuyerNote: strings.TrimSpace(info.Get("buyerNote").String()), + AdminNote: strings.TrimSpace(info.Get("adminNote").String()), + Remark: strings.TrimSpace(info.Get("remark").String()), + EffectiveMode: strings.TrimSpace(info.Get("effectiveMode").String()), + RenewDate: strings.TrimSpace(info.Get("renewDate").String()), + } + if value, ok := firstFloat64(info, unitPriceField, snapshotPath+"."+unitPriceField); ok { + contract.UnitPrice = value + } else { + contract.UnitPrice = -1 + } + if contract.SellerAgentID <= 0 { + contract.SellerAgentID = main.CreatedDept + } + if contract.CreatedDeptID <= 0 { + contract.CreatedDeptID = contract.SellerAgentID + } + contract.normalizePaymentContract() + if raw := strings.TrimSpace(info.Get("packageBeginTime").String()); raw != "" { + beginAt, err := parseExecutionTime(raw) + if err != nil { + return nil, gerror.Wrap(err, "parse package begin time failed") + } + contract.PackageBeginTime = beginAt + } + if err := contract.Validate(); err != nil { + return nil, err + } + return contract, nil +} + +func executionSourceBatchNo(main *rc.BatchMainRow, info *gjson.Json) string { + if main == nil || main.SourceType != consts.BatchSourceTypeBatchSelect { + return "" + } + sourceBatchNo := firstString(info, "sourceBatchNo", "changePrepareSnapshot.sourceBatchNo") + if sourceBatchNo == "" || sourceBatchNo == strings.TrimSpace(main.BatchNo) { + sourceBatchNo = strings.TrimSpace(main.SourceBatchNo) + } + if sourceBatchNo == strings.TrimSpace(main.BatchNo) { + return "" + } + return sourceBatchNo +} + +type executionDurationContract struct { + billingMethod int + unit string + unitText string + displayValue int + actualMonths int + giftMonths int +} + +func executionDurationFromInfo(info *gjson.Json, snapshotPath string) executionDurationContract { + rawDurationMonths := firstInt(info, "durationMonths") + displayValue := firstInt(info, "durationValue", "durationYears") + hasDisplayValue := hasJSONValue(info, "durationValue") || hasJSONValue(info, "durationYears") + if displayValue <= 0 { + displayValue = rawDurationMonths + } + + billingMethod := firstInt(info, "billingMethod", snapshotPath+".billingMethod") + unit := firstString(info, "renewDurationUnit", snapshotPath+".renewDurationUnit") + if strings.TrimSpace(unit) == "" { + if hasDisplayValue { + unit = renewDurationUnitForBillingMethod(billingMethod) + } else { + unit = consts.RenewDurationUnitMonth + } + } + unit = consts.NormalizeRenewDurationUnit(unit) + unitText := firstString(info, "renewDurationUnitText", snapshotPath+".renewDurationUnitText") + if strings.TrimSpace(unitText) == "" { + unitText = consts.RenewDurationUnitText(unit) + } + + actualMonths := firstInt(info, "actualDurationMonths") + if actualMonths <= 0 { + if hasDisplayValue { + actualMonths = consts.DurationValueToMonths(unit, displayValue) + } else { + actualMonths = rawDurationMonths + } + } + giftMonths := firstInt(info, "actualGiftDurationMonths") + if giftMonths <= 0 { + giftMonths = firstInt(info, "giftDurationMonths") + } + if hasDisplayValue && giftMonths > 0 && firstInt(info, "actualGiftDurationMonths") <= 0 { + giftMonths = consts.DurationValueToMonths(unit, giftMonths) + } + return executionDurationContract{ + billingMethod: billingMethod, + unit: unit, + unitText: unitText, + displayValue: displayValue, + actualMonths: actualMonths, + giftMonths: giftMonths, + } +} + +func normalizeRenewExecutionDuration(info *gjson.Json, duration executionDurationContract) executionDurationContract { + displayValue := firstInt(info, "durationValue", "durationYears") + if displayValue <= 0 { + displayValue = normalizeLegacyRenewDurationValue(firstInt(info, "durationMonths")) + } + if displayValue <= 0 { + return duration + } + duration.unit = consts.RenewDurationUnitYear + duration.unitText = consts.RenewDurationUnitText(duration.unit) + duration.displayValue = displayValue + duration.actualMonths = consts.DurationValueToMonths(duration.unit, displayValue) + return duration +} + +func (c *ExecutionContract) Validate() error { + if c == nil { + return gerror.New("execution contract cannot be nil") + } + if strings.TrimSpace(c.BatchNo) == "" { + return gerror.New("execution contract batch number is missing") + } + if c.BatchType != consts.BatchTypeRenew && c.BatchType != consts.BatchTypeChange { + return gerror.New("execution contract batch type is invalid") + } + if strings.TrimSpace(c.TenantID) == "" { + return gerror.New("execution contract tenant id is missing") + } + if c.CustomerAgentID <= 0 || c.SellerAgentID <= 0 || c.CreatedDeptID <= 0 || c.OperatorUserID <= 0 { + return gerror.New("execution contract organization or operator is incomplete") + } + if strings.TrimSpace(c.PackageCode) == "" || strings.TrimSpace(c.PackageName) == "" { + return gerror.New("execution contract package is incomplete") + } + if c.UnitPrice < 0 { + return gerror.New("execution contract unit price is missing") + } + if c.Quantity <= 0 { + return gerror.New("execution contract quantity must be greater than zero") + } + if c.DurationDisplayValue <= 0 { + c.DurationDisplayValue = c.DurationMonths + } + if c.DurationMonths <= 0 { + return gerror.New("execution contract duration months must be greater than zero") + } + if c.GiftDurationMonths < 0 { + return gerror.New("execution contract gift duration months cannot be negative") + } + c.normalizePaymentContract() + switch strings.TrimSpace(c.PaymentMode) { + case "offline": + if !c.OfflineContractCompleted { + return gerror.New("execution contract offline confirmation is missing") + } + case "balance": + if c.PayType != consts.PayTypeBalance { + return gerror.New("execution contract balance pay type is invalid") + } + if c.PayAmount < 0 || c.BalanceAmount < 0 { + return gerror.New("execution contract balance amount is invalid") + } + totalAmount := roundAmount(c.TotalAmount()) + if roundAmount(c.PayAmount) != totalAmount || roundAmount(c.BalanceAmount) != totalAmount { + return gerror.New("execution contract balance payment amount does not match total amount") + } + if strings.TrimSpace(c.BalanceDetailSerialNo) == "" && c.PayAmount > 0 { + return gerror.New("execution contract balance serial number is missing") + } + default: + return gerror.New("execution contract payment mode is invalid") + } + return nil +} + +func (c *ExecutionContract) normalizePaymentContract() { + if c == nil { + return + } + if strings.TrimSpace(c.PaymentMode) == "" { + if c.PayType == consts.PayTypeBalance || c.BalanceAmount > 0 || strings.TrimSpace(c.BalanceDetailSerialNo) != "" { + c.PaymentMode = "balance" + } else { + c.PaymentMode = "offline" + } + } + amount := c.TotalAmount() + if c.PayAmount <= 0 && amount > 0 { + c.PayAmount = amount + } + if c.PayStatus == 0 { + c.PayStatus = consts.PayStatusPaid + } + if c.PaymentMode == "offline" { + if c.PayType == 0 { + c.PayType = consts.PayTypeOffline + } + if c.OfflinePaidAmount <= 0 { + c.OfflinePaidAmount = amount + } + return + } + if c.PaymentMode == "balance" { + if c.PayType == 0 { + c.PayType = consts.PayTypeBalance + } + if c.BalanceAmount <= 0 { + c.BalanceAmount = amount + } + } +} + +func (c *ExecutionContract) TotalAmount() float64 { + if c == nil { + return 0 + } + return roundAmount(c.UnitPrice * float64(c.DisplayDurationValue()) * float64(c.Quantity)) +} + +func (c *ExecutionContract) ItemPrice() float64 { + if c == nil { + return 0 + } + return roundAmount(c.UnitPrice * float64(c.DisplayDurationValue())) +} + +func (c *ExecutionContract) DisplayDurationValue() int { + if c == nil { + return 0 + } + if c.DurationDisplayValue > 0 { + return c.DurationDisplayValue + } + return c.DurationMonths +} + +func (c *ExecutionContract) ServiceEnd(begin time.Time) time.Time { + if c == nil { + return begin + } + return addMonths(begin, c.DurationMonths+c.GiftDurationMonths) +} + +func BuildOrderDO(contract *ExecutionContract, orderSN string, now time.Time) (*rc.OrderDO, error) { + if err := contract.Validate(); err != nil { + return nil, err + } + orderSN = strings.TrimSpace(orderSN) + if orderSN == "" { + return nil, gerror.New("order number must be generated before building the order") + } + amount := contract.TotalAmount() + balanceAmount := float64(0) + offlinePaidAmount := amount + payType := consts.PayTypeOffline + payOrderSN := "" + if contract.PaymentMode == "balance" { + balanceAmount = contract.BalanceAmount + offlinePaidAmount = 0 + payType = consts.PayTypeBalance + payOrderSN = contract.BalanceDetailSerialNo + } + remark := contract.Remark + if remark == "" { + remark = "SIM card package business order" + } + payTime := gtime.NewFromTime(now) + if strings.TrimSpace(contract.PaymentTime) != "" { + if parsed, err := parseExecutionTime(strings.TrimSpace(contract.PaymentTime)); err == nil { + payTime = parsed + } + } + orderType := consts.OrderTypeCardRenew + if contract.BatchType == consts.BatchTypeChange { + orderType = consts.OrderTypeCardChange + } + return &rc.OrderDO{ + OrderSn: orderSN, + ParentOrderId: int64(0), + ParentOrderSn: "", + CustomerType: consts.CustomerTypeAgent, + CustomerAgentId: contract.CustomerAgentID, + EndUserId: int64(0), + AgentId: contract.SellerAgentID, + OrderType: orderType, + OrderStatus: consts.OrderStatusCompleted, + ShippingStatus: 0, + ReturnStatus: consts.OrderReturnStatusNone, + Consignee: "", + Mobile: "", + Email: "", + Address: "", + AreaCode: "", + BuyerNote: contract.BuyerNote, + AdminNote: contract.AdminNote, + Remark: remark, + ShippingMethod: consts.ShippingMethodNone, + LogisticsId: int64(0), + LogisticsName: "", + ShippingTypeId: int64(0), + ShippingTypeName: "", + TrackingNo: "", + PayType: payType, + PayStatus: consts.PayStatusPaid, + PayOrderSn: payOrderSN, + PayTime: payTime, + TotalAmount: amount, + ProductAmount: amount, + ShippingFee: 0, + ServiceFee: 0, + InvoiceFee: 0, + CouponAmount: 0, + DiscountAmount: 0, + Balance: balanceAmount, + PaidAmount: amount, + OnlinePaidAmount: 0, + OfflinePaidAmount: offlinePaidAmount, + UnpaidAmount: 0, + UnrefundAmount: 0, + IsNeedCommission: 0, + DistributionStatus: 0, + TenantId: contract.TenantID, + CreatedDept: contract.CreatedDeptID, + CreatedBy: contract.OperatorUserID, + CreatedAt: payTime, + UpdatedBy: contract.OperatorUserID, + UpdatedAt: payTime, + }, nil +} + +func BuildOrderItemDO(contract *ExecutionContract, product OrderProductContract, orderID int64, orderSN string, beginAt, endAt, now time.Time) (*rc.OrderItemDO, error) { + if err := contract.Validate(); err != nil { + return nil, err + } + if orderID <= 0 || strings.TrimSpace(orderSN) == "" { + return nil, gerror.New("order item requires a persisted order id and order number") + } + if product.ID <= 0 || strings.TrimSpace(product.Code) == "" || strings.TrimSpace(product.Name) == "" { + return nil, gerror.New("order product contract is incomplete") + } + if strings.TrimSpace(product.Code) != strings.TrimSpace(contract.PackageCode) { + return nil, gerror.New("order product does not match the confirmed package") + } + if !endAt.After(beginAt) { + return nil, gerror.New("order service end time must be after begin time") + } + + skuData, err := buildSKUData(contract, product, beginAt) + if err != nil { + return nil, err + } + extraSKUData, err := buildExtraSKUData(contract, beginAt, endAt) + if err != nil { + return nil, err + } + return &rc.OrderItemDO{ + OrderId: orderID, + OrderSn: strings.TrimSpace(orderSN), + CustomerType: consts.CustomerTypeAgent, + CustomerAgentId: contract.CustomerAgentID, + EndUserId: int64(0), + Price: contract.ItemPrice(), + Quantity: contract.Quantity, + ProductId: product.ID, + ProductName: product.Name, + ProductSn: product.Code, + PicThumb: "", + SkuId: int64(0), + SkuData: skuData, + DeliveryQuantity: 0, + ProductType: consts.ProductTypeCard, + IsGift: 0, + AgentId: contract.SellerAgentID, + PrepayPrice: 0, + Commission: "", + OriginPrice: contract.ItemPrice(), + PromotionData: "", + ExtraSkuData: extraSKUData, + PayMode: consts.PayModeYearMonth, + ServiceBeginTime: gtime.NewFromTime(beginAt), + ServiceEndTime: gtime.NewFromTime(endAt), + TenantId: contract.TenantID, + CreatedDept: contract.CreatedDeptID, + CreatedBy: contract.OperatorUserID, + CreatedAt: gtime.NewFromTime(now), + UpdatedBy: contract.OperatorUserID, + UpdatedAt: gtime.NewFromTime(now), + }, nil +} + +// BuildRenewCardUpdate returns the intentionally narrow card update used by +// renew execution. GoFrame ignores every nil field in this generated DO. +func BuildRenewCardUpdate(contract *ExecutionContract, endAt time.Time, packageStatus int, now time.Time) (*do.TSimCardInfo, error) { + if err := contract.Validate(); err != nil { + return nil, err + } + if contract.BatchType != consts.BatchTypeRenew { + return nil, gerror.New("renew card update requires a renew contract") + } + if endAt.IsZero() { + return nil, gerror.New("renew card end time cannot be empty") + } + if now.IsZero() { + return nil, gerror.New("renew card update time cannot be empty") + } + if packageStatus < consts.PackageStatusPending || packageStatus > consts.PackageStatusExpired { + return nil, gerror.New("renew card package status is invalid") + } + return &do.TSimCardInfo{ + FlowPkgEndAt: gtime.NewFromTime(endAt), + PackageStatus: packageStatus, + UpdatedBy: contract.OperatorUserID, + UpdatedAt: gtime.NewFromTime(now), + }, nil +} + +func buildSKUData(contract *ExecutionContract, product OrderProductContract, beginAt time.Time) (string, error) { + durationLabel := fmt.Sprintf("%d months", contract.DurationMonths) + if contract.DurationDisplayValue > 0 && strings.TrimSpace(contract.RenewDurationUnit) != "" { + durationLabel = renewDurationLabel(contract.DurationDisplayValue, contract.RenewDurationUnit) + } + data := []map[string]any{ + {"skuName": "package", "skuValue": product.Name, "packageCode": product.Code, "packageType": product.Type}, + {"skuName": "quantity", "skuValue": fmt.Sprintf("%d", contract.Quantity)}, + {"skuName": "effectiveTime", "skuValue": beginAt.Format("2006-01-02 15:04:05")}, + {"skuName": "duration", "skuValue": durationLabel}, + } + encoded, err := json.Marshal(data) + if err != nil { + return "", gerror.Wrap(err, "marshal order SKU data failed") + } + return string(encoded), nil +} + +func buildExtraSKUData(contract *ExecutionContract, beginAt, endAt time.Time) (string, error) { + data := map[string]any{ + "outboundBatchNo": contract.BatchNo, + "businessBatchNo": contract.BatchNo, + "offlineContractCompleted": contract.OfflineContractCompleted, + "packageBeginTime": beginAt.Format("2006-01-02 15:04:05"), + "packageEndTime": endAt.Format("2006-01-02 15:04:05"), + "durationMonths": contract.DurationMonths, + "durationValue": contract.DisplayDurationValue(), + "actualDurationMonths": contract.DurationMonths, + "durationUnit": strings.TrimSpace(contract.RenewDurationUnit), + "billingMethod": contract.BillingMethod, + "giftDurationMonths": contract.GiftDurationMonths, + "unitPrice": contract.UnitPrice, + "amount": contract.TotalAmount(), + "paymentMode": contract.PaymentMode, + "payType": contract.PayType, + "balanceAmount": contract.BalanceAmount, + "offlinePaidAmount": contract.OfflinePaidAmount, + } + if contract.BalanceDetailSerialNo != "" { + data["balanceDetailSerialNo"] = contract.BalanceDetailSerialNo + } + if contract.SourceBatchNo != "" { + data["sourceBatchNo"] = contract.SourceBatchNo + } + if contract.EffectiveMode != "" { + data["effectiveMode"] = contract.EffectiveMode + } + encoded, err := json.Marshal(data) + if err != nil { + return "", gerror.Wrap(err, "marshal order extra SKU data failed") + } + return string(encoded), nil +} + +func firstString(info *gjson.Json, paths ...string) string { + for _, path := range paths { + value := strings.TrimSpace(info.Get(path).String()) + if value != "" { + return value + } + } + return "" +} + +func hasJSONValue(info *gjson.Json, path string) bool { + if info == nil || strings.TrimSpace(path) == "" { + return false + } + value := info.Get(path) + return value != nil && !value.IsNil() +} + +func firstInt(info *gjson.Json, paths ...string) int { + return int(firstInt64(info, paths...)) +} + +func firstInt64(info *gjson.Json, paths ...string) int64 { + for _, path := range paths { + value := info.Get(path) + if value != nil && !value.IsNil() { + return value.Int64() + } + } + return 0 +} + +func firstFloat64(info *gjson.Json, paths ...string) (float64, bool) { + for _, path := range paths { + value := info.Get(path) + if value != nil && !value.IsNil() { + return value.Float64(), true + } + } + return 0, false +} + +func firstAmount(info *gjson.Json, paths ...string) float64 { + value, _ := firstFloat64(info, paths...) + return roundAmount(value) +} + +func parseExecutionTime(value string) (*gtime.Time, error) { + for _, layout := range []string{"2006-01-02 15:04:05", "2006-01-02"} { + parsed, err := time.ParseInLocation(layout, value, time.Local) + if err == nil { + return gtime.NewFromTime(parsed), nil + } + } + return nil, gerror.Newf("invalid execution time: %s", value) +} diff --git a/internal/service/renewchange/order_contract_test.go b/internal/service/renewchange/order_contract_test.go new file mode 100644 index 0000000..19a3957 --- /dev/null +++ b/internal/service/renewchange/order_contract_test.go @@ -0,0 +1,413 @@ +package renewchange + +import ( + "encoding/json" + "reflect" + "testing" + "time" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/util/gconv" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" +) + +func TestParseExecutionContractReadsConfirmedFlatContract(t *testing.T) { + main := &rc.BatchMainRow{ + BatchNo: "REN20260803100250ABCDEFGH", + BatchType: consts.BatchTypeRenew, + DeptId: 100, + CreatedDept: 100, + CreatedBy: 1, + TenantId: "000000", + BatchInfo: gjson.New(map[string]any{ + "customerAgentId": 200, + "packageCode": "PKG001", + "packageName": "Year package", + "renewUnitPrice": 10.25, + "quantity": 2, + "durationMonths": 12, + "giftDurationMonths": 1, + "offlineContractCompleted": true, + "buyerNote": "buyer", + "adminNote": "admin", + }), + } + contract, err := ParseExecutionContract(main) + if err != nil { + t.Fatalf("ParseExecutionContract returned error: %v", err) + } + if contract.DurationDisplayValue != 1 || contract.DurationMonths != 12 || contract.GiftDurationMonths != 1 { + t.Fatalf("duration = %d/%d+%d, want display 1 and actual 12+1", contract.DurationDisplayValue, contract.DurationMonths, contract.GiftDurationMonths) + } + if contract.CustomerAgentID != 200 || contract.SellerAgentID != 100 { + t.Fatalf("buyer/seller = %d/%d, want 200/100", contract.CustomerAgentID, contract.SellerAgentID) + } + if contract.TotalAmount() != 20.5 { + t.Fatalf("TotalAmount = %.2f, want 20.50", contract.TotalAmount()) + } + if contract.ItemPrice() != 10.25 { + t.Fatalf("ItemPrice = %.2f, want 10.25", contract.ItemPrice()) + } +} + +func TestParseExecutionContractRenewDurationValueOverridesLegacyMonthSnapshot(t *testing.T) { + main := &rc.BatchMainRow{ + BatchNo: "REN20260803100250LEGACY", + BatchType: consts.BatchTypeRenew, + DeptId: 100, + CreatedDept: 100, + CreatedBy: 1, + TenantId: "000000", + BatchInfo: gjson.New(map[string]any{ + "customerAgentId": 200, + "packageCode": "PKG001", + "packageName": "Year package", + "renewUnitPrice": 10.25, + "quantity": 1, + "durationValue": 1, + "durationYears": 1, + "durationMonths": 1, + "actualDurationMonths": 1, + "billingMethod": consts.BillingMethodMonth, + "renewDurationUnit": consts.RenewDurationUnitMonth, + "offlineContractCompleted": true, + }), + } + + contract, err := ParseExecutionContract(main) + if err != nil { + t.Fatalf("ParseExecutionContract returned error: %v", err) + } + if contract.RenewDurationUnit != consts.RenewDurationUnitYear { + t.Fatalf("renew duration unit = %q, want year", contract.RenewDurationUnit) + } + if contract.DurationDisplayValue != 1 || contract.DurationMonths != 12 { + t.Fatalf("duration = %d/%d, want display 1 and actual 12 months", contract.DurationDisplayValue, contract.DurationMonths) + } +} + +func TestParseExecutionContractFallsBackToLegacyNestedSnapshot(t *testing.T) { + main := &rc.BatchMainRow{ + BatchNo: "CHG20260803100250ABCDEFGH", + BatchType: consts.BatchTypeChange, + DeptId: 100, + CreatedDept: 100, + CreatedBy: 1, + TenantId: "000000", + BatchInfo: gjson.New(map[string]any{ + "changePriceSnapshot": map[string]any{ + "customerAgentId": 200, + "newPackageCode": "PKG002", + "newPackageName": "Changed package", + "changeUnitPrice": 8.5, + }, + "quantity": 1, + "durationMonths": 24, + "giftDurationMonths": 0, + "offlineContractCompleted": true, + }), + } + contract, err := ParseExecutionContract(main) + if err != nil { + t.Fatalf("ParseExecutionContract returned error: %v", err) + } + if contract.PackageCode != "PKG002" || contract.UnitPrice != 8.5 { + t.Fatalf("legacy snapshot contract = %s/%.2f, want PKG002/8.50", contract.PackageCode, contract.UnitPrice) + } +} + +func TestParseExecutionContractNormalizesSourceBatchBySourceType(t *testing.T) { + tests := []struct { + name string + sourceType int + mainSourceBatchNo string + infoSourceBatchNo string + want string + }{ + { + name: "input self reference is not a business source batch", + sourceType: consts.BatchSourceTypeBatchInput, + mainSourceBatchNo: "REN20260803161502D52JJU6B", + want: "", + }, + { + name: "file source ignores legacy source value", + sourceType: consts.BatchSourceTypeFile, + mainSourceBatchNo: "REN20260803161502D52JJU6B", + infoSourceBatchNo: "OUT20260803STALE", + want: "", + }, + { + name: "batch selection keeps confirmed source batch", + sourceType: consts.BatchSourceTypeBatchSelect, + mainSourceBatchNo: "OUT20260803MAIN", + infoSourceBatchNo: "OUT20260803CONFIRMED", + want: "OUT20260803CONFIRMED", + }, + { + name: "batch selection falls back to main source batch", + sourceType: consts.BatchSourceTypeBatchSelect, + mainSourceBatchNo: "OUT20260803MAIN", + want: "OUT20260803MAIN", + }, + { + name: "batch selection repairs self reference from main source batch", + sourceType: consts.BatchSourceTypeBatchSelect, + mainSourceBatchNo: "OUT20260803MAIN", + infoSourceBatchNo: "REN20260803161502D52JJU6B", + want: "OUT20260803MAIN", + }, + { + name: "legacy self reference is normalized to empty", + sourceType: consts.BatchSourceTypeBatchSelect, + mainSourceBatchNo: "REN20260803161502D52JJU6B", + want: "", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + info := map[string]any{ + "customerAgentId": 200, + "packageCode": "PKG001", + "packageName": "Year package", + "renewUnitPrice": 10.25, + "quantity": 1, + "durationMonths": 12, + "offlineContractCompleted": true, + } + if tt.infoSourceBatchNo != "" { + info["sourceBatchNo"] = tt.infoSourceBatchNo + } + main := &rc.BatchMainRow{ + BatchNo: "REN20260803161502D52JJU6B", + BatchType: consts.BatchTypeRenew, + SourceType: tt.sourceType, + SourceBatchNo: tt.mainSourceBatchNo, + DeptId: 100, + CreatedDept: 100, + CreatedBy: 1, + TenantId: "000000", + BatchInfo: gjson.New(info), + } + + contract, err := ParseExecutionContract(main) + if err != nil { + t.Fatalf("ParseExecutionContract returned error: %v", err) + } + if contract.SourceBatchNo != tt.want { + t.Fatalf("SourceBatchNo = %q, want %q", contract.SourceBatchNo, tt.want) + } + }) + } +} + +func TestParseExecutionContractRejectsMissingConfirmedFields(t *testing.T) { + base := validExecutionContract() + cases := []struct { + name string + mutate func(*ExecutionContract) + }{ + {name: "customer", mutate: func(c *ExecutionContract) { c.CustomerAgentID = 0 }}, + {name: "duration", mutate: func(c *ExecutionContract) { c.DurationMonths = 0 }}, + {name: "quantity", mutate: func(c *ExecutionContract) { c.Quantity = 0 }}, + {name: "package", mutate: func(c *ExecutionContract) { c.PackageCode = "" }}, + {name: "offline contract", mutate: func(c *ExecutionContract) { c.OfflineContractCompleted = false }}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := *base + tc.mutate(&got) + if err := got.Validate(); err == nil { + t.Fatalf("Validate accepted missing %s", tc.name) + } + }) + } +} + +func TestBuildOrderDocumentsUseAggregatePurchasePeriodPrice(t *testing.T) { + contract := validExecutionContract() + contract.UnitPrice = 10.25 + contract.Quantity = 2 + contract.DurationMonths = 12 + contract.GiftDurationMonths = 1 + now := time.Date(2026, 8, 3, 10, 2, 50, 0, time.Local) + orderSN := "SO20260803100250ABCDEFGH" + + order, err := BuildOrderDO(contract, orderSN, now) + if err != nil { + t.Fatalf("BuildOrderDO returned error: %v", err) + } + if gconv.String(order.OrderSn) != orderSN { + t.Fatalf("order_sn = %q, want %q", order.OrderSn, orderSN) + } + if gconv.Int64(order.CustomerAgentId) != 200 || gconv.Int64(order.AgentId) != 100 { + t.Fatalf("order buyer/seller = %v/%v, want 200/100", order.CustomerAgentId, order.AgentId) + } + if gconv.Int(order.OrderType) != consts.OrderTypeCardRenew { + t.Fatalf("order_type = %v, want renew", order.OrderType) + } + if gconv.Float64(order.TotalAmount) != 246 || gconv.Float64(order.ProductAmount) != 246 || gconv.Float64(order.PaidAmount) != 246 { + t.Fatalf("order amount fields = %v/%v/%v, want 246", order.TotalAmount, order.ProductAmount, order.PaidAmount) + } + if order.CreatedAt == nil || !order.CreatedAt.Time.Equal(now) || order.UpdatedAt == nil || !order.UpdatedAt.Time.Equal(now) { + t.Fatalf("order audit times = %v/%v, want %v", order.CreatedAt, order.UpdatedAt, now) + } + + beginAt := time.Date(2026, 8, 15, 0, 0, 0, 0, time.Local) + endAt := contract.ServiceEnd(beginAt) + item, err := BuildOrderItemDO(contract, OrderProductContract{ + ID: 300, Code: "PKG001", Name: "Year package", Type: 1, + }, 900, orderSN, beginAt, endAt, now) + if err != nil { + t.Fatalf("BuildOrderItemDO returned error: %v", err) + } + if gconv.Float64(item.Price) != 123 { + t.Fatalf("item price = %v, want totalAmount/quantity = 123", item.Price) + } + if gconv.Int(item.Quantity) != 2 { + t.Fatalf("item quantity = %v, want 2", item.Quantity) + } + if item.CreatedAt == nil || !item.CreatedAt.Time.Equal(now) || item.UpdatedAt == nil || !item.UpdatedAt.Time.Equal(now) { + t.Fatalf("item audit times = %v/%v, want %v", item.CreatedAt, item.UpdatedAt, now) + } + if got := item.ServiceEndTime.Time; !got.Equal(time.Date(2027, 9, 15, 0, 0, 0, 0, time.Local)) { + t.Fatalf("service end = %s, want 2027-09-15", got) + } + if item.CreatedAt == nil || item.UpdatedAt == nil || !item.CreatedAt.Time.Equal(now) || !item.UpdatedAt.Time.Equal(now) { + t.Fatalf("order item audit fields = %v/%v, want %v", item.CreatedAt, item.UpdatedAt, now) + } + + var extra map[string]any + if err = json.Unmarshal([]byte(gconv.String(item.ExtraSkuData)), &extra); err != nil { + t.Fatalf("extra_sku_data is invalid JSON: %v", err) + } + if gconv.String(extra["outboundBatchNo"]) != contract.BatchNo || gconv.Int(extra["durationMonths"]) != 12 || gconv.Int(extra["giftDurationMonths"]) != 1 { + t.Fatalf("extra duration = %v+%v, want 12+1", extra["durationMonths"], extra["giftDurationMonths"]) + } + if gconv.Float64(extra["amount"]) != 246 { + t.Fatalf("extra amount = %v, want 246", extra["amount"]) + } +} + +func TestBuildOrderUsesChangeOrderType(t *testing.T) { + contract := validExecutionContract() + contract.BatchType = consts.BatchTypeChange + + order, err := BuildOrderDO(contract, "SO20260803100250CHANGE01", time.Date(2026, 8, 3, 10, 2, 50, 0, time.Local)) + if err != nil { + t.Fatalf("BuildOrderDO returned error: %v", err) + } + if gconv.Int(order.OrderType) != consts.OrderTypeCardChange { + t.Fatalf("order_type = %v, want change", order.OrderType) + } +} + +func TestBuildOrderUsesBalancePaymentSnapshot(t *testing.T) { + contract := validExecutionContract() + contract.OfflineContractCompleted = false + contract.PaymentMode = "balance" + contract.PayType = consts.PayTypeBalance + contract.PayAmount = 120 + contract.BalanceAmount = 120 + contract.BalanceDetailSerialNo = "RENPAYREN20260803100250ABCDEFGH" + contract.UnitPrice = 10 + contract.Quantity = 1 + contract.DurationMonths = 12 + + order, err := BuildOrderDO(contract, "SO20260803100250ABCDEFGH", time.Date(2026, 8, 3, 10, 2, 50, 0, time.Local)) + if err != nil { + t.Fatalf("BuildOrderDO returned error: %v", err) + } + if gconv.Int(order.PayType) != consts.PayTypeBalance { + t.Fatalf("pay_type = %v, want balance", order.PayType) + } + if gconv.String(order.PayOrderSn) != contract.BalanceDetailSerialNo { + t.Fatalf("pay_order_sn = %q, want %q", order.PayOrderSn, contract.BalanceDetailSerialNo) + } + if gconv.Float64(order.Balance) != 120 || gconv.Float64(order.OfflinePaidAmount) != 0 { + t.Fatalf("balance/offline = %v/%v, want 120/0", order.Balance, order.OfflinePaidAmount) + } +} + +func TestBalancePaymentSnapshotMustMatchTotalAmount(t *testing.T) { + contract := validExecutionContract() + contract.OfflineContractCompleted = false + contract.PaymentMode = "balance" + contract.PayType = consts.PayTypeBalance + contract.PayAmount = 1 + contract.BalanceAmount = 1 + contract.BalanceDetailSerialNo = "RENPAYREN20260803100250ABCDEFGH" + + if err := contract.Validate(); err == nil { + t.Fatal("Validate accepted balance amount that does not match total amount") + } +} + +func TestBuildOrderRejectsEmptyOrderNumber(t *testing.T) { + if _, err := BuildOrderDO(validExecutionContract(), "", time.Now()); err == nil { + t.Fatal("BuildOrderDO accepted an empty order number") + } +} + +func TestBuildRenewCardUpdateOnlySetsRenewFields(t *testing.T) { + contract := validExecutionContract() + endAt := time.Date(2027, 8, 3, 0, 0, 0, 0, time.Local) + now := time.Date(2026, 8, 3, 10, 2, 50, 0, time.Local) + update, err := BuildRenewCardUpdate(contract, endAt, consts.PackageStatusEffective, now) + if err != nil { + t.Fatalf("BuildRenewCardUpdate returned error: %v", err) + } + if update.FlowPkgEndAt == nil || !update.FlowPkgEndAt.Time.Equal(endAt) { + t.Fatalf("flow_pkg_end_at = %v, want %v", update.FlowPkgEndAt, endAt) + } + if gconv.Int(update.PackageStatus) != consts.PackageStatusEffective || gconv.Int64(update.UpdatedBy) != contract.OperatorUserID { + t.Fatalf("package_status/updated_by = %v/%v", update.PackageStatus, update.UpdatedBy) + } + if update.UpdatedAt == nil || !update.UpdatedAt.Time.Equal(now) { + t.Fatalf("updated_at = %v, want %v", update.UpdatedAt, now) + } + assertOnlyRenewCardFieldsSet(t, update) +} + +func validExecutionContract() *ExecutionContract { + return &ExecutionContract{ + BatchNo: "REN20260803100250ABCDEFGH", + BatchType: consts.BatchTypeRenew, + TenantID: "000000", + CustomerAgentID: 200, + SellerAgentID: 100, + CreatedDeptID: 100, + OperatorUserID: 1, + PackageCode: "PKG001", + PackageName: "Year package", + UnitPrice: 10, + Quantity: 1, + DurationMonths: 12, + GiftDurationMonths: 0, + OfflineContractCompleted: true, + } +} + +func assertOnlyRenewCardFieldsSet(t *testing.T, update *do.TSimCardInfo) { + t.Helper() + allowed := map[string]bool{"FlowPkgEndAt": true, "PackageStatus": true, "UpdatedBy": true, "UpdatedAt": true} + value := reflect.ValueOf(update).Elem() + typ := value.Type() + for i := 0; i < value.NumField(); i++ { + fieldType := typ.Field(i) + if fieldType.Name == "Meta" || allowed[fieldType.Name] { + continue + } + field := value.Field(i) + switch field.Kind() { + case reflect.Interface, reflect.Pointer, reflect.Map, reflect.Slice: + if !field.IsNil() { + t.Fatalf("unexpected renew card field set: %s", fieldType.Name) + } + } + } +} diff --git a/internal/service/renewchange/prepare.go b/internal/service/renewchange/prepare.go new file mode 100644 index 0000000..0969101 --- /dev/null +++ b/internal/service/renewchange/prepare.go @@ -0,0 +1,1299 @@ +package renewchange + +import ( + "context" + "sort" + "strconv" + "strings" + "time" + + "github.com/gogf/gf/v2/container/gvar" + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" +) + +const prepareProcessingRetryAfter = 2 * time.Minute + +// PrepareExecutor 处理 batch_type=4 的预处理批次。 +// 它根据 batch_info.targetBatchType 把工作路由到 renew/change 的准备逻辑上。 +// +// 成功：保持 batch_type=4, batch_async_status=Success, run_result=Success； +// +// 写 batch_info 报价/客户/套餐快照；不改业务表。 +// +// 失败：转为目标业务类型后写 Failed；明细失败 + 原因；不改业务表。 +// 旧/重复消息：被处理过的批次（async != Processing）会直接安全忽略。 +type PrepareExecutor struct { + repo *Repository + nowFunc NowFunc +} + +// NewPrepareExecutor 返回默认的预处理执行器。 +func NewPrepareExecutor(repo *Repository) *PrepareExecutor { + if repo == nil { + repo = NewRepository() + } + return &PrepareExecutor{repo: repo, nowFunc: timeNow} +} + +// SetNowFunc 注入时钟，便于测试。 +func (p *PrepareExecutor) SetNowFunc(now NowFunc) { p.nowFunc = now } + +// CanExecuteProcessing allows a redelivered message to reclaim a stale prepare +// batch after a worker crash or shutdown between claim and finalization. +func (p *PrepareExecutor) CanExecuteProcessing(_ context.Context, main *entity.BatchMain) (bool, error) { + if main == nil || main.BatchType != consts.BatchTypePrepare || main.BatchAsyncStatus != consts.BatchAsyncStatusProcessing { + return false, nil + } + if main.StartTime == nil || main.StartTime.IsZero() { + return true, nil + } + return p.nowFunc().Sub(main.StartTime.Time) >= prepareProcessingRetryAfter, nil +} + +// Execute 实现 batch.Executor。 +func (p *PrepareExecutor) Execute(ctx context.Context, batchNo string) error { + batchNo = strings.TrimSpace(batchNo) + if batchNo == "" { + return gerror.New("prepare batch number is empty") + } + main, err := p.repo.LoadProcessingBatch(ctx, batchNo, consts.BatchTypePrepare) + if err != nil { + return err + } + targetType := readTargetBatchType(main.BatchInfo) + if targetType != consts.BatchTypeRenew && targetType != consts.BatchTypeChange { + return p.finalizeBatchFailure(ctx, main, "预处理批次缺少合法 targetBatchType（必须是 101 或 102）", nil) + } + details, err := p.repo.LoadRunningDetails(ctx, batchNo, consts.BatchTypePrepare) + if err != nil { + return gerror.Wrap(err, "load prepare details failed") + } + if len(details) == 0 { + return p.finalizeBatchFailure(ctx, main, "预处理批次无可处理的明细", nil) + } + options := loadPrepareOptions(main.BatchInfo) + result, err := p.validate(ctx, main, targetType, options, details) + if err != nil { + return gerror.Wrap(err, "prepare validate failed") + } + if result.hasFailure { + return p.finalizeBatchFailure(ctx, main, result.failureReason, mergeRowErrors(result.perRow, result.rowErrors)) + } + return p.finalizeBatchSuccess(ctx, main, targetType, result, details) +} + +type prepareOptions struct { + customerAgentID int64 + targetAgentID int64 + priceScope int + // change 专用 + newPackageCode string + durationValue int + durationSet bool +} + +type prepareOutcome struct { + hasFailure bool + failureReason string + perRow map[int]string + priceSnap *RenewPriceSnapshot + changeSnap *ChangePrepareSnapshot + changePrice *ChangePriceSnapshot + customer *customerInfo + packageInfo *packageInfo + rowErrors map[int]string + rows map[int]*RenewRow +} + +type customerInfo struct { + AgentID int64 `json:"agentId"` + Name string `json:"name"` + DeptID int64 `json:"deptId"` + TenantID string `json:"tenantId"` +} + +type packageInfo struct { + Code string `json:"code"` + Name string `json:"name"` + Type int `json:"type"` + FlowMb float64 `json:"flowMb"` + Duration int `json:"durationMonths"` + UnitPrice float64 `json:"unitPrice"` +} + +const renewSamePeriodFailureReason = "同一批续费卡当前套餐开始时间和到期时间必须一致" +const renewExpireWindowMonths = 6 +const renewExpireFailureReason = "距离到期超过6个月，不能续费" + +func (p *PrepareExecutor) validate(ctx context.Context, main *rc.BatchMainRow, targetType int, options prepareOptions, details []*rc.BatchDetailRow) (*prepareOutcome, error) { + now := p.nowFunc() + rows := make([]*RenewRow, 0, len(details)) + iccids := make([]string, 0, len(details)) + rowErrors := map[int]string{} + for _, d := range details { + if d == nil { + continue + } + iccid := strings.TrimSpace(d.ItemId) + if iccid == "" && d.Snapshot != nil { + iccid = prepareJSONValue(d.Snapshot, "iccid", "ICCID") + } + row := &RenewRow{ + RowNo: d.RowNo, + ICCID: iccid, + Reason: strings.TrimSpace(d.ErrorReason), + } + if d.Snapshot != nil { + row.MSISDN = prepareJSONValue(d.Snapshot, "msisdn", "MSISDN") + row.IMSI = prepareJSONValue(d.Snapshot, "imsi", "IMSI") + row.NewPkgCode = strings.TrimSpace(d.Snapshot.Get("newPackageCode").String()) + row.PackageCode = strings.TrimSpace(d.Snapshot.Get("packageCode").String()) + row.OldPkgCode = strings.TrimSpace(d.Snapshot.Get("oldPackageCode").String()) + row.OldPkgName = strings.TrimSpace(d.Snapshot.Get("oldPackageName").String()) + row.OutboundBatchNo = strings.TrimSpace(d.Snapshot.Get("outboundBatchNo").String()) + row.SourceBatchNo = strings.TrimSpace(d.Snapshot.Get("sourceBatchNo").String()) + } + rows = append(rows, row) + } + if err := p.resolvePrepareRowIdentities(ctx, main.TenantId, rows, rowErrors); err != nil { + return nil, err + } + iccids = collectPrepareIccids(rows, rowErrors) + cards, err := p.repo.Tables().LoadCardsByIccids(ctx, iccids) + if err != nil { + return nil, gerror.Wrap(err, "load sim cards failed") + } + var bindRows []*rc.SimCardBindPkg + if targetType == consts.BatchTypeRenew { + bindRows, err = p.repo.Tables().LoadRenewBindRowsByIccids(ctx, iccids) + } else { + bindRows, err = p.repo.Tables().LoadActiveBindRowsByIccids(ctx, iccids) + } + if err != nil { + return nil, gerror.Wrap(err, "load current subscriptions failed") + } + out := &prepareOutcome{ + perRow: map[int]string{}, + rowErrors: rowErrors, + rows: make(map[int]*RenewRow, len(rows)), + } + for _, row := range rows { + out.rows[row.RowNo] = row + } + sourceBatchNo := executionSourceBatchNo(main, main.BatchInfo) + // 选择基准套餐/价格：先按相同 packageCode 收敛；多个则统一失败。 + packageCodes := map[string]struct{}{} + customerIDs := map[int64]struct{}{} + for _, row := range rows { + if row.Reason != "" { + out.rowErrors[row.RowNo] = row.Reason + } + } + for _, row := range rows { + if strings.TrimSpace(out.rowErrors[row.RowNo]) != "" { + continue + } + if row.ICCID == "" { + out.rowErrors[row.RowNo] = appendIfEmpty(out.rowErrors[row.RowNo], "ICCID不能为空") + continue + } + card, ok := cards[row.ICCID] + if !ok || card == nil { + out.rowErrors[row.RowNo] = appendIfEmpty(out.rowErrors[row.RowNo], rcLocalize("errCardNotFound")) + continue + } + if card.TenantId != "" && main.TenantId != "" && card.TenantId != main.TenantId { + out.rowErrors[row.RowNo] = appendIfEmpty(out.rowErrors[row.RowNo], "ICCID不属于当前租户") + continue + } + if card.DeptId <= 0 { + out.rowErrors[row.RowNo] = appendIfEmpty(out.rowErrors[row.RowNo], rcLocalize("errCardNotFound")) + continue + } + if strings.TrimSpace(card.CurrentPackageCode) == "" { + out.rowErrors[row.RowNo] = appendIfEmpty(out.rowErrors[row.RowNo], rcLocalize("errCardNoCurrentPackage")) + continue + } + bind, futureBinds, bindErr := selectCurrentSubscription(bindRows, row.ICCID) + if bindErr != nil { + out.rowErrors[row.RowNo] = appendIfEmpty(out.rowErrors[row.RowNo], bindErr.Error()) + continue + } + if bindErr = validateCurrentSubscription(card, bind, subscriptionValidation{ + ICCID: row.ICCID, + MSISDN: row.MSISDN, + IMSI: row.IMSI, + TenantID: main.TenantId, + CustomerDeptID: card.DeptId, + PackageCode: card.CurrentPackageCode, + SourceBatchNo: sourceBatchNo, + }); bindErr != nil { + out.rowErrors[row.RowNo] = appendIfEmpty(out.rowErrors[row.RowNo], bindErr.Error()) + continue + } + periodBind := bind + if targetType == consts.BatchTypeRenew { + periodBind = renewBaseSubscriptionForCard(card, bind, futureBinds) + if periodBind == nil || periodBind.ExpectedExpireTime == nil { + out.rowErrors[row.RowNo] = appendIfEmpty(out.rowErrors[row.RowNo], "没有预计到期时间，不能续费") + continue + } + if renewExpireAfterAllowedWindow(periodBind.ExpectedExpireTime, now) { + out.rowErrors[row.RowNo] = appendIfEmpty(out.rowErrors[row.RowNo], renewExpireFailureReason) + continue + } + } + row.MSISDN = strings.TrimSpace(card.Msisdn) + row.IMSI = strings.TrimSpace(card.Imsi) + row.CustomerID = card.DeptId + row.PackageCode = strings.TrimSpace(card.CurrentPackageCode) + row.OldPkgCode = strings.TrimSpace(periodBind.PackageCode) + row.OldBindSubNo = strings.TrimSpace(periodBind.SubscriptionNo) + row.OldBindOrdNo = strings.TrimSpace(periodBind.OrderCode) + row.OldEffTime = periodBind.EffectiveTime + row.OldExpireTime = periodBind.ExpectedExpireTime + row.SourceBatchNo = sourceBatchNo + customerIDs[card.DeptId] = struct{}{} + packageCodes[row.PackageCode] = struct{}{} + } + if setPrepareFailureFromRows(out, rows) { + return out, nil + } + // 续费/变更专属校验 + if targetType == consts.BatchTypeRenew { + if len(packageCodes) > 1 { + out.rowErrors[0] = rcLocalize("errMultiplePackages") + out.hasFailure = true + out.failureReason = rcLocalize("errMultiplePackages") + return out, nil + } + if err := validateRenewSameCurrentPeriod(rows, out.rowErrors); err != nil { + out.rowErrors[0] = err.Error() + out.hasFailure = true + out.failureReason = err.Error() + return out, nil + } + } else { + if options.newPackageCode == "" { + out.rowErrors[0] = rcLocalize("errNewPackageNotInChange") + out.hasFailure = true + out.failureReason = "变更批次缺少新套餐编码" + return out, nil + } + } + // 同批客户单一性：多个客户按全批失败处理。 + if len(customerIDs) > 1 { + out.rowErrors[0] = rcLocalize("errMultipleCustomers") + out.hasFailure = true + out.failureReason = rcLocalize("errMultipleCustomers") + return out, nil + } + // 选择客户 + var customerID int64 + for id := range customerIDs { + customerID = id + } + // 加载套餐、客户、价格 + var ( + curPkg *rc.FlowPackageInfo + newPkg *rc.FlowPackageInfo + ) + for code := range packageCodes { + pkg, err := p.repo.Tables().LoadPackageByCode(ctx, main.TenantId, code) + if err != nil || pkg == nil { + out.rowErrors[0] = rcLocalize("errPkgNotFound") + out.hasFailure = true + out.failureReason = "当前套餐不存在或未启用" + return out, nil + } + if pkg.Enabled != 1 { + out.rowErrors[0] = rcLocalize("errPkgDisabled") + out.hasFailure = true + out.failureReason = "当前套餐未启用" + return out, nil + } + curPkg = pkg + for _, row := range rows { + if row.PackageCode == pkg.PackageCode { + row.OldPkgName = pkg.PackageName + } + } + } + if targetType == consts.BatchTypeChange { + newPkg, err = p.repo.Tables().LoadPackageByCode(ctx, main.TenantId, options.newPackageCode) + if err != nil || newPkg == nil { + out.rowErrors[0] = rcLocalize("errTargetPkgNotFound") + out.hasFailure = true + out.failureReason = "目标套餐不存在或未启用" + return out, nil + } + if newPkg.Enabled != 1 { + out.rowErrors[0] = rcLocalize("errPkgDisabled") + out.hasFailure = true + out.failureReason = "目标套餐未启用" + return out, nil + } + if curPkg != nil && curPkg.PackageType != newPkg.PackageType { + out.rowErrors[0] = rcLocalize("errPackageTypeMismatch") + out.hasFailure = true + out.failureReason = rcLocalize("errPackageTypeMismatch") + return out, nil + } + if curPkg != nil && curPkg.PackageCode == newPkg.PackageCode { + out.rowErrors[0] = rcLocalize("errNewPackageSameAsOld") + out.hasFailure = true + out.failureReason = rcLocalize("errNewPackageSameAsOld") + return out, nil + } + if curPkg.ChangeablePackageCodes == nil || curPkg.ChangeablePackageCodes.IsNil() { + out.rowErrors[0] = rcLocalize("errNoChangeablePackages") + out.hasFailure = true + out.failureReason = rcLocalize("errNoChangeablePackages") + return out, nil + } + allowed := false + for _, c := range curPkg.ChangeablePackageCodes.Interfaces() { + if strings.TrimSpace(stringify(c)) == newPkg.PackageCode { + allowed = true + break + } + } + if !allowed { + out.rowErrors[0] = rcLocalize("errNewPackageNotInChange") + out.hasFailure = true + out.failureReason = rcLocalize("errNewPackageNotInChange") + return out, nil + } + for _, row := range rows { + row.NewPkgCode = newPkg.PackageCode + } + } + customerName, err := p.repo.Tables().LoadDeptName(ctx, main.TenantId, customerID) + if err != nil || customerName == "" { + out.rowErrors[0] = rcLocalize("errDeptNotFound") + out.hasFailure = true + out.failureReason = rcLocalize("errDeptNotFound") + return out, nil + } + customer := &customerInfo{ + AgentID: options.customerAgentID, + Name: customerName, + DeptID: customerID, + TenantID: main.TenantId, + } + out.customer = customer + // 单卡校验 + for _, row := range rows { + if row.ICCID == "" { + continue + } + card := cards[row.ICCID] + if card == nil { + continue + } + // 续费时长校验 + if targetType == consts.BatchTypeRenew { + if curPkg.RenewableDurationConfig == nil || curPkg.RenewableDurationConfig.IsNil() { + out.rowErrors[row.RowNo] = appendIfEmpty(out.rowErrors[row.RowNo], rcLocalize("errRenewDurationEmpty")) + continue + } + opts, err := renewableYearOptions(curPkg.RenewableDurationConfig) + if err != nil || len(opts) == 0 { + out.rowErrors[row.RowNo] = appendIfEmpty(out.rowErrors[row.RowNo], rcLocalize("errRenewDurationEmpty")) + continue + } + if options.durationSet && !rc.ContainsInt(opts, options.durationValue) { + out.rowErrors[row.RowNo] = appendIfEmpty(out.rowErrors[row.RowNo], rcLocalize("errRenewableDurationMiss")) + continue + } + } + } + // 解析价格 + priceSnap, changePrice, err := p.resolvePrice(ctx, main, options, targetType, curPkg, newPkg, customer) + if err != nil { + out.rowErrors[0] = err.Error() + out.hasFailure = true + out.failureReason = err.Error() + return out, nil + } + out.priceSnap = priceSnap + if targetType == consts.BatchTypeRenew { + fillRenewBeginSnapshot(priceSnap, rows, out.rowErrors) + } + if changePrice != nil { + out.changePrice = changePrice + } + if targetType == consts.BatchTypeChange && curPkg != nil { + out.changeSnap = &ChangePrepareSnapshot{ + SourceBatchNo: executionSourceBatchNo(main, main.BatchInfo), + CustomerAgentId: customer.DeptID, + CustomerName: customer.Name, + OldPackageCode: curPkg.PackageCode, + OldPackageName: curPkg.PackageName, + OldPackageType: curPkg.PackageType, + OldPackageFlowMb: curPkg.FlowMb, + Quantity: len(rows), + } + } + if curPkg != nil { + out.packageInfo = &packageInfo{ + Code: curPkg.PackageCode, + Name: curPkg.PackageName, + Type: curPkg.PackageType, + FlowMb: curPkg.FlowMb, + Duration: options.durationValue, + UnitPrice: priceSnap.RenewUnitPrice, + } + } + // 把行级错误汇总到 hasFailure + for _, msg := range out.rowErrors { + if strings.TrimSpace(msg) != "" { + out.hasFailure = true + break + } + } + if out.hasFailure { + if out.failureReason == "" { + if len(rows) == 1 { + out.failureReason = strings.TrimSpace(out.rowErrors[rows[0].RowNo]) + } + if out.failureReason != "" { + return out, nil + } + out.failureReason = "同批存在不满足条件的卡，本次未执行" + } + } + return out, nil +} + +func setPrepareFailureFromRows(out *prepareOutcome, rows []*RenewRow) bool { + if out == nil { + return false + } + for _, msg := range out.rowErrors { + if strings.TrimSpace(msg) != "" { + out.hasFailure = true + break + } + } + if !out.hasFailure { + return false + } + if len(rows) == 1 { + out.failureReason = strings.TrimSpace(out.rowErrors[rows[0].RowNo]) + } + if out.failureReason == "" { + out.failureReason = "同批存在不满足条件的卡，本次未执行" + } + return true +} + +func (p *PrepareExecutor) resolvePrepareRowIdentities(ctx context.Context, tenantID string, rows []*RenewRow, rowErrors map[int]string) error { + keys := collectPrepareIdentityKeys(rows, rowErrors) + cards, err := p.repo.Tables().LoadCardsByIdentityValues(ctx, tenantID, keys) + if err != nil { + return gerror.Wrap(err, "load prepare identity cards failed") + } + resolvePrepareIdentityRowsFromCards(rows, rowErrors, cards) + return nil +} + +func collectPrepareIdentityKeys(rows []*RenewRow, rowErrors map[int]string) []string { + keys := make([]string, 0, len(rows)*3) + for _, row := range rows { + if row == nil || row.Reason != "" || strings.TrimSpace(rowErrors[row.RowNo]) != "" { + continue + } + keys = append(keys, row.ICCID, row.MSISDN, row.IMSI) + } + return uniquePrepareStrings(keys) +} + +func collectPrepareIccids(rows []*RenewRow, rowErrors map[int]string) []string { + seen := make(map[string]int, len(rows)) + iccids := make([]string, 0, len(rows)) + for _, row := range rows { + if row == nil || row.Reason != "" || strings.TrimSpace(rowErrors[row.RowNo]) != "" { + continue + } + row.ICCID = strings.TrimSpace(row.ICCID) + if row.ICCID == "" { + continue + } + if firstRowNo, exists := seen[row.ICCID]; exists { + reason := rcLocalize("errDuplicateICCID") + rowErrors[row.RowNo] = appendIfEmpty(rowErrors[row.RowNo], reason) + rowErrors[firstRowNo] = appendIfEmpty(rowErrors[firstRowNo], reason) + continue + } + seen[row.ICCID] = row.RowNo + iccids = append(iccids, row.ICCID) + } + return iccids +} + +func resolvePrepareIdentityRowsFromCards(rows []*RenewRow, rowErrors map[int]string, cards []*rc.SimCardInfo) { + resolver := newPrepareIdentityResolver(cards) + for _, row := range rows { + if row == nil || row.Reason != "" || strings.TrimSpace(rowErrors[row.RowNo]) != "" { + continue + } + identities := uniquePrepareStrings([]string{row.ICCID, row.MSISDN, row.IMSI}) + if len(identities) == 0 { + rowErrors[row.RowNo] = "ICCID/MSISDN/IMSI不能为空" + continue + } + card, err := resolver.resolve(identities) + if err != nil { + rowErrors[row.RowNo] = err.Error() + continue + } + if card == nil { + continue + } + if !prepareCardIdentityMatchesAll(card, identities) { + rowErrors[row.RowNo] = "ICCID/MSISDN/IMSI与卡信息不一致" + continue + } + row.ICCID = strings.TrimSpace(card.Iccid) + row.MSISDN = strings.TrimSpace(card.Msisdn) + row.IMSI = strings.TrimSpace(card.Imsi) + } +} + +type prepareIdentityResolver struct { + byValue map[string]map[int64]*rc.SimCardInfo +} + +func newPrepareIdentityResolver(cards []*rc.SimCardInfo) *prepareIdentityResolver { + resolver := &prepareIdentityResolver{byValue: map[string]map[int64]*rc.SimCardInfo{}} + for _, card := range cards { + resolver.addCard(card) + } + return resolver +} + +func (r *prepareIdentityResolver) addCard(card *rc.SimCardInfo) { + if r == nil || card == nil || card.Id <= 0 { + return + } + r.addValue(card.Iccid, card) + r.addValue(card.Msisdn, card) + r.addValue(card.Imsi, card) +} + +func (r *prepareIdentityResolver) addValue(value string, card *rc.SimCardInfo) { + value = strings.TrimSpace(value) + if r == nil || value == "" || card == nil || card.Id <= 0 { + return + } + if r.byValue == nil { + r.byValue = map[string]map[int64]*rc.SimCardInfo{} + } + matches := r.byValue[value] + if matches == nil { + matches = map[int64]*rc.SimCardInfo{} + r.byValue[value] = matches + } + matches[card.Id] = card +} + +func (r *prepareIdentityResolver) resolve(values []string) (*rc.SimCardInfo, error) { + if r == nil || len(values) == 0 { + return nil, nil + } + matches := map[int64]*rc.SimCardInfo{} + for _, value := range uniquePrepareStrings(values) { + for id, card := range r.byValue[strings.TrimSpace(value)] { + matches[id] = card + } + } + switch len(matches) { + case 0: + return nil, nil + case 1: + for _, card := range matches { + return card, nil + } + } + return nil, gerror.New("卡号对应多条卡记录，请输入 ICCID") +} + +func prepareCardIdentityMatchesAll(card *rc.SimCardInfo, values []string) bool { + if card == nil { + return false + } + for _, value := range values { + if !prepareCardIdentityMatches(card, value) { + return false + } + } + return true +} + +func prepareCardIdentityMatches(card *rc.SimCardInfo, value string) bool { + value = strings.TrimSpace(value) + if value == "" || card == nil { + return false + } + return value == strings.TrimSpace(card.Iccid) || + value == strings.TrimSpace(card.Msisdn) || + value == strings.TrimSpace(card.Imsi) +} + +func prepareJSONValue(json *gjson.Json, keys ...string) string { + if json == nil { + return "" + } + for _, key := range keys { + if value := strings.TrimSpace(json.Get(key).String()); value != "" { + return value + } + } + return "" +} + +func uniquePrepareStrings(list []string) []string { + if len(list) == 0 { + return nil + } + seen := make(map[string]struct{}, len(list)) + out := make([]string, 0, len(list)) + for _, value := range list { + value = strings.TrimSpace(value) + if value == "" { + continue + } + if _, ok := seen[value]; ok { + continue + } + seen[value] = struct{}{} + out = append(out, value) + } + sort.Strings(out) + return out +} + +func (p *PrepareExecutor) resolvePrice(ctx context.Context, main *rc.BatchMainRow, _ prepareOptions, targetType int, curPkg, newPkg *rc.FlowPackageInfo, customer *customerInfo) (*RenewPriceSnapshot, *ChangePriceSnapshot, error) { + tenantID := main.TenantId + pkg := newPkg + if pkg == nil { + pkg = curPkg + } + if pkg == nil { + return nil, nil, gerror.New("缺失套餐信息") + } + unitPrice := 0.0 + priceSource := PriceSourceSystem + var ( + agentDefaultID int64 + targetID int64 + ) + priceOwnerDeptID := int64(0) + if targetType == consts.BatchTypeRenew { + if customer == nil || customer.DeptID <= 0 { + return nil, nil, gerror.New("renew customer department is missing") + } + dept, err := p.repo.Tables().LoadDept(ctx, tenantID, customer.DeptID) + if err != nil { + return nil, nil, gerror.Wrap(err, "load renew customer department failed") + } + priceOwnerDeptID = renewPriceOwnerDeptID(dept) + salePrice, err := resolveRenewEffectiveSalePrice(ctx, p.repo.Tables(), tenantID, priceOwnerDeptID, customer.DeptID, pkg) + if err != nil { + return nil, nil, gerror.Wrap(err, "calculate renew effective price failed") + } + unitPrice = salePrice.AgentShipmentPrice + priceSource = salePrice.Rule.PriceSource + agentDefaultID = salePrice.Rule.AgentDefaultPriceId + targetID = salePrice.Rule.TargetPriceId + } else { + priceOwnerDeptID = main.DeptId + if priceOwnerDeptID <= 0 { + priceOwnerDeptID = main.CreatedDept + } + salePrice, err := resolveRenewEffectiveSalePrice(ctx, p.repo.Tables(), tenantID, priceOwnerDeptID, customer.DeptID, pkg) + if err != nil { + return nil, nil, gerror.Wrap(err, "calculate change effective price failed") + } + unitPrice = salePrice.AgentShipmentPrice + priceSource = salePrice.Rule.PriceSource + agentDefaultID = salePrice.Rule.AgentDefaultPriceId + targetID = salePrice.Rule.TargetPriceId + } + unitPrice = rc.RoundAmount(unitPrice) + if targetType == consts.BatchTypeRenew { + opts, _ := renewableYearOptions(curPkg.RenewableDurationConfig) + durationUnit := renewDurationUnitForPackage(curPkg) + renewableCodes, renewablePackages, err := p.renewablePackageOptions(ctx, tenantID, customer.DeptID, priceOwnerDeptID, curPkg) + if err != nil { + return nil, nil, err + } + return &RenewPriceSnapshot{ + CustomerAgentId: customer.DeptID, + PriceOwnerDeptId: priceOwnerDeptID, + PackageCode: curPkg.PackageCode, + PackageName: curPkg.PackageName, + BillingMethod: curPkg.BillingMethod, + RenewDurationUnit: durationUnit, + RenewDurationUnitText: consts.RenewDurationUnitText(durationUnit), + PriceSource: priceSource, + AgentDefaultPriceId: agentDefaultID, + TargetPriceId: targetID, + RenewUnitPrice: unitPrice, + RenewableDurationOptions: opts, + RenewablePackageCodes: renewableCodes, + RenewablePackages: renewablePackages, + }, nil, nil + } + oldOpts, _ := renewableYearOptions(curPkg.RenewableDurationConfig) + newOpts, _ := changeDurationOptions(pkg.RenewableDurationConfig) + oldDurationUnit := renewDurationUnitForPackage(curPkg) + renewSnap := &RenewPriceSnapshot{ + CustomerAgentId: customer.DeptID, + PackageCode: curPkg.PackageCode, + PackageName: curPkg.PackageName, + BillingMethod: curPkg.BillingMethod, + RenewDurationUnit: oldDurationUnit, + RenewDurationUnitText: consts.RenewDurationUnitText(oldDurationUnit), + PriceSource: priceSource, + AgentDefaultPriceId: agentDefaultID, + TargetPriceId: targetID, + RenewUnitPrice: rc.RoundAmount(curPkg.CostPrice), + RenewableDurationOptions: oldOpts, + } + changeSnap := &ChangePriceSnapshot{ + CustomerAgentId: customer.DeptID, + NewPackageCode: newPkg.PackageCode, + NewPackageName: newPkg.PackageName, + PriceSource: priceSource, + AgentDefaultPriceId: agentDefaultID, + TargetPriceId: targetID, + ChangeUnitPrice: unitPrice, + RenewableDurationOptions: newOpts, + } + return renewSnap, changeSnap, nil +} + +func fillRenewBeginSnapshot(snapshot *RenewPriceSnapshot, rows []*RenewRow, rowErrors map[int]string) { + if snapshot == nil { + return + } + dates := make([]string, 0, len(rows)) + for _, row := range rows { + if row == nil || row.OldExpireTime == nil { + continue + } + if strings.TrimSpace(rowErrors[row.RowNo]) != "" { + continue + } + renewBeginAt := renewBeginDateFromExpire(row.OldExpireTime) + if renewBeginAt == nil { + continue + } + date := strings.TrimSpace(renewBeginAt.Format("Y-m-d")) + if date != "" { + dates = append(dates, date) + } + } + sameDate, minDate, maxDate, same := summarizeRenewBeginDates(dates) + snapshot.RenewBeginDate = sameDate + snapshot.RenewBeginDateMin = minDate + snapshot.RenewBeginDateMax = maxDate + snapshot.SameRenewBeginDate = same +} + +func validateRenewSameCurrentPeriod(rows []*RenewRow, rowErrors map[int]string) error { + var beginTime, expireTime string + hasBase := false + for _, row := range rows { + if row == nil { + continue + } + if strings.TrimSpace(rowErrors[row.RowNo]) != "" { + continue + } + rowBegin := formatTime(row.OldEffTime) + rowExpire := formatTime(row.OldExpireTime) + if rowBegin == "" || rowExpire == "" { + return gerror.New(renewSamePeriodFailureReason) + } + if !hasBase { + beginTime = rowBegin + expireTime = rowExpire + hasBase = true + continue + } + if rowBegin != beginTime || rowExpire != expireTime { + return gerror.New(renewSamePeriodFailureReason) + } + } + return nil +} + +func summarizeRenewBeginDates(dates []string) (sameDate string, minDate string, maxDate string, same bool) { + if len(dates) == 0 { + return "", "", "", false + } + sort.Strings(dates) + minDate = dates[0] + maxDate = dates[len(dates)-1] + same = minDate == maxDate + if same { + sameDate = minDate + } + return sameDate, minDate, maxDate, same +} + +func renewPriceOwnerDeptID(dept *rc.SysDeptRow) int64 { + if dept == nil { + return 0 + } + return dept.ParentId +} + +func renewExpireAfterAllowedWindow(expireAt *gtime.Time, now time.Time) bool { + if expireAt == nil { + return false + } + return expireAt.Time.After(now.AddDate(0, renewExpireWindowMonths, 0)) +} + +func priceByPackage(pkg *rc.FlowPackageInfo, override *rc.FlowPackagePrice) float64 { + if pkg == nil { + return 0 + } + ratio := pkg.SysDefaultRatio + profit := pkg.SysDefaultProfit + if override != nil { + ratio = override.AgentCostRatio + profit = override.AgentCostProfit + } + return rc.RoundAmount(pkg.CostPrice*normalizePriceRatio(ratio) + profit) +} + +func (p *PrepareExecutor) renewablePackageOptions(ctx context.Context, tenantID string, customerDeptID, priceOwnerDeptID int64, currentPkg *rc.FlowPackageInfo) ([]string, []RenewPackageOption, error) { + if currentPkg == nil { + return nil, nil, gerror.New("当前套餐不存在或未启用") + } + codes := renewPackageCodesFromJSON(currentPkg.RenewablePackageCodes) + if len(codes) == 0 { + return nil, nil, gerror.New("当前套餐未配置可续费套餐") + } + options := make([]RenewPackageOption, 0, len(codes)) + validCodes := make([]string, 0, len(codes)) + for _, code := range codes { + pkg, err := p.repo.Tables().LoadPackageByCode(ctx, tenantID, code) + if err != nil || pkg == nil || pkg.Enabled != 1 { + continue + } + unitPrice, err := p.renewPackageUnitPrice(ctx, tenantID, customerDeptID, priceOwnerDeptID, pkg) + if err != nil { + return nil, nil, err + } + durations, err := renewableYearOptions(pkg.RenewableDurationConfig) + if err != nil { + continue + } + validCodes = append(validCodes, pkg.PackageCode) + options = append(options, RenewPackageOption{ + PackageCode: pkg.PackageCode, + PackageName: pkg.PackageName, + RenewUnitPrice: unitPrice, + RenewableDurationOptions: durations, + }) + } + if len(options) == 0 { + return nil, nil, gerror.New("当前套餐未配置可用的可续费套餐") + } + return validCodes, options, nil +} + +func (p *PrepareExecutor) renewPackageUnitPrice(ctx context.Context, tenantID string, customerDeptID, priceOwnerDeptID int64, pkg *rc.FlowPackageInfo) (float64, error) { + if pkg == nil { + return 0, gerror.New("续费套餐不存在") + } + salePrice, err := resolveRenewEffectiveSalePrice(ctx, p.repo.Tables(), tenantID, priceOwnerDeptID, customerDeptID, pkg) + if err != nil { + return 0, gerror.Wrap(err, "calculate renew package effective price failed") + } + return salePrice.AgentShipmentPrice, nil +} + +func renewPackageCodesFromJSON(values *gjson.Json) []string { + if values == nil || values.IsNil() { + return nil + } + seen := make(map[string]struct{}) + codes := make([]string, 0, len(values.Interfaces())) + for _, value := range values.Interfaces() { + code := strings.TrimSpace(gconvString(value)) + if code == "" { + continue + } + if _, ok := seen[code]; ok { + continue + } + seen[code] = struct{}{} + codes = append(codes, code) + } + return codes +} + +func normalizePriceRatio(ratio float64) float64 { + if ratio > 10 { + return ratio / 100 + } + return ratio +} + +func (p *PrepareExecutor) finalizeBatchFailure(ctx context.Context, main *rc.BatchMainRow, reason string, perRow map[int]string) error { + g.Log().Warningf(ctx, "prepare batch failed, batchNo:%s, reason:%s", main.BatchNo, reason) + now := p.nowFunc() + err := p.repo.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + finalBatchType := main.BatchType + // 明细级失败 + if perRow != nil { + for rowNo, msg := range perRow { + if rowNo <= 0 || msg == "" { + continue + } + if err := p.repo.MarkDetailFailed(ctx, tx, main.BatchNo, main.BatchType, rowNo, msg, main.CreatedBy); err != nil { + return err + } + } + } + // 兜底：所有仍是 running 的明细批量置为失败 + if err := p.repo.MarkAllDetailsFailed(ctx, tx, main.BatchNo, main.BatchType, reason, main.CreatedBy); err != nil { + return err + } + if targetType := readTargetBatchType(main.BatchInfo); targetType == consts.BatchTypeRenew || targetType == consts.BatchTypeChange { + convertResult, err := p.repo.ConvertPrepareToRenewChange(ctx, tx, main.BatchNo, targetType) + if err != nil { + return err + } + g.Log().Warningf(ctx, "prepare failure converted to target batch, batchNo:%s, targetType:%d, mainAffected:%d, detailAffected:%d, reason:%s", + main.BatchNo, targetType, convertResult.MainAffected, convertResult.DetailAffected, reason) + finalBatchType = targetType + } + return p.repo.FinalizeBatchMain(ctx, tx, main.BatchNo, finalBatchType, &rc.BatchMainRow{ + BatchAsyncStatus: failedStatus(), + RunResult: consts.BatchRunResultFailed, + FailReason: reason, + SuccessCount: 0, + FailCount: main.TotalCount, + ValidCount: 0, + TotalCount: main.TotalCount, + EndTime: gtime.NewFromTime(now), + UpdatedBy: main.CreatedBy, + }) + }) + if err != nil { + return gerror.Wrap(err, "finalize prepare failure failed") + } + return nil +} + +func (p *PrepareExecutor) finalizeBatchSuccess(ctx context.Context, main *rc.BatchMainRow, targetType int, out *prepareOutcome, details []*rc.BatchDetailRow) error { + now := p.nowFunc() + info := buildPrepareInfo(main.BatchInfo, targetType, out, len(details)) + tenantID := main.TenantId + err := p.repo.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + for _, d := range details { + if d == nil { + continue + } + snap := buildPrepareDetailSnapshot(d, out, targetType) + itemID := "" + if row := out.rows[d.RowNo]; row != nil { + itemID = row.ICCID + } + if err := p.repo.MarkPrepareDetailSuccess(ctx, tx, main.BatchNo, d.RowNo, itemID, snap, main.CreatedBy); err != nil { + return err + } + } + // 批量将所有 running 置为 success（兜底覆盖） + if err := p.repo.MarkAllDetailsSuccess(ctx, tx, main.BatchNo, main.BatchType, main.CreatedBy, nil); err != nil { + return err + } + if err := p.repo.UpdateBatchMainInfo(ctx, tx, main.BatchNo, main.BatchType, &rc.BatchMainRow{ + BatchType: main.BatchType, + BatchAsyncStatus: consts.BatchAsyncStatusSuccess, + RunResult: consts.BatchRunResultSuccess, + TotalCount: main.TotalCount, + ValidCount: len(details), + SuccessCount: len(details), + FailCount: 0, + BatchInfo: gjson.New(info), + EndTime: gtime.NewFromTime(now), + UpdatedBy: main.CreatedBy, + }); err != nil { + return err + } + return nil + }) + if err != nil { + return gerror.Wrap(err, "finalize prepare success failed") + } + g.Log().Infof(ctx, "prepare batch success, batchNo:%s, targetType:%d, tenant:%s", main.BatchNo, targetType, tenantID) + return nil +} + +func buildPrepareInfo(prev *gjson.Json, targetType int, out *prepareOutcome, quantity int) map[string]any { + info := map[string]any{} + if prev != nil { + for k, v := range prev.Map() { + info[k] = v + } + } + info["targetBatchType"] = targetType + info["quantity"] = quantity + if out.customer != nil { + info["customerAgentId"] = out.customer.DeptID + info["customerName"] = out.customer.Name + } + if out.priceSnap != nil { + info["priceSnapshot"] = out.priceSnap + if targetType == consts.BatchTypeRenew { + info["packageCode"] = out.priceSnap.PackageCode + info["packageName"] = out.priceSnap.PackageName + info["billingMethod"] = out.priceSnap.BillingMethod + info["renewDurationUnit"] = out.priceSnap.RenewDurationUnit + info["renewDurationUnitText"] = out.priceSnap.RenewDurationUnitText + info["priceSource"] = out.priceSnap.PriceSource + info["priceOwnerDeptId"] = out.priceSnap.PriceOwnerDeptId + info["agentDefaultPriceId"] = out.priceSnap.AgentDefaultPriceId + info["targetPriceId"] = out.priceSnap.TargetPriceId + info["renewUnitPrice"] = out.priceSnap.RenewUnitPrice + info["renewableDurationOptions"] = out.priceSnap.RenewableDurationOptions + info["renewablePackageCodes"] = out.priceSnap.RenewablePackageCodes + info["renewablePackages"] = out.priceSnap.RenewablePackages + info["renewBeginDate"] = out.priceSnap.RenewBeginDate + info["renewBeginDateMin"] = out.priceSnap.RenewBeginDateMin + info["renewBeginDateMax"] = out.priceSnap.RenewBeginDateMax + info["sameRenewBeginDate"] = out.priceSnap.SameRenewBeginDate + } + } + if out.changePrice != nil { + info["changePriceSnapshot"] = out.changePrice + if targetType == consts.BatchTypeChange { + info["newPackageCode"] = out.changePrice.NewPackageCode + info["newPackageName"] = out.changePrice.NewPackageName + info["priceSource"] = out.changePrice.PriceSource + info["agentDefaultPriceId"] = out.changePrice.AgentDefaultPriceId + info["targetPriceId"] = out.changePrice.TargetPriceId + info["changeUnitPrice"] = out.changePrice.ChangeUnitPrice + info["renewableDurationOptions"] = out.changePrice.RenewableDurationOptions + } + } + if out.changeSnap != nil { + info["changePrepareSnapshot"] = out.changeSnap + if targetType == consts.BatchTypeChange { + info["sourceBatchNo"] = out.changeSnap.SourceBatchNo + info["oldPackageCode"] = out.changeSnap.OldPackageCode + info["oldPackageName"] = out.changeSnap.OldPackageName + info["oldPackageType"] = out.changeSnap.OldPackageType + info["oldPackageFlowMb"] = out.changeSnap.OldPackageFlowMb + if quantity <= 0 { + info["quantity"] = out.changeSnap.Quantity + } + } + } + if out.customer != nil { + info["customer"] = out.customer + } + if out.packageInfo != nil { + info["packageInfo"] = out.packageInfo + } + info["prepareTime"] = timeNow().UTC().Format("2006-01-02 15:04:05") + info["status"] = "success" + return info +} + +func buildPrepareDetailSnapshot(d *rc.BatchDetailRow, out *prepareOutcome, targetType int) *gjson.Json { + base := map[string]any{} + if d.Snapshot != nil { + for k, v := range d.Snapshot.Map() { + base[k] = v + } + } + base["rowNo"] = d.RowNo + base["success"] = true + if row := out.rows[d.RowNo]; row != nil { + base["iccid"] = row.ICCID + base["msisdn"] = row.MSISDN + base["imsi"] = row.IMSI + base["oldPackageCode"] = row.OldPkgCode + base["oldPackageName"] = row.OldPkgName + base["oldSubscriptionNo"] = row.OldBindSubNo + base["oldOrderSn"] = row.OldBindOrdNo + base["sourceBatchNo"] = row.SourceBatchNo + if out.changePrice != nil { + base["newPackageCode"] = out.changePrice.NewPackageCode + base["newPackageName"] = out.changePrice.NewPackageName + } + } + if out.priceSnap != nil { + base["priceSnapshot"] = out.priceSnap + } + if out.changePrice != nil { + base["changePriceSnapshot"] = out.changePrice + } + if out.changeSnap != nil { + base["changePrepareSnapshot"] = out.changeSnap + } + if out.customer != nil { + base["customer"] = out.customer + } + if out.packageInfo != nil { + base["packageInfo"] = out.packageInfo + } + base["targetBatchType"] = targetType + return gjson.New(base) +} + +func loadPrepareOptions(info *gjson.Json) prepareOptions { + opts := prepareOptions{priceScope: PriceScopeAgentDefault} + if info == nil { + return opts + } + opts.customerAgentID = info.Get("customerAgentId").Int64() + opts.targetAgentID = info.Get("targetAgentId").Int64() + opts.priceScope = int(info.Get("priceScope").Int64()) + if opts.priceScope == 0 { + opts.priceScope = PriceScopeAgentDefault + } + opts.newPackageCode = strings.TrimSpace(info.Get("newPackageCode").String()) + if v := firstPrepareOptionValue(info, "durationValue", "durationYears"); v != nil { + opts.durationValue = int(v.Int64()) + opts.durationSet = opts.durationValue > 0 + } else if v := firstPrepareOptionValue(info, "durationMonths"); v != nil { + opts.durationValue = normalizeLegacyRenewDurationValue(int(v.Int64())) + opts.durationSet = opts.durationValue > 0 + } + return opts +} + +func firstPrepareOptionValue(info *gjson.Json, paths ...string) *gvar.Var { + if info == nil { + return nil + } + for _, path := range paths { + value := info.Get(path) + if value != nil && !value.IsNil() { + return value + } + } + return nil +} + +func mergeRowErrors(perRow, rowErrors map[int]string) map[int]string { + if len(perRow) == 0 && len(rowErrors) == 0 { + return nil + } + out := make(map[int]string, len(perRow)+len(rowErrors)) + for rowNo, msg := range perRow { + if strings.TrimSpace(msg) != "" { + out[rowNo] = msg + } + } + for rowNo, msg := range rowErrors { + if strings.TrimSpace(msg) == "" { + continue + } + if _, ok := out[rowNo]; !ok { + out[rowNo] = msg + } + } + return out +} + +func readTargetBatchType(info *gjson.Json) int { + if info == nil { + return 0 + } + if v := info.Get("targetBatchType"); v != nil { + return int(v.Int64()) + } + if v := info.Get("target_batch_type"); v != nil { + return int(v.Int64()) + } + return 0 +} + +func failedStatus() int { return consts.BatchAsyncStatusFailed } + +func rcLocalize(key string) string { + switch key { + case "errCardNotFound": + return "ICCID不存在正式卡记录" + case "errMsisdnMismatch": + return "MSISDN与SIM卡记录不一致" + case "errImsiMismatch": + return "IMSI与SIM卡记录不一致" + case "errCardNoCurrentPackage": + return "ICCID没有当前套餐" + case "errMultiplePackages": + return "当前批次下存在多个套餐，不能统一处理" + case "errMultipleCustomers": + return "当前批次下存在多个客户，不能统一处理" + case "errPkgNotFound": + return "套餐不存在" + case "errPkgDisabled": + return "套餐未启用或不存在" + case "errDeptNotFound": + return "客户不存在" + case "errNoChangeablePackages": + return "当前套餐未配置可变更套餐范围" + case "errNewPackageNotInChange": + return "新套餐不在当前套餐可变更范围内" + case "errNewPackageSameAsOld": + return "新套餐不能与当前套餐相同" + case "errPackageTypeMismatch": + return "目标套餐类型与当前套餐类型不一致" + case "errTargetPkgNotFound": + return "目标套餐不存在" + case "errRenewDurationEmpty": + return "续费/变更套餐未配置可续费时长" + case "errRenewableDurationMiss": + return "续费时长不在当前套餐可续费范围内" + } + return key +} + +func appendIfEmpty(existing, addition string) string { + addition = strings.TrimSpace(addition) + if addition == "" { + return existing + } + if existing == "" { + return addition + } + return existing + "；" + addition +} + +func stringify(value any) string { + switch v := value.(type) { + case string: + return v + case int: + return strconv.Itoa(v) + case int32: + return strconv.FormatInt(int64(v), 10) + case int64: + return strconv.FormatInt(v, 10) + case float32: + return strconv.FormatFloat(float64(v), 'f', -1, 32) + case float64: + return strconv.FormatFloat(v, 'f', -1, 64) + } + return "" +} diff --git a/internal/service/renewchange/prepare_test.go b/internal/service/renewchange/prepare_test.go new file mode 100644 index 0000000..8e3951b --- /dev/null +++ b/internal/service/renewchange/prepare_test.go @@ -0,0 +1,495 @@ +package renewchange + +import ( + "context" + "testing" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" +) + +func TestPrepareExecutorCanReclaimStaleProcessingBatch(t *testing.T) { + now := time.Date(2026, 8, 26, 14, 0, 0, 0, time.Local) + executor := &PrepareExecutor{nowFunc: func() time.Time { return now }} + + tests := []struct { + name string + main *entity.BatchMain + canRun bool + }{ + { + name: "fresh processing batch", + main: &entity.BatchMain{ + BatchType: consts.BatchTypePrepare, + BatchAsyncStatus: consts.BatchAsyncStatusProcessing, + StartTime: gtime.NewFromTime(now.Add(-prepareProcessingRetryAfter + time.Second)), + }, + }, + { + name: "stale processing batch", + main: &entity.BatchMain{ + BatchType: consts.BatchTypePrepare, + BatchAsyncStatus: consts.BatchAsyncStatusProcessing, + StartTime: gtime.NewFromTime(now.Add(-prepareProcessingRetryAfter)), + }, + canRun: true, + }, + { + name: "missing start time", + main: &entity.BatchMain{ + BatchType: consts.BatchTypePrepare, + BatchAsyncStatus: consts.BatchAsyncStatusProcessing, + }, + canRun: true, + }, + { + name: "terminal batch", + main: &entity.BatchMain{ + BatchType: consts.BatchTypePrepare, + BatchAsyncStatus: consts.BatchAsyncStatusSuccess, + }, + }, + } + + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + got, err := executor.CanExecuteProcessing(context.Background(), test.main) + if err != nil { + t.Fatalf("CanExecuteProcessing() error = %v", err) + } + if got != test.canRun { + t.Fatalf("CanExecuteProcessing() = %v, want %v", got, test.canRun) + } + }) + } +} + +func TestLoadPrepareOptionsDurationIsExplicitOnly(t *testing.T) { + opts := loadPrepareOptions(gjson.New(map[string]any{ + "targetBatchType": 101, + })) + if opts.durationSet { + t.Fatalf("durationSet = true, want false") + } + if opts.durationValue != 0 { + t.Fatalf("durationValue = %d, want 0", opts.durationValue) + } + + opts = loadPrepareOptions(gjson.New(map[string]any{ + "durationMonths": 6, + })) + if !opts.durationSet { + t.Fatalf("durationSet = false, want true") + } + if opts.durationValue != 6 { + t.Fatalf("durationValue = %d, want 6", opts.durationValue) + } + + opts = loadPrepareOptions(gjson.New(map[string]any{ + "durationMonths": 12, + })) + if !opts.durationSet { + t.Fatalf("durationSet = false, want true") + } + if opts.durationValue != 1 { + t.Fatalf("durationValue = %d, want 1", opts.durationValue) + } +} + +func TestChangeDurationOptionsUseMonthsAtExecutionBoundary(t *testing.T) { + got, err := changeDurationOptions(gjson.New([]int{1, 2, 3})) + if err != nil { + t.Fatalf("changeDurationOptions(years) error = %v", err) + } + if want := []int{12, 24, 36}; len(got) != len(want) { + t.Fatalf("changeDurationOptions(years) = %#v, want %#v", got, want) + } else { + for i := range want { + if got[i] != want[i] { + t.Fatalf("changeDurationOptions(years) = %#v, want %#v", got, want) + } + } + } + + got, err = changeDurationOptions(gjson.New([]int{12, 24, 36})) + if err != nil { + t.Fatalf("changeDurationOptions(legacy months) error = %v", err) + } + if want := []int{12, 24, 36}; len(got) != len(want) { + t.Fatalf("changeDurationOptions(legacy months) = %#v, want %#v", got, want) + } else { + for i := range want { + if got[i] != want[i] { + t.Fatalf("changeDurationOptions(legacy months) = %#v, want %#v", got, want) + } + } + } +} + +func TestMergeRowErrorsKeepsValidationReasons(t *testing.T) { + got := mergeRowErrors( + map[int]string{2: "parse failed"}, + map[int]string{1: "duration not allowed", 2: "validation failed", 0: "batch failed"}, + ) + + if got[1] != "duration not allowed" { + t.Fatalf("row 1 reason = %q", got[1]) + } + if got[2] != "parse failed" { + t.Fatalf("row 2 reason = %q", got[2]) + } + if got[0] != "batch failed" { + t.Fatalf("row 0 reason = %q", got[0]) + } +} + +func TestResolvePrepareIdentityRowsFromCardsCanonicalizesMSISDN(t *testing.T) { + rows := []*RenewRow{ + {RowNo: 1, ICCID: "13800008000"}, + {RowNo: 2, ICCID: "8986002", MSISDN: "bad-msisdn"}, + } + rowErrors := map[int]string{} + + resolvePrepareIdentityRowsFromCards(rows, rowErrors, []*rc.SimCardInfo{ + {Id: 1, Iccid: "8986001", Msisdn: "13800008000", Imsi: "460001"}, + {Id: 2, Iccid: "8986002", Msisdn: "13800008002", Imsi: "460002"}, + }) + + if rows[0].ICCID != "8986001" || rows[0].MSISDN != "13800008000" || rows[0].IMSI != "460001" { + t.Fatalf("row 1 was not canonicalized: %#v", rows[0]) + } + if rowErrors[1] != "" { + t.Fatalf("row 1 error = %q, want empty", rowErrors[1]) + } + if rowErrors[2] == "" { + t.Fatal("row 2 should fail when ICCID and MSISDN point to different identities") + } +} + +func TestCollectPrepareIccidsMarksDuplicateAfterCanonicalization(t *testing.T) { + rows := []*RenewRow{ + {RowNo: 1, ICCID: "8986001"}, + {RowNo: 2, ICCID: "8986001"}, + } + rowErrors := map[int]string{} + + got := collectPrepareIccids(rows, rowErrors) + + if len(got) != 1 || got[0] != "8986001" { + t.Fatalf("iccids = %#v, want single canonical ICCID", got) + } + if rowErrors[1] == "" || rowErrors[2] == "" { + t.Fatalf("duplicate errors not marked: %#v", rowErrors) + } +} + +func TestBuildPrepareInfoWritesRenewFlatContract(t *testing.T) { + got := buildPrepareInfo(nil, 101, &prepareOutcome{ + customer: &customerInfo{ + AgentID: 9, + DeptID: 117, + Name: "customer", + }, + priceSnap: &RenewPriceSnapshot{ + CustomerAgentId: 117, + PackageCode: "PKG-1", + PackageName: "package 1", + PriceSource: PriceSourceSystem, + AgentDefaultPriceId: 11, + TargetPriceId: 22, + RenewUnitPrice: 3.5, + RenewableDurationOptions: []int{1, 3, 12}, + RenewBeginDate: "2026-08-31", + RenewBeginDateMin: "2026-08-31", + RenewBeginDateMax: "2026-08-31", + SameRenewBeginDate: true, + }, + }, 2) + + if got["customerAgentId"] != int64(117) { + t.Fatalf("customerAgentId = %v, want 117", got["customerAgentId"]) + } + if got["packageCode"] != "PKG-1" { + t.Fatalf("packageCode = %v, want PKG-1", got["packageCode"]) + } + if got["quantity"] != 2 { + t.Fatalf("quantity = %v, want 2", got["quantity"]) + } + if got["renewUnitPrice"] != 3.5 { + t.Fatalf("renewUnitPrice = %v, want 3.5", got["renewUnitPrice"]) + } + if got["priceSnapshot"] == nil { + t.Fatalf("priceSnapshot is nil") + } + if got["renewBeginDate"] != "2026-08-31" { + t.Fatalf("renewBeginDate = %v, want 2026-08-31", got["renewBeginDate"]) + } + if got["sameRenewBeginDate"] != true { + t.Fatalf("sameRenewBeginDate = %v, want true", got["sameRenewBeginDate"]) + } +} + +func TestFillRenewBeginSnapshotSummarizesRenewBeginDate(t *testing.T) { + snapshot := &RenewPriceSnapshot{} + fillRenewBeginSnapshot(snapshot, []*RenewRow{ + {RowNo: 1, OldExpireTime: gtime.NewFromStr("2026-09-10 00:00:00")}, + {RowNo: 2, OldExpireTime: gtime.NewFromStr("2026-08-31 23:59:59")}, + }, nil) + + if snapshot.RenewBeginDate != "" { + t.Fatalf("renewBeginDate = %q, want empty for mixed dates", snapshot.RenewBeginDate) + } + if snapshot.RenewBeginDateMin != "2026-08-31" { + t.Fatalf("renewBeginDateMin = %q, want 2026-08-31", snapshot.RenewBeginDateMin) + } + if snapshot.RenewBeginDateMax != "2026-09-10" { + t.Fatalf("renewBeginDateMax = %q, want 2026-09-10", snapshot.RenewBeginDateMax) + } + if snapshot.SameRenewBeginDate { + t.Fatalf("sameRenewBeginDate = true, want false") + } + + fillRenewBeginSnapshot(snapshot, []*RenewRow{ + {RowNo: 1, OldExpireTime: gtime.NewFromStr("2026-10-01 00:00:00")}, + {RowNo: 2, OldExpireTime: gtime.NewFromStr("2026-10-01 23:59:59")}, + }, nil) + + if snapshot.RenewBeginDate != "2026-10-01" { + t.Fatalf("renewBeginDate = %q, want 2026-10-01", snapshot.RenewBeginDate) + } + if !snapshot.SameRenewBeginDate { + t.Fatalf("sameRenewBeginDate = false, want true") + } +} + +func TestRenewBeginDateFromExpireUsesExpireDate(t *testing.T) { + beginAt := renewBeginDateFromExpire(gtime.NewFromStr("2026-08-15 00:00:00")) + if beginAt == nil { + t.Fatal("renew begin date is nil") + } + if got := beginAt.Format("Y-m-d H:i:s"); got != "2026-08-15 23:59:59" { + t.Fatalf("renew begin date = %s, want 2026-08-15 23:59:59", got) + } + if got := addMonths(beginAt.Time, 12).Format("2006-01-02 15:04:05"); got != "2027-08-15 23:59:59" { + t.Fatalf("renew end date = %s, want 2027-08-15 23:59:59", got) + } + + nextBeginAt := renewBeginDateFromExpire(gtime.NewFromStr("2027-07-31 00:00:00")) + if nextBeginAt == nil { + t.Fatal("next renew begin date is nil") + } + if got := nextBeginAt.Format("Y-m-d H:i:s"); got != "2027-07-31 23:59:59" { + t.Fatalf("next renew begin date = %s, want 2027-07-31 23:59:59", got) + } + + giftBeginAt := renewBeginDateFromExpire(gtime.NewFromStr("2026-08-31 00:00:00")) + if giftBeginAt == nil { + t.Fatal("gift renew begin date is nil") + } + if got := giftBeginAt.Format("Y-m-d H:i:s"); got != "2026-08-31 23:59:59" { + t.Fatalf("gift renew begin date = %s, want 2026-08-31 23:59:59", got) + } + if got := addMonths(giftBeginAt.Time, 13).Format("2006-01-02 15:04:05"); got != "2027-09-30 23:59:59" { + t.Fatalf("renew end date with gift month = %s, want 2027-09-30 23:59:59", got) + } +} + +func TestValidateRenewSameCurrentPeriod(t *testing.T) { + sameRows := []*RenewRow{ + { + RowNo: 1, + OldEffTime: gtime.NewFromStr("2026-01-01 00:00:00"), + OldExpireTime: gtime.NewFromStr("2026-12-31 23:59:59"), + }, + { + RowNo: 2, + OldEffTime: gtime.NewFromStr("2026-01-01 00:00:00"), + OldExpireTime: gtime.NewFromStr("2026-12-31 23:59:59"), + }, + } + if err := validateRenewSameCurrentPeriod(sameRows, nil); err != nil { + t.Fatalf("validateRenewSameCurrentPeriod() error = %v", err) + } + + diffBeginRows := []*RenewRow{ + { + RowNo: 1, + OldEffTime: gtime.NewFromStr("2026-01-01 00:00:00"), + OldExpireTime: gtime.NewFromStr("2026-12-31 23:59:59"), + }, + { + RowNo: 2, + OldEffTime: gtime.NewFromStr("2026-02-01 00:00:00"), + OldExpireTime: gtime.NewFromStr("2026-12-31 23:59:59"), + }, + } + if err := validateRenewSameCurrentPeriod(diffBeginRows, nil); err == nil { + t.Fatalf("validateRenewSameCurrentPeriod() error = nil, want begin mismatch") + } + + diffExpireRows := []*RenewRow{ + { + RowNo: 1, + OldEffTime: gtime.NewFromStr("2026-01-01 00:00:00"), + OldExpireTime: gtime.NewFromStr("2026-12-31 23:59:59"), + }, + { + RowNo: 2, + OldEffTime: gtime.NewFromStr("2026-01-01 00:00:00"), + OldExpireTime: gtime.NewFromStr("2027-01-31 23:59:59"), + }, + } + if err := validateRenewSameCurrentPeriod(diffExpireRows, nil); err == nil { + t.Fatalf("validateRenewSameCurrentPeriod() error = nil, want expire mismatch") + } + + missingTimeRows := []*RenewRow{ + { + RowNo: 1, + OldEffTime: gtime.NewFromStr("2026-01-01 00:00:00"), + OldExpireTime: gtime.NewFromStr("2026-12-31 23:59:59"), + }, + {RowNo: 2, OldEffTime: gtime.NewFromStr("2026-01-01 00:00:00")}, + } + if err := validateRenewSameCurrentPeriod(missingTimeRows, nil); err == nil { + t.Fatalf("validateRenewSameCurrentPeriod() error = nil, want missing time") + } + + rowErrors := map[int]string{2: "existing validation error"} + if err := validateRenewSameCurrentPeriod(missingTimeRows, rowErrors); err != nil { + t.Fatalf("validateRenewSameCurrentPeriod() with rowErrors error = %v", err) + } +} + +func TestRenewPriceOwnerUsesCurrentCustomerParent(t *testing.T) { + if got := renewPriceOwnerDeptID(&rc.SysDeptRow{DeptId: 117, ParentId: 100}); got != 100 { + t.Fatalf("price owner dept = %d, want 100", got) + } + if got := renewPriceOwnerDeptID(&rc.SysDeptRow{DeptId: 100, ParentId: 0}); got != 0 { + t.Fatalf("top-level price owner dept = %d, want 0", got) + } +} + +func TestRenewExpireAfterAllowedWindow(t *testing.T) { + now := time.Date(2026, 8, 7, 16, 50, 39, 0, time.Local) + + if renewExpireAfterAllowedWindow(nil, now) { + t.Fatal("nil expire time should not be rejected") + } + if renewExpireAfterAllowedWindow(gtime.NewFromTime(now.AddDate(0, 6, 0)), now) { + t.Fatal("exactly 6 months ahead should be allowed") + } + if !renewExpireAfterAllowedWindow(gtime.NewFromTime(now.AddDate(0, 6, 0).Add(time.Second)), now) { + t.Fatal("beyond 6 months should be rejected") + } +} + +func TestBuildPrepareInfoWritesChangeFlatContract(t *testing.T) { + got := buildPrepareInfo(nil, 102, &prepareOutcome{ + customer: &customerInfo{ + DeptID: 117, + Name: "customer", + }, + changeSnap: &ChangePrepareSnapshot{ + SourceBatchNo: "CHG-1", + CustomerAgentId: 117, + CustomerName: "customer", + OldPackageCode: "OLD", + OldPackageName: "old package", + OldPackageType: 101, + OldPackageFlowMb: 1024, + Quantity: 1, + }, + changePrice: &ChangePriceSnapshot{ + CustomerAgentId: 117, + NewPackageCode: "NEW", + NewPackageName: "new package", + PriceSource: PriceSourceAgentDefault, + AgentDefaultPriceId: 33, + TargetPriceId: 44, + ChangeUnitPrice: 6.5, + RenewableDurationOptions: []int{1, 2}, + }, + }, 1) + + if got["customerAgentId"] != int64(117) { + t.Fatalf("customerAgentId = %v, want 117", got["customerAgentId"]) + } + if got["sourceBatchNo"] != "CHG-1" { + t.Fatalf("sourceBatchNo = %v, want CHG-1", got["sourceBatchNo"]) + } + if got["oldPackageCode"] != "OLD" { + t.Fatalf("oldPackageCode = %v, want OLD", got["oldPackageCode"]) + } + if got["newPackageCode"] != "NEW" { + t.Fatalf("newPackageCode = %v, want NEW", got["newPackageCode"]) + } + if got["changeUnitPrice"] != 6.5 { + t.Fatalf("changeUnitPrice = %v, want 6.5", got["changeUnitPrice"]) + } + if got["changePrepareSnapshot"] == nil || got["changePriceSnapshot"] == nil { + t.Fatalf("change snapshots are missing") + } +} + +func TestBuildPrepareDetailSnapshotIncludesChangeSubscriptionContract(t *testing.T) { + out := &prepareOutcome{ + rows: map[int]*RenewRow{ + 1: { + RowNo: 1, + ICCID: "8986001", + MSISDN: "13800000000", + IMSI: "460001", + OldPkgCode: "OLD", + OldPkgName: "old package", + OldBindSubNo: "SUB-OLD", + OldBindOrdNo: "SO-OLD", + SourceBatchNo: "OUT-1", + }, + }, + changePrice: &ChangePriceSnapshot{ + NewPackageCode: "NEW", + NewPackageName: "new package", + }, + } + snapshot := buildPrepareDetailSnapshot(&rc.BatchDetailRow{RowNo: 1}, out, 102) + want := map[string]string{ + "oldPackageCode": "OLD", + "oldPackageName": "old package", + "oldSubscriptionNo": "SUB-OLD", + "oldOrderSn": "SO-OLD", + "sourceBatchNo": "OUT-1", + "newPackageCode": "NEW", + "newPackageName": "new package", + } + for key, expected := range want { + if got := snapshot.Get(key).String(); got != expected { + t.Fatalf("%s = %q, want %q", key, got, expected) + } + } +} + +func TestPriceByPackageUsesAdminContractFormula(t *testing.T) { + pkg := &rc.FlowPackageInfo{ + CostPrice: 10, + SysDefaultRatio: 120, + SysDefaultProfit: 1.23, + } + + if got := priceByPackage(pkg, nil); got != 13.23 { + t.Fatalf("system price = %v, want 13.23", got) + } + + got := priceByPackage(pkg, &rc.FlowPackagePrice{ + AgentCostRatio: 80, + AgentCostProfit: 2.5, + }) + if got != 10.5 { + t.Fatalf("override price = %v, want 10.5", got) + } +} diff --git a/internal/service/renewchange/price_effective.go b/internal/service/renewchange/price_effective.go new file mode 100644 index 0000000..e20e911 --- /dev/null +++ b/internal/service/renewchange/price_effective.go @@ -0,0 +1,146 @@ +package renewchange + +import ( + "context" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/errors/gerror" + + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" +) + +type renewEffectivePriceStore interface { + LoadDept(ctx context.Context, tenantID string, deptID int64) (*rc.SysDeptRow, error) + LoadPriceOverride(ctx context.Context, tenantID string, deptID, targetAgentID int64, scopeType int, packageCode string) (*rc.FlowPackagePrice, error) +} + +type renewEffectivePriceRule struct { + PriceSource string + AgentDefaultPriceId int64 + TargetPriceId int64 + AgentCostRatio float64 + AgentCostProfit float64 +} + +type renewEffectiveSalePrice struct { + BaseAgentCost float64 + Rule renewEffectivePriceRule + AgentShipmentPrice float64 +} + +type EffectivePriceStore = renewEffectivePriceStore +type EffectivePriceRule = renewEffectivePriceRule +type EffectiveSalePrice = renewEffectiveSalePrice + +type renewTxEffectivePriceStore struct { + tx gdb.TX +} + +func (s renewTxEffectivePriceStore) LoadDept(ctx context.Context, tenantID string, deptID int64) (*rc.SysDeptRow, error) { + return loadRenewDeptTx(ctx, s.tx, tenantID, deptID) +} + +func (s renewTxEffectivePriceStore) LoadPriceOverride(ctx context.Context, tenantID string, deptID, targetAgentID int64, scopeType int, packageCode string) (*rc.FlowPackagePrice, error) { + return loadRenewPriceOverrideTx(ctx, s.tx, tenantID, deptID, targetAgentID, scopeType, packageCode) +} + +func ResolveEffectiveSalePrice(ctx context.Context, store EffectivePriceStore, tenantID string, ownerDeptID int64, targetDeptID int64, pkg *rc.FlowPackageInfo) (EffectiveSalePrice, error) { + return resolveRenewEffectiveSalePrice(ctx, store, tenantID, ownerDeptID, targetDeptID, pkg) +} + +func resolveRenewEffectiveSalePrice(ctx context.Context, store renewEffectivePriceStore, tenantID string, ownerDeptID int64, targetDeptID int64, pkg *rc.FlowPackageInfo) (renewEffectiveSalePrice, error) { + baseCost, err := resolveRenewEffectiveBaseCost(ctx, store, tenantID, ownerDeptID, pkg, make(map[int64]struct{})) + if err != nil { + return renewEffectiveSalePrice{}, err + } + rule, err := resolveRenewEffectivePriceRule(ctx, store, tenantID, ownerDeptID, targetDeptID, pkg) + if err != nil { + return renewEffectiveSalePrice{}, err + } + return renewEffectiveSalePrice{ + BaseAgentCost: baseCost, + Rule: rule, + AgentShipmentPrice: rc.RoundAmount(baseCost*rule.AgentCostRatio + rule.AgentCostProfit), + }, nil +} + +func resolveRenewEffectiveBaseCost(ctx context.Context, store renewEffectivePriceStore, tenantID string, deptID int64, pkg *rc.FlowPackageInfo, visited map[int64]struct{}) (float64, error) { + if pkg == nil || strings.TrimSpace(pkg.PackageCode) == "" { + return 0, gerror.New("套餐编码不能为空") + } + if deptID <= 0 { + return rc.RoundAmount(pkg.CostPrice), nil + } + if _, ok := visited[deptID]; ok { + return 0, gerror.New("组织定价层级存在循环，无法计算生效成本") + } + visited[deptID] = struct{}{} + + dept, err := store.LoadDept(ctx, tenantID, deptID) + if err != nil { + return 0, err + } + if dept == nil || dept.ParentId <= 0 { + return rc.RoundAmount(pkg.CostPrice), nil + } + + parentBaseCost, err := resolveRenewEffectiveBaseCost(ctx, store, tenantID, dept.ParentId, pkg, visited) + if err != nil { + return 0, err + } + parentRule, err := resolveRenewEffectivePriceRule(ctx, store, tenantID, dept.ParentId, deptID, pkg) + if err != nil { + return 0, err + } + return rc.RoundAmount(parentBaseCost*parentRule.AgentCostRatio + parentRule.AgentCostProfit), nil +} + +func resolveRenewEffectivePriceRule(ctx context.Context, store renewEffectivePriceStore, tenantID string, ownerDeptID int64, targetDeptID int64, pkg *rc.FlowPackageInfo) (renewEffectivePriceRule, error) { + rule := renewSystemPriceRule(pkg) + if ownerDeptID <= 0 { + return rule, nil + } + if targetDeptID > 0 { + targetPrice, err := store.LoadPriceOverride(ctx, tenantID, ownerDeptID, targetDeptID, PriceScopeTargetAgent, pkg.PackageCode) + if err != nil { + return rule, err + } + if targetPrice != nil { + return renewRuleFromOverride(PriceSourceTarget, targetPrice), nil + } + } + defaultPrice, err := store.LoadPriceOverride(ctx, tenantID, ownerDeptID, 0, PriceScopeAgentDefault, pkg.PackageCode) + if err != nil { + return rule, err + } + if defaultPrice != nil { + return renewRuleFromOverride(PriceSourceAgentDefault, defaultPrice), nil + } + return rule, nil +} + +func renewSystemPriceRule(pkg *rc.FlowPackageInfo) renewEffectivePriceRule { + if pkg == nil { + return renewEffectivePriceRule{PriceSource: PriceSourceSystem} + } + return renewEffectivePriceRule{ + PriceSource: PriceSourceSystem, + AgentCostRatio: normalizePriceRatio(pkg.SysDefaultRatio), + AgentCostProfit: pkg.SysDefaultProfit, + } +} + +func renewRuleFromOverride(source string, price *rc.FlowPackagePrice) renewEffectivePriceRule { + rule := renewEffectivePriceRule{ + PriceSource: source, + AgentCostRatio: normalizePriceRatio(price.AgentCostRatio), + AgentCostProfit: price.AgentCostProfit, + } + if source == PriceSourceTarget { + rule.TargetPriceId = price.Id + } else { + rule.AgentDefaultPriceId = price.Id + } + return rule +} diff --git a/internal/service/renewchange/price_effective_test.go b/internal/service/renewchange/price_effective_test.go new file mode 100644 index 0000000..d020814 --- /dev/null +++ b/internal/service/renewchange/price_effective_test.go @@ -0,0 +1,85 @@ +package renewchange + +import ( + "context" + "testing" + + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" +) + +type fakeRenewEffectivePriceStore struct { + depts map[int64]*rc.SysDeptRow + prices map[renewEffectivePriceKey]*rc.FlowPackagePrice +} + +type renewEffectivePriceKey struct { + deptID int64 + targetAgentID int64 + scopeType int + packageCode string +} + +func (s fakeRenewEffectivePriceStore) LoadDept(ctx context.Context, tenantID string, deptID int64) (*rc.SysDeptRow, error) { + return s.depts[deptID], nil +} + +func (s fakeRenewEffectivePriceStore) LoadPriceOverride(ctx context.Context, tenantID string, deptID, targetAgentID int64, scopeType int, packageCode string) (*rc.FlowPackagePrice, error) { + return s.prices[renewEffectivePriceKey{ + deptID: deptID, + targetAgentID: targetAgentID, + scopeType: scopeType, + packageCode: packageCode, + }], nil +} + +func TestResolveRenewEffectiveSalePriceUsesRecursiveParentCost(t *testing.T) { + store := fakeRenewEffectivePriceStore{ + depts: map[int64]*rc.SysDeptRow{ + 100: {DeptId: 100, ParentId: 0}, + 117: {DeptId: 117, ParentId: 100}, + 131: {DeptId: 131, ParentId: 117}, + }, + prices: map[renewEffectivePriceKey]*rc.FlowPackagePrice{ + {deptID: 100, targetAgentID: 117, scopeType: PriceScopeTargetAgent, packageCode: "yd-100m-3y"}: { + Id: 86, + AgentCostRatio: 1, + AgentCostProfit: 4, + }, + {deptID: 117, targetAgentID: 0, scopeType: PriceScopeAgentDefault, packageCode: "yd-100m-3y"}: { + Id: 91, + AgentCostRatio: 1, + AgentCostProfit: 1, + }, + {deptID: 117, targetAgentID: 131, scopeType: PriceScopeTargetAgent, packageCode: "yd-100m-3y"}: { + Id: 92, + AgentCostRatio: 1, + AgentCostProfit: 5, + }, + }, + } + + got, err := resolveRenewEffectiveSalePrice( + context.Background(), + store, + "000000", + 117, + 131, + &rc.FlowPackageInfo{ + PackageCode: "yd-100m-3y", + CostPrice: 10, + SysDefaultRatio: 1, + }, + ) + if err != nil { + t.Fatalf("resolveRenewEffectiveSalePrice() error = %v", err) + } + if got.BaseAgentCost != 14 { + t.Fatalf("base agent cost = %.2f, want 14.00", got.BaseAgentCost) + } + if got.AgentShipmentPrice != 19 { + t.Fatalf("agent shipment price = %.2f, want 19.00", got.AgentShipmentPrice) + } + if got.Rule.PriceSource != PriceSourceTarget || got.Rule.TargetPriceId != 92 { + t.Fatalf("rule = %+v, want target price 92", got.Rule) + } +} diff --git a/internal/service/renewchange/renew.go b/internal/service/renewchange/renew.go new file mode 100644 index 0000000..d0df3f4 --- /dev/null +++ b/internal/service/renewchange/renew.go @@ -0,0 +1,576 @@ +package renewchange + +import ( + "context" + "errors" + "strings" + "time" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" +) + +var errRenewAlreadyHandled = errors.New("renew batch was already handled") + +// RenewExecutor executes a confirmed type=101 batch. All business writes and +// the success state are committed by one database transaction. +type RenewExecutor struct { + repo *Repository + nowFunc NowFunc +} + +func NewRenewExecutor(repo *Repository) *RenewExecutor { + if repo == nil { + repo = NewRepository() + } + return &RenewExecutor{repo: repo, nowFunc: timeNow} +} + +func (r *RenewExecutor) SetNowFunc(now NowFunc) { r.nowFunc = now } + +func (r *RenewExecutor) Execute(ctx context.Context, batchNo string) error { + batchNo = strings.TrimSpace(batchNo) + if batchNo == "" { + return gerror.New("renew batch number is empty") + } + + main, err := r.repo.LoadProcessingBatch(ctx, batchNo, consts.BatchTypeRenew) + if err != nil { + return err + } + if strings.TrimSpace(main.OrderSn) != "" { + g.Log().Warningf(ctx, "renew batch already has order, skip duplicate: batchNo=%s orderSn=%s", batchNo, main.OrderSn) + return nil + } + details, err := r.repo.LoadRunningDetails(ctx, batchNo, consts.BatchTypeRenew) + if err != nil { + return gerror.Wrap(err, "load renew details failed") + } + if len(details) == 0 { + return r.finalizeFailure(ctx, main, "renew batch has no runnable details") + } + now := pnow(r.nowFunc) + orderSN, err := r.apply(ctx, main, details, now) + if errors.Is(err, errRenewAlreadyHandled) { + return nil + } + if err != nil { + return r.finalizeFailure(ctx, main, err.Error()) + } + + snap, _ := r.repo.ResolveResultSnapshot(ctx, main.BatchNo, main.BatchType) + _ = r.repo.WriteResultFileSafe(ctx, main.BatchNo, main.BatchType, main.TenantId, main.CreatedDept, main.CreatedBy, snap) + g.Log().Infof(ctx, "renew batch success, batchNo=%s, orderSn=%s, tenant=%s", main.BatchNo, orderSN, main.TenantId) + return nil +} + +type renewExecution struct { + detail *rc.BatchDetailRow + card *rc.SimCardInfo + bind *rc.SimCardBindPkg + begin time.Time + end time.Time +} + +func (r *RenewExecutor) apply(ctx context.Context, main *rc.BatchMainRow, details []*rc.BatchDetailRow, now time.Time) (string, error) { + iccids, err := strictIccidsFromDetails(details) + if err != nil { + return "", err + } + var orderSN string + err = r.repo.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + lockedMain, err := r.repo.Tables().LockRenewBatchMain(ctx, tx, main.BatchNo, consts.BatchTypeRenew) + if err != nil { + return err + } + if strings.TrimSpace(lockedMain.OrderSn) != "" || lockedMain.BatchAsyncStatus != consts.BatchAsyncStatusProcessing { + return errRenewAlreadyHandled + } + lockedDetails, err := r.repo.Tables().LockRenewDetails(ctx, tx, main.BatchNo, consts.BatchTypeRenew) + if err != nil { + return err + } + if err = validateRunningRenewDetails(lockedDetails, details); err != nil { + return err + } + details = lockedDetails + + cards, err := r.repo.Tables().LockCardsByIccids(ctx, tx, iccids) + if err != nil { + return err + } + bindRows, err := r.repo.Tables().LockRenewBindRowsByIccids(ctx, tx, iccids) + if err != nil { + return err + } + binds, futureBinds, err := resolveCurrentSubscriptions(bindRows, iccids) + if err != nil { + return err + } + contract, err := r.ensureRenewExecutionContract(ctx, tx, lockedMain, details, cards, binds, futureBinds, now) + if err != nil { + return err + } + executions, err := validateRenewExecutions(contract, details, cards, binds, futureBinds, now) + if err != nil { + return err + } + pkg, err := r.repo.Tables().LoadRenewPackage(ctx, tx, contract.TenantID, contract.PackageCode) + if err != nil { + return err + } + + orderSN, err = rc.NextOrderSN(now) + if err != nil { + return err + } + order, err := BuildOrderDO(contract, orderSN, now) + if err != nil { + return err + } + orderID, err := r.repo.Tables().InsertContractOrder(ctx, tx, order) + if err != nil { + return gerror.Wrap(err, "insert renew order failed") + } + minBegin, maxEnd := renewServiceRange(executions) + item, err := BuildOrderItemDO(contract, OrderProductContract{ + ID: pkg.Id, + Code: pkg.PackageCode, + Name: pkg.PackageName, + Type: pkg.PackageType, + }, orderID, orderSN, minBegin, maxEnd, now) + if err != nil { + return err + } + if err = r.repo.Tables().InsertContractOrderItem(ctx, tx, item); err != nil { + return gerror.Wrap(err, "insert renew order item failed") + } + + for _, execution := range executions { + subscriptionNo, err := generateRenewSubscriptionNo(now) + if err != nil { + return err + } + newBind := buildRenewBind(contract, pkg, execution, orderSN, subscriptionNo, now) + if err = r.repo.Tables().InsertBind(ctx, tx, newBind); err != nil { + return err + } + packageStatus := renewPackageStatus(now, execution.card.FlowPkgBeginAt, newBind.ExpectedExpireTime) + cardUpdate, err := BuildRenewCardUpdate(contract, execution.end, packageStatus, now) + if err != nil { + return err + } + if err = r.repo.Tables().UpdateRenewCard(ctx, tx, execution.card.Iccid, cardUpdate); err != nil { + return err + } + snapshot := buildRenewDetailSnapshot(execution, contract, pkg, orderSN, subscriptionNo) + if err = r.repo.MarkDetailSuccess(ctx, tx, main.BatchNo, consts.BatchTypeRenew, execution.detail.RowNo, snapshot, contract.OperatorUserID); err != nil { + return err + } + } + + info := mergeBatchInfo(lockedMain.BatchInfo, map[string]any{ + "orderSn": orderSN, + "packageCode": contract.PackageCode, + "packageName": pkg.PackageName, + "packageType": pkg.PackageType, + "billingMethod": contract.BillingMethod, + "renewDurationUnit": contract.RenewDurationUnit, + "renewDurationUnitText": contract.RenewDurationUnitText, + "durationValue": contract.DisplayDurationValue(), + "actualDurationMonths": contract.DurationMonths, + "durationMonths": contract.DurationMonths, + "giftDurationMonths": contract.GiftDurationMonths, + "amount": contract.TotalAmount(), + "quantity": contract.Quantity, + "renewTime": now.Format("2006-01-02 15:04:05"), + }) + if err = r.repo.UpdateBatchMainInfo(ctx, tx, main.BatchNo, consts.BatchTypeRenew, &rc.BatchMainRow{ + BatchType: consts.BatchTypeRenew, + BatchAsyncStatus: consts.BatchAsyncStatusSuccess, + RunResult: consts.BatchRunResultSuccess, + TotalCount: lockedMain.TotalCount, + ValidCount: len(executions), + SuccessCount: len(executions), + FailCount: 0, + BatchInfo: gjson.New(info), + OrderSn: orderSN, + StartTime: lockedMain.StartTime, + EndTime: gtime.NewFromTime(now), + TargetDeptId: contract.CustomerAgentID, + UpdatedBy: contract.OperatorUserID, + }); err != nil { + return err + } + return nil + }) + return orderSN, err +} + +func validateRenewExecutions(contract *ExecutionContract, details []*rc.BatchDetailRow, cards map[string]*rc.SimCardInfo, binds map[string]*rc.SimCardBindPkg, futureBinds map[string][]*rc.SimCardBindPkg, now time.Time) ([]renewExecution, error) { + if contract == nil { + return nil, gerror.New("renew execution contract is missing") + } + if contract.Quantity != len(details) { + return nil, gerror.New("renew detail quantity does not match confirmed contract") + } + seen := make(map[string]struct{}, len(details)) + result := make([]renewExecution, 0, len(details)) + for _, detail := range details { + if detail == nil { + return nil, gerror.New("renew detail is nil") + } + iccid := strings.TrimSpace(detail.ItemId) + if iccid == "" { + return nil, gerror.New("renew detail ICCID is empty") + } + if _, exists := seen[iccid]; exists { + return nil, gerror.Newf("renew detail ICCID is duplicated: %s", iccid) + } + seen[iccid] = struct{}{} + card := cards[iccid] + if card == nil { + return nil, gerror.Newf("renew card not found: %s", iccid) + } + bind := binds[iccid] + if bind == nil { + return nil, gerror.Newf("renew latest subscription not found: %s", iccid) + } + var expectedMSISDN, expectedIMSI string + if detail.Snapshot != nil { + expectedMSISDN = strings.TrimSpace(detail.Snapshot.Get("msisdn").String()) + expectedIMSI = strings.TrimSpace(detail.Snapshot.Get("imsi").String()) + } + if err := validateCurrentSubscription(card, bind, subscriptionValidation{ + ICCID: iccid, + MSISDN: expectedMSISDN, + IMSI: expectedIMSI, + TenantID: contract.TenantID, + CustomerDeptID: contract.CustomerAgentID, + PackageCode: contract.PackageCode, + SourceBatchNo: contract.SourceBatchNo, + }); err != nil { + return nil, gerror.Wrapf(err, "validate renew current subscription failed, iccid:%s", iccid) + } + // The latest subscription is the source of truth for the next renewal + // start. The legacy Admin transaction did not require the denormalized + // card expiration to equal that subscription; future subscriptions can + // legitimately make the two values differ. + renewBase := renewBaseSubscriptionForCard(card, bind, futureBinds[iccid]) + if err := validateRenewBaseWindow(renewBase, now, iccid); err != nil { + return nil, err + } + begin, ok := renewBeginTimeFromExpire(renewBase.ExpectedExpireTime) + if !ok { + return nil, gerror.Newf("ICCID没有预计到期时间：%s", strings.TrimSpace(iccid)) + } + if contract.PackageBeginTime != nil { + begin = contract.PackageBeginTime.Time + } + result = append(result, renewExecution{ + detail: detail, + card: card, + bind: renewBase, + begin: begin, + end: contract.ServiceEnd(begin), + }) + } + return result, nil +} + +func renewBaseSubscription(current *rc.SimCardBindPkg, future []*rc.SimCardBindPkg) *rc.SimCardBindPkg { + base := current + for _, bind := range future { + if bind == nil { + continue + } + if base == nil || subscriptionAfter(bind, base) { + base = bind + } + } + return base +} + +func renewBaseSubscriptionForCard(card *rc.SimCardInfo, current *rc.SimCardBindPkg, future []*rc.SimCardBindPkg) *rc.SimCardBindPkg { + base := renewBaseSubscription(current, future) + cardExpireAt := latestCardRenewExpireTime(card) + if cardExpireAt == nil { + return base + } + if base == nil { + return &rc.SimCardBindPkg{ + Iccid: strings.TrimSpace(card.Iccid), + PackageCode: strings.TrimSpace(card.CurrentPackageCode), + ExpectedExpireTime: cardExpireAt, + } + } + if base.ExpectedExpireTime != nil && !cardExpireAt.After(base.ExpectedExpireTime) { + return base + } + clone := *base + clone.ExpectedExpireTime = cardExpireAt + return &clone +} + +func latestCardRenewExpireTime(card *rc.SimCardInfo) *gtime.Time { + if card == nil { + return nil + } + var expireAt *gtime.Time + for _, candidate := range []*gtime.Time{ + card.FlowPkgEndAt, + card.FlowPkgPresentAt, + card.FlowPkgRealPresent, + } { + if candidate == nil || candidate.IsZero() { + continue + } + if expireAt == nil || candidate.After(expireAt) { + expireAt = candidate + } + } + return expireAt +} + +func validateRenewBaseWindow(base *rc.SimCardBindPkg, now time.Time, iccid string) error { + if base == nil || base.ExpectedExpireTime == nil { + return gerror.Newf("ICCID没有预计到期时间：%s", strings.TrimSpace(iccid)) + } + if renewExpireAfterAllowedWindow(base.ExpectedExpireTime, now) { + return gerror.Newf("%s：%s", renewExpireFailureReason, strings.TrimSpace(iccid)) + } + return nil +} + +func validateRunningRenewDetails(locked, loaded []*rc.BatchDetailRow) error { + if len(locked) != len(loaded) { + return gerror.New("renew detail count changed during execution") + } + for i, detail := range locked { + if detail == nil || detail.DetailStatus != consts.BatchDetailStatusRunning { + return gerror.New("renew detail is no longer running") + } + loadedDetail := loaded[i] + if loadedDetail == nil || detail.Id != loadedDetail.Id || detail.RowNo != loadedDetail.RowNo || + strings.TrimSpace(detail.ItemId) != strings.TrimSpace(loadedDetail.ItemId) { + return gerror.New("renew detail changed during execution") + } + } + return nil +} + +func renewServiceRange(executions []renewExecution) (time.Time, time.Time) { + minBegin := executions[0].begin + maxEnd := executions[0].end + for _, execution := range executions[1:] { + if execution.begin.Before(minBegin) { + minBegin = execution.begin + } + if execution.end.After(maxEnd) { + maxEnd = execution.end + } + } + return minBegin, maxEnd +} + +func generateRenewSubscriptionNo(now time.Time) (string, error) { + orderSN, err := rc.NextOrderSN(now) + if err != nil { + return "", err + } + return "SUB" + strings.TrimPrefix(orderSN, "SO"), nil +} + +func buildRenewBind(contract *ExecutionContract, pkg *rc.FlowPackageInfo, execution renewExecution, orderSN, subscriptionNo string, now time.Time) *rc.SimCardBindPkg { + remark := contract.Remark + if remark == "" { + remark = "card renew subscription" + } + return &rc.SimCardBindPkg{ + SubscriptionNo: subscriptionNo, + BatchNo: contract.BatchNo, + Iccid: execution.card.Iccid, + Msisdn: execution.card.Msisdn, + DeptId: contract.CustomerAgentID, + PackageType: pkg.PackageType, + PackageCode: pkg.PackageCode, + PreviousPackageCode: pkg.PackageCode, + BusinessType: consts.PackageBusinessTypeRenew, + EffectiveStatus: consts.EffectiveStatusPending, + HandleTime: gtime.NewFromTime(now), + EffectiveTime: gtime.NewFromTime(execution.begin), + ExpectedExpireTime: gtime.NewFromTime(execution.end), + DurationMonths: contract.DurationMonths, + GiftDurationMonths: contract.GiftDurationMonths, + OrderCode: orderSN, + Remark: remark, + TenantId: contract.TenantID, + CreatedDept: contract.CreatedDeptID, + CreatedBy: contract.OperatorUserID, + CreatedAt: gtime.NewFromTime(now), + UpdatedBy: contract.OperatorUserID, + UpdatedAt: gtime.NewFromTime(now), + } +} + +func buildRenewDetailSnapshot(execution renewExecution, contract *ExecutionContract, pkg *rc.FlowPackageInfo, orderSN, subscriptionNo string) *gjson.Json { + data := map[string]any{} + if execution.detail.Snapshot != nil { + for key, value := range execution.detail.Snapshot.Map() { + data[key] = value + } + } + data["iccid"] = execution.card.Iccid + data["msisdn"] = execution.card.Msisdn + data["imsi"] = execution.card.Imsi + data["renewBatchNo"] = contract.BatchNo + data["packageCode"] = pkg.PackageCode + data["packageName"] = pkg.PackageName + data["oldSubscriptionNo"] = execution.bind.SubscriptionNo + data["newSubscriptionNo"] = subscriptionNo + data["orderSn"] = orderSN + data["oldExpectedExpireTime"] = formatRenewTime(execution.bind.ExpectedExpireTime) + data["renewEffectiveTime"] = formatRenewTime(gtime.NewFromTime(execution.begin)) + data["renewExpectedExpireTime"] = formatRenewTime(gtime.NewFromTime(execution.end)) + data["cardFlowPkgEndAt"] = formatRenewTime(gtime.NewFromTime(execution.end)) + data["durationValue"] = contract.DisplayDurationValue() + data["actualDurationMonths"] = contract.DurationMonths + data["durationMonths"] = contract.DurationMonths + data["giftDurationMonths"] = contract.GiftDurationMonths + data["billingMethod"] = contract.BillingMethod + data["renewDurationUnit"] = contract.RenewDurationUnit + data["renewDurationUnitText"] = contract.RenewDurationUnitText + data["renewDate"] = contract.RenewDate + data["amount"] = contract.TotalAmount() + data["renewUnitPrice"] = contract.UnitPrice + data["priceSource"] = contract.PriceSource + data["rowNo"] = execution.detail.RowNo + if contract.SourceBatchNo != "" { + data["sourceBatchNo"] = contract.SourceBatchNo + } + return gjson.New(data) +} + +func formatRenewTime(value *gtime.Time) string { + if value == nil { + return "" + } + return value.Format("Y-m-d H:i:s") +} + +func renewPackageStatus(now time.Time, begin, end *gtime.Time) int { + if begin != nil && begin.After(gtime.NewFromTime(now)) { + return consts.PackageStatusPending + } + if end != nil && !end.After(gtime.NewFromTime(now)) { + return consts.PackageStatusExpired + } + return consts.PackageStatusEffective +} + +func strictIccidsFromDetails(details []*rc.BatchDetailRow) ([]string, error) { + seen := make(map[string]struct{}, len(details)) + result := make([]string, 0, len(details)) + for _, detail := range details { + if detail == nil { + return nil, gerror.New("renew detail is nil") + } + iccid := strings.TrimSpace(detail.ItemId) + if iccid == "" { + return nil, gerror.New("renew detail ICCID is empty") + } + if _, exists := seen[iccid]; exists { + return nil, gerror.Newf("renew detail ICCID is duplicated: %s", iccid) + } + seen[iccid] = struct{}{} + result = append(result, iccid) + } + return result, nil +} + +// uniqueIccidsFromDetails remains available to the change executor while the +// renew path uses strictIccidsFromDetails to reject malformed input. +func uniqueIccidsFromDetails(details []*rc.BatchDetailRow) []string { + seen := make(map[string]struct{}, len(details)) + result := make([]string, 0, len(details)) + for _, detail := range details { + if detail == nil { + continue + } + iccid := strings.TrimSpace(detail.ItemId) + if iccid == "" { + continue + } + if _, exists := seen[iccid]; exists { + continue + } + seen[iccid] = struct{}{} + result = append(result, iccid) + } + return result +} + +func (r *RenewExecutor) finalizeFailure(ctx context.Context, main *rc.BatchMainRow, reason string) error { + now := pnow(r.nowFunc) + contract, _ := ParseExecutionContract(main) + if contract == nil { + contract = renewRefundContractFromBatch(main) + } + nowTime := gtime.NewFromTime(now) + err := r.repo.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + lockedMain, err := r.repo.Tables().LockRenewBatchMain(ctx, tx, main.BatchNo, main.BatchType) + if err != nil { + return err + } + if contract != nil { + if err := refundRenewBalanceIfNeeded(ctx, tx, lockedMain, contract, reason, nowTime); err != nil { + return err + } + } + if err := r.repo.MarkAllDetailsFailed(ctx, tx, main.BatchNo, main.BatchType, reason, main.CreatedBy); err != nil { + return err + } + return r.repo.FinalizeBatchMain(ctx, tx, main.BatchNo, main.BatchType, &rc.BatchMainRow{ + BatchAsyncStatus: consts.BatchAsyncStatusFailed, + RunResult: consts.BatchRunResultFailed, + TotalCount: main.TotalCount, + FailCount: main.TotalCount, + FailReason: reason, + EndTime: nowTime, + UpdatedBy: main.CreatedBy, + }) + }) + if err != nil { + return gerror.Wrap(err, "finalize renew failure failed") + } + snap, _ := r.repo.ResolveResultSnapshot(ctx, main.BatchNo, main.BatchType) + _ = r.repo.WriteResultFileSafe(ctx, main.BatchNo, main.BatchType, main.TenantId, main.CreatedDept, main.CreatedBy, snap) + g.Log().Warningf(ctx, "renew batch failed, batchNo=%s, reason=%s, tenant=%s", main.BatchNo, reason, main.TenantId) + return nil +} + +func mergeBatchInfo(prev *gjson.Json, addition map[string]any) map[string]any { + out := map[string]any{} + if prev != nil { + for key, value := range prev.Map() { + out[key] = value + } + } + for key, value := range addition { + out[key] = value + } + return out +} + +func pnow(fn NowFunc) time.Time { + if fn == nil { + return timeNow() + } + return fn() +} diff --git a/internal/service/renewchange/renew_balance_test.go b/internal/service/renewchange/renew_balance_test.go new file mode 100644 index 0000000..be865af --- /dev/null +++ b/internal/service/renewchange/renew_balance_test.go @@ -0,0 +1,108 @@ +package renewchange + +import ( + "context" + "path/filepath" + "strings" + "testing" + "time" + + _ "github.com/gogf/gf/contrib/drivers/sqlite/v2" + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" + + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" +) + +func TestDeductRuntimeRenewBalanceRejectsInsufficientBalanceAtomically(t *testing.T) { + ctx := context.Background() + if err := gdb.SetConfig(gdb.Config{ + "default": gdb.ConfigGroup{{Type: "sqlite", Name: filepath.Join(t.TempDir(), "renew-balance.db")}}, + }); err != nil { + t.Fatalf("configure sqlite: %v", err) + } + defer g.DB().Close(ctx) + + createRenewBalanceTables(t, ctx) + if _, err := g.DB().Model(rc.TableAgentBalance).Ctx(ctx).Data(g.Map{ + "dept_id": 117, + "tenant_id": "000000", + "balance": 100, + "status": renewFinanceAccountStatusNormal, + }).Insert(); err != nil { + t.Fatalf("seed balance: %v", err) + } + + main := &rc.BatchMainRow{ + BatchNo: "REN20260806110000TEST", + TenantId: "000000", + CreatedBy: 60, + } + err := g.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + _, _, _, err := deductRuntimeRenewBalance(ctx, tx, main, 117, 576, time.Date(2026, 8, 6, 11, 0, 0, 0, time.Local)) + return err + }) + if err == nil || !strings.Contains(err.Error(), "账户余额不足") { + t.Fatalf("deduct error = %v, want insufficient balance", err) + } + + balance, err := g.DB().Model(rc.TableAgentBalance).Ctx(ctx). + Where("dept_id", 117). + Where("tenant_id", "000000"). + Value("balance") + if err != nil { + t.Fatalf("query balance: %v", err) + } + if got := balance.Float64(); got != 100 { + t.Fatalf("balance = %.2f, want unchanged 100.00", got) + } + + count, err := g.DB().Model(rc.TableBalanceDetail).Ctx(ctx). + Where("serial_no", "RENPAY"+main.BatchNo). + Count() + if err != nil { + t.Fatalf("count balance detail: %v", err) + } + if count != 0 { + t.Fatalf("balance detail count = %d, want 0", count) + } +} + +func createRenewBalanceTables(t *testing.T, ctx context.Context) { + t.Helper() + statements := []string{ + `CREATE TABLE t_agent_balance ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + dept_id INTEGER NOT NULL, + tenant_id TEXT NOT NULL, + balance REAL NOT NULL, + status INTEGER NOT NULL DEFAULT 0, + updated_by INTEGER, + updated_at DATETIME + )`, + `CREATE TABLE t_balance_detail ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + dept_id INTEGER NOT NULL, + serial_no TEXT NOT NULL, + transaction_time DATETIME, + income_expense_type INTEGER, + transaction_type INTEGER, + transaction_channel INTEGER, + transaction_status INTEGER, + amount REAL, + before_balance REAL, + balance_after REAL, + order_sn TEXT, + remark TEXT, + tenant_id TEXT NOT NULL, + created_dept INTEGER, + created_by INTEGER, + created_at DATETIME + )`, + } + for _, statement := range statements { + if _, err := g.DB().Exec(ctx, statement); err != nil { + t.Fatalf("create renew balance table: %v", err) + } + } +} diff --git a/internal/service/renewchange/renew_contract_runtime.go b/internal/service/renewchange/renew_contract_runtime.go new file mode 100644 index 0000000..4f4ab64 --- /dev/null +++ b/internal/service/renewchange/renew_contract_runtime.go @@ -0,0 +1,632 @@ +package renewchange + +import ( + "context" + "database/sql" + "errors" + "fmt" + "strings" + "time" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" +) + +const ( + renewFinanceAccountStatusNormal = 0 + renewFinanceIncomeExpenseExpense = 1 +) + +type renewPaymentAccountRow struct { + ID int64 `orm:"id"` + Balance float64 `orm:"balance"` + Status int `orm:"status"` +} + +func (r *RenewExecutor) ensureRenewExecutionContract( + ctx context.Context, + tx gdb.TX, + main *rc.BatchMainRow, + details []*rc.BatchDetailRow, + cards map[string]*rc.SimCardInfo, + binds map[string]*rc.SimCardBindPkg, + futureBinds map[string][]*rc.SimCardBindPkg, + now time.Time, +) (*ExecutionContract, error) { + if contract, err := ParseExecutionContract(main); err == nil { + if err = r.validateRenewRuntimePermissions(ctx, tx, main, contract); err != nil { + return nil, err + } + return contract, nil + } + + info := main.BatchInfo + if info == nil || info.IsNil() { + return nil, gerror.New("续费批次缺少导入参数") + } + durationValue := firstInt(info, "durationValue", "durationYears") + if durationValue <= 0 { + durationValue = normalizeLegacyRenewDurationValue(firstInt(info, "durationMonths")) + } + if durationValue <= 0 { + return nil, gerror.New("续费时长不能为空") + } + giftDurationMonths := firstInt(info, "giftDurationMonths") + if giftDurationMonths < 0 { + return nil, gerror.New("赠送时长不能小于0") + } + + iccids, err := strictIccidsFromDetails(details) + if err != nil { + return nil, err + } + customerID, packageCode, beginDates, err := runtimeRenewFacts(main, details, cards, binds, futureBinds, now) + if err != nil { + return nil, err + } + pkg, err := r.repo.Tables().LoadRenewPackage(ctx, tx, main.TenantId, packageCode) + if err != nil { + return nil, err + } + if pkg.Enabled != 1 { + return nil, gerror.New("当前套餐未启用") + } + durationUnit := renewDurationUnitForPackage(pkg) + actualDurationMonths := consts.DurationValueToMonths(durationUnit, durationValue) + actualGiftDurationMonths := consts.DurationValueToMonths(durationUnit, giftDurationMonths) + options, err := validateRuntimeRenewDuration(pkg, durationValue) + if err != nil { + return nil, err + } + + customerDept, err := loadRenewDeptTx(ctx, tx, main.TenantId, customerID) + if err != nil { + return nil, err + } + priceSnap, err := loadRuntimeRenewPriceSnapshot(ctx, tx, main.TenantId, pkg, customerDept, options, beginDates) + if err != nil { + return nil, err + } + amount := roundAmount(priceSnap.RenewUnitPrice * float64(durationValue) * float64(len(iccids))) + + mode := strings.TrimSpace(firstString(info, "paymentMode", "paymentSnapshot.paymentMode")) + payment, err := r.buildRuntimeRenewPayment(ctx, tx, main, mode, customerID, amount, now) + if err != nil { + return nil, err + } + + addition := map[string]any{ + "customerAgentId": customerID, + "customerName": strings.TrimSpace(customerDept.DeptName), + "packageCode": priceSnap.PackageCode, + "packageName": priceSnap.PackageName, + "priceSource": priceSnap.PriceSource, + "priceOwnerDeptId": priceSnap.PriceOwnerDeptId, + "agentDefaultPriceId": priceSnap.AgentDefaultPriceId, + "targetPriceId": priceSnap.TargetPriceId, + "renewUnitPrice": priceSnap.RenewUnitPrice, + "renewableDurationOptions": priceSnap.RenewableDurationOptions, + "renewablePackageCodes": priceSnap.RenewablePackageCodes, + "renewablePackages": priceSnap.RenewablePackages, + "billingMethod": priceSnap.BillingMethod, + "renewDurationUnit": priceSnap.RenewDurationUnit, + "renewDurationUnitText": priceSnap.RenewDurationUnitText, + "renewBeginDate": priceSnap.RenewBeginDate, + "renewBeginDateMin": priceSnap.RenewBeginDateMin, + "renewBeginDateMax": priceSnap.RenewBeginDateMax, + "sameRenewBeginDate": priceSnap.SameRenewBeginDate, + "priceSnapshot": priceSnap, + "quantity": len(iccids), + "durationValue": durationValue, + "durationMonths": actualDurationMonths, + "actualDurationMonths": actualDurationMonths, + "giftDurationMonths": giftDurationMonths, + "actualGiftDurationMonths": actualGiftDurationMonths, + "amount": amount, + } + for key, value := range payment { + addition[key] = value + } + merged := mergeBatchInfo(main.BatchInfo, addition) + main.BatchInfo = gjson.New(merged) + + contract, err := ParseExecutionContract(main) + if err != nil { + return nil, err + } + if err = r.validateRenewRuntimePermissions(ctx, tx, main, contract); err != nil { + return nil, err + } + return contract, nil +} + +func runtimeRenewFacts( + main *rc.BatchMainRow, + details []*rc.BatchDetailRow, + cards map[string]*rc.SimCardInfo, + binds map[string]*rc.SimCardBindPkg, + futureBinds map[string][]*rc.SimCardBindPkg, + now time.Time, +) (int64, string, []string, error) { + customerIDs := map[int64]struct{}{} + packageCodes := map[string]struct{}{} + beginDates := make([]string, 0, len(details)) + periodRows := make([]*RenewRow, 0, len(details)) + sourceBatchNo := executionSourceBatchNo(main, main.BatchInfo) + + for _, detail := range details { + if detail == nil { + return 0, "", nil, gerror.New("续费明细为空") + } + iccid := strings.TrimSpace(detail.ItemId) + card := cards[iccid] + bind := binds[iccid] + if card == nil { + return 0, "", nil, gerror.Newf("ICCID不存在：%s", iccid) + } + if bind == nil { + return 0, "", nil, gerror.Newf("ICCID没有订购记录：%s", iccid) + } + expectedMSISDN, expectedIMSI := snapshotIdentity(detail.Snapshot) + if err := validateCurrentSubscription(card, bind, subscriptionValidation{ + ICCID: iccid, + MSISDN: expectedMSISDN, + IMSI: expectedIMSI, + TenantID: main.TenantId, + CustomerDeptID: card.DeptId, + PackageCode: card.CurrentPackageCode, + SourceBatchNo: sourceBatchNo, + }); err != nil { + return 0, "", nil, gerror.Wrapf(err, "校验续费卡失败，ICCID:%s", iccid) + } + base := renewBaseSubscriptionForCard(card, bind, futureBinds[iccid]) + if err := validateRenewBaseWindow(base, now, iccid); err != nil { + return 0, "", nil, err + } + customerIDs[card.DeptId] = struct{}{} + packageCodes[strings.TrimSpace(card.CurrentPackageCode)] = struct{}{} + periodRows = append(periodRows, &RenewRow{ + RowNo: detail.RowNo, + ICCID: iccid, + OldEffTime: base.EffectiveTime, + OldExpireTime: base.ExpectedExpireTime, + }) + beginAt := renewBeginDateFromExpire(base.ExpectedExpireTime) + if beginAt != nil { + beginDates = append(beginDates, beginAt.Format("Y-m-d")) + } + } + if err := validateRenewSameCurrentPeriod(periodRows, nil); err != nil { + return 0, "", nil, err + } + if len(customerIDs) != 1 { + return 0, "", nil, gerror.New("当前批次下存在多个客户，不能统一续费") + } + if len(packageCodes) != 1 { + return 0, "", nil, gerror.New("当前批次下存在多个套餐，不能统一续费") + } + var customerID int64 + for id := range customerIDs { + customerID = id + } + var packageCode string + for code := range packageCodes { + packageCode = code + } + return customerID, packageCode, beginDates, nil +} + +func normalizeLegacyRenewDurationValue(raw int) int { + if raw <= 0 { + return 0 + } + if raw >= 12 && raw%12 == 0 { + return raw / 12 + } + return raw +} + +func validateRuntimeRenewDuration(pkg *rc.FlowPackageInfo, durationValue int) ([]int, error) { + if pkg == nil { + return nil, gerror.New("续费套餐不存在") + } + options, err := renewableYearOptions(pkg.RenewableDurationConfig) + if err != nil || len(options) == 0 { + return nil, gerror.Newf("当前套餐%s未配置可续费时长", runtimeRenewPackageLabel(pkg)) + } + if !rc.ContainsInt(options, durationValue) { + return nil, gerror.Newf( + "续费时长%d%s不在当前套餐%s可续费范围内，可选：%s", + durationValue, + consts.RenewDurationUnitText(renewDurationUnitForPackage(pkg)), + runtimeRenewPackageLabel(pkg), + formatRuntimeRenewDurationOptions(options, renewDurationUnitForPackage(pkg)), + ) + } + return options, nil +} + +func runtimeRenewPackageLabel(pkg *rc.FlowPackageInfo) string { + if pkg == nil { + return "" + } + name := strings.TrimSpace(pkg.PackageName) + code := strings.TrimSpace(pkg.PackageCode) + switch { + case name != "" && code != "": + return name + "(" + code + ")" + case name != "": + return name + default: + return code + } +} + +func formatRuntimeRenewDurationOptions(options []int, unit string) string { + if len(options) == 0 { + return "" + } + parts := make([]string, 0, len(options)) + for _, option := range options { + parts = append(parts, fmt.Sprintf("%d%s", option, consts.RenewDurationUnitText(unit))) + } + return strings.Join(parts, "、") +} + +func snapshotIdentity(snapshot *gjson.Json) (string, string) { + if snapshot == nil { + return "", "" + } + msisdn := strings.TrimSpace(snapshot.Get("msisdn").String()) + if msisdn == "" { + msisdn = strings.TrimSpace(snapshot.Get("MSISDN").String()) + } + imsi := strings.TrimSpace(snapshot.Get("imsi").String()) + if imsi == "" { + imsi = strings.TrimSpace(snapshot.Get("IMSI").String()) + } + return msisdn, imsi +} + +func loadRuntimeRenewPriceSnapshot( + ctx context.Context, + tx gdb.TX, + tenantID string, + pkg *rc.FlowPackageInfo, + customerDept *rc.SysDeptRow, + options []int, + beginDates []string, +) (*RenewPriceSnapshot, error) { + if pkg == nil || customerDept == nil { + return nil, gerror.New("续费套餐或客户信息缺失") + } + priceOwnerDeptID := renewPriceOwnerDeptID(customerDept) + salePrice, err := resolveRenewEffectiveSalePrice(ctx, renewTxEffectivePriceStore{tx: tx}, tenantID, priceOwnerDeptID, customerDept.DeptId, pkg) + if err != nil { + return nil, err + } + sameDate, minDate, maxDate, same := summarizeRenewBeginDates(beginDates) + durationUnit := renewDurationUnitForPackage(pkg) + renewableCodes, renewablePackages, err := runtimeRenewablePackageOptions(ctx, tx, tenantID, customerDept.DeptId, priceOwnerDeptID, pkg) + if err != nil { + return nil, err + } + return &RenewPriceSnapshot{ + CustomerAgentId: customerDept.DeptId, + PriceOwnerDeptId: priceOwnerDeptID, + PackageCode: pkg.PackageCode, + PackageName: pkg.PackageName, + BillingMethod: pkg.BillingMethod, + RenewDurationUnit: durationUnit, + RenewDurationUnitText: consts.RenewDurationUnitText(durationUnit), + PriceSource: salePrice.Rule.PriceSource, + AgentDefaultPriceId: salePrice.Rule.AgentDefaultPriceId, + TargetPriceId: salePrice.Rule.TargetPriceId, + RenewUnitPrice: salePrice.AgentShipmentPrice, + RenewableDurationOptions: options, + RenewablePackageCodes: renewableCodes, + RenewablePackages: renewablePackages, + RenewBeginDate: sameDate, + RenewBeginDateMin: minDate, + RenewBeginDateMax: maxDate, + SameRenewBeginDate: same, + }, nil +} + +func runtimeRenewablePackageOptions(ctx context.Context, tx gdb.TX, tenantID string, customerDeptID, priceOwnerDeptID int64, currentPkg *rc.FlowPackageInfo) ([]string, []RenewPackageOption, error) { + if currentPkg == nil { + return nil, nil, gerror.New("当前套餐不存在或未启用") + } + codes := renewPackageCodesFromJSON(currentPkg.RenewablePackageCodes) + if len(codes) == 0 { + return nil, nil, gerror.New("当前套餐未配置可续费套餐") + } + validCodes := make([]string, 0, len(codes)) + options := make([]RenewPackageOption, 0, len(codes)) + for _, code := range codes { + pkg, err := loadRenewPackageOptionTx(ctx, tx, tenantID, code) + if err != nil || pkg == nil || pkg.Enabled != 1 { + continue + } + unitPrice, err := runtimeRenewPackageUnitPrice(ctx, tx, tenantID, customerDeptID, priceOwnerDeptID, pkg) + if err != nil { + return nil, nil, err + } + durations, err := renewableYearOptions(pkg.RenewableDurationConfig) + if err != nil { + continue + } + validCodes = append(validCodes, pkg.PackageCode) + options = append(options, RenewPackageOption{ + PackageCode: pkg.PackageCode, + PackageName: pkg.PackageName, + RenewUnitPrice: unitPrice, + RenewableDurationOptions: durations, + }) + } + if len(options) == 0 { + return nil, nil, gerror.New("当前套餐未配置可用的可续费套餐") + } + return validCodes, options, nil +} + +func loadRenewPackageOptionTx(ctx context.Context, tx gdb.TX, tenantID, packageCode string) (*rc.FlowPackageInfo, error) { + var pkg rc.FlowPackageInfo + err := tx.Ctx(ctx).Model(rc.TableSimFlowPackageInfo). + Where("tenant_id", strings.TrimSpace(tenantID)). + Where("package_code", strings.TrimSpace(packageCode)). + LockShared(). + Scan(&pkg) + if err != nil { + if errors.Is(err, sql.ErrNoRows) { + return nil, nil + } + return nil, gerror.Wrap(err, "查询可续费套餐失败") + } + if pkg.Id == 0 { + return nil, nil + } + return &pkg, nil +} + +func runtimeRenewPackageUnitPrice(ctx context.Context, tx gdb.TX, tenantID string, customerDeptID, priceOwnerDeptID int64, pkg *rc.FlowPackageInfo) (float64, error) { + if pkg == nil { + return 0, gerror.New("续费套餐不存在") + } + salePrice, err := resolveRenewEffectiveSalePrice(ctx, renewTxEffectivePriceStore{tx: tx}, tenantID, priceOwnerDeptID, customerDeptID, pkg) + if err != nil { + return 0, err + } + return salePrice.AgentShipmentPrice, nil +} + +func (r *RenewExecutor) buildRuntimeRenewPayment(ctx context.Context, tx gdb.TX, main *rc.BatchMainRow, requestedMode string, payerDeptID int64, amount float64, now time.Time) (map[string]any, error) { + isTop, operatorDeptID, err := isRenewTopDeptTx(ctx, tx, main) + if err != nil { + return nil, err + } + if payerDeptID <= 0 { + payerDeptID = operatorDeptID + } + mode := strings.TrimSpace(requestedMode) + if mode == "" { + if isTop { + mode = "offline" + } else { + mode = "balance" + } + } + switch mode { + case "offline": + if !isTop { + return nil, gerror.New("当前组织续费必须使用余额支付") + } + if main.BatchInfo != nil && !main.BatchInfo.Get("offlineContractCompleted").Bool() { + return nil, gerror.New("请确认线下业务已完成") + } + return runtimePaymentSnapshot(mode, consts.PayTypeOffline, amount, 0, amount, 0, 0, "", payerDeptID, main.CreatedBy, isTop, now), nil + case "balance": + before, after, serialNo, err := deductRuntimeRenewBalance(ctx, tx, main, payerDeptID, amount, now) + if err != nil { + return nil, err + } + return runtimePaymentSnapshot(mode, consts.PayTypeBalance, amount, amount, 0, before, after, serialNo, payerDeptID, main.CreatedBy, isTop, now), nil + default: + return nil, gerror.New("续费付款方式不正确") + } +} + +func runtimePaymentSnapshot(mode string, payType int, payAmount, balanceAmount, offlineAmount, before, after float64, serialNo string, payerDeptID, payerUserID int64, isTop bool, now time.Time) map[string]any { + paymentTime := now.Format("2006-01-02 15:04:05") + snapshot := map[string]any{ + "paymentMode": mode, + "payType": payType, + "payStatus": consts.PayStatusPaid, + "payAmount": payAmount, + "balanceAmount": balanceAmount, + "offlinePaidAmount": offlineAmount, + "balanceBefore": before, + "balanceAfter": after, + "balanceDetailSerialNo": strings.TrimSpace(serialNo), + "paymentTime": paymentTime, + "payerDeptId": payerDeptID, + "payerUserId": payerUserID, + "isTopDept": isTop, + } + return map[string]any{ + "paymentMode": mode, + "payType": payType, + "payStatus": consts.PayStatusPaid, + "payAmount": payAmount, + "balanceAmount": balanceAmount, + "offlinePaidAmount": offlineAmount, + "balanceBefore": before, + "balanceAfter": after, + "balanceDetailSerialNo": strings.TrimSpace(serialNo), + "paymentTime": paymentTime, + "payerDeptId": payerDeptID, + "payerUserId": payerUserID, + "isTopDept": isTop, + "paymentSnapshot": snapshot, + } +} + +func deductRuntimeRenewBalance(ctx context.Context, tx gdb.TX, main *rc.BatchMainRow, payerDeptID int64, amount float64, now time.Time) (float64, float64, string, error) { + amount = roundAmount(amount) + if amount <= 0 { + return 0, 0, "", nil + } + serialNo := "RENPAY" + strings.TrimSpace(main.BatchNo) + if serialNo == "RENPAY" { + return 0, 0, "", gerror.New("续费余额流水号生成失败") + } + existingModel := tx.Ctx(ctx).Model(rc.TableBalanceDetail). + Where("tenant_id", main.TenantId). + Where("serial_no", serialNo) + existing, err := runtimeRenewLockForUpdate(existingModel).One() + if err != nil && !errors.Is(err, sql.ErrNoRows) { + return 0, 0, "", gerror.Wrap(err, "查询续费余额流水失败") + } + if existing != nil && !existing.IsEmpty() { + return 0, 0, "", gerror.New("当前续费批次已扣款，请勿重复提交") + } + + var account renewPaymentAccountRow + recordModel := tx.Ctx(ctx).Model(rc.TableAgentBalance). + Where("dept_id", payerDeptID). + Where("tenant_id", main.TenantId) + record, err := runtimeRenewLockForUpdate(recordModel).One() + if err != nil && !errors.Is(err, sql.ErrNoRows) { + return 0, 0, "", gerror.Wrap(err, "查询账户余额失败") + } + if record != nil && !record.IsEmpty() { + if err := record.Struct(&account); err != nil { + return 0, 0, "", gerror.Wrap(err, "解析账户余额失败") + } + } + if account.ID == 0 { + return 0, 0, "", gerror.New("账户余额不存在，请先充值") + } + if account.Status != renewFinanceAccountStatusNormal { + return 0, 0, "", gerror.New("账户余额状态异常，不能续费") + } + before := roundAmount(account.Balance) + if before+0.0001 < amount { + return 0, 0, "", gerror.Newf("账户余额不足，需支付%.2f元，当前余额%.2f元", amount, before) + } + after := roundAmount(before - amount) + nowTime := gtime.NewFromTime(now) + if _, err := tx.Ctx(ctx).Model(rc.TableAgentBalance).Where("id", account.ID).Data(g.Map{ + "balance": after, + "updated_by": main.CreatedBy, + "updated_at": nowTime, + }).Update(); err != nil { + return 0, 0, "", gerror.Wrap(err, "扣减账户余额失败") + } + if _, err := tx.Ctx(ctx).Model(rc.TableBalanceDetail).Data(g.Map{ + "dept_id": payerDeptID, + "serial_no": serialNo, + "transaction_time": nowTime, + "income_expense_type": renewFinanceIncomeExpenseExpense, + "transaction_type": renewFinanceTransactionTypeConsume, + "transaction_channel": renewFinanceTransactionChannelBalance, + "transaction_status": renewFinanceTransactionStatusPaid, + "amount": amount, + "before_balance": before, + "balance_after": after, + "order_sn": main.BatchNo, + "remark": fmt.Sprintf("SIM卡续费余额支付，批次号：%s", main.BatchNo), + "tenant_id": main.TenantId, + "created_dept": payerDeptID, + "created_by": main.CreatedBy, + "created_at": nowTime, + }).Insert(); err != nil { + return 0, 0, "", gerror.Wrap(err, "写入续费余额流水失败") + } + return before, after, serialNo, nil +} + +func runtimeRenewLockForUpdate(model *gdb.Model) *gdb.Model { + if g.DB().GetConfig().Type == "sqlite" { + return model + } + return model.LockUpdate() +} + +func (r *RenewExecutor) validateRenewRuntimePermissions(ctx context.Context, tx gdb.TX, main *rc.BatchMainRow, contract *ExecutionContract) error { + if contract == nil { + return gerror.New("续费执行合同不能为空") + } + isTop, _, err := isRenewTopDeptTx(ctx, tx, main) + if err != nil { + return err + } + if contract.PaymentMode == "offline" && !isTop { + return gerror.New("当前组织续费必须使用余额支付") + } + if contract.PackageBeginTime != nil && !isTop { + return gerror.New("当前组织不能修改续费日期") + } + return nil +} + +func isRenewTopDeptTx(ctx context.Context, tx gdb.TX, main *rc.BatchMainRow) (bool, int64, error) { + deptID := main.CreatedDept + if deptID <= 0 { + deptID = main.DeptId + } + if deptID <= 0 { + return false, 0, gerror.New("当前批次缺少操作组织") + } + dept, err := loadRenewDeptTx(ctx, tx, main.TenantId, deptID) + if err != nil { + return false, 0, err + } + return dept.ParentId == 0, deptID, nil +} + +func loadRenewDeptTx(ctx context.Context, tx gdb.TX, tenantID string, deptID int64) (*rc.SysDeptRow, error) { + var row rc.SysDeptRow + err := tx.Ctx(ctx).Model(rc.TableSysDept). + Where("tenant_id", tenantID). + Where("dept_id", deptID). + WhereNull("deleted_at"). + LockShared(). + Scan(&row) + if err != nil { + return nil, gerror.Wrap(err, "查询客户部门失败") + } + if row.DeptId == 0 { + return nil, gerror.New("客户部门不存在") + } + return &row, nil +} + +func loadRenewPriceOverrideTx(ctx context.Context, tx gdb.TX, tenantID string, deptID, targetAgentID int64, scopeType int, packageCode string) (*rc.FlowPackagePrice, error) { + var row rc.FlowPackagePrice + err := tx.Ctx(ctx).Model(rc.TableSimFlowPackagePrice). + Where("tenant_id", tenantID). + Where("dept_id", deptID). + Where("agent_scope_type", scopeType). + Where("target_agent_id", targetAgentID). + Where("package_code", strings.TrimSpace(packageCode)). + Where("enabled", 1). + LockShared(). + Scan(&row) + if err != nil { + if errors.Is(err, sql.ErrNoRows) { + return nil, nil + } + return nil, gerror.Wrap(err, "查询代理续费定价失败") + } + if row.Id == 0 { + return nil, nil + } + return &row, nil +} diff --git a/internal/service/renewchange/renew_refund.go b/internal/service/renewchange/renew_refund.go new file mode 100644 index 0000000..84aabc8 --- /dev/null +++ b/internal/service/renewchange/renew_refund.go @@ -0,0 +1,176 @@ +package renewchange + +import ( + "context" + "database/sql" + "errors" + "fmt" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" +) + +const ( + renewFinanceIncomeExpenseIncome = 0 + renewFinanceTransactionTypeConsume = 1 + renewFinanceTransactionChannelBalance = 3 + renewFinanceTransactionStatusPaid = 1 +) + +type renewBalanceAccountRow struct { + ID int64 `orm:"id"` + Balance float64 `orm:"balance"` +} + +type renewBalanceDetailRow struct { + ID int64 `orm:"id"` + DeptID int64 `orm:"dept_id"` + SerialNo string `orm:"serial_no"` + Amount float64 `orm:"amount"` + TransactionStatus int `orm:"transaction_status"` + TenantID string `orm:"tenant_id"` + CreatedDept int64 `orm:"created_dept"` + CreatedBy int64 `orm:"created_by"` +} + +func refundRenewBalanceIfNeeded(ctx context.Context, tx gdb.TX, main *rc.BatchMainRow, contract *ExecutionContract, reason string, now *gtime.Time) error { + if main == nil || contract == nil { + return nil + } + if contract.PaymentMode != "balance" || strings.TrimSpace(contract.BalanceDetailSerialNo) == "" || contract.BalanceAmount <= 0 { + return nil + } + if strings.TrimSpace(main.OrderSn) != "" { + return nil + } + refundSerialNo := "REFUND" + strings.TrimSpace(contract.BalanceDetailSerialNo) + var existing renewBalanceDetailRow + existingRecord, err := tx.Ctx(ctx).Model(rc.TableBalanceDetail). + Where("tenant_id", contract.TenantID). + Where("serial_no", refundSerialNo). + LockUpdate(). + One() + if err != nil && !errors.Is(err, sql.ErrNoRows) { + return gerror.Wrap(err, "query renew refund detail failed") + } + if existingRecord != nil && !existingRecord.IsEmpty() { + if err := existingRecord.Struct(&existing); err != nil { + return gerror.Wrap(err, "decode renew refund detail failed") + } + } + if existing.ID != 0 { + return nil + } + + var original renewBalanceDetailRow + originalRecord, err := tx.Ctx(ctx).Model(rc.TableBalanceDetail). + Where("tenant_id", contract.TenantID). + Where("serial_no", strings.TrimSpace(contract.BalanceDetailSerialNo)). + LockUpdate(). + One() + if err != nil && !errors.Is(err, sql.ErrNoRows) { + return gerror.Wrap(err, "query renew payment detail failed") + } + if originalRecord != nil && !originalRecord.IsEmpty() { + if err := originalRecord.Struct(&original); err != nil { + return gerror.Wrap(err, "decode renew payment detail failed") + } + } + if original.ID == 0 { + g.Log().Warningf(ctx, "renew balance refund skipped, original payment detail missing, batchNo=%s serialNo=%s", + main.BatchNo, contract.BalanceDetailSerialNo) + return nil + } + + payerDeptID := contract.PayerDeptID + if payerDeptID <= 0 { + payerDeptID = original.DeptID + } + var account renewBalanceAccountRow + accountRecord, err := tx.Ctx(ctx).Model(rc.TableAgentBalance). + Where("dept_id", payerDeptID). + Where("tenant_id", contract.TenantID). + LockUpdate(). + One() + if err != nil && !errors.Is(err, sql.ErrNoRows) { + return gerror.Wrap(err, "lock renew refund account failed") + } + if accountRecord != nil && !accountRecord.IsEmpty() { + if err := accountRecord.Struct(&account); err != nil { + return gerror.Wrap(err, "decode renew refund account failed") + } + } + if account.ID == 0 { + return gerror.New("renew refund account missing") + } + if now == nil { + now = gtime.Now() + } + amount := roundAmount(original.Amount) + before := roundAmount(account.Balance) + after := roundAmount(before + amount) + if _, err := tx.Ctx(ctx).Model(rc.TableAgentBalance).Where("id", account.ID).Data(g.Map{ + "balance": after, + "updated_by": contract.OperatorUserID, + "updated_at": now, + }).Update(); err != nil { + return gerror.Wrap(err, "refund renew balance failed") + } + remark := fmt.Sprintf("SIM卡续费批次失败退款，批次号：%s，原因：%s", main.BatchNo, strings.TrimSpace(reason)) + if _, err := tx.Ctx(ctx).Model(rc.TableBalanceDetail).Data(g.Map{ + "dept_id": payerDeptID, + "serial_no": refundSerialNo, + "transaction_time": now, + "income_expense_type": renewFinanceIncomeExpenseIncome, + "transaction_type": renewFinanceTransactionTypeConsume, + "transaction_channel": renewFinanceTransactionChannelBalance, + "transaction_status": renewFinanceTransactionStatusPaid, + "amount": amount, + "before_balance": before, + "balance_after": after, + "order_sn": main.BatchNo, + "remark": remark, + "tenant_id": contract.TenantID, + "created_dept": original.CreatedDept, + "created_by": original.CreatedBy, + "created_at": now, + }).Insert(); err != nil { + return gerror.Wrap(err, "insert renew refund detail failed") + } + g.Log().Warningf(ctx, "renew balance refunded, batchNo=%s serialNo=%s refundSerialNo=%s amount=%.2f reason=%s", + main.BatchNo, contract.BalanceDetailSerialNo, refundSerialNo, amount, reason) + return nil +} + +func renewRefundContractFromBatch(main *rc.BatchMainRow) *ExecutionContract { + if main == nil || main.BatchInfo == nil || main.BatchInfo.IsNil() { + return nil + } + info := main.BatchInfo + contract := &ExecutionContract{ + BatchNo: strings.TrimSpace(main.BatchNo), + BatchType: main.BatchType, + TenantID: strings.TrimSpace(main.TenantId), + CustomerAgentID: firstInt64(info, "customerAgentId", "priceSnapshot.customerAgentId"), + SellerAgentID: main.DeptId, + CreatedDeptID: main.CreatedDept, + OperatorUserID: main.CreatedBy, + PaymentMode: firstString(info, "paymentMode", "paymentSnapshot.paymentMode"), + PayType: firstInt(info, "payType", "paymentSnapshot.payType"), + PayAmount: firstAmount(info, "payAmount", "paymentSnapshot.payAmount"), + BalanceAmount: firstAmount(info, "balanceAmount", "paymentSnapshot.balanceAmount"), + BalanceDetailSerialNo: firstString(info, "balanceDetailSerialNo", "paymentSnapshot.balanceDetailSerialNo"), + PayerDeptID: firstInt64(info, "payerDeptId", "paymentSnapshot.payerDeptId"), + PayerUserID: firstInt64(info, "payerUserId", "paymentSnapshot.payerUserId"), + } + contract.normalizePaymentContract() + if contract.PaymentMode != "balance" || strings.TrimSpace(contract.BalanceDetailSerialNo) == "" || contract.BalanceAmount <= 0 { + return nil + } + return contract +} diff --git a/internal/service/renewchange/renew_test.go b/internal/service/renewchange/renew_test.go new file mode 100644 index 0000000..9b110af --- /dev/null +++ b/internal/service/renewchange/renew_test.go @@ -0,0 +1,543 @@ +package renewchange + +import ( + "strings" + "testing" + "time" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" +) + +func TestValidateRuntimeRenewDurationRejectsUnsupportedOption(t *testing.T) { + pkg := &rc.FlowPackageInfo{ + PackageCode: "PKG001", + BillingMethod: consts.BillingMethodMonth, + PackageName: "测试套餐", + RenewableDurationConfig: gjson.New([]any{1, 2, 3}), + } + + options, err := validateRuntimeRenewDuration(pkg, 1) + if err != nil { + t.Fatalf("validateRuntimeRenewDuration returned error: %v", err) + } + if len(options) != 3 { + t.Fatalf("options = %v, want 3 entries", options) + } + + _, err = validateRuntimeRenewDuration(pkg, 4) + if err == nil { + t.Fatal("expected unsupported renew duration to be rejected") + } + if got := err.Error(); !strings.Contains(got, "4年") || !strings.Contains(got, "测试套餐(PKG001)") || !strings.Contains(got, "3年") { + t.Fatalf("error message %q does not include duration/package/options", got) + } +} + +func TestValidateRuntimeRenewDurationNormalizesLegacyMonthOptions(t *testing.T) { + pkg := &rc.FlowPackageInfo{ + PackageCode: "PKG001", + PackageName: "测试套餐", + RenewableDurationConfig: gjson.New([]any{12, 24, 36}), + } + + options, err := validateRuntimeRenewDuration(pkg, 2) + if err != nil { + t.Fatalf("validateRuntimeRenewDuration returned error: %v", err) + } + if got := options; len(got) != 3 || got[0] != 1 || got[1] != 2 || got[2] != 3 { + t.Fatalf("options = %v, want [1 2 3]", got) + } +} + +func TestAddMonthsClampsToTargetMonthEnd(t *testing.T) { + begin := time.Date(2027, time.January, 31, 0, 0, 0, 0, time.UTC) + want := time.Date(2027, time.February, 28, 0, 0, 0, 0, time.UTC) + if got := addMonths(begin, 1); !got.Equal(want) { + t.Fatalf("addMonths() = %v, want %v", got, want) + } +} + +func TestRenewBaseSubscriptionForCardPrefersLaterCardExpire(t *testing.T) { + card := &rc.SimCardInfo{ + Iccid: "8986001", + CurrentPackageCode: "PKG001", + FlowPkgEndAt: gtime.NewFromStr("2026-08-31 00:00:00"), + FlowPkgPresentAt: gtime.NewFromStr("2026-09-30 00:00:00"), + FlowPkgRealPresent: gtime.NewFromStr("2026-10-31 00:00:00"), + } + bind := &rc.SimCardBindPkg{ + Iccid: "8986001", + PackageCode: "PKG001", + ExpectedExpireTime: gtime.NewFromStr("2026-08-31 00:00:00"), + } + + if got := renewBaseSubscriptionForCard(card, bind, nil); got == nil || got.ExpectedExpireTime.Format("Y-m-d") != "2026-10-31" { + t.Fatalf("renew base expire = %v, want 2026-10-31", got) + } +} + +func TestParseExecutionContractConvertsYearDurationToActualMonths(t *testing.T) { + main := &rc.BatchMainRow{ + BatchNo: "REN20260806000000TEST", + BatchType: consts.BatchTypeRenew, + TenantId: "000000", + DeptId: 1000, + CreatedDept: 1000, + CreatedBy: 1, + BatchInfo: gjson.New(map[string]any{ + "customerAgentId": 1001, + "packageCode": "PKG001", + "packageName": "Test package", + "renewUnitPrice": 120, + "quantity": 2, + "billingMethod": consts.BillingMethodYear, + "renewDurationUnit": consts.RenewDurationUnitYear, + "durationValue": 1, + "durationMonths": 1, + "offlineContractCompleted": true, + "paymentMode": "offline", + }), + } + + contract, err := ParseExecutionContract(main) + if err != nil { + t.Fatalf("ParseExecutionContract returned error: %v", err) + } + if contract.DurationDisplayValue != 1 { + t.Fatalf("display duration = %d, want 1", contract.DurationDisplayValue) + } + if contract.DurationMonths != 12 { + t.Fatalf("actual duration months = %d, want 12", contract.DurationMonths) + } + if got := contract.TotalAmount(); got != 240 { + t.Fatalf("total amount = %v, want 240", got) + } +} + +func TestRuntimeRenewFactsRequiresSameCurrentPeriod(t *testing.T) { + main := &rc.BatchMainRow{ + BatchNo: "REN20260806000000TEST", + TenantId: "000000", + } + begin := gtime.NewFromStr("2026-01-01 00:00:00") + expire := gtime.NewFromStr("2026-12-31 23:59:59") + details := []*rc.BatchDetailRow{ + {RowNo: 1, ItemId: "898600000000000001"}, + {RowNo: 2, ItemId: "898600000000000002"}, + } + cards := map[string]*rc.SimCardInfo{ + details[0].ItemId: { + Iccid: details[0].ItemId, + DeptId: 117, + CurrentPackageCode: "PKG001", + PackageStatus: consts.PackageStatusEffective, + TenantId: main.TenantId, + }, + details[1].ItemId: { + Iccid: details[1].ItemId, + DeptId: 117, + CurrentPackageCode: "PKG001", + PackageStatus: consts.PackageStatusEffective, + TenantId: main.TenantId, + }, + } + binds := map[string]*rc.SimCardBindPkg{ + details[0].ItemId: { + Iccid: details[0].ItemId, + DeptId: 117, + PackageCode: "PKG001", + EffectiveTime: begin, + ExpectedExpireTime: expire, + TenantId: main.TenantId, + }, + details[1].ItemId: { + Iccid: details[1].ItemId, + DeptId: 117, + PackageCode: "PKG001", + EffectiveTime: begin, + ExpectedExpireTime: expire, + TenantId: main.TenantId, + }, + } + + now := time.Date(2026, 8, 1, 0, 0, 0, 0, time.Local) + if _, _, _, err := runtimeRenewFacts(main, details, cards, binds, nil, now); err != nil { + t.Fatalf("runtimeRenewFacts() error = %v", err) + } + + binds[details[1].ItemId].ExpectedExpireTime = gtime.NewFromStr("2027-01-31 23:59:59") + if _, _, _, err := runtimeRenewFacts(main, details, cards, binds, nil, now); err == nil { + t.Fatalf("runtimeRenewFacts() error = nil, want same period failure") + } +} + +func TestValidateRenewExecutionsUsesConfirmedDuration(t *testing.T) { + now := time.Date(2026, 8, 3, 10, 0, 0, 0, time.UTC) + base := now.AddDate(0, 1, 0) + contract := testRenewContract(12, 2) + details := []*rc.BatchDetailRow{{RowNo: 1, ItemId: "8986000000000000001"}} + cards := map[string]*rc.SimCardInfo{ + details[0].ItemId: { + Iccid: details[0].ItemId, + DeptId: contract.CustomerAgentID, + CurrentPackageCode: contract.PackageCode, + PackageStatus: consts.PackageStatusEffective, + FlowPkgBeginAt: gtime.NewFromTime(now.AddDate(0, -1, 0)), + FlowPkgEndAt: gtime.NewFromTime(base), + OutboundBatchNo: contract.SourceBatchNo, + TenantId: contract.TenantID, + }, + } + binds := map[string]*rc.SimCardBindPkg{ + details[0].ItemId: { + Iccid: details[0].ItemId, + BatchNo: contract.SourceBatchNo, + Msisdn: cards[details[0].ItemId].Msisdn, + DeptId: contract.CustomerAgentID, + PackageCode: contract.PackageCode, + ExpectedExpireTime: gtime.NewFromTime(base), + TenantId: contract.TenantID, + }, + } + + got, err := validateRenewExecutions(contract, details, cards, binds, nil, now) + if err != nil { + t.Fatalf("validateRenewExecutions returned error: %v", err) + } + renewBegin := renewBeginDateFromExpire(gtime.NewFromTime(base)) + wantEnd := addMonths(renewBegin.Time, 14) + if len(got) != 1 || !got[0].end.Equal(wantEnd) { + t.Fatalf("renew end = %v, want %v", got[0].end, wantEnd) + } +} + +func TestValidateRenewExecutionsUsesLatestBindExpiration(t *testing.T) { + now := time.Date(2026, 8, 3, 10, 0, 0, 0, time.UTC) + contract := testRenewContract(12, 0) + detail := &rc.BatchDetailRow{RowNo: 1, ItemId: "8986000000000000002"} + cardEnd := gtime.NewFromTime(time.Date(2026, 9, 1, 0, 0, 0, 0, time.UTC)) + cards := map[string]*rc.SimCardInfo{ + detail.ItemId: { + Iccid: detail.ItemId, + DeptId: contract.CustomerAgentID, + CurrentPackageCode: contract.PackageCode, + PackageStatus: consts.PackageStatusEffective, + FlowPkgEndAt: cardEnd, + OutboundBatchNo: contract.SourceBatchNo, + TenantId: contract.TenantID, + }, + } + binds := map[string]*rc.SimCardBindPkg{ + detail.ItemId: { + Iccid: detail.ItemId, + BatchNo: contract.SourceBatchNo, + Msisdn: cards[detail.ItemId].Msisdn, + DeptId: contract.CustomerAgentID, + PackageCode: contract.PackageCode, + ExpectedExpireTime: gtime.NewFromTime(cardEnd.Time.AddDate(0, 0, 1)), + TenantId: contract.TenantID, + }, + } + + got, err := validateRenewExecutions(contract, []*rc.BatchDetailRow{detail}, cards, binds, nil, now) + if err != nil { + t.Fatalf("validateRenewExecutions returned error: %v", err) + } + renewBegin := renewBeginDateFromExpire(gtime.NewFromTime(cardEnd.Time.AddDate(0, 0, 1))) + wantEnd := addMonths(renewBegin.Time, contract.DurationMonths) + if len(got) != 1 || !got[0].end.Equal(wantEnd) { + t.Fatalf("renew end = %v, want %v", got[0].end, wantEnd) + } +} + +func TestValidateRenewExecutionsAllowsRenewalFromExpiredSubscription(t *testing.T) { + now := time.Date(2026, 8, 3, 10, 0, 0, 0, time.UTC) + contract := testRenewContract(12, 0) + detail := &rc.BatchDetailRow{RowNo: 1, ItemId: "8986000000000000006"} + expiredAt := time.Date(2026, 7, 1, 0, 0, 0, 0, time.UTC) + cards := map[string]*rc.SimCardInfo{ + detail.ItemId: { + Iccid: detail.ItemId, + DeptId: contract.CustomerAgentID, + CurrentPackageCode: contract.PackageCode, + PackageStatus: consts.PackageStatusExpired, + OutboundBatchNo: contract.SourceBatchNo, + TenantId: contract.TenantID, + }, + } + binds := map[string]*rc.SimCardBindPkg{ + detail.ItemId: { + Iccid: detail.ItemId, + BatchNo: contract.SourceBatchNo, + DeptId: contract.CustomerAgentID, + PackageCode: contract.PackageCode, + EffectiveStatus: consts.EffectiveStatusExpired, + ExpectedExpireTime: gtime.NewFromTime(expiredAt), + TenantId: contract.TenantID, + }, + } + + got, err := validateRenewExecutions(contract, []*rc.BatchDetailRow{detail}, cards, binds, nil, now) + if err != nil { + t.Fatalf("validateRenewExecutions returned error: %v", err) + } + renewBegin := renewBeginDateFromExpire(gtime.NewFromTime(expiredAt)) + wantEnd := addMonths(renewBegin.Time, contract.DurationMonths) + if len(got) != 1 || !got[0].begin.Equal(renewBegin.Time) || !got[0].end.Equal(wantEnd) { + t.Fatalf("renew range = %v - %v, want %v - %v", got[0].begin, got[0].end, renewBegin.Time, wantEnd) + } +} + +func TestValidateRenewExecutionsAppendsAfterFuturePendingExpiration(t *testing.T) { + now := time.Date(2026, 8, 3, 10, 0, 0, 0, time.UTC) + contract := testRenewContract(12, 0) + detail := &rc.BatchDetailRow{RowNo: 1, ItemId: "8986000000000000005"} + currentExpire := time.Date(2026, 9, 1, 0, 0, 0, 0, time.UTC) + futureExpire := time.Date(2027, 1, 1, 0, 0, 0, 0, time.UTC) + cards := map[string]*rc.SimCardInfo{ + detail.ItemId: { + Iccid: detail.ItemId, + DeptId: contract.CustomerAgentID, + CurrentPackageCode: contract.PackageCode, + PackageStatus: consts.PackageStatusEffective, + OutboundBatchNo: contract.SourceBatchNo, + TenantId: contract.TenantID, + }, + } + binds := map[string]*rc.SimCardBindPkg{ + detail.ItemId: { + Iccid: detail.ItemId, + BatchNo: contract.SourceBatchNo, + DeptId: contract.CustomerAgentID, + PackageCode: contract.PackageCode, + ExpectedExpireTime: gtime.NewFromTime(currentExpire), + TenantId: contract.TenantID, + }, + } + futureBinds := map[string][]*rc.SimCardBindPkg{ + detail.ItemId: { + { + Iccid: detail.ItemId, + BatchNo: "REN-FUTURE", + DeptId: contract.CustomerAgentID, + PackageCode: contract.PackageCode, + ExpectedExpireTime: gtime.NewFromTime(futureExpire), + TenantId: contract.TenantID, + }, + }, + } + + got, err := validateRenewExecutions(contract, []*rc.BatchDetailRow{detail}, cards, binds, futureBinds, now) + if err != nil { + t.Fatalf("validateRenewExecutions returned error: %v", err) + } + renewBegin := renewBeginDateFromExpire(gtime.NewFromTime(futureExpire)) + wantEnd := addMonths(renewBegin.Time, contract.DurationMonths) + if len(got) != 1 || !got[0].begin.Equal(renewBegin.Time) || !got[0].end.Equal(wantEnd) { + t.Fatalf("renew range = %v - %v, want %v - %v", got[0].begin, got[0].end, renewBegin.Time, wantEnd) + } +} + +func TestValidateRenewExecutionsRejectsPendingExpirationBeyondWindow(t *testing.T) { + now := time.Date(2026, 8, 3, 10, 0, 0, 0, time.UTC) + contract := testRenewContract(12, 0) + detail := &rc.BatchDetailRow{RowNo: 1, ItemId: "8986000000000000007"} + currentExpire := time.Date(2026, 9, 1, 0, 0, 0, 0, time.UTC) + futureExpire := time.Date(2027, 9, 1, 0, 0, 0, 0, time.UTC) + cards := map[string]*rc.SimCardInfo{ + detail.ItemId: { + Iccid: detail.ItemId, + DeptId: contract.CustomerAgentID, + CurrentPackageCode: contract.PackageCode, + PackageStatus: consts.PackageStatusEffective, + OutboundBatchNo: contract.SourceBatchNo, + TenantId: contract.TenantID, + }, + } + binds := map[string]*rc.SimCardBindPkg{ + detail.ItemId: { + Iccid: detail.ItemId, + BatchNo: contract.SourceBatchNo, + DeptId: contract.CustomerAgentID, + PackageCode: contract.PackageCode, + ExpectedExpireTime: gtime.NewFromTime(currentExpire), + TenantId: contract.TenantID, + }, + } + futureBinds := map[string][]*rc.SimCardBindPkg{ + detail.ItemId: { + { + Iccid: detail.ItemId, + BatchNo: "REN-FUTURE", + DeptId: contract.CustomerAgentID, + PackageCode: contract.PackageCode, + ExpectedExpireTime: gtime.NewFromTime(futureExpire), + TenantId: contract.TenantID, + }, + }, + } + + _, err := validateRenewExecutions(contract, []*rc.BatchDetailRow{detail}, cards, binds, futureBinds, now) + if err == nil || !strings.Contains(err.Error(), renewExpireFailureReason) { + t.Fatalf("validateRenewExecutions error = %v, want renew window failure", err) + } +} + +func TestValidateRunningRenewDetailsRejectsChangedSnapshot(t *testing.T) { + cases := []struct { + name string + locked []*rc.BatchDetailRow + loaded []*rc.BatchDetailRow + }{ + { + name: "count", + locked: []*rc.BatchDetailRow{{Id: 1, RowNo: 1, ItemId: "8986001", DetailStatus: consts.BatchDetailStatusRunning}}, + loaded: nil, + }, + { + name: "id", + locked: []*rc.BatchDetailRow{{Id: 2, RowNo: 1, ItemId: "8986001", DetailStatus: consts.BatchDetailStatusRunning}}, + loaded: []*rc.BatchDetailRow{{Id: 1, RowNo: 1, ItemId: "8986001", DetailStatus: consts.BatchDetailStatusRunning}}, + }, + { + name: "row number", + locked: []*rc.BatchDetailRow{{Id: 1, RowNo: 2, ItemId: "8986001", DetailStatus: consts.BatchDetailStatusRunning}}, + loaded: []*rc.BatchDetailRow{{Id: 1, RowNo: 1, ItemId: "8986001", DetailStatus: consts.BatchDetailStatusRunning}}, + }, + { + name: "iccid", + locked: []*rc.BatchDetailRow{{Id: 1, RowNo: 1, ItemId: "8986002", DetailStatus: consts.BatchDetailStatusRunning}}, + loaded: []*rc.BatchDetailRow{{Id: 1, RowNo: 1, ItemId: "8986001", DetailStatus: consts.BatchDetailStatusRunning}}, + }, + { + name: "status", + locked: []*rc.BatchDetailRow{{Id: 1, RowNo: 1, ItemId: "8986001", DetailStatus: consts.BatchDetailStatusSuccess}}, + loaded: []*rc.BatchDetailRow{{Id: 1, RowNo: 1, ItemId: "8986001", DetailStatus: consts.BatchDetailStatusRunning}}, + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + if err := validateRunningRenewDetails(tc.locked, tc.loaded); err == nil { + t.Fatal("expected changed renew details to be rejected") + } + }) + } +} + +func TestValidateRenewExecutionsUsesConfirmedPackageBeginTime(t *testing.T) { + now := time.Date(2026, 8, 3, 10, 0, 0, 0, time.Local) + contract := testRenewContract(12, 0) + confirmedBegin := time.Date(2026, 10, 1, 0, 0, 0, 0, time.Local) + contract.PackageBeginTime = gtime.NewFromTime(confirmedBegin) + detail := &rc.BatchDetailRow{RowNo: 1, ItemId: "8986000000000000004"} + latestExpire := time.Date(2026, 9, 1, 0, 0, 0, 0, time.Local) + cards := map[string]*rc.SimCardInfo{ + detail.ItemId: { + Iccid: detail.ItemId, + DeptId: contract.CustomerAgentID, + CurrentPackageCode: contract.PackageCode, + PackageStatus: consts.PackageStatusEffective, + OutboundBatchNo: contract.SourceBatchNo, + TenantId: contract.TenantID, + }, + } + binds := map[string]*rc.SimCardBindPkg{ + detail.ItemId: { + Iccid: detail.ItemId, + BatchNo: contract.SourceBatchNo, + DeptId: contract.CustomerAgentID, + PackageCode: contract.PackageCode, + ExpectedExpireTime: gtime.NewFromTime(latestExpire), + TenantId: contract.TenantID, + }, + } + + got, err := validateRenewExecutions(contract, []*rc.BatchDetailRow{detail}, cards, binds, nil, now) + if err != nil { + t.Fatalf("validateRenewExecutions returned error: %v", err) + } + wantEnd := addMonths(confirmedBegin, contract.DurationMonths) + if len(got) != 1 || !got[0].begin.Equal(confirmedBegin) || !got[0].end.Equal(wantEnd) { + t.Fatalf("renew range = %v - %v, want %v - %v", got[0].begin, got[0].end, confirmedBegin, wantEnd) + } +} + +func TestBuildRenewBindUsesConfirmedContract(t *testing.T) { + now := time.Date(2026, 8, 3, 10, 0, 0, 0, time.UTC) + begin := now.AddDate(0, 1, 0) + contract := testRenewContract(12, 3) + pkg := &rc.FlowPackageInfo{PackageCode: contract.PackageCode, PackageType: 7} + execution := renewExecution{ + card: &rc.SimCardInfo{Iccid: "8986000000000000003", Msisdn: "13800000000"}, + bind: &rc.SimCardBindPkg{ExpectedExpireTime: gtime.NewFromTime(begin)}, + begin: begin, + end: contract.ServiceEnd(begin), + } + + bind := buildRenewBind(contract, pkg, execution, "SO20260803100000ABC", "SUB20260803100000ABC", now) + if bind.DurationMonths != 12 || bind.GiftDurationMonths != 3 { + t.Fatalf("duration = %d+%d, want 12+3", bind.DurationMonths, bind.GiftDurationMonths) + } + if bind.PackageType != 7 || bind.EffectiveTime.Time != begin { + t.Fatalf("bind contract fields were not preserved: %+v", bind) + } +} + +func TestBuildRenewDetailSnapshotUsesConfirmedBatchAmount(t *testing.T) { + contract := testRenewContract(12, 0) + contract.Quantity = 2 + execution := renewExecution{ + detail: &rc.BatchDetailRow{RowNo: 1}, + card: &rc.SimCardInfo{Iccid: "8986000000000000003"}, + bind: &rc.SimCardBindPkg{}, + begin: time.Date(2026, 8, 3, 0, 0, 0, 0, time.UTC), + end: time.Date(2027, 8, 3, 0, 0, 0, 0, time.UTC), + } + pkg := &rc.FlowPackageInfo{PackageCode: contract.PackageCode, PackageName: contract.PackageName} + + snapshot := buildRenewDetailSnapshot(execution, contract, pkg, "SO-1", "SUB-1") + if got := snapshot.Get("amount").Float64(); got != 240 { + t.Fatalf("detail amount = %v, want confirmed batch amount 240", got) + } +} + +func TestRenewPackageStatus(t *testing.T) { + now := time.Date(2026, 8, 3, 10, 0, 0, 0, time.UTC) + if got := renewPackageStatus(now, gtime.NewFromTime(now.AddDate(0, 1, 0)), gtime.NewFromTime(now.AddDate(0, 13, 0))); got != consts.PackageStatusPending { + t.Fatalf("future package status = %d, want pending", got) + } + if got := renewPackageStatus(now, gtime.NewFromTime(now.AddDate(0, -1, 0)), gtime.NewFromTime(now.AddDate(0, 1, 0))); got != consts.PackageStatusEffective { + t.Fatalf("active package status = %d, want effective", got) + } + if got := renewPackageStatus(now, gtime.NewFromTime(now.AddDate(0, -2, 0)), gtime.NewFromTime(now.AddDate(0, -1, 0))); got != consts.PackageStatusExpired { + t.Fatalf("expired package status = %d, want expired", got) + } +} + +func testRenewContract(duration, gift int) *ExecutionContract { + return &ExecutionContract{ + BatchNo: "REN20260803100000TEST", + BatchType: consts.BatchTypeRenew, + SourceBatchNo: "OUT20260801TEST", + TenantID: "000000", + CustomerAgentID: 1001, + SellerAgentID: 1000, + CreatedDeptID: 1000, + OperatorUserID: 1, + PackageCode: "PKG001", + PackageName: "Test package", + PriceSource: PriceSourceSystem, + UnitPrice: 10, + Quantity: 1, + DurationMonths: duration, + GiftDurationMonths: gift, + OfflineContractCompleted: true, + BuyerNote: "buyer", + AdminNote: "admin", + Remark: "renew test", + } +} diff --git a/internal/service/renewchange/repository.go b/internal/service/renewchange/repository.go new file mode 100644 index 0000000..7a4595f --- /dev/null +++ b/internal/service/renewchange/repository.go @@ -0,0 +1,396 @@ +package renewchange + +import ( + "context" + "strconv" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + batchrepo "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/batch" + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +// Repository orchestrates table access + batch status writes for renew/change +// service. It embeds the typed table repository and adds batch lifecycle +// helpers that align with the Admin side batch_main/batch_detail contract. +type Repository struct { + tables *rc.Repository + batchRepo *batchrepo.Repository + results batchresult.Generator +} + +// NewRepository builds a service-level repository backed by the typed table +// repository (g.DB() default) and the shared batch repository. +func NewRepository() *Repository { + return &Repository{ + tables: rc.New(), + batchRepo: batchrepo.New(), + results: batchresult.New(), + } +} + +// newRepositoryWithDeps allows tests to inject custom dependencies. +func newRepositoryWithDeps(tables *rc.Repository, batchRepo *batchrepo.Repository, results batchresult.Generator) *Repository { + return &Repository{tables: tables, batchRepo: batchRepo, results: results} +} + +// Tables exposes the low-level repository for tests. +func (r *Repository) Tables() *rc.Repository { return r.tables } + +// Batch exposes the batch repository. +func (r *Repository) Batch() *batchrepo.Repository { return r.batchRepo } + +// Results exposes the result file generator. +func (r *Repository) Results() batchresult.Generator { return r.results } + +// DB exposes the underlying gdb.DB handle. +func (r *Repository) DB() gdb.DB { return r.tables.DB() } + +// LoadProcessingBatch loads a processing renew/change/prepare batch by +// (batchNo, batchType). It returns an error if the batch is not in +// processing state so the caller can log and skip safely. +func (r *Repository) LoadProcessingBatch(ctx context.Context, batchNo string, batchType int) (*rc.BatchMainRow, error) { + main, err := r.tables.LoadBatchMain(ctx, batchNo, batchType) + if err != nil { + return nil, err + } + if main == nil { + return nil, gerror.Newf("batch %s (type %d) does not exist", batchNo, batchType) + } + if main.BatchAsyncStatus != batch.AsyncStatusProcessing { + return nil, gerror.Newf("batch %s (type %d) is not in processing state: %d", batchNo, batchType, main.BatchAsyncStatus) + } + return main, nil +} + +// LoadRunningDetails returns the running batch details ordered by row_no. +func (r *Repository) LoadRunningDetails(ctx context.Context, batchNo string, batchType int) ([]*rc.BatchDetailRow, error) { + return r.tables.LoadBatchDetails(ctx, batchNo, batchType, []int{consts.BatchDetailStatusRunning}) +} + +// LoadAllDetails returns every detail regardless of status, used for the +// confirm step (renew/change) to inspect previously committed results. +func (r *Repository) LoadAllDetails(ctx context.Context, batchNo string, batchType int) ([]*rc.BatchDetailRow, error) { + return r.tables.LoadBatchDetails(ctx, batchNo, batchType, nil) +} + +type ConvertPrepareResult struct { + MainAffected int64 + DetailAffected int64 +} + +// ConvertPrepareToRenewChange switches the prepare batch (type=4) to the +// matching business type (101/102). The transaction must be the one used for +// the detail update. +func (r *Repository) ConvertPrepareToRenewChange(ctx context.Context, tx gdb.TX, batchNo string, targetType int) (*ConvertPrepareResult, error) { + detailResult, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypePrepare). + Data(do.BatchDetail{BatchType: targetType, UpdatedAt: gtime.Now()}). + Update() + if err != nil { + return nil, gerror.Wrap(err, "convert prepare details to target type failed") + } + detailAffected, err := detailResult.RowsAffected() + if err != nil { + return nil, err + } + mainResult, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypePrepare). + Data(do.BatchMain{BatchType: targetType, UpdatedAt: gtime.Now()}). + Update() + if err != nil { + return nil, gerror.Wrap(err, "convert prepare batch type failed") + } + mainAffected, err := mainResult.RowsAffected() + if err != nil { + return nil, err + } + if mainAffected == 0 { + return nil, gerror.New("prepare batch main already converted or missing") + } + return &ConvertPrepareResult{ + MainAffected: mainAffected, + DetailAffected: detailAffected, + }, nil +} + +// MarkDetailFailed writes a single detail as failed with the given reason. +func (r *Repository) MarkDetailFailed(ctx context.Context, tx gdb.TX, batchNo string, batchType, rowNo int, reason string, userID int64) error { + return r.tables.UpdateBatchDetailSnapshot(ctx, tx, batchNo, batchType, rowNo, consts.BatchDetailStatusFailed, nil, reason, userID) +} + +// MarkDetailSuccess writes a single detail as success with the given snapshot. +func (r *Repository) MarkDetailSuccess(ctx context.Context, tx gdb.TX, batchNo string, batchType, rowNo int, snapshot *gjson.Json, userID int64) error { + return r.tables.UpdateBatchDetailSnapshot(ctx, tx, batchNo, batchType, rowNo, consts.BatchDetailStatusSuccess, snapshot, "", userID) +} + +func (r *Repository) MarkPrepareDetailSuccess(ctx context.Context, tx gdb.TX, batchNo string, rowNo int, itemID string, snapshot *gjson.Json, userID int64) error { + data := map[string]any{ + dao.BatchDetail.Columns().DetailStatus: consts.BatchDetailStatusSuccess, + dao.BatchDetail.Columns().Snapshot: snapshot, + dao.BatchDetail.Columns().UpdatedBy: userID, + dao.BatchDetail.Columns().UpdatedAt: gtime.Now(), + } + if itemID = strings.TrimSpace(itemID); itemID != "" { + data[dao.BatchDetail.Columns().ItemId] = itemID + } + _, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypePrepare). + Where(dao.BatchDetail.Columns().RowNo, rowNo). + Data(data). + Update() + return gerror.Wrap(err, "mark prepare detail success failed") +} + +// MarkAllDetailsFailed marks every unfinished detail as failed in a single update. +func (r *Repository) MarkAllDetailsFailed(ctx context.Context, tx gdb.TX, batchNo string, batchType int, reason string, userID int64) error { + _, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().BatchType, batchType). + WhereIn(dao.BatchDetail.Columns().DetailStatus, []int{consts.BatchDetailStatusPending, consts.BatchDetailStatusRunning}). + Data(do.BatchDetail{ + DetailStatus: consts.BatchDetailStatusFailed, + ErrorReason: reason, + UpdatedBy: userID, + UpdatedAt: gtime.Now(), + }). + Update() + return gerror.Wrap(err, "mark all running details failed") +} + +// MarkAllDetailsSuccess marks every running detail as success. +func (r *Repository) MarkAllDetailsSuccess(ctx context.Context, tx gdb.TX, batchNo string, batchType int, userID int64, extra map[string]any) error { + data := map[string]any{ + "detail_status": consts.BatchDetailStatusSuccess, + "updated_by": userID, + "updated_at": gtime.Now(), + } + for k, v := range extra { + data[k] = v + } + _, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().BatchType, batchType). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + Data(data). + Update() + return gerror.Wrap(err, "mark all running details success") +} + +// UpdateBatchMainInfo merges new entries into batch_main.batch_info and +// updates the lifecycle counters in a single statement. +func (r *Repository) UpdateBatchMainInfo(ctx context.Context, tx gdb.TX, batchNo string, batchType int, data *rc.BatchMainRow) error { + return r.tables.UpdateBatchMainStatus(ctx, tx, batchNo, batchType, data) +} + +// FinalizeBatchMain updates only the lifecycle fields (status, counts, +// fail_reason, end_time) while keeping batch_info untouched. +func (r *Repository) FinalizeBatchMain(ctx context.Context, tx gdb.TX, batchNo string, batchType int, data *rc.BatchMainRow) error { + _, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Where(dao.BatchMain.Columns().BatchType, batchType). + Data(do.BatchMain{ + BatchAsyncStatus: data.BatchAsyncStatus, + RunResult: data.RunResult, + SuccessCount: data.SuccessCount, + FailCount: data.FailCount, + ValidCount: data.ValidCount, + TotalCount: data.TotalCount, + FailReason: data.FailReason, + EndTime: data.EndTime, + UpdatedBy: data.UpdatedBy, + }). + Update() + return gerror.Wrap(err, "finalize batch main failed") +} + +// ResolveResultSnapshot returns the per-row data needed to render the result +// file. It always includes every detail (success and failure) for the batch. +func (r *Repository) ResolveResultSnapshot(ctx context.Context, batchNo string, batchType int) ([]ResultRow, error) { + details, err := r.tables.LoadBatchDetails(ctx, batchNo, batchType, nil) + if err != nil { + return nil, err + } + rows := make([]ResultRow, 0, len(details)) + for _, d := range details { + if d == nil { + continue + } + msisdn, imsi := "", "" + targetCode, packageCode, effectiveMode, orderNum := "", "", "", "" + if d.Snapshot != nil { + msisdn = strings.TrimSpace(d.Snapshot.Get("msisdn").String()) + imsi = strings.TrimSpace(d.Snapshot.Get("imsi").String()) + targetCode = strings.TrimSpace(d.Snapshot.Get("targetCode").String()) + if targetCode == "" { + targetCode = strings.TrimSpace(d.ItemId) + } + packageCode = strings.TrimSpace(d.Snapshot.Get("packageCode").String()) + effectiveMode = strings.TrimSpace(d.Snapshot.Get("effectiveMode").String()) + orderNum = strings.TrimSpace(d.Snapshot.Get("orderSn").String()) + if orderNum == "" { + orderNum = strings.TrimSpace(d.Snapshot.Get("orderNo").String()) + } + } + rows = append(rows, ResultRow{ + RowNo: d.RowNo, + ICCID: strings.TrimSpace(d.ItemId), + TargetCode: targetCode, + PackageCode: packageCode, + EffectiveMode: effectiveMode, + MSISDN: msisdn, + IMSI: imsi, + Success: d.DetailStatus == consts.BatchDetailStatusSuccess, + Reason: strings.TrimSpace(d.ErrorReason), + OrderNum: orderNum, + Snap: d.Snapshot, + DetailID: d.Id, + DetailCode: d.ErrorCode, + }) + } + return rows, nil +} + +// WriteResultFileSafe persists a result Excel and OSS/download records. It +// never panics on a bad batch type; missing configuration is logged and the +// business outcome is preserved. The caller must not abort the batch when +// this returns an error. +func (r *Repository) WriteResultFileSafe(ctx context.Context, batchNo string, batchType int, tenantID string, deptID, userID int64, rows []ResultRow) error { + if r.results == nil { + return nil + } + brRows := make([]batchresult.Row, 0, len(rows)) + for _, row := range rows { + brRows = append(brRows, batchresult.Row{ + RowNo: row.RowNo, + ObjectCode: row.TargetCode, + TargetCode: row.TargetCode, + PackageCode: row.PackageCode, + EffectiveMode: row.EffectiveMode, + ICCID: row.ICCID, + MSISDN: row.MSISDN, + IMSI: row.IMSI, + Success: row.Success, + Reason: row.Reason, + OrderNum: row.OrderNum, + }) + } + request := batchresult.Request{ + BatchNo: batchNo, + BatchType: batchType, + TenantID: strings.TrimSpace(tenantID), + Rows: brRows, + } + artifact, err := r.results.Create(ctx, request) + if err != nil { + g.Log().Warningf(ctx, "renew/change result file generation failed, batchNo:%s, batchType:%d, err:%+v", batchNo, batchType, err) + return err + } + if err = r.persistArtifact(ctx, artifact, batchNo, batchType, tenantID, deptID, userID, len(rows)); err != nil { + g.Log().Warningf(ctx, "renew/change result file bookkeeping failed, batchNo:%s, batchType:%d, err:%+v", batchNo, batchType, err) + return err + } + return nil +} + +func (r *Repository) persistArtifact(ctx context.Context, artifact batchresult.Artifact, batchNo string, batchType int, tenantID string, deptID, userID int64, total int) error { + return r.tables.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + ossID, err := r.tables.InsertSysOss(ctx, tx, &rc.SysOssRow{ + TenantId: tenantID, + FileName: artifact.FileName, + OriginalName: artifact.OriginalName, + FileSuffix: artifact.FileSuffix, + Path: artifact.Path, + Url: artifact.URL, + Md5: artifact.MD5, + FileSize: artifact.Size, + Service: "local", + CreatedDept: deptID, + CreatedBy: userID, + UpdatedBy: userID, + }) + if err != nil { + return err + } + successCount, failCount := 0, 0 + main, mErr := r.tables.LoadBatchMain(ctx, batchNo, batchType) + if mErr == nil && main != nil { + successCount = main.SuccessCount + failCount = main.FailCount + } + _, err = r.tables.InsertBatchDownload(ctx, tx, &rc.ResourceBatchDownloadRow{ + BatchNo: batchNo, + TaskName: batchresult.TaskName(batchType), + ResourceType: consts.ResourceTypeExcel, + ExportScene: batchType, + TotalCount: total, + SuccessCount: successCount, + FailCount: failCount, + DownloadStatus: consts.DownloadStatusGenerated, + OssId: ossID, + FileName: artifact.FileName, + FileExt: artifact.FileSuffix, + FileUrl: artifact.URL, + FilePath: artifact.Path, + FileSize: artifact.Size, + FileMd5: artifact.MD5, + DownloadCount: 0, + DeptId: deptID, + StartTime: gtime.Now(), + FinishTime: gtime.Now(), + TenantId: tenantID, + CreatedDept: deptID, + CreatedBy: userID, + UpdatedBy: userID, + }) + return err + }) +} + +// LoadSystemContext returns tenant id and a default user id (CreatedBy) for +// the given batch. Centralized so individual executors stay slim. +func (r *Repository) LoadSystemContext(ctx context.Context, batchNo string, batchType int) (string, int64, int64, error) { + main, err := r.tables.LoadBatchMain(ctx, batchNo, batchType) + if err != nil { + return "", 0, 0, err + } + if main == nil { + return "", 0, 0, gerror.Newf("batch %s (type %d) does not exist", batchNo, batchType) + } + return main.TenantId, main.CreatedBy, main.CreatedDept, nil +} + +// ResultRow is the per-row information needed by the result file. +type ResultRow struct { + RowNo int + ICCID string + TargetCode string + PackageCode string + EffectiveMode string + MSISDN string + IMSI string + Success bool + Reason string + OrderNum string + Upstream int + Snap *gjson.Json + DetailID int64 + DetailCode string +} + +func tenantIDtoString(tenantID int64) string { + return strconv.FormatInt(tenantID, 10) +} diff --git a/internal/service/renewchange/subscription_validation.go b/internal/service/renewchange/subscription_validation.go new file mode 100644 index 0000000..33f9cec --- /dev/null +++ b/internal/service/renewchange/subscription_validation.go @@ -0,0 +1,209 @@ +package renewchange + +import ( + "strings" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" +) + +type subscriptionValidation struct { + ICCID string + MSISDN string + IMSI string + TenantID string + CustomerDeptID int64 + PackageCode string + SourceBatchNo string +} + +// resolveCurrentSubscriptions selects the current package fact per ICCID. +// It keeps future pending subscriptions for change-package cleanup, while +// allowing renew to append after the farthest pending subscription. +func resolveCurrentSubscriptions(rows []*rc.SimCardBindPkg, iccids []string) (map[string]*rc.SimCardBindPkg, map[string][]*rc.SimCardBindPkg, error) { + grouped := make(map[string][]*rc.SimCardBindPkg, len(iccids)) + for _, row := range rows { + if row == nil { + continue + } + iccid := strings.TrimSpace(row.Iccid) + if iccid == "" { + continue + } + grouped[iccid] = append(grouped[iccid], row) + } + + current := make(map[string]*rc.SimCardBindPkg, len(iccids)) + future := make(map[string][]*rc.SimCardBindPkg, len(iccids)) + for _, rawICCID := range iccids { + iccid := strings.TrimSpace(rawICCID) + selected, pending, err := selectCurrentSubscription(grouped[iccid], iccid) + if err != nil { + return nil, nil, err + } + current[iccid] = selected + future[iccid] = pending + } + return current, future, nil +} + +func selectCurrentSubscription(rows []*rc.SimCardBindPkg, iccid string) (*rc.SimCardBindPkg, []*rc.SimCardBindPkg, error) { + var ( + effective *rc.SimCardBindPkg + pending []*rc.SimCardBindPkg + expired []*rc.SimCardBindPkg + ) + for _, row := range rows { + if row == nil || strings.TrimSpace(row.Iccid) != strings.TrimSpace(iccid) { + continue + } + if consts.IsAddPackageType(row.PackageType) { + continue + } + switch row.EffectiveStatus { + case consts.EffectiveStatusEffective: + if effective != nil { + return nil, nil, gerror.Newf("同一张卡存在多个生效中的套餐订购记录：%s", iccid) + } + effective = row + case consts.EffectiveStatusPending: + pending = append(pending, row) + case consts.EffectiveStatusExpired: + expired = append(expired, row) + } + } + sortPendingSubscriptions(pending) + if effective != nil { + return effective, pending, nil + } + if len(pending) > 0 { + return pending[0], pending[1:], nil + } + if latest := latestExpiredSubscription(expired); latest != nil { + return latest, nil, nil + } + return nil, nil, gerror.Newf("ICCID没有当前套餐订购记录：%s", iccid) +} + +func latestExpiredSubscription(rows []*rc.SimCardBindPkg) *rc.SimCardBindPkg { + var latest *rc.SimCardBindPkg + for _, row := range rows { + if row == nil { + continue + } + if latest == nil || subscriptionAfter(row, latest) { + latest = row + } + } + return latest +} + +func sortPendingSubscriptions(rows []*rc.SimCardBindPkg) { + if len(rows) <= 1 { + return + } + for i := 1; i < len(rows); i++ { + current := rows[i] + j := i - 1 + for j >= 0 && subscriptionAfter(rows[j], current) { + rows[j+1] = rows[j] + j-- + } + rows[j+1] = current + } +} + +func subscriptionAfter(left, right *rc.SimCardBindPkg) bool { + if compareTime(expectedExpireTime(left), expectedExpireTime(right)) != 0 { + return compareTime(expectedExpireTime(left), expectedExpireTime(right)) > 0 + } + if compareTime(effectiveTime(left), effectiveTime(right)) != 0 { + return compareTime(effectiveTime(left), effectiveTime(right)) > 0 + } + return strings.TrimSpace(subscriptionNo(left)) > strings.TrimSpace(subscriptionNo(right)) +} + +func expectedExpireTime(row *rc.SimCardBindPkg) *gtime.Time { + if row == nil { + return nil + } + return row.ExpectedExpireTime +} + +func effectiveTime(row *rc.SimCardBindPkg) *gtime.Time { + if row == nil { + return nil + } + return row.EffectiveTime +} + +func subscriptionNo(row *rc.SimCardBindPkg) string { + if row == nil { + return "" + } + return row.SubscriptionNo +} + +func compareTime(left, right *gtime.Time) int { + switch { + case left == nil && right == nil: + return 0 + case left == nil: + return -1 + case right == nil: + return 1 + case left.Before(right): + return -1 + case left.After(right): + return 1 + default: + return 0 + } +} + +func validateCurrentSubscription(card *rc.SimCardInfo, bind *rc.SimCardBindPkg, expected subscriptionValidation) error { + if card == nil || bind == nil { + return gerror.New("卡信息或当前订购记录缺失") + } + iccid := strings.TrimSpace(expected.ICCID) + if iccid == "" || strings.TrimSpace(card.Iccid) != iccid || strings.TrimSpace(bind.Iccid) != iccid { + return gerror.New("ICCID不一致") + } + if strings.TrimSpace(expected.TenantID) != "" && + (strings.TrimSpace(card.TenantId) != expected.TenantID || strings.TrimSpace(bind.TenantId) != expected.TenantID) { + return gerror.New("租户不一致") + } + // t_sim_card_info.dept_id is the current card owner. A card allocation + // changes that field, while the historical bind row keeps the department + // that originally purchased the subscription. + if expected.CustomerDeptID <= 0 || card.DeptId != expected.CustomerDeptID { + return gerror.New("客户部门不一致") + } + packageCode := strings.TrimSpace(expected.PackageCode) + if packageCode == "" || strings.TrimSpace(card.CurrentPackageCode) != packageCode || strings.TrimSpace(bind.PackageCode) != packageCode { + return gerror.New("当前套餐不一致") + } + if card.PackageStatus == consts.PackageStatusNone { + return gerror.New("卡没有当前套餐") + } + if msisdn := strings.TrimSpace(expected.MSISDN); msisdn != "" && strings.TrimSpace(card.Msisdn) != msisdn { + return gerror.New("MSISDN不一致") + } + if bindMSISDN := strings.TrimSpace(bind.Msisdn); bindMSISDN != "" && strings.TrimSpace(card.Msisdn) != bindMSISDN { + return gerror.New("订购记录MSISDN不一致") + } + if imsi := strings.TrimSpace(expected.IMSI); imsi != "" && strings.TrimSpace(card.Imsi) != imsi { + return gerror.New("IMSI不一致") + } + if sourceBatchNo := strings.TrimSpace(expected.SourceBatchNo); sourceBatchNo != "" { + bindBatchNo := strings.TrimSpace(bind.BatchNo) + outboundBatchNo := strings.TrimSpace(card.OutboundBatchNo) + if bindBatchNo != sourceBatchNo && outboundBatchNo != sourceBatchNo { + return gerror.New("当前订购来源批次不一致") + } + } + return nil +} diff --git a/internal/service/renewchange/subscription_validation_test.go b/internal/service/renewchange/subscription_validation_test.go new file mode 100644 index 0000000..f94962c --- /dev/null +++ b/internal/service/renewchange/subscription_validation_test.go @@ -0,0 +1,293 @@ +package renewchange + +import ( + "strings" + "testing" + + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" +) + +func TestResolveCurrentSubscriptionsRejectsMissingAndDuplicatePriorities(t *testing.T) { + tests := []struct { + name string + rows []*rc.SimCardBindPkg + }{ + {name: "missing"}, + { + name: "duplicate effective", + rows: []*rc.SimCardBindPkg{ + {Iccid: "8986001", EffectiveStatus: consts.EffectiveStatusEffective}, + {Iccid: "8986001", EffectiveStatus: consts.EffectiveStatusEffective}, + }, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if _, _, err := resolveCurrentSubscriptions(tt.rows, []string{"8986001"}); err == nil { + t.Fatal("resolveCurrentSubscriptions() error = nil") + } + }) + } +} + +func TestSelectCurrentSubscriptionIgnoresCardAddPackage(t *testing.T) { + iccid := "8986000000000000103" + formal := &rc.SimCardBindPkg{ + Iccid: iccid, + PackageType: consts.PackageTypeCard, + PackageCode: "CARD-FORMAL", + EffectiveStatus: consts.EffectiveStatusEffective, + } + addPackage := &rc.SimCardBindPkg{ + Iccid: iccid, + PackageType: consts.PackageTypeCardAddPackage, + PackageCode: "CARD-ADD", + EffectiveStatus: consts.EffectiveStatusEffective, + } + + current, future, err := selectCurrentSubscription([]*rc.SimCardBindPkg{formal, addPackage}, iccid) + if err != nil { + t.Fatalf("selectCurrentSubscription() error = %v", err) + } + if current != formal { + t.Fatalf("current package = %+v, want formal package", current) + } + if len(future) != 0 { + t.Fatalf("future package count = %d, want 0", len(future)) + } +} + +func TestSelectCurrentSubscriptionIgnoresPoolAddPackage(t *testing.T) { + iccid := "8986000000000000104" + formal := &rc.SimCardBindPkg{ + Iccid: iccid, + PackageType: consts.PackageTypePool, + PackageCode: "POOL-FORMAL", + EffectiveStatus: consts.EffectiveStatusEffective, + } + addPackage := &rc.SimCardBindPkg{ + Iccid: iccid, + PackageType: consts.PackageTypePoolAddPackage, + PackageCode: "POOL-ADD", + EffectiveStatus: consts.EffectiveStatusEffective, + } + + current, _, err := selectCurrentSubscription([]*rc.SimCardBindPkg{addPackage, formal}, iccid) + if err != nil { + t.Fatalf("selectCurrentSubscription() error = %v", err) + } + if current != formal { + t.Fatalf("current package = %+v, want formal package", current) + } +} + +func TestSelectCurrentSubscriptionRejectsTwoEffectiveFormalPackages(t *testing.T) { + iccid := "8986000000000000102" + rows := []*rc.SimCardBindPkg{ + { + Iccid: iccid, + PackageType: consts.PackageTypeCard, + PackageCode: "FORMAL-1", + EffectiveStatus: consts.EffectiveStatusEffective, + }, + { + Iccid: iccid, + PackageType: consts.PackageTypePool, + PackageCode: "FORMAL-2", + EffectiveStatus: consts.EffectiveStatusEffective, + }, + } + + _, _, err := selectCurrentSubscription(rows, iccid) + if err == nil { + t.Fatal("selectCurrentSubscription() error = nil, want duplicate formal package error") + } + if !strings.Contains(err.Error(), "多个生效中的套餐订购记录") { + t.Fatalf("error = %q, want duplicate formal package message", err.Error()) + } +} + +func TestResolveCurrentSubscriptionsAllowsChainedPendingRenewals(t *testing.T) { + rows := []*rc.SimCardBindPkg{ + {Iccid: "8986001", SubscriptionNo: "SUB-E", EffectiveStatus: consts.EffectiveStatusEffective}, + {Iccid: "8986001", SubscriptionNo: "SUB-P2", EffectiveStatus: consts.EffectiveStatusPending, ExpectedExpireTime: gtime.NewFromStr("2028-01-01 00:00:00")}, + {Iccid: "8986001", SubscriptionNo: "SUB-P1", EffectiveStatus: consts.EffectiveStatusPending, ExpectedExpireTime: gtime.NewFromStr("2027-01-01 00:00:00")}, + } + current, future, err := resolveCurrentSubscriptions(rows, []string{"8986001"}) + if err != nil { + t.Fatalf("resolveCurrentSubscriptions() error = %v", err) + } + if current["8986001"].SubscriptionNo != "SUB-E" { + t.Fatalf("current subscription = %q, want SUB-E", current["8986001"].SubscriptionNo) + } + if len(future["8986001"]) != 2 { + t.Fatalf("future subscriptions len = %d, want 2", len(future["8986001"])) + } + if future["8986001"][0].SubscriptionNo != "SUB-P1" || future["8986001"][1].SubscriptionNo != "SUB-P2" { + t.Fatalf("future subscriptions order = %+v, want SUB-P1,SUB-P2", future["8986001"]) + } +} + +func TestResolveCurrentSubscriptionsFallsBackToLatestExpiredSubscription(t *testing.T) { + rows := []*rc.SimCardBindPkg{ + { + Iccid: "8986001", + SubscriptionNo: "SUB-X1", + EffectiveStatus: consts.EffectiveStatusExpired, + ExpectedExpireTime: gtime.NewFromStr("2026-01-01 00:00:00"), + }, + { + Iccid: "8986001", + SubscriptionNo: "SUB-X2", + EffectiveStatus: consts.EffectiveStatusExpired, + ExpectedExpireTime: gtime.NewFromStr("2026-07-01 00:00:00"), + }, + { + Iccid: "8986001", + SubscriptionNo: "SUB-T", + EffectiveStatus: consts.EffectiveStatusTerminated, + }, + } + current, future, err := resolveCurrentSubscriptions(rows, []string{"8986001"}) + if err != nil { + t.Fatalf("resolveCurrentSubscriptions() error = %v", err) + } + if current["8986001"] == nil || current["8986001"].SubscriptionNo != "SUB-X2" { + t.Fatalf("current subscription = %+v, want SUB-X2", current["8986001"]) + } + if len(future["8986001"]) != 0 { + t.Fatalf("future subscriptions = %+v, want empty", future["8986001"]) + } +} + +func TestResolveCurrentSubscriptionsPrefersPendingOverExpiredHistory(t *testing.T) { + rows := []*rc.SimCardBindPkg{ + { + Iccid: "8986001", + SubscriptionNo: "SUB-X", + EffectiveStatus: consts.EffectiveStatusExpired, + ExpectedExpireTime: gtime.NewFromStr("2026-07-01 00:00:00"), + }, + { + Iccid: "8986001", + SubscriptionNo: "SUB-P", + EffectiveStatus: consts.EffectiveStatusPending, + ExpectedExpireTime: gtime.NewFromStr("2027-07-01 00:00:00"), + }, + } + current, future, err := resolveCurrentSubscriptions(rows, []string{"8986001"}) + if err != nil { + t.Fatalf("resolveCurrentSubscriptions() error = %v", err) + } + if current["8986001"] == nil || current["8986001"].SubscriptionNo != "SUB-P" { + t.Fatalf("current subscription = %+v, want SUB-P", current["8986001"]) + } + if len(future["8986001"]) != 0 { + t.Fatalf("future subscriptions = %+v, want empty", future["8986001"]) + } +} + +func TestResolveCurrentSubscriptionsStillRejectsOnlyTerminatedOrVoidedHistory(t *testing.T) { + rows := []*rc.SimCardBindPkg{ + {Iccid: "8986001", EffectiveStatus: consts.EffectiveStatusTerminated}, + {Iccid: "8986001", EffectiveStatus: consts.EffectiveStatusVoided}, + } + if _, _, err := resolveCurrentSubscriptions(rows, []string{"8986001"}); err == nil { + t.Fatal("terminated/voided history was accepted as current subscription") + } +} + +func TestValidateCurrentSubscriptionChecksPackageAndBusinessSource(t *testing.T) { + card, bind, expected := validSubscriptionValidationFixture() + if err := validateCurrentSubscription(card, bind, expected); err != nil { + t.Fatalf("validateCurrentSubscription() error = %v", err) + } + + bind.PackageCode = "OTHER" + if err := validateCurrentSubscription(card, bind, expected); err == nil { + t.Fatal("package mismatch was accepted") + } + bind.PackageCode = expected.PackageCode + bind.BatchNo = "OUT-OTHER" + card.OutboundBatchNo = "OUT-OTHER" + if err := validateCurrentSubscription(card, bind, expected); err == nil { + t.Fatal("source batch mismatch was accepted") + } +} + +func TestValidateCurrentSubscriptionIgnoresSourceForInputSelfReference(t *testing.T) { + card, bind, expected := validSubscriptionValidationFixture() + bind.BatchNo = "OUT-ACTUAL" + card.OutboundBatchNo = "" + expected.SourceBatchNo = "" + if err := validateCurrentSubscription(card, bind, expected); err != nil { + t.Fatalf("input source should be ignored, got %v", err) + } +} + +func TestValidateCurrentSubscriptionRejectsIdentityChanges(t *testing.T) { + tests := []struct { + name string + mutate func(*rc.SimCardInfo, *rc.SimCardBindPkg, *subscriptionValidation) + }{ + {name: "ICCID", mutate: func(card *rc.SimCardInfo, _ *rc.SimCardBindPkg, _ *subscriptionValidation) { card.Iccid = "8986002" }}, + {name: "MSISDN", mutate: func(card *rc.SimCardInfo, _ *rc.SimCardBindPkg, _ *subscriptionValidation) { + card.Msisdn = "13900000000" + }}, + {name: "IMSI", mutate: func(card *rc.SimCardInfo, _ *rc.SimCardBindPkg, _ *subscriptionValidation) { card.Imsi = "460002" }}, + {name: "tenant", mutate: func(card *rc.SimCardInfo, _ *rc.SimCardBindPkg, _ *subscriptionValidation) { card.TenantId = "OTHER" }}, + {name: "customer", mutate: func(card *rc.SimCardInfo, _ *rc.SimCardBindPkg, _ *subscriptionValidation) { card.DeptId = 118 }}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + card, bind, expected := validSubscriptionValidationFixture() + tt.mutate(card, bind, &expected) + if err := validateCurrentSubscription(card, bind, expected); err == nil { + t.Fatalf("%s change was accepted", tt.name) + } + }) + } +} + +func TestValidateCurrentSubscriptionAllowsAllocatedCardWithHistoricalBindDept(t *testing.T) { + card, bind, expected := validSubscriptionValidationFixture() + bind.DeptId = 100 + if err := validateCurrentSubscription(card, bind, expected); err != nil { + t.Fatalf("allocated card with historical bind department was rejected: %v", err) + } +} + +func validSubscriptionValidationFixture() (*rc.SimCardInfo, *rc.SimCardBindPkg, subscriptionValidation) { + expected := subscriptionValidation{ + ICCID: "8986001", + MSISDN: "13800000000", + IMSI: "460001", + TenantID: "000000", + CustomerDeptID: 117, + PackageCode: "PKG-OLD", + SourceBatchNo: "OUT-1", + } + card := &rc.SimCardInfo{ + Iccid: expected.ICCID, + Msisdn: expected.MSISDN, + Imsi: expected.IMSI, + DeptId: expected.CustomerDeptID, + CurrentPackageCode: expected.PackageCode, + PackageStatus: consts.PackageStatusEffective, + OutboundBatchNo: expected.SourceBatchNo, + TenantId: expected.TenantID, + } + bind := &rc.SimCardBindPkg{ + Iccid: expected.ICCID, + Msisdn: expected.MSISDN, + DeptId: expected.CustomerDeptID, + PackageCode: expected.PackageCode, + EffectiveStatus: consts.EffectiveStatusEffective, + BatchNo: expected.SourceBatchNo, + TenantId: expected.TenantID, + } + return card, bind, expected +} diff --git a/internal/service/renewchange/types.go b/internal/service/renewchange/types.go new file mode 100644 index 0000000..32a92b1 --- /dev/null +++ b/internal/service/renewchange/types.go @@ -0,0 +1,343 @@ +package renewchange + +import ( + "context" + "fmt" + "sort" + "strings" + "time" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + rc "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/renewchange" +) + +const ( + SourceTypeExcel = 3 + SourceTypeInput = 2 + + // 价格来源 + PriceSourceSystem = "system" + PriceSourceAgentDefault = "agentDefault" + PriceSourceTarget = "target" + + // Agent 价格 scope_type + PriceScopeAgentDefault = 0 + PriceScopeTargetAgent = 1 +) + +// RenewRow 续费/变更行（按业务语义复用同一行结构）。 +type RenewRow struct { + RowNo int + ICCID string + MSISDN string + IMSI string + NewPkgCode string + Reason string + PackageCode string + CustomerID int64 + // 续费使用 + OutboundBatchNo string + // 变更使用 + OldPkgCode string + OldPkgName string + OldBindSubNo string + OldBindOrdNo string + OldEffTime *gtime.Time + OldExpireTime *gtime.Time + // 准备结果快照 + SourceBatchNo string +} + +// RenewPriceSnapshot 续费价格快照（与 Admin 对齐）。 +type RenewPriceSnapshot struct { + CustomerAgentId int64 `json:"customerAgentId"` + PriceOwnerDeptId int64 `json:"priceOwnerDeptId"` + PackageCode string `json:"packageCode"` + PackageName string `json:"packageName"` + BillingMethod int `json:"billingMethod"` + RenewDurationUnit string `json:"renewDurationUnit"` + RenewDurationUnitText string `json:"renewDurationUnitText"` + PriceSource string `json:"priceSource"` + AgentDefaultPriceId int64 `json:"agentDefaultPriceId"` + TargetPriceId int64 `json:"targetPriceId"` + RenewUnitPrice float64 `json:"renewUnitPrice"` + RenewableDurationOptions []int `json:"renewableDurationOptions"` + RenewablePackageCodes []string `json:"renewablePackageCodes"` + RenewablePackages []RenewPackageOption `json:"renewablePackages"` + RenewBeginDate string `json:"renewBeginDate"` + RenewBeginDateMin string `json:"renewBeginDateMin"` + RenewBeginDateMax string `json:"renewBeginDateMax"` + SameRenewBeginDate bool `json:"sameRenewBeginDate"` +} + +type RenewPackageOption struct { + PackageCode string `json:"packageCode"` + PackageName string `json:"packageName"` + RenewUnitPrice float64 `json:"renewUnitPrice"` + RenewableDurationOptions []int `json:"renewableDurationOptions"` +} + +// ChangePriceSnapshot 变更价格快照。 +type ChangePriceSnapshot struct { + CustomerAgentId int64 `json:"customerAgentId"` + NewPackageCode string `json:"newPackageCode"` + NewPackageName string `json:"newPackageName"` + PriceSource string `json:"priceSource"` + AgentDefaultPriceId int64 `json:"agentDefaultPriceId"` + TargetPriceId int64 `json:"targetPriceId"` + ChangeUnitPrice float64 `json:"changeUnitPrice"` + RenewableDurationOptions []int `json:"renewableDurationOptions"` +} + +// ChangePrepareSnapshot 变更预处理快照。 +type ChangePrepareSnapshot struct { + SourceBatchNo string `json:"sourceBatchNo"` + CustomerAgentId int64 `json:"customerAgentId"` + CustomerName string `json:"customerName"` + OldPackageCode string `json:"oldPackageCode"` + OldPackageName string `json:"oldPackageName"` + OldPackageType int `json:"oldPackageType"` + OldPackageFlowMb float64 `json:"oldPackageFlowMb"` + Quantity int `json:"quantity"` +} + +// Task 续费/变更通用任务。 +type Task struct { + Batch *rc.BatchMainRow + Rows []*RenewRow + FailureReason string + ValidCount int + SuccessCount int + FailCount int + + // 续费 + PriceSnapshot *RenewPriceSnapshot + + // 变更 + PrepareSnapshot *ChangePrepareSnapshot + PriceSnapshot2 *ChangePriceSnapshot +} + +// Executor 抽象。 +type Executor interface { + Execute(ctx context.Context, batchNo string) error +} + +// NowFunc 提供时钟，便于测试。 +type NowFunc func() time.Time + +// timeNow 默认时钟。 +func timeNow() time.Time { + return time.Now() +} + +// addMonths 在给定时间上增加 N 个月，保持日时分秒。 +func addMonths(t time.Time, months int) time.Time { + if months <= 0 { + return t + } + targetMonth := time.Date( + t.Year(), + t.Month()+time.Month(months), + 1, + t.Hour(), + t.Minute(), + t.Second(), + t.Nanosecond(), + t.Location(), + ) + day := t.Day() + lastDay := time.Date( + targetMonth.Year(), + targetMonth.Month()+1, + 0, + t.Hour(), + t.Minute(), + t.Second(), + t.Nanosecond(), + t.Location(), + ).Day() + if day > lastDay { + day = lastDay + } + return time.Date( + targetMonth.Year(), + targetMonth.Month(), + day, + t.Hour(), + t.Minute(), + t.Second(), + t.Nanosecond(), + t.Location(), + ) +} + +func renewBeginDateFromExpire(expireAt *gtime.Time) *gtime.Time { + if expireAt == nil { + return nil + } + return dayEndAt(expireAt) +} + +func renewBeginTimeFromExpire(expireAt *gtime.Time) (time.Time, bool) { + beginAt := renewBeginDateFromExpire(expireAt) + if beginAt == nil { + return time.Time{}, false + } + return beginAt.Time, true +} + +func previousMonthEndAt(t *gtime.Time) *gtime.Time { + if t == nil { + return nil + } + tt := t.Time + return gtime.NewFromTime(time.Date(tt.Year(), tt.Month(), 0, 23, 59, 59, 0, tt.Location())) +} + +func dayEndAt(t *gtime.Time) *gtime.Time { + if t == nil { + return nil + } + tt := t.Time + return gtime.NewFromTime(time.Date(tt.Year(), tt.Month(), tt.Day(), 23, 59, 59, 0, tt.Location())) +} + +func renewDurationUnitForBillingMethod(billingMethod int) string { + return consts.BillingMethodDurationUnit(billingMethod) +} + +func renewDurationUnitForPackage(pkg *rc.FlowPackageInfo) string { + return consts.RenewDurationUnitYear +} + +func renewableYearOptions(config *gjson.Json) ([]int, error) { + values, err := rc.RenewableDurationOptions(config) + if err != nil { + return nil, err + } + legacyMonthConfig := len(values) > 0 + for _, value := range values { + if value < 12 || value%12 != 0 { + legacyMonthConfig = false + break + } + } + if !legacyMonthConfig { + return values, nil + } + years := make([]int, 0, len(values)) + for _, value := range values { + years = append(years, value/12) + } + sort.Ints(years) + return years, nil +} + +// changeDurationOptions returns months because the change contract and the +// subscription records store the actual service duration in months. The +// package configuration is normally expressed in years, while legacy rows +// may still contain month values such as [12, 24, 36]. +func changeDurationOptions(config *gjson.Json) ([]int, error) { + values, err := rc.RenewableDurationOptions(config) + if err != nil { + return nil, err + } + legacyMonthConfig := len(values) > 0 + for _, value := range values { + if value < 12 || value%12 != 0 { + legacyMonthConfig = false + break + } + } + if legacyMonthConfig { + return values, nil + } + months := make([]int, 0, len(values)) + for _, value := range values { + if value > 0 { + months = append(months, value*12) + } + } + sort.Ints(months) + if len(months) == 0 { + return nil, gerror.New("package has no valid change duration options") + } + return months, nil +} + +func renewDurationLabel(value int, unit string) string { + if value <= 0 { + return "" + } + return fmt.Sprintf("%d%s", value, consts.RenewDurationUnitText(unit)) +} + +func parseRenewDate(value string, now NowFunc) (*gtime.Time, error) { + value = strings.TrimSpace(value) + if value == "" { + t := now() + return gtime.NewFromTime(t), nil + } + parsed, err := time.ParseInLocation("2006-01-02", value, time.Local) + if err != nil { + parsed, err = time.ParseInLocation("2006-01-02 15:04:05", value, time.Local) + if err != nil { + return nil, err + } + } + return gtime.NewFromTime(parsed), nil +} + +func formatTime(t *gtime.Time) string { + if t == nil { + return "" + } + return t.Format("Y-m-d H:i:s") +} + +func snapshotIntSlice(j *gjson.Json, path string) []int { + if j == nil { + return nil + } + raw := j.Get(path) + if raw == nil || raw.IsNil() { + return nil + } + values := raw.Interfaces() + out := make([]int, 0, len(values)) + for _, v := range values { + switch n := v.(type) { + case int: + out = append(out, n) + case int32: + out = append(out, int(n)) + case int64: + out = append(out, int(n)) + case float32: + out = append(out, int(n)) + case float64: + out = append(out, int(n)) + } + } + sort.Ints(out) + return out +} + +// TxFn 在事务内执行回调。 +type TxFn func(ctx context.Context, fn func(ctx context.Context, tx gdb.TX) error) error + +// DefaultTx 真实事务实现。 +func DefaultTx(database gdb.DB) TxFn { + if database == nil { + database = rc.DB() + } + return func(ctx context.Context, fn func(ctx context.Context, tx gdb.TX) error) error { + return database.Transaction(ctx, fn) + } +} diff --git a/internal/service/simstatus/handler.go b/internal/service/simstatus/handler.go new file mode 100644 index 0000000..45e7679 --- /dev/null +++ b/internal/service/simstatus/handler.go @@ -0,0 +1,131 @@ +package simstatus + +import ( + "context" + "fmt" + "strings" + + zsetconsumer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consumer/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + simstatusrepo "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/simstatus" +) + +const ( + telecomRetryableCode = "S00004" + telecomStatusTemplate = "sim_dx_v1" + chanGatewayInternalError = 9999 + // tokenExpiredCode is the carrier code for an expired/invalid upstream + // token. The gateway already invalidates and refreshes the token once on + // this code; when it still reaches the worker, retrying in a later round + // lets a refreshed token succeed instead of parking the card as a terminal + // business failure. + tokenExpiredCode = 12021 +) + +type StatusClient interface { + QuerySimCardStatus(context.Context, *model.SimCardStatusInput) ([]model.SimBaseRes[model.SimCardStatusItem], error) +} + +type StatusRepository interface { + ApplyBatch(context.Context, []simstatusrepo.Result) error +} + +type Handler struct { + client StatusClient + repo StatusRepository + networkRetries int +} + +func NewHandler(client StatusClient, repo StatusRepository, networkRetries ...int) (*Handler, error) { + if client == nil || repo == nil { + return nil, fmt.Errorf("sim status handler dependencies are incomplete") + } + retries := 0 + if len(networkRetries) > 0 && networkRetries[0] > 0 { + retries = networkRetries[0] + } + return &Handler{client: client, repo: repo, networkRetries: retries}, nil +} + +func (h *Handler) Query(ctx context.Context, task queue.Task) zsetconsumer.Result { + result := zsetconsumer.Result{Task: task} + input := &model.SimCardStatusInput{SimKeys: []*model.SimKey{{ + ChanKey: task.ChanKey, + ICCID: task.ICCID, + IMSI: task.IMSI, + MSISDN: task.MSISDN, + }}} + var ( + output []model.SimBaseRes[model.SimCardStatusItem] + err error + ) + for attempt := 0; attempt <= h.networkRetries; attempt++ { + output, err = h.client.QuerySimCardStatus(ctx, input) + if err == nil { + break + } + if ctx.Err() != nil { + break + } + } + if err != nil { + result.ErrorClass = "network" + result.Retryable = true + result.Message = err.Error() + return result + } + if len(output) == 0 { + result.ErrorClass = "protocol" + result.Retryable = true + result.Message = "chan returned empty status result" + return result + } + item := output[0] + if item.SimKey == nil { + item.SimKey = &model.SimKey{ChanKey: task.ChanKey, ICCID: task.ICCID, IMSI: task.IMSI, MSISDN: task.MSISDN} + } + if task.ICCID != "" && item.SimKey.ICCID != task.ICCID { + result.ErrorClass = "protocol" + result.Retryable = true + result.Message = "chan returned mismatched card identifier" + return result + } + result.Code = item.Code + result.Message = item.Message + result.ErrCode = item.ErrCode + if item.Code != 0 { + result.ErrorClass = "business" + if model.IsTelecomTemplate(task.TempKey) && strings.EqualFold(strings.TrimSpace(item.ErrCode), telecomRetryableCode) { + result.ErrorClass = "carrier_retry" + result.Retryable = true + } else if item.Code == chanGatewayInternalError && strings.TrimSpace(item.ErrCode) == "" { + result.ErrorClass = "gateway" + result.Retryable = true + } else if item.Code == tokenExpiredCode { + result.ErrorClass = "token_expired" + result.Retryable = true + } + return result + } + result.Success = true + status := item.ResData.CardStatus + result.CardStatus = &status + return result +} + +func (h *Handler) Apply(ctx context.Context, results []zsetconsumer.Result) error { + updates := make([]simstatusrepo.Result, 0, len(results)) + for _, result := range results { + if result.Success && result.CardStatus != nil { + updates = append(updates, simstatusrepo.Result{CardID: result.Task.CardID, CardStatus: *result.CardStatus}) + } + } + return h.repo.ApplyBatch(ctx, updates) +} + +// ShouldRetryNextRound is the full-status-sync retry policy. It is passed to +// the Run-aware ZSet consumer only; single-card tasks retain lease recovery. +func ShouldRetryNextRound(result zsetconsumer.Result) bool { + return !result.Success && result.Retryable +} diff --git a/internal/service/simstatus/handler_test.go b/internal/service/simstatus/handler_test.go new file mode 100644 index 0000000..52bcf2b --- /dev/null +++ b/internal/service/simstatus/handler_test.go @@ -0,0 +1,158 @@ +package simstatus + +import ( + "context" + "errors" + "testing" + + zsetconsumer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consumer/zset" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + simstatusrepo "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/simstatus" +) + +type fakeStatusClient struct { + responses [][]model.SimBaseRes[model.SimCardStatusItem] + errors []error + calls int +} + +func (f *fakeStatusClient) QuerySimCardStatus(context.Context, *model.SimCardStatusInput) ([]model.SimBaseRes[model.SimCardStatusItem], error) { + index := f.calls + f.calls++ + var err error + if index < len(f.errors) { + err = f.errors[index] + } + if err != nil { + return nil, err + } + if index < len(f.responses) { + return f.responses[index], nil + } + return nil, nil +} + +type fakeStatusRepository struct { + results []simstatusrepo.Result +} + +func (f *fakeStatusRepository) ApplyBatch(_ context.Context, results []simstatusrepo.Result) error { + f.results = append(f.results, results...) + return nil +} + +func TestHandlerUsesTaskCardWhenResponseOmitsSimKey(t *testing.T) { + client := &fakeStatusClient{responses: [][]model.SimBaseRes[model.SimCardStatusItem]{{ + {Code: 0, ResData: model.SimCardStatusItem{CardStatus: 2}}, + }}} + repo := &fakeStatusRepository{} + handler, err := NewHandler(client, repo) + if err != nil { + t.Fatal(err) + } + + task := queue.Task{CardID: 42, ICCID: "8986000000000042", ChanKey: "mobile-a", TempKey: "sim_yd_v5"} + result := handler.Query(context.Background(), task) + if !result.Success || result.CardStatus == nil || *result.CardStatus != 2 { + t.Fatalf("unexpected result: %+v", result) + } + if err := handler.Apply(context.Background(), []zsetconsumer.Result{result}); err != nil { + t.Fatal(err) + } + if len(repo.results) != 1 || repo.results[0].CardID != 42 || repo.results[0].CardStatus != 2 { + t.Fatalf("unexpected updates: %+v", repo.results) + } +} + +func TestHandlerRetriesNetworkErrorsOnlyForConfiguredAttempts(t *testing.T) { + client := &fakeStatusClient{ + errors: []error{errors.New("temporary-1"), errors.New("temporary-2")}, + responses: [][]model.SimBaseRes[model.SimCardStatusItem]{nil, nil, { + {Code: 0, ResData: model.SimCardStatusItem{CardStatus: 4}}, + }}, + } + handler, err := NewHandler(client, &fakeStatusRepository{}, 2) + if err != nil { + t.Fatal(err) + } + result := handler.Query(context.Background(), queue.Task{CardID: 1, ICCID: "iccid-1", ChanKey: "mobile-a", TempKey: "sim_yd_v5"}) + if !result.Success || client.calls != 3 { + t.Fatalf("success=%v calls=%d result=%+v", result.Success, client.calls, result) + } +} + +func TestHandlerClassifiesRetryableStatusFailures(t *testing.T) { + tests := []struct { + name string + item model.SimBaseRes[model.SimCardStatusItem] + tempKey string + errorClass string + retryable bool + }{ + { + name: "telecom S00004", + item: model.SimBaseRes[model.SimCardStatusItem]{Code: 9999, ErrCode: telecomRetryableCode}, + tempKey: telecomStatusTemplate, + errorClass: "carrier_retry", + retryable: true, + }, + { + name: "telecom v3 S00004", + item: model.SimBaseRes[model.SimCardStatusItem]{Code: 9999, ErrCode: telecomRetryableCode}, + tempKey: "sim_dx_v3", + errorClass: "carrier_retry", + retryable: true, + }, + { + name: "gateway error without carrier code", + item: model.SimBaseRes[model.SimCardStatusItem]{Code: chanGatewayInternalError}, + tempKey: telecomStatusTemplate, + errorClass: "gateway", + retryable: true, + }, + { + name: "token expired is retryable", + item: model.SimBaseRes[model.SimCardStatusItem]{Code: tokenExpiredCode}, + tempKey: "sim_yd_v5", + errorClass: "token_expired", + retryable: true, + }, + { + name: "known business error remains terminal", + item: model.SimBaseRes[model.SimCardStatusItem]{Code: 11011, ErrCode: "11011"}, + tempKey: telecomStatusTemplate, + errorClass: "business", + retryable: false, + }, + { + name: "non telecom S00004 remains terminal", + item: model.SimBaseRes[model.SimCardStatusItem]{Code: 9999, ErrCode: telecomRetryableCode}, + tempKey: "sim_yd_v5", + errorClass: "business", + retryable: false, + }, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + client := &fakeStatusClient{responses: [][]model.SimBaseRes[model.SimCardStatusItem]{{test.item}}} + handler, err := NewHandler(client, &fakeStatusRepository{}) + if err != nil { + t.Fatal(err) + } + result := handler.Query(context.Background(), queue.Task{CardID: 1, ICCID: "iccid-1", ChanKey: "telecom-a", TempKey: test.tempKey}) + if result.ErrorClass != test.errorClass || result.Retryable != test.retryable || result.BlockChannel || result.Success { + t.Fatalf("unexpected result: %+v", result) + } + }) + } +} + +func TestShouldRetryNextRoundUsesRetryableFlag(t *testing.T) { + if !ShouldRetryNextRound(zsetconsumer.Result{Retryable: true}) { + t.Fatal("retryable status failure should advance to the next round") + } + if ShouldRetryNextRound(zsetconsumer.Result{Success: true, Retryable: true}) || ShouldRetryNextRound(zsetconsumer.Result{}) { + t.Fatal("successful and terminal status results must not advance rounds") + } +} diff --git a/internal/service/trafficcutoff/executor.go b/internal/service/trafficcutoff/executor.go new file mode 100644 index 0000000..798987c --- /dev/null +++ b/internal/service/trafficcutoff/executor.go @@ -0,0 +1,172 @@ +package trafficcutoff + +import ( + "context" + "strings" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +const ( + cascadeFailureReason = "same batch has invalid traffic cutoff rows; update skipped" + validationSkipReason = "batch validation failed; traffic cutoff update skipped" + emptyICCIDReason = "ICCID cannot be empty" + duplicateICCIDReason = "duplicate ICCID in batch" + unauthorizedReason = "ICCID is outside the current operator authorization scope" + cardMissingReason = "ICCID does not exist in SIM card records" + statusCanceledReason = "ICCID is already canceled" + statusInconsistentReason = "same batch contains different card statuses" +) + +type executor struct { + repository Repository +} + +func NewExecutor(repository Repository) Executor { + return &executor{repository: repository} +} + +func (e *executor) Execute(ctx context.Context, batchNo string) error { + task, err := e.repository.Load(ctx, strings.TrimSpace(batchNo)) + if err != nil { + return gerror.Wrap(err, "load traffic cutoff batch") + } + if !validateTask(task) { + if err = e.repository.SaveValidationFailure(ctx, task); err != nil { + return gerror.Wrap(err, "save traffic cutoff validation failure") + } + e.saveResultFile(ctx, task) + return nil + } + if err = e.repository.CommitBusiness(ctx, task); err != nil { + return gerror.Wrap(err, "commit traffic cutoff business") + } + e.saveResultFile(ctx, task) + return nil +} + +func (e *executor) saveResultFile(ctx context.Context, task *Task) { + request := batchresult.Request{ + BatchNo: task.Batch.BatchNo, + BatchType: task.Batch.BatchType, + TenantID: task.Batch.TenantID, + Rows: task.ResultRows(), + } + if err := e.repository.SaveResultFile(ctx, task, request); err != nil { + g.Log().Errorf(ctx, "save traffic cutoff result file status failed, batchNo:%s, err:%+v", task.Batch.BatchNo, err) + } +} + +func validateTask(task *Task) bool { + if task == nil || len(task.Details) == 0 { + if task != nil { + task.FailureReason = "traffic cutoff batch has no valid details" + } + return false + } + if task.TrafficCutoffStatus != 1 { + task.FailureReason = "traffic cutoff batch must enable cutoff" + markAllFailed(task, task.FailureReason) + return false + } + if task.TrafficCutoffThreshold <= 0 { + task.FailureReason = "traffic cutoff threshold must be greater than 0" + markAllFailed(task, task.FailureReason) + return false + } + + hasFailure := false + iccids := make(map[string]int, len(task.Details)) + statusSet := make(map[int]struct{}) + for _, detail := range task.Details { + if detail == nil { + hasFailure = true + continue + } + validateDetail(detail) + if detail.Reason != "" { + hasFailure = true + continue + } + iccids[detail.ICCID]++ + statusSet[detail.BossCardStatusBefore] = struct{}{} + } + for _, detail := range task.Details { + if detail == nil || detail.Reason != "" { + continue + } + if iccids[detail.ICCID] > 1 { + detail.Reason = duplicateICCIDReason + hasFailure = true + } + } + if len(statusSet) > 1 { + for _, detail := range task.Details { + if detail != nil && detail.Reason == "" { + detail.Reason = statusInconsistentReason + } + } + hasFailure = true + } + if !hasFailure { + task.ValidationOK = true + return true + } + for _, detail := range task.Details { + if detail != nil && detail.Reason == "" { + detail.Reason = cascadeFailureReason + } + } + task.FailureReason = validationSkipReason + return false +} + +func validateDetail(detail *Detail) { + if detail.Reason != "" { + return + } + detail.ICCID = strings.TrimSpace(detail.ICCID) + detail.InputMSISDN = strings.TrimSpace(detail.InputMSISDN) + detail.InputIMSI = strings.TrimSpace(detail.InputIMSI) + if detail.ICCID == "" { + detail.Reason = emptyICCIDReason + return + } + if !detail.Authorized { + detail.Reason = unauthorizedReason + return + } + if detail.Card == nil { + detail.Reason = cardMissingReason + return + } + detail.MSISDN = strings.TrimSpace(detail.Card.MSISDN) + detail.IMSI = strings.TrimSpace(detail.Card.IMSI) + detail.BossCardStatusBefore = detail.Card.BossCardStatus + detail.TrafficCutoffStatusBefore = detail.Card.TrafficCutoffStatus + detail.TrafficCutoffThresholdBefore = detail.Card.TrafficCutoffThreshold + + if detail.InputMSISDN != "" && detail.InputMSISDN != detail.MSISDN { + detail.Reason = "MSISDN does not match the SIM card record" + return + } + if detail.InputIMSI != "" && detail.InputIMSI != detail.IMSI { + detail.Reason = "IMSI does not match the SIM card record" + return + } + if detail.BossCardStatusBefore == BossCardStatusCanceled { + detail.Reason = statusCanceledReason + return + } +} + +func markAllFailed(task *Task, reason string) { + for _, detail := range task.Details { + if detail != nil { + detail.Reason = reason + } + } +} diff --git a/internal/service/trafficcutoff/executor_test.go b/internal/service/trafficcutoff/executor_test.go new file mode 100644 index 0000000..9b1fa1f --- /dev/null +++ b/internal/service/trafficcutoff/executor_test.go @@ -0,0 +1,178 @@ +package trafficcutoff + +import ( + "context" + "errors" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +func TestValidateTaskFailsOnCanceledCard(t *testing.T) { + task := validTask() + task.Details[0].Card.BossCardStatus = BossCardStatusCanceled + + if validateTask(task) { + t.Fatal("expected canceled card to fail validation") + } + if task.Details[0].Reason != statusCanceledReason { + t.Fatalf("reason = %q, want %q", task.Details[0].Reason, statusCanceledReason) + } +} + +func TestValidateTaskFailsOnMixedCardStatus(t *testing.T) { + task := validTask() + task.Details = append(task.Details, &Detail{ + RowNo: 2, + ICCID: "898600000000002", + Authorized: true, + Card: &Card{ + ID: 2, + ICCID: "898600000000002", + MSISDN: "13800000002", + IMSI: "460001000000002", + BossCardStatus: 5, + }, + }) + + if validateTask(task) { + t.Fatal("expected mixed status batch to fail validation") + } + for _, detail := range task.Details { + if detail.Reason != statusInconsistentReason { + t.Fatalf("detail %d reason = %q, want %q", detail.RowNo, detail.Reason, statusInconsistentReason) + } + } +} + +func TestValidateTaskFailsOnUnauthorizedAndCascades(t *testing.T) { + task := validTask() + task.Details = append(task.Details, &Detail{ + RowNo: 2, + ICCID: "898600000000002", + Authorized: false, + Card: &Card{ + ID: 2, + ICCID: "898600000000002", + MSISDN: "13800000002", + IMSI: "460001000000002", + BossCardStatus: 4, + }, + }) + + if validateTask(task) { + t.Fatal("expected unauthorized row to fail validation") + } + if task.Details[1].Reason != unauthorizedReason { + t.Fatalf("unauthorized reason = %q", task.Details[1].Reason) + } + if task.Details[0].Reason != cascadeFailureReason { + t.Fatalf("cascade reason = %q", task.Details[0].Reason) + } +} + +func TestExecuteHappyPathCommitsAndSavesResult(t *testing.T) { + task := validTask() + repo := &fakeRepository{task: task} + + if err := NewExecutor(repo).Execute(context.Background(), "DCT-1"); err != nil { + t.Fatalf("Execute() error = %v", err) + } + if !repo.committed { + t.Fatal("expected business commit") + } + if repo.validationSaved { + t.Fatal("did not expect validation failure save") + } + if !repo.resultSaved { + t.Fatal("expected result file save") + } + if repo.resultRequest.BatchType != 107 || len(repo.resultRequest.Rows) != 1 { + t.Fatalf("unexpected result request: %+v", repo.resultRequest) + } +} + +func TestExecuteValidationFailureSkipsCommit(t *testing.T) { + task := validTask() + task.Details[0].Authorized = false + repo := &fakeRepository{task: task} + + if err := NewExecutor(repo).Execute(context.Background(), "DCT-2"); err != nil { + t.Fatalf("Execute() error = %v", err) + } + if !repo.validationSaved { + t.Fatal("expected validation failure save") + } + if repo.committed { + t.Fatal("did not expect business commit") + } + if !repo.resultSaved { + t.Fatal("expected result file save") + } +} + +func TestExecuteCommitFailurePropagatesAndSkipsResult(t *testing.T) { + task := validTask() + commitErr := errors.New("transaction failed") + repo := &fakeRepository{task: task, commitErr: commitErr} + + err := NewExecutor(repo).Execute(context.Background(), "DCT-3") + if err == nil || !errors.Is(err, commitErr) { + t.Fatalf("Execute() error = %v, want wrap of %v", err, commitErr) + } + if repo.resultSaved { + t.Fatal("result file should not be saved after commit error") + } +} + +func validTask() *Task { + return &Task{ + Batch: Batch{BatchNo: "DCT-1", BatchType: 107, TenantID: "000000"}, + TrafficCutoffStatus: 1, + TrafficCutoffThreshold: 1024.5, + ThresholdUnit: "MB", + Details: []*Detail{ + { + RowNo: 1, + ICCID: "898600000000001", + Authorized: true, + Card: &Card{ + ID: 1, + ICCID: "898600000000001", + MSISDN: "13800000001", + IMSI: "460001000000001", + BossCardStatus: 4, + }, + }, + }, + } +} + +type fakeRepository struct { + task *Task + committed bool + commitErr error + validationSaved bool + resultSaved bool + resultRequest batchresult.Request +} + +func (r *fakeRepository) Load(_ context.Context, _ string) (*Task, error) { + return r.task, nil +} + +func (r *fakeRepository) SaveValidationFailure(_ context.Context, _ *Task) error { + r.validationSaved = true + return nil +} + +func (r *fakeRepository) CommitBusiness(_ context.Context, _ *Task) error { + r.committed = true + return r.commitErr +} + +func (r *fakeRepository) SaveResultFile(_ context.Context, _ *Task, request batchresult.Request) error { + r.resultSaved = true + r.resultRequest = request + return nil +} diff --git a/internal/service/trafficcutoff/types.go b/internal/service/trafficcutoff/types.go new file mode 100644 index 0000000..33fa368 --- /dev/null +++ b/internal/service/trafficcutoff/types.go @@ -0,0 +1,135 @@ +package trafficcutoff + +import ( + "context" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +const BossCardStatusCanceled = 9 + +type Batch struct { + ID int64 + BatchNo string + BatchType int + TenantID string + DeptID int64 + CreatedBy int64 + CreatedDept int64 + StartTime *gtime.Time + BatchInfo *gjson.Json +} + +type Card struct { + ID int64 + ICCID string + MSISDN string + IMSI string + BossCardStatus int + TrafficCutoffStatus int + TrafficCutoffThreshold float64 +} + +type Detail struct { + ID int64 + RowNo int + ICCID string + InputMSISDN string + InputIMSI string + MSISDN string + IMSI string + Authorized bool + BossCardStatusBefore int + TrafficCutoffStatusBefore int + TrafficCutoffThresholdBefore float64 + TrafficCutoffCrossMonthHold int + TrafficCutoffStatusAfter int + TrafficCutoffThresholdAfter float64 + Reason string + Card *Card +} + +type Task struct { + Batch Batch + TrafficCutoffStatus int + TrafficCutoffThreshold float64 + ThresholdUnit string + Details []*Detail + ValidationOK bool + FailureReason string +} + +type Repository interface { + Load(ctx context.Context, batchNo string) (*Task, error) + SaveValidationFailure(ctx context.Context, task *Task) error + CommitBusiness(ctx context.Context, task *Task) error + SaveResultFile(ctx context.Context, task *Task, request batchresult.Request) error +} + +type Executor interface { + Execute(ctx context.Context, batchNo string) error +} + +func (d *Detail) Snapshot(task *Task) *gjson.Json { + if d == nil { + return gjson.New(nil) + } + return gjson.New(map[string]any{ + "rowNo": d.RowNo, + "ICCID": d.ICCID, + "MSISDN": d.MSISDN, + "IMSI": d.IMSI, + "inputMsisdn": d.InputMSISDN, + "inputImsi": d.InputIMSI, + "authorized": d.Authorized, + "operation": "dataCut", + "trafficCutoffStatusBefore": d.TrafficCutoffStatusBefore, + "trafficCutoffThresholdBefore": d.TrafficCutoffThresholdBefore, + "trafficCutoffStatusAfter": d.TrafficCutoffStatusAfter, + "trafficCutoffThresholdAfter": d.TrafficCutoffThresholdAfter, + "trafficCutoffThreshold": task.TrafficCutoffThreshold, + "trafficCutoffCrossMonthHold": d.TrafficCutoffCrossMonthHold, + "thresholdUnit": task.ThresholdUnit, + "bossCardStatusBefore": d.BossCardStatusBefore, + "errorReason": d.Reason, + }) +} + +func (t *Task) ResultRows() []batchresult.Row { + rows := make([]batchresult.Row, 0, len(t.Details)) + for _, detail := range t.Details { + if detail == nil { + continue + } + rows = append(rows, batchresult.Row{ + RowNo: detail.RowNo, + ICCID: detail.ICCID, + MSISDN: detail.MSISDN, + IMSI: detail.IMSI, + Operation: "达量断网", + Success: detail.Reason == "", + Reason: detail.Reason, + }) + } + return rows +} + +func (t *Task) SuccessCount() int { + count := 0 + for _, detail := range t.Details { + if detail != nil && detail.Reason == "" { + count++ + } + } + return count +} + +func (t *Task) FailCount() int { + if t == nil { + return 0 + } + return len(t.Details) - t.SuccessCount() +} diff --git a/internal/service/trafficcutoffconfig/executor.go b/internal/service/trafficcutoffconfig/executor.go new file mode 100644 index 0000000..1f836eb --- /dev/null +++ b/internal/service/trafficcutoffconfig/executor.go @@ -0,0 +1,142 @@ +package trafficcutoffconfig + +import ( + "context" + "strings" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +const ( + noDetailsReason = "达量断网配置批次没有可处理明细" + validationFailureReason = "部分达量断网配置校验失败" + emptyICCIDReason = "ICCID不能为空" + duplicateICCIDReason = "ICCID在批次内重复" + unauthorizedReason = "ICCID不在当前操作员授权范围内" + cardMissingReason = "ICCID不存在或不属于当前租户" + msisdnMismatchReason = "MSISDN与卡档案不一致" + imsiMismatchReason = "IMSI与卡档案不一致" + flowPoolCardReason = "流量池卡不支持达量断网配置" + packageTotalExceededReason = "达量断网阈值不能大于套餐总流量" +) + +type executor struct { + repository *Repository +} + +const trafficCutoffConfigProcessingRetryAfter = 2 * time.Minute + +func NewExecutor() *executor { + return &executor{repository: NewRepository()} +} + +func (e *executor) CanExecuteProcessing(_ context.Context, main *entity.BatchMain) (bool, error) { + if main == nil || main.BatchType != consts.BatchTypeTrafficCutoffConfig || main.BatchAsyncStatus != consts.BatchAsyncStatusProcessing { + return false, nil + } + if main.StartTime == nil || main.StartTime.IsZero() { + return true, nil + } + return gtime.Now().Sub(main.StartTime) >= trafficCutoffConfigProcessingRetryAfter, nil +} + +func (e *executor) Execute(ctx context.Context, batchNo string) error { + task, err := e.repository.Load(ctx, strings.TrimSpace(batchNo)) + if err != nil { + return gerror.Wrap(err, "load traffic cutoff config batch failed") + } + validateTask(task) + return gerror.Wrap(e.repository.Commit(ctx, task), "commit traffic cutoff config batch failed") +} + +func validateTask(task *Task) { + if task == nil || len(task.Details) == 0 { + if task != nil { + task.FailureReason = noDetailsReason + } + return + } + + iccidCounts := make(map[string]int, len(task.Details)) + for _, detail := range task.Details { + if detail == nil { + continue + } + detail.ICCID = strings.TrimSpace(detail.ICCID) + detail.MSISDN = strings.TrimSpace(detail.MSISDN) + detail.IMSI = strings.TrimSpace(detail.IMSI) + if detail.ICCID != "" { + iccidCounts[detail.ICCID]++ + } + } + + for _, detail := range task.Details { + validateDetail(task, detail) + if detail != nil && detail.Reason == "" && iccidCounts[detail.ICCID] > 1 { + detail.Reason = duplicateICCIDReason + } + } + _, failCount := countResult(task) + if failCount > 0 { + task.FailureReason = validationFailureReason + } else { + task.FailureReason = "" + } +} + +func validateDetail(task *Task, detail *Detail) { + if detail == nil || detail.Reason != "" { + return + } + if detail.ICCID == "" { + detail.Reason = emptyICCIDReason + return + } + if !detail.Authorized { + detail.Reason = unauthorizedReason + return + } + if detail.Card == nil { + detail.Reason = cardMissingReason + return + } + detail.Card.ICCID = strings.TrimSpace(detail.Card.ICCID) + detail.Card.MSISDN = strings.TrimSpace(detail.Card.MSISDN) + detail.Card.IMSI = strings.TrimSpace(detail.Card.IMSI) + if detail.MSISDN != "" && detail.MSISDN != detail.Card.MSISDN { + detail.Reason = msisdnMismatchReason + return + } + if detail.IMSI != "" && detail.IMSI != detail.Card.IMSI { + detail.Reason = imsiMismatchReason + return + } + if detail.Card.IsFlowPool != 0 { + detail.Reason = flowPoolCardReason + return + } + if task.Config.TrafficCutoffStatus == 1 && + detail.Card.PackageTotalFlowMb > 0 && + task.Config.TrafficCutoffThreshold > detail.Card.PackageTotalFlowMb { + detail.Reason = packageTotalExceededReason + } +} + +func countResult(task *Task) (successCount, failCount int) { + if task == nil { + return 0, 0 + } + for _, detail := range task.Details { + if detail != nil && detail.Success() { + successCount++ + } else { + failCount++ + } + } + return successCount, failCount +} diff --git a/internal/service/trafficcutoffconfig/executor_test.go b/internal/service/trafficcutoffconfig/executor_test.go new file mode 100644 index 0000000..9be444b --- /dev/null +++ b/internal/service/trafficcutoffconfig/executor_test.go @@ -0,0 +1,190 @@ +package trafficcutoffconfig + +import "testing" + +func TestValidateTaskAllowsDifferentCardStatusesAndMissingEffectivePackage(t *testing.T) { + task := validConfigTask(512) + task.Details = []*Detail{ + validConfigDetail(1, "898600000000001"), + validConfigDetail(2, "898600000000002"), + validConfigDetail(3, "898600000000003"), + } + task.Details[0].Card.BossCardStatus = 1 + task.Details[0].Card.PackageStatus = 0 + task.Details[0].Card.PackageTotalFlowMb = 0 + task.Details[1].Card.BossCardStatus = 4 + task.Details[1].Card.PackageStatus = 1 + task.Details[1].Card.PackageTotalFlowMb = 0 + task.Details[2].Card.BossCardStatus = 9 + task.Details[2].Card.PackageStatus = 3 + task.Details[2].Card.PackageTotalFlowMb = 0 + + validateTask(task) + + if success, fail := countResult(task); success != 3 || fail != 0 { + t.Fatalf("countResult() = (%d, %d), want (3, 0); task = %+v", success, fail, task) + } +} + +func TestValidateTaskAllowsPartialSuccessForThresholdGreaterThanPositivePackageTotal(t *testing.T) { + task := validConfigTask(512) + task.Details = []*Detail{ + validConfigDetail(1, "898600000000001"), + validConfigDetail(2, "898600000000002"), + } + task.Details[0].Card.PackageTotalFlowMb = 128 + task.Details[1].Card.PackageTotalFlowMb = 0 + + validateTask(task) + + assertDetailReason(t, task.Details[0], packageTotalExceededReason) + assertDetailReason(t, task.Details[1], "") + if success, fail := countResult(task); success != 1 || fail != 1 { + t.Fatalf("countResult() = (%d, %d), want (1, 1)", success, fail) + } +} + +func TestValidateTaskAllowsPartialSuccessForInvalidRows(t *testing.T) { + tests := []struct { + name string + details []*Detail + wantReason map[int]string + }{ + { + name: "empty ICCID", + details: []*Detail{ + {RowNo: 1, ICCID: " ", Authorized: true, Card: validConfigCard(1, "")}, + validConfigDetail(2, "898600000000002"), + }, + wantReason: map[int]string{1: emptyICCIDReason, 2: ""}, + }, + { + name: "duplicate ICCID", + details: []*Detail{ + validConfigDetail(1, "898600000000001"), + validConfigDetail(2, "898600000000001"), + validConfigDetail(3, "898600000000003"), + }, + wantReason: map[int]string{1: duplicateICCIDReason, 2: duplicateICCIDReason, 3: ""}, + }, + { + name: "unauthorized", + details: []*Detail{ + validConfigDetail(1, "898600000000001"), + validConfigDetail(2, "898600000000002"), + }, + wantReason: map[int]string{1: unauthorizedReason, 2: ""}, + }, + { + name: "missing card", + details: []*Detail{ + {RowNo: 1, ICCID: "898600000000001", Authorized: true}, + validConfigDetail(2, "898600000000002"), + }, + wantReason: map[int]string{1: cardMissingReason, 2: ""}, + }, + { + name: "MSISDN mismatch", + details: []*Detail{ + validConfigDetail(1, "898600000000001"), + validConfigDetail(2, "898600000000002"), + }, + wantReason: map[int]string{1: msisdnMismatchReason, 2: ""}, + }, + { + name: "IMSI mismatch", + details: []*Detail{ + validConfigDetail(1, "898600000000001"), + validConfigDetail(2, "898600000000002"), + }, + wantReason: map[int]string{1: imsiMismatchReason, 2: ""}, + }, + { + name: "flow pool card", + details: []*Detail{ + validConfigDetail(1, "898600000000001"), + validConfigDetail(2, "898600000000002"), + }, + wantReason: map[int]string{1: flowPoolCardReason, 2: ""}, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + task := validConfigTask(512) + task.Details = tt.details + switch tt.name { + case "unauthorized": + task.Details[0].Authorized = false + case "MSISDN mismatch": + task.Details[0].MSISDN = "13900009999" + case "IMSI mismatch": + task.Details[0].IMSI = "460009999999999" + case "flow pool card": + task.Details[0].Card.IsFlowPool = 1 + } + + validateTask(task) + + for _, detail := range task.Details { + assertDetailReason(t, detail, tt.wantReason[detail.RowNo]) + } + if success, fail := countResult(task); success != 1 || fail != len(task.Details)-1 { + t.Fatalf("countResult() = (%d, %d), want (1, %d)", success, fail, len(task.Details)-1) + } + }) + } +} + +func validConfigTask(threshold float64) *Task { + return &Task{ + Batch: Batch{ + BatchNo: "TCC-41", + BatchType: 41, + TenantID: "000000", + CreatedBy: 7, + }, + Config: Config{ + TrafficCutoffStatus: 1, + TrafficCutoffThreshold: threshold, + }, + Details: []*Detail{validConfigDetail(1, "898600000000001")}, + } +} + +func validConfigDetail(rowNo int, iccid string) *Detail { + card := validConfigCard(rowNo, iccid) + return &Detail{ + ID: int64(rowNo), + RowNo: rowNo, + ICCID: iccid, + MSISDN: card.MSISDN, + IMSI: card.IMSI, + Authorized: true, + Card: card, + } +} + +func validConfigCard(rowNo int, iccid string) *Card { + return &Card{ + ID: int64(rowNo), + ICCID: iccid, + MSISDN: "1380000000" + string(rune('0'+rowNo)), + IMSI: "46000100000000" + string(rune('0'+rowNo)), + PackageStatus: 0, + BossCardStatus: rowNo, + PackageTotalFlowMb: 1024, + TrafficCutoffStatus: 0, + TrafficCutoffThreshold: 0, + } +} + +func assertDetailReason(t *testing.T, detail *Detail, want string) { + t.Helper() + if detail == nil { + t.Fatal("detail is nil") + } + if detail.Reason != want { + t.Fatalf("row %d reason = %q, want %q", detail.RowNo, detail.Reason, want) + } +} diff --git a/internal/service/trafficcutoffconfig/repository.go b/internal/service/trafficcutoffconfig/repository.go new file mode 100644 index 0000000..7c21172 --- /dev/null +++ b/internal/service/trafficcutoffconfig/repository.go @@ -0,0 +1,499 @@ +package trafficcutoffconfig + +import ( + "context" + "math" + "sort" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +type Repository struct { + db gdb.DB + results batchresult.Generator + lockEnabled bool +} + +func NewRepository() *Repository { + return &Repository{results: batchresult.New(), lockEnabled: true} +} + +func (r *Repository) database() gdb.DB { + if r.db != nil { + return r.db + } + return g.DB() +} + +func (r *Repository) applyLock(model *gdb.Model) *gdb.Model { + if r.lockEnabled { + return model.LockUpdate() + } + return model +} + +func (r *Repository) Load(ctx context.Context, batchNo string) (*Task, error) { + batchNo = strings.TrimSpace(batchNo) + if batchNo == "" { + return nil, gerror.New("达量断网配置批次号不能为空") + } + + var main *entity.BatchMain + if err := r.database().Model(dao.BatchMain.Table()).Safe().Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeTrafficCutoffConfig). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Scan(&main); err != nil { + return nil, gerror.Wrap(err, "查询达量断网配置批次主表失败") + } + if main == nil || main.Id == 0 { + return nil, gerror.New("处理中达量断网配置批次不存在") + } + config, err := configFromBatchInfo(main.BatchInfo) + if err != nil { + return nil, err + } + + var batchDetails []*entity.BatchDetail + if err = r.database().Model(dao.BatchDetail.Table()).Safe().Ctx(ctx). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeTrafficCutoffConfig). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + OrderAsc(dao.BatchDetail.Columns().RowNo). + Scan(&batchDetails); err != nil { + return nil, gerror.Wrap(err, "查询达量断网配置批次明细失败") + } + if len(batchDetails) == 0 { + return nil, gerror.New("达量断网配置批次没有运行中明细") + } + + task := &Task{ + Batch: Batch{ + ID: main.Id, + BatchNo: main.BatchNo, + BatchType: main.BatchType, + TenantID: main.TenantId, + CreatedDept: main.CreatedDept, + CreatedBy: main.CreatedBy, + StartTime: main.StartTime, + BatchInfo: main.BatchInfo, + }, + Config: config, + Details: make([]*Detail, 0, len(batchDetails)), + } + for _, batchDetail := range batchDetails { + if batchDetail == nil { + continue + } + detail := &Detail{ + ID: batchDetail.Id, + RowNo: batchDetail.RowNo, + ICCID: strings.TrimSpace(batchDetail.ItemId), + Reason: strings.TrimSpace(batchDetail.ErrorReason), + } + if batchDetail.Snapshot != nil { + if value := firstSnapshot(batchDetail.Snapshot, "ICCID", "iccid"); value != "" { + detail.ICCID = value + } + detail.MSISDN = firstSnapshot(batchDetail.Snapshot, "MSISDN", "msisdn", "inputMsisdn", "input_msisdn") + detail.IMSI = firstSnapshot(batchDetail.Snapshot, "IMSI", "imsi", "inputImsi", "input_imsi") + detail.Authorized = batchDetail.Snapshot.Get("authorized").Bool() + } + task.Details = append(task.Details, detail) + } + if len(task.Details) == 0 { + return nil, gerror.New("达量断网配置批次没有可读取明细") + } + if err = r.loadCards(ctx, task); err != nil { + return nil, err + } + return task, nil +} + +func firstSnapshot(snapshot *gjson.Json, keys ...string) string { + if snapshot == nil { + return "" + } + for _, key := range keys { + if value := strings.TrimSpace(snapshot.Get(key).String()); value != "" { + return value + } + } + return "" +} + +func (r *Repository) loadCards(ctx context.Context, task *Task) error { + iccids := make([]string, 0, len(task.Details)) + for _, detail := range task.Details { + if detail != nil && strings.TrimSpace(detail.ICCID) != "" { + iccids = append(iccids, strings.TrimSpace(detail.ICCID)) + } + } + iccids = uniqueSortedStrings(iccids) + if len(iccids) == 0 { + return nil + } + + var cards []*entity.TSimCardInfo + columns := dao.TSimCardInfo.Columns() + if err := r.database().Model(dao.TSimCardInfo.Table()).Safe().Ctx(ctx). + Where(columns.TenantId, task.Batch.TenantID). + WhereIn(columns.Iccid, iccids). + Scan(&cards); err != nil { + return gerror.Wrap(err, "查询SIM卡档案失败") + } + byICCID := make(map[string]*entity.TSimCardInfo, len(cards)) + for _, card := range cards { + if card != nil { + byICCID[strings.TrimSpace(card.Iccid)] = card + } + } + for _, detail := range task.Details { + if detail == nil { + continue + } + card := byICCID[strings.TrimSpace(detail.ICCID)] + if card == nil { + continue + } + detail.Card = &Card{ + ID: card.Id, + ICCID: strings.TrimSpace(card.Iccid), + MSISDN: strings.TrimSpace(card.Msisdn), + IMSI: strings.TrimSpace(card.Imsi), + CurrentPackageCode: strings.TrimSpace(card.CurrentPackageCode), + PackageStatus: card.PackageStatus, + BossCardStatus: card.BossCardStatus, + IsFlowPool: card.IsFlowPool, + PackageTotalFlowMb: card.PackageTotalFlowMb, + TrafficCutoffStatus: card.TrafficCutoffStatus, + TrafficCutoffThreshold: card.TrafficCutoffThreshold, + } + } + return nil +} + +func (r *Repository) Commit(ctx context.Context, task *Task) error { + if task == nil { + return gerror.New("达量断网配置任务不能为空") + } + successCount, failCount := countResult(task) + artifact, artifactErr := r.createResultFile(ctx, task) + if artifactErr != nil { + g.Log().Warningf(ctx, "traffic cutoff config result file generation failed, batchNo:%s, err:%+v", task.Batch.BatchNo, artifactErr) + } + + now := gtime.Now() + err := r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + if successCount > 0 { + if err := r.updateAllCardConfigs(ctx, tx, task); err != nil { + return err + } + } + if err := r.updateDetails(ctx, tx, task); err != nil { + return err + } + + var ossID, downloadID int64 + if artifactErr == nil { + var err error + ossID, downloadID, err = r.persistArtifact(ctx, tx, task, artifact, successCount, failCount, now) + if err != nil { + return err + } + } + return r.finalizeBatch(ctx, tx, task, successCount, failCount, ossID, downloadID, now, artifactErr != nil) + }) + if err != nil && artifactErr == nil && artifact.Path != "" { + batchresult.RemoveLocalFile(artifact) + } + return gerror.Wrap(err, "保存达量断网配置批次结果失败") +} + +func (r *Repository) updateAllCardConfigs(ctx context.Context, tx gdb.TX, task *Task) error { + cardIDs := successDetailCardIDs(task.Details) + if len(cardIDs) == 0 { + return nil + } + columns := dao.TSimCardInfo.Columns() + var lockedCards []*entity.TSimCardInfo + model := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()).Safe(). + Fields(columns.Id). + Where(columns.TenantId, task.Batch.TenantID). + WhereIn(columns.Id, cardIDs) + if err := r.applyLock(model).Scan(&lockedCards); err != nil { + return gerror.Wrap(err, "锁定达量断网配置卡档案失败") + } + if len(lockedCards) != len(cardIDs) { + return gerror.New("部分达量断网配置卡档案已发生变化") + } + _, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()).Safe(). + Where(columns.TenantId, task.Batch.TenantID). + WhereIn(columns.Id, cardIDs). + Data(trafficCutoffConfigData(task)).Update() + if err != nil { + return gerror.Wrap(err, "保存达量断网配置失败") + } + return nil +} + +func (r *Repository) updateDetails(ctx context.Context, tx gdb.TX, task *Task) error { + for _, detail := range task.Details { + if detail == nil { + continue + } + if err := r.updateDetail(ctx, tx, task, detail); err != nil { + return err + } + } + return nil +} + +func (r *Repository) updateDetail(ctx context.Context, tx gdb.TX, task *Task, detail *Detail) error { + status := consts.BatchDetailStatusFailed + if detail.Success() { + status = consts.BatchDetailStatusSuccess + } + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeTrafficCutoffConfig). + Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning). + Data(do.BatchDetail{ + DetailStatus: status, + ErrorReason: detail.Reason, + Snapshot: detail.Snapshot(task.Config), + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "更新达量断网配置批次明细失败") + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.Newf("达量断网配置批次明细%d已不是运行中状态", detail.ID) + } + return nil +} + +func (r *Repository) finalizeBatch(ctx context.Context, tx gdb.TX, task *Task, successCount, failCount int, ossID, downloadID int64, now *gtime.Time, resultFileFailed bool) error { + batchAsyncStatus, runResult := batchResult(successCount, failCount) + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeTrafficCutoffConfig). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Data(do.BatchMain{ + BatchAsyncStatus: batchAsyncStatus, + RunResult: runResult, + TotalCount: len(task.Details), + ValidCount: successCount, + SuccessCount: successCount, + FailCount: failCount, + FailReason: task.FailureReason, + BatchInfo: mergeBatchInfo(task.Batch.BatchInfo, ossID, downloadID, resultFileFailed), + EndTime: now, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "更新达量断网配置批次主表失败") + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.New("达量断网配置批次已不是处理中状态") + } + return nil +} + +func (r *Repository) createResultFile(ctx context.Context, task *Task) (batchresult.Artifact, error) { + if r.results == nil { + return batchresult.Artifact{}, gerror.New("result generator is not configured") + } + return r.results.Create(ctx, batchresult.Request{ + BatchNo: task.Batch.BatchNo, + BatchType: task.Batch.BatchType, + TenantID: task.Batch.TenantID, + Rows: task.ResultRows(), + }) +} + +func (r *Repository) persistArtifact(ctx context.Context, tx gdb.TX, task *Task, artifact batchresult.Artifact, successCount, failCount int, now *gtime.Time) (int64, int64, error) { + ossID, err := tx.Ctx(ctx).Model(dao.SysOss.Table()).Data(do.SysOss{ + TenantId: task.Batch.TenantID, + FileName: artifact.FileName, + OriginalName: artifact.OriginalName, + FileSuffix: artifact.FileSuffix, + Path: artifact.Path, + Url: artifact.URL, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + Service: "local", + Md5: artifact.MD5, + FileSize: artifact.Size, + }).InsertAndGetId() + if err != nil { + return 0, 0, gerror.Wrap(err, "写入达量断网配置结果文件OSS失败") + } + downloadID, err := tx.Ctx(ctx).Model(dao.TResourceBatchDownload.Table()).Data(do.TResourceBatchDownload{ + BatchNo: task.Batch.BatchNo, + TaskName: batchresult.TaskName(task.Batch.BatchType), + ResourceType: consts.ResourceTypeExcel, + ExportScene: consts.BatchTypeTrafficCutoffConfig, + TotalCount: len(task.Details), + SuccessCount: successCount, + FailCount: failCount, + DownloadStatus: consts.DownloadStatusGenerated, + OssId: ossID, + FileName: artifact.FileName, + FileExt: artifact.FileSuffix, + FileUrl: artifact.URL, + FilePath: artifact.Path, + FileSize: artifact.Size, + FileMd5: artifact.MD5, + DownloadCount: 0, + MaxDownloadCount: 0, + DeptId: task.Batch.CreatedDept, + StartTime: task.Batch.StartTime, + FinishTime: now, + TenantId: task.Batch.TenantID, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + }).InsertAndGetId() + if err != nil { + return 0, 0, gerror.Wrap(err, "写入达量断网配置结果下载记录失败") + } + return ossID, downloadID, nil +} + +func configFromBatchInfo(info *gjson.Json) (Config, error) { + if info == nil || info.IsNil() { + return Config{}, gerror.New("达量断网配置批次缺少批次参数") + } + status := info.Get("trafficCutoffStatus") + if status.IsNil() { + return Config{}, gerror.New("达量断网配置批次缺少trafficCutoffStatus") + } + config := Config{ + TrafficCutoffCrossMonthHold: info.Get("trafficCutoffCrossMonthHold").Int(), + TrafficCutoffStatus: status.Int(), + TrafficCutoffThreshold: info.Get("trafficCutoffThreshold").Float64(), + } + if config.TrafficCutoffStatus != 0 && config.TrafficCutoffStatus != 1 && config.TrafficCutoffStatus != 2 { + return Config{}, gerror.New("达量断网启停状态无效") + } + if config.TrafficCutoffThreshold < 0 { + return Config{}, gerror.New("达量断网阈值不能小于0") + } + if config.TrafficCutoffStatus == 0 && config.TrafficCutoffThreshold != 0 { + return Config{}, gerror.New("未配置达量断网时阈值必须为0") + } + if config.TrafficCutoffStatus == 1 && config.TrafficCutoffThreshold <= 0 { + return Config{}, gerror.New("开启达量断网时阈值必须大于0") + } + if config.TrafficCutoffCrossMonthHold != 0 && config.TrafficCutoffCrossMonthHold != 1 { + return Config{}, gerror.New("invalid cross-month traffic cutoff hold value") + } + if config.TrafficCutoffStatus != 1 && config.TrafficCutoffCrossMonthHold != 0 { + return Config{}, gerror.New("cross-month hold requires traffic cutoff to be enabled") + } + rounded := math.Round(config.TrafficCutoffThreshold*1000) / 1000 + if math.Abs(config.TrafficCutoffThreshold-rounded) > 0.0000001 { + return Config{}, gerror.New("达量断网阈值最多保留3位小数") + } + return config, nil +} + +func trafficCutoffConfigData(task *Task) do.TSimCardTrafficCutoffConfig { + data := do.TSimCardTrafficCutoffConfig{ + TrafficCutoffStatus: task.Config.TrafficCutoffStatus, + TrafficCutoffThreshold: task.Config.TrafficCutoffThreshold, + TrafficCutoffCrossMonthHold: task.Config.TrafficCutoffCrossMonthHold, + UpdatedBy: task.Batch.CreatedBy, + } + if task.Config.TrafficCutoffStatus != 1 || task.Config.TrafficCutoffCrossMonthHold == 0 { + data.TrafficCutoffTriggeredAt = gdb.Raw("NULL") + } + return data +} + +func mergeBatchInfo(info *gjson.Json, ossID, downloadID int64, resultFileFailed bool) *gjson.Json { + data := make(map[string]any) + if info != nil { + for key, value := range info.Map() { + data[key] = value + } + } + if ossID > 0 { + data["resultFileOssId"] = ossID + } + if resultFileFailed { + data["resultFile"] = map[string]any{"status": "failed"} + } else if downloadID > 0 { + data["resultDownloadId"] = downloadID + data["resultFile"] = map[string]any{"status": "generated"} + } else { + data["resultFile"] = map[string]any{"status": "skipped"} + } + return gjson.New(data) +} + +func batchResult(successCount, failCount int) (int, int) { + if failCount == 0 { + return consts.BatchAsyncStatusSuccess, consts.BatchRunResultSuccess + } + if successCount > 0 { + return consts.BatchAsyncStatusPartialSuccess, consts.BatchRunResultPartialSuccess + } + return consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed +} + +func successDetailCardIDs(details []*Detail) []int64 { + ids := make([]int64, 0, len(details)) + seen := make(map[int64]struct{}, len(details)) + for _, detail := range details { + if detail != nil && detail.Success() && detail.Card != nil && detail.Card.ID > 0 { + if _, exists := seen[detail.Card.ID]; exists { + continue + } + seen[detail.Card.ID] = struct{}{} + ids = append(ids, detail.Card.ID) + } + } + sort.Slice(ids, func(i, j int) bool { return ids[i] < ids[j] }) + return ids +} + +func uniqueSortedStrings(values []string) []string { + seen := make(map[string]struct{}, len(values)) + out := make([]string, 0, len(values)) + for _, value := range values { + value = strings.TrimSpace(value) + if value == "" { + continue + } + if _, ok := seen[value]; ok { + continue + } + seen[value] = struct{}{} + out = append(out, value) + } + sort.Strings(out) + return out +} diff --git a/internal/service/trafficcutoffconfig/repository_test.go b/internal/service/trafficcutoffconfig/repository_test.go new file mode 100644 index 0000000..bb68e48 --- /dev/null +++ b/internal/service/trafficcutoffconfig/repository_test.go @@ -0,0 +1,462 @@ +package trafficcutoffconfig + +import ( + "context" + "testing" + + _ "github.com/gogf/gf/contrib/drivers/sqlite/v2" + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + + batchdomain "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" +) + +const ( + testTenantID = "000000" + testDeptID = int64(100) +) + +func TestExecuteAllSuccessUpdatesAllCardsAndFinalizesSuccess(t *testing.T) { + ctx := context.Background() + repo := newConfigTestRepository(t) + batchNo := "TCC-41-SUCCESS" + seedConfigProcessingBatch(t, ctx, repo.db, batchNo, 512) + seedConfigRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", true) + seedConfigRunningDetail(t, ctx, repo.db, batchNo, 2, "898600000000002", "13800000002", "460001000000002", true) + seedConfigCard(t, ctx, repo.db, cardSeed{ICCID: "898600000000001", MSISDN: "13800000001", IMSI: "460001000000001", BossStatus: 1, PackageStatus: 0, PackageTotal: 0}) + seedConfigCard(t, ctx, repo.db, cardSeed{ICCID: "898600000000002", MSISDN: "13800000002", IMSI: "460001000000002", BossStatus: 9, PackageStatus: 3, PackageTotal: 0}) + + if err := (&executor{repository: repo}).Execute(ctx, batchNo); err != nil { + t.Fatalf("Execute() error = %v", err) + } + + assertConfigBatchMain(t, ctx, repo.db, batchNo, consts.BatchAsyncStatusSuccess, consts.BatchRunResultSuccess, 2, 0) + assertConfigDetailStatusCount(t, ctx, repo.db, batchNo, consts.BatchDetailStatusSuccess, 2) + assertConfigDetailStatusCount(t, ctx, repo.db, batchNo, consts.BatchDetailStatusFailed, 0) + assertConfigTrafficCutoff(t, ctx, repo.db, "898600000000001", 1, 512, 7) + assertConfigTrafficCutoff(t, ctx, repo.db, "898600000000002", 1, 512, 7) + assertNoPartialSuccess(t, ctx, repo.db, batchNo) +} + +func TestExecuteDuplicateRowsMarksWholeBatchFailedWithoutCardUpdate(t *testing.T) { + ctx := context.Background() + repo := newConfigTestRepository(t) + batchNo := "TCC-41-DUPLICATE" + seedConfigProcessingBatch(t, ctx, repo.db, batchNo, 512) + seedConfigRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", true) + seedConfigRunningDetail(t, ctx, repo.db, batchNo, 2, "898600000000001", "13800000001", "460001000000001", true) + seedConfigCard(t, ctx, repo.db, cardSeed{ICCID: "898600000000001", MSISDN: "13800000001", IMSI: "460001000000001", BossStatus: 2, PackageStatus: 2, PackageTotal: 1024}) + + if err := (&executor{repository: repo}).Execute(ctx, batchNo); err != nil { + t.Fatalf("Execute() error = %v", err) + } + + assertConfigBatchMain(t, ctx, repo.db, batchNo, consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed, 0, 2) + assertConfigDetailStatusCount(t, ctx, repo.db, batchNo, consts.BatchDetailStatusSuccess, 0) + assertConfigDetailStatusCount(t, ctx, repo.db, batchNo, consts.BatchDetailStatusFailed, 2) + assertAnyDetailReason(t, ctx, repo.db, batchNo, duplicateICCIDReason) + assertAllConfigTrafficCutoffUnchanged(t, ctx, repo.db) + assertNoPartialSuccess(t, ctx, repo.db, batchNo) +} + +func TestExecuteInvalidRowsAllowPartialSuccess(t *testing.T) { + tests := []struct { + name string + threshold float64 + validICCID string + invalidICCID string + mutate func(ctx context.Context, t *testing.T, db gdb.DB, batchNo string) + wantReason string + }{ + { + name: "unauthorized", + threshold: 512, + validICCID: "898600000000002", + invalidICCID: "898600000000001", + wantReason: unauthorizedReason, + mutate: func(ctx context.Context, t *testing.T, db gdb.DB, batchNo string) { + seedConfigRunningDetail(t, ctx, db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", false) + seedConfigRunningDetail(t, ctx, db, batchNo, 2, "898600000000002", "13800000002", "460001000000002", true) + seedConfigCard(t, ctx, db, cardSeed{ICCID: "898600000000001", MSISDN: "13800000001", IMSI: "460001000000001", BossStatus: 2, PackageStatus: 2, PackageTotal: 1024}) + seedConfigCard(t, ctx, db, cardSeed{ICCID: "898600000000002", MSISDN: "13800000002", IMSI: "460001000000002", BossStatus: 2, PackageStatus: 2, PackageTotal: 1024}) + }, + }, + { + name: "missing", + threshold: 512, + validICCID: "898600000000002", + wantReason: cardMissingReason, + mutate: func(ctx context.Context, t *testing.T, db gdb.DB, batchNo string) { + seedConfigRunningDetail(t, ctx, db, batchNo, 1, "898600000000404", "13800000404", "460001000000404", true) + seedConfigRunningDetail(t, ctx, db, batchNo, 2, "898600000000002", "13800000002", "460001000000002", true) + seedConfigCard(t, ctx, db, cardSeed{ICCID: "898600000000002", MSISDN: "13800000002", IMSI: "460001000000002", BossStatus: 2, PackageStatus: 2, PackageTotal: 1024}) + }, + }, + { + name: "MSISDN mismatch", + threshold: 512, + validICCID: "898600000000002", + invalidICCID: "898600000000001", + wantReason: msisdnMismatchReason, + mutate: func(ctx context.Context, t *testing.T, db gdb.DB, batchNo string) { + seedConfigRunningDetail(t, ctx, db, batchNo, 1, "898600000000001", "13900009999", "460001000000001", true) + seedConfigRunningDetail(t, ctx, db, batchNo, 2, "898600000000002", "13800000002", "460001000000002", true) + seedConfigCard(t, ctx, db, cardSeed{ICCID: "898600000000001", MSISDN: "13800000001", IMSI: "460001000000001", BossStatus: 2, PackageStatus: 2, PackageTotal: 1024}) + seedConfigCard(t, ctx, db, cardSeed{ICCID: "898600000000002", MSISDN: "13800000002", IMSI: "460001000000002", BossStatus: 2, PackageStatus: 2, PackageTotal: 1024}) + }, + }, + { + name: "IMSI mismatch", + threshold: 512, + validICCID: "898600000000002", + invalidICCID: "898600000000001", + wantReason: imsiMismatchReason, + mutate: func(ctx context.Context, t *testing.T, db gdb.DB, batchNo string) { + seedConfigRunningDetail(t, ctx, db, batchNo, 1, "898600000000001", "13800000001", "460009999999999", true) + seedConfigRunningDetail(t, ctx, db, batchNo, 2, "898600000000002", "13800000002", "460001000000002", true) + seedConfigCard(t, ctx, db, cardSeed{ICCID: "898600000000001", MSISDN: "13800000001", IMSI: "460001000000001", BossStatus: 2, PackageStatus: 2, PackageTotal: 1024}) + seedConfigCard(t, ctx, db, cardSeed{ICCID: "898600000000002", MSISDN: "13800000002", IMSI: "460001000000002", BossStatus: 2, PackageStatus: 2, PackageTotal: 1024}) + }, + }, + { + name: "flow pool", + threshold: 512, + validICCID: "898600000000002", + invalidICCID: "898600000000001", + wantReason: flowPoolCardReason, + mutate: func(ctx context.Context, t *testing.T, db gdb.DB, batchNo string) { + seedConfigRunningDetail(t, ctx, db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", true) + seedConfigRunningDetail(t, ctx, db, batchNo, 2, "898600000000002", "13800000002", "460001000000002", true) + seedConfigCard(t, ctx, db, cardSeed{ICCID: "898600000000001", MSISDN: "13800000001", IMSI: "460001000000001", BossStatus: 2, PackageStatus: 2, PackageTotal: 1024, IsFlowPool: 1}) + seedConfigCard(t, ctx, db, cardSeed{ICCID: "898600000000002", MSISDN: "13800000002", IMSI: "460001000000002", BossStatus: 2, PackageStatus: 2, PackageTotal: 1024}) + }, + }, + { + name: "package total exceeded", + threshold: 2048, + validICCID: "898600000000002", + invalidICCID: "898600000000001", + wantReason: packageTotalExceededReason, + mutate: func(ctx context.Context, t *testing.T, db gdb.DB, batchNo string) { + seedConfigRunningDetail(t, ctx, db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", true) + seedConfigRunningDetail(t, ctx, db, batchNo, 2, "898600000000002", "13800000002", "460001000000002", true) + seedConfigCard(t, ctx, db, cardSeed{ICCID: "898600000000001", MSISDN: "13800000001", IMSI: "460001000000001", BossStatus: 2, PackageStatus: 2, PackageTotal: 1024}) + seedConfigCard(t, ctx, db, cardSeed{ICCID: "898600000000002", MSISDN: "13800000002", IMSI: "460001000000002", BossStatus: 2, PackageStatus: 2, PackageTotal: 0}) + }, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + ctx := context.Background() + repo := newConfigTestRepository(t) + batchNo := "TCC-41-" + tt.name + seedConfigProcessingBatch(t, ctx, repo.db, batchNo, tt.threshold) + tt.mutate(ctx, t, repo.db, batchNo) + + if err := (&executor{repository: repo}).Execute(ctx, batchNo); err != nil { + t.Fatalf("Execute() error = %v", err) + } + + assertConfigBatchMain(t, ctx, repo.db, batchNo, consts.BatchAsyncStatusPartialSuccess, consts.BatchRunResultPartialSuccess, 1, 1) + assertConfigDetailStatusCount(t, ctx, repo.db, batchNo, consts.BatchDetailStatusSuccess, 1) + assertConfigDetailStatusCount(t, ctx, repo.db, batchNo, consts.BatchDetailStatusFailed, 1) + assertAnyDetailReason(t, ctx, repo.db, batchNo, tt.wantReason) + assertConfigTrafficCutoff(t, ctx, repo.db, tt.validICCID, 1, tt.threshold, 7) + if tt.invalidICCID != "" { + assertConfigTrafficCutoff(t, ctx, repo.db, tt.invalidICCID, 0, 0, 0) + } + }) + } +} + +func TestExecuteDuplicateMessageDoesNotReapplyBusinessUpdate(t *testing.T) { + ctx := context.Background() + repo := newConfigTestRepository(t) + batchNo := "TCC-41-REPLAY" + seedConfigProcessingBatch(t, ctx, repo.db, batchNo, 512) + seedConfigRunningDetail(t, ctx, repo.db, batchNo, 1, "898600000000001", "13800000001", "460001000000001", true) + seedConfigCard(t, ctx, repo.db, cardSeed{ICCID: "898600000000001", MSISDN: "13800000001", IMSI: "460001000000001", BossStatus: 2, PackageStatus: 0, PackageTotal: 0}) + + exec := &executor{repository: repo} + if err := exec.Execute(ctx, batchNo); err != nil { + t.Fatalf("first Execute() error = %v", err) + } + assertConfigTrafficCutoff(t, ctx, repo.db, "898600000000001", 1, 512, 7) + if _, err := repo.db.Model(dao.TSimCardInfo.Table()).Ctx(ctx). + Where(dao.TSimCardInfo.Columns().Iccid, "898600000000001"). + Data(do.TSimCardInfo{TrafficCutoffStatus: 0, TrafficCutoffThreshold: 0, UpdatedBy: 99}). + Update(); err != nil { + t.Fatalf("reset card after first execute: %v", err) + } + + if err := exec.Execute(ctx, batchNo); err == nil { + t.Fatal("second Execute() error = nil, want non-nil for finalized duplicate message") + } + assertConfigBatchMain(t, ctx, repo.db, batchNo, consts.BatchAsyncStatusSuccess, consts.BatchRunResultSuccess, 1, 0) + assertConfigTrafficCutoff(t, ctx, repo.db, "898600000000001", 0, 0, 99) +} + +type cardSeed struct { + ICCID string + MSISDN string + IMSI string + BossStatus int + PackageStatus int + PackageTotal float64 + IsFlowPool int +} + +func newConfigTestRepository(t *testing.T) *Repository { + t.Helper() + ctx := context.Background() + db, err := gdb.New(gdb.ConfigNode{ + Type: "sqlite", + Link: "sqlite::@file(:memory:)", + }) + if err != nil { + t.Fatalf("create sqlite database: %v", err) + } + statements := []string{ + `CREATE TABLE batch_main ( + source_channel TEXT NOT NULL DEFAULT 'legacy', + execution_mode INTEGER NOT NULL DEFAULT 0, + id INTEGER PRIMARY KEY AUTOINCREMENT, + batch_no TEXT NOT NULL, + batch_type INTEGER NOT NULL, + batch_async_status INTEGER NOT NULL, + run_result INTEGER NOT NULL DEFAULT 0, + total_count INTEGER NOT NULL DEFAULT 0, + valid_count INTEGER NOT NULL DEFAULT 0, + success_count INTEGER NOT NULL DEFAULT 0, + fail_count INTEGER NOT NULL DEFAULT 0, + fail_reason TEXT, + start_time DATETIME, + end_time DATETIME, + created_by INTEGER NOT NULL DEFAULT 0, + created_dept INTEGER NOT NULL DEFAULT 0, + tenant_id TEXT NOT NULL DEFAULT '', + batch_info TEXT, + updated_by INTEGER NOT NULL DEFAULT 0 + )`, + `CREATE TABLE batch_detail ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + batch_no TEXT NOT NULL, + batch_type INTEGER NOT NULL, + detail_status INTEGER NOT NULL, + error_reason TEXT, + snapshot TEXT, + item_id TEXT, + row_no INTEGER NOT NULL DEFAULT 0, + updated_by INTEGER NOT NULL DEFAULT 0 + )`, + `CREATE TABLE t_sim_card_info ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + iccid TEXT NOT NULL, + msisdn TEXT, + imsi TEXT, + current_package_code TEXT, + package_status INTEGER NOT NULL DEFAULT 0, + boss_card_status INTEGER NOT NULL DEFAULT 0, + is_flow_pool INTEGER NOT NULL DEFAULT 0, + package_total_flow_mb REAL NOT NULL DEFAULT 0, + traffic_cutoff_status INTEGER NOT NULL DEFAULT 0, + traffic_cutoff_threshold REAL NOT NULL DEFAULT 0, + tenant_id TEXT NOT NULL DEFAULT '', + updated_by INTEGER NOT NULL DEFAULT 0 + )`, + } + for _, statement := range statements { + if _, err = db.Exec(ctx, statement); err != nil { + t.Fatalf("create test schema: %v", err) + } + } + t.Cleanup(func() { _ = db.Close(ctx) }) + return &Repository{db: db} +} + +func seedConfigProcessingBatch(t *testing.T, ctx context.Context, db gdb.DB, batchNo string, threshold float64) { + t.Helper() + info := gjson.New(map[string]any{ + "operation": "traffic_cutoff_config", + "trafficCutoffStatus": 1, + "trafficCutoffThreshold": threshold, + "thresholdUnit": "MB", + }) + if _, err := db.Model(dao.BatchMain.Table()).Ctx(ctx).Data(do.BatchMain{ + BatchNo: batchNo, + BatchType: consts.BatchTypeTrafficCutoffConfig, + BatchAsyncStatus: consts.BatchAsyncStatusProcessing, + RunResult: batchdomain.RunResultRunning, + CreatedBy: 7, + CreatedDept: testDeptID, + TenantId: testTenantID, + BatchInfo: info, + }).Insert(); err != nil { + t.Fatalf("insert batch main: %v", err) + } +} + +func TestConfigFromBatchInfoValidatesThreeStateContract(t *testing.T) { + tests := []struct { + name string + info map[string]any + wantStatus int + wantErr bool + }{ + {name: "unconfigured", info: map[string]any{"trafficCutoffStatus": 0, "trafficCutoffThreshold": 0}}, + {name: "enabled", info: map[string]any{"trafficCutoffStatus": 1, "trafficCutoffThreshold": 1.125}, wantStatus: 1}, + {name: "disabled with retained threshold", info: map[string]any{"trafficCutoffStatus": 2, "trafficCutoffThreshold": 1.125}, wantStatus: 2}, + {name: "old field only", info: map[string]any{"trafficCutoffEnabled": 1, "trafficCutoffThreshold": 1}, wantErr: true}, + {name: "unconfigured nonzero threshold", info: map[string]any{"trafficCutoffStatus": 0, "trafficCutoffThreshold": 1}, wantErr: true}, + {name: "enabled zero threshold", info: map[string]any{"trafficCutoffStatus": 1, "trafficCutoffThreshold": 0}, wantErr: true}, + {name: "negative threshold", info: map[string]any{"trafficCutoffStatus": 2, "trafficCutoffThreshold": -1}, wantErr: true}, + {name: "too many decimals", info: map[string]any{"trafficCutoffStatus": 2, "trafficCutoffThreshold": 1.0001}, wantErr: true}, + {name: "invalid status", info: map[string]any{"trafficCutoffStatus": 3, "trafficCutoffThreshold": 0}, wantErr: true}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := configFromBatchInfo(gjson.New(tt.info)) + if (err != nil) != tt.wantErr { + t.Fatalf("configFromBatchInfo() error = %v, wantErr %v", err, tt.wantErr) + } + if got.TrafficCutoffStatus != tt.wantStatus { + t.Fatalf("configFromBatchInfo() status = %d, want %d", got.TrafficCutoffStatus, tt.wantStatus) + } + }) + } +} + +func seedConfigRunningDetail(t *testing.T, ctx context.Context, db gdb.DB, batchNo string, rowNo int, iccid, msisdn, imsi string, authorized bool) { + t.Helper() + snapshot := gjson.New(map[string]any{ + "ICCID": iccid, + "MSISDN": msisdn, + "IMSI": imsi, + "authorized": authorized, + }) + if _, err := db.Model(dao.BatchDetail.Table()).Ctx(ctx).Data(do.BatchDetail{ + BatchNo: batchNo, + BatchType: consts.BatchTypeTrafficCutoffConfig, + DetailStatus: consts.BatchDetailStatusRunning, + RowNo: rowNo, + ItemId: iccid, + Snapshot: snapshot, + }).Insert(); err != nil { + t.Fatalf("insert batch detail: %v", err) + } +} + +func seedConfigCard(t *testing.T, ctx context.Context, db gdb.DB, seed cardSeed) { + t.Helper() + if _, err := db.Model(dao.TSimCardInfo.Table()).Ctx(ctx).Data(do.TSimCardInfo{ + Iccid: seed.ICCID, + Msisdn: seed.MSISDN, + Imsi: seed.IMSI, + CurrentPackageCode: "PKG-" + seed.ICCID, + PackageStatus: seed.PackageStatus, + BossCardStatus: seed.BossStatus, + IsFlowPool: seed.IsFlowPool, + PackageTotalFlowMb: seed.PackageTotal, + TrafficCutoffStatus: 0, + TrafficCutoffThreshold: 0, + TenantId: testTenantID, + }).Insert(); err != nil { + t.Fatalf("insert sim card: %v", err) + } +} + +func assertConfigBatchMain(t *testing.T, ctx context.Context, db gdb.DB, batchNo string, wantAsync, wantRun, wantSuccess, wantFail int) { + t.Helper() + var row struct { + Async int `orm:"batch_async_status"` + Run int `orm:"run_result"` + Success int `orm:"success_count"` + Fail int `orm:"fail_count"` + } + if err := db.Model(dao.BatchMain.Table()).Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Fields(dao.BatchMain.Columns().BatchAsyncStatus, dao.BatchMain.Columns().RunResult, dao.BatchMain.Columns().SuccessCount, dao.BatchMain.Columns().FailCount). + Scan(&row); err != nil { + t.Fatalf("query batch main: %v", err) + } + if row.Async != wantAsync || row.Run != wantRun || row.Success != wantSuccess || row.Fail != wantFail { + t.Fatalf("batch main = %+v, want async=%d run=%d success=%d fail=%d", row, wantAsync, wantRun, wantSuccess, wantFail) + } +} + +func assertConfigDetailStatusCount(t *testing.T, ctx context.Context, db gdb.DB, batchNo string, status, want int) { + t.Helper() + count, err := db.Model(dao.BatchDetail.Table()).Ctx(ctx). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().DetailStatus, status). + Count() + if err != nil { + t.Fatalf("count details: %v", err) + } + if count != want { + t.Fatalf("detail status %d count = %d, want %d", status, count, want) + } +} + +func assertConfigTrafficCutoff(t *testing.T, ctx context.Context, db gdb.DB, iccid string, enabled int, threshold float64, updatedBy int64) { + t.Helper() + var row struct { + Enabled int `orm:"traffic_cutoff_status"` + Threshold float64 `orm:"traffic_cutoff_threshold"` + UpdatedBy int64 `orm:"updated_by"` + } + if err := db.Model(dao.TSimCardInfo.Table()).Ctx(ctx). + Where(dao.TSimCardInfo.Columns().Iccid, iccid). + Fields(dao.TSimCardInfo.Columns().TrafficCutoffStatus, dao.TSimCardInfo.Columns().TrafficCutoffThreshold, dao.TSimCardInfo.Columns().UpdatedBy). + Scan(&row); err != nil { + t.Fatalf("query traffic cutoff fields: %v", err) + } + if row.Enabled != enabled || row.Threshold != threshold || row.UpdatedBy != updatedBy { + t.Fatalf("%s cutoff = (%d, %v, updatedBy=%d), want (%d, %v, updatedBy=%d)", iccid, row.Enabled, row.Threshold, row.UpdatedBy, enabled, threshold, updatedBy) + } +} + +func assertAllConfigTrafficCutoffUnchanged(t *testing.T, ctx context.Context, db gdb.DB) { + t.Helper() + count, err := db.Model(dao.TSimCardInfo.Table()).Ctx(ctx). + Where(dao.TSimCardInfo.Columns().TrafficCutoffStatus, 1). + Count() + if err != nil { + t.Fatalf("count updated cards: %v", err) + } + if count != 0 { + t.Fatalf("updated card count = %d, want 0", count) + } +} + +func assertAnyDetailReason(t *testing.T, ctx context.Context, db gdb.DB, batchNo, want string) { + t.Helper() + count, err := db.Model(dao.BatchDetail.Table()).Ctx(ctx). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().ErrorReason, want). + Count() + if err != nil { + t.Fatalf("count detail reason: %v", err) + } + if count == 0 { + t.Fatalf("detail reason %q count = 0, want at least 1", want) + } +} + +func assertNoPartialSuccess(t *testing.T, ctx context.Context, db gdb.DB, batchNo string) { + t.Helper() + count, err := db.Model(dao.BatchMain.Table()).Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusPartialSuccess). + Count() + if err != nil { + t.Fatalf("count partial success batch main: %v", err) + } + if count != 0 { + t.Fatalf("partial success batch count = %d, want 0", count) + } +} diff --git a/internal/service/trafficcutoffconfig/types.go b/internal/service/trafficcutoffconfig/types.go new file mode 100644 index 0000000..1ad4eeb --- /dev/null +++ b/internal/service/trafficcutoffconfig/types.go @@ -0,0 +1,125 @@ +package trafficcutoffconfig + +import ( + "strings" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +type Batch struct { + ID int64 + BatchNo string + BatchType int + TenantID string + CreatedDept int64 + CreatedBy int64 + StartTime *gtime.Time + BatchInfo *gjson.Json +} + +type Config struct { + TrafficCutoffCrossMonthHold int + TrafficCutoffStatus int + TrafficCutoffThreshold float64 +} + +type Card struct { + ID int64 + ICCID string + MSISDN string + IMSI string + CurrentPackageCode string + PackageStatus int + BossCardStatus int + IsFlowPool int + PackageTotalFlowMb float64 + TrafficCutoffStatus int + TrafficCutoffThreshold float64 +} + +type Detail struct { + ID int64 + RowNo int + ICCID string + MSISDN string + IMSI string + Authorized bool + Card *Card + Reason string +} + +type Task struct { + Batch Batch + Config Config + Details []*Detail + FailureReason string +} + +func (d *Detail) Success() bool { + return d != nil && strings.TrimSpace(d.Reason) == "" +} + +func (d *Detail) DisplayCode() string { + if d == nil { + return "" + } + if d.Card != nil && strings.TrimSpace(d.Card.ICCID) != "" { + return strings.TrimSpace(d.Card.ICCID) + } + return strings.TrimSpace(d.ICCID) +} + +func (d *Detail) Snapshot(config Config) *gjson.Json { + if d == nil { + return gjson.New(nil) + } + data := map[string]any{ + "rowNo": d.RowNo, + "ICCID": d.ICCID, + "MSISDN": d.MSISDN, + "IMSI": d.IMSI, + "authorized": d.Authorized, + "trafficCutoffStatusAfter": config.TrafficCutoffStatus, + "trafficCutoffThresholdAfter": config.TrafficCutoffThreshold, + "trafficCutoffCrossMonthHoldAfter": config.TrafficCutoffCrossMonthHold, + "thresholdUnit": "MB", + "errorReason": d.Reason, + } + if d.Card != nil { + data["cardId"] = d.Card.ID + data["cardIccid"] = d.Card.ICCID + data["cardMsisdn"] = d.Card.MSISDN + data["cardImsi"] = d.Card.IMSI + data["currentPackageCode"] = d.Card.CurrentPackageCode + data["packageStatus"] = d.Card.PackageStatus + data["bossCardStatus"] = d.Card.BossCardStatus + data["isFlowPool"] = d.Card.IsFlowPool + data["packageTotalFlowMb"] = d.Card.PackageTotalFlowMb + data["trafficCutoffStatusBefore"] = d.Card.TrafficCutoffStatus + data["trafficCutoffThresholdBefore"] = d.Card.TrafficCutoffThreshold + } + return gjson.New(data) +} + +func (t *Task) ResultRows() []batchresult.Row { + rows := make([]batchresult.Row, 0, len(t.Details)) + for _, detail := range t.Details { + if detail == nil { + continue + } + rows = append(rows, batchresult.Row{ + RowNo: detail.RowNo, + ObjectCode: detail.DisplayCode(), + ICCID: detail.ICCID, + MSISDN: detail.MSISDN, + IMSI: detail.IMSI, + Operation: "达量断网配置", + Success: detail.Success(), + Reason: detail.Reason, + }) + } + return rows +} diff --git a/internal/service/trafficusage/service.go b/internal/service/trafficusage/service.go new file mode 100644 index 0000000..2581b85 --- /dev/null +++ b/internal/service/trafficusage/service.go @@ -0,0 +1,158 @@ +package trafficusage + +import ( + "context" + "math" + "strings" + "time" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "github.com/gogf/gf/v2/errors/gerror" +) + +type UpstreamClient interface { + QuerySimDatausageDaily(context.Context, *model.SimDataUsageDailyInput) ([]model.SimBaseRes[model.SimDataUsageDailyOutputItem], error) + QuerySimDatausageMonthly(context.Context, *model.SimDataUsageMonthlyInput) ([]model.SimBaseRes[model.SimDataUsageMonthlyOutputItem], error) +} + +type Repository interface { + UpdateDaily(context.Context, []model.TrafficUsageDailyItem) error + UpdateMonthly(context.Context, []model.TrafficUsageMonthlyItem) error +} + +type Service struct { + upstream UpstreamClient + repository Repository +} + +type Failure struct { + Cause error + UsageMb *float64 + Permanent bool +} + +func (failure *Failure) Error() string { return failure.Cause.Error() } +func (failure *Failure) Unwrap() error { return failure.Cause } + +func New(upstream UpstreamClient, repository Repository) *Service { + return &Service{upstream: upstream, repository: repository} +} + +var usageLocation = time.FixedZone("UTC+8", 8*60*60) + +func validateMessage(simKey *model.SimKey, card model.TrafficUsageSimCard, queryDate, layout string) (time.Time, error) { + if simKey == nil || card.Iccid == "" || simKey.ICCID != card.Iccid { + return time.Time{}, &Failure{Cause: gerror.New("invalid traffic usage card identity"), Permanent: true} + } + date, err := time.ParseInLocation(layout, queryDate, usageLocation) + if err != nil { + return time.Time{}, &Failure{Cause: gerror.Wrap(err, "invalid traffic usage period"), Permanent: true} + } + return date, nil +} + +func validateUsage(usage float64) error { + if math.IsNaN(usage) || math.IsInf(usage, 0) || usage < 0 { + return &Failure{Cause: gerror.New("invalid traffic usage amount"), Permanent: true} + } + return nil +} + +func (s *Service) RetryDaily(ctx context.Context, message model.TrafficUsageDailyRetryMessage) error { + date, err := validateMessage(message.SimKey, message.Card, message.QueryDate, "20060102") + if err != nil { + return err + } + usage := message.UsageMb + if usage == nil { + output, err := s.upstream.QuerySimDatausageDaily(ctx, &model.SimDataUsageDailyInput{ + SimKeys: []*model.SimKey{message.SimKey}, + ReqData: model.SimDataUsageDailyInputItem{QueryDate: message.QueryDate}, + }) + if err != nil { + return err + } + if len(output) != 1 || output[0].SimKey == nil || output[0].SimKey.ICCID != message.Card.Iccid { + return gerror.New("daily usage response missing, failed or card identity mismatched") + } + if output[0].Code != 0 { + return &Failure{Cause: gerror.Newf("daily usage query failed: %s", output[0].Message), Permanent: isPermanentUsageResponse(output[0].Message)} + } + if queryDate, ok := normalizeDailyQueryDate(output[0].ResData.QueryDate); !ok || queryDate != message.QueryDate { + return gerror.New("daily usage response period mismatched") + } + usage = &output[0].ResData.DataAmount + } + if err := validateUsage(*usage); err != nil { + return err + } + err = s.repository.UpdateDaily(ctx, []model.TrafficUsageDailyItem{{ + TrafficUsageBase: model.TrafficUsageBase{Iccid: message.Card.Iccid, UsageMb: *usage, PackageCode: message.Card.PackageCode}, Date: date, + }}) + if err != nil { + return &Failure{Cause: err, UsageMb: usage} + } + return nil +} + +func (s *Service) RetryMonthly(ctx context.Context, message model.TrafficUsageMonthlyRetryMessage) error { + month, err := validateMessage(message.SimKey, message.Card, message.QueryDate, "200601") + if err != nil { + return err + } + usage := message.UsageMb + if usage == nil { + output, err := s.upstream.QuerySimDatausageMonthly(ctx, &model.SimDataUsageMonthlyInput{ + SimKeys: []*model.SimKey{message.SimKey}, + ReqData: model.SimDataUsageMonthlyInputItem{QueryDate: message.QueryDate}, + }) + if err != nil { + return err + } + if len(output) != 1 || output[0].SimKey == nil || output[0].SimKey.ICCID != message.Card.Iccid { + return gerror.New("monthly usage response missing, failed or card identity mismatched") + } + if output[0].Code != 0 { + return &Failure{Cause: gerror.Newf("monthly usage query failed: %s", output[0].Message), Permanent: isPermanentUsageResponse(output[0].Message)} + } + if queryMonth, ok := normalizeMonthlyQueryDate(output[0].ResData.DeadLine); !ok || queryMonth != message.QueryDate { + return gerror.New("monthly usage response period mismatched") + } + usage = &output[0].ResData.DataAmount + } + if err := validateUsage(*usage); err != nil { + return err + } + err = s.repository.UpdateMonthly(ctx, []model.TrafficUsageMonthlyItem{{ + TrafficUsageBase: model.TrafficUsageBase{Iccid: message.Card.Iccid, UsageMb: *usage, PackageCode: message.Card.PackageCode}, Month: month, + }}) + if err != nil { + return &Failure{Cause: err, UsageMb: usage} + } + return nil +} + +func normalizeDailyQueryDate(value string) (string, bool) { + for _, layout := range []string{"20060102", "2006-01-02"} { + parsed, err := time.ParseInLocation(layout, value, usageLocation) + if err == nil { + return parsed.Format("20060102"), true + } + } + return "", false +} + +func normalizeMonthlyQueryDate(value string) (string, bool) { + for _, layout := range []string{"200601", "2006-01", "20060102", "2006-01-02"} { + parsed, err := time.ParseInLocation(layout, value, usageLocation) + if err == nil { + return parsed.Format("200601"), true + } + } + return "", false +} + +func isPermanentUsageResponse(message string) bool { + message = strings.ToLower(strings.TrimSpace(message)) + return strings.Contains(message, "unsupported") || strings.Contains(message, "does not support") || strings.Contains(message, "not supported") +} diff --git a/internal/service/trafficusage/service_test.go b/internal/service/trafficusage/service_test.go new file mode 100644 index 0000000..47cd63b --- /dev/null +++ b/internal/service/trafficusage/service_test.go @@ -0,0 +1,229 @@ +package trafficusage + +import ( + "context" + "errors" + "io" + "os" + "testing" + + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +func TestMain(m *testing.M) { + g.Log().SetWriter(io.Discard) + os.Exit(m.Run()) +} + +type upstreamStub struct { + dailyCalls int + monthlyCalls int + dailyOutput []model.SimBaseRes[model.SimDataUsageDailyOutputItem] + monthlyOutput []model.SimBaseRes[model.SimDataUsageMonthlyOutputItem] + dailyErr error + monthlyErr error +} + +func (s *upstreamStub) QuerySimDatausageDaily(_ context.Context, _ *model.SimDataUsageDailyInput) ([]model.SimBaseRes[model.SimDataUsageDailyOutputItem], error) { + s.dailyCalls++ + return s.dailyOutput, s.dailyErr +} + +func (s *upstreamStub) QuerySimDatausageMonthly(_ context.Context, _ *model.SimDataUsageMonthlyInput) ([]model.SimBaseRes[model.SimDataUsageMonthlyOutputItem], error) { + s.monthlyCalls++ + return s.monthlyOutput, s.monthlyErr +} + +type repositoryStub struct { + dailyCalls int + monthlyCalls int + dailyItems []model.TrafficUsageDailyItem + monthlyItems []model.TrafficUsageMonthlyItem + dailyErr error + monthlyErr error +} + +func (s *repositoryStub) UpdateDaily(_ context.Context, items []model.TrafficUsageDailyItem) error { + s.dailyCalls++ + s.dailyItems = items + return s.dailyErr +} + +func (s *repositoryStub) UpdateMonthly(_ context.Context, items []model.TrafficUsageMonthlyItem) error { + s.monthlyCalls++ + s.monthlyItems = items + return s.monthlyErr +} + +func TestRetryDailySuccess(t *testing.T) { + upstream := &upstreamStub{ + dailyOutput: []model.SimBaseRes[model.SimDataUsageDailyOutputItem]{ + { + SimKey: &model.SimKey{ICCID: "89860001"}, + ResData: model.SimDataUsageDailyOutputItem{QueryDate: "20260714", DataAmount: 12.5}, + }, + }, + } + repository := &repositoryStub{} + service := New(upstream, repository) + message := model.TrafficUsageDailyRetryMessage{ + SimKey: &model.SimKey{ICCID: "89860001"}, + Card: model.TrafficUsageSimCard{Iccid: "89860001", PackageCode: "PKG-1"}, + QueryDate: "20260714", + } + + if err := service.RetryDaily(context.Background(), message); err != nil { + t.Fatalf("RetryDaily returned error: %v", err) + } + if upstream.dailyCalls != 1 || repository.dailyCalls != 1 { + t.Fatalf("unexpected calls: upstream=%d repository=%d", upstream.dailyCalls, repository.dailyCalls) + } + if len(repository.dailyItems) != 1 { + t.Fatalf("expected one daily item, got %d", len(repository.dailyItems)) + } + item := repository.dailyItems[0] + if item.Iccid != "89860001" || item.UsageMb != 12.5 || item.PackageCode != "PKG-1" { + t.Fatalf("unexpected daily item: %+v", item) + } + if got := item.Date.Format("20060102"); got != "20260714" { + t.Fatalf("unexpected daily date: %s", got) + } +} + +func TestRetryDailyPreservesRetryBehavior(t *testing.T) { + t.Run("period mismatch", func(t *testing.T) { + upstream := &upstreamStub{dailyOutput: []model.SimBaseRes[model.SimDataUsageDailyOutputItem]{{SimKey: &model.SimKey{ICCID: "89860001"}, ResData: model.SimDataUsageDailyOutputItem{QueryDate: "20260713", DataAmount: 1}}}} + repository := &repositoryStub{} + service := New(upstream, repository) + if err := service.RetryDaily(context.Background(), validDailyMessage()); err == nil { + t.Fatal("expected period mismatch failure") + } + if repository.dailyCalls != 0 { + t.Fatal("period mismatch was persisted") + } + }) + t.Run("unsupported channel is permanent", func(t *testing.T) { + upstream := &upstreamStub{dailyOutput: []model.SimBaseRes[model.SimDataUsageDailyOutputItem]{{SimKey: &model.SimKey{ICCID: "89860001"}, Code: 1, Message: "daily usage query is not supported"}}} + service := New(upstream, &repositoryStub{}) + err := service.RetryDaily(context.Background(), validDailyMessage()) + var failure *Failure + if !errors.As(err, &failure) || !failure.Permanent { + t.Fatalf("expected permanent failure, got %v", err) + } + }) + + t.Run("upstream failure", func(t *testing.T) { + upstream := &upstreamStub{dailyErr: errors.New("upstream failed")} + repository := &repositoryStub{} + service := New(upstream, repository) + + if err := service.RetryDaily(context.Background(), validDailyMessage()); err == nil { + t.Fatal("expected retryable failure") + } + if upstream.dailyCalls != 1 || repository.dailyCalls != 0 { + t.Fatalf("unexpected calls: upstream=%d repository=%d", upstream.dailyCalls, repository.dailyCalls) + } + }) + + t.Run("repository failure", func(t *testing.T) { + upstream := &upstreamStub{dailyOutput: validDailyOutput()} + repository := &repositoryStub{dailyErr: errors.New("repository failed")} + service := New(upstream, repository) + + if err := service.RetryDaily(context.Background(), validDailyMessage()); err == nil { + t.Fatal("expected persistence failure") + } + if upstream.dailyCalls != 1 || repository.dailyCalls != 1 { + t.Fatalf("unexpected calls: upstream=%d repository=%d", upstream.dailyCalls, repository.dailyCalls) + } + }) + + t.Run("invalid date", func(t *testing.T) { + upstream := &upstreamStub{} + repository := &repositoryStub{} + service := New(upstream, repository) + message := validDailyMessage() + message.QueryDate = "invalid" + + if err := service.RetryDaily(context.Background(), message); err == nil { + t.Fatal("expected invalid date failure") + } + if upstream.dailyCalls != 0 || repository.dailyCalls != 0 { + t.Fatalf("invalid date reached dependencies") + } + }) +} + +func TestRetryMonthlySuccessAndRetry(t *testing.T) { + t.Run("success", func(t *testing.T) { + upstream := &upstreamStub{ + monthlyOutput: []model.SimBaseRes[model.SimDataUsageMonthlyOutputItem]{ + { + SimKey: &model.SimKey{ICCID: "89860002"}, + ResData: model.SimDataUsageMonthlyOutputItem{DeadLine: "202607", DataAmount: 25.5}, + }, + }, + } + repository := &repositoryStub{} + service := New(upstream, repository) + message := model.TrafficUsageMonthlyRetryMessage{ + SimKey: &model.SimKey{ICCID: "89860002"}, + Card: model.TrafficUsageSimCard{Iccid: "89860002", PackageCode: "PKG-2"}, + QueryDate: "202607", + } + + if err := service.RetryMonthly(context.Background(), message); err != nil { + t.Fatalf("RetryMonthly returned error: %v", err) + } + if upstream.monthlyCalls != 1 || repository.monthlyCalls != 1 { + t.Fatalf("unexpected calls: upstream=%d repository=%d", upstream.monthlyCalls, repository.monthlyCalls) + } + if len(repository.monthlyItems) != 1 { + t.Fatalf("expected one monthly item, got %d", len(repository.monthlyItems)) + } + item := repository.monthlyItems[0] + if item.Iccid != "89860002" || item.UsageMb != 25.5 || item.PackageCode != "PKG-2" { + t.Fatalf("unexpected monthly item: %+v", item) + } + if got := item.Month.Format("200601"); got != "202607" { + t.Fatalf("unexpected month: %s", got) + } + }) + + t.Run("upstream failure", func(t *testing.T) { + upstream := &upstreamStub{monthlyErr: errors.New("upstream failed")} + repository := &repositoryStub{} + service := New(upstream, repository) + message := model.TrafficUsageMonthlyRetryMessage{ + SimKey: &model.SimKey{ICCID: "89860002"}, + Card: model.TrafficUsageSimCard{Iccid: "89860002"}, + QueryDate: "202607", + } + + if err := service.RetryMonthly(context.Background(), message); err == nil { + t.Fatal("expected retryable failure") + } + if upstream.monthlyCalls != 1 || repository.monthlyCalls != 0 { + t.Fatalf("unexpected calls: upstream=%d repository=%d", upstream.monthlyCalls, repository.monthlyCalls) + } + }) +} + +func validDailyMessage() model.TrafficUsageDailyRetryMessage { + return model.TrafficUsageDailyRetryMessage{ + SimKey: &model.SimKey{ICCID: "89860001"}, + Card: model.TrafficUsageSimCard{Iccid: "89860001"}, + QueryDate: "20260714", + } +} + +func validDailyOutput() []model.SimBaseRes[model.SimDataUsageDailyOutputItem] { + return []model.SimBaseRes[model.SimDataUsageDailyOutputItem]{ + { + SimKey: &model.SimKey{ICCID: "89860001"}, + ResData: model.SimDataUsageDailyOutputItem{QueryDate: "20260714", DataAmount: 1}, + }, + } +} diff --git a/internal/service/trafficusage/snapshot_test.go b/internal/service/trafficusage/snapshot_test.go new file mode 100644 index 0000000..4eabf25 --- /dev/null +++ b/internal/service/trafficusage/snapshot_test.go @@ -0,0 +1,54 @@ +package trafficusage + +import ( + "context" + "errors" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "testing" +) + +func TestPersistenceFailureRetainsSnapshot(t *testing.T) { + upstream := &upstreamStub{dailyOutput: validDailyOutput()} + repository := &repositoryStub{dailyErr: errors.New("database offline")} + service := New(upstream, repository) + message := validDailyMessage() + var failure *Failure + if err := service.RetryDaily(context.Background(), message); !errors.As(err, &failure) || failure.UsageMb == nil || *failure.UsageMb != 1 { + t.Fatalf("snapshot missing: %v", err) + } + message.UsageMb = failure.UsageMb + repository.dailyErr = nil + if err := service.RetryDaily(context.Background(), message); err != nil { + t.Fatal(err) + } + if upstream.dailyCalls != 1 || repository.dailyCalls != 2 { + t.Fatal("persistence retry queried upstream again") + } +} + +func TestZeroSnapshotBypassesUpstream(t *testing.T) { + upstream, repository := &upstreamStub{}, &repositoryStub{} + service := New(upstream, repository) + zero := 0.0 + message := validDailyMessage() + message.UsageMb = &zero + if err := service.RetryDaily(context.Background(), message); err != nil { + t.Fatal(err) + } + monthly := model.TrafficUsageMonthlyRetryMessage{SimKey: message.SimKey, Card: message.Card, QueryDate: "202607", UsageMb: &zero} + if err := service.RetryMonthly(context.Background(), monthly); err != nil { + t.Fatal(err) + } + if upstream.dailyCalls != 0 || upstream.monthlyCalls != 0 || repository.dailyCalls != 1 || repository.monthlyCalls != 1 { + t.Fatal("zero snapshot was not persisted directly") + } +} + +func TestRejectMismatchedOrMissingResult(t *testing.T) { + for _, output := range [][]model.SimBaseRes[model.SimDataUsageDailyOutputItem]{nil, {{}}, {{SimKey: &model.SimKey{ICCID: "wrong"}}}, {validDailyOutput()[0], validDailyOutput()[0]}} { + upstream, repository := &upstreamStub{dailyOutput: output}, &repositoryStub{} + if err := New(upstream, repository).RetryDaily(context.Background(), validDailyMessage()); err == nil || repository.dailyCalls != 0 { + t.Fatal("invalid result accepted") + } + } +} diff --git a/internal/service/unsub/executor.go b/internal/service/unsub/executor.go new file mode 100644 index 0000000..a0fae9d --- /dev/null +++ b/internal/service/unsub/executor.go @@ -0,0 +1,125 @@ +package unsub + +import ( + "context" + "strings" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/packagecustomer" +) + +const ( + cascadeFailureReason = "same batch has invalid unsubscribe rows; this row was skipped" + validationSkipReason = "batch validation failed; unsub request was skipped" + emptyICCIDReason = "ICCID cannot be empty" + duplicateICCIDReason = "duplicate ICCID in batch" +) + +type executor struct { + repository Repository +} + +func NewExecutor(repository Repository) Executor { + return &executor{repository: repository} +} + +func (e *executor) Execute(ctx context.Context, batchNo string) error { + task, err := e.repository.Load(ctx, strings.TrimSpace(batchNo)) + if err != nil { + return gerror.Wrap(err, "load unsub batch failed") + } + if !validateTask(task) { + if err = e.repository.SaveValidationFailure(ctx, task); err != nil { + return gerror.Wrap(err, "save unsub validation failure failed") + } + e.saveResultFile(ctx, task) + return nil + } + if err = e.repository.CommitBusiness(ctx, task); err != nil { + return gerror.Wrap(err, "commit unsub business failed") + } + packagecustomer.PublishSyncForDeptsSafe(ctx, task.Batch.TenantID, task.Batch.CreatedBy, unsubPackageSyncDeptIDs(task), "unsub", task.Batch.BatchNo) + e.saveResultFile(ctx, task) + return nil +} + +func unsubPackageSyncDeptIDs(task *Task) []int64 { + if task == nil { + return nil + } + deptIds := []int64{task.Batch.DeptID, task.Batch.CreatedDept} + for _, detail := range task.Details { + if detail == nil || strings.TrimSpace(detail.Reason) != "" { + continue + } + if detail.CardDeptID > 0 { + deptIds = append(deptIds, detail.CardDeptID) + } + } + return deptIds +} + +func (e *executor) saveResultFile(ctx context.Context, task *Task) { + request := batchresult.Request{ + BatchNo: task.Batch.BatchNo, + BatchType: task.Batch.BatchType, + TenantID: task.Batch.TenantID, + Rows: task.ResultRows(), + } + if err := e.repository.SaveResultFile(ctx, task, request); err != nil { + g.Log().Errorf(ctx, "save unsub result file status failed, batchNo:%s, err:%+v", task.Batch.BatchNo, err) + } +} + +func validateTask(task *Task) bool { + if task == nil || len(task.Details) == 0 { + if task != nil { + task.FailureReason = "unsub batch has no valid details" + } + return false + } + hasFailure := false + iccids := make(map[string]int, len(task.Details)) + for _, detail := range task.Details { + if detail == nil { + hasFailure = true + continue + } + if detail.Reason != "" { + hasFailure = true + continue + } + iccid := strings.TrimSpace(detail.ICCID) + if iccid == "" { + detail.Reason = emptyICCIDReason + hasFailure = true + continue + } + iccids[iccid]++ + } + for _, detail := range task.Details { + if detail == nil { + continue + } + if detail.Reason != "" { + continue + } + if iccids[strings.TrimSpace(detail.ICCID)] > 1 { + detail.Reason = duplicateICCIDReason + hasFailure = true + } + } + if !hasFailure { + return true + } + for _, detail := range task.Details { + if detail != nil && detail.Reason == "" { + detail.Reason = cascadeFailureReason + } + } + task.FailureReason = validationSkipReason + return false +} diff --git a/internal/service/unsub/executor_test.go b/internal/service/unsub/executor_test.go new file mode 100644 index 0000000..bc98cb5 --- /dev/null +++ b/internal/service/unsub/executor_test.go @@ -0,0 +1,195 @@ +package unsub + +import ( + "context" + "errors" + "testing" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +func TestValidateTaskFailsWhenAnyDetailHasPreExistingErrorReason(t *testing.T) { + task := &Task{ + Batch: Batch{BatchNo: "B-1"}, + Details: []*Detail{ + {RowNo: 1, ICCID: "8986"}, + {RowNo: 2, ICCID: "8987", Reason: "snapshot has reason"}, + }, + } + if validateTask(task) { + t.Fatal("expected validation to fail when any detail has a pre-existing reason") + } + if task.Details[0].Reason != cascadeFailureReason { + t.Fatalf("expected cascade reason on clean detail, got %q", task.Details[0].Reason) + } + if task.FailureReason == "" { + t.Fatal("expected task failure reason to be populated") + } +} + +func TestValidateTaskFailsOnEmptyICCID(t *testing.T) { + task := &Task{ + Batch: Batch{BatchNo: "B-2"}, + Details: []*Detail{ + {RowNo: 1, ICCID: ""}, + }, + } + if validateTask(task) { + t.Fatal("expected empty ICCID to fail validation") + } + if task.Details[0].Reason != emptyICCIDReason { + t.Fatalf("expected empty ICCID reason, got %q", task.Details[0].Reason) + } +} + +func TestValidateTaskFailsOnDuplicateICCID(t *testing.T) { + task := &Task{ + Batch: Batch{BatchNo: "B-3"}, + Details: []*Detail{ + {RowNo: 1, ICCID: "8986"}, + {RowNo: 2, ICCID: "8986"}, + }, + } + if validateTask(task) { + t.Fatal("expected duplicate ICCID to fail validation") + } + if task.Details[0].Reason != duplicateICCIDReason || task.Details[1].Reason != duplicateICCIDReason { + t.Fatalf("expected both duplicates flagged, got %q / %q", task.Details[0].Reason, task.Details[1].Reason) + } +} + +func TestValidateTaskPassesOnCleanDetails(t *testing.T) { + task := &Task{ + Batch: Batch{BatchNo: "B-4"}, + Details: []*Detail{ + {RowNo: 1, ICCID: "8986"}, + {RowNo: 2, ICCID: "8987"}, + }, + } + if !validateTask(task) { + t.Fatal("expected validation to pass for clean details") + } +} + +func TestExecuteHappyPathCommitsAndSavesResult(t *testing.T) { + task := &Task{ + Batch: Batch{BatchNo: "B-5", BatchType: 108, TenantID: "000000"}, + Details: []*Detail{ + {RowNo: 1, ICCID: "8986"}, + }, + } + repo := &fakeRepository{task: task} + if err := NewExecutor(repo).Execute(context.Background(), "B-5"); err != nil { + t.Fatalf("Execute() error = %v", err) + } + if !repo.committed { + t.Fatal("expected business to be committed") + } + if !repo.resultSaved { + t.Fatal("expected result file to be saved") + } + if repo.validationSaved { + t.Fatal("did not expect validation failure path on happy path") + } +} + +func TestExecuteValidationFailureSkipsCommit(t *testing.T) { + task := &Task{ + Batch: Batch{BatchNo: "B-6", BatchType: 108}, + Details: []*Detail{ + {RowNo: 1, ICCID: "8986"}, + {RowNo: 2, ICCID: "8987", Reason: "snapshot error"}, + }, + } + repo := &fakeRepository{task: task} + if err := NewExecutor(repo).Execute(context.Background(), "B-6"); err != nil { + t.Fatalf("Execute() error = %v", err) + } + if !repo.validationSaved { + t.Fatal("expected validation failure to be persisted") + } + if repo.committed { + t.Fatal("did not expect business to be committed when validation fails") + } + if !repo.resultSaved { + t.Fatal("expected result file to be saved even when validation fails") + } +} + +func TestExecuteCommitFailurePropagatesAndSkipsResult(t *testing.T) { + task := &Task{ + Batch: Batch{BatchNo: "B-7", BatchType: 108}, + Details: []*Detail{ + {RowNo: 1, ICCID: "8986"}, + }, + } + commitErr := errors.New("transaction failed") + repo := &fakeRepository{task: task, commitErr: commitErr} + err := NewExecutor(repo).Execute(context.Background(), "B-7") + if err == nil || !errors.Is(err, commitErr) { + t.Fatalf("Execute() error = %v, want wrap of %v", err, commitErr) + } + if repo.validationSaved { + t.Fatal("validation path should not run when commit failure happens before it") + } + if repo.resultSaved { + t.Fatal("result file should not be saved when commit fails") + } +} + +func TestExecuteResultFileFailureOnlyMarksBatchInfo(t *testing.T) { + task := &Task{ + Batch: Batch{BatchNo: "B-8", BatchType: 108, TenantID: "000000"}, + Details: []*Detail{ + {RowNo: 1, ICCID: "8986"}, + }, + } + repo := &fakeRepository{task: task, resultErr: errors.New("disk full")} + if err := NewExecutor(repo).Execute(context.Background(), "B-8"); err != nil { + t.Fatalf("Execute() error = %v, want nil when only result file fails", err) + } + if !repo.committed { + t.Fatal("expected business to be committed before result file failure") + } + if !repo.resultSaved { + t.Fatal("expected result file save attempt to be recorded") + } +} + +type fakeRepository struct { + task *Task + committed bool + commitErr error + validationSaved bool + resultSaved bool + resultErr error + loadErr error + loadCalls int + saveCalls int + resultRequest batchresult.Request +} + +func (r *fakeRepository) Load(_ context.Context, _ string) (*Task, error) { + r.loadCalls++ + if r.loadErr != nil { + return nil, r.loadErr + } + return r.task, nil +} + +func (r *fakeRepository) SaveValidationFailure(_ context.Context, _ *Task) error { + r.validationSaved = true + return nil +} + +func (r *fakeRepository) CommitBusiness(_ context.Context, _ *Task) error { + r.committed = true + return r.commitErr +} + +func (r *fakeRepository) SaveResultFile(_ context.Context, _ *Task, request batchresult.Request) error { + r.saveCalls++ + r.resultRequest = request + r.resultSaved = true + return r.resultErr +} diff --git a/internal/service/unsub/types.go b/internal/service/unsub/types.go new file mode 100644 index 0000000..f0a4c24 --- /dev/null +++ b/internal/service/unsub/types.go @@ -0,0 +1,142 @@ +package unsub + +import ( + "context" + "strings" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +type Batch struct { + ID int64 + BatchNo string + BatchType int + TenantID string + DeptID int64 + CreatedBy int64 + CreatedDept int64 + StartTime *gtime.Time + BatchInfo *gjson.Json + SourceBatchNo string + Remark string +} + +type Detail struct { + ID int64 + RowNo int + ICCID string + MSISDN string + IMSI string + Reason string + + Authorized bool + Remark string + SourceBatchNo string + + SubscriptionNo string + OrderCode string + PackageCode string + PackageType int + EffectiveTime string + ExpectedExpireTime string + ActualExpireTime string + ReturnedToStock bool + StockDeptID int64 + CardDeptID int64 + StockBatchNo string +} + +type Task struct { + Batch Batch + Details []*Detail + FailureReason string +} + +type OrderRefund struct { + Amount float64 + SerialNo string + Status string + Reason string +} + +type Repository interface { + Load(ctx context.Context, batchNo string) (*Task, error) + SaveValidationFailure(ctx context.Context, task *Task) error + CommitBusiness(ctx context.Context, task *Task) error + SaveResultFile(ctx context.Context, task *Task, request batchresult.Request) error +} + +type Executor interface { + Execute(ctx context.Context, batchNo string) error +} + +func (d *Detail) Snapshot() *gjson.Json { + if d == nil { + return gjson.New(nil) + } + return gjson.New(map[string]any{ + "rowNo": d.RowNo, + "ICCID": d.ICCID, + "MSISDN": d.MSISDN, + "IMSI": d.IMSI, + "authorized": d.Authorized, + "remark": d.Remark, + "sourceBatchNo": d.SourceBatchNo, + "subscriptionNo": d.SubscriptionNo, + "orderCode": d.OrderCode, + "packageCode": d.PackageCode, + "packageType": d.PackageType, + "effectiveTime": d.EffectiveTime, + "expectedExpireTime": d.ExpectedExpireTime, + "actualExpireTime": d.ActualExpireTime, + "returnedToStock": d.ReturnedToStock, + "stockDeptId": d.StockDeptID, + "cardDeptId": d.CardDeptID, + "stockBatchNo": d.StockBatchNo, + "errorReason": d.Reason, + }) +} + +func (t *Task) ResultRows() []batchresult.Row { + rows := make([]batchresult.Row, 0, len(t.Details)) + for _, detail := range t.Details { + if detail == nil { + continue + } + rows = append(rows, batchresult.Row{ + RowNo: detail.RowNo, + ICCID: detail.ICCID, + MSISDN: detail.MSISDN, + IMSI: detail.IMSI, + Success: detail.Reason == "", + Reason: detail.Reason, + }) + } + return rows +} + +func trimSnapshotString(value any) string { + return strings.TrimSpace(toStringValue(value)) +} + +func toStringValue(value any) string { + if value == nil { + return "" + } + switch v := value.(type) { + case string: + return v + case *string: + if v == nil { + return "" + } + return *v + case []byte: + return string(v) + default: + return "" + } +} diff --git a/internal/service/warningconfig/executor.go b/internal/service/warningconfig/executor.go new file mode 100644 index 0000000..5983bda --- /dev/null +++ b/internal/service/warningconfig/executor.go @@ -0,0 +1,214 @@ +package warningconfig + +import ( + "context" + "strings" + "time" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +const warningConfigProcessingRetryAfter = 2 * time.Minute + +type executor struct { + repository *Repository +} + +func NewExecutor() *executor { + return &executor{repository: NewRepository()} +} + +func (e *executor) CanExecuteProcessing(_ context.Context, main *entity.BatchMain) (bool, error) { + if main == nil || main.BatchType != consts.BatchTypeSimWarningConfig || main.BatchAsyncStatus != consts.BatchAsyncStatusProcessing { + return false, nil + } + if main.StartTime == nil || main.StartTime.IsZero() { + return true, nil + } + return gtime.Now().Sub(main.StartTime) >= warningConfigProcessingRetryAfter, nil +} + +func (e *executor) Execute(ctx context.Context, batchNo string) error { + task, err := e.repository.Load(ctx, strings.TrimSpace(batchNo)) + if err != nil { + return gerror.Wrap(err, "load warning config batch failed") + } + validateTask(task) + return gerror.Wrap(e.repository.Commit(ctx, task), "commit warning config batch failed") +} + +func validateTask(task *Task) { + if task == nil || len(task.Details) == 0 { + if task != nil { + task.FailureReason = "预警配置批次没有可处理明细" + } + return + } + for _, detail := range task.Details { + validateDetail(task, detail) + } + _, failCount := countResult(task) + if failCount > 0 { + markAtomicFailure(task, "预警配置批次存在失败明细，整批未执行") + } +} + +func markAtomicFailure(task *Task, reason string) { + if task == nil { + return + } + reason = strings.TrimSpace(reason) + if reason == "" { + reason = "本批次存在失败明细，整批未执行" + } + task.FailureReason = reason + for _, detail := range task.Details { + if detail != nil && strings.TrimSpace(detail.Reason) == "" { + detail.Reason = "本批次存在失败明细，整批未执行" + } + } +} + +func validateDetail(task *Task, detail *Detail) { + if detail == nil || detail.Reason != "" { + return + } + config := detail.Config + if config.ItemType == warnItemTypeFlowPool { + validateFlowPoolDetail(config, detail) + return + } + validateCardDetail(config, detail) +} + +func validateCardDetail(config Config, detail *Detail) { + detail.ICCID = strings.TrimSpace(detail.ICCID) + detail.MSISDN = strings.TrimSpace(detail.MSISDN) + detail.IMSI = strings.TrimSpace(detail.IMSI) + if detail.ICCID == "" { + detail.Reason = "ICCID不能为空" + return + } + if !detail.Authorized { + detail.Reason = "ICCID不在当前操作员授权范围内" + return + } + if detail.Card == nil { + detail.Reason = "ICCID不存在或不属于当前租户" + return + } + if detail.MSISDN != "" && detail.MSISDN != detail.Card.MSISDN { + detail.Reason = "MSISDN与卡档案不一致" + return + } + if detail.IMSI != "" && detail.IMSI != detail.Card.IMSI { + detail.Reason = "IMSI与卡档案不一致" + return + } + if detail.Card.BossCardStatus == consts.SimBossCardStatusStopped { + detail.Reason = "卡已停机，不能配置预警" + return + } + if detail.Card.BossCardStatus != consts.SimBossCardStatusActivated { + detail.Reason = "当前卡状态为" + bossCardStatusText(detail.Card.BossCardStatus) + "，只有已激活卡可以配置预警" + return + } + if detail.Card.PackageStatus != packageStatusEffective { + detail.Reason = "套餐未生效，不能配置预警" + return + } + if detail.Card.IsFlowPool != 0 { + detail.Reason = "流量池卡暂不支持单卡预警配置" + return + } + if strings.TrimSpace(detail.Card.CurrentPackageCode) == "" { + detail.Reason = "当前套餐编号为空，不能配置预警" + return + } + switch config.WarnType { + case warnTypeCardTraffic: + validateTrafficDetail(config, detail) + case warnTypeExpireDay, warnTypeExpireMonth: + if detail.Card.FlowPkgEndAt == nil || detail.Card.FlowPkgEndAt.IsZero() { + detail.Reason = "套餐到期时间为空，不能配置到期预警" + } + default: + detail.Reason = "预警类型无效" + } +} + +func validateFlowPoolDetail(config Config, detail *Detail) { + detail.FlowPoolCode = strings.TrimSpace(detail.FlowPoolCode) + if detail.FlowPoolCode == "" { + detail.Reason = "流量池编号不能为空" + return + } + if !detail.Authorized { + detail.Reason = "流量池不在当前操作员授权范围内" + return + } + if detail.Pool == nil { + detail.Reason = "流量池不存在或不属于当前租户" + return + } + if detail.Pool.PoolStatus != flowPoolStatusNormal { + detail.Reason = "流量池状态不是正常，不能配置预警" + return + } + if detail.Pool.TotalFlowMb <= 0 { + detail.Reason = "流量池总流量为空，不能配置预警" + return + } + if config.WarnType != warnTypeFlowPoolPercent || config.WarnUnit != warnUnitPercent { + detail.Reason = "流量池预警仅支持百分比配置" + return + } + if config.AutoAction != 0 { + detail.Reason = "流量池预警暂不支持第三次关停" + } +} + +func validateTrafficDetail(config Config, detail *Detail) { + if detail.Card.PackageTotalFlowMb <= 0 { + detail.Reason = "套餐总流量为空，不能配置流量预警" + return + } + if config.WarnUnit != warnUnitMb { + return + } + total := detail.Card.PackageTotalFlowMb + if config.FirstWarn > total || config.SecondWarn > total || config.ThirdWarn > total { + detail.Reason = "流量阈值不能大于套餐总流量" + } +} + +func countResult(task *Task) (successCount, failCount int) { + if task == nil { + return 0, 0 + } + for _, detail := range task.Details { + if detail != nil && detail.Success() { + successCount++ + } else { + failCount++ + } + } + return successCount, failCount +} + +func bossCardStatusText(status int) string { + switch status { + case consts.SimBossCardStatusActivated: + return "已激活" + case consts.SimBossCardStatusStopped: + return "已停机" + case 1: + return "待激活" + default: + return "未知状态" + } +} diff --git a/internal/service/warningconfig/executor_test.go b/internal/service/warningconfig/executor_test.go new file mode 100644 index 0000000..b8f3bce --- /dev/null +++ b/internal/service/warningconfig/executor_test.go @@ -0,0 +1,124 @@ +package warningconfig + +import ( + "context" + "testing" + "time" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +func TestCanExecuteProcessingAllowsStaleWarningConfigBatch(t *testing.T) { + executor := &executor{} + main := &entity.BatchMain{ + BatchType: consts.BatchTypeSimWarningConfig, + BatchAsyncStatus: consts.BatchAsyncStatusProcessing, + StartTime: gtime.Now().Add(-warningConfigProcessingRetryAfter - time.Second), + } + + canExecute, err := executor.CanExecuteProcessing(context.Background(), main) + if err != nil || !canExecute { + t.Fatalf("CanExecuteProcessing() = %t, %v; want true, nil", canExecute, err) + } +} + +func TestConfigFromDetailSnapshotUsesRowValues(t *testing.T) { + fallback := Config{ + ItemType: warnItemTypeCard, + WarnUnit: warnUnitMb, + WarnMethods: warnDefaultMethod, + IsEnabled: 1, + } + snapshot := gjson.New(map[string]any{ + "configSet": true, + "configItemType": warnItemTypeCard, + "warnUnit": warnUnitMb, + "firstWarn": 100, + "secondWarn": 500, + "thirdWarn": 900, + "warnMethods": "3,1,0", + "warnPhones": "13800008000", + "warnEmails": "test@example.com", + "autoAction": warnAutoActionSuspend, + "isEnabled": 1, + "remark": "row config", + }) + + got := configFromDetailSnapshot(snapshot, fallback) + if got.FirstWarn != 100 || got.SecondWarn != 500 || got.ThirdWarn != 900 { + t.Fatalf("unexpected thresholds: %#v", got) + } + if got.WarnMethods != "3,1,0" || got.AutoAction != warnAutoActionSuspend { + t.Fatalf("unexpected row notification config: %#v", got) + } +} + +func TestCanExecuteProcessingSkipsFreshOrDifferentBatch(t *testing.T) { + executor := &executor{} + tests := []struct { + name string + main *entity.BatchMain + }{ + { + name: "fresh processing batch", + main: &entity.BatchMain{ + BatchType: consts.BatchTypeSimWarningConfig, + BatchAsyncStatus: consts.BatchAsyncStatusProcessing, + StartTime: gtime.Now(), + }, + }, + { + name: "different batch type", + main: &entity.BatchMain{ + BatchType: consts.BatchTypeSimCardStart, + BatchAsyncStatus: consts.BatchAsyncStatusProcessing, + StartTime: gtime.Now().Add(-warningConfigProcessingRetryAfter - time.Second), + }, + }, + { + name: "pending warning config batch", + main: &entity.BatchMain{ + BatchType: consts.BatchTypeSimWarningConfig, + BatchAsyncStatus: 1, + StartTime: gtime.Now().Add(-warningConfigProcessingRetryAfter - time.Second), + }, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + canExecute, err := executor.CanExecuteProcessing(context.Background(), tt.main) + if err != nil || canExecute { + t.Fatalf("CanExecuteProcessing() = %t, %v; want false, nil", canExecute, err) + } + }) + } +} + +func TestValidateTaskMarksWholeBatchFailed(t *testing.T) { + task := &Task{ + Details: []*Detail{ + {ICCID: "8986000000000000001"}, + {ICCID: "8986000000000000002", Reason: "卡不存在"}, + }, + } + + validateTask(task) + + for _, detail := range task.Details { + if detail.Success() { + t.Fatalf("detail %s remained successful after atomic failure", detail.ICCID) + } + } + if task.FailureReason == "" { + t.Fatal("expected batch failure reason") + } + status, result := batchResult(countResult(task)) + if status != consts.BatchAsyncStatusFailed || result != consts.BatchRunResultFailed { + t.Fatalf("batchResult() = %d, %d; want failed, failed", status, result) + } +} diff --git a/internal/service/warningconfig/notification_test.go b/internal/service/warningconfig/notification_test.go new file mode 100644 index 0000000..3ec57d8 --- /dev/null +++ b/internal/service/warningconfig/notification_test.go @@ -0,0 +1,37 @@ +package warningconfig + +import ( + "github.com/gogf/gf/v2/encoding/gjson" + "testing" +) + +func TestNotificationFieldPresenceSurvivesSnapshot(t *testing.T) { + for _, supplied := range []bool{false, true} { + data := map[string]any{"itemType": 1, "warnUnit": 1, "firstWarn": 100, "isEnabled": 1} + if supplied { + data["appNotifyEnabled"] = 0 + data["wxTemplateId"] = "" + } + config, err := configFromBatchInfo(gjson.New(data)) + if err != nil { + t.Fatal(err) + } + if config.AppNotifyProvided != supplied || config.WxTemplateProvided != supplied { + t.Fatalf("presence lost: %+v", config) + } + detail := &Detail{Config: config} + snapshot := detail.Snapshot(config) + restored := configFromDetailSnapshot(snapshot, Config{}) + if restored.AppNotifyProvided != supplied || restored.WxTemplateProvided != supplied { + t.Fatalf("snapshot presence lost: %+v", restored) + } + } +} + +func TestNotificationSnapshotPreservesFallbackWhenAbsent(t *testing.T) { + fallback := Config{AppNotifyEnabled: 1, AppNotifyProvided: true, WxTemplateID: "existing", WxTemplateProvided: true} + restored := configFromDetailSnapshot(gjson.New(map[string]any{"configSet": true, "warnUnit": 1}), fallback) + if restored.AppNotifyEnabled != 1 || restored.WxTemplateID != "existing" { + t.Fatalf("fallback overwritten: %+v", restored) + } +} diff --git a/internal/service/warningconfig/repository.go b/internal/service/warningconfig/repository.go new file mode 100644 index 0000000..779ad07 --- /dev/null +++ b/internal/service/warningconfig/repository.go @@ -0,0 +1,714 @@ +package warningconfig + +import ( + "context" + "strings" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +const packageStatusEffective = consts.PackageStatusEffective + +type Repository struct { + db gdb.DB + results batchresult.Generator +} + +func NewRepository() *Repository { + return &Repository{results: batchresult.New()} +} + +func (r *Repository) database() gdb.DB { + if r.db != nil { + return r.db + } + return g.DB() +} + +func (r *Repository) Load(ctx context.Context, batchNo string) (*Task, error) { + if batchNo == "" { + return nil, gerror.New("预警配置批次号不能为空") + } + var main *entity.BatchMain + if err := r.database().Model(dao.BatchMain.Table()).Safe().Ctx(ctx). + Where(dao.BatchMain.Columns().BatchNo, batchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeSimWarningConfig). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Scan(&main); err != nil { + return nil, gerror.Wrap(err, "查询预警配置批次主表失败") + } + if main == nil || main.Id == 0 { + return nil, gerror.New("处理中预警配置批次不存在") + } + config, err := configFromBatchInfo(main.BatchInfo) + if err != nil { + return nil, err + } + + var batchDetails []*entity.BatchDetail + if err = r.database().Model(dao.BatchDetail.Table()).Safe().Ctx(ctx). + Where(dao.BatchDetail.Columns().BatchNo, batchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeSimWarningConfig). + OrderAsc(dao.BatchDetail.Columns().RowNo). + Scan(&batchDetails); err != nil { + return nil, gerror.Wrap(err, "查询预警配置批次明细失败") + } + if len(batchDetails) == 0 { + return nil, gerror.New("预警配置批次没有运行中明细") + } + + task := &Task{ + Batch: Batch{ + ID: main.Id, + BatchNo: main.BatchNo, + BatchType: main.BatchType, + TenantID: main.TenantId, + CreatedDept: main.CreatedDept, + CreatedBy: main.CreatedBy, + StartTime: main.StartTime, + BatchInfo: main.BatchInfo, + }, + Config: config, + Details: make([]*Detail, 0, len(batchDetails)), + } + for _, batchDetail := range batchDetails { + if batchDetail == nil { + continue + } + detail := &Detail{ + ID: batchDetail.Id, + RowNo: batchDetail.RowNo, + DetailStatus: batchDetail.DetailStatus, + ItemType: config.ItemType, + Reason: strings.TrimSpace(batchDetail.ErrorReason), + Config: config, + } + if detail.DetailStatus != consts.BatchDetailStatusPending && + detail.DetailStatus != consts.BatchDetailStatusRunning && + detail.Reason == "" { + detail.Reason = "批次明细已处理，不能重复执行" + } + if config.ItemType == warnItemTypeFlowPool { + detail.FlowPoolCode = strings.TrimSpace(batchDetail.ItemId) + } else { + detail.ICCID = strings.TrimSpace(batchDetail.ItemId) + } + if batchDetail.Snapshot != nil { + if value := batchDetail.Snapshot.Get("itemType").Int(); value > 0 { + detail.ItemType = value + } + if value := strings.TrimSpace(batchDetail.Snapshot.Get("ICCID").String()); value != "" { + detail.ICCID = value + } + detail.MSISDN = strings.TrimSpace(batchDetail.Snapshot.Get("MSISDN").String()) + detail.IMSI = strings.TrimSpace(batchDetail.Snapshot.Get("IMSI").String()) + if value := strings.TrimSpace(batchDetail.Snapshot.Get("flowPoolCode").String()); value != "" { + detail.FlowPoolCode = value + } + detail.Authorized = batchDetail.Snapshot.Get("authorized").Bool() + detail.Config = configFromDetailSnapshot(batchDetail.Snapshot, config) + } + task.Details = append(task.Details, detail) + } + if len(task.Details) == 0 { + return nil, gerror.New("预警配置批次没有可读取明细") + } + if err = r.loadItems(ctx, task); err != nil { + return nil, err + } + return task, nil +} + +func (r *Repository) loadItems(ctx context.Context, task *Task) error { + if task == nil { + return nil + } + if task.Config.ItemType == warnItemTypeFlowPool { + return r.loadFlowPools(ctx, task) + } + return r.loadCards(ctx, task) +} + +func (r *Repository) loadCards(ctx context.Context, task *Task) error { + iccids := make([]string, 0, len(task.Details)) + for _, detail := range task.Details { + if detail != nil && strings.TrimSpace(detail.ICCID) != "" { + iccids = append(iccids, strings.TrimSpace(detail.ICCID)) + } + } + if len(iccids) == 0 { + return nil + } + var cards []*entity.TSimCardInfo + columns := dao.TSimCardInfo.Columns() + if err := r.database().Model(dao.TSimCardInfo.Table()).Safe().Ctx(ctx). + Where(columns.TenantId, task.Batch.TenantID). + WhereIn(columns.Iccid, iccids). + Scan(&cards); err != nil { + return gerror.Wrap(err, "查询SIM卡档案失败") + } + byICCID := make(map[string]*entity.TSimCardInfo, len(cards)) + for _, card := range cards { + if card != nil { + byICCID[strings.TrimSpace(card.Iccid)] = card + } + } + for _, detail := range task.Details { + if detail == nil { + continue + } + card := byICCID[strings.TrimSpace(detail.ICCID)] + if card == nil { + continue + } + detail.Card = &Card{ + ID: card.Id, + ICCID: strings.TrimSpace(card.Iccid), + MSISDN: strings.TrimSpace(card.Msisdn), + IMSI: strings.TrimSpace(card.Imsi), + CurrentPackageCode: strings.TrimSpace(card.CurrentPackageCode), + PackageStatus: card.PackageStatus, + BossCardStatus: card.BossCardStatus, + IsFlowPool: card.IsFlowPool, + FlowPkgEndAt: card.FlowPkgEndAt, + PackageTotalFlowMb: card.PackageTotalFlowMb, + } + } + return nil +} + +func (r *Repository) loadFlowPools(ctx context.Context, task *Task) error { + codes := make([]string, 0, len(task.Details)) + for _, detail := range task.Details { + if detail != nil && strings.TrimSpace(detail.FlowPoolCode) != "" { + codes = append(codes, strings.TrimSpace(detail.FlowPoolCode)) + } + } + if len(codes) == 0 { + return nil + } + var pools []*entity.TSimFlowPool + columns := dao.TSimFlowPool.Columns() + if err := r.database().Model(dao.TSimFlowPool.Table()).Safe().Ctx(ctx). + Fields( + columns.Id, + columns.FlowPoolCode, + columns.TotalFlowMb, + columns.UsedFlowMb, + columns.RemainFlowMb, + columns.PoolStatus, + ). + Where(columns.TenantId, task.Batch.TenantID). + WhereIn(columns.FlowPoolCode, codes). + Scan(&pools); err != nil { + return gerror.Wrap(err, "查询流量池档案失败") + } + byCode := make(map[string]*entity.TSimFlowPool, len(pools)) + for _, pool := range pools { + if pool != nil { + byCode[strings.TrimSpace(pool.FlowPoolCode)] = pool + } + } + for _, detail := range task.Details { + if detail == nil { + continue + } + pool := byCode[strings.TrimSpace(detail.FlowPoolCode)] + if pool == nil { + continue + } + detail.Pool = &Pool{ + ID: pool.Id, + FlowPoolCode: strings.TrimSpace(pool.FlowPoolCode), + TotalFlowMb: pool.TotalFlowMb, + UsedFlowMb: pool.UsedFlowMb, + RemainFlowMb: pool.RemainFlowMb, + PoolStatus: pool.PoolStatus, + } + } + return nil +} + +func (r *Repository) Commit(ctx context.Context, task *Task) error { + if task == nil { + return gerror.New("预警配置任务不能为空") + } + + if _, failCount := countResult(task); failCount > 0 { + markAtomicFailure(task, "预警配置批次存在失败明细，整批未执行") + } + successCount, failCount := countResult(task) + artifact, artifactErr := r.createResultFile(ctx, task) + if artifactErr != nil { + g.Log().Warningf(ctx, "warning config result file generation failed, batchNo:%s, err:%+v", task.Batch.BatchNo, artifactErr) + } + + now := gtime.Now() + err := r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + for _, detail := range task.Details { + if detail == nil { + continue + } + if detail.Success() { + if err := r.upsertWarningConfig(ctx, tx, task, detail, now); err != nil { + return err + } + } + if err := r.updateDetail(ctx, tx, task, detail, now); err != nil { + return err + } + } + var ossID, downloadID int64 + if artifactErr == nil { + var err error + ossID, downloadID, err = r.persistArtifact(ctx, tx, task, artifact, successCount, failCount, now) + if err != nil { + return err + } + } + return r.finalizeBatch(ctx, tx, task, successCount, failCount, ossID, downloadID, now) + }) + if err != nil && artifactErr == nil && artifact.Path != "" { + batchresult.RemoveLocalFile(artifact) + } + return gerror.Wrap(err, "保存预警配置批次结果失败") +} + +func (r *Repository) upsertWarningConfig(ctx context.Context, tx gdb.TX, task *Task, detail *Detail, now *gtime.Time) error { + itemID, itemCode, err := warningConfigItemSnapshot(task, detail) + if err != nil { + return err + } + config := detail.Config + columns := dao.TWarnConfig.Columns() + excludeFields := "id,tenant_id,item_type,item_id,warn_unit,created_dept,created_by,created_at" + if !config.AppNotifyProvided { + excludeFields += ",app_notify_enabled" + } + if !config.WxTemplateProvided { + excludeFields += ",wx_template_id" + } + _, err = tx.Ctx(ctx).Model(dao.TWarnConfig.Table()).Safe(). + Data(do.TWarnConfig{ + TenantId: task.Batch.TenantID, + BatchNo: task.Batch.BatchNo, + ItemType: config.ItemType, + ItemId: itemID, + ItemCode: itemCode, + WarnUnit: config.WarnUnit, + FirstWarn: config.FirstWarn, + SecondWarn: config.SecondWarn, + ThirdWarn: config.ThirdWarn, + WarnMethods: config.WarnMethods, + WarnPhones: config.WarnPhones, + WarnEmails: config.WarnEmails, + AutoAction: config.AutoAction, + IsEnabled: config.IsEnabled, + WxTemplateId: config.WxTemplateID, + AppNotifyEnabled: config.AppNotifyEnabled, + Remark: config.Remark, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + CreatedAt: now, + UpdatedBy: task.Batch.CreatedBy, + UpdatedAt: now, + }). + OnConflict(columns.TenantId, columns.ItemType, columns.ItemId, columns.WarnUnit). + OnDuplicateEx(excludeFields). + Save() + return gerror.Wrap(err, "写入预警配置失败") +} + +func warningConfigItemSnapshot(task *Task, detail *Detail) (int64, string, error) { + if task == nil || detail == nil { + return 0, "", gerror.New("预警配置明细为空") + } + if detail.Config.ItemType == warnItemTypeFlowPool { + if detail.Pool == nil { + return 0, "", gerror.New("预警配置明细缺少流量池档案") + } + itemCode := strings.TrimSpace(detail.Pool.FlowPoolCode) + if itemCode == "" { + itemCode = strings.TrimSpace(detail.FlowPoolCode) + } + return detail.Pool.ID, itemCode, nil + } + if detail.Card == nil { + return 0, "", gerror.New("预警配置明细缺少SIM卡档案") + } + itemCode := strings.TrimSpace(detail.Card.ICCID) + if itemCode == "" { + itemCode = strings.TrimSpace(detail.ICCID) + } + return detail.Card.ID, itemCode, nil +} + +func (r *Repository) updateDetail(ctx context.Context, tx gdb.TX, task *Task, detail *Detail, now *gtime.Time) error { + if detail == nil || + (detail.DetailStatus != consts.BatchDetailStatusPending && + detail.DetailStatus != consts.BatchDetailStatusRunning) { + return nil + } + status := consts.BatchDetailStatusFailed + if detail.Success() { + status = consts.BatchDetailStatusSuccess + } + result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()). + Where(dao.BatchDetail.Columns().Id, detail.ID). + Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchDetail.Columns().BatchType, consts.BatchTypeSimWarningConfig). + WhereIn(dao.BatchDetail.Columns().DetailStatus, []int{ + consts.BatchDetailStatusPending, + consts.BatchDetailStatusRunning, + }). + Data(do.BatchDetail{ + DetailStatus: status, + ErrorReason: detail.Reason, + Snapshot: detail.Snapshot(detail.Config), + UpdatedBy: task.Batch.CreatedBy, + UpdatedAt: now, + }).Update() + if err != nil { + return gerror.Wrap(err, "更新预警配置批次明细失败") + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.Newf("预警配置批次明细%d已不是运行中状态", detail.ID) + } + return nil +} + +func (r *Repository) finalizeBatch(ctx context.Context, tx gdb.TX, task *Task, successCount, failCount int, ossID, downloadID int64, now *gtime.Time) error { + batchAsyncStatus, runResult := batchResult(successCount, failCount) + result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()). + Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo). + Where(dao.BatchMain.Columns().BatchType, consts.BatchTypeSimWarningConfig). + Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing). + Data(do.BatchMain{ + BatchAsyncStatus: batchAsyncStatus, + RunResult: runResult, + TotalCount: len(task.Details), + ValidCount: successCount, + SuccessCount: successCount, + FailCount: failCount, + FailReason: task.FailureReason, + BatchInfo: mergeBatchInfo(task.Batch.BatchInfo, ossID, downloadID), + EndTime: now, + UpdatedBy: task.Batch.CreatedBy, + }).Update() + if err != nil { + return gerror.Wrap(err, "更新预警配置批次主表失败") + } + affected, err := result.RowsAffected() + if err != nil { + return err + } + if affected != 1 { + return gerror.New("预警配置批次已不是处理中状态") + } + return nil +} + +func (r *Repository) createResultFile(ctx context.Context, task *Task) (batchresult.Artifact, error) { + if r.results == nil { + return batchresult.Artifact{}, gerror.New("result generator is not configured") + } + return r.results.Create(ctx, batchresult.Request{ + BatchNo: task.Batch.BatchNo, + BatchType: task.Batch.BatchType, + TenantID: task.Batch.TenantID, + Rows: task.ResultRows(), + }) +} + +func (r *Repository) persistArtifact(ctx context.Context, tx gdb.TX, task *Task, artifact batchresult.Artifact, successCount, failCount int, now *gtime.Time) (int64, int64, error) { + ossID, err := tx.Ctx(ctx).Model(dao.SysOss.Table()).Data(do.SysOss{ + TenantId: task.Batch.TenantID, + FileName: artifact.FileName, + OriginalName: artifact.OriginalName, + FileSuffix: artifact.FileSuffix, + Path: artifact.Path, + Url: artifact.URL, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + Service: "local", + Md5: artifact.MD5, + FileSize: artifact.Size, + }).InsertAndGetId() + if err != nil { + return 0, 0, gerror.Wrap(err, "写入预警配置结果文件OSS失败") + } + downloadID, err := tx.Ctx(ctx).Model(dao.TResourceBatchDownload.Table()).Data(do.TResourceBatchDownload{ + BatchNo: task.Batch.BatchNo, + TaskName: batchresult.TaskName(task.Batch.BatchType), + ResourceType: consts.ResourceTypeExcel, + ExportScene: consts.BatchTypeSimWarningConfig, + TotalCount: len(task.Details), + SuccessCount: successCount, + FailCount: failCount, + DownloadStatus: consts.DownloadStatusGenerated, + OssId: ossID, + FileName: artifact.FileName, + FileExt: artifact.FileSuffix, + FileUrl: artifact.URL, + FilePath: artifact.Path, + FileSize: artifact.Size, + FileMd5: artifact.MD5, + DownloadCount: 0, + MaxDownloadCount: 0, + DeptId: task.Batch.CreatedDept, + StartTime: task.Batch.StartTime, + FinishTime: now, + TenantId: task.Batch.TenantID, + CreatedDept: task.Batch.CreatedDept, + CreatedBy: task.Batch.CreatedBy, + UpdatedBy: task.Batch.CreatedBy, + }).InsertAndGetId() + if err != nil { + return 0, 0, gerror.Wrap(err, "写入预警配置结果下载记录失败") + } + return ossID, downloadID, nil +} + +func configFromBatchInfo(info *gjson.Json) (Config, error) { + if info == nil { + return Config{}, gerror.New("预警配置批次缺少批次参数") + } + config := Config{ + ItemType: info.Get("itemType").Int(), + WarnUnit: info.Get("warnUnit").Int(), + FirstWarn: info.Get("firstWarn").Float64(), + SecondWarn: info.Get("secondWarn").Float64(), + ThirdWarn: info.Get("thirdWarn").Float64(), + WarnMethods: strings.TrimSpace(info.Get("warnMethods").String()), + WarnPhones: strings.TrimSpace(info.Get("warnPhones").String()), + WarnEmails: strings.TrimSpace(info.Get("warnEmails").String()), + AutoAction: info.Get("autoAction").Int(), + IsEnabled: info.Get("isEnabled").Int(), + WxTemplateID: strings.TrimSpace(info.Get("wxTemplateId").String()), + AppNotifyEnabled: info.Get("appNotifyEnabled").Int(), + AppNotifyProvided: !info.Get("appNotifyEnabled").IsNil(), + WxTemplateProvided: !info.Get("wxTemplateId").IsNil(), + Remark: strings.TrimSpace(info.Get("remark").String()), + } + if config.WarnMethods == "" { + config.WarnMethods = warnDefaultMethod + } + if config.ItemType == 0 { + config.ItemType = warnItemTypeCard + } + config.WarnType = deriveWarnType(config.ItemType, config.WarnUnit) + if config.ItemType != warnItemTypeCard && config.ItemType != warnItemTypeFlowPool { + return Config{}, gerror.New("预警配置对象类型无效") + } + if config.WarnUnit == warnUnitDay || config.WarnUnit == warnUnitMonth { + return Config{}, gerror.New("日期预警由系统默认规则处理，不支持手动配置") + } + if config.AppNotifyEnabled != 0 && config.AppNotifyEnabled != 1 { + return Config{}, gerror.New("微信提醒启停状态无效") + } + if info.Get("rowConfig").Bool() { + if err := validateWarnTypeUnit(config.ItemType, config.WarnType, config.WarnUnit); err != nil { + return Config{}, err + } + return config, nil + } + if config.IsEnabled != 0 && config.IsEnabled != 1 { + return Config{}, gerror.New("预警配置启停状态无效") + } + if config.AutoAction != 0 && config.AutoAction != warnAutoActionSuspend { + return Config{}, gerror.New("预警配置自动动作无效") + } + if err := validateWarnTypeUnit(config.ItemType, config.WarnType, config.WarnUnit); err != nil { + return Config{}, err + } + if config.ItemType == warnItemTypeFlowPool && config.AutoAction != 0 { + return Config{}, gerror.New("流量池预警暂不支持第三次关停") + } + if err := validateConfigThresholds(&config); err != nil { + return Config{}, err + } + return config, nil +} + +func configFromDetailSnapshot(snapshot *gjson.Json, fallback Config) Config { + if snapshot == nil { + return fallback + } + if !snapshot.Get("configSet").Bool() && + snapshot.Get("configItemType").Int() == 0 && + snapshot.Get("warnUnit").Int() == 0 { + return fallback + } + config := fallback + if value := snapshot.Get("configItemType").Int(); value > 0 { + config.ItemType = value + } + if value := snapshot.Get("warnUnit").Int(); value > 0 { + config.WarnUnit = value + } + config.FirstWarn = snapshot.Get("firstWarn").Float64() + config.SecondWarn = snapshot.Get("secondWarn").Float64() + config.ThirdWarn = snapshot.Get("thirdWarn").Float64() + if value := strings.TrimSpace(snapshot.Get("warnMethods").String()); value != "" { + config.WarnMethods = value + } + config.WarnPhones = strings.TrimSpace(snapshot.Get("warnPhones").String()) + config.WarnEmails = strings.TrimSpace(snapshot.Get("warnEmails").String()) + config.AutoAction = snapshot.Get("autoAction").Int() + if snapshot.Get("isEnabled").Val() != nil { + config.IsEnabled = snapshot.Get("isEnabled").Int() + } + if !snapshot.Get("wxTemplateId").IsNil() { + config.WxTemplateID = strings.TrimSpace(snapshot.Get("wxTemplateId").String()) + config.WxTemplateProvided = true + } + if snapshot.Get("appNotifyEnabled").Val() != nil { + config.AppNotifyEnabled = snapshot.Get("appNotifyEnabled").Int() + config.AppNotifyProvided = true + } + config.Remark = strings.TrimSpace(snapshot.Get("remark").String()) + config.WarnType = deriveWarnType(config.ItemType, config.WarnUnit) + return config +} + +func validateConfigThresholds(config *Config) error { + if config == nil { + return gerror.New("预警配置参数不能为空") + } + if config.FirstWarn <= 0 { + return gerror.New("第一档阈值必须大于0") + } + if config.SecondWarn < 0 || config.ThirdWarn < 0 { + return gerror.New("预警阈值不能小于0") + } + if config.AutoAction == warnAutoActionSuspend && config.ThirdWarn <= 0 { + return gerror.New("开启第三次自动停机时必须配置第三档阈值") + } + switch config.WarnType { + case warnTypeCardTraffic: + if config.WarnUnit == warnUnitPercent { + config.FirstWarn = normalizePercentThreshold(config.FirstWarn) + config.SecondWarn = normalizePercentThreshold(config.SecondWarn) + config.ThirdWarn = normalizePercentThreshold(config.ThirdWarn) + if config.FirstWarn > 100 || config.SecondWarn > 100 || config.ThirdWarn > 100 { + return gerror.New("百分比阈值不能大于100") + } + } + return validateGreaterThresholds(config) + case warnTypeFlowPoolPercent: + config.FirstWarn = normalizePercentThreshold(config.FirstWarn) + config.SecondWarn = normalizePercentThreshold(config.SecondWarn) + config.ThirdWarn = normalizePercentThreshold(config.ThirdWarn) + if config.FirstWarn > 100 || config.SecondWarn > 100 || config.ThirdWarn > 100 { + return gerror.New("百分比阈值不能大于100") + } + return validateGreaterThresholds(config) + default: + return gerror.New("预警类型无效") + } +} + +func validateGreaterThresholds(config *Config) error { + if config.SecondWarn > 0 && config.SecondWarn <= config.FirstWarn { + return gerror.New("流量预警第二档阈值必须大于第一档") + } + if config.ThirdWarn > 0 { + if config.SecondWarn <= 0 { + return gerror.New("配置第三档时必须先配置第二档") + } + if config.ThirdWarn <= config.SecondWarn { + return gerror.New("流量预警第三档阈值必须大于第二档") + } + } + return nil +} + +func normalizePercentThreshold(value float64) float64 { + if value > 0 && value <= 1 { + return value * 100 + } + return value +} + +func deriveWarnType(itemType, warnUnit int) int { + if itemType == warnItemTypeFlowPool { + return warnTypeFlowPoolPercent + } + switch warnUnit { + case warnUnitMb, warnUnitPercent: + return warnTypeCardTraffic + case warnUnitDay: + return warnTypeExpireDay + case warnUnitMonth: + return warnTypeExpireMonth + default: + return 0 + } +} + +func validateWarnTypeUnit(itemType, warnType, warnUnit int) error { + switch warnType { + case warnTypeCardTraffic: + if itemType != warnItemTypeCard { + return gerror.New("单卡流量预警只能配置单卡对象") + } + if warnUnit != warnUnitMb && warnUnit != warnUnitPercent { + return gerror.New("单卡流量预警仅支持MB或百分比单位") + } + case warnTypeExpireDay: + return gerror.New("日期预警由系统默认规则处理，不支持手动配置") + case warnTypeExpireMonth: + return gerror.New("日期预警由系统默认规则处理，不支持手动配置") + case warnTypeFlowPoolPercent: + if itemType != warnItemTypeFlowPool { + return gerror.New("流量池预警只能配置流量池对象") + } + if warnUnit != warnUnitPercent { + return gerror.New("流量池预警仅支持百分比单位") + } + default: + return gerror.New("预警类型无效") + } + return nil +} + +func mergeBatchInfo(info *gjson.Json, ossID, downloadID int64) *gjson.Json { + data := make(map[string]any) + if info != nil { + for key, value := range info.Map() { + data[key] = value + } + } + if ossID > 0 { + data["resultFileOssId"] = ossID + } + if downloadID > 0 { + data["resultDownloadId"] = downloadID + data["resultFile"] = map[string]any{"status": "generated"} + } else { + data["resultFile"] = map[string]any{"status": "skipped"} + } + return gjson.New(data) +} + +func batchResult(successCount, failCount int) (int, int) { + if failCount > 0 || successCount == 0 { + return consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed + } + return consts.BatchAsyncStatusSuccess, consts.BatchRunResultSuccess +} diff --git a/internal/service/warningconfig/types.go b/internal/service/warningconfig/types.go new file mode 100644 index 0000000..94ead19 --- /dev/null +++ b/internal/service/warningconfig/types.go @@ -0,0 +1,201 @@ +package warningconfig + +import ( + "strings" + + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult" +) + +const ( + warnItemTypeCard = 1 + warnItemTypeFlowPool = 2 + + warnUnitMb = 1 + warnUnitPercent = 2 + warnUnitDay = 3 + warnUnitMonth = 4 + + warnTypeCardTraffic = 1 + warnTypeExpireDay = 2 + warnTypeExpireMonth = 3 + warnTypeFlowPoolPercent = 4 + + warnAutoActionSuspend = 3 + warnDefaultMethod = "3" + + flowPoolStatusNormal = 0 +) + +type Batch struct { + ID int64 + BatchNo string + BatchType int + TenantID string + CreatedDept int64 + CreatedBy int64 + StartTime *gtime.Time + BatchInfo *gjson.Json +} + +type Config struct { + ItemType int + WarnType int + WarnUnit int + FirstWarn float64 + SecondWarn float64 + ThirdWarn float64 + WarnMethods string + WarnPhones string + WarnEmails string + AutoAction int + IsEnabled int + WxTemplateID string + AppNotifyEnabled int + AppNotifyProvided bool + WxTemplateProvided bool + Remark string +} + +type Card struct { + ID int64 + ICCID string + MSISDN string + IMSI string + CurrentPackageCode string + PackageStatus int + BossCardStatus int + IsFlowPool int + FlowPkgEndAt *gtime.Time + PackageTotalFlowMb float64 +} + +type Pool struct { + ID int64 + FlowPoolCode string + TotalFlowMb float64 + UsedFlowMb float64 + RemainFlowMb float64 + PoolStatus int +} + +type Detail struct { + ID int64 + RowNo int + DetailStatus int + ItemType int + ICCID string + MSISDN string + IMSI string + FlowPoolCode string + Authorized bool + Card *Card + Pool *Pool + Reason string + Config Config +} + +type Task struct { + Batch Batch + Config Config + Details []*Detail + FailureReason string +} + +func (d *Detail) Success() bool { + return d != nil && strings.TrimSpace(d.Reason) == "" +} + +func (d *Detail) DisplayCode() string { + if d == nil { + return "" + } + if d.ItemType == warnItemTypeFlowPool { + if d.Pool != nil && strings.TrimSpace(d.Pool.FlowPoolCode) != "" { + return strings.TrimSpace(d.Pool.FlowPoolCode) + } + return strings.TrimSpace(d.FlowPoolCode) + } + if d.Card != nil && strings.TrimSpace(d.Card.ICCID) != "" { + return strings.TrimSpace(d.Card.ICCID) + } + return strings.TrimSpace(d.ICCID) +} + +func (d *Detail) Snapshot(config Config) *gjson.Json { + if d == nil { + return gjson.New(nil) + } + data := map[string]any{ + "rowNo": d.RowNo, + "itemType": d.ItemType, + "ICCID": d.ICCID, + "MSISDN": d.MSISDN, + "IMSI": d.IMSI, + "flowPoolCode": d.FlowPoolCode, + "authorized": d.Authorized, + "configItemType": config.ItemType, + "configSet": true, + "warnUnit": config.WarnUnit, + "firstWarn": config.FirstWarn, + "secondWarn": config.SecondWarn, + "thirdWarn": config.ThirdWarn, + "warnMethods": config.WarnMethods, + "warnPhones": config.WarnPhones, + "warnEmails": config.WarnEmails, + "autoAction": config.AutoAction, + "isEnabled": config.IsEnabled, + "remark": config.Remark, + "errorReason": d.Reason, + } + if config.WxTemplateProvided { + data["wxTemplateId"] = config.WxTemplateID + } + if config.AppNotifyProvided { + data["appNotifyEnabled"] = config.AppNotifyEnabled + } + if d.Card != nil { + data["cardId"] = d.Card.ID + data["cardIccid"] = d.Card.ICCID + data["cardMsisdn"] = d.Card.MSISDN + data["cardImsi"] = d.Card.IMSI + data["currentPackageCode"] = d.Card.CurrentPackageCode + data["packageStatus"] = d.Card.PackageStatus + data["bossCardStatus"] = d.Card.BossCardStatus + data["isFlowPool"] = d.Card.IsFlowPool + data["packageTotalFlowMb"] = d.Card.PackageTotalFlowMb + if d.Card.FlowPkgEndAt != nil { + data["flowPkgEndAt"] = d.Card.FlowPkgEndAt.String() + } + } + if d.Pool != nil { + data["poolId"] = d.Pool.ID + data["poolFlowPoolCode"] = d.Pool.FlowPoolCode + data["poolTotalFlowMb"] = d.Pool.TotalFlowMb + data["poolUsedFlowMb"] = d.Pool.UsedFlowMb + data["poolRemainFlowMb"] = d.Pool.RemainFlowMb + data["poolStatus"] = d.Pool.PoolStatus + } + return gjson.New(data) +} + +func (t *Task) ResultRows() []batchresult.Row { + rows := make([]batchresult.Row, 0, len(t.Details)) + for _, detail := range t.Details { + if detail == nil { + continue + } + rows = append(rows, batchresult.Row{ + RowNo: detail.RowNo, + ObjectCode: detail.DisplayCode(), + ICCID: detail.ICCID, + MSISDN: detail.MSISDN, + IMSI: detail.IMSI, + Success: detail.Success(), + Reason: detail.Reason, + }) + } + return rows +} diff --git a/internal/service/workorder/notice.go b/internal/service/workorder/notice.go new file mode 100644 index 0000000..41b0d07 --- /dev/null +++ b/internal/service/workorder/notice.go @@ -0,0 +1,25 @@ +package workorder + +import ( + "context" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/encoding/gjson" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/os/gtime" + "net/url" +) + +func saveAutoCloseNotice(ctx context.Context, tx gdb.TX, current *entity.TWoMain) error { + if current.CreatorUserId <= 0 { + return nil + } + id, err := dao.TMessageLog.Ctx(ctx).TX(tx).Data(do.TMessageLog{MessageType: 1, SendUserType: current.CreatorType, UserIds: gjson.New([]int64{current.CreatorUserId}), MessageTitle: "工单已自动完结", MessageContent: "工单解决后72小时未收到未解决反馈，现已自动完结。您可以查看记录并评价服务。", MessageLink: "/pages/business/views/workOrderDetail/index?woNo=" + url.QueryEscape(current.WoNo), SendTime: gtime.Now(), IsRecall: 0, TenantId: current.TenantId, CreatedDept: current.CreatorDeptId, CreatedBy: 0}).InsertAndGetId() + if err != nil { + return gerror.Wrap(err, "保存自动完结通知失败") + } + _, err = dao.TUserMessage.Ctx(ctx).TX(tx).Data(do.TUserMessage{MessageLogId: id, UserId: current.CreatorUserId, IsRead: 0, TenantId: current.TenantId, CreatedDept: current.CreatorDeptId, CreatedBy: 0}).Insert() + return gerror.Wrap(err, "保存自动完结用户通知失败") +} diff --git a/internal/service/workorder/service.go b/internal/service/workorder/service.go new file mode 100644 index 0000000..5300217 --- /dev/null +++ b/internal/service/workorder/service.go @@ -0,0 +1,105 @@ +package workorder + +import ( + "context" + "time" + + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity" +) + +const ( + statusResolved = 3 + statusClosed = 4 + operationClose = 4 +) + +type Service struct{} + +func New() *Service { + return &Service{} +} + +func (s *Service) AutoCloseResolved(ctx context.Context, message model.WorkOrderAutoCloseMessage) error { + if message.WoNo == "" { + return nil + } + g.Log().Infof(ctx, "work order auto close received, woNo:%s", message.WoNo) + return g.DB().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error { + current := entity.TWoMain{} + if err := tx.Ctx(ctx).Model(dao.TWoMain.Table()). + Where(dao.TWoMain.Columns().WoNo, message.WoNo). + Scan(¤t); err != nil { + return gerror.Wrap(err, "query work order failed") + } + if current.WoNo == "" || current.CurrentStatus != statusResolved { + return nil + } + if current.ResolveTime == nil || current.ResolveTime.Add(72*time.Hour).After(gtime.Now()) { + return nil + } + if message.ResolveTimeTs > 0 && (current.ResolveTime == nil || current.ResolveTime.Timestamp() != message.ResolveTimeTs) { + g.Log().Infof(ctx, "work order auto close skipped, resolve time changed, woNo:%s, messageResolveTime:%d", message.WoNo, message.ResolveTimeTs) + return nil + } + + operatorID := int64(0) + now := gtime.Now() + result, err := tx.Ctx(ctx).Model(dao.TWoMain.Table()). + Where(dao.TWoMain.Columns().Id, current.Id). + Where(dao.TWoMain.Columns().TenantId, current.TenantId). + Where(dao.TWoMain.Columns().ResolveTime, current.ResolveTime). + Where(dao.TWoMain.Columns().CurrentStatus, statusResolved). + Data(do.TWoMain{ + CurrentStatus: statusClosed, + CloseTime: now, + UpdatedBy: operatorID, + }).Update() + if err != nil { + return gerror.Wrap(err, "auto close work order failed") + } + affected, err := result.RowsAffected() + if err != nil || affected == 0 { + return err + } + + _, err = tx.Ctx(ctx).Model(dao.TWoRecord.Table()).Data(do.TWoRecord{ + WoNo: message.WoNo, + OperationType: operationClose, + OperationDeptId: current.HandlerDeptId, + OperationUserId: operatorID, + BeforeStatus: statusResolved, + AfterStatus: statusClosed, + TenantId: current.TenantId, + CreatedDept: current.CreatedDept, + CreatedBy: operatorID, + }).Insert() + if err != nil { + return gerror.Wrap(err, "保存自动完结记录失败") + } + return saveAutoCloseNotice(ctx, tx, ¤t) + }) +} + +func (s *Service) CloseOverdue(ctx context.Context) error { + var rows []*entity.TWoMain + if err := dao.TWoMain.Ctx(ctx).Where(dao.TWoMain.Columns().CurrentStatus, statusResolved).WhereLTE(dao.TWoMain.Columns().ResolveTime, gtime.Now().Add(-72*time.Hour)).OrderAsc(dao.TWoMain.Columns().ResolveTime).Limit(100).Scan(&rows); err != nil { + return gerror.Wrap(err, "查询到期工单失败") + } + for _, row := range rows { + if row == nil || row.ResolveTime == nil { + continue + } + if err := s.AutoCloseResolved(ctx, model.WorkOrderAutoCloseMessage{WoNo: row.WoNo, ResolveTimeTs: row.ResolveTime.Timestamp()}); err != nil { + g.Log().Errorf(ctx, "工单补偿关闭失败 woNo:%s err:%v", row.WoNo, err) + } + } + return nil +} diff --git a/internal/service/workorder/service_test.go b/internal/service/workorder/service_test.go new file mode 100644 index 0000000..82a32c3 --- /dev/null +++ b/internal/service/workorder/service_test.go @@ -0,0 +1,244 @@ +package workorder + +import ( + "context" + "io" + "os" + "path/filepath" + "testing" + "time" + + _ "github.com/gogf/gf/contrib/drivers/sqlite/v2" + "github.com/gogf/gf/v2/database/gdb" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model" +) + +func TestMain(m *testing.M) { + g.Log().SetWriter(io.Discard) + code := m.Run() + _ = os.RemoveAll("resource") + os.Exit(code) +} + +func TestAutoCloseResolved(t *testing.T) { + ctx := context.Background() + if err := gdb.SetConfig(gdb.Config{ + "default": gdb.ConfigGroup{ + { + Type: "sqlite", + Name: filepath.Join(t.TempDir(), "workorder.db"), + }, + }, + }); err != nil { + t.Fatalf("configure test database: %v", err) + } + defer g.DB().Close(ctx) + + createWorkOrderTables(t, ctx) + service := New() + + t.Run("resolved order is closed once", func(t *testing.T) { + insertWorkOrder(t, ctx, "WO-RESOLVED", statusResolved) + message := model.WorkOrderAutoCloseMessage{WoNo: "WO-RESOLVED"} + if err := service.AutoCloseResolved(ctx, message); err != nil { + t.Fatalf("AutoCloseResolved returned error: %v", err) + } + if status := workOrderStatus(t, ctx, message.WoNo); status != statusClosed { + t.Fatalf("expected closed status, got %d", status) + } + if count := workOrderRecordCount(t, ctx, message.WoNo); count != 1 { + t.Fatalf("expected one close record, got %d", count) + } + + if err := service.AutoCloseResolved(ctx, message); err != nil { + t.Fatalf("duplicate AutoCloseResolved returned error: %v", err) + } + if count := workOrderRecordCount(t, ctx, message.WoNo); count != 1 { + t.Fatalf("duplicate message added records, got %d", count) + } + }) + + t.Run("non-resolved order is ignored", func(t *testing.T) { + insertWorkOrder(t, ctx, "WO-PROCESSING", 2) + if err := service.AutoCloseResolved(ctx, model.WorkOrderAutoCloseMessage{WoNo: "WO-PROCESSING"}); err != nil { + t.Fatalf("AutoCloseResolved returned error: %v", err) + } + if status := workOrderStatus(t, ctx, "WO-PROCESSING"); status != 2 { + t.Fatalf("non-resolved order changed status to %d", status) + } + if count := workOrderRecordCount(t, ctx, "WO-PROCESSING"); count != 0 { + t.Fatalf("non-resolved order added %d records", count) + } + }) + + t.Run("stale resolve message is ignored", func(t *testing.T) { + const woNo = "WO-STALE-MESSAGE" + insertWorkOrder(t, ctx, woNo, statusResolved) + resolveTime := gtime.Now().Add(-73 * time.Hour) + if _, err := g.DB().Model("t_wo_main").Ctx(ctx).Where("wo_no", woNo).Data(g.Map{"resolve_time": resolveTime}).Update(); err != nil { + t.Fatalf("set resolve time: %v", err) + } + if err := service.AutoCloseResolved(ctx, model.WorkOrderAutoCloseMessage{WoNo: woNo, ResolveTimeTs: resolveTime.Timestamp() - 1}); err != nil { + t.Fatalf("AutoCloseResolved returned error: %v", err) + } + if status := workOrderStatus(t, ctx, woNo); status != statusResolved { + t.Fatalf("stale message changed status to %d", status) + } + if count := workOrderRecordCount(t, ctx, woNo); count != 0 { + t.Fatalf("stale message added %d records", count) + } + }) + + t.Run("record failure rolls back status update", func(t *testing.T) { + for _, statement := range []string{ + `CREATE TABLE t_message_log (message_log_id INTEGER PRIMARY KEY AUTOINCREMENT, message_type INTEGER, send_user_type INTEGER, user_ids TEXT, message_title TEXT, message_content TEXT, message_link TEXT, send_time DATETIME, is_recall INTEGER, tenant_id TEXT, created_dept INTEGER, created_by INTEGER, created_at DATETIME, updated_at DATETIME)`, + `CREATE TABLE t_user_message (message_id INTEGER PRIMARY KEY AUTOINCREMENT, message_log_id INTEGER, user_id INTEGER, is_read INTEGER, tenant_id TEXT, created_dept INTEGER, created_by INTEGER, created_at DATETIME, updated_at DATETIME)`, + } { + if _, err := g.DB().Exec(ctx, statement); err != nil { + t.Fatal(err) + } + } + insertWorkOrder(t, ctx, "WO-NOTICE", statusResolved) + if _, err := g.DB().Exec(ctx, `UPDATE t_wo_main SET creator_user_id = 88, creator_dept_id = 30, creator_type = 1 WHERE wo_no = ?`, "WO-NOTICE"); err != nil { + t.Fatal(err) + } + for attempt := 0; attempt < 2; attempt++ { + if err := service.AutoCloseResolved(ctx, model.WorkOrderAutoCloseMessage{WoNo: "WO-NOTICE"}); err != nil { + t.Fatal(err) + } + } + count, err := g.DB().Model("t_user_message").Ctx(ctx).Where("user_id", 88).Where("tenant_id", "tenant").Count() + if err != nil || count != 1 { + t.Fatalf("expected one tenant-scoped notice, count=%d err=%v", count, err) + } + insertWorkOrder(t, ctx, "WO-NOTICE-ROLLBACK", statusResolved) + if _, err := g.DB().Exec(ctx, `UPDATE t_wo_main SET creator_user_id = 88 WHERE wo_no = ?`, "WO-NOTICE-ROLLBACK"); err != nil { + t.Fatal(err) + } + if _, err := g.DB().Exec(ctx, `DROP TABLE t_user_message`); err != nil { + t.Fatal(err) + } + if err := service.AutoCloseResolved(ctx, model.WorkOrderAutoCloseMessage{WoNo: "WO-NOTICE-ROLLBACK"}); err == nil { + t.Fatal("expected notice failure") + } + if status := workOrderStatus(t, ctx, "WO-NOTICE-ROLLBACK"); status != statusResolved { + t.Fatalf("notice failure did not roll back: %d", status) + } + if _, err := g.DB().Exec(ctx, `UPDATE t_wo_main SET creator_user_id = 0 WHERE wo_no = ?`, "WO-NOTICE-ROLLBACK"); err != nil { + t.Fatal(err) + } + for _, name := range []string{"WO-NO-TIME", "WO-EARLY"} { + insertWorkOrder(t, ctx, name, statusResolved) + var resolveTime *gtime.Time + if name == "WO-EARLY" { + resolveTime = gtime.Now() + } + if _, err := g.DB().Exec(ctx, `UPDATE t_wo_main SET resolve_time = ? WHERE wo_no = ?`, resolveTime, name); err != nil { + t.Fatal(err) + } + if err := service.AutoCloseResolved(ctx, model.WorkOrderAutoCloseMessage{WoNo: name}); err != nil { + t.Fatal(err) + } + if status := workOrderStatus(t, ctx, name); status != statusResolved { + t.Fatalf("premature close: %s status=%d", name, status) + } + } + insertWorkOrder(t, ctx, "WO-SCAN", statusResolved) + if err := service.CloseOverdue(ctx); err != nil { + t.Fatal(err) + } + if status := workOrderStatus(t, ctx, "WO-SCAN"); status != statusClosed { + t.Fatalf("overdue scan did not close order: %d", status) + } + insertWorkOrder(t, ctx, "WO-ROLLBACK", statusResolved) + if _, err := g.DB().Exec(ctx, `DROP TABLE t_wo_record`); err != nil { + t.Fatalf("drop record table: %v", err) + } + if err := service.AutoCloseResolved(ctx, model.WorkOrderAutoCloseMessage{WoNo: "WO-ROLLBACK"}); err == nil { + t.Fatal("expected record insert error") + } + if status := workOrderStatus(t, ctx, "WO-ROLLBACK"); status != statusResolved { + t.Fatalf("transaction did not roll back, status=%d", status) + } + }) +} + +func createWorkOrderTables(t *testing.T, ctx context.Context) { + t.Helper() + statements := []string{ + `CREATE TABLE t_wo_main ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + wo_no TEXT NOT NULL UNIQUE, + creator_user_id INTEGER, + creator_dept_id INTEGER, + creator_type INTEGER, + current_status INTEGER NOT NULL, + resolve_time DATETIME, + handler_dept_id INTEGER, + handler_user_id INTEGER, + tenant_id TEXT, + created_dept INTEGER, + created_by INTEGER, + updated_by INTEGER, + close_time DATETIME, + created_at DATETIME, + updated_at DATETIME + )`, + `CREATE TABLE t_wo_record ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + wo_no TEXT NOT NULL, + operation_type INTEGER, + operation_dept_id INTEGER, + operation_user_id INTEGER, + before_status INTEGER, + after_status INTEGER, + tenant_id TEXT, + created_dept INTEGER, + created_by INTEGER, + created_at DATETIME, + updated_at DATETIME + )`, + } + for _, statement := range statements { + if _, err := g.DB().Exec(ctx, statement); err != nil { + t.Fatalf("create test table: %v", err) + } + } +} + +func insertWorkOrder(t *testing.T, ctx context.Context, woNo string, status int) { + t.Helper() + _, err := g.DB().Exec(ctx, ` + INSERT INTO t_wo_main ( + wo_no, current_status, handler_dept_id, handler_user_id, + tenant_id, created_dept, created_by, updated_by + ) VALUES (?, ?, ?, ?, ?, ?, ?, ?) + `, woNo, status, 10, 20, "tenant", 30, 40, 50) + if err != nil { + t.Fatalf("insert work order: %v", err) + } + if _, err := g.DB().Exec(ctx, `UPDATE t_wo_main SET resolve_time = ? WHERE wo_no = ?`, gtime.Now().Add(-73*time.Hour), woNo); err != nil { + t.Fatal(err) + } +} + +func workOrderStatus(t *testing.T, ctx context.Context, woNo string) int { + t.Helper() + value, err := g.DB().Model("t_wo_main").Ctx(ctx).Where("wo_no", woNo).Value("current_status") + if err != nil { + t.Fatalf("query work order status: %v", err) + } + return value.Int() +} + +func workOrderRecordCount(t *testing.T, ctx context.Context, woNo string) int { + t.Helper() + count, err := g.DB().Model("t_wo_record").Ctx(ctx).Where("wo_no", woNo).Count() + if err != nil { + t.Fatalf("count work order records: %v", err) + } + return count +} diff --git a/internal/testutil/redis.go b/internal/testutil/redis.go new file mode 100644 index 0000000..4a0a256 --- /dev/null +++ b/internal/testutil/redis.go @@ -0,0 +1,65 @@ +// Package testutil provides isolated infrastructure for integration tests. +package testutil + +import ( + "context" + "net" + "os" + "os/exec" + "testing" + "time" + + _ "github.com/gogf/gf/contrib/nosql/redis/v2" + "github.com/gogf/gf/v2/database/gredis" +) + +// Redis starts a disposable loopback-only server with persistence disabled. +// Tests never use the application's Redis configuration or an existing server. +func Redis(t *testing.T) *gredis.Redis { + t.Helper() + binary := os.Getenv("REDIS_SERVER_BINARY") + if binary == "" { + var err error + binary, err = exec.LookPath("redis-server") + if err != nil { + t.Skip("real Redis tests require redis-server on PATH or REDIS_SERVER_BINARY") + } + } + listener, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + address := listener.Addr().String() + _, port, err := net.SplitHostPort(address) + if err != nil { + t.Fatal(err) + } + _ = listener.Close() + command := exec.Command(binary, "--bind", "127.0.0.1", "--port", port, "--save", "", "--appendonly", "no") + command.Dir = t.TempDir() + if err = command.Start(); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { + _ = command.Process.Kill() + _ = command.Wait() + }) + client, err := gredis.New(&gredis.Config{Address: address, DialTimeout: 100 * time.Millisecond, ReadTimeout: time.Second, WriteTimeout: time.Second}) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = client.Close(context.Background()) }) + for deadline := time.Now().Add(5 * time.Second); time.Now().Before(deadline); { + var connection net.Conn + connection, err = net.DialTimeout("tcp", address, 50*time.Millisecond) + if err == nil { + _ = connection.Close() + if _, err = client.Do(context.Background(), "PING"); err == nil { + return client + } + } + time.Sleep(20 * time.Millisecond) + } + t.Fatalf("isolated Redis did not start: %v", err) + return nil +} diff --git a/internal/worker/historyusage_scan_config.go b/internal/worker/historyusage_scan_config.go new file mode 100644 index 0000000..4021c8a --- /dev/null +++ b/internal/worker/historyusage_scan_config.go @@ -0,0 +1,31 @@ +package worker + +import ( + "context" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/producer/dailyusage" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/producer/monthlyusagehistory" +) + +// Missing settings retain the serial path; explicit invalid values fail closed. +// Enabling channel mode requires validating idx_card_chan_key on the target DB. +func loadHistoryUsageScanSettings(ctx context.Context, section string) (string, int, error) { + mode := g.Cfg().MustGet(ctx, section+".producer.scanMode", "serial").String() + concurrency := g.Cfg().MustGet(ctx, section+".producer.scanConcurrency", 4).Int() + var err error + switch section { + case "trafficUsageDailySync": + err = dailyusage.ValidateScanSettings(mode, concurrency) + case "trafficUsageMonthlyHistorySync": + err = monthlyusagehistory.ValidateScanSettings(mode, concurrency) + default: + return "", 0, gerror.Newf("unknown history usage config section %q", section) + } + if err != nil { + return "", 0, gerror.Wrap(err, "load history usage scan settings") + } + return mode, concurrency, nil +} diff --git a/internal/worker/historyusage_scan_config_test.go b/internal/worker/historyusage_scan_config_test.go new file mode 100644 index 0000000..4a017a1 --- /dev/null +++ b/internal/worker/historyusage_scan_config_test.go @@ -0,0 +1,42 @@ +package worker + +import ( + "context" + "fmt" + "testing" + + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gcfg" +) + +func TestHistoryUsageScanSettings(t *testing.T) { + old := g.Cfg().GetAdapter() + t.Cleanup(func() { g.Cfg().SetAdapter(old) }) + for _, section := range []string{"trafficUsageDailySync", "trafficUsageMonthlyHistorySync"} { + for _, test := range []struct { + name, fields, mode string + concurrency int + invalid bool + }{ + {"omitted", "pageSize: 100", "serial", 4, false}, + {"channel", "scanMode: channel\n scanConcurrency: 2", "channel", 2, false}, + {"rollback", "scanMode: serial\n scanConcurrency: 1", "serial", 1, false}, + {"zero", "scanMode: channel\n scanConcurrency: 0", "", 0, true}, + {"negative", "scanConcurrency: -1", "", 0, true}, + {"empty_mode", "scanMode: ''", "", 0, true}, + {"invalid_mode", "scanMode: pool", "", 0, true}, + } { + t.Run(section+"/"+test.name, func(t *testing.T) { + adapter, err := gcfg.NewAdapterContent(fmt.Sprintf("%s:\n producer:\n %s\n", section, test.fields)) + if err != nil { + t.Fatal(err) + } + g.Cfg().SetAdapter(adapter) + mode, concurrency, err := loadHistoryUsageScanSettings(context.Background(), section) + if (err != nil) != test.invalid || mode != test.mode || concurrency != test.concurrency { + t.Fatalf("mode=%q concurrency=%d err=%v", mode, concurrency, err) + } + }) + } + } +} diff --git a/internal/worker/imei_config_test.go b/internal/worker/imei_config_test.go new file mode 100644 index 0000000..08ed3d4 --- /dev/null +++ b/internal/worker/imei_config_test.go @@ -0,0 +1,157 @@ +package worker + +import ( + "testing" + + "github.com/gogf/gf/v2/os/gcfg" +) + +// loadImeiTestConfig loads the real shipped config file. +// +// A plain g.Cfg() lookup is not enough here: the test binary does not resolve +// `config_default.yaml` (GoFrame searches for a file literally named +// `config`), so every MustGet would silently fall back to the default argument +// and a typo in the YAML key would go undetected. Loading the adapter +// explicitly makes the assertions below test the file, not the defaults. +func loadImeiTestConfig(t *testing.T) *gcfg.Config { + t.Helper() + adapter, err := gcfg.NewAdapterFile("config_default.yaml") + if err != nil { + t.Fatalf("load config_default.yaml: %v", err) + } + // Resolve relative to the package directory (internal/worker) up to the + // repository root, where config/ lives. + adapter.AddPath("../../config") + return gcfg.NewWithAdapter(adapter) +} + +// TestImeiConfigFileResolves 验证 imeiFullSync 配置段真实存在于发布配置中， +// 且与状态链路结构对齐。键名拼错或段落缺失会让链路静默使用零值。 +func TestImeiConfigFileResolves(t *testing.T) { + config := loadImeiTestConfig(t) + + // enabled 必须存在且为 false：未完成预演验证前不得默认开启。 + enabled, err := config.Get(nil, "imeiFullSync.enabled") + if err != nil { + t.Fatalf("读取 imeiFullSync.enabled 失败（配置段可能缺失）: %v", err) + } + if enabled.Bool() { + t.Fatal("imeiFullSync.enabled 必须为 false（预演验证后再开）") + } + + prefix, err := config.Get(nil, "imeiFullSync.redisKeyPrefix") + if err != nil || prefix.String() != "imei:" { + t.Fatalf("redisKeyPrefix = %q (err=%v), want imei:", prefix.String(), err) + } + + // 三条链路的前缀必须互不相同，否则 Run 命名空间会串场。 + for _, other := range []struct { + key string + want string + }{ + {"trafficUsageFullSync.redisKeyPrefix", "flow:"}, + {"simStatusQueueSync.redisKeyPrefix", "sim-status:"}, + } { + value, err := config.Get(nil, other.key) + if err != nil { + t.Fatalf("读取 %s 失败: %v", other.key, err) + } + if value.String() == prefix.String() { + t.Fatalf("imei 前缀 %q 与 %s (%q) 重复", prefix.String(), other.key, value.String()) + } + } + + // 关键数值项逐条断言（对齐状态链路的键名）。 + checks := []struct { + key string + want string + }{ + {"imeiFullSync.producer.pageSize", "1000"}, + {"imeiFullSync.producer.runLockKey", "imei:producer:active"}, + {"imeiFullSync.producer.runLockTTL", "5m"}, + {"imeiFullSync.consumer.lockKey", "imei:consumer:active"}, + {"imeiFullSync.consumer.claimBatch", "100"}, + {"imeiFullSync.consumer.mobileConcurrency", "100"}, + {"imeiFullSync.consumer.telecomConcurrency", "8"}, + {"imeiFullSync.consumer.networkRetries", "2"}, + {"imeiFullSync.consumer.chanClientTimeout", "30s"}, + {"imeiFullSync.retry.maxRounds", "5"}, + {"imeiFullSync.retry.failureRetention", "720h"}, + // 失败池告警阀值：漏配会静默退化为默认 50000，但键名拼错时 + // 我们希望测试直接报错，而不是让链路跑在一个意外阀值上。 + {"imeiFullSync.consumer.failurePoolAlertThreshold", "50000"}, + } + for _, check := range checks { + value, err := config.Get(nil, check.key) + if err != nil { + t.Fatalf("读取 %s 失败: %v", check.key, err) + } + if value.String() != check.want { + t.Fatalf("%s = %q, want %q", check.key, value.String(), check.want) + } + } + + // consumer.enabled 与 producer 同为 false：链路默认不跑。 + consumerEnabled, err := config.Get(nil, "imeiFullSync.consumer.enabled") + if err != nil { + t.Fatalf("读取 consumer.enabled 失败: %v", err) + } + if consumerEnabled.Bool() { + t.Fatal("imeiFullSync.consumer.enabled 必须为 false") + } +} + +// TestImeiTemplateQPSInConfigFile 模板限速必须覆盖生产实际存在的移动族模板， +// 否则会退回通用 1.5 QPS（255,976 张卡将耗时数天）。 +func TestImeiTemplateQPSInConfigFile(t *testing.T) { + config := loadImeiTestConfig(t) + + for _, tempKey := range []string{ + "sim_yd_v5", + "sim_yd_mock_v5", + "sim_onelink_tengyu_v1", + "sim_onelink_xuzhou_mobile_v1", + "sim_onelink_lelian_v1", + "sim_onelink_beixin_v1", + } { + value, err := config.Get(nil, "imeiFullSync.consumer.templateQPS."+tempKey) + if err != nil { + t.Fatalf("templateQPS 缺少 %s（会退回通用 1.5 QPS）: %v", tempKey, err) + } + if value.Float64() != 100 { + t.Fatalf("templateQPS.%s = %v, want 100", tempKey, value.Float64()) + } + } + + // 电信额度与状态/流量链路共享，必须压低。 + telecom, err := config.Get(nil, "imeiFullSync.consumer.templateQPS.sim_dx_v1") + if err != nil { + t.Fatalf("templateQPS 缺少 sim_dx_v1: %v", err) + } + if telecom.Float64() != 5 { + t.Fatalf("templateQPS.sim_dx_v1 = %v, want 5（电信额度共享，必须压低）", telecom.Float64()) + } +} + +// TestFailurePoolAlertConfiguredForEveryChain 三条全量同步链路都要有失败池 +// 告警阀值。漏配某一条时该链路会退化为「无告警」，而这正是内存问题最需要 +// 被看见的地方，所以这里显式要求三条都在配置文件里出现。 +func TestFailurePoolAlertConfiguredForEveryChain(t *testing.T) { + config := loadImeiTestConfig(t) + + chains := []string{ + "trafficUsageFullSync", + "simStatusQueueSync", + "imeiFullSync", + } + for _, chain := range chains { + key := chain + ".consumer.failurePoolAlertThreshold" + value, err := config.Get(nil, key) + if err != nil { + t.Fatalf("%s 未配置（该链路将无失败池告警）: %v", key, err) + } + if value.Int64() <= 0 { + t.Fatalf("%s = %d，应为正数（0 表示关闭告警）", key, value.Int64()) + } + } +} diff --git a/internal/worker/monthlyusage_config.go b/internal/worker/monthlyusage_config.go new file mode 100644 index 0000000..88f4ce7 --- /dev/null +++ b/internal/worker/monthlyusage_config.go @@ -0,0 +1,31 @@ +package worker + +import ( + "context" + + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/producer/monthlyusage" +) + +// Startup-only. Explicit zero/negative concurrency and invalid modes fail +// initialization; omitted concurrency defaults to four channel queries. +func loadMonthlyUsageProducerConfig(ctx context.Context) (monthlyusage.Config, error) { + config := monthlyusage.Config{ + ScanConcurrency: g.Cfg().MustGet(ctx, "trafficUsageFullSync.producer.scanConcurrency", 4).Int(), + PageSize: g.Cfg().MustGet(ctx, "trafficUsageFullSync.producer.pageSize", 100).Int(), + GlobalHighWaterMark: g.Cfg().MustGet(ctx, "trafficUsageFullSync.producer.globalHighWaterMark", 0).Int64(), + ChannelHighWaterMark: g.Cfg().MustGet(ctx, "trafficUsageFullSync.producer.channelHighWaterMark", 0).Int64(), + PollInterval: g.Cfg().MustGet(ctx, "trafficUsageFullSync.producer.pollInterval", "2s").Duration(), + RunLockKey: g.Cfg().MustGet(ctx, "trafficUsageFullSync.producer.runLockKey", "flow:producer:active").String(), + RunLockTTL: g.Cfg().MustGet(ctx, "trafficUsageFullSync.producer.runLockTTL", "5m").Duration(), + RoundRetryEnabled: true, + MaxRounds: g.Cfg().MustGet(ctx, "trafficUsageFullSync.retry.maxRounds", 5).Int(), + FailureRetention: g.Cfg().MustGet(ctx, "trafficUsageFullSync.retry.failureRetention", "720h").Duration(), + } + if err := monthlyusage.ValidateScanSettings(config.ScanConcurrency); err != nil { + return monthlyusage.Config{}, gerror.Wrap(err, "load monthly usage producer config") + } + return config, nil +} diff --git a/internal/worker/monthlyusage_config_test.go b/internal/worker/monthlyusage_config_test.go new file mode 100644 index 0000000..378ef7c --- /dev/null +++ b/internal/worker/monthlyusage_config_test.go @@ -0,0 +1,39 @@ +package worker + +import ( + "context" + "testing" + + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gcfg" +) + +func TestMonthlyUsageScanStartupConfig(t *testing.T) { + old := g.Cfg().GetAdapter() + defer g.Cfg().SetAdapter(old) + for _, test := range []struct { + name, yaml string + concurrency int + invalid bool + }{ + {"omitted", "{}", 4, false}, + {"channel", "trafficUsageFullSync:\n producer:\n scanMode: channel\n scanConcurrency: 4", 4, false}, + {"zero", "trafficUsageFullSync:\n producer:\n scanConcurrency: 0", 0, true}, + {"negative", "trafficUsageFullSync:\n producer:\n scanConcurrency: -1", 0, true}, + } { + t.Run(test.name, func(t *testing.T) { + adapter, err := gcfg.NewAdapterContent(test.yaml) + if err != nil { + t.Fatal(err) + } + g.Cfg().SetAdapter(adapter) + config, err := loadMonthlyUsageProducerConfig(context.Background()) + if (err != nil) != test.invalid { + t.Fatalf("config=%+v err=%v", config, err) + } + if !test.invalid && config.ScanConcurrency != test.concurrency { + t.Fatalf("config=%+v", config) + } + }) + } +} diff --git a/internal/worker/worker.go b/internal/worker/worker.go new file mode 100644 index 0000000..b02d55e --- /dev/null +++ b/internal/worker/worker.go @@ -0,0 +1,752 @@ +package worker + +import ( + "context" + "os" + "os/signal" + "strings" + "syscall" + "time" + + "github.com/gogf/gf/v2/database/gredis" + "github.com/gogf/gf/v2/errors/gerror" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gtime" + "github.com/gogf/gf/v2/util/gmode" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/client/upstream" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts" + rabbitmqconsumer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consumer/rabbitmq" + zsetconsumer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consumer/zset" + fullsyncarchive "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/archive" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/httpapi" + fullsyncruntime "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/runtime" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/tdengine" + dailyusageproducer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/producer/dailyusage" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/producer/imei" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/producer/monthlyusage" + monthlyhistoryproducer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/producer/monthlyusagehistory" + simstatusproducer "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/producer/simstatus" + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset" + batchrepository "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/batch" + cancelrepository "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/cancel" + datarepository "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/data" + imeirepository "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/imei" + inboundrepository "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/inbound" + lifecyclerepository "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/lifecycle" + monthlyusagerepository "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/monthlyusage" + simstatusrepository "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/simstatus" + trafficusagerepository "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/trafficusage" + unsubrepository "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/unsub" + addpackageservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/addpackage" + allocateservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/allocate" + balanceservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/balance" + cancelservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/cancel" + dailyusageservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/dailyusage" + dataservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/data" + imeiservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/imei" + inboundservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/inbound" + lifecycleservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/lifecycle" + monthlyusageservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/monthlyusage" + monthlyhistoryservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/monthlyusagehistory" + packagecustomerservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/packagecustomer" + renewchangeservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/renewchange" + simstatusservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/simstatus" + trafficcutoffconfigservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/trafficcutoffconfig" + trafficusageservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/trafficusage" + unsubservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/unsub" + warningconfigservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/warningconfig" + workorderservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/workorder" +) + +const ( + shutdownTimeout = 10 * time.Second + consumerRestartInitialDelay = 2 * time.Second + consumerRestartMaxDelay = 30 * time.Second + consumerRestartBackoffResetAfter = time.Minute +) + +const monthlyUsageTemplateQPSConfigKey = "trafficUsageFullSync.consumer.templateQPS" + +func loadMonthlyUsageTemplateQPS(ctx context.Context) map[string]float64 { + value, err := g.Cfg().Get(ctx, monthlyUsageTemplateQPSConfigKey) + if err != nil { + g.Log().Warningf(ctx, "load monthly usage template QPS config failed: %+v", err) + return nil + } + if value == nil || value.IsNil() { + return nil + } + var configured map[string]float64 + if err = value.Scan(&configured); err != nil { + g.Log().Warningf(ctx, "parse monthly usage template QPS config failed: %+v", err) + return nil + } + return configured +} + +func loadSimStatusTemplateQPS(ctx context.Context) map[string]float64 { + value, err := g.Cfg().Get(ctx, "simStatusQueueSync.consumer.templateQPS") + if err != nil || value == nil || value.IsNil() { + return nil + } + var configured map[string]float64 + if err = value.Scan(&configured); err != nil { + g.Log().Warningf(ctx, "parse sim status template QPS config failed: %+v", err) + return nil + } + return configured +} + +func loadImeiTemplateQPS(ctx context.Context) map[string]float64 { + value, err := g.Cfg().Get(ctx, "imeiFullSync.consumer.templateQPS") + if err != nil || value == nil || value.IsNil() { + return nil + } + var configured map[string]float64 + if err = value.Scan(&configured); err != nil { + g.Log().Warningf(ctx, "parse imei template QPS config failed: %+v", err) + return nil + } + return configured +} + +func loadSimStatusRunPrefix(ctx context.Context) (string, error) { + prefix := strings.TrimSpace(g.Cfg().MustGet(ctx, "simStatusQueueSync.redisKeyPrefix", "sim-status:").String()) + if prefix == "" { + return "", gerror.New("sim status redisKeyPrefix must be nonempty") + } + return prefix, nil +} + +func loadDailyUsageTemplateQPS(ctx context.Context) map[string]float64 { + value, err := g.Cfg().Get(ctx, "trafficUsageDailySync.consumer.templateQPS") + if err != nil || value == nil || value.IsNil() { + return nil + } + var configured map[string]float64 + if err = value.Scan(&configured); err != nil { + g.Log().Warningf(ctx, "parse daily usage template QPS config failed: %+v", err) + return nil + } + return configured +} + +func loadDailyUsageRunPrefix(ctx context.Context) (string, error) { + prefix := strings.TrimSpace(g.Cfg().MustGet(ctx, "trafficUsageDailySync.redisKeyPrefix", "daily:").String()) + if prefix == "" { + return "", gerror.New("daily usage redisKeyPrefix must be nonempty") + } + return prefix, nil +} + +func loadImeiRunPrefix(ctx context.Context) (string, error) { + prefix := strings.TrimSpace(g.Cfg().MustGet(ctx, "imeiFullSync.redisKeyPrefix", "imei:").String()) + if prefix == "" { + return "", gerror.New("imei redisKeyPrefix must be nonempty") + } + return prefix, nil +} + +func loadMonthlyHistoryRunPrefix(ctx context.Context) (string, error) { + prefix := strings.TrimSpace(g.Cfg().MustGet(ctx, "trafficUsageMonthlyHistorySync.redisKeyPrefix", "monthly-history:").String()) + if prefix == "" { + return "", gerror.New("monthly usage history redisKeyPrefix must be nonempty") + } + return prefix, nil +} + +func loadMonthlyHistoryTemplateQPS(ctx context.Context) map[string]float64 { + value, err := g.Cfg().Get(ctx, "trafficUsageMonthlyHistorySync.consumer.templateQPS") + if err != nil || value == nil || value.IsNil() { + return nil + } + var configured map[string]float64 + if err = value.Scan(&configured); err != nil { + g.Log().Warningf(ctx, "parse monthly usage history template QPS config failed: %+v", err) + return nil + } + return configured +} + +func superviseZSetConsumer(ctx context.Context, eventPrefix string, run func(context.Context) error) { + retryDelay := consumerRestartInitialDelay + for { + startedAt := time.Now() + runErr := run(ctx) + if ctx.Err() != nil { + return + } + if time.Since(startedAt) >= consumerRestartBackoffResetAfter { + retryDelay = consumerRestartInitialDelay + } + + errorText := "" + if runErr != nil { + errorText = runErr.Error() + } + g.Log().Warningf(ctx, "event=%s_consumer status=restart_scheduled retryDelayMs=%d error=%q", eventPrefix, retryDelay.Milliseconds(), errorText) + + timer := time.NewTimer(retryDelay) + select { + case <-ctx.Done(): + timer.Stop() + return + case <-timer.C: + } + + retryDelay *= 2 + if retryDelay > consumerRestartMaxDelay { + retryDelay = consumerRestartMaxDelay + } + } +} + +func loadFullSyncArchivePrefixes(ctx context.Context) ([]string, error) { + trafficPrefix := strings.TrimSpace(g.Cfg().MustGet(ctx, "trafficUsageFullSync.redisKeyPrefix", "flow:").String()) + if trafficPrefix == "" { + trafficPrefix = "flow:" + } + statusPrefix, err := loadSimStatusRunPrefix(ctx) + if err != nil { + return nil, err + } + dailyPrefix, err := loadDailyUsageRunPrefix(ctx) + if err != nil { + return nil, err + } + monthlyHistoryPrefix, err := loadMonthlyHistoryRunPrefix(ctx) + if err != nil { + return nil, err + } + imeiPrefix, err := loadImeiRunPrefix(ctx) + if err != nil { + return nil, err + } + // Duplicate prefixes would make Catalog and SimpleReplay read the same run + // hash twice. Deduplicate while preserving the traffic-first order. + prefixes := make([]string, 0, 5) + seen := make(map[string]struct{}, 5) + for _, prefix := range []string{trafficPrefix, statusPrefix, dailyPrefix, monthlyHistoryPrefix, imeiPrefix} { + if _, exists := seen[prefix]; exists { + continue + } + seen[prefix] = struct{}{} + prefixes = append(prefixes, prefix) + } + return prefixes, nil +} + +func Run(ctx context.Context) error { + workerCtx, stop := signal.NotifyContext(ctx, os.Interrupt, syscall.SIGTERM) + defer stop() + + if err := initialize(workerCtx); err != nil { + return err + } + + var ( + upstreamClient = upstream.New() + usageRepository = trafficusagerepository.New() + trafficUsage = trafficusageservice.New(upstreamClient, usageRepository) + workOrder = workorderservice.New() + balance = balanceservice.New() + inboundRepo = inboundrepository.New() + inboundExecutor = inboundservice.NewExecutorWithUpstream(inboundRepo, upstreamClient) + lifecycleRepo = lifecyclerepository.New() + lifecycleExecutor = lifecycleservice.NewExecutor(lifecycleRepo, upstreamClient) + cancelRepo = cancelrepository.New() + cancelExecutor = cancelservice.NewExecutor(cancelRepo) + renewChangeRepo = renewchangeservice.NewRepository() + addPackageCardExecutor = addpackageservice.NewCardExecutor(renewChangeRepo) + addPackagePoolExecutor = addpackageservice.NewPoolExecutor(renewChangeRepo) + renewPrepareExecutor = renewchangeservice.NewPrepareExecutor(renewChangeRepo) + prepareExecutor = addpackageservice.NewPrepareMux(renewChangeRepo, renewPrepareExecutor) + renewExecutor = renewchangeservice.NewRenewExecutor(renewChangeRepo) + changeExecutor = renewchangeservice.NewChangeExecutor(renewChangeRepo) + dataRepo = datarepository.New() + dataExecutor = dataservice.NewExecutor(dataRepo, upstreamClient) + unsubRepo = unsubrepository.New() + unsubExecutor = unsubservice.NewExecutor(unsubRepo) + restartExecutor = lifecycleservice.NewRestartExecutor(lifecycleRepo, upstreamClient) + batchRepository = batchrepository.New() + batchDispatcher = batch.NewDispatcher(lifecycleExecutor) + batchRuntime = batch.NewRuntime( + batchRepository, + batchDispatcher, + batch.WithExecutionTimeout(g.Cfg().MustGet(workerCtx, "rabbitmq.batchExecute.executionTimeout", "0s").Duration()), + batch.WithFinalizationTimeout(g.Cfg().MustGet(workerCtx, "rabbitmq.batchExecute.finalizationTimeout", "10s").Duration()), + ) + allocateExecutor = allocateservice.NewExecutor() + warningConfigExecutor = warningconfigservice.NewExecutor() + trafficCutoffConfigExecutor = trafficcutoffconfigservice.NewExecutor() + packageCustomerSync = packagecustomerservice.New() + ) + var ( + monthlyProducer *monthlyusage.Producer + monthlyConsumer *zsetconsumer.Consumer + statusProducer *simstatusproducer.Producer + statusConsumer *zsetconsumer.Consumer + dailyProducer *dailyusageproducer.Producer + dailyConsumer *zsetconsumer.Consumer + monthlyHistoryProducer *monthlyhistoryproducer.Producer + monthlyHistoryConsumer *zsetconsumer.Consumer + imeiProducer *imei.Producer + imeiConsumer *zsetconsumer.Consumer + ) + if g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.enabled", false).Bool() { + prefix := g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.redisKeyPrefix", "flow:").String() + store, err := zset.New(g.Redis(), prefix) + if err != nil { + return gerror.Wrap(err, "initialize monthly usage Redis store failed") + } + store.EnableArchiveIndex(g.Cfg().MustGet(workerCtx, "fullSyncOps.archive.enabled", false).Bool()) + producerConfig, configErr := loadMonthlyUsageProducerConfig(workerCtx) + if configErr != nil { + return configErr + } + var supplyTracker *supply.Tracker + if g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.supplyObservation.enabled", false).Bool() { + supplyTracker = supply.NewTracker(g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.consumer.enabled", false).Bool()) + producerConfig.SupplyObservation = supplyTracker + } + monthlyProducer, err = monthlyusage.New(&monthlyusage.DatabaseScanner{}, store, producerConfig) + if err != nil { + return gerror.Wrap(err, "initialize monthly usage producer failed") + } + monthlyUsageClient := upstream.NewWithCurrentUsageClient( + g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.consumer.chanClientName", "").String(), + g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.consumer.chanClientTimeout", "90s").Duration(), + ) + usageHandler, handlerErr := monthlyusageservice.NewHandler(monthlyUsageClient, monthlyusagerepository.NewRepository()) + if handlerErr != nil { + return gerror.Wrap(handlerErr, "initialize monthly usage handler failed") + } + monthlyConsumer, err = zsetconsumer.New(store, usageHandler, zsetconsumer.Config{ + SupplyObservation: supplyTracker, + LockKey: g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.consumer.lockKey", "flow:consumer:active").String(), + LockTTL: g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.consumer.lockTTL", "1m").Duration(), + PollInterval: g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.consumer.pollInterval", "1s").Duration(), + LeaseTTL: g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.consumer.leaseTTL", "3m").Duration(), + TaskTimeout: g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.consumer.taskTimeout", "2m").Duration(), + ClaimBatch: g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.consumer.claimBatch", 100).Int(), + MobileConcurrency: g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.consumer.mobileConcurrency", 30).Int(), + TelecomConcurrency: g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.consumer.telecomConcurrency", 20).Int(), + GenericConcurrency: g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.consumer.genericConcurrency", 1).Int(), + GenericQPS: g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.consumer.genericQPS", 1.5).Float64(), + TemplateQPS: loadMonthlyUsageTemplateQPS(workerCtx), + RoundRetryEnabled: g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.retry.enabled", true).Bool(), + MaxRounds: g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.retry.maxRounds", 5).Int(), + FailureRetention: g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.retry.failureRetention", "720h").Duration(), + ShouldRetryNextRound: func(result zsetconsumer.Result) bool { + return !result.Success && result.Retryable + }, + }) + if err != nil { + return gerror.Wrap(err, "initialize monthly usage consumer failed") + } + if g.Cfg().MustGet(workerCtx, "trafficUsageFullSync.consumer.enabled", false).Bool() { + go superviseZSetConsumer(workerCtx, "monthly_usage", monthlyConsumer.Run) + } + } + if g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.enabled", false).Bool() { + prefix, prefixErr := loadDailyUsageRunPrefix(workerCtx) + if prefixErr != nil { + return prefixErr + } + store, err := zset.New(g.Redis(), prefix) + if err != nil { + return gerror.Wrap(err, "initialize daily usage Redis store failed") + } + store.EnableArchiveIndex(g.Cfg().MustGet(workerCtx, "fullSyncOps.archive.enabled", false).Bool()) + dailyScanMode, dailyScanConcurrency, scanErr := loadHistoryUsageScanSettings(workerCtx, "trafficUsageDailySync") + if scanErr != nil { + return scanErr + } + dailyProducer, err = dailyusageproducer.New(&dailyusageproducer.DatabaseScanner{}, store, dailyusageproducer.Config{ + ScanMode: dailyScanMode, + ScanConcurrency: dailyScanConcurrency, + PageSize: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.producer.pageSize", 100).Int(), + GlobalHighWaterMark: int64(g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.producer.globalHighWaterMark", 0).Int64()), + ChannelHighWaterMark: int64(g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.producer.channelHighWaterMark", 0).Int64()), + PollInterval: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.producer.pollInterval", "2s").Duration(), + RunLockKey: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.producer.runLockKey", "daily:producer:active").String(), + RunLockTTL: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.producer.runLockTTL", "5m").Duration(), + RoundRetryEnabled: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.retry.enabled", true).Bool(), + MaxRounds: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.retry.maxRounds", 3).Int(), + FailureRetention: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.retry.failureRetention", "720h").Duration(), + }) + if err != nil { + return gerror.Wrap(err, "initialize daily usage producer failed") + } + dailyHandler, handlerErr := dailyusageservice.NewHandler(upstream.New(), trafficusagerepository.New()) + if handlerErr != nil { + return gerror.Wrap(handlerErr, "initialize daily usage handler failed") + } + dailyConsumer, err = zsetconsumer.New(store, dailyHandler, zsetconsumer.Config{ + RunQueueEnabled: true, + RoundRetryEnabled: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.retry.enabled", true).Bool(), + MaxRounds: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.retry.maxRounds", 3).Int(), + FailureRetention: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.retry.failureRetention", "720h").Duration(), + ShouldRetryNextRound: dailyusageservice.ShouldRetryNextRound, + BlockedTasksAsFailures: true, + EventPrefix: "daily_usage", + LockKey: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.consumer.lockKey", "daily:consumer:active").String(), + LockTTL: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.consumer.lockTTL", "1m").Duration(), + PollInterval: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.consumer.pollInterval", "1s").Duration(), + LeaseTTL: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.consumer.leaseTTL", "3m").Duration(), + TaskTimeout: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.consumer.taskTimeout", "2m").Duration(), + ClaimBatch: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.consumer.claimBatch", 100).Int(), + MobileConcurrency: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.consumer.mobileConcurrency", 30).Int(), + TelecomConcurrency: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.consumer.telecomConcurrency", 8).Int(), + GenericConcurrency: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.consumer.genericConcurrency", 1).Int(), + GenericQPS: g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.consumer.genericQPS", 1.5).Float64(), + TemplateQPS: loadDailyUsageTemplateQPS(workerCtx), + }) + if err != nil { + return gerror.Wrap(err, "initialize daily usage consumer failed") + } + if g.Cfg().MustGet(workerCtx, "trafficUsageDailySync.consumer.enabled", false).Bool() { + go superviseZSetConsumer(workerCtx, "daily_usage", dailyConsumer.Run) + } + } + if g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.enabled", false).Bool() { + prefix, prefixErr := loadMonthlyHistoryRunPrefix(workerCtx) + if prefixErr != nil { + return prefixErr + } + store, err := zset.New(g.Redis(), prefix) + if err != nil { + return gerror.Wrap(err, "initialize monthly usage history Redis store failed") + } + store.EnableArchiveIndex(g.Cfg().MustGet(workerCtx, "fullSyncOps.archive.enabled", false).Bool()) + monthlyHistoryScanMode, monthlyHistoryScanConcurrency, scanErr := loadHistoryUsageScanSettings(workerCtx, "trafficUsageMonthlyHistorySync") + if scanErr != nil { + return scanErr + } + monthlyHistoryProducer, err = monthlyhistoryproducer.New(&monthlyhistoryproducer.DatabaseScanner{}, store, monthlyhistoryproducer.Config{ + ScanMode: monthlyHistoryScanMode, + ScanConcurrency: monthlyHistoryScanConcurrency, + PageSize: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.producer.pageSize", 100).Int(), + GlobalHighWaterMark: int64(g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.producer.globalHighWaterMark", 0).Int64()), + ChannelHighWaterMark: int64(g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.producer.channelHighWaterMark", 0).Int64()), + PollInterval: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.producer.pollInterval", "2s").Duration(), + RunLockKey: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.producer.runLockKey", "monthly-history:producer:active").String(), + RunLockTTL: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.producer.runLockTTL", "5m").Duration(), + RoundRetryEnabled: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.retry.enabled", true).Bool(), + MaxRounds: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.retry.maxRounds", 3).Int(), + FailureRetention: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.retry.failureRetention", "720h").Duration(), + }) + if err != nil { + return gerror.Wrap(err, "initialize monthly usage history producer failed") + } + monthlyHistoryHandler, handlerErr := monthlyhistoryservice.NewHandler(upstream.New(), trafficusagerepository.New()) + if handlerErr != nil { + return gerror.Wrap(handlerErr, "initialize monthly usage history handler failed") + } + monthlyHistoryConsumer, err = zsetconsumer.New(store, monthlyHistoryHandler, zsetconsumer.Config{ + RunQueueEnabled: true, + RoundRetryEnabled: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.retry.enabled", true).Bool(), + MaxRounds: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.retry.maxRounds", 3).Int(), + FailureRetention: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.retry.failureRetention", "720h").Duration(), + ShouldRetryNextRound: monthlyhistoryservice.ShouldRetryNextRound, + BlockedTasksAsFailures: true, + EventPrefix: "monthly_usage_history", + LockKey: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.consumer.lockKey", "monthly-history:consumer:active").String(), + LockTTL: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.consumer.lockTTL", "1m").Duration(), + PollInterval: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.consumer.pollInterval", "1s").Duration(), + LeaseTTL: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.consumer.leaseTTL", "3m").Duration(), + TaskTimeout: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.consumer.taskTimeout", "2m").Duration(), + ClaimBatch: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.consumer.claimBatch", 100).Int(), + MobileConcurrency: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.consumer.mobileConcurrency", 30).Int(), + TelecomConcurrency: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.consumer.telecomConcurrency", 8).Int(), + GenericConcurrency: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.consumer.genericConcurrency", 1).Int(), + GenericQPS: g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.consumer.genericQPS", 1.5).Float64(), + TemplateQPS: loadMonthlyHistoryTemplateQPS(workerCtx), + }) + if err != nil { + return gerror.Wrap(err, "initialize monthly usage history consumer failed") + } + if g.Cfg().MustGet(workerCtx, "trafficUsageMonthlyHistorySync.consumer.enabled", false).Bool() { + go superviseZSetConsumer(workerCtx, "monthly_usage_history", monthlyHistoryConsumer.Run) + } + } + if g.Cfg().MustGet(workerCtx, "simStatusQueueSync.enabled", false).Bool() { + prefix, prefixErr := loadSimStatusRunPrefix(workerCtx) + if prefixErr != nil { + return prefixErr + } + store, err := zset.New(g.Redis(), prefix) + if err != nil { + return gerror.Wrap(err, "initialize sim status Redis store failed") + } + store.EnableArchiveIndex(g.Cfg().MustGet(workerCtx, "fullSyncOps.archive.enabled", false).Bool()) + statusProducer, err = simstatusproducer.New(&simstatusproducer.DatabaseScanner{}, store, simstatusproducer.Config{ + PageSize: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.producer.pageSize", 1000).Int(), + GlobalHighWaterMark: int64(g.Cfg().MustGet(workerCtx, "simStatusQueueSync.producer.globalHighWaterMark", 0).Int64()), + PollInterval: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.producer.pollInterval", "1s").Duration(), + RunLockKey: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.producer.runLockKey", "sim-status:producer:active").String(), + RunLockTTL: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.producer.runLockTTL", "5m").Duration(), + }) + if err != nil { + return gerror.Wrap(err, "initialize sim status producer failed") + } + statusHandler, handlerErr := simstatusservice.NewHandler( + upstream.New(), + simstatusrepository.NewRepository(), + g.Cfg().MustGet(workerCtx, "simStatusQueueSync.consumer.networkRetries", 2).Int(), + ) + if handlerErr != nil { + return gerror.Wrap(handlerErr, "initialize sim status handler failed") + } + statusConsumer, err = zsetconsumer.New(store, statusHandler, zsetconsumer.Config{ + RunQueueEnabled: true, + RoundRetryEnabled: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.retry.enabled", true).Bool(), + MaxRounds: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.retry.maxRounds", 3).Int(), + FailureRetention: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.retry.failureRetention", "720h").Duration(), + ShouldRetryNextRound: simstatusservice.ShouldRetryNextRound, + BlockedTasksAsFailures: true, + EventPrefix: "sim_status_queue", + LockKey: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.consumer.lockKey", "sim-status:consumer:active").String(), + LockTTL: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.consumer.lockTTL", "1m").Duration(), + PollInterval: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.consumer.pollInterval", "1s").Duration(), + LeaseTTL: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.consumer.leaseTTL", "3m").Duration(), + TaskTimeout: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.consumer.taskTimeout", "2m").Duration(), + ClaimBatch: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.consumer.claimBatch", 100).Int(), + MobileConcurrency: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.consumer.mobileConcurrency", 100).Int(), + TelecomConcurrency: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.consumer.telecomConcurrency", 8).Int(), + GenericConcurrency: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.consumer.genericConcurrency", 1).Int(), + GenericQPS: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.consumer.genericQPS", 1.5).Float64(), + TemplateQPS: loadSimStatusTemplateQPS(workerCtx), + FailurePoolAlertThreshold: g.Cfg().MustGet(workerCtx, "simStatusQueueSync.consumer.failurePoolAlertThreshold", 50000).Int64(), + }) + if err != nil { + return gerror.Wrap(err, "initialize sim status consumer failed") + } + if g.Cfg().MustGet(workerCtx, "simStatusQueueSync.consumer.enabled", false).Bool() { + go superviseZSetConsumer(workerCtx, "sim_status_queue", statusConsumer.Run) + } + } + if g.Cfg().MustGet(workerCtx, "imeiFullSync.enabled", false).Bool() { + prefix, prefixErr := loadImeiRunPrefix(workerCtx) + if prefixErr != nil { + return prefixErr + } + store, err := zset.New(g.Redis(), prefix) + if err != nil { + return gerror.Wrap(err, "initialize imei Redis store failed") + } + store.EnableArchiveIndex(g.Cfg().MustGet(workerCtx, "fullSyncOps.archive.enabled", false).Bool()) + // g.Redis may be initialized from g.Cfg without registering GetConfig. + leaseConfig, configured := gredis.GetConfig() + if !configured { + value, configErr := g.Cfg().Get(workerCtx, "redis.default") + if configErr != nil { + return gerror.Wrap(configErr, "load IMEI lease Redis configuration failed") + } + leaseConfig, err = gredis.ConfigFromMap(value.Map()) + if err != nil { + return gerror.Wrap(err, "parse IMEI lease Redis configuration failed") + } + } + imeiProducer, err = imei.New(&imei.DatabaseScanner{}, store, imei.Config{ + PageSize: g.Cfg().MustGet(workerCtx, "imeiFullSync.producer.pageSize", 1000).Int(), + GlobalHighWaterMark: int64(g.Cfg().MustGet(workerCtx, "imeiFullSync.producer.globalHighWaterMark", 0).Int64()), + PollInterval: g.Cfg().MustGet(workerCtx, "imeiFullSync.producer.pollInterval", "1s").Duration(), + RunLockKey: g.Cfg().MustGet(workerCtx, "imeiFullSync.producer.runLockKey", "imei:producer:active").String(), + RunLockTTL: g.Cfg().MustGet(workerCtx, "imeiFullSync.producer.runLockTTL", "5m").Duration(), + LeaseRedisConfig: leaseConfig, + }) + if err != nil { + return gerror.Wrap(err, "initialize imei producer failed") + } + imeiHandler, handlerErr := imeiservice.NewHandler( + upstream.NewWithImeiClient( + g.Cfg().MustGet(workerCtx, "imeiFullSync.consumer.chanClientName", "").String(), + g.Cfg().MustGet(workerCtx, "imeiFullSync.consumer.chanClientTimeout", "30s").Duration(), + ), + imeirepository.NewRepository(), + g.Cfg().MustGet(workerCtx, "imeiFullSync.consumer.networkRetries", 2).Int(), + ) + if handlerErr != nil { + return gerror.Wrap(handlerErr, "initialize imei handler failed") + } + imeiConsumer, err = zsetconsumer.New(store, imeiHandler, zsetconsumer.Config{ + RunQueueEnabled: true, + RoundRetryEnabled: g.Cfg().MustGet(workerCtx, "imeiFullSync.retry.enabled", true).Bool(), + MaxRounds: g.Cfg().MustGet(workerCtx, "imeiFullSync.retry.maxRounds", 5).Int(), + FailureRetention: g.Cfg().MustGet(workerCtx, "imeiFullSync.retry.failureRetention", "720h").Duration(), + ShouldRetryNextRound: imeiservice.ShouldRetryNextRound, + BlockedTasksAsFailures: true, + EventPrefix: "imei_queue", + LockKey: g.Cfg().MustGet(workerCtx, "imeiFullSync.consumer.lockKey", "imei:consumer:active").String(), + LockTTL: g.Cfg().MustGet(workerCtx, "imeiFullSync.consumer.lockTTL", "1m").Duration(), + PollInterval: g.Cfg().MustGet(workerCtx, "imeiFullSync.consumer.pollInterval", "1s").Duration(), + LeaseTTL: g.Cfg().MustGet(workerCtx, "imeiFullSync.consumer.leaseTTL", "3m").Duration(), + TaskTimeout: g.Cfg().MustGet(workerCtx, "imeiFullSync.consumer.taskTimeout", "2m").Duration(), + ClaimBatch: g.Cfg().MustGet(workerCtx, "imeiFullSync.consumer.claimBatch", 100).Int(), + MobileConcurrency: g.Cfg().MustGet(workerCtx, "imeiFullSync.consumer.mobileConcurrency", 100).Int(), + TelecomConcurrency: g.Cfg().MustGet(workerCtx, "imeiFullSync.consumer.telecomConcurrency", 8).Int(), + GenericConcurrency: g.Cfg().MustGet(workerCtx, "imeiFullSync.consumer.genericConcurrency", 1).Int(), + GenericQPS: g.Cfg().MustGet(workerCtx, "imeiFullSync.consumer.genericQPS", 1.5).Float64(), + TemplateQPS: loadImeiTemplateQPS(workerCtx), + // 失败池是唯一随卡数线性增长的每卡结构（单条约 1KB）， + // 且会保留整个 failureRetention。超阀值只告警不截断。 + FailurePoolAlertThreshold: g.Cfg().MustGet(workerCtx, "imeiFullSync.consumer.failurePoolAlertThreshold", 50000).Int64(), + }) + if err != nil { + return gerror.Wrap(err, "initialize imei consumer failed") + } + if g.Cfg().MustGet(workerCtx, "imeiFullSync.consumer.enabled", false).Bool() { + go superviseZSetConsumer(workerCtx, "imei_queue", imeiConsumer.Run) + } + } + var rabbitMQConsumers *rabbitmqconsumer.Consumers + if monthlyProducer != nil { + rabbitMQConsumers = rabbitmqconsumer.NewWithPackageCustomerSyncAndMonthlyUsage(trafficUsage, workOrder, balance, packageCustomerSync, monthlyProducer, batchRuntime) + } else { + rabbitMQConsumers = rabbitmqconsumer.NewWithPackageCustomerSync(trafficUsage, workOrder, balance, packageCustomerSync, batchRuntime) + } + if statusProducer != nil { + rabbitMQConsumers.WithSimStatusQueue(statusProducer, statusProducer) + } + if dailyProducer != nil { + rabbitMQConsumers.WithDailyUsageStart(dailyProducer) + } + if monthlyHistoryProducer != nil { + rabbitMQConsumers.WithMonthlyHistoryStart(monthlyHistoryProducer) + } + if imeiProducer != nil { + rabbitMQConsumers.WithImeiQueue(imeiProducer) + } + var ( + fullSyncHTTP *httpapi.Server + fullSyncArchives []*fullsyncarchive.Service + ) + { // The authenticated control plane is always available. + prefixes, err := loadFullSyncArchivePrefixes(workerCtx) + if err != nil { + return err + } + stores := make([]*zset.Store, 0, len(prefixes)) + for _, prefix := range prefixes { + store, e := zset.New(g.Redis(), prefix) + if e != nil { + return e + } + stores = append(stores, store) + } + archiveRoot := g.Cfg().MustGet(workerCtx, "fullSyncOps.archive.rootDir", "/app/logs/full-sync").String() + repositories := make([]*fullsyncruntime.Repository, 0, len(stores)) + for _, store := range stores { + repo, e := fullsyncruntime.New(store, archiveRoot) + if e != nil { + return gerror.Wrap(e, "initialize full-sync runtime repository failed") + } + repositories = append(repositories, repo) + } + catalog, err := fullsyncruntime.NewCatalog(repositories...) + if err != nil { + return gerror.Wrap(err, "initialize full-sync run catalog failed") + } + simple, err := fullsyncruntime.NewSimpleReplayWithDefault(g.Cfg().MustGet(workerCtx, "fullSyncOps.replay.maxRounds", fullsyncruntime.DefaultReplayMaxRounds).Int(), stores...) + if err != nil { + return err + } + // Only Admin exposes replay to users. This listener stays inside the + // Docker network; the Worker service must not publish port 18081. + fullSyncHTTP, err = httpapi.NewServer(httpapi.Config{Addr: ":18081", ShutdownTimeout: g.Cfg().MustGet(workerCtx, "fullSyncOps.control.shutdownTimeout", "10s").Duration()}, httpapi.Dependencies{Runs: catalog, List: catalog, Failures: catalog, Replays: repositories[0], Actions: repositories[0], Simple: simple}) + if err != nil { + return gerror.Wrap(err, "initialize full-sync control server failed") + } + if err := fullSyncHTTP.Start(workerCtx); err != nil { + return gerror.Wrap(err, "start full-sync control server failed") + } + simple.Start(workerCtx) + } + if g.Cfg().MustGet(workerCtx, "fullSyncOps.archive.enabled", false).Bool() { + prefixes, err := loadFullSyncArchivePrefixes(workerCtx) + if err != nil { + return gerror.Wrap(err, "load full-sync archive Redis prefixes failed") + } + for _, prefix := range prefixes { + store, err := zset.New(g.Redis(), prefix) + if err != nil { + return gerror.Wrapf(err, "initialize full-sync archive Redis store failed for prefix %q", prefix) + } + store.EnableArchiveIndex(true) + repo, err := fullsyncruntime.New(store, g.Cfg().MustGet(workerCtx, "fullSyncOps.archive.rootDir", "/app/logs/full-sync").String()) + if err != nil { + return gerror.Wrapf(err, "initialize full-sync archive repository failed for prefix %q", prefix) + } + service := fullsyncarchive.NewService(fullsyncarchive.Config{Root: g.Cfg().MustGet(workerCtx, "fullSyncOps.archive.rootDir", "/app/logs/full-sync").String(), Interval: g.Cfg().MustGet(workerCtx, "fullSyncOps.archive.pollInterval", "10s").Duration(), InitialBackoff: g.Cfg().MustGet(workerCtx, "fullSyncOps.archive.retryMin", "5s").Duration(), MaxBackoff: g.Cfg().MustGet(workerCtx, "fullSyncOps.archive.retryMax", "5m").Duration()}, repo, repo, repo) + if err := service.Start(workerCtx); err != nil { + return gerror.Wrapf(err, "start full-sync archive service failed for prefix %q", prefix) + } + fullSyncArchives = append(fullSyncArchives, service) + } + } + batchDispatcher.Register(consts.BatchTypeInbound, inboundExecutor) + batchDispatcher.Register(consts.BatchTypeAllocate, allocateExecutor) + batchDispatcher.Register(consts.BatchTypeCardAddPackage, addPackageCardExecutor) + batchDispatcher.Register(consts.BatchTypePoolAddPackage, addPackagePoolExecutor) + batchDispatcher.Register(consts.BatchTypePrepare, prepareExecutor) + batchDispatcher.Register(consts.BatchTypeRenew, renewExecutor) + batchDispatcher.Register(consts.BatchTypeChange, changeExecutor) + batchDispatcher.Register(consts.BatchTypeDataCut, dataExecutor) + batchDispatcher.Register(consts.BatchTypeDataStart, dataExecutor) + batchDispatcher.Register(consts.BatchTypeDataStop, dataExecutor) + batchDispatcher.Register(consts.BatchTypeSimCardUnsub, unsubExecutor) + batchDispatcher.Register(consts.BatchTypeSimCardRestart, restartExecutor) + batchDispatcher.Register(consts.BatchTypeSimCardCancel, cancelExecutor) + batchDispatcher.Register(consts.BatchTypeSimWarningConfig, warningConfigExecutor) + batchDispatcher.Register(consts.BatchTypeTrafficCutoffConfig, trafficCutoffConfigExecutor) + if err := rabbitMQConsumers.Start(workerCtx); err != nil { + _ = rabbitMQConsumers.Stop(context.Background()) + return gerror.Wrap(err, "start RabbitMQ consumers failed") + } + + g.Log().Info(workerCtx, "iotlcm worker started") + <-workerCtx.Done() + g.Log().Info(ctx, "iotlcm worker stopping") + + shutdownCtx, cancel := context.WithTimeout(context.Background(), shutdownTimeout) + defer cancel() + if err := rabbitMQConsumers.Stop(shutdownCtx); err != nil { + return gerror.Wrap(err, "stop RabbitMQ consumers failed") + } + for _, service := range fullSyncArchives { + if err := service.Stop(); err != nil { + return gerror.Wrap(err, "stop full-sync archive service failed") + } + } + if fullSyncHTTP != nil { + if err := fullSyncHTTP.Wait(); err != nil { + return gerror.Wrap(err, "stop full-sync control server failed") + } + } + if err := tdengine.Close(); err != nil { + return gerror.Wrap(err, "close TDengine failed") + } + return nil +} + +func initialize(ctx context.Context) error { + if err := g.DB().PingMaster(); err != nil { + return gerror.Wrap(err, "connect database failed") + } + if err := gtime.SetTimeZone("Asia/Shanghai"); err != nil { + return gerror.Wrap(err, "set timezone failed") + } + + mode := g.Cfg().MustGet(ctx, "system.gfmode", gmode.NOT_SET).String() + for _, supported := range []string{gmode.DEVELOP, gmode.TESTING, gmode.STAGING, gmode.PRODUCT} { + if mode == supported { + gmode.Set(mode) + break + } + } + return nil +} diff --git a/internal/worker/worker_test.go b/internal/worker/worker_test.go new file mode 100644 index 0000000..70c8e41 --- /dev/null +++ b/internal/worker/worker_test.go @@ -0,0 +1,198 @@ +package worker + +import ( + "context" + "reflect" + "testing" + + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gcfg" +) + +func TestLoadMonthlyUsageTemplateQPS(t *testing.T) { + oldAdapter := g.Cfg().GetAdapter() + adapter, err := gcfg.NewAdapterContent(` +trafficUsageFullSync: + consumer: + templateQPS: + sim_yd_v5: 30 + sim_dx_v1: 25 + qibin_direct: 1.5 +`) + if err != nil { + t.Fatalf("create config adapter: %v", err) + } + g.Cfg().SetAdapter(adapter) + defer g.Cfg().SetAdapter(oldAdapter) + + got := loadMonthlyUsageTemplateQPS(context.Background()) + if got["sim_yd_v5"] != 30 || got["sim_dx_v1"] != 25 || got["qibin_direct"] != 1.5 { + t.Fatalf("template QPS = %#v, want configured values", got) + } +} + +func TestLoadSimStatusRunPrefix(t *testing.T) { + oldAdapter := g.Cfg().GetAdapter() + defer g.Cfg().SetAdapter(oldAdapter) + for _, test := range []struct { + name, config, want string + invalid bool + }{ + {"default", "simStatusQueueSync:\n enabled: true\n", "sim-status:", false}, + {"custom", "simStatusQueueSync:\n redisKeyPrefix: 'status:'", "status:", false}, + {"empty", "simStatusQueueSync:\n redisKeyPrefix: ''", "", true}, + } { + t.Run(test.name, func(t *testing.T) { + adapter, err := gcfg.NewAdapterContent(test.config) + if err != nil { + t.Fatal(err) + } + g.Cfg().SetAdapter(adapter) + got, err := loadSimStatusRunPrefix(context.Background()) + if got != test.want || (err != nil) != test.invalid { + t.Fatalf("prefix=%q err=%v", got, err) + } + }) + } +} + +func TestLoadDailyUsageTemplateQPS(t *testing.T) { + oldAdapter := g.Cfg().GetAdapter() + adapter, err := gcfg.NewAdapterContent(` +trafficUsageDailySync: + consumer: + templateQPS: + sim_yd_v5: 30 + sim_dx_v1: 25 +`) + if err != nil { + t.Fatalf("create config adapter: %v", err) + } + g.Cfg().SetAdapter(adapter) + defer g.Cfg().SetAdapter(oldAdapter) + + got := loadDailyUsageTemplateQPS(context.Background()) + if got["sim_yd_v5"] != 30 || got["sim_dx_v1"] != 25 { + t.Fatalf("daily usage template QPS = %#v, want the daily-configured values", got) + } + // The daily chain must not silently inherit the monthly chain's rates; the + // two run against the same carrier quota and must be tuned independently. + monthly := loadMonthlyUsageTemplateQPS(context.Background()) + if len(monthly) != 0 { + t.Fatalf("monthly template QPS = %#v, want it unaffected by the daily block", monthly) + } +} + +func TestLoadDailyUsageRunPrefix(t *testing.T) { + oldAdapter := g.Cfg().GetAdapter() + defer g.Cfg().SetAdapter(oldAdapter) + for _, test := range []struct { + name, config, want string + invalid bool + }{ + {"default", "trafficUsageDailySync:\n enabled: true\n", "daily:", false}, + {"custom", "trafficUsageDailySync:\n redisKeyPrefix: 'daily-usage:'", "daily-usage:", false}, + {"empty", "trafficUsageDailySync:\n redisKeyPrefix: ''", "", true}, + } { + t.Run(test.name, func(t *testing.T) { + adapter, err := gcfg.NewAdapterContent(test.config) + if err != nil { + t.Fatal(err) + } + g.Cfg().SetAdapter(adapter) + got, err := loadDailyUsageRunPrefix(context.Background()) + if got != test.want || (err != nil) != test.invalid { + t.Fatalf("prefix=%q err=%v", got, err) + } + }) + } +} + +func TestLoadMonthlyHistoryRunPrefix(t *testing.T) { + oldAdapter := g.Cfg().GetAdapter() + defer g.Cfg().SetAdapter(oldAdapter) + for _, test := range []struct { + name, config, want string + invalid bool + }{ + {"default", "trafficUsageMonthlyHistorySync:\n enabled: true\n", "monthly-history:", false}, + {"custom", "trafficUsageMonthlyHistorySync:\n redisKeyPrefix: 'mh:'", "mh:", false}, + {"empty", "trafficUsageMonthlyHistorySync:\n redisKeyPrefix: ''", "", true}, + } { + t.Run(test.name, func(t *testing.T) { + adapter, err := gcfg.NewAdapterContent(test.config) + if err != nil { + t.Fatal(err) + } + g.Cfg().SetAdapter(adapter) + got, err := loadMonthlyHistoryRunPrefix(context.Background()) + if got != test.want || (err != nil) != test.invalid { + t.Fatalf("prefix=%q err=%v", got, err) + } + }) + } +} + +// TestMonthlyHistoryPrefixIsIsolatedFromCurrentMonthSync pins the separation of +// the two monthly chains: trafficUsageFullSync fills the MySQL current-month +// snapshot, while trafficUsageMonthlyHistorySync fills the TDengine monthly +// table. Sharing a prefix would let one chain's producer lock block the other. +func TestMonthlyHistoryPrefixIsIsolatedFromCurrentMonthSync(t *testing.T) { + oldAdapter := g.Cfg().GetAdapter() + defer g.Cfg().SetAdapter(oldAdapter) + adapter, err := gcfg.NewAdapterContent(` +trafficUsageFullSync: + redisKeyPrefix: 'flow:' +trafficUsageMonthlyHistorySync: + redisKeyPrefix: 'monthly-history:' +`) + if err != nil { + t.Fatal(err) + } + g.Cfg().SetAdapter(adapter) + prefixes, err := loadFullSyncArchivePrefixes(context.Background()) + if err != nil { + t.Fatal(err) + } + seen := map[string]bool{} + for _, p := range prefixes { + seen[p] = true + } + if !seen["flow:"] || !seen["monthly-history:"] { + t.Fatalf("prefixes=%q, want both the current-month and monthly-history chains present", prefixes) + } +} + +func TestLoadFullSyncArchivePrefixes(t *testing.T) { + oldAdapter := g.Cfg().GetAdapter() + defer g.Cfg().SetAdapter(oldAdapter) + for _, test := range []struct { + name, config string + want []string + }{ + {"defaults", "{}", []string{"flow:", "sim-status:", "daily:", "monthly-history:", "imei:"}}, + {"disabled IMEI remains visible", "imeiFullSync:\n enabled: false", []string{"flow:", "sim-status:", "daily:", "monthly-history:", "imei:"}}, + {"custom", "trafficUsageFullSync:\n redisKeyPrefix: 'usage:'\nsimStatusQueueSync:\n redisKeyPrefix: 'status:'\ntrafficUsageDailySync:\n redisKeyPrefix: 'daily-usage:'\ntrafficUsageMonthlyHistorySync:\n redisKeyPrefix: 'monthly-history:'\nimeiFullSync:\n redisKeyPrefix: 'imei-usage:'", []string{"usage:", "status:", "daily-usage:", "monthly-history:", "imei-usage:"}}, + // Duplicate prefixes would make Catalog and SimpleReplay read the same + // run hash twice, so the list is deduplicated with the first occurrence + // keeping its position. + {"shared prefix", "trafficUsageFullSync:\n redisKeyPrefix: 'sync:'\nsimStatusQueueSync:\n redisKeyPrefix: 'sync:'\ntrafficUsageDailySync:\n redisKeyPrefix: 'sync:'\ntrafficUsageMonthlyHistorySync:\n redisKeyPrefix: 'sync:'\nimeiFullSync:\n redisKeyPrefix: 'sync:'", []string{"sync:"}}, + {"daily shares the status prefix", "simStatusQueueSync:\n redisKeyPrefix: 'daily:'\ntrafficUsageDailySync:\n redisKeyPrefix: 'daily:'", []string{"flow:", "daily:", "monthly-history:", "imei:"}}, + {"monthly history shares the daily prefix", "trafficUsageDailySync:\n redisKeyPrefix: 'usage:'\ntrafficUsageMonthlyHistorySync:\n redisKeyPrefix: 'usage:'", []string{"flow:", "sim-status:", "usage:", "imei:"}}, + } { + t.Run(test.name, func(t *testing.T) { + adapter, err := gcfg.NewAdapterContent(test.config) + if err != nil { + t.Fatal(err) + } + g.Cfg().SetAdapter(adapter) + got, err := loadFullSyncArchivePrefixes(context.Background()) + if err != nil { + t.Fatal(err) + } + if !reflect.DeepEqual(got, test.want) { + t.Fatalf("prefixes=%q, want %q", got, test.want) + } + }) + } +} diff --git a/main.go b/main.go new file mode 100644 index 0000000..c72fda8 --- /dev/null +++ b/main.go @@ -0,0 +1,30 @@ +package main + +import ( + _ "github.com/gogf/gf/contrib/drivers/mysql/v2" + _ "github.com/gogf/gf/contrib/drivers/sqlite/v2" + _ "github.com/gogf/gf/contrib/nosql/redis/v2" + "github.com/gogf/gf/v2/frame/g" + "github.com/gogf/gf/v2/os/gctx" + + "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/cmd" + _ "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/packed" +) + +var ( + BuildVersion = "0.0" + BuildTime = "" + CommitID = "" +) + +func main() { + ctx := gctx.GetInitCtx() + g.Log().Noticef( + ctx, + "iotlcm-worker-svr starting, version:%s, buildTime:%s, commitID:%s", + BuildVersion, + BuildTime, + CommitID, + ) + cmd.Main.Run(ctx) +} diff --git a/manifest/deploy/kustomize/base/deployment.yaml b/manifest/deploy/kustomize/base/deployment.yaml new file mode 100644 index 0000000..3673830 --- /dev/null +++ b/manifest/deploy/kustomize/base/deployment.yaml @@ -0,0 +1,21 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: iotlcm-worker-svr + labels: + app: iotlcm-worker-svr +spec: + replicas: 1 + selector: + matchLabels: + app: iotlcm-worker-svr + template: + metadata: + labels: + app: iotlcm-worker-svr + spec: + terminationGracePeriodSeconds: 15 + containers: + - name: worker + image: iotlcm-worker-svr + imagePullPolicy: Always diff --git a/manifest/deploy/kustomize/base/kustomization.yaml b/manifest/deploy/kustomize/base/kustomization.yaml new file mode 100644 index 0000000..9c2d28b --- /dev/null +++ b/manifest/deploy/kustomize/base/kustomization.yaml @@ -0,0 +1,4 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - deployment.yaml diff --git a/manifest/deploy/kustomize/overlays/develop/deployment.yaml b/manifest/deploy/kustomize/overlays/develop/deployment.yaml new file mode 100644 index 0000000..3896238 --- /dev/null +++ b/manifest/deploy/kustomize/overlays/develop/deployment.yaml @@ -0,0 +1,10 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: iotlcm-worker-svr +spec: + template: + spec: + containers: + - name: worker + image: iotlcm-worker-svr:develop diff --git a/manifest/deploy/kustomize/overlays/develop/kustomization.yaml b/manifest/deploy/kustomize/overlays/develop/kustomization.yaml new file mode 100644 index 0000000..c2ca26e --- /dev/null +++ b/manifest/deploy/kustomize/overlays/develop/kustomization.yaml @@ -0,0 +1,7 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: default +resources: + - ../../base +patchesStrategicMerge: + - deployment.yaml diff --git a/manifest/docker/Dockerfile b/manifest/docker/Dockerfile new file mode 100644 index 0000000..285bf2b --- /dev/null +++ b/manifest/docker/Dockerfile @@ -0,0 +1,9 @@ +FROM loads/alpine:3.8 + +ENV WORKDIR=/app +ADD resource $WORKDIR/ +ADD ./temp/linux_amd64/iotlcm-worker-svr $WORKDIR/iotlcm-worker-svr +RUN chmod +x $WORKDIR/iotlcm-worker-svr + +WORKDIR $WORKDIR +CMD ./iotlcm-worker-svr diff --git a/manifest/docker/docker-compose.yml b/manifest/docker/docker-compose.yml new file mode 100644 index 0000000..b8ab6f8 --- /dev/null +++ b/manifest/docker/docker-compose.yml @@ -0,0 +1,10 @@ +services: + iotlcm-worker-svr: + image: your-registry/iotlcm/iotlcm-worker-svr:${IMAGE_TAG:-latest} + container_name: iotlcm-worker-svr + restart: unless-stopped + volumes: + - ${LOG_DIR:-/data/app/iotlcm-worker-svr/logs}:/app/logs + - ${FULL_SYNC_ARCHIVE_DIR:-/data/logs/iotlcm-worker-svr/full-sync}:/app/logs/full-sync + # Must match the Admin service upload mount. + - ${UPLOAD_DIR:-/data/app/iotlcmadmin-web-svc/upload}:/app/upload diff --git a/manifest/docker/docker.sh b/manifest/docker/docker.sh new file mode 100644 index 0000000..ff393f9 --- /dev/null +++ b/manifest/docker/docker.sh @@ -0,0 +1,8 @@ +#!/bin/bash + +# This shell is executed before docker build. + + + + + diff --git a/manifest/docker/run.sh b/manifest/docker/run.sh new file mode 100644 index 0000000..2d3363a --- /dev/null +++ b/manifest/docker/run.sh @@ -0,0 +1,26 @@ +#!/bin/bash +set -euo pipefail + +CONTAINER_NAME="${CONTAINER_NAME:-iotlcm-worker-svr}" +IMAGE="${IMAGE:-your-registry/iotlcm/iotlcm-worker-svr:latest}" +LOG_DIR="${LOG_DIR:-/data/app/iotlcm-worker-svr/logs}" +FULL_SYNC_ARCHIVE_DIR="${FULL_SYNC_ARCHIVE_DIR:-/data/logs/iotlcm-worker-svr/full-sync}" + +# Must match the Admin service upload mount, otherwise Worker cannot read +# Admin-created local OSS files such as async inbound Excel sources. +UPLOAD_DIR="${UPLOAD_DIR:-/data/app/iotlcmadmin-web-svc/upload}" + +mkdir -p "${LOG_DIR}" "${FULL_SYNC_ARCHIVE_DIR}" "${UPLOAD_DIR}" + +if docker ps -a --format '{{.Names}}' | grep -qx "${CONTAINER_NAME}"; then + docker stop "${CONTAINER_NAME}" >/dev/null 2>&1 || true + docker rm "${CONTAINER_NAME}" >/dev/null +fi + +docker run -d \ + --name "${CONTAINER_NAME}" \ + --restart unless-stopped \ + -v "${LOG_DIR}:/app/logs" \ + -v "${FULL_SYNC_ARCHIVE_DIR}:/app/logs/full-sync" \ + -v "${UPLOAD_DIR}:/app/upload" \ + "${IMAGE}"