sync: 同步 v1.0.4 开源代码并脱敏
This commit is contained in:
@@ -11,3 +11,4 @@
|
|||||||
*.ttf binary
|
*.ttf binary
|
||||||
*.pdf binary
|
*.pdf binary
|
||||||
*.exe binary
|
*.exe binary
|
||||||
|
*.go linguist-language=Go
|
||||||
|
|||||||
@@ -1,120 +1,27 @@
|
|||||||
# iotlcm-worker-svr
|
# iotlcm-worker-svr
|
||||||
|
|
||||||
iotLCM 物联网卡全生命周期管理平台的**异步执行中枢** —— RabbitMQ 消费者 + 全量同步编排器。
|
Dedicated RabbitMQ consumer service for IotLCM asynchronous work.
|
||||||
|
|
||||||
平台的大部分耗时操作(批量开停复机、续费变更、流量/状态/IMEI 全量同步、达量断网、工单自动关闭)都在本服务里异步执行。
|
## Consumers
|
||||||
|
|
||||||
## 特性
|
The service always registers these business consumers when RabbitMQ is enabled:
|
||||||
|
|
||||||
- **多队列消费者**:注册业务消费者处理重试队列、批次执行、工单等
|
- `iotlcm.traffic_usage.daily.retry`
|
||||||
- **全量同步编排**:卡状态 / 本月流量 / 日流量 / 月流量历史 / IMEI 五条全量同步链路,统一的 Producer(扫描 + 供料)→ Consumer(领取 + 执行)模型
|
- `iotlcm.traffic_usage.monthly.retry`
|
||||||
- **通道级并发与限速**:按通道模板配置 QPS,按通道隔离并发度
|
- `iotlcm.work_order.auto_close`
|
||||||
- **多轮执行**:失败卡进入下一轮重试,单轮上限与轮次上限可配
|
|
||||||
- **运行归档**:每轮执行产出终态 JSON 归档,供通知与排障使用
|
|
||||||
- **可观测**:阶段耗时、库存水位、缺料观测等遥测写入归档
|
|
||||||
|
|
||||||
## 技术栈
|
For local Admin-to-Worker integration checks, set `workerDiagnostic.enabled: true` to
|
||||||
|
register `iotlcm.worker.diagnostic`. Diagnostic messages are logged and acknowledged;
|
||||||
|
they do not access MySQL, TDengine, or Upstream. The diagnostic consumer is disabled
|
||||||
|
in the default and build configurations.
|
||||||
|
|
||||||
- Go 1.25 + GoFrame v2
|
## SIM status synchronization
|
||||||
- RabbitMQ(`github.com/rabbitmq/amqp091-go`)
|
|
||||||
- MySQL(业务库)、Redis(任务队列与锁)、TDengine(用量时序)
|
|
||||||
- 支付宝(余额充值)、excelize(Excel 导入导出)
|
|
||||||
|
|
||||||
## 快速开始
|
Status sync uses run-scoped queues and durable outcome counters. See
|
||||||
|
[Run accounting and rollout](SIM_STATUS_RUN_ACCOUNTING.md) for the new
|
||||||
|
`simStatusQueueSync.redisKeyPrefix` setting and tests.
|
||||||
|
|
||||||
```bash
|
|
||||||
go mod tidy
|
|
||||||
go run .
|
|
||||||
```
|
|
||||||
|
|
||||||
## 配置
|
## 角色权限同步
|
||||||
|
|
||||||
配置文件在 `config/config_default.yaml`(默认配置)与 `config/config.yaml`(部署时覆盖)。**公开版里的凭据与地址全部是占位符**:
|
配置 `roleSync.enabled: true` 后,独立接管管理后端的持久化角色权限任务。部署顺序与回滚见 `docs/角色权限任务切换.md`。默认关闭,必须先执行迁移并移除旧管理后端执行入口。
|
||||||
|
|
||||||
| 配置项 | 占位 | 说明 |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `database.default.link` | `${DB_USER}:${DB_PASSWORD}@tcp(${DB_HOST}:3306)` | 业务库 |
|
|
||||||
| `redis.default.address` | `${REDIS_ADDR}` | 任务队列 |
|
|
||||||
| `redis.default.pass` | `${REDIS_PASSWORD}` | 缓存口令 |
|
|
||||||
|
|
||||||
上游凭据(AppKey / AppSecret)**不写在本仓**,存放在通道表 `t_sim_channels` 与通道模板表里,由平台写入。
|
|
||||||
|
|
||||||
## 目录结构
|
|
||||||
|
|
||||||
```text
|
|
||||||
iotlcm-worker-svr/
|
|
||||||
├── internal/
|
|
||||||
│ ├── cmd/ # 启动装配
|
|
||||||
│ ├── consumer/
|
|
||||||
│ │ ├── rabbitmq/ # RabbitMQ 消费者(批次、重试、工单等)
|
|
||||||
│ │ └── zset/ # 全量同步消费者(ZSet 任务队列)
|
|
||||||
│ ├── producer/
|
|
||||||
│ │ ├── simstatus/ # 卡状态全量同步
|
|
||||||
│ │ ├── monthlyusage/ # 本月流量全量同步
|
|
||||||
│ │ ├── dailyusage/ # 日流量全量同步
|
|
||||||
│ │ ├── monthlyusagehistory/ # 月流量历史同步
|
|
||||||
│ │ └── imei/ # IMEI 全量同步
|
|
||||||
│ ├── fullsync/ # 全量同步运行时、归档、契约、HTTP 诊断
|
|
||||||
│ ├── service/ # 各业务动作执行器
|
|
||||||
│ ├── repository/ # 数据访问
|
|
||||||
│ ├── queue/zset/ # 任务队列与锁
|
|
||||||
│ └── library/rabbitmq/ # MQ 封装
|
|
||||||
├── config/ # 配置
|
|
||||||
├── build/ # Dockerfile 与镜像内配置
|
|
||||||
├── manifest/deploy/kustomize/ # K8s 部署清单
|
|
||||||
└── manifest/docker/ # compose 与启动脚本
|
|
||||||
```
|
|
||||||
|
|
||||||
## 全量同步模型
|
|
||||||
|
|
||||||
```text
|
|
||||||
Producer(扫描卡 → 按通道分片入队)
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
ZSet 任务队列(Redis)
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
Consumer(按通道 QPS 领取 → 调上游 → 写库 → 结算)
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
运行归档(终态 JSON,供 Bot 通知与排障)
|
|
||||||
```
|
|
||||||
|
|
||||||
关键设计:
|
|
||||||
|
|
||||||
- **通道隔离**:每个通道独立并发与 QPS,一个慢通道不拖垮其他通道
|
|
||||||
- **供料与消费解耦**:Producer 只负责入队,Consumer 按节奏领取,避免上游被瞬时打满
|
|
||||||
- **多轮重试**:单卡失败进入下一轮,轮次上限与单轮上限均可配
|
|
||||||
- **归档去重**:按 `runId` 标识一次执行,Bot 据此去重推送
|
|
||||||
|
|
||||||
## 容器化构建
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker build -f build/Dockerfile -t iotlcm-worker-svr:latest .
|
|
||||||
```
|
|
||||||
|
|
||||||
镜像内不含任何凭据;`config/config.yaml` 以只读方式挂载覆盖。
|
|
||||||
|
|
||||||
## 测试
|
|
||||||
|
|
||||||
```bash
|
|
||||||
go test ./...
|
|
||||||
```
|
|
||||||
|
|
||||||
测试使用本地替身,不访问真实运营商接口。
|
|
||||||
|
|
||||||
> **已知失败(与上游主分支一致,非本仓引入)**:
|
|
||||||
> `TestValidateUpstreamBasicInfoAcceptsMatchingIdentifiers`、
|
|
||||||
> `TestConfiguredInboundBasicInfoClientUsesDedicatedEndpoint`、
|
|
||||||
> `TestAllocateProcessingRetryAfterHasLargeBatchDefault`。
|
|
||||||
> 这三项在源仓库同一提交上同样失败,属既有缺陷。
|
|
||||||
|
|
||||||
## 相关项目
|
|
||||||
|
|
||||||
- [iotlcm-bot](https://git.5iot.com/iotlcm-public/iotlcm-bot) —— 读取归档 JSON 并推送飞书卡片
|
|
||||||
- [iotlcm-upstream-chan](https://git.5iot.com/iotlcm-public/iotlcm-upstream-chan) —— 运营商上游适配
|
|
||||||
- [iotlcm-admin-svr](https://git.5iot.com/iotlcm-public/iotlcm-admin-svr) —— 管理后台(发起批次与定时任务)
|
|
||||||
|
|
||||||
## License
|
|
||||||
|
|
||||||
[Apache License 2.0](./LICENSE)
|
|
||||||
|
|||||||
+8
-8
@@ -1,32 +1,32 @@
|
|||||||
FROM golang:1.25-alpine AS builder
|
ARG BASE_REGISTRY=docker.io/library
|
||||||
|
FROM ${BASE_REGISTRY}/golang:1.25-alpine AS builder
|
||||||
LABEL stage=gobuilder
|
LABEL stage=gobuilder
|
||||||
|
|
||||||
ENV GOPROXY=https://goproxy.cn,direct
|
ENV GOPROXY=https://goproxy.cn,direct
|
||||||
ENV TZ=Asia/Shanghai
|
ENV TZ=Asia/Shanghai
|
||||||
|
|
||||||
|
# RUN apk update --no-cache && apk add --no-cache expect git tzdata
|
||||||
WORKDIR /build
|
WORKDIR /build
|
||||||
|
|
||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
|
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
|
|
||||||
COPY . .
|
COPY . .
|
||||||
|
|
||||||
ARG BuildVersion
|
ARG BuildVersion
|
||||||
ARG BuildTime
|
ARG BuildTime
|
||||||
ARG CommitID
|
ARG CommitID
|
||||||
|
|
||||||
RUN go build -v -o /build/iotlcm-worker-svr -ldflags "-X main.BuildVersion=${BuildVersion} -X main.CommitID=${CommitID} -X main.BuildTime=${BuildTime}"
|
RUN go build -v -x -o /build/iotlcm-worker-svr -ldflags "-X main.BuildVersion=${BuildVersion} -X main.CommitID=${CommitID} -X main.BuildTime=${BuildTime}"
|
||||||
|
|
||||||
FROM alpine:3.23
|
FROM ${BASE_REGISTRY}/alpine:3.23 AS alpine
|
||||||
|
|
||||||
RUN apk add --no-cache ca-certificates tzdata
|
RUN apk add --no-cache ca-certificates tzdata
|
||||||
|
|
||||||
ENV TZ=Asia/Shanghai
|
ENV TZ=Asia/Shanghai
|
||||||
|
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
COPY --from=builder /build/iotlcm-worker-svr /app/iotlcm-worker-svr
|
COPY --from=builder /build/iotlcm-worker-svr /app/iotlcm-worker-svr
|
||||||
COPY --from=builder /build/build/build_config.yaml /app/config/config.yaml
|
COPY --from=builder /build/build/config.example.yaml /app/config/config.yaml
|
||||||
|
|
||||||
CMD ["./iotlcm-worker-svr"]
|
CMD ["./iotlcm-worker-svr"]
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
# config.yaml 配置文件
|
# config.yaml 配置文件
|
||||||
# @Link https://github.com/xiujiecn/xiu-admin
|
# @Link https://github.com/xiujiecn/xiu-admin
|
||||||
# @Copyright Copyright (c) 2025 LiXiujie
|
# @Copyright Copyright (c) 2025 LiXiujie
|
||||||
# @Author Lxj <li@xiujie.cn>
|
# @Author Lxj <xxx@example.com>
|
||||||
# @License https://github.com/xiujiecn/xiu-admin/blob/master/LICENSE
|
# @License https://github.com/xiujiecn/xiu-admin/blob/master/LICENSE
|
||||||
|
|
||||||
# 系统配置
|
# 系统配置
|
||||||
@@ -40,7 +40,7 @@ database:
|
|||||||
level: "all"
|
level: "all"
|
||||||
stdout: true
|
stdout: true
|
||||||
default: # 分组名称,可自定义,默认为default
|
default: # 分组名称,可自定义,默认为default
|
||||||
link: "mysql:${DB_USER}:${DB_PASSWORD}@tcp(${DB_HOST}:3306)/db_iotlcm?charset=utf8mb4&parseTime=True&loc=Asia%2FShanghai"
|
link: "${DB_DSN}"
|
||||||
debug: true #开启调试模式
|
debug: true #开启调试模式
|
||||||
charset: "utf8mb4" # (可选)数据库编码(如: utf8mb4/utf8/gbk/gb2312),一般设置为utf8mb4。默认为utf8。
|
charset: "utf8mb4" # (可选)数据库编码(如: utf8mb4/utf8/gbk/gb2312),一般设置为utf8mb4。默认为utf8。
|
||||||
dryRun: false #ORM空跑(只读不写)
|
dryRun: false #ORM空跑(只读不写)
|
||||||
@@ -49,7 +49,7 @@ database:
|
|||||||
maxLifetime: 120 #(单位秒)连接对象可重复使用的时间长度
|
maxLifetime: 120 #(单位秒)连接对象可重复使用的时间长度
|
||||||
memorydb:
|
memorydb:
|
||||||
# link: "sqlite::@file(jieiotmem.db)?journal_mode=WAL"
|
# link: "sqlite::@file(jieiotmem.db)?journal_mode=WAL"
|
||||||
link: "sqlite::@file(:memory:)"
|
link: "${DB_DSN}"
|
||||||
# link: ""
|
# link: ""
|
||||||
debug: true
|
debug: true
|
||||||
maxIdle: 1
|
maxIdle: 1
|
||||||
@@ -67,9 +67,9 @@ redis:
|
|||||||
default:
|
default:
|
||||||
mode: "single" #redis模式,支持:single|cluster|sentinel
|
mode: "single" #redis模式,支持:single|cluster|sentinel
|
||||||
sentinelMasterName: "iotlcm-master" #哨兵模式主节点名称
|
sentinelMasterName: "iotlcm-master" #哨兵模式主节点名称
|
||||||
address: 192.0.2.29:6379
|
address: 127.0.0.1:6379
|
||||||
db: 0
|
db: 0
|
||||||
pass: "${REDIS_PASSWORD}"
|
pass: "${SERVICE_PASSWORD}"
|
||||||
minIdle: 5 #最小空闲连接数
|
minIdle: 5 #最小空闲连接数
|
||||||
maxIdle: 30 #允许闲置的最大连接数
|
maxIdle: 30 #允许闲置的最大连接数
|
||||||
maxActive: 5000 #最大活跃连接数
|
maxActive: 5000 #最大活跃连接数
|
||||||
@@ -78,8 +78,8 @@ redis:
|
|||||||
|
|
||||||
|
|
||||||
alipay:
|
alipay:
|
||||||
appId: "${MOBILE_APP_ID}"
|
appId: "${APP_ID}"
|
||||||
privateKey: "MIIEowIBAAKCAQEAg+4BpLO2Fw5GMbrihnugyQqiLBp2RhSbSsvjNyzaG10u900N1DcVlh8HNdcus5AGjfbiUwMQwv7Os+NnFrEZA3CtWLXH+uczHIpcMqJJhoVgA//4gf2TX9K4OSQ1pgJeAhTS9WWMEB7DQLb7A6xGjvBdGNbPAzucGhUmtG1tgZSVOcpWqN2h4dQxkZhfR+6m3AqWAPahpQXsNINHewetQ4mIG+b3+z/OknEll0RDRmw7OihOJ24B51Phy7lewl160LgLpLffyaQUUXcFdcNo+HvWqtIT0mH8WWHAgqfN/WVkMz/WVjE0rzbIVgmpQc7+hMbrcafTmeq0wdnVXrKS6QIDAQABAoIBAEagy1ZuMyRiqueCojTWyHv4altsCzd3Xq+LOWfXZihSGYuhP68FklPc5pJHUaQljazAp+dC891AZk5dhlnak4aI8vRT1OyYi39S/wrIYM5YZs+dYF/O7e/VsnQ800+9YrCAj3N6WqJiparmirkrqFZrjZt3nDmh1u8Ltb3P+1jr90MBEdS+BXaEQP/KGxFiKZyoWFn9B4l0aPFkPExF+hgtRpP3gxFcyljQKGVmSUIkT6V08F1oRm72NQOhcizCh9lX+5HYLTkUVQVg6FHwdZs/KeJcZu6Y829pNSNeJmPbR3VPUpTbvwb0qeEoc42T3H/C0xXUeTYizUpHmhJJqnUCgYEA3Y7Xz+MUk98zRJnlr2BDUc+Qnhv3wmSA5LCcRbxTNNBp4TXjnZknY6Ms9D+nJupL4nw6QSYkO6O+tAEmU6gaSDEHKI1uKqU0zOdbGIHNkMqka3rinQcMJNNRZ8T1FLCPFna7S1FNlxulZgWxjqxP/KvWOEpCWLCLzwZNk8fEd+MCgYEAmHBNbaMmT4WvLevY/OMv/5e5csnDFIcsNJKvFLZZMBLuKT/26YN2ryVUivY+TTyHL5zk4VybeauTdQHIvX6sNJoh2Ii6vthKfSufL4rkMob8PHiG33iHBx3VxAwiVXfu3JHVnbCPcXTOzsdRzT17X3rODK9t8319qfVjd0Gxi8MCgYEAqgNc0+ea+MXD35aAzA0AW5z9inZHruHUjqh/onubFeH//ZOOsHa/5RNoAZ1+tLwBszBEu417LFDAm/xt3rAWSuNkbNfqe8r+M2YtXybsCXmWpetMTdcEmJWX2BspPdEmW8SMgIVJFvVd/GXOFH9hebCa1+MT+l6vShO6H/ahaqUCgYAzon6nr4bPkvHWtTfH2VCt2//mIO+iaXMQz8rcUjP9s7vWXyNMz8oI47UyaNyLnwAPoa5po8vKN9L35EXR4jGEk9z5Er9m267HnVoLaZmIf+66sPzGf3YFdBavlIiguMJBFmjsRB0iijVvF1bfFX9PxTI8W8P7oTtKsaMihy8TrwKBgCgjA8DL8HJbRwZ+V/KWOh8rzO5cBUUH3ThIj6u7Wn4tMlscfi+KFKTXghjSORfIVx8v9v2aNZKwGZ547AOPRXhDF09cLAcCfl7uOrwsftziIeBQnrqtpIqcVTgRwGMIMBJEtQEks5N/E/d+dSL1YQKC76mVC8/RMf52WYKH1QCa"
|
privateKey: "${PRIVATE_KEY}"
|
||||||
alipayPublicKey: "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA8Df1EPFkA2xy8NgF0UdRYxsshslUi2XSGTG01oQKsrYNB+rhAw9MJdMlunVQxSrBZsw5N7htXGdiTceaLo/n8Wuibp0cRHdErJSp9GhP4pW2IR3FoCCPYlJlyiCUR/NInI3XMai0UdjgiPX03bvYLRAKE8lLiOSgxNagQmOFLmQYRYfFhGbxDe98XMg83YdSWvZKrlM+vzgk8/aPgq7BFLpbRKqLvd8OXXBHDCaWMSSGwCN0whhdEiwbIhSVgwTgkhWXUuNDmT741/+7D46djp8wX/84SDts0fWb63/in9ATTGtNivJX7HPDyqrmcN2iUi5xFHS9XwYBdT4yu/xFQwIDAQAB"
|
alipayPublicKey: "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA8Df1EPFkA2xy8NgF0UdRYxsshslUi2XSGTG01oQKsrYNB+rhAw9MJdMlunVQxSrBZsw5N7htXGdiTceaLo/n8Wuibp0cRHdErJSp9GhP4pW2IR3FoCCPYlJlyiCUR/NInI3XMai0UdjgiPX03bvYLRAKE8lLiOSgxNagQmOFLmQYRYfFhGbxDe98XMg83YdSWvZKrlM+vzgk8/aPgq7BFLpbRKqLvd8OXXBHDCaWMSSGwCN0whhdEiwbIhSVgwTgkhWXUuNDmT741/+7D46djp8wX/84SDts0fWb63/in9ATTGtNivJX7HPDyqrmcN2iUi5xFHS9XwYBdT4yu/xFQwIDAQAB"
|
||||||
notifyUrl: "https://evolution-starless-municipal.ngrok-free.dev/api/v1/pay/alipay/notify"
|
notifyUrl: "https://evolution-starless-municipal.ngrok-free.dev/api/v1/pay/alipay/notify"
|
||||||
returnUrl: "http://localhost:5666/enterpriseManagement/tBalanceDetail/tAgentBalance"
|
returnUrl: "http://localhost:5666/enterpriseManagement/tBalanceDetail/tAgentBalance"
|
||||||
@@ -88,9 +88,9 @@ alipay:
|
|||||||
# RabbitMQ 配置(worker 核心依赖)
|
# RabbitMQ 配置(worker 核心依赖)
|
||||||
rabbitmq:
|
rabbitmq:
|
||||||
enabled: true
|
enabled: true
|
||||||
# 示例:amqp://guest:guest@127.0.0.1:5672/
|
# 示例:${SERVICE_DSN}
|
||||||
# dsn: "amqp://admin:admin@127.0.0.1:5672/"
|
# dsn: "${SERVICE_DSN}"
|
||||||
dsn: "amqp://root:sdyz2026@192.0.2.29:5672/"
|
dsn: "${SERVICE_DSN}"
|
||||||
balanceRecharge:
|
balanceRecharge:
|
||||||
retryDelay: "1m" # Alipay 查询失败后的延时重试时间
|
retryDelay: "1m" # Alipay 查询失败后的延时重试时间
|
||||||
maxRetry: 100 # 最大重试次数,超过后投递到 iotlcm.balance.recharge.dlq
|
maxRetry: 100 # 最大重试次数,超过后投递到 iotlcm.balance.recharge.dlq
|
||||||
@@ -135,7 +135,7 @@ trafficUsageFullSync:
|
|||||||
runLockTTL: "5m"
|
runLockTTL: "5m"
|
||||||
consumer:
|
consumer:
|
||||||
enabled: true
|
enabled: true
|
||||||
chanClientName: "192.0.2.29:8110"
|
chanClientName: "127.0.0.1:8110"
|
||||||
chanClientTimeout: "90s"
|
chanClientTimeout: "90s"
|
||||||
lockKey: "flow:consumer:active"
|
lockKey: "flow:consumer:active"
|
||||||
lockTTL: "1m"
|
lockTTL: "1m"
|
||||||
@@ -327,10 +327,10 @@ tsdb:
|
|||||||
database: "TdEngine" #可选择 TdEngine、Influxdb
|
database: "TdEngine" #可选择 TdEngine、Influxdb
|
||||||
tdengine:
|
tdengine:
|
||||||
# type: "taosRestful" #http连接方式,端口是6041
|
# type: "taosRestful" #http连接方式,端口是6041
|
||||||
# dsn: "root:taosdata@http(127.0.0.1:6041)/"
|
# dsn: "${SERVICE_DSN}"
|
||||||
type: "taosWS" #websocket连接方式,端口是6041
|
type: "taosWS" #websocket连接方式,端口是6041
|
||||||
# dsn: "root:taosdata@ws(127.0.0.1:6041)/"
|
# dsn: "${SERVICE_DSN}"
|
||||||
dsn: "root:sdyz2026__@ws(192.0.2.29:6041)/"
|
dsn: "${SERVICE_DSN}"
|
||||||
dbName: "iotlcm"
|
dbName: "iotlcm"
|
||||||
maxOpenConns: 100
|
maxOpenConns: 100
|
||||||
maxIdleConns: 10
|
maxIdleConns: 10
|
||||||
@@ -1,7 +1,10 @@
|
|||||||
|
roleSync:
|
||||||
|
enabled: false
|
||||||
|
|
||||||
# config_default.yaml 配置文件
|
# config_default.yaml 配置文件
|
||||||
# @Link https://github.com/xiujiecn/xiu-admin
|
# @Link https://github.com/xiujiecn/xiu-admin
|
||||||
# @Copyright Copyright (c) 2025 LiXiujie
|
# @Copyright Copyright (c) 2025 LiXiujie
|
||||||
# @Author Lxj <li@xiujie.cn>
|
# @Author Lxj <xxx@example.com>
|
||||||
# @License https://github.com/xiujiecn/xiu-admin/blob/master/LICENSE
|
# @License https://github.com/xiujiecn/xiu-admin/blob/master/LICENSE
|
||||||
|
|
||||||
# 系统配置
|
# 系统配置
|
||||||
@@ -28,8 +31,8 @@ fullSyncOps:
|
|||||||
|
|
||||||
# Alipay credentials are injected by the deployment configuration.
|
# Alipay credentials are injected by the deployment configuration.
|
||||||
alipay:
|
alipay:
|
||||||
appId: ""
|
appId: "${APP_ID}"
|
||||||
privateKey: ""
|
privateKey: "${PRIVATE_KEY}"
|
||||||
alipayPublicKey: ""
|
alipayPublicKey: ""
|
||||||
isProduction: false
|
isProduction: false
|
||||||
|
|
||||||
@@ -347,7 +350,7 @@ database:
|
|||||||
level: "all"
|
level: "all"
|
||||||
stdout: true
|
stdout: true
|
||||||
default: # 分组名称,可自定义,默认为default
|
default: # 分组名称,可自定义,默认为default
|
||||||
link: "mysql:${DB_USER}:${DB_PASSWORD}@tcp(${DB_HOST}:3306)/iotlcm?charset=utf8mb4&parseTime=True&loc=Asia%2FShanghai"
|
link: "${DB_DSN}"
|
||||||
debug: true #开启调试模式
|
debug: true #开启调试模式
|
||||||
charset: "utf8mb4" # (可选)数据库编码(如: utf8mb4/utf8/gbk/gb2312),一般设置为utf8mb4。默认为utf8。
|
charset: "utf8mb4" # (可选)数据库编码(如: utf8mb4/utf8/gbk/gb2312),一般设置为utf8mb4。默认为utf8。
|
||||||
dryRun: false #ORM空跑(只读不写)
|
dryRun: false #ORM空跑(只读不写)
|
||||||
@@ -365,10 +368,10 @@ redis:
|
|||||||
# 单实例配置
|
# 单实例配置
|
||||||
default:
|
default:
|
||||||
mode: "single" #redis模式,支持:single|cluster|sentinel
|
mode: "single" #redis模式,支持:single|cluster|sentinel
|
||||||
sentinelMasterName: "${REDIS_SENTINEL_MASTER}" #哨兵模式主节点名称
|
sentinelMasterName: "xiujie-master" #哨兵模式主节点名称
|
||||||
address: 192.0.2.200:6379
|
address: 127.0.0.1:6379
|
||||||
db: 0
|
db: 0
|
||||||
pass: "${REDIS_PASSWORD}"
|
pass: "${SERVICE_PASSWORD}"
|
||||||
minIdle: 5 #最小空闲连接数
|
minIdle: 5 #最小空闲连接数
|
||||||
maxIdle: 30 #允许闲置的最大连接数
|
maxIdle: 30 #允许闲置的最大连接数
|
||||||
maxActive: 5000 #最大活跃连接数
|
maxActive: 5000 #最大活跃连接数
|
||||||
|
|||||||
@@ -0,0 +1,24 @@
|
|||||||
|
# 角色权限任务切换
|
||||||
|
|
||||||
|
管理后端仅保存角色并事务记录 sys_role_sync_task,Worker 接管执行。任务数据格式和迁移脚本与管理后端一致。
|
||||||
|
|
||||||
|
## 启用顺序
|
||||||
|
|
||||||
|
1. 备份并执行 resource/migrations/20261014_role_sync_outbox.sql;已执行时无需重复迁移。
|
||||||
|
2. 停止全部旧管理后端执行实例,更新后端。不要与旧消费者混跑。
|
||||||
|
3. 核对 Worker 和管理后端使用同环境数据库、RabbitMQ vhost;数据库账号须有角色、菜单、机构、任务及权限版本表的相应读写权限。
|
||||||
|
4. 配置 roleSync.enabled: true,更新 Worker;确认 role sync Worker started 日志及 iotlcm-worker-agent-role-sync 消费者。
|
||||||
|
5. 核对旧主队列、retry、dlq 积压。完整来源的旧消息转为确定性 ID 的数据库任务;不完整消息拒绝处理,不能盲目清空队列。
|
||||||
|
6. 保存成功仅表示来源权限和任务已提交;通过管理后端任务查询与重试接口观察结果。
|
||||||
|
|
||||||
|
## 验收
|
||||||
|
|
||||||
|
使用隔离环境验证正常同步、断开 RabbitMQ、消费者断线重连、Worker 重启、重复消息、部分失败、模板切换、多实例撤权和租约丢失。监控 pending/retry 积压、最老任务时间和终态失败。SQLite 测试不能代替 MySQL 行锁及真实消息队列验收。
|
||||||
|
|
||||||
|
## 回滚
|
||||||
|
|
||||||
|
关闭 Worker 的 roleSync.enabled 前先记录未完成任务;恢复旧后端消费者与停止新消费者要协调,禁止长期双执行。保留任务表和权限版本表,不删除真实角色绑定。未启用执行器期间任务保留 pending,恢复后处理。
|
||||||
|
|
||||||
|
## 检查补充
|
||||||
|
|
||||||
|
无效 JSON 和缺少来源信息的旧消息先确认发布到 iotlcm.agent.role.sync.dlq,再确认原消息;隔离发布失败时原消息重新入队。消息引用不存在的任务只记警告,不无限重投;有效任务载荷必须与任务表的租户及来源字段一致。缺失默认角色直接失败,不自动生成孤立角色,修复绑定须业务确认。权限相关数据库查询使用主库,避免从库延迟将旧权限缓存到新版本。
|
||||||
@@ -7,7 +7,9 @@ require (
|
|||||||
github.com/gogf/gf/contrib/drivers/sqlite/v2 v2.10.0
|
github.com/gogf/gf/contrib/drivers/sqlite/v2 v2.10.0
|
||||||
github.com/gogf/gf/contrib/nosql/redis/v2 v2.10.0
|
github.com/gogf/gf/contrib/nosql/redis/v2 v2.10.0
|
||||||
github.com/gogf/gf/v2 v2.10.0
|
github.com/gogf/gf/v2 v2.10.0
|
||||||
|
github.com/google/uuid v1.6.0
|
||||||
github.com/rabbitmq/amqp091-go v1.10.0
|
github.com/rabbitmq/amqp091-go v1.10.0
|
||||||
|
github.com/redis/go-redis/v9 v9.18.0
|
||||||
github.com/smartwalle/alipay/v3 v3.2.29
|
github.com/smartwalle/alipay/v3 v3.2.29
|
||||||
github.com/taosdata/driver-go/v3 v3.8.0
|
github.com/taosdata/driver-go/v3 v3.8.0
|
||||||
github.com/xuri/excelize/v2 v2.9.0
|
github.com/xuri/excelize/v2 v2.9.0
|
||||||
@@ -27,7 +29,6 @@ require (
|
|||||||
github.com/go-logr/logr v1.4.3 // indirect
|
github.com/go-logr/logr v1.4.3 // indirect
|
||||||
github.com/go-logr/stdr v1.2.2 // indirect
|
github.com/go-logr/stdr v1.2.2 // indirect
|
||||||
github.com/go-sql-driver/mysql v1.7.1 // indirect
|
github.com/go-sql-driver/mysql v1.7.1 // indirect
|
||||||
github.com/google/uuid v1.6.0 // indirect
|
|
||||||
github.com/gorilla/websocket v1.5.3 // indirect
|
github.com/gorilla/websocket v1.5.3 // indirect
|
||||||
github.com/grokify/html-strip-tags-go v0.1.0 // indirect
|
github.com/grokify/html-strip-tags-go v0.1.0 // indirect
|
||||||
github.com/json-iterator/go v1.1.12 // indirect
|
github.com/json-iterator/go v1.1.12 // indirect
|
||||||
@@ -42,7 +43,6 @@ require (
|
|||||||
github.com/olekukonko/ll v0.0.9 // indirect
|
github.com/olekukonko/ll v0.0.9 // indirect
|
||||||
github.com/olekukonko/tablewriter v1.1.0 // indirect
|
github.com/olekukonko/tablewriter v1.1.0 // indirect
|
||||||
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
|
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
|
||||||
github.com/redis/go-redis/v9 v9.18.0 // indirect
|
|
||||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||||
github.com/richardlehane/mscfb v1.0.4 // indirect
|
github.com/richardlehane/mscfb v1.0.4 // indirect
|
||||||
github.com/richardlehane/msoleps v1.0.4 // indirect
|
github.com/richardlehane/msoleps v1.0.4 // indirect
|
||||||
|
|||||||
@@ -0,0 +1,17 @@
|
|||||||
|
# CLI tool, only in development environment.
|
||||||
|
# https://goframe.org/docs/cli
|
||||||
|
gfcli:
|
||||||
|
gen:
|
||||||
|
dao:
|
||||||
|
- link: "${DB_DSN}"
|
||||||
|
tables: "sys_role,sys_role_menu,sys_role_dept,sys_config,sys_user_role,batch_main,batch_detail,sys_dept,sys_oss,t_resource_batch_download,t_sim_card_cancellation,t_sim_card_info,t_sim_flow_pool,t_warn_config,t_warn_log,t_wo_main,t_wo_record,t_message_log,t_user_message"
|
||||||
|
descriptionTag: true
|
||||||
|
noModelComment: true
|
||||||
|
jsonCase: "CamelLower"
|
||||||
|
gJsonSupport: true
|
||||||
|
group: "default"
|
||||||
|
|
||||||
|
docker:
|
||||||
|
build: "-a amd64 -s linux -p temp -ew"
|
||||||
|
tagPrefixes:
|
||||||
|
- iotlcm-worker-svr
|
||||||
@@ -1,17 +0,0 @@
|
|||||||
# CLI tool, only in development environment.
|
|
||||||
# https://goframe.org/docs/cli
|
|
||||||
gfcli:
|
|
||||||
gen:
|
|
||||||
dao:
|
|
||||||
- link: "mysql:${DB_USER}:${DB_PASSWORD}@tcp(${DB_HOST}:3306)/db_iotlcm?charset=utf8mb4&parseTime=True&loc=Asia%2FShanghai"
|
|
||||||
tables: "batch_main,batch_detail,sys_dept,sys_oss,t_resource_batch_download,t_sim_card_cancellation,t_sim_card_info,t_sim_flow_pool,t_warn_config,t_warn_log,t_wo_main,t_wo_record,t_message_log,t_user_message"
|
|
||||||
descriptionTag: true
|
|
||||||
noModelComment: true
|
|
||||||
jsonCase: "CamelLower"
|
|
||||||
gJsonSupport: true
|
|
||||||
group: "default"
|
|
||||||
|
|
||||||
docker:
|
|
||||||
build: "-a amd64 -s linux -p temp -ew"
|
|
||||||
tagPrefixes:
|
|
||||||
- iotlcm-worker-svr
|
|
||||||
@@ -112,6 +112,11 @@ type SelfClaimingProcessingExecutor interface {
|
|||||||
ClaimsProcessingReentry() bool
|
ClaimsProcessingReentry() bool
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// DurableFailureHandler takes ownership before any generic MarkFailed fallback.
|
||||||
|
type DurableFailureHandler interface {
|
||||||
|
ProtectExecutionFailure(ctx context.Context, batchNo string, batchType int, reason string) (bool, error)
|
||||||
|
}
|
||||||
|
|
||||||
type ExecutionFailureHandler interface {
|
type ExecutionFailureHandler interface {
|
||||||
HandleExecutionFailure(ctx context.Context, batchNo string, batchType int, reason string) error
|
HandleExecutionFailure(ctx context.Context, batchNo string, batchType int, reason string) error
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -136,6 +136,64 @@ func (s *repositoryStub) ReclaimProcessing(_ context.Context, batchNo string, ba
|
|||||||
return s.reclaimed, s.reclaimErr
|
return s.reclaimed, s.reclaimErr
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type durableExecutorStub struct {
|
||||||
|
execute func(context.Context) error
|
||||||
|
protectErr error
|
||||||
|
protectCalls int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *durableExecutorStub) Execute(ctx context.Context, _ string) error {
|
||||||
|
if s.execute != nil {
|
||||||
|
return s.execute(ctx)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (s *durableExecutorStub) ProtectExecutionFailure(ctx context.Context, _ string, _ int, _ string) (bool, error) {
|
||||||
|
s.protectCalls++
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return true, ctx.Err()
|
||||||
|
}
|
||||||
|
return true, s.protectErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRuntimeDurableOwnsEveryFailurePath(t *testing.T) {
|
||||||
|
for _, kind := range []string{"error", "deadline-error", "deadline-nil", "failure-reason", "retry-exhausted", "protect-db-down"} {
|
||||||
|
t.Run(kind, func(t *testing.T) {
|
||||||
|
repo := &repositoryStub{mains: []*entity.BatchMain{{BatchNo: "DURABLE", BatchType: Type109, BatchAsyncStatus: AsyncStatusPending}}, claimed: true}
|
||||||
|
exe := &durableExecutorStub{}
|
||||||
|
msg := Message{BatchNo: "DURABLE", BatchType: Type109}
|
||||||
|
switch kind {
|
||||||
|
case "error":
|
||||||
|
exe.execute = func(context.Context) error { return errors.New("执行中断") }
|
||||||
|
case "deadline-error":
|
||||||
|
exe.execute = func(ctx context.Context) error { <-ctx.Done(); return ctx.Err() }
|
||||||
|
case "deadline-nil":
|
||||||
|
exe.execute = func(ctx context.Context) error { <-ctx.Done(); return nil }
|
||||||
|
case "failure-reason", "retry-exhausted":
|
||||||
|
msg.FailureReason = "重试耗尽或失败重投"
|
||||||
|
repo.mains[0].BatchAsyncStatus = AsyncStatusProcessing
|
||||||
|
case "protect-db-down":
|
||||||
|
exe.execute = func(context.Context) error { return errors.New("执行中断") }
|
||||||
|
exe.protectErr = errors.New("数据库不可用")
|
||||||
|
}
|
||||||
|
dispatcher := NewDispatcher(&executorStub{})
|
||||||
|
dispatcher.Register(Type109, exe)
|
||||||
|
err := NewRuntime(repo, dispatcher, WithExecutionTimeout(time.Millisecond)).Handle(context.Background(), msg)
|
||||||
|
if kind == "protect-db-down" {
|
||||||
|
var failure *FailureMarkError
|
||||||
|
if !errors.As(err, &failure) {
|
||||||
|
t.Fatalf("保护失败应可重投: %v", err)
|
||||||
|
}
|
||||||
|
} else if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if exe.protectCalls != 1 || repo.failCalls != 0 {
|
||||||
|
t.Fatalf("通用MarkFailed越过durable保护: protect=%d failed=%d", exe.protectCalls, repo.failCalls)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestMessageJSONContract(t *testing.T) {
|
func TestMessageJSONContract(t *testing.T) {
|
||||||
body, err := json.Marshal(Message{BatchNo: "BATCH-1", BatchType: Type103})
|
body, err := json.Marshal(Message{BatchNo: "BATCH-1", BatchType: Type103})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import (
|
|||||||
"github.com/gogf/gf/v2/errors/gerror"
|
"github.com/gogf/gf/v2/errors/gerror"
|
||||||
"github.com/gogf/gf/v2/frame/g"
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/batchfields"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -84,9 +85,6 @@ func (r *Runtime) Handle(ctx context.Context, message Message) error {
|
|||||||
}
|
}
|
||||||
if main.BatchAsyncStatus != AsyncStatusPending {
|
if main.BatchAsyncStatus != AsyncStatusPending {
|
||||||
if main.BatchAsyncStatus == AsyncStatusProcessing {
|
if main.BatchAsyncStatus == AsyncStatusProcessing {
|
||||||
if message.Retryable {
|
|
||||||
return r.reclaimAndExecute(ctx, message, executor, main)
|
|
||||||
}
|
|
||||||
if processingExecutor, ok := executor.(ProcessingExecutor); ok {
|
if processingExecutor, ok := executor.(ProcessingExecutor); ok {
|
||||||
canExecute, checkErr := processingExecutor.CanExecuteProcessing(ctx, main)
|
canExecute, checkErr := processingExecutor.CanExecuteProcessing(ctx, main)
|
||||||
if checkErr != nil {
|
if checkErr != nil {
|
||||||
@@ -98,6 +96,8 @@ func (r *Runtime) Handle(ctx context.Context, message Message) error {
|
|||||||
}
|
}
|
||||||
return r.reclaimAndExecute(ctx, message, executor, main)
|
return r.reclaimAndExecute(ctx, message, executor, main)
|
||||||
}
|
}
|
||||||
|
} else if message.Retryable {
|
||||||
|
return r.reclaimAndExecute(ctx, message, executor, main)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
logUnclaimable(ctx, main)
|
logUnclaimable(ctx, main)
|
||||||
@@ -168,9 +168,19 @@ func (r *Runtime) execute(ctx context.Context, message Message, executor Executo
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (r *Runtime) markExecutionFailed(ctx context.Context, message Message, executor Executor, reason string) error {
|
func (r *Runtime) markExecutionFailed(ctx context.Context, message Message, executor Executor, reason string) error {
|
||||||
|
reason = batchfields.FailureReason(reason)
|
||||||
finalizationCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), r.finalizationTimeout)
|
finalizationCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), r.finalizationTimeout)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|
||||||
|
if handler, ok := executor.(DurableFailureHandler); ok {
|
||||||
|
handled, err := handler.ProtectExecutionFailure(finalizationCtx, message.BatchNo, message.BatchType, reason)
|
||||||
|
if err != nil {
|
||||||
|
return &FailureMarkError{Reason: reason, Err: err}
|
||||||
|
}
|
||||||
|
if handled {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
failed, markErr := r.repository.MarkFailed(finalizationCtx, message.BatchNo, message.BatchType, reason)
|
failed, markErr := r.repository.MarkFailed(finalizationCtx, message.BatchNo, message.BatchType, reason)
|
||||||
if markErr != nil {
|
if markErr != nil {
|
||||||
return &FailureMarkError{Reason: reason, Err: markErr}
|
return &FailureMarkError{Reason: reason, Err: markErr}
|
||||||
|
|||||||
@@ -0,0 +1,154 @@
|
|||||||
|
package upstream
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/errors/gerror"
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
)
|
||||||
|
|
||||||
|
// apnRequiredSchema 是异步APN提交依赖的表与关键列,对应 20261015 的四份迁移脚本。
|
||||||
|
// 缺表或缺列意味着迁移没有执行完,此时提交必然写不进去,应在发请求前就拒绝。
|
||||||
|
var apnRequiredSchema = map[string][]string{
|
||||||
|
"t_apn_operation": {"operation_id", "chan_key", "request_hash", "response_json", "revision"},
|
||||||
|
"data_operation_parent": {"batch_no", "ready", "finalize_token", "finalize_until"},
|
||||||
|
"data_operation_sub_batch": {"operation_id", "batch_no", "chan_key", "request_data", "status", "poll_count", "lease_token"},
|
||||||
|
"data_operation_sub_detail": {"detail_id", "operation_id", "batch_no", "iccid"},
|
||||||
|
"data_operation_card_claim": {"tenant_id", "iccid", "batch_id", "active_batch_id"},
|
||||||
|
}
|
||||||
|
|
||||||
|
// apnReadinessTTL 是依赖检查结果的缓存时间:迁移完成后最迟一分钟内自动放行,无需重启。
|
||||||
|
const apnReadinessTTL = time.Minute
|
||||||
|
|
||||||
|
// apnReadinessVerifyTimeout 是单次结构校验的独立超时,与调用方上下文解耦。
|
||||||
|
const apnReadinessVerifyTimeout = 5 * time.Second
|
||||||
|
|
||||||
|
type apnReadinessCache struct {
|
||||||
|
ttl time.Duration
|
||||||
|
mu sync.Mutex
|
||||||
|
checkedAt time.Time
|
||||||
|
err error
|
||||||
|
// verify 默认指向 verifyAPNSchema;独立出来是为了让缓存行为可测。
|
||||||
|
verify func(context.Context) error
|
||||||
|
}
|
||||||
|
|
||||||
|
var defaultAPNReadiness = &apnReadinessCache{ttl: apnReadinessTTL, verify: verifyAPNSchema}
|
||||||
|
|
||||||
|
// check 返回异步APN提交的依赖状态,结果按 TTL 缓存,避免每次提交都查库。
|
||||||
|
func (c *apnReadinessCache) check(ctx context.Context) error {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
|
if !c.checkedAt.IsZero() && time.Since(c.checkedAt) < c.ttl {
|
||||||
|
return c.err
|
||||||
|
}
|
||||||
|
verify := c.verify
|
||||||
|
if verify == nil {
|
||||||
|
verify = verifyAPNSchema
|
||||||
|
}
|
||||||
|
// 校验使用独立超时,不受调用方取消影响;否则一次取消就会把失败结果缓存给全进程。
|
||||||
|
verifyCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), apnReadinessVerifyTimeout)
|
||||||
|
defer cancel()
|
||||||
|
c.err = verify(verifyCtx)
|
||||||
|
if c.err != nil && (errors.Is(c.err, context.Canceled) || errors.Is(c.err, context.DeadlineExceeded)) {
|
||||||
|
// 只是本次校验没跑完,不代表依赖缺失,不写入缓存。
|
||||||
|
c.checkedAt = time.Time{}
|
||||||
|
return c.err
|
||||||
|
}
|
||||||
|
c.checkedAt = time.Now()
|
||||||
|
return c.err
|
||||||
|
}
|
||||||
|
|
||||||
|
// verifyAPNSchema 直接查询库里的实际表结构,而不是依赖人工开开关确认「迁移已执行」。
|
||||||
|
func verifyAPNSchema(ctx context.Context) (err error) {
|
||||||
|
if !databaseConfigured(ctx) {
|
||||||
|
// 未配置数据库连接(测试桩、纯传输层用例)时跳过;生产 worker 不存在该情况。
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
// g.DB() 在缺配置时会 panic,这里统一转成可读错误,避免拖垮提交链路。
|
||||||
|
defer func() {
|
||||||
|
if recovered := recover(); recovered != nil {
|
||||||
|
err = gerror.Newf("无法获取数据库连接,不能确认异步APN依赖是否就绪:%v", recovered)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
db := g.DB()
|
||||||
|
if db == nil {
|
||||||
|
return gerror.New("无法获取数据库连接,不能确认异步APN依赖是否就绪")
|
||||||
|
}
|
||||||
|
config := db.GetConfig()
|
||||||
|
if strings.EqualFold(config.Type, "sqlite") {
|
||||||
|
// 本地/单元测试使用 SQLite,没有 information_schema,跳过结构检查。
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
names := make([]string, 0, len(apnRequiredSchema))
|
||||||
|
for name := range apnRequiredSchema {
|
||||||
|
names = append(names, name)
|
||||||
|
}
|
||||||
|
sort.Strings(names)
|
||||||
|
placeholders := make([]string, 0, len(names))
|
||||||
|
args := make([]interface{}, 0, len(names))
|
||||||
|
for _, name := range names {
|
||||||
|
placeholders = append(placeholders, "?")
|
||||||
|
args = append(args, name)
|
||||||
|
}
|
||||||
|
records, err := db.GetAll(ctx, fmt.Sprintf(
|
||||||
|
"SELECT TABLE_NAME AS table_name, COLUMN_NAME AS column_name FROM information_schema.columns WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME IN (%s)",
|
||||||
|
strings.Join(placeholders, ",")), args...)
|
||||||
|
if err != nil {
|
||||||
|
return gerror.Wrap(err, "查询异步APN依赖表结构失败")
|
||||||
|
}
|
||||||
|
existing := make(map[string]map[string]bool, len(records))
|
||||||
|
for _, record := range records {
|
||||||
|
table := record["table_name"].String()
|
||||||
|
if existing[table] == nil {
|
||||||
|
existing[table] = make(map[string]bool)
|
||||||
|
}
|
||||||
|
existing[table][strings.ToLower(record["column_name"].String())] = true
|
||||||
|
}
|
||||||
|
return validateAPNSchema(existing)
|
||||||
|
}
|
||||||
|
|
||||||
|
// databaseConfigured 判断当前配置里是否声明了数据库,用于区分测试桩与真实运行环境。
|
||||||
|
func databaseConfigured(ctx context.Context) bool {
|
||||||
|
value := g.Cfg().MustGet(ctx, "database")
|
||||||
|
return value != nil && !value.IsNil() && !value.IsEmpty()
|
||||||
|
}
|
||||||
|
|
||||||
|
// validateAPNSchema 比对必需的表与列,返回可直接据此执行迁移的错误信息。
|
||||||
|
func validateAPNSchema(existing map[string]map[string]bool) error {
|
||||||
|
tables := make([]string, 0, len(apnRequiredSchema))
|
||||||
|
for table := range apnRequiredSchema {
|
||||||
|
tables = append(tables, table)
|
||||||
|
}
|
||||||
|
sort.Strings(tables)
|
||||||
|
var missingTables []string
|
||||||
|
var missingColumns []string
|
||||||
|
for _, table := range tables {
|
||||||
|
columns, ok := existing[table]
|
||||||
|
if !ok {
|
||||||
|
missingTables = append(missingTables, table)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
required := append([]string(nil), apnRequiredSchema[table]...)
|
||||||
|
sort.Strings(required)
|
||||||
|
for _, column := range required {
|
||||||
|
if !columns[column] {
|
||||||
|
missingColumns = append(missingColumns, table+"."+column)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case len(missingTables) > 0:
|
||||||
|
return gerror.Newf("异步APN依赖未就绪:缺少数据表 %s,请先执行 20261015 迁移脚本",
|
||||||
|
strings.Join(missingTables, "、"))
|
||||||
|
case len(missingColumns) > 0:
|
||||||
|
return gerror.Newf("异步APN依赖未就绪:缺少字段 %s,请先执行 20261015 迁移脚本",
|
||||||
|
strings.Join(missingColumns, "、"))
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,138 @@
|
|||||||
|
package upstream
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
"github.com/gogf/gf/v2/os/gcfg"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
func fullAPNSchema() map[string]map[string]bool {
|
||||||
|
schema := make(map[string]map[string]bool, len(apnRequiredSchema))
|
||||||
|
for table, columns := range apnRequiredSchema {
|
||||||
|
schema[table] = make(map[string]bool, len(columns))
|
||||||
|
for _, column := range columns {
|
||||||
|
schema[table][column] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return schema
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestValidateAPNSchemaRequiresAllTablesAndColumns 固化依赖判据:
|
||||||
|
// 缺表、缺列都必须给出可据此执行迁移的明确原因,而不是笼统的「未就绪」。
|
||||||
|
func TestValidateAPNSchemaRequiresAllTablesAndColumns(t *testing.T) {
|
||||||
|
if err := validateAPNSchema(fullAPNSchema()); err != nil {
|
||||||
|
t.Fatalf("完整结构不应报错:%v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
missingTable := fullAPNSchema()
|
||||||
|
delete(missingTable, "data_operation_card_claim")
|
||||||
|
err := validateAPNSchema(missingTable)
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "data_operation_card_claim") || !strings.Contains(err.Error(), "缺少数据表") {
|
||||||
|
t.Fatalf("缺表未被识别:%v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
missingColumn := fullAPNSchema()
|
||||||
|
delete(missingColumn["t_apn_operation"], "revision")
|
||||||
|
err = validateAPNSchema(missingColumn)
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "t_apn_operation.revision") || !strings.Contains(err.Error(), "缺少字段") {
|
||||||
|
t.Fatalf("缺列未被识别:%v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// 表全部缺失时优先报缺表,避免一次报几十个字段干扰排查。
|
||||||
|
err = validateAPNSchema(map[string]map[string]bool{})
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "缺少数据表") {
|
||||||
|
t.Fatalf("空库应优先报缺表:%v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestAsyncApnSubmitBlockedWhenDependencyNotReady 验证依赖未就绪时在发请求前就拒绝,
|
||||||
|
// 并把错误标记为「未发送」,避免上游误判为已办理。
|
||||||
|
func TestAsyncApnSubmitBlockedWhenDependencyNotReady(t *testing.T) {
|
||||||
|
oldAdapter := g.Cfg().GetAdapter()
|
||||||
|
defer g.Cfg().SetAdapter(oldAdapter)
|
||||||
|
oldReady := apnDependencyReady
|
||||||
|
defer func() { apnDependencyReady = oldReady }()
|
||||||
|
apnDependencyReady = func(ctx context.Context) error {
|
||||||
|
return errors.New("缺少数据表 data_operation_card_claim")
|
||||||
|
}
|
||||||
|
|
||||||
|
probes, submits := 0, 0
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.URL.Path == "/sim/query/sim-apn-capability" {
|
||||||
|
probes++
|
||||||
|
return
|
||||||
|
}
|
||||||
|
submits++
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
adapter, err := gcfg.NewAdapterContent(fmt.Sprintf(`{"upstream":{"proxy":{"address":%q,"timeout":"1s"}}}`, strings.TrimPrefix(server.URL, "http://")))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
g.Cfg().SetAdapter(adapter)
|
||||||
|
|
||||||
|
_, err = New().SubmitSimApnFunction(context.Background(), &model.SimCardApnChangeInput{
|
||||||
|
SimKeys: []*model.SimKey{{ChanKey: "test", ICCID: "fake-card"}},
|
||||||
|
ReqData: model.SimCardApnChangeInputItem{OperationID: "fake-operation"},
|
||||||
|
})
|
||||||
|
var notSent interface{ APNRequestNotSent() bool }
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "依赖未就绪") {
|
||||||
|
t.Fatalf("依赖未就绪应拒绝提交:%v", err)
|
||||||
|
}
|
||||||
|
if !errors.As(err, ¬Sent) || !notSent.APNRequestNotSent() {
|
||||||
|
t.Fatalf("依赖拒绝必须标记为未发送:%v", err)
|
||||||
|
}
|
||||||
|
if probes != 0 || submits != 0 {
|
||||||
|
t.Fatalf("依赖未就绪时不应发出任何请求:probes=%d submits=%d", probes, submits)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestReadinessCacheDoesNotCacheCanceledVerification 覆盖审查发现的场景:
|
||||||
|
// 调用方取消导致的校验失败不能进进程级缓存,否则会误挡后续所有提交。
|
||||||
|
func TestReadinessCacheDoesNotCacheCanceledVerification(t *testing.T) {
|
||||||
|
cache := &apnReadinessCache{ttl: time.Minute}
|
||||||
|
calls := 0
|
||||||
|
cache.verify = func(ctx context.Context) error {
|
||||||
|
calls++
|
||||||
|
if calls == 1 {
|
||||||
|
return context.Canceled
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err := cache.check(context.Background()); !errors.Is(err, context.Canceled) {
|
||||||
|
t.Fatalf("首次应返回取消错误:%v", err)
|
||||||
|
}
|
||||||
|
if err := cache.check(context.Background()); err != nil {
|
||||||
|
t.Fatalf("取消不应被缓存,第二次应重新校验并放行:%v", err)
|
||||||
|
}
|
||||||
|
if calls != 2 {
|
||||||
|
t.Fatalf("取消后应重新校验,实际调用%d次", calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestReadinessCacheReusesResult 保证依赖检查不会每次提交都查库,同时迁移完成后能自动恢复。
|
||||||
|
func TestReadinessCacheReusesResult(t *testing.T) {
|
||||||
|
cache := &apnReadinessCache{ttl: time.Minute}
|
||||||
|
calls := 0
|
||||||
|
cache.verify = func(ctx context.Context) error {
|
||||||
|
calls++
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
for i := 0; i < 3; i++ {
|
||||||
|
if err := cache.check(context.Background()); err != nil {
|
||||||
|
t.Fatalf("第%d次检查报错:%v", i+1, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if calls != 1 {
|
||||||
|
t.Fatalf("缓存期内应只查一次库,实际%d次", calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -19,6 +19,22 @@ import (
|
|||||||
const inboundBasicInfoClientNameEnv = "UPSTREAM_INBOUND_BASIC_INFO_CLIENT_NAME"
|
const inboundBasicInfoClientNameEnv = "UPSTREAM_INBOUND_BASIC_INFO_CLIENT_NAME"
|
||||||
const upstreamProxyMaxTimeoutConfigKey = "upstream.proxy.maxTimeout"
|
const upstreamProxyMaxTimeoutConfigKey = "upstream.proxy.maxTimeout"
|
||||||
|
|
||||||
|
// apnAsyncProtocolVersionConfigKey 是异步APN提交的兼容开关。
|
||||||
|
// 默认值改为开启:常规发布不再需要人工逐环境开开关,兼容性由提交前的协议能力探测兜底。
|
||||||
|
// 只有需要临时回退到旧协议时,才显式配为 0。
|
||||||
|
const apnAsyncProtocolVersionConfigKey = "upstream.proxy.apnAsyncProtocolVersion"
|
||||||
|
|
||||||
|
// defaultApnAsyncProtocolVersion 默认开启异步APN提交。
|
||||||
|
const defaultApnAsyncProtocolVersion = 1
|
||||||
|
|
||||||
|
// defaultUpstreamProxyMaxTimeout 是单次批量转发的兜底上限。
|
||||||
|
// 必须大于 upstream-proxy 的 gateway.batchTimeout(默认 20 分钟),
|
||||||
|
// 让超时由更靠近通道的一层先判定,避免 worker 提前放弃而通道仍在办理,造成结果未知。
|
||||||
|
const defaultUpstreamProxyMaxTimeout = 25 * time.Minute
|
||||||
|
|
||||||
|
// defaultUpstreamProxyTimeout 是未配置单卡预算时的默认值。
|
||||||
|
const defaultUpstreamProxyTimeout = 10 * time.Second
|
||||||
|
|
||||||
type Client struct {
|
type Client struct {
|
||||||
currentUsageClientName string
|
currentUsageClientName string
|
||||||
currentUsageTimeout time.Duration
|
currentUsageTimeout time.Duration
|
||||||
@@ -134,6 +150,106 @@ func (c *Client) ChangeSimApnFunction(ctx context.Context, input *model.SimCardA
|
|||||||
return requestAll[model.SimCardApnChangeOutputItem](ctx, "/sim/change/sim-apn", input, len(input.SimKeys))
|
return requestAll[model.SimCardApnChangeOutputItem](ctx, "/sim/change/sim-apn", input, len(input.SimKeys))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// apnRequestNotSentError 仅描述任何APN写请求之前的能力拒绝。
|
||||||
|
// 提交开始后的通信错误绝不能使用此类型。
|
||||||
|
type apnRequestNotSentError struct{ cause error }
|
||||||
|
|
||||||
|
func (e *apnRequestNotSentError) Error() string { return e.cause.Error() }
|
||||||
|
func (e *apnRequestNotSentError) Unwrap() error { return e.cause }
|
||||||
|
func (e *apnRequestNotSentError) APNRequestNotSent() bool { return true }
|
||||||
|
|
||||||
|
func (c *Client) CheckSimApnAsyncCapability(ctx context.Context, input *model.SimCardApnChangeInput) (err error) {
|
||||||
|
defer func() {
|
||||||
|
if err != nil {
|
||||||
|
err = &apnRequestNotSentError{cause: err}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
if !apnAsyncSubmitEnabled(ctx) {
|
||||||
|
return gerror.New("异步APN提交已被显式关闭(upstream.proxy.apnAsyncProtocolVersion=0);清除该配置或置为1后重试")
|
||||||
|
}
|
||||||
|
if err := apnDependencyReady(ctx); err != nil {
|
||||||
|
return gerror.Wrap(err, "异步APN依赖未就绪,未发送办理请求")
|
||||||
|
}
|
||||||
|
if input == nil || strings.TrimSpace(input.ReqData.OperationID) == "" {
|
||||||
|
return gerror.New("异步APN提交能力探测缺少operationId")
|
||||||
|
}
|
||||||
|
return c.checkSimApnProtocol(ctx, input)
|
||||||
|
}
|
||||||
|
|
||||||
|
// apnDependencyReady 检查本地真实依赖(迁移表结构);测试可替换以便固定结论。
|
||||||
|
var apnDependencyReady = func(ctx context.Context) error {
|
||||||
|
return defaultAPNReadiness.check(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
// apnAsyncSubmitEnabled 判断应急总闸是否被显式关闭。
|
||||||
|
// 默认放行:是否可提交由程序自行检查真实依赖决定——
|
||||||
|
// - 本地依赖:apn_readiness 核对迁移表结构是否就绪;
|
||||||
|
// - 远端兼容:每次提交前的协议能力探测,旧版 chan/proxy 会明确拒绝而不是按同步语义执行。
|
||||||
|
// 因此常规发布不再需要人工逐环境开开关;仅回退到旧协议时把配置显式设为 0。
|
||||||
|
func apnAsyncSubmitEnabled(ctx context.Context) bool {
|
||||||
|
return g.Cfg().MustGet(ctx, apnAsyncProtocolVersionConfigKey, defaultApnAsyncProtocolVersion).Int() == 1
|
||||||
|
}
|
||||||
|
|
||||||
|
// 仅验证远端协议:已有受理映射的只读补查不受新增提交门禁控制。
|
||||||
|
// 旧版代理/通道缺少能力接口时必须拒绝,不能把查询当作普通办理转发。
|
||||||
|
func (c *Client) checkSimApnProtocol(ctx context.Context, input *model.SimCardApnChangeInput) error {
|
||||||
|
if len(input.SimKeys) == 0 || len(input.SimKeys) > 100 || (strings.TrimSpace(input.ReqData.OperationID) == "" && strings.TrimSpace(input.ReqData.QueryJobID) == "") {
|
||||||
|
return gerror.New("异步APN能力探测需要operationId和1至100张卡快照")
|
||||||
|
}
|
||||||
|
type capability struct {
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
}
|
||||||
|
probeCtx, cancel := context.WithTimeout(ctx, 10*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
items, err := requestAll[capability](probeCtx, "/sim/query/sim-apn-capability", input, len(input.SimKeys))
|
||||||
|
if err != nil {
|
||||||
|
return gerror.Wrap(err, "异步APN协议能力探测失败,未发送办理或恢复请求")
|
||||||
|
}
|
||||||
|
if len(items) != len(input.SimKeys) {
|
||||||
|
return gerror.New("异步APN协议能力响应卡数不匹配,禁止发送办理请求")
|
||||||
|
}
|
||||||
|
seen := make(map[model.SimKey]bool)
|
||||||
|
for _, item := range items {
|
||||||
|
if item.SimKey == nil || item.Code != 0 || item.ResData.ProtocolVersion != 1 || seen[*item.SimKey] {
|
||||||
|
return gerror.New("通道未就绪或不支持异步APN协议,禁止发送办理请求")
|
||||||
|
}
|
||||||
|
seen[*item.SimKey] = true
|
||||||
|
}
|
||||||
|
for _, key := range input.SimKeys {
|
||||||
|
if key == nil || !seen[*key] {
|
||||||
|
return gerror.New("异步APN协议能力响应卡关联不匹配")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// SubmitSimApnFunction 仅提交异步批量任务;受理状态不能按办理成功回写。
|
||||||
|
func (c *Client) SubmitSimApnFunction(ctx context.Context, input *model.SimCardApnChangeInput) ([]model.SimBaseRes[model.SimCardApnChangeOutputItem], error) {
|
||||||
|
if err := c.CheckSimApnAsyncCapability(ctx, input); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
request := *input
|
||||||
|
request.ReqData.SubmitOnly = true
|
||||||
|
request.ReqData.QueryJobID = ""
|
||||||
|
request.ReqData.RecoverOnly = false
|
||||||
|
return c.ChangeSimApnFunction(ctx, &request)
|
||||||
|
}
|
||||||
|
|
||||||
|
// QuerySimApnFunctionResult 根据已持久化的jobId只查一次,不重新提交。
|
||||||
|
func (c *Client) QuerySimApnFunctionResult(ctx context.Context, input *model.SimCardApnChangeInput) ([]model.SimBaseRes[model.SimCardApnChangeOutputItem], error) {
|
||||||
|
if strings.TrimSpace(input.ReqData.QueryJobID) == "" && strings.TrimSpace(input.ReqData.OperationID) == "" {
|
||||||
|
return nil, gerror.New("异步APN结果查询缺少jobId和operationId")
|
||||||
|
}
|
||||||
|
request := *input
|
||||||
|
request.ReqData.QueryJobID = strings.TrimSpace(request.ReqData.QueryJobID)
|
||||||
|
request.ReqData.SubmitOnly = false
|
||||||
|
request.ReqData.RecoverOnly = request.ReqData.QueryJobID == ""
|
||||||
|
if err := c.checkSimApnProtocol(ctx, &request); err != nil {
|
||||||
|
return nil, gerror.Wrap(err, "异步APN只读查询协议未就绪,未发送查询请求")
|
||||||
|
}
|
||||||
|
return c.ChangeSimApnFunction(ctx, &request)
|
||||||
|
}
|
||||||
|
|
||||||
func request[T any](ctx context.Context, path string, input any, itemCount int) ([]model.SimBaseRes[T], error) {
|
func request[T any](ctx context.Context, path string, input any, itemCount int) ([]model.SimBaseRes[T], error) {
|
||||||
items, err := requestAll[T](ctx, path, input, itemCount)
|
items, err := requestAll[T](ctx, path, input, itemCount)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -173,19 +289,13 @@ func requestAllWithAddressTimeout[T any](ctx context.Context, path string, input
|
|||||||
}
|
}
|
||||||
|
|
||||||
client := g.Client()
|
client := g.Client()
|
||||||
timeout := timeoutOverride
|
if path == "/sim/change/sim-apn" || path == "/sim/query/sim-apn-capability" {
|
||||||
if timeout <= 0 {
|
client.SetContentType("application/json")
|
||||||
timeout = config.Timeout
|
|
||||||
}
|
|
||||||
if timeout <= 0 {
|
|
||||||
timeout = 10 * time.Second
|
|
||||||
}
|
|
||||||
if itemCount > 1 {
|
|
||||||
timeout *= time.Duration(itemCount)
|
|
||||||
}
|
|
||||||
if maxTimeout := g.Cfg().MustGet(ctx, upstreamProxyMaxTimeoutConfigKey, 2*time.Minute).Duration(); maxTimeout > 0 && timeout > maxTimeout {
|
|
||||||
timeout = maxTimeout
|
|
||||||
}
|
}
|
||||||
|
timeout := resolveUpstreamTimeout(path, itemCount, config.Timeout,
|
||||||
|
timeoutOverride,
|
||||||
|
configDuration(ctx, upstreamProxyMaxTimeoutConfigKey, defaultUpstreamProxyMaxTimeout),
|
||||||
|
configDuration(ctx, "upstream.proxy.dataServiceTimeout", 240*time.Second))
|
||||||
setClientTimeout(client, timeout)
|
setClientTimeout(client, timeout)
|
||||||
clientResponse, err := client.Post(ctx, fmt.Sprintf("http://%s%s", address, path), input)
|
clientResponse, err := client.Post(ctx, fmt.Sprintf("http://%s%s", address, path), input)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -239,3 +349,45 @@ func setClientTimeout(client *gclient.Client, timeout time.Duration) {
|
|||||||
transport.ResponseHeaderTimeout = timeout
|
transport.ResponseHeaderTimeout = timeout
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// resolveUpstreamTimeout 计算单次转发预算:
|
||||||
|
// 普通批量按「单卡预算 × 卡数」放大,并以 maxTimeout 兜底,避免批量操作被默认 2 分钟上限提前判失败;
|
||||||
|
// 数据开停沿用独立预算,优先于按卡计时。
|
||||||
|
func resolveUpstreamTimeout(path string, itemCount int, perCard, timeoutOverride, maxTimeout, dataServiceTimeout time.Duration) time.Duration {
|
||||||
|
if itemCount < 1 {
|
||||||
|
itemCount = 1
|
||||||
|
}
|
||||||
|
timeout := timeoutOverride
|
||||||
|
if timeout <= 0 {
|
||||||
|
timeout = perCard
|
||||||
|
}
|
||||||
|
if timeout <= 0 {
|
||||||
|
timeout = defaultUpstreamProxyTimeout
|
||||||
|
}
|
||||||
|
timeout *= time.Duration(itemCount)
|
||||||
|
if path == "/sim/change/sim-apn" {
|
||||||
|
if dataServiceTimeout > 0 {
|
||||||
|
return dataServiceTimeout
|
||||||
|
}
|
||||||
|
return 240 * time.Second
|
||||||
|
}
|
||||||
|
if maxTimeout > 0 && timeout > maxTimeout {
|
||||||
|
timeout = maxTimeout
|
||||||
|
}
|
||||||
|
return timeout
|
||||||
|
}
|
||||||
|
|
||||||
|
// configDuration 读取时长配置,兼容写成 "20m" 字符串或秒数整数两种形式。
|
||||||
|
func configDuration(ctx context.Context, key string, fallback time.Duration) time.Duration {
|
||||||
|
value := g.Cfg().MustGet(ctx, key)
|
||||||
|
if value == nil || value.IsNil() {
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
if duration := value.Duration(); duration >= time.Second {
|
||||||
|
return duration
|
||||||
|
}
|
||||||
|
if seconds := value.Int(); seconds > 0 {
|
||||||
|
return time.Duration(seconds) * time.Second
|
||||||
|
}
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
|||||||
@@ -1,16 +1,399 @@
|
|||||||
package upstream
|
package upstream
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"bufio"
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
"os"
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
"strings"
|
"strings"
|
||||||
|
"sync"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
"github.com/gogf/gf/v2/os/gcfg"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
func startAPNHTTPChainHelper(t *testing.T, module string, extraEnv ...string) string {
|
||||||
|
t.Helper()
|
||||||
|
root, err := filepath.Abs(filepath.Join("..", "..", "..", "..", module))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
buildCtx, buildCancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||||
|
defer buildCancel()
|
||||||
|
binary := filepath.Join(t.TempDir(), module+"-http-test.exe")
|
||||||
|
build := exec.CommandContext(buildCtx, "go", "-C", root, "test", "-work", "-c", "-o", binary, "./internal/cmd")
|
||||||
|
build.Env = append(os.Environ(), "GOTMPDIR="+t.TempDir())
|
||||||
|
if output, err := build.CombinedOutput(); err != nil {
|
||||||
|
t.Fatalf("%s模拟进程编译失败:%v\n%s", module, err, output)
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 170*time.Second)
|
||||||
|
cmd := exec.CommandContext(ctx, binary, "-test.run=^TestAPNHTTPHarness$", "-test.v", "-test.count=1")
|
||||||
|
cmd.Dir = filepath.Join(root, "internal", "cmd")
|
||||||
|
cmd.Env = append(append(os.Environ(), "APN_HTTP_CHAIN_HELPER=1"), extraEnv...)
|
||||||
|
stdout, err := cmd.StdoutPipe()
|
||||||
|
if err != nil {
|
||||||
|
cancel()
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cmd.Stderr = cmd.Stdout
|
||||||
|
if err := cmd.Start(); err != nil {
|
||||||
|
cancel()
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ready := make(chan string, 1)
|
||||||
|
var mu sync.Mutex
|
||||||
|
var log strings.Builder
|
||||||
|
go func() {
|
||||||
|
scanner := bufio.NewScanner(stdout)
|
||||||
|
for scanner.Scan() {
|
||||||
|
line := scanner.Text()
|
||||||
|
mu.Lock()
|
||||||
|
log.WriteString(line + "\n")
|
||||||
|
mu.Unlock()
|
||||||
|
if strings.HasPrefix(line, "APN_CHAIN_READY=") {
|
||||||
|
ready <- strings.TrimPrefix(line, "APN_CHAIN_READY=")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
done := make(chan error, 1)
|
||||||
|
go func() { done <- cmd.Wait() }()
|
||||||
|
var address string
|
||||||
|
select {
|
||||||
|
case address = <-ready:
|
||||||
|
case err := <-done:
|
||||||
|
cancel()
|
||||||
|
mu.Lock()
|
||||||
|
text := log.String()
|
||||||
|
mu.Unlock()
|
||||||
|
t.Fatalf("%s模拟进程启动即退出:%v\n%s", module, err, text)
|
||||||
|
case <-ctx.Done():
|
||||||
|
cancel()
|
||||||
|
<-done
|
||||||
|
mu.Lock()
|
||||||
|
text := log.String()
|
||||||
|
mu.Unlock()
|
||||||
|
t.Fatalf("%s模拟进程启动失败:%s", module, text)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
requestCtx, stop := context.WithTimeout(context.Background(), 5*time.Second)
|
||||||
|
defer stop()
|
||||||
|
req, _ := http.NewRequestWithContext(requestCtx, http.MethodPost, address+"/__test/stop", nil)
|
||||||
|
if res, err := http.DefaultClient.Do(req); err == nil {
|
||||||
|
_ = res.Body.Close()
|
||||||
|
if res.StatusCode != http.StatusOK {
|
||||||
|
t.Errorf("%s停止请求失败:%d", module, res.StatusCode)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
t.Errorf("%s停止请求失败:%v", module, err)
|
||||||
|
}
|
||||||
|
if err := <-done; err != nil {
|
||||||
|
mu.Lock()
|
||||||
|
text := log.String()
|
||||||
|
mu.Unlock()
|
||||||
|
t.Errorf("%s模拟进程异常退出:%v\n%s", module, err, text)
|
||||||
|
} else {
|
||||||
|
t.Logf("%s helper正常shutdown并退出0", module)
|
||||||
|
}
|
||||||
|
cancel()
|
||||||
|
})
|
||||||
|
return address
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAPNHTTPChainWorkerClientProxyChanCarrier(t *testing.T) {
|
||||||
|
if os.Getenv("APN_HTTP_CHAIN_INTEGRATION") != "1" {
|
||||||
|
t.Skip("未启用本机跨模块HTTP及专用MySQL模拟链")
|
||||||
|
}
|
||||||
|
chanURL := startAPNHTTPChainHelper(t, "iotlcm-upstream-chan")
|
||||||
|
proxyURL := startAPNHTTPChainHelper(t, "iotlcm-upstream-proxy", "APN_HTTP_CHAIN_CHAN="+chanURL)
|
||||||
|
oldAdapter := g.Cfg().GetAdapter()
|
||||||
|
defer g.Cfg().SetAdapter(oldAdapter)
|
||||||
|
adapter, err := gcfg.NewAdapterContent(fmt.Sprintf(`{"upstream":{"proxy":{"address":%q,"timeout":"2s","apnAsyncProtocolVersion":1}}}`, strings.TrimPrefix(proxyURL, "http://")))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
g.Cfg().SetAdapter(adapter)
|
||||||
|
client := New()
|
||||||
|
expectedWrites := map[string]int{}
|
||||||
|
expectedQueries := map[string]int{}
|
||||||
|
expectedSubmits := 0
|
||||||
|
for _, scenario := range []string{"drop-response", "top-error", "lost-response", "failed-task"} {
|
||||||
|
t.Run(scenario, func(t *testing.T) {
|
||||||
|
input := &model.SimCardApnChangeInput{ReqData: model.SimCardApnChangeInputItem{OperationID: fmt.Sprintf("http-chain-%s-%d", scenario, time.Now().UnixNano()), OperType: "1", ApnName: "CMIOT"}}
|
||||||
|
prefix := scenario + "-"
|
||||||
|
if scenario == "lost-response" {
|
||||||
|
prefix = "lost-"
|
||||||
|
}
|
||||||
|
if scenario == "failed-task" {
|
||||||
|
prefix = "failed-"
|
||||||
|
}
|
||||||
|
for i := 0; i < 10; i++ {
|
||||||
|
input.SimKeys = append(input.SimKeys, &model.SimKey{ChanKey: "fake-channel", ICCID: fmt.Sprintf("fake-card-%s-%d", scenario, i), MSISDN: fmt.Sprintf("%s%d", prefix, i)})
|
||||||
|
}
|
||||||
|
out, err := client.SubmitSimApnFunction(context.Background(), input)
|
||||||
|
if err != nil || len(out) != 10 {
|
||||||
|
t.Fatalf("首发结果无效:%+v %v", out, err)
|
||||||
|
}
|
||||||
|
for _, item := range out {
|
||||||
|
if scenario != "failed-task" && !item.ResData.Uncertain {
|
||||||
|
t.Fatalf("丢响应被确认:%+v", item)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
res, err := http.Post(chanURL+"/__test/restart", "application/json", nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
_ = res.Body.Close()
|
||||||
|
recovered, err := client.QuerySimApnFunctionResult(context.Background(), input)
|
||||||
|
if err != nil || len(recovered) != 10 {
|
||||||
|
t.Fatalf("恢复失败:%v", err)
|
||||||
|
}
|
||||||
|
duplicate, err := client.SubmitSimApnFunction(context.Background(), input)
|
||||||
|
if err != nil || len(duplicate) != 10 {
|
||||||
|
t.Fatalf("重复恢复失败:%v", err)
|
||||||
|
}
|
||||||
|
if scenario == "lost-response" {
|
||||||
|
for i, item := range recovered {
|
||||||
|
if !item.ResData.Uncertain || item.ResData.OrderNum != "" || !duplicate[i].ResData.Uncertain {
|
||||||
|
t.Fatalf("无jobId必须未知:%+v", item)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
job := recovered[0].ResData.OrderNum
|
||||||
|
for i, item := range recovered {
|
||||||
|
if job == "" || item.ResData.OrderNum != job || !item.ResData.Pending || duplicate[i].ResData.OrderNum != job {
|
||||||
|
t.Fatalf("恢复丢失已持久化jobId:%+v", item)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
expectedQueries[job] = 3
|
||||||
|
input.ReqData.QueryJobID = job
|
||||||
|
for attempt := 0; attempt < 3; attempt++ {
|
||||||
|
result, e := client.QuerySimApnFunctionResult(context.Background(), input)
|
||||||
|
if e != nil || len(result) != 10 {
|
||||||
|
t.Fatalf("单次查询失败:%v", e)
|
||||||
|
}
|
||||||
|
for _, item := range result {
|
||||||
|
if attempt < 2 && (!item.ResData.Pending || item.Message == "") {
|
||||||
|
t.Fatalf("14008/处理中不得终态:%+v", item)
|
||||||
|
}
|
||||||
|
if attempt == 2 && (item.ResData.Pending || item.ResData.Uncertain || (scenario == "failed-task" && item.Code != 9999) || (scenario != "failed-task" && item.Code != 0)) {
|
||||||
|
t.Fatalf("终态语义错误:%+v", item)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
statsResponse, e := http.Get(chanURL + "/__test/stats")
|
||||||
|
if e != nil {
|
||||||
|
t.Fatal(e)
|
||||||
|
}
|
||||||
|
var stats struct {
|
||||||
|
Writes map[string]int `json:"writes"`
|
||||||
|
Queries map[string]int `json:"queries"`
|
||||||
|
Submits int `json:"submits"`
|
||||||
|
Unexpected int `json:"unexpected"`
|
||||||
|
}
|
||||||
|
e = json.NewDecoder(statsResponse.Body).Decode(&stats)
|
||||||
|
_ = statsResponse.Body.Close()
|
||||||
|
expectedSubmits++
|
||||||
|
for _, key := range input.SimKeys {
|
||||||
|
expectedWrites[key.MSISDN] = 1
|
||||||
|
}
|
||||||
|
if e != nil || stats.Submits != expectedSubmits || stats.Unexpected != 0 || len(stats.Writes) != len(expectedWrites) || len(stats.Queries) != len(expectedQueries) {
|
||||||
|
t.Fatalf("额外或缺失上游请求:stats=%+v err=%v", stats, e)
|
||||||
|
}
|
||||||
|
for key, count := range expectedWrites {
|
||||||
|
if stats.Writes[key] != count {
|
||||||
|
t.Fatalf("逐卡办理次数错误:%s=%d,预期%d", key, stats.Writes[key], count)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for job, count := range expectedQueries {
|
||||||
|
if stats.Queries[job] != count {
|
||||||
|
t.Fatalf("纯查询次数错误:%s=%d,预期%d", job, stats.Queries[job], count)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
counts := make([]int, 0, len(input.SimKeys))
|
||||||
|
for _, key := range input.SimKeys {
|
||||||
|
counts = append(counts, stats.Writes[key.MSISDN])
|
||||||
|
}
|
||||||
|
t.Logf("十卡实际办理计数=%v 累计提交=%d 无额外请求;丢失jobId=%v 明确失败终态=%v", counts, stats.Submits, scenario == "lost-response", scenario == "failed-task")
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAsyncApnCapabilityGateBlocksOldChanBeforeSubmit(t *testing.T) {
|
||||||
|
oldAdapter := g.Cfg().GetAdapter()
|
||||||
|
defer g.Cfg().SetAdapter(oldAdapter)
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
enabled int
|
||||||
|
version int
|
||||||
|
wantError bool
|
||||||
|
}{{"显式关闭", 0, 1, true}, {"旧版无能力", 1, 0, true}, {"能力就绪", 1, 1, false}} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
probes, submits := 0, 0
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.URL.Path == "/sim/query/sim-apn-capability" {
|
||||||
|
probes++
|
||||||
|
_, _ = fmt.Fprintf(w, `{"code":0,"data":[{"code":0,"simKey":{"chanKey":"test","iccid":"fake-card"},"resData":{"protocolVersion":%d}}]}`, tc.version)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
submits++
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"data":[{"code":0,"resData":{"pending":true,"orderNum":"fake-job"}}]}`))
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
adapter, err := gcfg.NewAdapterContent(fmt.Sprintf(`{"upstream":{"proxy":{"address":%q,"timeout":"1s","apnAsyncProtocolVersion":%d}}}`, strings.TrimPrefix(server.URL, "http://"), tc.enabled))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
g.Cfg().SetAdapter(adapter)
|
||||||
|
_, err = New().SubmitSimApnFunction(context.Background(), &model.SimCardApnChangeInput{SimKeys: []*model.SimKey{{ChanKey: "test", ICCID: "fake-card"}}, ReqData: model.SimCardApnChangeInputItem{OperationID: "fake-operation"}})
|
||||||
|
var notSent interface{ APNRequestNotSent() bool }
|
||||||
|
if (err != nil) != tc.wantError || (tc.wantError && (submits != 0 || !errors.As(err, ¬Sent) || !notSent.APNRequestNotSent())) || (!tc.wantError && (submits != 1 || probes != 1)) {
|
||||||
|
t.Fatalf("能力门禁失效:probes=%d submits=%d err=%v", probes, submits, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestAsyncApnSubmitDefaultsEnabledWithoutGateConfig 固化默认开启:
|
||||||
|
// 配置里不写 apnAsyncProtocolVersion 时,提交不应再被门禁拦下,兼容性交给协议能力探测。
|
||||||
|
func TestAsyncApnSubmitDefaultsEnabledWithoutGateConfig(t *testing.T) {
|
||||||
|
oldAdapter := g.Cfg().GetAdapter()
|
||||||
|
defer g.Cfg().SetAdapter(oldAdapter)
|
||||||
|
probes, submits := 0, 0
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.URL.Path == "/sim/query/sim-apn-capability" {
|
||||||
|
probes++
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"data":[{"code":0,"simKey":{"chanKey":"test","iccid":"fake-card"},"resData":{"protocolVersion":1}}]}`))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
submits++
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"data":[{"code":0,"resData":{"pending":true,"orderNum":"fake-job"}}]}`))
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
adapter, err := gcfg.NewAdapterContent(fmt.Sprintf(`{"upstream":{"proxy":{"address":%q,"timeout":"1s"}}}`, strings.TrimPrefix(server.URL, "http://")))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
g.Cfg().SetAdapter(adapter)
|
||||||
|
if _, err = New().SubmitSimApnFunction(context.Background(), &model.SimCardApnChangeInput{SimKeys: []*model.SimKey{{ChanKey: "test", ICCID: "fake-card"}}, ReqData: model.SimCardApnChangeInputItem{OperationID: "fake-operation"}}); err != nil {
|
||||||
|
t.Fatalf("未配置开关时提交应默认放行:%v", err)
|
||||||
|
}
|
||||||
|
if probes != 1 || submits != 1 {
|
||||||
|
t.Fatalf("默认放行后仍应做协议探测:probes=%d submits=%d", probes, submits)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAsyncApnReadOnlyQueryBypassesSubmitGateButRequiresProtocol(t *testing.T) {
|
||||||
|
oldAdapter := g.Cfg().GetAdapter()
|
||||||
|
defer g.Cfg().SetAdapter(oldAdapter)
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name, jobID, operationID string
|
||||||
|
version int
|
||||||
|
missingProbe bool
|
||||||
|
wantQuery bool
|
||||||
|
}{
|
||||||
|
{name: "existing_job", jobID: "existing-job", version: 1, wantQuery: true},
|
||||||
|
{name: "existing_operation", operationID: "existing-operation", version: 1, wantQuery: true},
|
||||||
|
{name: "legacy_protocol", jobID: "existing-job", version: 0},
|
||||||
|
{name: "missing_capability", jobID: "existing-job", missingProbe: true},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
probes, queries := 0, 0
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.URL.Path == "/sim/query/sim-apn-capability" {
|
||||||
|
probes++
|
||||||
|
if tc.missingProbe {
|
||||||
|
_ = json.NewEncoder(w).Encode(map[string]any{"code": 404, "message": "missing capability"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
_, _ = fmt.Fprintf(w, `{"code":0,"data":[{"code":0,"simKey":{"chanKey":"test","iccid":"fake-card"},"resData":{"protocolVersion":%d}}]}`, tc.version)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if r.URL.Path != "/sim/change/sim-apn" {
|
||||||
|
t.Errorf("unexpected endpoint: %s", r.URL.Path)
|
||||||
|
}
|
||||||
|
queries++
|
||||||
|
var request model.SimCardApnChangeInput
|
||||||
|
if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
if request.ReqData.SubmitOnly || request.ReqData.QueryJobID != tc.jobID || request.ReqData.RecoverOnly != (tc.jobID == "") {
|
||||||
|
t.Errorf("query accidentally submitted: %+v", request.ReqData)
|
||||||
|
}
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"data":[{"code":0,"simKey":{"chanKey":"test","iccid":"fake-card"},"resData":{"orderNum":"existing-job","pending":true}}]}`))
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
adapter, err := gcfg.NewAdapterContent(fmt.Sprintf(`{"upstream":{"proxy":{"address":%q,"timeout":"1s","apnAsyncProtocolVersion":0}}}`, strings.TrimPrefix(server.URL, "http://")))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
g.Cfg().SetAdapter(adapter)
|
||||||
|
result, err := New().QuerySimApnFunctionResult(context.Background(), &model.SimCardApnChangeInput{
|
||||||
|
SimKeys: []*model.SimKey{{ChanKey: "test", ICCID: "fake-card"}},
|
||||||
|
ReqData: model.SimCardApnChangeInputItem{OperationID: tc.operationID, QueryJobID: tc.jobID},
|
||||||
|
})
|
||||||
|
if probes != 1 || (err == nil) != tc.wantQuery || queries != map[bool]int{true: 1, false: 0}[tc.wantQuery] {
|
||||||
|
t.Fatalf("read-only protocol guard failed: probes=%d queries=%d err=%v result=%+v", probes, queries, err, result)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAsyncApnSecondProbeRejectionIsTypedAndTransportIsNot(t *testing.T) {
|
||||||
|
oldAdapter := g.Cfg().GetAdapter()
|
||||||
|
defer g.Cfg().SetAdapter(oldAdapter)
|
||||||
|
for _, probeFails := range []bool{true, false} {
|
||||||
|
probes, submits := 0, 0
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
// 夹具不共享连接:并发跑包时复用空闲连接会被对端关闭,
|
||||||
|
// 使失败从"提交阶段"漂移到"只读探测阶段",造成与被测语义无关的抖动。
|
||||||
|
w.Header().Set("Connection", "close")
|
||||||
|
if r.URL.Path == "/sim/query/sim-apn-capability" {
|
||||||
|
probes++
|
||||||
|
if probes == 2 && probeFails {
|
||||||
|
http.Error(w, "模拟第二次探测失败", http.StatusNotFound)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
fmt.Fprint(w, `{"code":0,"data":[{"code":0,"simKey":{"chanKey":"test","iccid":"fake-card"},"resData":{"protocolVersion":1}}]}`)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
submits++
|
||||||
|
conn, _, err := w.(http.Hijacker).Hijack()
|
||||||
|
if err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
_ = conn.Close()
|
||||||
|
}))
|
||||||
|
adapter, err := gcfg.NewAdapterContent(fmt.Sprintf(`{"upstream":{"proxy":{"address":%q,"timeout":"1s","apnAsyncProtocolVersion":1}}}`, strings.TrimPrefix(server.URL, "http://")))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
g.Cfg().SetAdapter(adapter)
|
||||||
|
input := &model.SimCardApnChangeInput{SimKeys: []*model.SimKey{{ChanKey: "test", ICCID: "fake-card"}}, ReqData: model.SimCardApnChangeInputItem{OperationID: "fake-operation"}}
|
||||||
|
client := New()
|
||||||
|
if err := client.CheckSimApnAsyncCapability(context.Background(), input); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
_, err = client.SubmitSimApnFunction(context.Background(), input)
|
||||||
|
server.Close()
|
||||||
|
var notSent interface{ APNRequestNotSent() bool }
|
||||||
|
isNotSent := errors.As(err, ¬Sent) && notSent.APNRequestNotSent()
|
||||||
|
if err == nil || isNotSent != probeFails || (probeFails && submits != 0) || (!probeFails && submits != 1) || probes != 2 {
|
||||||
|
t.Fatalf("零写类型边界错误:probeFails=%v probes=%d submits=%d err=%v", probeFails, probes, submits, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestSimCardStatusChangeProtocol(t *testing.T) {
|
func TestSimCardStatusChangeProtocol(t *testing.T) {
|
||||||
input := model.SimCardStatusChangeInput{
|
input := model.SimCardStatusChangeInput{
|
||||||
SimKeys: []*model.SimKey{{ChanKey: "channel-a", ICCID: "89860001"}},
|
SimKeys: []*model.SimKey{{ChanKey: "channel-a", ICCID: "89860001"}},
|
||||||
@@ -89,6 +472,17 @@ func TestConfiguredInboundBasicInfoClientUsesDedicatedEndpoint(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestDecodeAsyncApnResponseKeepsUnconfirmedState(t *testing.T) {
|
||||||
|
body := []byte(`{"code":0,"data":[{"code":0,"resData":{"orderNum":"job-1","pending":true}},{"code":1,"message":"响应未确认","resData":{"orderNum":"job-2","uncertain":true}}]}`)
|
||||||
|
items, err := decodeResponse[model.SimCardApnChangeOutputItem](body)
|
||||||
|
if err != nil || len(items) != 2 {
|
||||||
|
t.Fatalf("解析异步响应失败:%v", err)
|
||||||
|
}
|
||||||
|
if !items[0].ResData.Pending || items[0].ResData.OrderNum != "job-1" || !items[1].ResData.Uncertain {
|
||||||
|
t.Fatalf("异步待查字段丢失:%+v", items)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestSimCardApnChangeProtocol(t *testing.T) {
|
func TestSimCardApnChangeProtocol(t *testing.T) {
|
||||||
input := model.SimCardApnChangeInput{
|
input := model.SimCardApnChangeInput{
|
||||||
SimKeys: []*model.SimKey{{ChanKey: "channel-a", ICCID: "89860001"}},
|
SimKeys: []*model.SimKey{{ChanKey: "channel-a", ICCID: "89860001"}},
|
||||||
|
|||||||
@@ -0,0 +1,49 @@
|
|||||||
|
package upstream
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestResolveUpstreamTimeoutScalesWithCardCount 覆盖批量操作预算:
|
||||||
|
// worker 不能再把批量请求压回 2 分钟,否则通道仍在按卡串行办理时会被提前判失败。
|
||||||
|
func TestResolveUpstreamTimeoutScalesWithCardCount(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
name string
|
||||||
|
path string
|
||||||
|
itemCount int
|
||||||
|
perCard time.Duration
|
||||||
|
override time.Duration
|
||||||
|
maxTimeout time.Duration
|
||||||
|
apnTimeout time.Duration
|
||||||
|
want time.Duration
|
||||||
|
}{
|
||||||
|
{"单卡", "/sim/change/sim-status", 1, 10 * time.Second, 0, defaultUpstreamProxyMaxTimeout, 240 * time.Second, 10 * time.Second},
|
||||||
|
{"零卡按单卡处理", "/sim/change/sim-status", 0, 10 * time.Second, 0, defaultUpstreamProxyMaxTimeout, 240 * time.Second, 10 * time.Second},
|
||||||
|
{"50卡按单卡预算放大", "/sim/change/sim-status", 50, 10 * time.Second, 0, defaultUpstreamProxyMaxTimeout, 240 * time.Second, 500 * time.Second},
|
||||||
|
{"100卡恰好低于上限", "/sim/change/sim-status", 100, 10 * time.Second, 0, defaultUpstreamProxyMaxTimeout, 240 * time.Second, 1000 * time.Second},
|
||||||
|
{"超大批量按上限截断", "/sim/query/usage", 1000, 10 * time.Second, 0, defaultUpstreamProxyMaxTimeout, 240 * time.Second, defaultUpstreamProxyMaxTimeout},
|
||||||
|
{"单卡预算缺失用默认值", "/sim/query/usage", 2, 0, 0, defaultUpstreamProxyMaxTimeout, 240 * time.Second, 20 * time.Second},
|
||||||
|
{"显式覆盖仍按卡数放大", "/sim/query/usage", 3, 10 * time.Second, 2 * time.Second, defaultUpstreamProxyMaxTimeout, 240 * time.Second, 6 * time.Second},
|
||||||
|
{"数据开停使用独立预算", "/sim/change/sim-apn", 50, 10 * time.Second, 0, defaultUpstreamProxyMaxTimeout, 240 * time.Second, 240 * time.Second},
|
||||||
|
{"数据开停预算缺失回退", "/sim/change/sim-apn", 1, 10 * time.Second, 0, defaultUpstreamProxyMaxTimeout, 0, 240 * time.Second},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
t.Run(c.name, func(t *testing.T) {
|
||||||
|
got := resolveUpstreamTimeout(c.path, c.itemCount, c.perCard, c.override, c.maxTimeout, c.apnTimeout)
|
||||||
|
if got != c.want {
|
||||||
|
t.Fatalf("resolveUpstreamTimeout=%s,期望%s", got, c.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestUpstreamMaxTimeoutOutlastsProxyBatchTimeout 固化跨服务预算约束:
|
||||||
|
// worker 上限必须大于 proxy 的 gateway.batchTimeout(默认 20 分钟),
|
||||||
|
// 使超时由更靠近通道的一层先判定,避免 worker 提前放弃而通道仍在办理。
|
||||||
|
func TestUpstreamMaxTimeoutOutlastsProxyBatchTimeout(t *testing.T) {
|
||||||
|
const proxyBatchTimeout = 20 * time.Minute
|
||||||
|
if defaultUpstreamProxyMaxTimeout <= proxyBatchTimeout {
|
||||||
|
t.Fatalf("worker 上限%s 必须大于 proxy 批量上限%s", defaultUpstreamProxyMaxTimeout, proxyBatchTimeout)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -2,12 +2,58 @@ package cmd
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/errors/gerror"
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
"github.com/gogf/gf/v2/os/gcmd"
|
"github.com/gogf/gf/v2/os/gcmd"
|
||||||
|
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/client/upstream"
|
||||||
|
datarepository "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/data"
|
||||||
|
dataservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/data"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/worker"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/worker"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
var QueryAPNReview = gcmd.Command{
|
||||||
|
Name: "query-apn-review",
|
||||||
|
Usage: "query-apn-review --operation-id=<operationId> | --batch-no=<batchNo>",
|
||||||
|
Brief: "只读查询待核查APN子批并同步已确认结果(传 batch-no 时同步该父批下全部待核查子批)",
|
||||||
|
Arguments: []gcmd.Argument{
|
||||||
|
{Name: "operation-id", Brief: "待核查子批operationId"},
|
||||||
|
{Name: "batch-no", Brief: "待核查父批批次号,按父批同步全部待核查子批"},
|
||||||
|
},
|
||||||
|
Func: func(ctx context.Context, parser *gcmd.Parser) error {
|
||||||
|
operationID := strings.TrimSpace(parser.GetOpt("operation-id").String())
|
||||||
|
batchNo := strings.TrimSpace(parser.GetOpt("batch-no").String())
|
||||||
|
if operationID == "" && batchNo == "" {
|
||||||
|
return gerror.New("必须提供 operation-id 或 batch-no")
|
||||||
|
}
|
||||||
|
if operationID != "" && batchNo != "" {
|
||||||
|
return gerror.New("operation-id 与 batch-no 不能同时提供")
|
||||||
|
}
|
||||||
|
executor := dataservice.NewExecutor(datarepository.New(), upstream.New())
|
||||||
|
if batchNo != "" {
|
||||||
|
queried, err := executor.QueryReviewBatch(ctx, batchNo)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
g.Log().Infof(ctx, "APN待核查批量同步完成 batchNo=%s 已同步=%d", batchNo, queried)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err := executor.QueryReview(ctx, operationID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
g.Log().Infof(ctx, "APN待核查查询完成 operationId=%s", operationID)
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
func init() {
|
||||||
|
if err := Main.AddCommand(&QueryAPNReview); err != nil {
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
var Main = gcmd.Command{
|
var Main = gcmd.Command{
|
||||||
Name: "main",
|
Name: "main",
|
||||||
Usage: "main",
|
Usage: "main",
|
||||||
|
|||||||
@@ -1,8 +1,9 @@
|
|||||||
package consts
|
package consts
|
||||||
|
|
||||||
// BatchSubBatchSize 是批量开停数据 / 停复机执行时,单个上游通道每次请求携带的卡数量上限。
|
// BatchSubBatchSize 是批量开停数据 / 停复机执行时,单个上游通道每次请求携带的卡数量上限。
|
||||||
// 原来一个通道整组一次性发给 upstream-chan,大批量会触发 worker 侧 HTTP 超时(默认 2 分钟上限)导致整批失败。
|
// upstream-chan 对同一通道按卡串行调用运营商接口,单个请求耗时随卡数线性增长;
|
||||||
// 按此值拆分为多个小请求后,每个请求都能在超时上限内返回。
|
// 因此超时预算必须按「单卡预算 × 卡数」计算并配置足够上限(见 upstream.proxy.maxTimeout),
|
||||||
|
// 不能依赖固定 2 分钟上限,否则请求仍在上游办理就会被判失败。
|
||||||
const BatchSubBatchSize = 100
|
const BatchSubBatchSize = 100
|
||||||
|
|
||||||
// BatchSubBatchConcurrency 是同一通道内子批次的并发度,受信号量限制:
|
// BatchSubBatchConcurrency 是同一通道内子批次的并发度,受信号量限制:
|
||||||
|
|||||||
@@ -71,17 +71,23 @@ const (
|
|||||||
BatchAsyncStatusSuccess = 3
|
BatchAsyncStatusSuccess = 3
|
||||||
BatchAsyncStatusFailed = 4
|
BatchAsyncStatusFailed = 4
|
||||||
BatchAsyncStatusPartialSuccess = 5
|
BatchAsyncStatusPartialSuccess = 5
|
||||||
|
BatchAsyncStatusUnknown = 6
|
||||||
|
BatchAsyncStatusReview = 6
|
||||||
|
|
||||||
BatchRunResultPending = 0
|
BatchRunResultPending = 0
|
||||||
BatchRunResultRunning = 1
|
BatchRunResultRunning = 1
|
||||||
BatchRunResultSuccess = 2
|
BatchRunResultSuccess = 2
|
||||||
BatchRunResultFailed = 3
|
BatchRunResultFailed = 3
|
||||||
BatchRunResultPartialSuccess = 4
|
BatchRunResultPartialSuccess = 4
|
||||||
|
BatchRunResultUnknown = 5
|
||||||
|
BatchRunResultReview = 5
|
||||||
|
|
||||||
BatchDetailStatusPending = 0
|
BatchDetailStatusPending = 0
|
||||||
BatchDetailStatusRunning = 1
|
BatchDetailStatusRunning = 1
|
||||||
BatchDetailStatusSuccess = 2
|
BatchDetailStatusSuccess = 2
|
||||||
BatchDetailStatusFailed = 3
|
BatchDetailStatusFailed = 3
|
||||||
|
BatchDetailStatusUnknown = 4
|
||||||
|
BatchDetailStatusNotExecuted = 5
|
||||||
|
|
||||||
ResourceTypeExcel = 1
|
ResourceTypeExcel = 1
|
||||||
DownloadStatusGenerated = 1
|
DownloadStatusGenerated = 1
|
||||||
|
|||||||
@@ -25,6 +25,9 @@ const (
|
|||||||
QueueBatchRestartResumeDelay = "iotlcm.batch.restart.resume.delay"
|
QueueBatchRestartResumeDelay = "iotlcm.batch.restart.resume.delay"
|
||||||
QueuePackageCustomerSync = "iotlcm.package_customer.sync"
|
QueuePackageCustomerSync = "iotlcm.package_customer.sync"
|
||||||
QueueChannelRefresh = "iotlcm.channel.refresh"
|
QueueChannelRefresh = "iotlcm.channel.refresh"
|
||||||
|
// QueueDataReviewQuery 承载"手动同步上游结果"命令:Admin 发布父批号,
|
||||||
|
// Worker 只对既有 operationId 做只读补查并回写,绝不重发办理。
|
||||||
|
QueueDataReviewQuery = "iotlcm.data.review.query"
|
||||||
// QueueBalanceRechargeDeadLetter 保存重试次数耗尽后仍失败的充值超时消息,
|
// QueueBalanceRechargeDeadLetter 保存重试次数耗尽后仍失败的充值超时消息,
|
||||||
// 供人工排查(例如支付宝凭据未配置导致的永久失败),不再自动重投。
|
// 供人工排查(例如支付宝凭据未配置导致的永久失败),不再自动重投。
|
||||||
QueueBalanceRechargeDeadLetter = "iotlcm.balance.recharge.dlq"
|
QueueBalanceRechargeDeadLetter = "iotlcm.balance.recharge.dlq"
|
||||||
|
|||||||
@@ -0,0 +1,11 @@
|
|||||||
|
package consts
|
||||||
|
|
||||||
|
const (
|
||||||
|
DefaultSystemTenantCode = "000000"
|
||||||
|
SuperAdminRoleId = int64(1)
|
||||||
|
SysRoleMenuDataScopeRole = "0"
|
||||||
|
SysRoleDataScopeAll = "1"
|
||||||
|
SysRoleDataScopeCustom = "2"
|
||||||
|
SysRoleDataScopeDeptAndBelow = "4"
|
||||||
|
QueueAgentRoleSync = "iotlcm.agent.role.sync"
|
||||||
|
)
|
||||||
@@ -0,0 +1,247 @@
|
|||||||
|
package rabbitmq
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
_ "github.com/gogf/gf/contrib/drivers/sqlite/v2"
|
||||||
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch"
|
||||||
|
mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity"
|
||||||
|
batchrepo "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/batch"
|
||||||
|
datarepo "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/repository/data"
|
||||||
|
dataservice "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/data"
|
||||||
|
)
|
||||||
|
|
||||||
|
type durableBatchRepo struct{ failed int }
|
||||||
|
|
||||||
|
func (*durableBatchRepo) FindMain(context.Context, string) (*entity.BatchMain, error) {
|
||||||
|
return &entity.BatchMain{BatchNo: "APN-RECOVER", BatchType: 109, BatchAsyncStatus: batch.AsyncStatusPending}, nil
|
||||||
|
}
|
||||||
|
func (*durableBatchRepo) MarkProcessing(context.Context, string, int) (bool, error) { return true, nil }
|
||||||
|
func (r *durableBatchRepo) MarkFailed(context.Context, string, int, string) (bool, error) {
|
||||||
|
r.failed++
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
func (*durableBatchRepo) MarkSuccess(context.Context, string, int, int, int, string) (bool, error) {
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type durableBatchExecutor struct {
|
||||||
|
calls, protections int
|
||||||
|
retryable bool
|
||||||
|
protectionErr error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *durableBatchExecutor) Execute(context.Context, string) error {
|
||||||
|
e.calls++
|
||||||
|
if e.retryable {
|
||||||
|
return batch.Retryable(errors.New("准备中断"))
|
||||||
|
}
|
||||||
|
return errors.New("本地执行中断")
|
||||||
|
}
|
||||||
|
func (e *durableBatchExecutor) ProtectExecutionFailure(ctx context.Context, _ string, _ int, _ string) (bool, error) {
|
||||||
|
e.protections++
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return true, ctx.Err()
|
||||||
|
}
|
||||||
|
return true, e.protectionErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDurableAPNConsumerNeverFallsThroughToGenericFailure(t *testing.T) {
|
||||||
|
repo := &durableBatchRepo{}
|
||||||
|
executor := &durableBatchExecutor{}
|
||||||
|
dispatcher := batch.NewDispatcher(nil)
|
||||||
|
dispatcher.Register(109, executor)
|
||||||
|
consumers := &Consumers{batch: batch.NewRuntime(repo, dispatcher)}
|
||||||
|
if err := consumers.handleBatchExecute(context.Background(), batch.Message{BatchNo: "APN-RECOVER", BatchType: 109}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if executor.calls != 1 || executor.protections != 1 || repo.failed != 0 {
|
||||||
|
t.Fatalf("消费者普通错误越过durable保护: calls=%d protections=%d failed=%d", executor.calls, executor.protections, repo.failed)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDurableAPNConsumerRetryExhaustionProtectsBeforePublish(t *testing.T) {
|
||||||
|
repo := &durableBatchRepo{}
|
||||||
|
dbDown := errors.New("模拟保护数据库不可用")
|
||||||
|
executor := &durableBatchExecutor{retryable: true, protectionErr: dbDown}
|
||||||
|
dispatcher := batch.NewDispatcher(nil)
|
||||||
|
dispatcher.Register(109, executor)
|
||||||
|
consumers := &Consumers{batch: batch.NewRuntime(repo, dispatcher), batchExecuteMaxRetries: 0}
|
||||||
|
ctx := mq.WithDeliveryMetadata(context.Background(), mq.DeliveryMetadata{Headers: map[string]any{batchRetryCountHeader: 3}})
|
||||||
|
err := consumers.handleBatchExecute(ctx, batch.Message{BatchNo: "APN-RECOVER", BatchType: 109})
|
||||||
|
// 保护失败必须在消息发布前返回,测试不会连接真实RabbitMQ。
|
||||||
|
if !errors.Is(err, dbDown) || executor.calls != 1 || executor.protections != 1 || repo.failed != 0 {
|
||||||
|
t.Fatalf("重试耗尽错误终结或重发: err=%v calls=%d protections=%d failed=%d", err, executor.calls, executor.protections, repo.failed)
|
||||||
|
}
|
||||||
|
executor.protectionErr = nil
|
||||||
|
if err := consumers.handleBatchExecute(context.Background(), batch.Message{BatchNo: "APN-RECOVER", BatchType: 109, FailureReason: "重试耗尽"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if executor.calls != 1 || executor.protections != 2 || repo.failed != 0 {
|
||||||
|
t.Fatal("失败重投重新执行或MarkFailed")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// reentryWriteCounter 同时覆盖同步写、异步提交与结果查询,重入准备期间均不得调用。
|
||||||
|
type reentryWriteCounter struct{ syncWrites, asyncWrites, queries int }
|
||||||
|
|
||||||
|
func (u *reentryWriteCounter) ChangeSimApnFunction(context.Context, *model.SimCardApnChangeInput) ([]model.SimBaseRes[model.SimCardApnChangeOutputItem], error) {
|
||||||
|
u.syncWrites++
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
func (u *reentryWriteCounter) SubmitSimApnFunction(context.Context, *model.SimCardApnChangeInput) ([]model.SimBaseRes[model.SimCardApnChangeOutputItem], error) {
|
||||||
|
u.asyncWrites++
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
func (u *reentryWriteCounter) QuerySimApnFunctionResult(context.Context, *model.SimCardApnChangeInput) ([]model.SimBaseRes[model.SimCardApnChangeOutputItem], error) {
|
||||||
|
u.queries++
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAPNProcessingRabbitMQReentryRequiresDurableParent(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
if err := gdb.SetConfig(gdb.Config{"default": gdb.ConfigGroup{{Type: "sqlite", Link: "sqlite::@file(" + filepath.ToSlash(filepath.Join(t.TempDir(), "apn-reentry.db")) + ")"}}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
db := g.DB()
|
||||||
|
t.Cleanup(func() {
|
||||||
|
if err := db.Close(ctx); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
for _, sql := range []string{
|
||||||
|
"CREATE TABLE batch_main (id INTEGER PRIMARY KEY,batch_no TEXT UNIQUE,batch_type INTEGER,batch_async_status INTEGER,tenant_id TEXT,start_time DATETIME,batch_info TEXT,run_result INTEGER DEFAULT 0,valid_count INTEGER DEFAULT 0,success_count INTEGER DEFAULT 0,fail_count INTEGER DEFAULT 0,fail_reason TEXT,end_time DATETIME,updated_at DATETIME,created_by INTEGER,created_dept INTEGER,updated_by INTEGER)",
|
||||||
|
"CREATE TABLE batch_detail (id INTEGER PRIMARY KEY AUTOINCREMENT,batch_no TEXT,batch_type INTEGER,row_no INTEGER,item_id TEXT,detail_status INTEGER,error_reason TEXT,snapshot TEXT,updated_by INTEGER)",
|
||||||
|
"CREATE TABLE t_sim_card_info (id INTEGER PRIMARY KEY AUTOINCREMENT,tenant_id TEXT,iccid TEXT,msisdn TEXT,imsi TEXT,chan_key TEXT,data_service_status INTEGER)",
|
||||||
|
"CREATE TABLE data_operation_parent (batch_no TEXT PRIMARY KEY,ready INTEGER DEFAULT 0,finalize_token TEXT,finalize_until DATETIME,recovery_attempt_at DATETIME,last_reason TEXT)",
|
||||||
|
"CREATE TABLE data_operation_sub_batch (id INTEGER PRIMARY KEY AUTOINCREMENT,batch_no TEXT,batch_type INTEGER,chan_key TEXT,operation_id TEXT UNIQUE,detail_ids TEXT,request_data TEXT,job_id TEXT,status TEXT,poll_count INTEGER DEFAULT 0,next_check_at DATETIME,submitted_at DATETIME,lease_token TEXT,lease_until DATETIME,created_at DATETIME DEFAULT CURRENT_TIMESTAMP,last_reason TEXT)",
|
||||||
|
"CREATE TABLE data_operation_sub_detail (detail_id INTEGER PRIMARY KEY,operation_id TEXT,batch_no TEXT,iccid TEXT,UNIQUE(batch_no,iccid))",
|
||||||
|
"CREATE TABLE data_operation_card_claim (tenant_id TEXT,iccid TEXT,batch_id INTEGER,operation_id TEXT,active_batch_id INTEGER DEFAULT 0,active_operation_id TEXT DEFAULT '',PRIMARY KEY(tenant_id,iccid))",
|
||||||
|
} {
|
||||||
|
if _, err := db.Exec(ctx, sql); err != nil {
|
||||||
|
t.Fatalf("初始化本机 SQLite fixture: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
old, retryable bool
|
||||||
|
}{
|
||||||
|
{name: "legacy-ordinary", old: true},
|
||||||
|
{name: "legacy-retryable", old: true, retryable: true},
|
||||||
|
{name: "durable-ordinary"},
|
||||||
|
{name: "durable-retryable", retryable: true},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
status := batch.AsyncStatusPending
|
||||||
|
if tc.old {
|
||||||
|
status = batch.AsyncStatusProcessing
|
||||||
|
}
|
||||||
|
// 使用 NULL 旧首发时间以确保 ReclaimProcessing 的 WHERE start_time IS NULL 可以命中。
|
||||||
|
if _, err := db.Exec(ctx, "INSERT INTO batch_main(batch_no,batch_type,batch_async_status,tenant_id) VALUES (?,109,?,'tenant')", tc.name, status); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for i := 0; i < 10; i++ {
|
||||||
|
iccid := tc.name + "-card-" + string(rune('0'+i))
|
||||||
|
detailStatus := batch.DetailStatusPending
|
||||||
|
if tc.old {
|
||||||
|
detailStatus = batch.DetailStatusRunning
|
||||||
|
}
|
||||||
|
if _, err := db.Exec(ctx, "INSERT INTO batch_detail(batch_no,batch_type,row_no,item_id,detail_status,snapshot) VALUES (?,109,?,?,?,?)", tc.name, i+1, iccid, detailStatus, `{"authorized":true}`); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := db.Exec(ctx, "INSERT INTO t_sim_card_info(tenant_id,iccid,msisdn,imsi,chan_key,data_service_status) VALUES ('tenant',?,?,?,'mock-apn',0)", iccid, "130000000"+string(rune('0'+i)), "460000000"+string(rune('0'+i))); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
mainRepo := batchrepo.New()
|
||||||
|
if tc.old && tc.retryable {
|
||||||
|
// 先证实同一 fixture 的旧批确实可被重领,再复位空首发时间。
|
||||||
|
claimed, err := mainRepo.ReclaimProcessing(ctx, tc.name, 109, nil)
|
||||||
|
if err != nil || !claimed {
|
||||||
|
t.Fatalf("重领 CAS 必须可命中,避免假阳性: claimed=%t err=%v", claimed, err)
|
||||||
|
}
|
||||||
|
if _, err := db.Exec(ctx, "UPDATE batch_main SET start_time=NULL WHERE batch_no=?", tc.name); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !tc.old {
|
||||||
|
claimed, err := mainRepo.MarkProcessing(ctx, tc.name, 109)
|
||||||
|
if err != nil || !claimed {
|
||||||
|
t.Fatalf("新批需同事务领取 Processing 与 ready=0 父凭据: claimed=%t err=%v", claimed, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
parent, err := db.GetValue(ctx, "SELECT COUNT(*) FROM data_operation_parent WHERE batch_no=? AND ready=0", tc.name)
|
||||||
|
if err != nil || parent.Int() != 1 && !tc.old || parent.Int() != 0 && tc.old {
|
||||||
|
t.Fatalf("前置凭据不符: old=%t ready0=%d err=%v", tc.old, parent.Int(), err)
|
||||||
|
}
|
||||||
|
before, err := db.GetValue(ctx, "SELECT start_time FROM batch_main WHERE batch_no=?", tc.name)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
upstream := &reentryWriteCounter{}
|
||||||
|
dispatcher := batch.NewDispatcher(nil)
|
||||||
|
dispatcher.Register(109, dataservice.NewExecutor(datarepo.New(), upstream))
|
||||||
|
consumer := &Consumers{batch: batch.NewRuntime(mainRepo, dispatcher)}
|
||||||
|
err = consumer.handleBatchExecute(ctx, batch.Message{BatchNo: tc.name, BatchType: 109, Retryable: tc.retryable})
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("重入应安全跳过旧批或准备新批,不能进入消息重发路径: %v", err)
|
||||||
|
}
|
||||||
|
if upstream.syncWrites != 0 || upstream.asyncWrites != 0 || upstream.queries != 0 {
|
||||||
|
t.Errorf("重入意外调用上游: sync=%d submit=%d query=%d", upstream.syncWrites, upstream.asyncWrites, upstream.queries)
|
||||||
|
}
|
||||||
|
main, err := mainRepo.FindMain(ctx, tc.name)
|
||||||
|
if err != nil || main == nil || main.BatchAsyncStatus != batch.AsyncStatusProcessing {
|
||||||
|
t.Fatalf("批状态被意外终结: main=%+v err=%v", main, err)
|
||||||
|
}
|
||||||
|
remaining, err := db.GetValue(ctx, "SELECT COUNT(*) FROM batch_detail WHERE batch_no=? AND detail_status=1", tc.name)
|
||||||
|
if err != nil || remaining.Int() != 10 {
|
||||||
|
t.Fatalf("运行中明细不能被更改: count=%d err=%v", remaining.Int(), err)
|
||||||
|
}
|
||||||
|
intent, err := db.GetValue(ctx, "SELECT COUNT(*) FROM data_operation_sub_batch WHERE batch_no=? AND status='prepared'", tc.name)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if tc.old {
|
||||||
|
if intent.Int() != 0 {
|
||||||
|
t.Errorf("无凭据历史批不应产生新提交意图: %d", intent.Int())
|
||||||
|
}
|
||||||
|
assigned, err := db.GetValue(ctx, "SELECT COUNT(*) FROM data_operation_sub_detail WHERE batch_no=?", tc.name)
|
||||||
|
if err != nil || assigned.Int() != 0 {
|
||||||
|
t.Errorf("历史批不可冻结明细: count=%d err=%v", assigned.Int(), err)
|
||||||
|
}
|
||||||
|
claims, err := db.GetValue(ctx, "SELECT COUNT(*) FROM data_operation_card_claim WHERE operation_id!='' AND iccid LIKE ?", tc.name+"-%")
|
||||||
|
if err != nil || claims.Int() != 0 {
|
||||||
|
t.Errorf("历史批不可建立卡级提交占位: count=%d err=%v", claims.Int(), err)
|
||||||
|
}
|
||||||
|
parent, err = db.GetValue(ctx, "SELECT COUNT(*) FROM data_operation_parent WHERE batch_no=?", tc.name)
|
||||||
|
if err != nil || parent.Int() != 0 {
|
||||||
|
t.Errorf("历史批不可补造父凭据: count=%d err=%v", parent.Int(), err)
|
||||||
|
}
|
||||||
|
after, err := db.GetValue(ctx, "SELECT start_time FROM batch_main WHERE batch_no=?", tc.name)
|
||||||
|
if err != nil || after.String() != before.String() {
|
||||||
|
t.Errorf("旧批重入不可重领取: before=%s after=%s err=%v", before.String(), after.String(), err)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if intent.Int() != 1 {
|
||||||
|
t.Errorf("新批 ready=0 未正确恢复成 10 卡一个冻结意图: %d", intent.Int())
|
||||||
|
}
|
||||||
|
parent, err = db.GetValue(ctx, "SELECT COUNT(*) FROM data_operation_parent WHERE batch_no=? AND ready=1", tc.name)
|
||||||
|
if err != nil || parent.Int() != 1 {
|
||||||
|
t.Errorf("新批冻结意图与 ready=1 未原子提交: count=%d err=%v", parent.Int(), err)
|
||||||
|
}
|
||||||
|
assigned, err := db.GetValue(ctx, "SELECT COUNT(*) FROM data_operation_sub_detail WHERE batch_no=?", tc.name)
|
||||||
|
if err != nil || assigned.Int() != 10 {
|
||||||
|
t.Errorf("新批冻结明细必须覆盖全部 10 卡: count=%d err=%v", assigned.Int(), err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -11,9 +11,9 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
|
||||||
"github.com/gogf/gf/v2/frame/g"
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
"github.com/gogf/gf/v2/os/gcfg"
|
"github.com/gogf/gf/v2/os/gcfg"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
)
|
)
|
||||||
|
|
||||||
func setChannelRefreshTestConfig(t *testing.T, gateway, proxy string) {
|
func setChannelRefreshTestConfig(t *testing.T, gateway, proxy string) {
|
||||||
|
|||||||
@@ -56,6 +56,11 @@ type ImeiQueueStartHandler interface {
|
|||||||
Produce(context.Context, model.ImeiQueueStartMessage) error
|
Produce(context.Context, model.ImeiQueueStartMessage) error
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// DataReviewQueryHandler 执行"手动同步上游结果":只读补查既有 operationId。
|
||||||
|
type DataReviewQueryHandler interface {
|
||||||
|
QueryReviewBatch(ctx context.Context, batchNo string) (int, error)
|
||||||
|
}
|
||||||
|
|
||||||
type Consumers struct {
|
type Consumers struct {
|
||||||
trafficUsage TrafficUsageHandler
|
trafficUsage TrafficUsageHandler
|
||||||
workOrder WorkOrderHandler
|
workOrder WorkOrderHandler
|
||||||
@@ -72,6 +77,14 @@ type Consumers struct {
|
|||||||
simStatusQueueStart SimStatusQueueStartHandler
|
simStatusQueueStart SimStatusQueueStartHandler
|
||||||
simStatusQueueTask SimStatusQueueTaskHandler
|
simStatusQueueTask SimStatusQueueTaskHandler
|
||||||
imeiQueueStart ImeiQueueStartHandler
|
imeiQueueStart ImeiQueueStartHandler
|
||||||
|
dataReview DataReviewQueryHandler
|
||||||
|
}
|
||||||
|
|
||||||
|
// WithDataReviewQuery adds the manual "sync upstream result" consumer without
|
||||||
|
// changing any existing RabbitMQ consumer wiring.
|
||||||
|
func (c *Consumers) WithDataReviewQuery(handler DataReviewQueryHandler) *Consumers {
|
||||||
|
c.dataReview = handler
|
||||||
|
return c
|
||||||
}
|
}
|
||||||
|
|
||||||
// WithDailyUsageStart adds the previous-day usage producer without changing any
|
// WithDailyUsageStart adds the previous-day usage producer without changing any
|
||||||
@@ -145,6 +158,7 @@ func (c *Consumers) initializers(workerDiagnosticEnabled bool) []func(context.Co
|
|||||||
c.initBalanceRechargeTimeout,
|
c.initBalanceRechargeTimeout,
|
||||||
c.initBatchExecute,
|
c.initBatchExecute,
|
||||||
c.initChannelRefresh,
|
c.initChannelRefresh,
|
||||||
|
c.initDataReviewQuery,
|
||||||
}
|
}
|
||||||
if c.packageCustomerSync != nil {
|
if c.packageCustomerSync != nil {
|
||||||
initializers = append(initializers, c.initPackageCustomerSync)
|
initializers = append(initializers, c.initPackageCustomerSync)
|
||||||
|
|||||||
@@ -300,7 +300,7 @@ func TestHandleWorkerDiagnostic(t *testing.T) {
|
|||||||
|
|
||||||
message := model.WorkerDiagnosticMessage{
|
message := model.WorkerDiagnosticMessage{
|
||||||
TraceID: "worker-test-1",
|
TraceID: "worker-test-1",
|
||||||
SentAt: 1780000000000,
|
SentAt: 1784030000000,
|
||||||
Message: "worker ping",
|
Message: "worker ping",
|
||||||
}
|
}
|
||||||
if err := consumers.handleWorkerDiagnostic(context.Background(), message); err != nil {
|
if err := consumers.handleWorkerDiagnostic(context.Background(), message); err != nil {
|
||||||
@@ -311,14 +311,14 @@ func TestHandleWorkerDiagnostic(t *testing.T) {
|
|||||||
func TestWorkerDiagnosticMessageJSONContract(t *testing.T) {
|
func TestWorkerDiagnosticMessageJSONContract(t *testing.T) {
|
||||||
message := model.WorkerDiagnosticMessage{
|
message := model.WorkerDiagnosticMessage{
|
||||||
TraceID: "worker-test-1",
|
TraceID: "worker-test-1",
|
||||||
SentAt: 1780000000000,
|
SentAt: 1784030000000,
|
||||||
Message: "worker ping",
|
Message: "worker ping",
|
||||||
}
|
}
|
||||||
body, err := json.Marshal(message)
|
body, err := json.Marshal(message)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("marshal diagnostic message: %v", err)
|
t.Fatalf("marshal diagnostic message: %v", err)
|
||||||
}
|
}
|
||||||
const expected = `{"traceId":"worker-test-1","sentAt":1780000000000,"message":"worker ping"}`
|
const expected = `{"traceId":"worker-test-1","sentAt":1784030000000,"message":"worker ping"}`
|
||||||
if string(body) != expected {
|
if string(body) != expected {
|
||||||
t.Fatalf("unexpected diagnostic JSON: %s", body)
|
t.Fatalf("unexpected diagnostic JSON: %s", body)
|
||||||
}
|
}
|
||||||
@@ -517,10 +517,11 @@ func TestMonthlyHistoryStartConsumerIsOptIn(t *testing.T) {
|
|||||||
|
|
||||||
func TestConsumerInitializersRespectWorkerDiagnosticSwitch(t *testing.T) {
|
func TestConsumerInitializersRespectWorkerDiagnosticSwitch(t *testing.T) {
|
||||||
consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{})
|
consumers := New(&trafficUsageHandlerStub{}, &workOrderHandlerStub{}, &balanceRechargeHandlerStub{})
|
||||||
if got := len(consumers.initializers(false)); got != 6 {
|
// 手动同步上游结果(initDataReviewQuery)随业务消费者常驻,无处理器时自跳过。
|
||||||
t.Fatalf("diagnostic disabled: expected 6 business consumers, got %d", got)
|
if got := len(consumers.initializers(false)); got != 7 {
|
||||||
|
t.Fatalf("diagnostic disabled: expected 7 business consumers, got %d", got)
|
||||||
}
|
}
|
||||||
if got := len(consumers.initializers(true)); got != 7 {
|
if got := len(consumers.initializers(true)); got != 8 {
|
||||||
t.Fatalf("diagnostic enabled: expected 7 consumers, got %d", got)
|
t.Fatalf("diagnostic enabled: expected 8 consumers, got %d", got)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,46 @@
|
|||||||
|
package rabbitmq
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts"
|
||||||
|
mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (c *Consumers) initDataReviewQuery(ctx context.Context) error {
|
||||||
|
if c.dataReview == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if !mq.Enabled(ctx) {
|
||||||
|
g.Log().Info(ctx, "data review query consumer skipped, RabbitMQ disabled")
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err := mq.ConsumeJSON[model.DataReviewQueryMessage](
|
||||||
|
ctx,
|
||||||
|
consts.QueueDataReviewQuery,
|
||||||
|
"iotlcm-data-review-query",
|
||||||
|
c.handleDataReviewQuery,
|
||||||
|
); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
g.Log().Infof(ctx, "data review query consumer started, queue:%s", consts.QueueDataReviewQuery)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Consumers) handleDataReviewQuery(ctx context.Context, message model.DataReviewQueryMessage) error {
|
||||||
|
if c.dataReview == nil || message.BatchNo == "" {
|
||||||
|
g.Log().Errorf(ctx, "data review query message dropped, invalid message:%+v", message)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
queried, err := c.dataReview.QueryReviewBatch(ctx, message.BatchNo)
|
||||||
|
if err != nil {
|
||||||
|
// 手动同步失败只记日志,不重投:重投也不会改变上游结果,且可能放大查询压力。
|
||||||
|
g.Log().Errorf(ctx, "手动同步APN上游结果失败 batchNo=%s 已同步=%d error=%v", message.BatchNo, queried, err)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
g.Log().Infof(ctx, "手动同步APN上游结果完成 batchNo=%s 已同步=%d", message.BatchNo, queried)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -6,11 +6,11 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/errors/gerror"
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq"
|
mq "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/trafficusage"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/trafficusage"
|
||||||
"github.com/gogf/gf/v2/errors/gerror"
|
|
||||||
"github.com/gogf/gf/v2/frame/g"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
var trafficUsagePublish = mq.PublishBytesWithHeaders
|
var trafficUsagePublish = mq.PublishBytesWithHeaders
|
||||||
|
|||||||
@@ -4,10 +4,10 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
|
amqp "github.com/rabbitmq/amqp091-go"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/trafficusage"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/trafficusage"
|
||||||
amqp "github.com/rabbitmq/amqp091-go"
|
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -0,0 +1,101 @@
|
|||||||
|
// ==========================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// ==========================================================================
|
||||||
|
|
||||||
|
package internal
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysConfigDao is the data access object for the table sys_config.
|
||||||
|
type SysConfigDao struct {
|
||||||
|
table string // table is the underlying table name of the DAO.
|
||||||
|
group string // group is the database configuration group name of the current DAO.
|
||||||
|
columns SysConfigColumns // columns contains all the column names of Table for convenient usage.
|
||||||
|
handlers []gdb.ModelHandler // handlers for customized model modification.
|
||||||
|
}
|
||||||
|
|
||||||
|
// SysConfigColumns defines and stores column names for the table sys_config.
|
||||||
|
type SysConfigColumns struct {
|
||||||
|
ConfigId string //
|
||||||
|
TenantId string //
|
||||||
|
ConfigName string //
|
||||||
|
ConfigKey string //
|
||||||
|
ConfigValue string //
|
||||||
|
ConfigType string //
|
||||||
|
CreatedDept string //
|
||||||
|
CreatedBy string //
|
||||||
|
CreatedAt string //
|
||||||
|
UpdatedBy string //
|
||||||
|
UpdatedAt string //
|
||||||
|
Remark string //
|
||||||
|
}
|
||||||
|
|
||||||
|
// sysConfigColumns holds the columns for the table sys_config.
|
||||||
|
var sysConfigColumns = SysConfigColumns{
|
||||||
|
ConfigId: "config_id",
|
||||||
|
TenantId: "tenant_id",
|
||||||
|
ConfigName: "config_name",
|
||||||
|
ConfigKey: "config_key",
|
||||||
|
ConfigValue: "config_value",
|
||||||
|
ConfigType: "config_type",
|
||||||
|
CreatedDept: "created_dept",
|
||||||
|
CreatedBy: "created_by",
|
||||||
|
CreatedAt: "created_at",
|
||||||
|
UpdatedBy: "updated_by",
|
||||||
|
UpdatedAt: "updated_at",
|
||||||
|
Remark: "remark",
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSysConfigDao creates and returns a new DAO object for table data access.
|
||||||
|
func NewSysConfigDao(handlers ...gdb.ModelHandler) *SysConfigDao {
|
||||||
|
return &SysConfigDao{
|
||||||
|
group: "default",
|
||||||
|
table: "sys_config",
|
||||||
|
columns: sysConfigColumns,
|
||||||
|
handlers: handlers,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// DB retrieves and returns the underlying raw database management object of the current DAO.
|
||||||
|
func (dao *SysConfigDao) DB() gdb.DB {
|
||||||
|
return g.DB(dao.group)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Table returns the table name of the current DAO.
|
||||||
|
func (dao *SysConfigDao) Table() string {
|
||||||
|
return dao.table
|
||||||
|
}
|
||||||
|
|
||||||
|
// Columns returns all column names of the current DAO.
|
||||||
|
func (dao *SysConfigDao) Columns() SysConfigColumns {
|
||||||
|
return dao.columns
|
||||||
|
}
|
||||||
|
|
||||||
|
// Group returns the database configuration group name of the current DAO.
|
||||||
|
func (dao *SysConfigDao) Group() string {
|
||||||
|
return dao.group
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation.
|
||||||
|
func (dao *SysConfigDao) Ctx(ctx context.Context) *gdb.Model {
|
||||||
|
model := dao.DB().Model(dao.table)
|
||||||
|
for _, handler := range dao.handlers {
|
||||||
|
model = handler(model)
|
||||||
|
}
|
||||||
|
return model.Safe().Ctx(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Transaction wraps the transaction logic using function f.
|
||||||
|
// It rolls back the transaction and returns the error if function f returns a non-nil error.
|
||||||
|
// It commits the transaction and returns nil if function f returns nil.
|
||||||
|
//
|
||||||
|
// Note: Do not commit or roll back the transaction in function f,
|
||||||
|
// as it is automatically handled by this function.
|
||||||
|
func (dao *SysConfigDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) {
|
||||||
|
return dao.Ctx(ctx).Transaction(ctx, f)
|
||||||
|
}
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
// ==========================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// ==========================================================================
|
||||||
|
|
||||||
|
package internal
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysPermissionRevisionDao is the data access object for the table sys_permission_revision.
|
||||||
|
type SysPermissionRevisionDao struct {
|
||||||
|
table string // table is the underlying table name of the DAO.
|
||||||
|
group string // group is the database configuration group name of the current DAO.
|
||||||
|
columns SysPermissionRevisionColumns // columns contains all the column names of Table for convenient usage.
|
||||||
|
handlers []gdb.ModelHandler // handlers for customized model modification.
|
||||||
|
}
|
||||||
|
|
||||||
|
// SysPermissionRevisionColumns defines and stores column names for the table sys_permission_revision.
|
||||||
|
type SysPermissionRevisionColumns struct {
|
||||||
|
Id string //
|
||||||
|
Revision string //
|
||||||
|
UpdatedAt string //
|
||||||
|
}
|
||||||
|
|
||||||
|
// sysPermissionRevisionColumns holds the columns for the table sys_permission_revision.
|
||||||
|
var sysPermissionRevisionColumns = SysPermissionRevisionColumns{
|
||||||
|
Id: "id",
|
||||||
|
Revision: "revision",
|
||||||
|
UpdatedAt: "updated_at",
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSysPermissionRevisionDao creates and returns a new DAO object for table data access.
|
||||||
|
func NewSysPermissionRevisionDao(handlers ...gdb.ModelHandler) *SysPermissionRevisionDao {
|
||||||
|
return &SysPermissionRevisionDao{
|
||||||
|
group: "default",
|
||||||
|
table: "sys_permission_revision",
|
||||||
|
columns: sysPermissionRevisionColumns,
|
||||||
|
handlers: handlers,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// DB retrieves and returns the underlying raw database management object of the current DAO.
|
||||||
|
func (dao *SysPermissionRevisionDao) DB() gdb.DB {
|
||||||
|
return g.DB(dao.group)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Table returns the table name of the current DAO.
|
||||||
|
func (dao *SysPermissionRevisionDao) Table() string {
|
||||||
|
return dao.table
|
||||||
|
}
|
||||||
|
|
||||||
|
// Columns returns all column names of the current DAO.
|
||||||
|
func (dao *SysPermissionRevisionDao) Columns() SysPermissionRevisionColumns {
|
||||||
|
return dao.columns
|
||||||
|
}
|
||||||
|
|
||||||
|
// Group returns the database configuration group name of the current DAO.
|
||||||
|
func (dao *SysPermissionRevisionDao) Group() string {
|
||||||
|
return dao.group
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation.
|
||||||
|
func (dao *SysPermissionRevisionDao) Ctx(ctx context.Context) *gdb.Model {
|
||||||
|
model := dao.DB().Model(dao.table)
|
||||||
|
for _, handler := range dao.handlers {
|
||||||
|
model = handler(model)
|
||||||
|
}
|
||||||
|
return model.Safe().Ctx(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Transaction wraps the transaction logic using function f.
|
||||||
|
// It rolls back the transaction and returns the error if function f returns a non-nil error.
|
||||||
|
// It commits the transaction and returns nil if function f returns nil.
|
||||||
|
//
|
||||||
|
// Note: Do not commit or roll back the transaction in function f,
|
||||||
|
// as it is automatically handled by this function.
|
||||||
|
func (dao *SysPermissionRevisionDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) {
|
||||||
|
return dao.Ctx(ctx).Transaction(ctx, f)
|
||||||
|
}
|
||||||
@@ -0,0 +1,119 @@
|
|||||||
|
// ==========================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// ==========================================================================
|
||||||
|
|
||||||
|
package internal
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysRoleDao is the data access object for the table sys_role.
|
||||||
|
type SysRoleDao struct {
|
||||||
|
table string // table is the underlying table name of the DAO.
|
||||||
|
group string // group is the database configuration group name of the current DAO.
|
||||||
|
columns SysRoleColumns // columns contains all the column names of Table for convenient usage.
|
||||||
|
handlers []gdb.ModelHandler // handlers for customized model modification.
|
||||||
|
}
|
||||||
|
|
||||||
|
// SysRoleColumns defines and stores column names for the table sys_role.
|
||||||
|
type SysRoleColumns struct {
|
||||||
|
RoleId string //
|
||||||
|
TenantId string //
|
||||||
|
DeptId string //
|
||||||
|
RoleName string //
|
||||||
|
RoleKey string //
|
||||||
|
RoleSort string //
|
||||||
|
DataScope string //
|
||||||
|
MenuCheckStrictly string //
|
||||||
|
DeptCheckStrictly string //
|
||||||
|
Status string //
|
||||||
|
IsPass string //
|
||||||
|
SourceRoleId string //
|
||||||
|
SourceDeptId string //
|
||||||
|
CreatedDept string //
|
||||||
|
CreatedBy string //
|
||||||
|
CreatedAt string //
|
||||||
|
UpdatedBy string //
|
||||||
|
UpdatedAt string //
|
||||||
|
DeletedBy string //
|
||||||
|
DeletedAt string //
|
||||||
|
Remark string //
|
||||||
|
}
|
||||||
|
|
||||||
|
// sysRoleColumns holds the columns for the table sys_role.
|
||||||
|
var sysRoleColumns = SysRoleColumns{
|
||||||
|
RoleId: "role_id",
|
||||||
|
TenantId: "tenant_id",
|
||||||
|
DeptId: "dept_id",
|
||||||
|
RoleName: "role_name",
|
||||||
|
RoleKey: "role_key",
|
||||||
|
RoleSort: "role_sort",
|
||||||
|
DataScope: "data_scope",
|
||||||
|
MenuCheckStrictly: "menu_check_strictly",
|
||||||
|
DeptCheckStrictly: "dept_check_strictly",
|
||||||
|
Status: "status",
|
||||||
|
IsPass: "is_pass",
|
||||||
|
SourceRoleId: "source_role_id",
|
||||||
|
SourceDeptId: "source_dept_id",
|
||||||
|
CreatedDept: "created_dept",
|
||||||
|
CreatedBy: "created_by",
|
||||||
|
CreatedAt: "created_at",
|
||||||
|
UpdatedBy: "updated_by",
|
||||||
|
UpdatedAt: "updated_at",
|
||||||
|
DeletedBy: "deleted_by",
|
||||||
|
DeletedAt: "deleted_at",
|
||||||
|
Remark: "remark",
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSysRoleDao creates and returns a new DAO object for table data access.
|
||||||
|
func NewSysRoleDao(handlers ...gdb.ModelHandler) *SysRoleDao {
|
||||||
|
return &SysRoleDao{
|
||||||
|
group: "default",
|
||||||
|
table: "sys_role",
|
||||||
|
columns: sysRoleColumns,
|
||||||
|
handlers: handlers,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// DB retrieves and returns the underlying raw database management object of the current DAO.
|
||||||
|
func (dao *SysRoleDao) DB() gdb.DB {
|
||||||
|
return g.DB(dao.group)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Table returns the table name of the current DAO.
|
||||||
|
func (dao *SysRoleDao) Table() string {
|
||||||
|
return dao.table
|
||||||
|
}
|
||||||
|
|
||||||
|
// Columns returns all column names of the current DAO.
|
||||||
|
func (dao *SysRoleDao) Columns() SysRoleColumns {
|
||||||
|
return dao.columns
|
||||||
|
}
|
||||||
|
|
||||||
|
// Group returns the database configuration group name of the current DAO.
|
||||||
|
func (dao *SysRoleDao) Group() string {
|
||||||
|
return dao.group
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation.
|
||||||
|
func (dao *SysRoleDao) Ctx(ctx context.Context) *gdb.Model {
|
||||||
|
model := dao.DB().Model(dao.table)
|
||||||
|
for _, handler := range dao.handlers {
|
||||||
|
model = handler(model)
|
||||||
|
}
|
||||||
|
return model.Safe().Ctx(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Transaction wraps the transaction logic using function f.
|
||||||
|
// It rolls back the transaction and returns the error if function f returns a non-nil error.
|
||||||
|
// It commits the transaction and returns nil if function f returns nil.
|
||||||
|
//
|
||||||
|
// Note: Do not commit or roll back the transaction in function f,
|
||||||
|
// as it is automatically handled by this function.
|
||||||
|
func (dao *SysRoleDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) {
|
||||||
|
return dao.Ctx(ctx).Transaction(ctx, f)
|
||||||
|
}
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
// ==========================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// ==========================================================================
|
||||||
|
|
||||||
|
package internal
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysRoleDeptDao is the data access object for the table sys_role_dept.
|
||||||
|
type SysRoleDeptDao struct {
|
||||||
|
table string // table is the underlying table name of the DAO.
|
||||||
|
group string // group is the database configuration group name of the current DAO.
|
||||||
|
columns SysRoleDeptColumns // columns contains all the column names of Table for convenient usage.
|
||||||
|
handlers []gdb.ModelHandler // handlers for customized model modification.
|
||||||
|
}
|
||||||
|
|
||||||
|
// SysRoleDeptColumns defines and stores column names for the table sys_role_dept.
|
||||||
|
type SysRoleDeptColumns struct {
|
||||||
|
TenantId string //
|
||||||
|
RoleId string //
|
||||||
|
DeptId string //
|
||||||
|
}
|
||||||
|
|
||||||
|
// sysRoleDeptColumns holds the columns for the table sys_role_dept.
|
||||||
|
var sysRoleDeptColumns = SysRoleDeptColumns{
|
||||||
|
TenantId: "tenant_id",
|
||||||
|
RoleId: "role_id",
|
||||||
|
DeptId: "dept_id",
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSysRoleDeptDao creates and returns a new DAO object for table data access.
|
||||||
|
func NewSysRoleDeptDao(handlers ...gdb.ModelHandler) *SysRoleDeptDao {
|
||||||
|
return &SysRoleDeptDao{
|
||||||
|
group: "default",
|
||||||
|
table: "sys_role_dept",
|
||||||
|
columns: sysRoleDeptColumns,
|
||||||
|
handlers: handlers,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// DB retrieves and returns the underlying raw database management object of the current DAO.
|
||||||
|
func (dao *SysRoleDeptDao) DB() gdb.DB {
|
||||||
|
return g.DB(dao.group)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Table returns the table name of the current DAO.
|
||||||
|
func (dao *SysRoleDeptDao) Table() string {
|
||||||
|
return dao.table
|
||||||
|
}
|
||||||
|
|
||||||
|
// Columns returns all column names of the current DAO.
|
||||||
|
func (dao *SysRoleDeptDao) Columns() SysRoleDeptColumns {
|
||||||
|
return dao.columns
|
||||||
|
}
|
||||||
|
|
||||||
|
// Group returns the database configuration group name of the current DAO.
|
||||||
|
func (dao *SysRoleDeptDao) Group() string {
|
||||||
|
return dao.group
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation.
|
||||||
|
func (dao *SysRoleDeptDao) Ctx(ctx context.Context) *gdb.Model {
|
||||||
|
model := dao.DB().Model(dao.table)
|
||||||
|
for _, handler := range dao.handlers {
|
||||||
|
model = handler(model)
|
||||||
|
}
|
||||||
|
return model.Safe().Ctx(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Transaction wraps the transaction logic using function f.
|
||||||
|
// It rolls back the transaction and returns the error if function f returns a non-nil error.
|
||||||
|
// It commits the transaction and returns nil if function f returns nil.
|
||||||
|
//
|
||||||
|
// Note: Do not commit or roll back the transaction in function f,
|
||||||
|
// as it is automatically handled by this function.
|
||||||
|
func (dao *SysRoleDeptDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) {
|
||||||
|
return dao.Ctx(ctx).Transaction(ctx, f)
|
||||||
|
}
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
// ==========================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// ==========================================================================
|
||||||
|
|
||||||
|
package internal
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysRoleMenuDao is the data access object for the table sys_role_menu.
|
||||||
|
type SysRoleMenuDao struct {
|
||||||
|
table string // table is the underlying table name of the DAO.
|
||||||
|
group string // group is the database configuration group name of the current DAO.
|
||||||
|
columns SysRoleMenuColumns // columns contains all the column names of Table for convenient usage.
|
||||||
|
handlers []gdb.ModelHandler // handlers for customized model modification.
|
||||||
|
}
|
||||||
|
|
||||||
|
// SysRoleMenuColumns defines and stores column names for the table sys_role_menu.
|
||||||
|
type SysRoleMenuColumns struct {
|
||||||
|
TenantId string //
|
||||||
|
RoleId string //
|
||||||
|
MenuId string //
|
||||||
|
DataScope string //
|
||||||
|
}
|
||||||
|
|
||||||
|
// sysRoleMenuColumns holds the columns for the table sys_role_menu.
|
||||||
|
var sysRoleMenuColumns = SysRoleMenuColumns{
|
||||||
|
TenantId: "tenant_id",
|
||||||
|
RoleId: "role_id",
|
||||||
|
MenuId: "menu_id",
|
||||||
|
DataScope: "data_scope",
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSysRoleMenuDao creates and returns a new DAO object for table data access.
|
||||||
|
func NewSysRoleMenuDao(handlers ...gdb.ModelHandler) *SysRoleMenuDao {
|
||||||
|
return &SysRoleMenuDao{
|
||||||
|
group: "default",
|
||||||
|
table: "sys_role_menu",
|
||||||
|
columns: sysRoleMenuColumns,
|
||||||
|
handlers: handlers,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// DB retrieves and returns the underlying raw database management object of the current DAO.
|
||||||
|
func (dao *SysRoleMenuDao) DB() gdb.DB {
|
||||||
|
return g.DB(dao.group)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Table returns the table name of the current DAO.
|
||||||
|
func (dao *SysRoleMenuDao) Table() string {
|
||||||
|
return dao.table
|
||||||
|
}
|
||||||
|
|
||||||
|
// Columns returns all column names of the current DAO.
|
||||||
|
func (dao *SysRoleMenuDao) Columns() SysRoleMenuColumns {
|
||||||
|
return dao.columns
|
||||||
|
}
|
||||||
|
|
||||||
|
// Group returns the database configuration group name of the current DAO.
|
||||||
|
func (dao *SysRoleMenuDao) Group() string {
|
||||||
|
return dao.group
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation.
|
||||||
|
func (dao *SysRoleMenuDao) Ctx(ctx context.Context) *gdb.Model {
|
||||||
|
model := dao.DB().Model(dao.table)
|
||||||
|
for _, handler := range dao.handlers {
|
||||||
|
model = handler(model)
|
||||||
|
}
|
||||||
|
return model.Safe().Ctx(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Transaction wraps the transaction logic using function f.
|
||||||
|
// It rolls back the transaction and returns the error if function f returns a non-nil error.
|
||||||
|
// It commits the transaction and returns nil if function f returns nil.
|
||||||
|
//
|
||||||
|
// Note: Do not commit or roll back the transaction in function f,
|
||||||
|
// as it is automatically handled by this function.
|
||||||
|
func (dao *SysRoleMenuDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) {
|
||||||
|
return dao.Ctx(ctx).Transaction(ctx, f)
|
||||||
|
}
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
// ==========================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// ==========================================================================
|
||||||
|
|
||||||
|
package internal
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysRoleSyncTaskDao is the data access object for the table sys_role_sync_task.
|
||||||
|
type SysRoleSyncTaskDao struct {
|
||||||
|
table string // table is the underlying table name of the DAO.
|
||||||
|
group string // group is the database configuration group name of the current DAO.
|
||||||
|
columns SysRoleSyncTaskColumns // columns contains all the column names of Table for convenient usage.
|
||||||
|
handlers []gdb.ModelHandler // handlers for customized model modification.
|
||||||
|
}
|
||||||
|
|
||||||
|
// SysRoleSyncTaskColumns defines and stores column names for the table sys_role_sync_task.
|
||||||
|
type SysRoleSyncTaskColumns struct {
|
||||||
|
TaskId string //
|
||||||
|
TenantId string //
|
||||||
|
SourceRoleId string //
|
||||||
|
SourceDeptId string //
|
||||||
|
ActorId string //
|
||||||
|
Status string //
|
||||||
|
Attempts string //
|
||||||
|
NextAttemptAt string //
|
||||||
|
LeaseUntil string //
|
||||||
|
LeaseToken string //
|
||||||
|
Payload string //
|
||||||
|
LastError string //
|
||||||
|
TotalCount string //
|
||||||
|
SuccessCount string //
|
||||||
|
FailedCount string //
|
||||||
|
CreatedAt string //
|
||||||
|
UpdatedAt string //
|
||||||
|
}
|
||||||
|
|
||||||
|
// sysRoleSyncTaskColumns holds the columns for the table sys_role_sync_task.
|
||||||
|
var sysRoleSyncTaskColumns = SysRoleSyncTaskColumns{
|
||||||
|
TaskId: "task_id",
|
||||||
|
TenantId: "tenant_id",
|
||||||
|
SourceRoleId: "source_role_id",
|
||||||
|
SourceDeptId: "source_dept_id",
|
||||||
|
ActorId: "actor_id",
|
||||||
|
Status: "status",
|
||||||
|
Attempts: "attempts",
|
||||||
|
NextAttemptAt: "next_attempt_at",
|
||||||
|
LeaseUntil: "lease_until",
|
||||||
|
LeaseToken: "lease_token",
|
||||||
|
Payload: "payload",
|
||||||
|
LastError: "last_error",
|
||||||
|
TotalCount: "total_count",
|
||||||
|
SuccessCount: "success_count",
|
||||||
|
FailedCount: "failed_count",
|
||||||
|
CreatedAt: "created_at",
|
||||||
|
UpdatedAt: "updated_at",
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSysRoleSyncTaskDao creates and returns a new DAO object for table data access.
|
||||||
|
func NewSysRoleSyncTaskDao(handlers ...gdb.ModelHandler) *SysRoleSyncTaskDao {
|
||||||
|
return &SysRoleSyncTaskDao{
|
||||||
|
group: "default",
|
||||||
|
table: "sys_role_sync_task",
|
||||||
|
columns: sysRoleSyncTaskColumns,
|
||||||
|
handlers: handlers,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// DB retrieves and returns the underlying raw database management object of the current DAO.
|
||||||
|
func (dao *SysRoleSyncTaskDao) DB() gdb.DB {
|
||||||
|
return g.DB(dao.group)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Table returns the table name of the current DAO.
|
||||||
|
func (dao *SysRoleSyncTaskDao) Table() string {
|
||||||
|
return dao.table
|
||||||
|
}
|
||||||
|
|
||||||
|
// Columns returns all column names of the current DAO.
|
||||||
|
func (dao *SysRoleSyncTaskDao) Columns() SysRoleSyncTaskColumns {
|
||||||
|
return dao.columns
|
||||||
|
}
|
||||||
|
|
||||||
|
// Group returns the database configuration group name of the current DAO.
|
||||||
|
func (dao *SysRoleSyncTaskDao) Group() string {
|
||||||
|
return dao.group
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation.
|
||||||
|
func (dao *SysRoleSyncTaskDao) Ctx(ctx context.Context) *gdb.Model {
|
||||||
|
model := dao.DB().Model(dao.table)
|
||||||
|
for _, handler := range dao.handlers {
|
||||||
|
model = handler(model)
|
||||||
|
}
|
||||||
|
return model.Safe().Ctx(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Transaction wraps the transaction logic using function f.
|
||||||
|
// It rolls back the transaction and returns the error if function f returns a non-nil error.
|
||||||
|
// It commits the transaction and returns nil if function f returns nil.
|
||||||
|
//
|
||||||
|
// Note: Do not commit or roll back the transaction in function f,
|
||||||
|
// as it is automatically handled by this function.
|
||||||
|
func (dao *SysRoleSyncTaskDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) {
|
||||||
|
return dao.Ctx(ctx).Transaction(ctx, f)
|
||||||
|
}
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
// ==========================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// ==========================================================================
|
||||||
|
|
||||||
|
package internal
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysUserRoleDao is the data access object for the table sys_user_role.
|
||||||
|
type SysUserRoleDao struct {
|
||||||
|
table string // table is the underlying table name of the DAO.
|
||||||
|
group string // group is the database configuration group name of the current DAO.
|
||||||
|
columns SysUserRoleColumns // columns contains all the column names of Table for convenient usage.
|
||||||
|
handlers []gdb.ModelHandler // handlers for customized model modification.
|
||||||
|
}
|
||||||
|
|
||||||
|
// SysUserRoleColumns defines and stores column names for the table sys_user_role.
|
||||||
|
type SysUserRoleColumns struct {
|
||||||
|
TenantId string //
|
||||||
|
UserId string //
|
||||||
|
RoleId string //
|
||||||
|
}
|
||||||
|
|
||||||
|
// sysUserRoleColumns holds the columns for the table sys_user_role.
|
||||||
|
var sysUserRoleColumns = SysUserRoleColumns{
|
||||||
|
TenantId: "tenant_id",
|
||||||
|
UserId: "user_id",
|
||||||
|
RoleId: "role_id",
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSysUserRoleDao creates and returns a new DAO object for table data access.
|
||||||
|
func NewSysUserRoleDao(handlers ...gdb.ModelHandler) *SysUserRoleDao {
|
||||||
|
return &SysUserRoleDao{
|
||||||
|
group: "default",
|
||||||
|
table: "sys_user_role",
|
||||||
|
columns: sysUserRoleColumns,
|
||||||
|
handlers: handlers,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// DB retrieves and returns the underlying raw database management object of the current DAO.
|
||||||
|
func (dao *SysUserRoleDao) DB() gdb.DB {
|
||||||
|
return g.DB(dao.group)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Table returns the table name of the current DAO.
|
||||||
|
func (dao *SysUserRoleDao) Table() string {
|
||||||
|
return dao.table
|
||||||
|
}
|
||||||
|
|
||||||
|
// Columns returns all column names of the current DAO.
|
||||||
|
func (dao *SysUserRoleDao) Columns() SysUserRoleColumns {
|
||||||
|
return dao.columns
|
||||||
|
}
|
||||||
|
|
||||||
|
// Group returns the database configuration group name of the current DAO.
|
||||||
|
func (dao *SysUserRoleDao) Group() string {
|
||||||
|
return dao.group
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ctx creates and returns a Model for the current DAO. It automatically sets the context for the current operation.
|
||||||
|
func (dao *SysUserRoleDao) Ctx(ctx context.Context) *gdb.Model {
|
||||||
|
model := dao.DB().Model(dao.table)
|
||||||
|
for _, handler := range dao.handlers {
|
||||||
|
model = handler(model)
|
||||||
|
}
|
||||||
|
return model.Safe().Ctx(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Transaction wraps the transaction logic using function f.
|
||||||
|
// It rolls back the transaction and returns the error if function f returns a non-nil error.
|
||||||
|
// It commits the transaction and returns nil if function f returns nil.
|
||||||
|
//
|
||||||
|
// Note: Do not commit or roll back the transaction in function f,
|
||||||
|
// as it is automatically handled by this function.
|
||||||
|
func (dao *SysUserRoleDao) Transaction(ctx context.Context, f func(ctx context.Context, tx gdb.TX) error) (err error) {
|
||||||
|
return dao.Ctx(ctx).Transaction(ctx, f)
|
||||||
|
}
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package dao
|
||||||
|
|
||||||
|
import (
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal"
|
||||||
|
)
|
||||||
|
|
||||||
|
// sysConfigDao is the data access object for the table sys_config.
|
||||||
|
// You can define custom methods on it to extend its functionality as needed.
|
||||||
|
type sysConfigDao struct {
|
||||||
|
*internal.SysConfigDao
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
// SysConfig is a globally accessible object for table sys_config operations.
|
||||||
|
SysConfig = sysConfigDao{internal.NewSysConfigDao()}
|
||||||
|
)
|
||||||
|
|
||||||
|
// Add your custom methods and functionality below.
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package dao
|
||||||
|
|
||||||
|
import (
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal"
|
||||||
|
)
|
||||||
|
|
||||||
|
// sysPermissionRevisionDao is the data access object for the table sys_permission_revision.
|
||||||
|
// You can define custom methods on it to extend its functionality as needed.
|
||||||
|
type sysPermissionRevisionDao struct {
|
||||||
|
*internal.SysPermissionRevisionDao
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
// SysPermissionRevision is a globally accessible object for table sys_permission_revision operations.
|
||||||
|
SysPermissionRevision = sysPermissionRevisionDao{internal.NewSysPermissionRevisionDao()}
|
||||||
|
)
|
||||||
|
|
||||||
|
// Add your custom methods and functionality below.
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package dao
|
||||||
|
|
||||||
|
import (
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal"
|
||||||
|
)
|
||||||
|
|
||||||
|
// sysRoleDao is the data access object for the table sys_role.
|
||||||
|
// You can define custom methods on it to extend its functionality as needed.
|
||||||
|
type sysRoleDao struct {
|
||||||
|
*internal.SysRoleDao
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
// SysRole is a globally accessible object for table sys_role operations.
|
||||||
|
SysRole = sysRoleDao{internal.NewSysRoleDao()}
|
||||||
|
)
|
||||||
|
|
||||||
|
// Add your custom methods and functionality below.
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package dao
|
||||||
|
|
||||||
|
import (
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal"
|
||||||
|
)
|
||||||
|
|
||||||
|
// sysRoleDeptDao is the data access object for the table sys_role_dept.
|
||||||
|
// You can define custom methods on it to extend its functionality as needed.
|
||||||
|
type sysRoleDeptDao struct {
|
||||||
|
*internal.SysRoleDeptDao
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
// SysRoleDept is a globally accessible object for table sys_role_dept operations.
|
||||||
|
SysRoleDept = sysRoleDeptDao{internal.NewSysRoleDeptDao()}
|
||||||
|
)
|
||||||
|
|
||||||
|
// Add your custom methods and functionality below.
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package dao
|
||||||
|
|
||||||
|
import (
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal"
|
||||||
|
)
|
||||||
|
|
||||||
|
// sysRoleMenuDao is the data access object for the table sys_role_menu.
|
||||||
|
// You can define custom methods on it to extend its functionality as needed.
|
||||||
|
type sysRoleMenuDao struct {
|
||||||
|
*internal.SysRoleMenuDao
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
// SysRoleMenu is a globally accessible object for table sys_role_menu operations.
|
||||||
|
SysRoleMenu = sysRoleMenuDao{internal.NewSysRoleMenuDao()}
|
||||||
|
)
|
||||||
|
|
||||||
|
// Add your custom methods and functionality below.
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package dao
|
||||||
|
|
||||||
|
import (
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal"
|
||||||
|
)
|
||||||
|
|
||||||
|
// sysRoleSyncTaskDao is the data access object for the table sys_role_sync_task.
|
||||||
|
// You can define custom methods on it to extend its functionality as needed.
|
||||||
|
type sysRoleSyncTaskDao struct {
|
||||||
|
*internal.SysRoleSyncTaskDao
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
// SysRoleSyncTask is a globally accessible object for table sys_role_sync_task operations.
|
||||||
|
SysRoleSyncTask = sysRoleSyncTaskDao{internal.NewSysRoleSyncTaskDao()}
|
||||||
|
)
|
||||||
|
|
||||||
|
// Add your custom methods and functionality below.
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package dao
|
||||||
|
|
||||||
|
import (
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao/internal"
|
||||||
|
)
|
||||||
|
|
||||||
|
// sysUserRoleDao is the data access object for the table sys_user_role.
|
||||||
|
// You can define custom methods on it to extend its functionality as needed.
|
||||||
|
type sysUserRoleDao struct {
|
||||||
|
*internal.SysUserRoleDao
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
// SysUserRole is a globally accessible object for table sys_user_role operations.
|
||||||
|
SysUserRole = sysUserRoleDao{internal.NewSysUserRoleDao()}
|
||||||
|
)
|
||||||
|
|
||||||
|
// Add your custom methods and functionality below.
|
||||||
@@ -14,4 +14,18 @@
|
|||||||
- 可控时间:阶段累计、处理与库存排除、unknown、采样缺口、并发事件、256 KiB 排名与恢复总量。
|
- 可控时间:阶段累计、处理与库存排除、unknown、采样缺口、并发事件、256 KiB 排名与恢复总量。
|
||||||
- `go build ./...` 已通过。
|
- `go build ./...` 已通过。
|
||||||
- 本机 CGO 关闭且未发现 C 编译器,race 尚未执行成功。
|
- 本机 CGO 关闭且未发现 C 编译器,race 尚未执行成功。
|
||||||
- 全仓测试非全绿;
|
- 全仓测试非全绿;未改动 origin/main 4369472 上复现 TestConfiguredInboundBasicInfoClientUsesDedicatedEndpoint、TestValidateUpstreamBasicInfoAcceptsMatchingIdentifiers、TestAllocateProcessingRetryAfterHasLargeBatchDefault 失败,后两项出现配置搜索路径 panic。
|
||||||
|
|
||||||
|
## 预演门禁与回退记录(仅只读核对,尚未部署)
|
||||||
|
|
||||||
|
环境仅 127.0.0.1。
|
||||||
|
|
||||||
|
- worker 原镜像:sha256:136196e9b05cbf926d4cf51451de99c22c5aa35479f5b60a31d7b0d65c4d9fd3
|
||||||
|
- bot 原镜像:sha256:671ebc8f8777b72ca168e24631a5fbb4a1d1ca499b81b3a83415a39c801b170f
|
||||||
|
- Compose 目录:/home/xjzl/iotlcm-deploy/iotlcmpreview
|
||||||
|
- worker Compose 组合:docker-compose.yml + docker-compose.channel-scan-30a8a34.yml;bot 来自 docker-compose.yml。
|
||||||
|
- worker /app/logs/full-sync 与 bot /data/full-sync 均挂载 /data/logs/iotlcm-worker-svr/full-sync;bot state 挂载 /data/app/iotlcm-bot/state。
|
||||||
|
- 递归归档基础校验:1 份,未发现基础格式/终态/汇总计数错误;尚需使用 Bot 完整 Validate 及其实际配置再次核验。
|
||||||
|
- 本机 Docker daemon 查询超时;Jenkins list_jobs 返回 401。尚未构建独立镜像、运行 race、保存完整部署配置、检查活跃 Run/Replay 或白名单、重建服务或触发管理端。
|
||||||
|
|
||||||
|
发布前仍需补齐以上门禁。仅服务级重建 worker/bot,显式 enabled=true,不替换归档/state 目录。先关闭 enabled 并重启 worker 可回退观测;必要时恢复原两服务 digest 和原 Compose/config。不清 Run、归档或发送状态。本轮没有进行规模压测,也不证明全程无缺料或八百万卡容量。
|
||||||
|
|||||||
@@ -0,0 +1,50 @@
|
|||||||
|
package batchfields
|
||||||
|
|
||||||
|
import (
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
"unicode/utf8"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
ItemIDLimit = 64
|
||||||
|
DetailReasonLimit = 512
|
||||||
|
FailureReasonByteLimit = 4096
|
||||||
|
)
|
||||||
|
|
||||||
|
var mysqlErrorAfterSQL = regexp.MustCompile(`(?i): Error [0-9]{3,5} \([0-9A-Z]{5}\): [^\r\n]{0,160}`)
|
||||||
|
|
||||||
|
func DetailReason(value string) string {
|
||||||
|
runes := []rune(strings.TrimSpace(value))
|
||||||
|
if len(runes) > DetailReasonLimit {
|
||||||
|
return string(runes[:DetailReasonLimit])
|
||||||
|
}
|
||||||
|
return string(runes)
|
||||||
|
}
|
||||||
|
|
||||||
|
func ItemID(value string) string {
|
||||||
|
if utf8.RuneCountInString(value) > ItemIDLimit {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
|
||||||
|
func FailureReason(value string) string {
|
||||||
|
value = strings.ToValidUTF8(strings.TrimSpace(value), "")
|
||||||
|
upper := strings.ToUpper(value)
|
||||||
|
for _, marker := range []string{": INSERT INTO ", ": UPDATE ", ": SELECT ", ": DELETE FROM "} {
|
||||||
|
if index := strings.Index(upper, marker); index >= 0 {
|
||||||
|
databaseError := mysqlErrorAfterSQL.FindString(value[index+len(marker):])
|
||||||
|
value = value[:index] + " (SQL omitted)" + databaseError
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(value) <= FailureReasonByteLimit {
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
end := FailureReasonByteLimit - 3
|
||||||
|
for end > 0 && !utf8.RuneStart(value[end]) {
|
||||||
|
end--
|
||||||
|
}
|
||||||
|
return value[:end] + "..."
|
||||||
|
}
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
package batchfields
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"unicode/utf8"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestItemIDRejectsOversizedCardIdentifier(t *testing.T) {
|
||||||
|
if got := ItemID(strings.Repeat("9", 65)); got != "" {
|
||||||
|
t.Fatalf("oversized identifier = %q, want empty", got)
|
||||||
|
}
|
||||||
|
if got := ItemID(strings.Repeat("9", 64)); len(got) != 64 {
|
||||||
|
t.Fatalf("64-character identifier = %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDetailReasonFitsColumn(t *testing.T) {
|
||||||
|
if got := DetailReason(strings.Repeat("错", 513)); utf8.RuneCountInString(got) != 512 {
|
||||||
|
t.Fatalf("detail reason length = %d", utf8.RuneCountInString(got))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFailureReasonOmitsStatementAndFitsColumn(t *testing.T) {
|
||||||
|
value := "insert failed: INSERT INTO batch_detail VALUES (" + strings.Repeat("错", 30000) + ")"
|
||||||
|
got := FailureReason(value)
|
||||||
|
if strings.Contains(got, "VALUES") || len(got) > FailureReasonByteLimit || !utf8.ValidString(got) {
|
||||||
|
t.Fatalf("unsafe failure reason: %q", got)
|
||||||
|
}
|
||||||
|
got = FailureReason(strings.Repeat("错", 30000))
|
||||||
|
if len(got) > FailureReasonByteLimit || !utf8.ValidString(got) {
|
||||||
|
t.Fatalf("invalid truncated UTF-8 reason")
|
||||||
|
}
|
||||||
|
got = FailureReason("insert detail failed: INSERT INTO batch_detail VALUES ('" + strings.Repeat("9", 1000) + "'): Error 1406 (22001): Data too long for column 'item_id' at row 1")
|
||||||
|
if strings.Contains(got, "VALUES") || !strings.Contains(got, "Error 1406 (22001)") {
|
||||||
|
t.Fatalf("SQL should be removed but database error preserved: %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -241,24 +241,29 @@ func publishWithConfirm(ctx context.Context, ch *amqp.Channel, queue string, mes
|
|||||||
|
|
||||||
// Consume 启动消费者,使用手动 Ack,handler 成功才确认消息。
|
// Consume 启动消费者,使用手动 Ack,handler 成功才确认消息。
|
||||||
func Consume(ctx context.Context, queue, consumer string, handler Handler) error {
|
func Consume(ctx context.Context, queue, consumer string, handler Handler) error {
|
||||||
|
_, err := ConsumeWithDone(ctx, queue, consumer, handler)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func ConsumeWithDone(ctx context.Context, queue, consumer string, handler Handler) (<-chan struct{}, error) {
|
||||||
ch, err := Channel(ctx)
|
ch, err := Channel(ctx)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return nil, err
|
||||||
}
|
}
|
||||||
if err = declareQueue(ch, queue); err != nil {
|
if err = declareQueue(ch, queue); err != nil {
|
||||||
_ = ch.Close()
|
_ = ch.Close()
|
||||||
return err
|
return nil, err
|
||||||
}
|
}
|
||||||
if err = ch.Qos(1, 0, false); err != nil {
|
if err = ch.Qos(1, 0, false); err != nil {
|
||||||
_ = ch.Close()
|
_ = ch.Close()
|
||||||
return gerror.Wrap(err, "rabbitmq set qos failed")
|
return nil, gerror.Wrap(err, "rabbitmq set qos failed")
|
||||||
}
|
}
|
||||||
|
|
||||||
// autoAck=false,业务处理成功后手动 Ack,处理失败时 Nack 并重新入队。
|
// autoAck=false,业务处理成功后手动 Ack,处理失败时 Nack 并重新入队。
|
||||||
deliveries, err := ch.Consume(queue, consumer, false, false, false, false, nil)
|
deliveries, err := ch.Consume(queue, consumer, false, false, false, false, nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
_ = ch.Close()
|
_ = ch.Close()
|
||||||
return gerror.Wrap(err, "rabbitmq consume failed")
|
return nil, gerror.Wrap(err, "rabbitmq consume failed")
|
||||||
}
|
}
|
||||||
|
|
||||||
consumerState := &managedConsumer{
|
consumerState := &managedConsumer{
|
||||||
@@ -269,6 +274,8 @@ func Consume(ctx context.Context, queue, consumer string, handler Handler) error
|
|||||||
registerConsumer(consumerState)
|
registerConsumer(consumerState)
|
||||||
go func() {
|
go func() {
|
||||||
defer close(consumerState.done)
|
defer close(consumerState.done)
|
||||||
|
defer ch.Close()
|
||||||
|
defer unregisterConsumer(consumerState)
|
||||||
for {
|
for {
|
||||||
if ctx.Err() != nil {
|
if ctx.Err() != nil {
|
||||||
return
|
return
|
||||||
@@ -302,7 +309,7 @@ func Consume(ctx context.Context, queue, consumer string, handler Handler) error
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}()
|
}()
|
||||||
return nil
|
return consumerState.done, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// ConsumeJSON 启动 JSON 消费者,统一完成反序列化和 Ack/Nack 处理。
|
// ConsumeJSON 启动 JSON 消费者,统一完成反序列化和 Ack/Nack 处理。
|
||||||
@@ -423,3 +430,16 @@ func cloneHeaders(headers amqp.Table) map[string]any {
|
|||||||
func isClosedError(err error) bool {
|
func isClosedError(err error) bool {
|
||||||
return err == nil || errors.Is(err, amqp.ErrClosed)
|
return err == nil || errors.Is(err, amqp.ErrClosed)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func unregisterConsumer(target *managedConsumer) {
|
||||||
|
consumerMu.Lock()
|
||||||
|
defer consumerMu.Unlock()
|
||||||
|
for index, consumer := range consumers {
|
||||||
|
if consumer == target {
|
||||||
|
remaining := make([]*managedConsumer, 0, len(consumers)-1)
|
||||||
|
remaining = append(remaining, consumers[:index]...)
|
||||||
|
consumers = append(remaining, consumers[index+1:]...)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -149,3 +149,22 @@ func resetRabbitMQState(t *testing.T) {
|
|||||||
connMu.Unlock()
|
connMu.Unlock()
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestUnregisterConsumerPreservesShutdownSnapshot(t *testing.T) {
|
||||||
|
consumerMu.Lock()
|
||||||
|
original := consumers
|
||||||
|
first, second, third := &managedConsumer{}, &managedConsumer{}, &managedConsumer{}
|
||||||
|
consumers = []*managedConsumer{first, second, third}
|
||||||
|
snapshot := consumers
|
||||||
|
consumerMu.Unlock()
|
||||||
|
defer func() { consumerMu.Lock(); consumers = original; consumerMu.Unlock() }()
|
||||||
|
unregisterConsumer(second)
|
||||||
|
consumerMu.Lock()
|
||||||
|
defer consumerMu.Unlock()
|
||||||
|
if len(consumers) != 2 || consumers[0] != first || consumers[1] != third {
|
||||||
|
t.Fatal("consumer not removed")
|
||||||
|
}
|
||||||
|
if snapshot[1] != second || snapshot[2] != third {
|
||||||
|
t.Fatal("shutdown snapshot mutated")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,151 @@
|
|||||||
|
package rolepolicy
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"github.com/gogf/gf/v2/errors/gerror"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
const TemplateConfigKey = "agent.role.template.role_id"
|
||||||
|
|
||||||
|
func Template(ctx context.Context) (*entity.SysRole, error) {
|
||||||
|
var configs []*entity.SysConfig
|
||||||
|
if err := dao.SysConfig.Ctx(ctx).Master().Where("tenant_id", consts.DefaultSystemTenantCode).Where("config_key", TemplateConfigKey).Scan(&configs); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if len(configs) == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
if len(configs) != 1 {
|
||||||
|
return nil, gerror.New("代理模板角色系统参数重复")
|
||||||
|
}
|
||||||
|
return ValidateTemplate(ctx, configs[0].ConfigValue)
|
||||||
|
}
|
||||||
|
|
||||||
|
func ValidateTemplate(ctx context.Context, value string) (*entity.SysRole, error) {
|
||||||
|
roleID, err := strconv.ParseInt(strings.TrimSpace(value), 10, 64)
|
||||||
|
if err != nil || roleID <= 0 || roleID == consts.SuperAdminRoleId {
|
||||||
|
return nil, gerror.New("代理模板参数必须填写非超级管理员的有效角色ID")
|
||||||
|
}
|
||||||
|
var role *entity.SysRole
|
||||||
|
if err = dao.SysRole.Ctx(ctx).Master().Where("role_id", roleID).Where("tenant_id", consts.DefaultSystemTenantCode).Scan(&role); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if role == nil || role.Status != "0" || (role.CreatedDept > 0 && role.CreatedDept != role.DeptId) || role.DeptId <= 0 {
|
||||||
|
return nil, gerror.New("代理模板参数对应角色须正常启用、绑定本级机构且非上级继承角色")
|
||||||
|
}
|
||||||
|
if role.DataScope != consts.SysRoleDataScopeDeptAndBelow {
|
||||||
|
return nil, gerror.New("代理模板数据范围必须为本机构及下级")
|
||||||
|
}
|
||||||
|
var dept *entity.SysDept
|
||||||
|
if err := dao.SysDept.Ctx(ctx).Master().Where("tenant_id", role.TenantId).Where("dept_id", role.DeptId).Where("status", "0").Scan(&dept); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if dept == nil {
|
||||||
|
return nil, gerror.New("代理模板所属机构不存在或已停用")
|
||||||
|
}
|
||||||
|
invalidScopes, err := dao.SysRoleMenu.Ctx(ctx).Master().Where("tenant_id", role.TenantId).Where("role_id", role.RoleId).WhereIn("data_scope", []string{consts.SysRoleDataScopeAll, consts.SysRoleDataScopeCustom}).Count()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if invalidScopes > 0 {
|
||||||
|
return nil, gerror.New("代理模板按钮不允许全部或自定义数据范围")
|
||||||
|
}
|
||||||
|
return role, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func MenuIDs(ctx context.Context, roleID int64) ([]int64, error) {
|
||||||
|
var menus []*entity.SysRoleMenu
|
||||||
|
if err := dao.SysRoleMenu.Ctx(ctx).Master().Where("role_id", roleID).Scan(&menus); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ids := make([]int64, 0, len(menus))
|
||||||
|
for _, menu := range menus {
|
||||||
|
ids = append(ids, menu.MenuId)
|
||||||
|
}
|
||||||
|
return ids, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func Intersect(left, right []int64) []int64 {
|
||||||
|
allowed := make(map[int64]bool, len(right))
|
||||||
|
for _, id := range right {
|
||||||
|
allowed[id] = true
|
||||||
|
}
|
||||||
|
result := make([]int64, 0, len(left))
|
||||||
|
for _, id := range left {
|
||||||
|
if allowed[id] {
|
||||||
|
result = append(result, id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
func CurrentSuperAdmin(ctx context.Context, userID int64) (bool, error) {
|
||||||
|
if userID <= 0 {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
var relations []*entity.SysUserRole
|
||||||
|
if err := dao.SysUserRole.Ctx(ctx).Master().Where("tenant_id", consts.DefaultSystemTenantCode).Where("user_id", userID).Where("role_id", consts.SuperAdminRoleId).Scan(&relations); err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
if len(relations) == 0 {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
var role *entity.SysRole
|
||||||
|
if err := dao.SysRole.Ctx(ctx).Master().Where("tenant_id", consts.DefaultSystemTenantCode).Where("role_id", consts.SuperAdminRoleId).Where("status", "0").Scan(&role); err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
return role != nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// RestrictMenus validates organization membership and role bindings. Templates are
|
||||||
|
// overwrite sources, not runtime permission ceilings: platform role edits must
|
||||||
|
// remain effective until the next explicit template distribution.
|
||||||
|
func RestrictMenus(ctx context.Context, tenantID string, deptID int64, ids []int64) ([]int64, error) {
|
||||||
|
template, err := Template(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if template == nil {
|
||||||
|
return ids, nil
|
||||||
|
}
|
||||||
|
seen := make(map[int64]bool)
|
||||||
|
for deptID > 0 {
|
||||||
|
if seen[deptID] {
|
||||||
|
return nil, gerror.New("机构上下级存在循环")
|
||||||
|
}
|
||||||
|
seen[deptID] = true
|
||||||
|
var dept *entity.SysDept
|
||||||
|
if err := dao.SysDept.Ctx(ctx).Master().Where("tenant_id", tenantID).Where("dept_id", deptID).Scan(&dept); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if dept == nil || dept.Status != "0" {
|
||||||
|
return []int64{}, nil
|
||||||
|
}
|
||||||
|
if tenantID == template.TenantId && deptID == template.DeptId {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if dept.DeptRole == consts.SuperAdminRoleId {
|
||||||
|
return nil, gerror.Newf("机构%d不能使用超级管理员角色作为代理权限上限,请核对机构角色绑定", dept.DeptId)
|
||||||
|
}
|
||||||
|
if dept.DeptRole <= 0 {
|
||||||
|
return nil, gerror.Newf("机构%d未关联默认角色,请联系平台管理员核对机构角色绑定", dept.DeptId)
|
||||||
|
}
|
||||||
|
var role *entity.SysRole
|
||||||
|
if err := dao.SysRole.Ctx(ctx).Master().Where("tenant_id", tenantID).Where("dept_id", deptID).Where("role_id", dept.DeptRole).Where("status", "0").Scan(&role); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if role == nil {
|
||||||
|
return nil, gerror.Newf("机构%d默认角色不存在、已停用或归属不匹配,请联系平台管理员核对", dept.DeptId)
|
||||||
|
}
|
||||||
|
deptID = dept.ParentId
|
||||||
|
}
|
||||||
|
if tenantID == template.TenantId && !seen[template.DeptId] {
|
||||||
|
return []int64{}, nil
|
||||||
|
}
|
||||||
|
return ids, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
package rolesync
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/rabbitmq"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
var publishRoleSync = func(ctx context.Context, message *model.AgentRoleSyncMessage) error {
|
||||||
|
return rabbitmq.Publish(ctx, consts.QueueAgentRoleSync, message)
|
||||||
|
}
|
||||||
@@ -0,0 +1,61 @@
|
|||||||
|
package tdengine
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
"github.com/gogf/gf/v2/os/gcfg"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestGetConfigDatabaseRouting(t *testing.T) {
|
||||||
|
previous := g.Cfg().GetAdapter()
|
||||||
|
t.Cleanup(func() { g.Cfg().SetAdapter(previous) })
|
||||||
|
for _, test := range []struct {
|
||||||
|
name string
|
||||||
|
fields string
|
||||||
|
fallback string
|
||||||
|
daily string
|
||||||
|
monthly string
|
||||||
|
}{
|
||||||
|
{"defaults", "", "iotlcm_daily", "iotlcm_daily", "iotlcm_daily"},
|
||||||
|
{"legacy", " dbName: legacy\n", "legacy", "legacy", "legacy"},
|
||||||
|
{"split", " dbName: legacy\n dailyDbName: iotlcm_daily\n monthlyDbName: iotlcm_monthly\n", "legacy", "iotlcm_daily", "iotlcm_monthly"},
|
||||||
|
{"daily_only", " dbName: legacy\n dailyDbName: iotlcm_daily\n", "legacy", "iotlcm_daily", "legacy"},
|
||||||
|
{"monthly_only", " dbName: legacy\n monthlyDbName: iotlcm_monthly\n", "legacy", "legacy", "iotlcm_monthly"},
|
||||||
|
} {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
adapter, err := gcfg.NewAdapterContent("tsdb:\n database: TdEngine\n tdengine:\n batchSize: 20\n" + test.fields)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
g.Cfg().SetAdapter(adapter)
|
||||||
|
config := GetConfig(context.Background())
|
||||||
|
if config.DbName != test.fallback {
|
||||||
|
t.Fatalf("fallback=%q want=%q", config.DbName, test.fallback)
|
||||||
|
}
|
||||||
|
for _, stable := range []string{config.DailyStable, config.PoolDailyStable, config.PoolCardCountDailyStable} {
|
||||||
|
if got := config.DbNameFor(stable); got != test.daily {
|
||||||
|
t.Errorf("%s database=%q want=%q", stable, got, test.daily)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, stable := range []string{config.MonthlyStable, config.PoolMonthlyStable, config.PoolCardCountMonthlyStable} {
|
||||||
|
if got := config.DbNameFor(stable); got != test.monthly {
|
||||||
|
t.Errorf("%s database=%q want=%q", stable, got, test.monthly)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got := config.DbNameFor("other_stable"); got != test.fallback {
|
||||||
|
t.Errorf("unknown stable database=%q", got)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDatabaseRoutingEmptySplitNames(t *testing.T) {
|
||||||
|
config := Config{DbName: "legacy", DailyStable: "daily", MonthlyStable: "monthly"}
|
||||||
|
for _, stable := range []string{"daily", "monthly", "other"} {
|
||||||
|
if got := config.DbNameFor(stable); got != "legacy" {
|
||||||
|
t.Errorf("%s database=%q", stable, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
package model
|
||||||
|
|
||||||
|
type AgentRoleSyncMessage struct {
|
||||||
|
TemplateRole bool `json:"templateRole,omitempty"`
|
||||||
|
Superseded bool `json:"superseded,omitempty"`
|
||||||
|
TaskId string `json:"taskId,omitempty"`
|
||||||
|
TotalCount int `json:"totalCount,omitempty"`
|
||||||
|
SuccessCount int `json:"successCount,omitempty"`
|
||||||
|
FailedCount int `json:"failedCount,omitempty"`
|
||||||
|
TenantId string `json:"tenantId"`
|
||||||
|
SourceRoleId int64 `json:"sourceRoleId"`
|
||||||
|
SourceDeptId int64 `json:"sourceDeptId"`
|
||||||
|
ActorId int64 `json:"actorId"`
|
||||||
|
RetryCount int `json:"retryCount"`
|
||||||
|
ChildDeptIds []int64 `json:"childDeptIds,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type AgentRoleSyncDeadLetterMessage struct {
|
||||||
|
Message AgentRoleSyncMessage `json:"message"`
|
||||||
|
Error string `json:"error"`
|
||||||
|
FailedAt string `json:"failedAt"`
|
||||||
|
}
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
package model
|
||||||
|
|
||||||
|
// DataReviewQueryMessage 是"手动同步上游结果"命令。
|
||||||
|
// 只按既有 operationId/jobId 查询一次,不重发办理、不清零自动补查计数。
|
||||||
|
type DataReviewQueryMessage struct {
|
||||||
|
BatchNo string `json:"batchNo"`
|
||||||
|
RequestedBy string `json:"requestedBy,omitempty"`
|
||||||
|
}
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package do
|
||||||
|
|
||||||
|
import (
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
"github.com/gogf/gf/v2/os/gtime"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysConfig is the golang structure of table sys_config for DAO operations like Where/Data.
|
||||||
|
type SysConfig struct {
|
||||||
|
g.Meta `orm:"table:sys_config, do:true"`
|
||||||
|
ConfigId any //
|
||||||
|
TenantId any //
|
||||||
|
ConfigName any //
|
||||||
|
ConfigKey any //
|
||||||
|
ConfigValue any //
|
||||||
|
ConfigType any //
|
||||||
|
CreatedDept any //
|
||||||
|
CreatedBy any //
|
||||||
|
CreatedAt *gtime.Time //
|
||||||
|
UpdatedBy any //
|
||||||
|
UpdatedAt *gtime.Time //
|
||||||
|
Remark any //
|
||||||
|
}
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package do
|
||||||
|
|
||||||
|
import (
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
"github.com/gogf/gf/v2/os/gtime"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysPermissionRevision is the golang structure of table sys_permission_revision for DAO operations like Where/Data.
|
||||||
|
type SysPermissionRevision struct {
|
||||||
|
g.Meta `orm:"table:sys_permission_revision, do:true"`
|
||||||
|
Id any //
|
||||||
|
Revision any //
|
||||||
|
UpdatedAt *gtime.Time //
|
||||||
|
}
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package do
|
||||||
|
|
||||||
|
import (
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
"github.com/gogf/gf/v2/os/gtime"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysRole is the golang structure of table sys_role for DAO operations like Where/Data.
|
||||||
|
type SysRole struct {
|
||||||
|
g.Meta `orm:"table:sys_role, do:true"`
|
||||||
|
RoleId any //
|
||||||
|
TenantId any //
|
||||||
|
DeptId any //
|
||||||
|
RoleName any //
|
||||||
|
RoleKey any //
|
||||||
|
RoleSort any //
|
||||||
|
DataScope any //
|
||||||
|
MenuCheckStrictly any //
|
||||||
|
DeptCheckStrictly any //
|
||||||
|
Status any //
|
||||||
|
IsPass any //
|
||||||
|
SourceRoleId any //
|
||||||
|
SourceDeptId any //
|
||||||
|
CreatedDept any //
|
||||||
|
CreatedBy any //
|
||||||
|
CreatedAt *gtime.Time //
|
||||||
|
UpdatedBy any //
|
||||||
|
UpdatedAt *gtime.Time //
|
||||||
|
DeletedBy any //
|
||||||
|
DeletedAt *gtime.Time //
|
||||||
|
Remark any //
|
||||||
|
}
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package do
|
||||||
|
|
||||||
|
import (
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysRoleDept is the golang structure of table sys_role_dept for DAO operations like Where/Data.
|
||||||
|
type SysRoleDept struct {
|
||||||
|
g.Meta `orm:"table:sys_role_dept, do:true"`
|
||||||
|
TenantId any //
|
||||||
|
RoleId any //
|
||||||
|
DeptId any //
|
||||||
|
}
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package do
|
||||||
|
|
||||||
|
import (
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysRoleMenu is the golang structure of table sys_role_menu for DAO operations like Where/Data.
|
||||||
|
type SysRoleMenu struct {
|
||||||
|
g.Meta `orm:"table:sys_role_menu, do:true"`
|
||||||
|
TenantId any //
|
||||||
|
RoleId any //
|
||||||
|
MenuId any //
|
||||||
|
DataScope any //
|
||||||
|
}
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package do
|
||||||
|
|
||||||
|
import (
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
"github.com/gogf/gf/v2/os/gtime"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysRoleSyncTask is the golang structure of table sys_role_sync_task for DAO operations like Where/Data.
|
||||||
|
type SysRoleSyncTask struct {
|
||||||
|
g.Meta `orm:"table:sys_role_sync_task, do:true"`
|
||||||
|
TaskId any //
|
||||||
|
TenantId any //
|
||||||
|
SourceRoleId any //
|
||||||
|
SourceDeptId any //
|
||||||
|
ActorId any //
|
||||||
|
Status any //
|
||||||
|
Attempts any //
|
||||||
|
NextAttemptAt any //
|
||||||
|
LeaseUntil any //
|
||||||
|
LeaseToken any //
|
||||||
|
Payload any //
|
||||||
|
LastError any //
|
||||||
|
TotalCount any //
|
||||||
|
SuccessCount any //
|
||||||
|
FailedCount any //
|
||||||
|
CreatedAt *gtime.Time //
|
||||||
|
UpdatedAt *gtime.Time //
|
||||||
|
}
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package do
|
||||||
|
|
||||||
|
import (
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysUserRole is the golang structure of table sys_user_role for DAO operations like Where/Data.
|
||||||
|
type SysUserRole struct {
|
||||||
|
g.Meta `orm:"table:sys_user_role, do:true"`
|
||||||
|
TenantId any //
|
||||||
|
UserId any //
|
||||||
|
RoleId any //
|
||||||
|
}
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package entity
|
||||||
|
|
||||||
|
import (
|
||||||
|
"github.com/gogf/gf/v2/os/gtime"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysConfig is the golang structure for table sys_config.
|
||||||
|
type SysConfig struct {
|
||||||
|
ConfigId int64 `json:"configId" orm:"config_id" ` //
|
||||||
|
TenantId string `json:"tenantId" orm:"tenant_id" ` //
|
||||||
|
ConfigName string `json:"configName" orm:"config_name" ` //
|
||||||
|
ConfigKey string `json:"configKey" orm:"config_key" ` //
|
||||||
|
ConfigValue string `json:"configValue" orm:"config_value" ` //
|
||||||
|
ConfigType string `json:"configType" orm:"config_type" ` //
|
||||||
|
CreatedDept int64 `json:"createdDept" orm:"created_dept" ` //
|
||||||
|
CreatedBy int64 `json:"createdBy" orm:"created_by" ` //
|
||||||
|
CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" ` //
|
||||||
|
UpdatedBy int64 `json:"updatedBy" orm:"updated_by" ` //
|
||||||
|
UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" ` //
|
||||||
|
Remark string `json:"remark" orm:"remark" ` //
|
||||||
|
}
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package entity
|
||||||
|
|
||||||
|
import (
|
||||||
|
"github.com/gogf/gf/v2/os/gtime"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysPermissionRevision is the golang structure for table sys_permission_revision.
|
||||||
|
type SysPermissionRevision struct {
|
||||||
|
Id int `json:"id" orm:"id" ` //
|
||||||
|
Revision int64 `json:"revision" orm:"revision" ` //
|
||||||
|
UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" ` //
|
||||||
|
}
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package entity
|
||||||
|
|
||||||
|
import (
|
||||||
|
"github.com/gogf/gf/v2/os/gtime"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysRole is the golang structure for table sys_role.
|
||||||
|
type SysRole struct {
|
||||||
|
RoleId int64 `json:"roleId" orm:"role_id" ` //
|
||||||
|
TenantId string `json:"tenantId" orm:"tenant_id" ` //
|
||||||
|
DeptId int64 `json:"deptId" orm:"dept_id" ` //
|
||||||
|
RoleName string `json:"roleName" orm:"role_name" ` //
|
||||||
|
RoleKey string `json:"roleKey" orm:"role_key" ` //
|
||||||
|
RoleSort int64 `json:"roleSort" orm:"role_sort" ` //
|
||||||
|
DataScope string `json:"dataScope" orm:"data_scope" ` //
|
||||||
|
MenuCheckStrictly int64 `json:"menuCheckStrictly" orm:"menu_check_strictly" ` //
|
||||||
|
DeptCheckStrictly int64 `json:"deptCheckStrictly" orm:"dept_check_strictly" ` //
|
||||||
|
Status string `json:"status" orm:"status" ` //
|
||||||
|
IsPass int64 `json:"isPass" orm:"is_pass" ` //
|
||||||
|
SourceRoleId int64 `json:"sourceRoleId" orm:"source_role_id" ` //
|
||||||
|
SourceDeptId int64 `json:"sourceDeptId" orm:"source_dept_id" ` //
|
||||||
|
CreatedDept int64 `json:"createdDept" orm:"created_dept" ` //
|
||||||
|
CreatedBy int64 `json:"createdBy" orm:"created_by" ` //
|
||||||
|
CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" ` //
|
||||||
|
UpdatedBy int64 `json:"updatedBy" orm:"updated_by" ` //
|
||||||
|
UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" ` //
|
||||||
|
DeletedBy int64 `json:"deletedBy" orm:"deleted_by" ` //
|
||||||
|
DeletedAt *gtime.Time `json:"deletedAt" orm:"deleted_at" ` //
|
||||||
|
Remark string `json:"remark" orm:"remark" ` //
|
||||||
|
}
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package entity
|
||||||
|
|
||||||
|
// SysRoleDept is the golang structure for table sys_role_dept.
|
||||||
|
type SysRoleDept struct {
|
||||||
|
TenantId string `json:"tenantId" orm:"tenant_id" ` //
|
||||||
|
RoleId int64 `json:"roleId" orm:"role_id" ` //
|
||||||
|
DeptId int64 `json:"deptId" orm:"dept_id" ` //
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package entity
|
||||||
|
|
||||||
|
// SysRoleMenu is the golang structure for table sys_role_menu.
|
||||||
|
type SysRoleMenu struct {
|
||||||
|
TenantId string `json:"tenantId" orm:"tenant_id" ` //
|
||||||
|
RoleId int64 `json:"roleId" orm:"role_id" ` //
|
||||||
|
MenuId int64 `json:"menuId" orm:"menu_id" ` //
|
||||||
|
DataScope string `json:"dataScope" orm:"data_scope" ` //
|
||||||
|
}
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package entity
|
||||||
|
|
||||||
|
import (
|
||||||
|
"github.com/gogf/gf/v2/os/gtime"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SysRoleSyncTask is the golang structure for table sys_role_sync_task.
|
||||||
|
type SysRoleSyncTask struct {
|
||||||
|
TaskId string `json:"taskId" orm:"task_id" ` //
|
||||||
|
TenantId string `json:"tenantId" orm:"tenant_id" ` //
|
||||||
|
SourceRoleId int64 `json:"sourceRoleId" orm:"source_role_id" ` //
|
||||||
|
SourceDeptId int64 `json:"sourceDeptId" orm:"source_dept_id" ` //
|
||||||
|
ActorId int64 `json:"actorId" orm:"actor_id" ` //
|
||||||
|
Status string `json:"status" orm:"status" ` //
|
||||||
|
Attempts int `json:"attempts" orm:"attempts" ` //
|
||||||
|
NextAttemptAt int64 `json:"nextAttemptAt" orm:"next_attempt_at" ` //
|
||||||
|
LeaseUntil int64 `json:"leaseUntil" orm:"lease_until" ` //
|
||||||
|
LeaseToken string `json:"leaseToken" orm:"lease_token" ` //
|
||||||
|
Payload string `json:"payload" orm:"payload" ` //
|
||||||
|
LastError string `json:"lastError" orm:"last_error" ` //
|
||||||
|
TotalCount int `json:"totalCount" orm:"total_count" ` //
|
||||||
|
SuccessCount int `json:"successCount" orm:"success_count" ` //
|
||||||
|
FailedCount int `json:"failedCount" orm:"failed_count" ` //
|
||||||
|
CreatedAt *gtime.Time `json:"createdAt" orm:"created_at" ` //
|
||||||
|
UpdatedAt *gtime.Time `json:"updatedAt" orm:"updated_at" ` //
|
||||||
|
}
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
// =================================================================================
|
||||||
|
// Code generated and maintained by GoFrame CLI tool. DO NOT EDIT.
|
||||||
|
// =================================================================================
|
||||||
|
|
||||||
|
package entity
|
||||||
|
|
||||||
|
// SysUserRole is the golang structure for table sys_user_role.
|
||||||
|
type SysUserRole struct {
|
||||||
|
TenantId string `json:"tenantId" orm:"tenant_id" ` //
|
||||||
|
UserId int64 `json:"userId" orm:"user_id" ` //
|
||||||
|
RoleId int64 `json:"roleId" orm:"role_id" ` //
|
||||||
|
}
|
||||||
@@ -101,10 +101,16 @@ type SimCardStatusChangeOutputItem struct {
|
|||||||
type SimCardApnChangeInput = SimBaseReq[SimCardApnChangeInputItem]
|
type SimCardApnChangeInput = SimBaseReq[SimCardApnChangeInputItem]
|
||||||
|
|
||||||
type SimCardApnChangeInputItem struct {
|
type SimCardApnChangeInputItem struct {
|
||||||
OperType string `json:"operType"`
|
OperType string `json:"operType"`
|
||||||
ApnName string `json:"apnName"`
|
ApnName string `json:"apnName"`
|
||||||
|
SubmitOnly bool `json:"submitOnly,omitempty"`
|
||||||
|
QueryJobID string `json:"queryJobId,omitempty"`
|
||||||
|
OperationID string `json:"operationId,omitempty"`
|
||||||
|
RecoverOnly bool `json:"recoverOnly,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type SimCardApnChangeOutputItem struct {
|
type SimCardApnChangeOutputItem struct {
|
||||||
OrderNum string `json:"orderNum"`
|
OrderNum string `json:"orderNum"`
|
||||||
|
Pending bool `json:"pending,omitempty"`
|
||||||
|
Uncertain bool `json:"uncertain,omitempty"`
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,10 +12,10 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/container/gvar"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil"
|
||||||
"github.com/gogf/gf/v2/container/gvar"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// datasetScanner applies the SQL eligibility predicate before LIMIT; metadata
|
// datasetScanner applies the SQL eligibility predicate before LIMIT; metadata
|
||||||
|
|||||||
@@ -7,9 +7,9 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
|
||||||
_ "github.com/gogf/gf/contrib/drivers/sqlite/v2"
|
_ "github.com/gogf/gf/contrib/drivers/sqlite/v2"
|
||||||
"github.com/gogf/gf/v2/database/gdb"
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
||||||
)
|
)
|
||||||
|
|
||||||
// SQLite runs the SAME predicate and ordering with only the two MySQL hints
|
// SQLite runs the SAME predicate and ordering with only the two MySQL hints
|
||||||
|
|||||||
@@ -16,11 +16,11 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/database/gredis"
|
||||||
|
goredis "github.com/redis/go-redis/v9"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset"
|
queue "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil"
|
||||||
"github.com/gogf/gf/v2/database/gredis"
|
|
||||||
goredis "github.com/redis/go-redis/v9"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// This proxy withholds a real Redis request/reply without closing its TCP socket.
|
// This proxy withholds a real Redis request/reply without closing its TCP socket.
|
||||||
|
|||||||
@@ -12,9 +12,9 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/container/gvar"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil"
|
||||||
"github.com/gogf/gf/v2/container/gvar"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// datasetScanner applies the SQL eligibility predicate before LIMIT; metadata
|
// datasetScanner applies the SQL eligibility predicate before LIMIT; metadata
|
||||||
|
|||||||
@@ -7,9 +7,9 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
|
||||||
_ "github.com/gogf/gf/contrib/drivers/sqlite/v2"
|
_ "github.com/gogf/gf/contrib/drivers/sqlite/v2"
|
||||||
"github.com/gogf/gf/v2/database/gdb"
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
||||||
)
|
)
|
||||||
|
|
||||||
// SQLite runs the SAME predicate and ordering with only the two MySQL hints
|
// SQLite runs the SAME predicate and ordering with only the two MySQL hints
|
||||||
|
|||||||
@@ -2,8 +2,8 @@ package monthlyusage
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply"
|
|
||||||
"github.com/gogf/gf/v2/frame/g"
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply"
|
||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -3,11 +3,11 @@ package monthlyusage
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"github.com/gogf/gf/v2/container/gvar"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil"
|
||||||
"github.com/gogf/gf/v2/container/gvar"
|
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|||||||
@@ -12,10 +12,10 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/container/gvar"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/testutil"
|
||||||
"github.com/gogf/gf/v2/container/gvar"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// datasetScanner applies the SQL eligibility predicate before LIMIT; metadata
|
// datasetScanner applies the SQL eligibility predicate before LIMIT; metadata
|
||||||
|
|||||||
@@ -7,9 +7,9 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
|
||||||
_ "github.com/gogf/gf/contrib/drivers/sqlite/v2"
|
_ "github.com/gogf/gf/contrib/drivers/sqlite/v2"
|
||||||
"github.com/gogf/gf/v2/database/gdb"
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
||||||
)
|
)
|
||||||
|
|
||||||
// SQLite runs the SAME predicate and ordering with only the two MySQL hints
|
// SQLite runs the SAME predicate and ordering with only the two MySQL hints
|
||||||
|
|||||||
@@ -15,9 +15,9 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/errors/gerror"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
"github.com/gogf/gf/v2/errors/gerror"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// ErrRunNotFound reports that a run's metadata hash does not exist. Callers use
|
// ErrRunNotFound reports that a run's metadata hash does not exist. Callers use
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ import (
|
|||||||
|
|
||||||
batchdomain "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch"
|
batchdomain "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/batchfields"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity"
|
||||||
)
|
)
|
||||||
@@ -71,6 +72,11 @@ func (r *Repository) MarkProcessing(ctx context.Context, batchNo string, batchTy
|
|||||||
Update(); updateErr != nil {
|
Update(); updateErr != nil {
|
||||||
return gerror.Wrap(updateErr, "mark pending batch details running failed")
|
return gerror.Wrap(updateErr, "mark pending batch details running failed")
|
||||||
}
|
}
|
||||||
|
if batchType == 107 || batchType == 109 || batchType == 110 {
|
||||||
|
if _, insertErr := tx.Exec("INSERT INTO data_operation_parent (batch_no,ready) VALUES (?,0)", batchNo); insertErr != nil {
|
||||||
|
return gerror.Wrap(insertErr, "create durable APN parent marker failed")
|
||||||
|
}
|
||||||
|
}
|
||||||
claimed = true
|
claimed = true
|
||||||
return nil
|
return nil
|
||||||
})
|
})
|
||||||
@@ -114,7 +120,7 @@ func (r *Repository) MarkFailed(ctx context.Context, batchNo string, batchType i
|
|||||||
Data(do.BatchMain{
|
Data(do.BatchMain{
|
||||||
BatchAsyncStatus: batchdomain.AsyncStatusFailed,
|
BatchAsyncStatus: batchdomain.AsyncStatusFailed,
|
||||||
RunResult: batchdomain.RunResultFailed,
|
RunResult: batchdomain.RunResultFailed,
|
||||||
FailReason: reason,
|
FailReason: batchfields.FailureReason(reason),
|
||||||
EndTime: gtime.Now(),
|
EndTime: gtime.Now(),
|
||||||
}).
|
}).
|
||||||
Update()
|
Update()
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ import (
|
|||||||
|
|
||||||
batchdomain "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch"
|
batchdomain "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/batch"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/batchfields"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity"
|
||||||
)
|
)
|
||||||
@@ -150,6 +151,26 @@ func TestMarkFailedTruncatesDetailReason(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestMarkFailedKeepsMainReasonWithinTextColumn(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
repository := newTestRepository(t, ctx)
|
||||||
|
insertBatch(t, ctx, repository.db, "B-LONG", batchdomain.Type103)
|
||||||
|
if claimed, err := repository.MarkProcessing(ctx, "B-LONG", batchdomain.Type103); err != nil || !claimed {
|
||||||
|
t.Fatalf("claim batch = (%t, %v)", claimed, err)
|
||||||
|
}
|
||||||
|
reason := "database error: INSERT INTO batch_detail VALUES (" + strings.Repeat("错", 30000) + ")"
|
||||||
|
if marked, err := repository.MarkFailed(ctx, "B-LONG", batchdomain.Type103, reason); err != nil || !marked {
|
||||||
|
t.Fatalf("mark failed = (%t, %v)", marked, err)
|
||||||
|
}
|
||||||
|
var main entity.BatchMain
|
||||||
|
if err := repository.db.Model(dao.BatchMain.Table()).Ctx(ctx).Where(dao.BatchMain.Columns().BatchNo, "B-LONG").Scan(&main); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(main.FailReason) > batchfields.FailureReasonByteLimit || strings.Contains(main.FailReason, "VALUES") {
|
||||||
|
t.Fatalf("unsafe failure reason: %q", main.FailReason)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func newTestRepository(t *testing.T, ctx context.Context) *Repository {
|
func newTestRepository(t *testing.T, ctx context.Context) *Repository {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
db, err := gdb.New(gdb.ConfigNode{
|
db, err := gdb.New(gdb.ConfigNode{
|
||||||
@@ -192,6 +213,42 @@ func insertBatch(t *testing.T, ctx context.Context, db gdb.DB, batchNo string, b
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestMarkProcessingAPNMarkerIsAtomic(t *testing.T) {
|
||||||
|
for _, batchType := range []int{107, 109, 110} {
|
||||||
|
t.Run(string(rune('A'+batchType-107)), func(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
repo := newTestRepository(t, ctx)
|
||||||
|
insertBatch(t, ctx, repo.db, "APN", batchType)
|
||||||
|
claimed, err := repo.MarkProcessing(ctx, "APN", batchType)
|
||||||
|
if err == nil || claimed {
|
||||||
|
t.Fatalf("缺少迁移必须安全失败: claimed=%t err=%v", claimed, err)
|
||||||
|
}
|
||||||
|
assertStatus(t, ctx, repo.db, "batch_main", "batch_async_status", batchdomain.AsyncStatusPending)
|
||||||
|
main, err := repo.FindMain(ctx, "APN")
|
||||||
|
if err != nil || main.StartTime != nil {
|
||||||
|
t.Fatalf("失败未回滚首发时间: %+v %v", main, err)
|
||||||
|
}
|
||||||
|
pending, err := repo.db.Model("batch_detail").Ctx(ctx).Where("detail_status", batchdomain.DetailStatusPending).Count()
|
||||||
|
if err != nil || pending != 1 {
|
||||||
|
t.Fatalf("明细没有安全回滚: %d %v", pending, err)
|
||||||
|
}
|
||||||
|
if _, err := repo.db.Exec(ctx, "CREATE TABLE data_operation_parent(batch_no TEXT PRIMARY KEY,ready INTEGER DEFAULT 0)"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if claimed, err := repo.MarkProcessing(ctx, "APN", batchType); err != nil || !claimed {
|
||||||
|
t.Fatalf("新批领取: %t %v", claimed, err)
|
||||||
|
}
|
||||||
|
n, err := repo.db.Model("data_operation_parent").Ctx(ctx).Where("batch_no", "APN").Where("ready", 0).Count()
|
||||||
|
if err != nil || n != 1 {
|
||||||
|
t.Fatalf("新批缺少原子凭据: %d %v", n, err)
|
||||||
|
}
|
||||||
|
if claimed, err := repo.MarkProcessing(ctx, "APN", batchType); err != nil || claimed {
|
||||||
|
t.Fatalf("重投不应重复创建凭据: %t %v", claimed, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func assertStatus(t *testing.T, ctx context.Context, db gdb.DB, table, column string, expected int) {
|
func assertStatus(t *testing.T, ctx context.Context, db gdb.DB, table, column string, expected int) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
actual, err := db.Model(table).Ctx(ctx).Value(column)
|
actual, err := db.Model(table).Ctx(ctx).Value(column)
|
||||||
|
|||||||
@@ -230,7 +230,7 @@ func (r *Repository) SaveValidationFailure(ctx context.Context, task *cancelserv
|
|||||||
}
|
}
|
||||||
reason := task.FailureReason
|
reason := task.FailureReason
|
||||||
if reason == "" {
|
if reason == "" {
|
||||||
reason = "batch validation failed; cancellation was skipped"
|
reason = "批次校验未通过,已跳过取消操作"
|
||||||
}
|
}
|
||||||
now := gtime.Now()
|
now := gtime.Now()
|
||||||
return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error {
|
return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error {
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
"github.com/gogf/gf/v2/database/gdb"
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
"github.com/gogf/gf/v2/encoding/gjson"
|
"github.com/gogf/gf/v2/encoding/gjson"
|
||||||
@@ -13,6 +14,7 @@ import (
|
|||||||
|
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/batchfields"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult"
|
||||||
@@ -36,14 +38,30 @@ func (r *Repository) database() gdb.DB {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (r *Repository) Load(ctx context.Context, batchNo string) (*data.Task, error) {
|
func (r *Repository) Load(ctx context.Context, batchNo string) (*data.Task, error) {
|
||||||
|
return r.loadTask(ctx, batchNo, false)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Repository) loadTask(ctx context.Context, batchNo string, allDetails bool) (*data.Task, error) {
|
||||||
|
return r.loadTaskFiltered(ctx, batchNo, allDetails, nil)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Repository) loadTaskDetails(ctx context.Context, batchNo string, ids []int64) (*data.Task, error) {
|
||||||
|
return r.loadTaskFiltered(ctx, batchNo, true, ids)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Repository) loadTaskFiltered(ctx context.Context, batchNo string, allDetails bool, ids []int64) (*data.Task, error) {
|
||||||
if batchNo == "" {
|
if batchNo == "" {
|
||||||
return nil, gerror.New("data service batch number is empty")
|
return nil, gerror.New("data service batch number is empty")
|
||||||
}
|
}
|
||||||
batch := entity.BatchMain{}
|
batch := entity.BatchMain{}
|
||||||
|
statuses := []int{consts.BatchAsyncStatusProcessing}
|
||||||
|
if allDetails {
|
||||||
|
statuses = append(statuses, consts.BatchAsyncStatusReview)
|
||||||
|
}
|
||||||
if err := r.database().Model(dao.BatchMain.Table()).Safe().Ctx(ctx).
|
if err := r.database().Model(dao.BatchMain.Table()).Safe().Ctx(ctx).
|
||||||
Where(dao.BatchMain.Columns().BatchNo, batchNo).
|
Where(dao.BatchMain.Columns().BatchNo, batchNo).
|
||||||
WhereIn(dao.BatchMain.Columns().BatchType, []int{consts.BatchTypeDataCut, consts.BatchTypeDataStart, consts.BatchTypeDataStop}).
|
WhereIn(dao.BatchMain.Columns().BatchType, []int{consts.BatchTypeDataCut, consts.BatchTypeDataStart, consts.BatchTypeDataStop}).
|
||||||
Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing).
|
WhereIn(dao.BatchMain.Columns().BatchAsyncStatus, statuses).
|
||||||
Scan(&batch); err != nil {
|
Scan(&batch); err != nil {
|
||||||
return nil, gerror.Wrap(err, "query data service batch")
|
return nil, gerror.Wrap(err, "query data service batch")
|
||||||
}
|
}
|
||||||
@@ -56,16 +74,18 @@ func (r *Repository) Load(ctx context.Context, batchNo string) (*data.Task, erro
|
|||||||
}
|
}
|
||||||
|
|
||||||
var batchDetails []*entity.BatchDetail
|
var batchDetails []*entity.BatchDetail
|
||||||
if err = r.database().Model(dao.BatchDetail.Table()).Safe().Ctx(ctx).
|
detailQuery := r.database().Model(dao.BatchDetail.Table()).Safe().Ctx(ctx).
|
||||||
Where(dao.BatchDetail.Columns().BatchNo, batchNo).
|
Where(dao.BatchDetail.Columns().BatchNo, batchNo).
|
||||||
Where(dao.BatchDetail.Columns().BatchType, batch.BatchType).
|
Where(dao.BatchDetail.Columns().BatchType, batch.BatchType)
|
||||||
Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning).
|
if len(ids) > 0 {
|
||||||
OrderAsc(dao.BatchDetail.Columns().RowNo).
|
detailQuery = detailQuery.WhereIn("id", ids)
|
||||||
Scan(&batchDetails); err != nil {
|
|
||||||
return nil, gerror.Wrap(err, "query running data service details")
|
|
||||||
}
|
}
|
||||||
if len(batchDetails) == 0 {
|
if !allDetails {
|
||||||
return nil, gerror.New("data service batch has no running details")
|
detailQuery = detailQuery.Where(dao.BatchDetail.Columns().DetailStatus, consts.BatchDetailStatusRunning).
|
||||||
|
Where("NOT EXISTS (SELECT 1 FROM data_operation_sub_detail assigned WHERE assigned.detail_id = batch_detail.id)")
|
||||||
|
}
|
||||||
|
if err = detailQuery.OrderAsc(dao.BatchDetail.Columns().RowNo).Scan(&batchDetails); err != nil {
|
||||||
|
return nil, gerror.Wrap(err, "query running data service details")
|
||||||
}
|
}
|
||||||
|
|
||||||
task := &data.Task{
|
task := &data.Task{
|
||||||
@@ -88,10 +108,11 @@ func (r *Repository) Load(ctx context.Context, batchNo string) (*data.Task, erro
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
detail := &data.Detail{
|
detail := &data.Detail{
|
||||||
ID: batchDetail.Id,
|
ID: batchDetail.Id,
|
||||||
RowNo: batchDetail.RowNo,
|
DetailStatus: batchDetail.DetailStatus,
|
||||||
ICCID: strings.TrimSpace(batchDetail.ItemId),
|
RowNo: batchDetail.RowNo,
|
||||||
Reason: strings.TrimSpace(batchDetail.ErrorReason),
|
ICCID: strings.TrimSpace(batchDetail.ItemId),
|
||||||
|
Reason: strings.TrimSpace(batchDetail.ErrorReason),
|
||||||
}
|
}
|
||||||
if batchDetail.Snapshot != nil {
|
if batchDetail.Snapshot != nil {
|
||||||
if value := strings.TrimSpace(batchDetail.Snapshot.Get("ICCID").String()); value != "" {
|
if value := strings.TrimSpace(batchDetail.Snapshot.Get("ICCID").String()); value != "" {
|
||||||
@@ -102,14 +123,17 @@ func (r *Repository) Load(ctx context.Context, batchNo string) (*data.Task, erro
|
|||||||
detail.MSISDN = detail.InputMSISDN
|
detail.MSISDN = detail.InputMSISDN
|
||||||
detail.IMSI = detail.InputIMSI
|
detail.IMSI = detail.InputIMSI
|
||||||
detail.Authorized = batchDetail.Snapshot.Get("authorized").Bool()
|
detail.Authorized = batchDetail.Snapshot.Get("authorized").Bool()
|
||||||
|
detail.ValidationPassed = batchDetail.Snapshot.Get("validationPassed").Bool()
|
||||||
detail.ChanKey = strings.TrimSpace(batchDetail.Snapshot.Get("chanKey").String())
|
detail.ChanKey = strings.TrimSpace(batchDetail.Snapshot.Get("chanKey").String())
|
||||||
detail.TrafficCutoffCrossMonthHold = batchDetail.Snapshot.Get("trafficCutoffCrossMonthHold").Int()
|
detail.TrafficCutoffCrossMonthHold = batchDetail.Snapshot.Get("trafficCutoffCrossMonthHold").Int()
|
||||||
|
detail.OrderNum = batchDetail.Snapshot.Get("orderNum").String()
|
||||||
|
detail.UpstreamCode = batchDetail.Snapshot.Get("upstreamCode").Int()
|
||||||
|
detail.UpstreamReason = batchDetail.Snapshot.Get("upstreamMessage").String()
|
||||||
|
detail.StatusBefore = batchDetail.Snapshot.Get("statusBefore").Int()
|
||||||
|
detail.StatusAfter = batchDetail.Snapshot.Get("statusAfter").Int()
|
||||||
}
|
}
|
||||||
task.Details = append(task.Details, detail)
|
task.Details = append(task.Details, detail)
|
||||||
}
|
}
|
||||||
if len(task.Details) == 0 {
|
|
||||||
return nil, gerror.New("data service batch has no readable running details")
|
|
||||||
}
|
|
||||||
if err = r.loadCards(ctx, task); err != nil {
|
if err = r.loadCards(ctx, task); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -229,6 +253,7 @@ func (r *Repository) SaveValidationFailure(ctx context.Context, task *data.Task)
|
|||||||
if detail == nil {
|
if detail == nil {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
detail.DetailStatus = consts.BatchDetailStatusFailed
|
||||||
result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()).
|
result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()).
|
||||||
Where(dao.BatchDetail.Columns().Id, detail.ID).
|
Where(dao.BatchDetail.Columns().Id, detail.ID).
|
||||||
Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo).
|
Where(dao.BatchDetail.Columns().BatchNo, task.Batch.BatchNo).
|
||||||
@@ -263,7 +288,7 @@ func (r *Repository) SaveResult(ctx context.Context, task *data.Task, detail *da
|
|||||||
if resolved.Success {
|
if resolved.Success {
|
||||||
if detail.Card == nil {
|
if detail.Card == nil {
|
||||||
resolved.Success = false
|
resolved.Success = false
|
||||||
resolved.Message = "SIM card record is missing"
|
resolved.Message = "卡记录不存在,未办理"
|
||||||
} else {
|
} else {
|
||||||
result, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()).
|
result, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()).
|
||||||
Where(dao.TSimCardInfo.Columns().Id, detail.Card.ID).
|
Where(dao.TSimCardInfo.Columns().Id, detail.Card.ID).
|
||||||
@@ -332,16 +357,37 @@ func dataServiceUpdateData(task *data.Task, detail *data.Detail) do.TSimCardTraf
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (r *Repository) Finalize(ctx context.Context, task *data.Task) error {
|
func (r *Repository) Finalize(ctx context.Context, task *data.Task) error {
|
||||||
successCount := 0
|
successCount, unknownCount := 0, 0
|
||||||
for _, detail := range task.Details {
|
for _, detail := range task.Details {
|
||||||
if detail != nil && detail.Reason == "" {
|
if detail != nil && detail.DetailStatus == consts.BatchDetailStatusUnknown {
|
||||||
|
unknownCount++
|
||||||
|
} else if detail != nil && detail.DetailStatus == consts.BatchDetailStatusSuccess {
|
||||||
successCount++
|
successCount++
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
failCount := len(task.Details) - successCount
|
failCount := len(task.Details) - successCount - unknownCount
|
||||||
batchAsyncStatus, runResult := batchResult(successCount, failCount)
|
batchAsyncStatus, runResult := batchResult(successCount, failCount)
|
||||||
if task.FailureReason == "" && failCount > 0 {
|
if unknownCount > 0 {
|
||||||
task.FailureReason = strconv.Itoa(failCount) + " SIM cards failed"
|
batchAsyncStatus, runResult = consts.BatchAsyncStatusReview, consts.BatchRunResultReview
|
||||||
|
}
|
||||||
|
reasons := make([]string, 0, 3)
|
||||||
|
seenReasons := map[string]bool{}
|
||||||
|
for _, detail := range task.Details {
|
||||||
|
if detail != nil && detail.Reason != "" && !seenReasons[detail.Reason] && len(reasons) < 3 {
|
||||||
|
reasons = append(reasons, detail.Reason)
|
||||||
|
seenReasons[detail.Reason] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(task.Details) == 0 {
|
||||||
|
if task.FailureReason == "" {
|
||||||
|
task.FailureReason = "数据开停批次没有有效明细"
|
||||||
|
}
|
||||||
|
task.FailureReason = batchfields.FailureReason(task.FailureReason)
|
||||||
|
} else {
|
||||||
|
task.FailureReason = ""
|
||||||
|
if failCount > 0 || unknownCount > 0 {
|
||||||
|
task.FailureReason = batchfields.FailureReason("成功=" + strconv.Itoa(successCount) + ",明确失败=" + strconv.Itoa(failCount) + ",待核查=" + strconv.Itoa(unknownCount) + ";" + strings.Join(reasons, ";"))
|
||||||
|
}
|
||||||
}
|
}
|
||||||
validCount := 0
|
validCount := 0
|
||||||
if task.ValidationOK {
|
if task.ValidationOK {
|
||||||
@@ -353,6 +399,26 @@ func (r *Repository) Finalize(ctx context.Context, task *data.Task) error {
|
|||||||
}
|
}
|
||||||
now := gtime.Now()
|
now := gtime.Now()
|
||||||
err = r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error {
|
err = r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error {
|
||||||
|
var main struct {
|
||||||
|
Status int `orm:"batch_async_status"`
|
||||||
|
}
|
||||||
|
if err := r.subLock(tx.Ctx(ctx).Model(dao.BatchMain.Table()).Fields("batch_async_status").Where("batch_no", task.Batch.BatchNo)).Scan(&main); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if main.Status != consts.BatchAsyncStatusProcessing && !(ctx.Value(finalizeTokenKey{}) != nil && main.Status == consts.BatchAsyncStatusReview) {
|
||||||
|
return gerror.New("data service batch is no longer processing")
|
||||||
|
}
|
||||||
|
if token, ok := ctx.Value(finalizeTokenKey{}).(string); ok {
|
||||||
|
var lease struct {
|
||||||
|
Token string `orm:"finalize_token"`
|
||||||
|
}
|
||||||
|
if err := r.subLock(tx.Ctx(ctx).Model(parentTable).Fields("finalize_token").Where("batch_no", task.Batch.BatchNo).WhereGT("finalize_until", time.Now())).Scan(&lease); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if lease.Token != token {
|
||||||
|
return gerror.New("APN最终汇总租约已过期")
|
||||||
|
}
|
||||||
|
}
|
||||||
ossID, err := tx.Ctx(ctx).Model(dao.SysOss.Table()).Data(do.SysOss{
|
ossID, err := tx.Ctx(ctx).Model(dao.SysOss.Table()).Data(do.SysOss{
|
||||||
TenantId: task.Batch.TenantID,
|
TenantId: task.Batch.TenantID,
|
||||||
FileName: artifact.FileName,
|
FileName: artifact.FileName,
|
||||||
@@ -399,9 +465,13 @@ func (r *Repository) Finalize(ctx context.Context, task *data.Task) error {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return gerror.Wrap(err, "insert data service result download")
|
return gerror.Wrap(err, "insert data service result download")
|
||||||
}
|
}
|
||||||
|
allowedStatuses := []int{consts.BatchAsyncStatusProcessing}
|
||||||
|
if ctx.Value(finalizeTokenKey{}) != nil {
|
||||||
|
allowedStatuses = append(allowedStatuses, consts.BatchAsyncStatusReview)
|
||||||
|
}
|
||||||
result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()).
|
result, err := tx.Ctx(ctx).Model(dao.BatchMain.Table()).
|
||||||
Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo).
|
Where(dao.BatchMain.Columns().BatchNo, task.Batch.BatchNo).
|
||||||
Where(dao.BatchMain.Columns().BatchAsyncStatus, consts.BatchAsyncStatusProcessing).
|
WhereIn(dao.BatchMain.Columns().BatchAsyncStatus, allowedStatuses).
|
||||||
Data(do.BatchMain{
|
Data(do.BatchMain{
|
||||||
BatchAsyncStatus: batchAsyncStatus,
|
BatchAsyncStatus: batchAsyncStatus,
|
||||||
ValidCount: validCount,
|
ValidCount: validCount,
|
||||||
@@ -423,6 +493,10 @@ func (r *Repository) Finalize(ctx context.Context, task *data.Task) error {
|
|||||||
if affected != 1 {
|
if affected != 1 {
|
||||||
return gerror.New("data service batch is no longer processing")
|
return gerror.New("data service batch is no longer processing")
|
||||||
}
|
}
|
||||||
|
if token, ok := ctx.Value(finalizeTokenKey{}).(string); ok {
|
||||||
|
_, err = tx.Ctx(ctx).Model(parentTable).Where("batch_no", task.Batch.BatchNo).Where("finalize_token", token).Data(g.Map{"finalize_token": gdb.Raw("NULL"), "finalize_until": gdb.Raw("NULL")}).Update()
|
||||||
|
return err
|
||||||
|
}
|
||||||
return nil
|
return nil
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -432,6 +506,36 @@ func (r *Repository) Finalize(ctx context.Context, task *data.Task) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const subDetailOwnershipTable = "data_operation_sub_detail"
|
||||||
|
|
||||||
|
// markFrozenValidation 以“是否已被冻结进持久化子批”判定明细是否通过提交前校验。
|
||||||
|
// 只有通过校验的明细才会被冻结提交,因此冻结归属才是有效数的可信依据;
|
||||||
|
// authorized 只表示运营商授权范围,不能当作校验通过的证据。
|
||||||
|
func (r *Repository) markFrozenValidation(ctx context.Context, task *data.Task) error {
|
||||||
|
if task == nil || len(task.Details) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var rows []struct {
|
||||||
|
DetailID int64 `orm:"detail_id"`
|
||||||
|
}
|
||||||
|
if err := r.database().Model(subDetailOwnershipTable).Ctx(ctx).
|
||||||
|
Fields("detail_id").Where("batch_no", task.Batch.BatchNo).Scan(&rows); err != nil {
|
||||||
|
return gerror.Wrap(err, "query frozen data service detail ownership")
|
||||||
|
}
|
||||||
|
frozen := make(map[int64]struct{}, len(rows))
|
||||||
|
for _, row := range rows {
|
||||||
|
frozen[row.DetailID] = struct{}{}
|
||||||
|
}
|
||||||
|
for _, detail := range task.Details {
|
||||||
|
if detail == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
_, owned := frozen[detail.ID]
|
||||||
|
detail.ValidationPassed = owned
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
func validDetailCount(task *data.Task) int {
|
func validDetailCount(task *data.Task) int {
|
||||||
if task == nil {
|
if task == nil {
|
||||||
return 0
|
return 0
|
||||||
@@ -460,12 +564,13 @@ func artifactRequest(task *data.Task) batchresult.Request {
|
|||||||
}
|
}
|
||||||
rows = append(rows, batchresult.Row{
|
rows = append(rows, batchresult.Row{
|
||||||
RowNo: detail.RowNo,
|
RowNo: detail.RowNo,
|
||||||
|
DetailStatus: detail.DetailStatus,
|
||||||
ICCID: detail.ICCID,
|
ICCID: detail.ICCID,
|
||||||
MSISDN: detail.MSISDN,
|
MSISDN: detail.MSISDN,
|
||||||
IMSI: detail.IMSI,
|
IMSI: detail.IMSI,
|
||||||
Operation: operationLabel(task.Operation),
|
Operation: operationLabel(task.Operation),
|
||||||
ApnName: task.ApnName,
|
ApnName: task.ApnName,
|
||||||
Success: detail.Reason == "",
|
Success: detail.DetailStatus == consts.BatchDetailStatusSuccess,
|
||||||
Reason: detail.Reason,
|
Reason: detail.Reason,
|
||||||
UpstreamCode: detail.UpstreamCode,
|
UpstreamCode: detail.UpstreamCode,
|
||||||
OrderNum: detail.OrderNum,
|
OrderNum: detail.OrderNum,
|
||||||
@@ -497,6 +602,13 @@ func buildBatchInfo(task *data.Task, ossID, downloadID int64) *gjson.Json {
|
|||||||
info["operation"] = task.Operation.Name
|
info["operation"] = task.Operation.Name
|
||||||
info["operType"] = task.Operation.ApnOperType
|
info["operType"] = task.Operation.ApnOperType
|
||||||
info["apnName"] = task.ApnName
|
info["apnName"] = task.ApnName
|
||||||
|
unknownCount := 0
|
||||||
|
for _, detail := range task.Details {
|
||||||
|
if detail != nil && detail.DetailStatus == consts.BatchDetailStatusUnknown {
|
||||||
|
unknownCount++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
info["apnUnknownCount"] = unknownCount
|
||||||
info["resultFileOssId"] = ossID
|
info["resultFileOssId"] = ossID
|
||||||
info["resultDownloadId"] = downloadID
|
info["resultDownloadId"] = downloadID
|
||||||
return gjson.New(info)
|
return gjson.New(info)
|
||||||
@@ -504,10 +616,10 @@ func buildBatchInfo(task *data.Task, ossID, downloadID int64) *gjson.Json {
|
|||||||
|
|
||||||
func batchResult(successCount, failCount int) (int, int) {
|
func batchResult(successCount, failCount int) (int, int) {
|
||||||
switch {
|
switch {
|
||||||
case failCount == 0:
|
|
||||||
return consts.BatchAsyncStatusSuccess, consts.BatchRunResultSuccess
|
|
||||||
case successCount == 0:
|
case successCount == 0:
|
||||||
return consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed
|
return consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed
|
||||||
|
case failCount == 0:
|
||||||
|
return consts.BatchAsyncStatusSuccess, consts.BatchRunResultSuccess
|
||||||
default:
|
default:
|
||||||
return consts.BatchAsyncStatusPartialSuccess, consts.BatchRunResultPartialSuccess
|
return consts.BatchAsyncStatusPartialSuccess, consts.BatchRunResultPartialSuccess
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,237 @@
|
|||||||
|
package data
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
_ "github.com/gogf/gf/contrib/drivers/sqlite/v2"
|
||||||
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult"
|
||||||
|
service "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/data"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestAccuracyEmptyValidationBatchFinalizesFailed(t *testing.T) {
|
||||||
|
for _, batchType := range []int{107, 109, 110} {
|
||||||
|
t.Run(fmt.Sprint(batchType), func(t *testing.T) {
|
||||||
|
repo, generator, ctx := accuracyTestRepository(t, batchType)
|
||||||
|
if err := service.NewExecutor(repo, &accuracyNoUpstream{t: t}).Execute(ctx, "ACCURACY"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
row, err := repo.database().GetOne(ctx, "SELECT batch_async_status,run_result,valid_count,success_count,fail_count,fail_reason FROM batch_main WHERE batch_no='ACCURACY'")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if row["batch_async_status"].Int() != consts.BatchAsyncStatusFailed || row["run_result"].Int() != consts.BatchRunResultFailed {
|
||||||
|
t.Fatalf("零明细校验失败被误报成功: %+v", row)
|
||||||
|
}
|
||||||
|
if row["valid_count"].Int() != 0 || row["success_count"].Int() != 0 || row["fail_count"].Int() != 0 || row["fail_reason"].String() == "" {
|
||||||
|
t.Fatalf("空批次不得虚构行数或丢失失败原因: %+v", row)
|
||||||
|
}
|
||||||
|
if len(generator.requests) != 1 || len(generator.requests[0].Rows) != 0 {
|
||||||
|
t.Fatalf("空批次结果文件行数错误: %+v", generator.requests)
|
||||||
|
}
|
||||||
|
download, err := repo.database().GetOne(ctx, "SELECT total_count,success_count,fail_count FROM t_resource_batch_download WHERE batch_no='ACCURACY'")
|
||||||
|
if err != nil || download["total_count"].Int() != 0 || download["success_count"].Int() != 0 || download["fail_count"].Int() != 0 {
|
||||||
|
t.Fatalf("空批次下载统计错误: %+v err=%v", download, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAccuracyAuthorizedSnapshotDoesNotPassValidation(t *testing.T) {
|
||||||
|
repo, _, ctx := accuracyTestRepository(t, 110)
|
||||||
|
if _, err := repo.database().Exec(ctx, `INSERT INTO batch_detail (id,batch_no,batch_type,row_no,item_id,detail_status,snapshot) VALUES (1,'ACCURACY',110,1,'',1,'{"authorized":true}')`); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
task, err := repo.Load(ctx, "ACCURACY")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(task.Details) != 1 || !task.Details[0].Authorized || task.Details[0].ValidationPassed {
|
||||||
|
t.Fatalf("授权标志不能被当成校验通过: %+v", task.Details)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAccuracyFrozenOwnershipDrivesValidCount(t *testing.T) {
|
||||||
|
repo, _, ctx := accuracyTestRepository(t, 110)
|
||||||
|
// 冻结归属才代表已通过提交前校验:授权标记不等于校验通过,
|
||||||
|
// 未被冻结进子批的明细(校验被拒)也不得计入有效数。
|
||||||
|
if _, err := repo.database().Exec(ctx, "INSERT INTO data_operation_sub_detail (detail_id,operation_id,batch_no,iccid) VALUES (1,'op-1','ACCURACY','frozen-card')"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
task := &service.Task{Batch: service.Batch{BatchNo: "ACCURACY", BatchType: 110}, Details: []*service.Detail{
|
||||||
|
{ID: 1, Authorized: true, DetailStatus: consts.BatchDetailStatusUnknown, Reason: "补查窗口耗尽,结果未知,禁止重发"},
|
||||||
|
{ID: 2, Authorized: true, DetailStatus: consts.BatchDetailStatusFailed, Reason: "ICCID does not exist in SIM card records"},
|
||||||
|
}}
|
||||||
|
if err := repo.markFrozenValidation(ctx, task); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !task.Details[0].ValidationPassed || task.Details[1].ValidationPassed {
|
||||||
|
t.Fatalf("冻结归属判定错误: %+v", task.Details)
|
||||||
|
}
|
||||||
|
task.ValidationOK = true
|
||||||
|
if count := validDetailCount(task); count != 1 {
|
||||||
|
t.Fatalf("恢复时有效数错误: got=%d want=1", count)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAccuracyFinalizeEmptyBatchPreservesValidationReason(t *testing.T) {
|
||||||
|
repo, _, ctx := accuracyTestRepository(t, 109)
|
||||||
|
operation, err := service.OperationByBatchType(109)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
task := &service.Task{Batch: service.Batch{BatchNo: "ACCURACY", BatchType: 109}, Operation: operation, FailureReason: "导入文件没有可执行行"}
|
||||||
|
if err := repo.Finalize(ctx, task); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
row, err := repo.database().GetOne(ctx, "SELECT batch_async_status,run_result,fail_reason FROM batch_main WHERE batch_no='ACCURACY'")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if row["batch_async_status"].Int() != consts.BatchAsyncStatusFailed || row["run_result"].Int() != consts.BatchRunResultFailed || row["fail_reason"].String() != "导入文件没有可执行行" || task.FailureReason != "导入文件没有可执行行" {
|
||||||
|
t.Fatalf("空批次原始校验原因被覆盖: %+v task=%+v", row, task)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAccuracyValidationFailureKeepsPerRowReasons(t *testing.T) {
|
||||||
|
for _, batchType := range []int{107, 109, 110} {
|
||||||
|
t.Run(fmt.Sprint(batchType), func(t *testing.T) {
|
||||||
|
repo, generator, ctx := accuracyTestRepository(t, batchType)
|
||||||
|
operation, err := service.OperationByBatchType(batchType)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
task := &service.Task{Batch: service.Batch{BatchNo: "ACCURACY", BatchType: batchType}, Operation: operation, Details: []*service.Detail{
|
||||||
|
{ID: 1, RowNo: 1, ICCID: "same-card", Reason: "MSISDN does not match the SIM card record"},
|
||||||
|
{ID: 2, RowNo: 2, ICCID: "same-card", Reason: "IMSI does not match the SIM card record"},
|
||||||
|
}}
|
||||||
|
for _, detail := range task.Details {
|
||||||
|
if _, err := repo.database().Exec(ctx, "INSERT INTO batch_detail (id,batch_no,batch_type,row_no,item_id,detail_status) VALUES (?,'ACCURACY',?,?,?,1)", detail.ID, batchType, detail.RowNo, detail.ICCID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := repo.SaveValidationFailure(ctx, task); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := repo.Finalize(ctx, task); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(generator.requests) != 1 || len(generator.requests[0].Rows) != len(task.Details) {
|
||||||
|
t.Fatalf("逐行导出缺失: %+v", generator.requests)
|
||||||
|
}
|
||||||
|
for i, detail := range task.Details {
|
||||||
|
row, err := repo.database().GetOne(ctx, "SELECT detail_status,error_reason FROM batch_detail WHERE id=?", detail.ID)
|
||||||
|
if err != nil || row["detail_status"].Int() != consts.BatchDetailStatusFailed || row["error_reason"].String() != detail.Reason {
|
||||||
|
t.Fatalf("逐行原因落库错误: row=%+v detail=%+v err=%v", row, detail, err)
|
||||||
|
}
|
||||||
|
exported := generator.requests[0].Rows[i]
|
||||||
|
if exported.Success || exported.DetailStatus != consts.BatchDetailStatusFailed || exported.Reason != detail.Reason {
|
||||||
|
t.Fatalf("逐行原因导出被改写: %+v", exported)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAccuracyFinalizeUnknownStillRequiresReview(t *testing.T) {
|
||||||
|
for _, statuses := range [][]int{{consts.BatchDetailStatusUnknown}, {consts.BatchDetailStatusSuccess, consts.BatchDetailStatusUnknown}, {consts.BatchDetailStatusFailed, consts.BatchDetailStatusUnknown}} {
|
||||||
|
t.Run(fmt.Sprint(statuses), func(t *testing.T) {
|
||||||
|
repo, _, ctx := accuracyTestRepository(t, 110)
|
||||||
|
task := &service.Task{Batch: service.Batch{BatchNo: "ACCURACY", BatchType: 110}}
|
||||||
|
for i, status := range statuses {
|
||||||
|
task.Details = append(task.Details, &service.Detail{ID: int64(i + 1), DetailStatus: status, Reason: "该卡结果待核查"})
|
||||||
|
}
|
||||||
|
if err := repo.Finalize(ctx, task); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
row, err := repo.database().GetOne(ctx, "SELECT batch_async_status,run_result,fail_count,batch_info FROM batch_main WHERE batch_no='ACCURACY'")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
wantFailed := 0
|
||||||
|
if statuses[0] == consts.BatchDetailStatusFailed {
|
||||||
|
wantFailed = 1
|
||||||
|
}
|
||||||
|
if row["batch_async_status"].Int() != consts.BatchAsyncStatusReview || row["run_result"].Int() != consts.BatchRunResultReview || row["fail_count"].Int() != wantFailed {
|
||||||
|
t.Fatalf("未知被误计成功或明确失败: %+v", row)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAccuracyBatchResultCounts(t *testing.T) {
|
||||||
|
for _, tc := range []struct {
|
||||||
|
success, failed int
|
||||||
|
status, result int
|
||||||
|
}{{0, 0, consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed}, {0, 2, consts.BatchAsyncStatusFailed, consts.BatchRunResultFailed}, {2, 0, consts.BatchAsyncStatusSuccess, consts.BatchRunResultSuccess}, {1, 1, consts.BatchAsyncStatusPartialSuccess, consts.BatchRunResultPartialSuccess}} {
|
||||||
|
t.Run(fmt.Sprintf("success=%d/failed=%d", tc.success, tc.failed), func(t *testing.T) {
|
||||||
|
status, result := batchResult(tc.success, tc.failed)
|
||||||
|
if status != tc.status || result != tc.result {
|
||||||
|
t.Fatalf("汇总结果=(%d,%d),期望=(%d,%d)", status, result, tc.status, tc.result)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func accuracyTestRepository(t *testing.T, batchType int) (*Repository, *accuracyArtifactGenerator, context.Context) {
|
||||||
|
t.Helper()
|
||||||
|
ctx := context.Background()
|
||||||
|
db, err := gdb.New(gdb.ConfigNode{Type: "sqlite", Link: "sqlite::@file(" + filepath.ToSlash(filepath.Join(t.TempDir(), "accuracy.db")) + ")"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
if err := db.Close(ctx); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
for _, statement := range []string{
|
||||||
|
"CREATE TABLE batch_main (id INTEGER PRIMARY KEY,batch_no TEXT,batch_type INTEGER,batch_async_status INTEGER,tenant_id TEXT,start_time DATETIME,created_by INTEGER,created_dept INTEGER,valid_count INTEGER,success_count INTEGER,fail_count INTEGER,run_result INTEGER,fail_reason TEXT,batch_info TEXT,end_time DATETIME,updated_by INTEGER)",
|
||||||
|
"CREATE TABLE batch_detail (id INTEGER PRIMARY KEY,batch_no TEXT,batch_type INTEGER,row_no INTEGER,item_id TEXT,detail_status INTEGER,error_code TEXT,error_reason TEXT,snapshot TEXT,updated_by INTEGER)",
|
||||||
|
"CREATE TABLE data_operation_parent (batch_no TEXT PRIMARY KEY,ready INTEGER DEFAULT 0,finalize_token TEXT,finalize_until DATETIME,last_reason TEXT)",
|
||||||
|
"CREATE TABLE data_operation_sub_batch (id INTEGER PRIMARY KEY,batch_no TEXT,status TEXT)",
|
||||||
|
"CREATE TABLE data_operation_sub_detail (detail_id INTEGER PRIMARY KEY, operation_id TEXT, batch_no TEXT, iccid TEXT, UNIQUE(batch_no,iccid))",
|
||||||
|
"CREATE TABLE sys_oss (id INTEGER PRIMARY KEY AUTOINCREMENT,tenant_id TEXT,file_name TEXT,original_name TEXT,file_suffix TEXT,path TEXT,url TEXT,created_dept INTEGER,created_by INTEGER,updated_by INTEGER,service TEXT,md5 TEXT,file_size INTEGER)",
|
||||||
|
"CREATE TABLE t_resource_batch_download (id INTEGER PRIMARY KEY AUTOINCREMENT,batch_no TEXT,task_name TEXT,resource_type INTEGER,export_scene INTEGER,total_count INTEGER,success_count INTEGER,fail_count INTEGER,download_status INTEGER,oss_id INTEGER,file_name TEXT,file_ext TEXT,file_url TEXT,file_path TEXT,file_size INTEGER,file_md5 TEXT,download_count INTEGER,max_download_count INTEGER,dept_id INTEGER,start_time DATETIME,finish_time DATETIME,tenant_id TEXT,created_dept INTEGER,created_by INTEGER,updated_by INTEGER)",
|
||||||
|
} {
|
||||||
|
if _, err := db.Exec(ctx, statement); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := db.Exec(ctx, "INSERT INTO batch_main (id,batch_no,batch_type,batch_async_status,tenant_id) VALUES (1,'ACCURACY',?,2,'isolated')", batchType); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := db.Exec(ctx, "INSERT INTO data_operation_parent (batch_no,ready) VALUES ('ACCURACY',0)"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
generator := &accuracyArtifactGenerator{}
|
||||||
|
return &Repository{db: db, resultFiles: generator}, generator, ctx
|
||||||
|
}
|
||||||
|
|
||||||
|
type accuracyArtifactGenerator struct{ requests []batchresult.Request }
|
||||||
|
|
||||||
|
func (g *accuracyArtifactGenerator) Create(_ context.Context, request batchresult.Request) (batchresult.Artifact, error) {
|
||||||
|
g.requests = append(g.requests, request)
|
||||||
|
return batchresult.Artifact{FileName: "accuracy.xlsx", OriginalName: request.BatchNo + "-result.xlsx", FileSuffix: ".xlsx", Path: "accuracy.xlsx", URL: "/accuracy.xlsx"}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type accuracyNoUpstream struct{ t *testing.T }
|
||||||
|
|
||||||
|
func (u *accuracyNoUpstream) ChangeSimApnFunction(context.Context, *model.SimCardApnChangeInput) ([]model.SimBaseRes[model.SimCardApnChangeOutputItem], error) {
|
||||||
|
u.t.Fatal("隔离回归不得调用同步上游")
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
func (u *accuracyNoUpstream) SubmitSimApnFunction(context.Context, *model.SimCardApnChangeInput) ([]model.SimBaseRes[model.SimCardApnChangeOutputItem], error) {
|
||||||
|
u.t.Fatal("隔离回归不得提交上游")
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
func (u *accuracyNoUpstream) QuerySimApnFunctionResult(context.Context, *model.SimCardApnChangeInput) ([]model.SimBaseRes[model.SimCardApnChangeOutputItem], error) {
|
||||||
|
u.t.Fatal("隔离回归不得查询上游")
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,841 @@
|
|||||||
|
package data
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"reflect"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
"github.com/google/uuid"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/batchfields"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
|
data "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/data"
|
||||||
|
)
|
||||||
|
|
||||||
|
const subTable = "data_operation_sub_batch"
|
||||||
|
const claimTable = "data_operation_card_claim"
|
||||||
|
const parentTable = "data_operation_parent"
|
||||||
|
const subLease = 5 * time.Minute
|
||||||
|
|
||||||
|
type finalizeTokenKey struct{}
|
||||||
|
type preparingKey struct{}
|
||||||
|
|
||||||
|
func (r *Repository) subLock(q *gdb.Model) *gdb.Model {
|
||||||
|
if strings.EqualFold(r.database().GetConfig().Type, "sqlite") {
|
||||||
|
return q
|
||||||
|
}
|
||||||
|
return q.LockUpdate()
|
||||||
|
}
|
||||||
|
func (r *Repository) lockSubCards(ctx context.Context, tx gdb.TX, tenantID string, keys []*model.SimKey) ([]cardClaim, error) {
|
||||||
|
keys = append([]*model.SimKey(nil), keys...)
|
||||||
|
sort.Slice(keys, func(i, j int) bool { return keys[i].ICCID < keys[j].ICCID })
|
||||||
|
claims := make([]cardClaim, len(keys))
|
||||||
|
for i, key := range keys {
|
||||||
|
if err := r.subLock(tx.Ctx(ctx).Model(claimTable).Where("tenant_id", tenantID).Where("iccid", key.ICCID)).Scan(&claims[i]); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return claims, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func noRows(err error) error {
|
||||||
|
if errors.Is(err, sql.ErrNoRows) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
func (r *Repository) ignoreInsert(ctx context.Context, tx gdb.TX, table string, values g.Map) error {
|
||||||
|
_, err := tx.Ctx(ctx).Model(table).Data(values).InsertIgnore()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
func (r *Repository) durableTx(ctx context.Context, fn func(context.Context, gdb.TX) error) error {
|
||||||
|
var err error
|
||||||
|
for attempt := 0; attempt < 3; attempt++ {
|
||||||
|
err = r.database().Transaction(ctx, fn)
|
||||||
|
if err == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
text := err.Error()
|
||||||
|
if !strings.Contains(text, "1213") && !strings.Contains(text, "1205") && !strings.Contains(text, "database is locked") {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return ctx.Err()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
type subRow struct {
|
||||||
|
ID int64 `orm:"id"`
|
||||||
|
BatchNo string `orm:"batch_no"`
|
||||||
|
BatchType int `orm:"batch_type"`
|
||||||
|
ChanKey string `orm:"chan_key"`
|
||||||
|
OperationID string `orm:"operation_id"`
|
||||||
|
DetailIDs string `orm:"detail_ids"`
|
||||||
|
RequestData string `orm:"request_data"`
|
||||||
|
JobID string `orm:"job_id"`
|
||||||
|
Status string `orm:"status"`
|
||||||
|
PollCount int `orm:"poll_count"`
|
||||||
|
SubmittedAt *time.Time `orm:"submitted_at"`
|
||||||
|
NextCheckAt *time.Time `orm:"next_check_at"`
|
||||||
|
LeaseToken string `orm:"lease_token"`
|
||||||
|
LeaseUntil *time.Time `orm:"lease_until"`
|
||||||
|
CreatedAt *time.Time `orm:"created_at"`
|
||||||
|
}
|
||||||
|
type cardClaim struct {
|
||||||
|
BatchID int64 `orm:"batch_id"`
|
||||||
|
OperationID string `orm:"operation_id"`
|
||||||
|
ActiveBatchID int64 `orm:"active_batch_id"`
|
||||||
|
ActiveOperationID string `orm:"active_operation_id"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func decodeSub(row subRow) (data.SubBatch, error) {
|
||||||
|
out := data.SubBatch{ID: row.ID, BatchNo: row.BatchNo, BatchType: row.BatchType, ChanKey: row.ChanKey, OperationID: row.OperationID, JobID: row.JobID, Status: row.Status, PollCount: row.PollCount, LeaseToken: row.LeaseToken}
|
||||||
|
if row.SubmittedAt != nil {
|
||||||
|
out.SubmittedAt = *row.SubmittedAt
|
||||||
|
}
|
||||||
|
if row.NextCheckAt != nil {
|
||||||
|
out.NextCheckAt = *row.NextCheckAt
|
||||||
|
}
|
||||||
|
if row.LeaseUntil != nil {
|
||||||
|
out.LeaseUntil = *row.LeaseUntil
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal([]byte(row.DetailIDs), &out.DetailIDs); err != nil {
|
||||||
|
return out, err
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal([]byte(row.RequestData), &out.Request); err != nil {
|
||||||
|
return out, err
|
||||||
|
}
|
||||||
|
if len(out.DetailIDs) == 0 || len(out.DetailIDs) != len(out.Request.SimKeys) {
|
||||||
|
return out, fmt.Errorf("APN冻结请求与明细数不匹配: %s", row.OperationID)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Repository) HasDurableParent(ctx context.Context, batchNo string) (bool, error) {
|
||||||
|
n, err := r.database().Model(parentTable).Ctx(ctx).Where("batch_no", batchNo).Count()
|
||||||
|
return n == 1, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// ResumeOrPrepare只恢复由Pending领取事务写入的凭据,绝不接管无映射的历史同步批。
|
||||||
|
func (r *Repository) ResumeOrPrepare(ctx context.Context, batchNo string, prepare func(context.Context) error) error {
|
||||||
|
if ctx.Value(preparingKey{}) != nil {
|
||||||
|
return prepare(ctx)
|
||||||
|
}
|
||||||
|
err := r.durableTx(ctx, func(ctx context.Context, tx gdb.TX) error {
|
||||||
|
var main struct {
|
||||||
|
ID int64 `orm:"id"`
|
||||||
|
Status int `orm:"batch_async_status"`
|
||||||
|
}
|
||||||
|
if err := noRows(r.subLock(tx.Ctx(ctx).Model(dao.BatchMain.Table()).Fields("id,batch_async_status").Where("batch_no", batchNo)).Scan(&main)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if main.ID == 0 || main.Status != consts.BatchAsyncStatusProcessing {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var parent *struct {
|
||||||
|
Ready int `orm:"ready"`
|
||||||
|
}
|
||||||
|
if err := noRows(r.subLock(tx.Ctx(ctx).Model(parentTable).Where("batch_no", batchNo)).Scan(&parent)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if parent == nil {
|
||||||
|
return fmt.Errorf("历史APN批次缺少持久化凭据,须人工核查,禁止自动提交: %s", batchNo)
|
||||||
|
}
|
||||||
|
if parent.Ready == 1 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err := prepare(context.WithValue(ctx, preparingKey{}, true)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
_, err := tx.Ctx(ctx).Model(parentTable).Where("batch_no", batchNo).Data(g.Map{"ready": 1}).Update()
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return r.FinalizeReadyBatch(ctx, batchNo)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Repository) PrepareSubBatch(ctx context.Context, batch data.SubBatch) error {
|
||||||
|
return r.durableTx(ctx, func(ctx context.Context, tx gdb.TX) error {
|
||||||
|
var main struct {
|
||||||
|
ID int64 `orm:"id"`
|
||||||
|
TenantID string `orm:"tenant_id"`
|
||||||
|
}
|
||||||
|
if err := r.subLock(tx.Ctx(ctx).Model(dao.BatchMain.Table()).Fields("id,tenant_id").Where("batch_no", batch.BatchNo).Where("batch_async_status", consts.BatchAsyncStatusProcessing)).Scan(&main); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if main.ID == 0 || len(batch.DetailIDs) == 0 || len(batch.DetailIDs) > 100 || len(batch.DetailIDs) != len(batch.Request.SimKeys) {
|
||||||
|
return fmt.Errorf("APN准备参数或父批状态不合法")
|
||||||
|
}
|
||||||
|
keys := append([]*model.SimKey(nil), batch.Request.SimKeys...)
|
||||||
|
for _, key := range keys {
|
||||||
|
if key == nil {
|
||||||
|
return fmt.Errorf("APN卡标识为空")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Slice(keys, func(i, j int) bool { return keys[i].ICCID < keys[j].ICCID })
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for _, key := range keys {
|
||||||
|
if key == nil || key.ICCID == "" || key.ChanKey != batch.ChanKey || seen[key.ICCID] {
|
||||||
|
return fmt.Errorf("APN子批卡重复或跨通道")
|
||||||
|
}
|
||||||
|
seen[key.ICCID] = true
|
||||||
|
if err := r.ignoreInsert(ctx, tx, claimTable, g.Map{"tenant_id": main.TenantID, "iccid": key.ICCID, "batch_id": 0, "operation_id": "", "active_batch_id": 0, "active_operation_id": ""}); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var claim cardClaim
|
||||||
|
if err := r.subLock(tx.Ctx(ctx).Model(claimTable).Where("tenant_id", main.TenantID).Where("iccid", key.ICCID)).Scan(&claim); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if claim.BatchID < main.ID {
|
||||||
|
if _, err := tx.Ctx(ctx).Model(claimTable).Where("tenant_id", main.TenantID).Where("iccid", key.ICCID).WhereLT("batch_id", main.ID).Data(g.Map{"batch_id": main.ID, "operation_id": batch.OperationID}).Update(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
} else if claim.BatchID == main.ID && claim.OperationID != batch.OperationID {
|
||||||
|
return fmt.Errorf("同批ICCID重复,禁止二次提交: %s", key.ICCID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var old subRow
|
||||||
|
if err := noRows(r.subLock(tx.Ctx(ctx).Model(subTable).Where("operation_id", batch.OperationID)).Scan(&old)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if old.ID != 0 {
|
||||||
|
stored, err := decodeSub(old)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if stored.BatchNo != batch.BatchNo || stored.ChanKey != batch.ChanKey || !reflect.DeepEqual(stored.Request, batch.Request) || !reflect.DeepEqual(stored.DetailIDs, batch.DetailIDs) {
|
||||||
|
return fmt.Errorf("operationId绑定了不同请求: %s", batch.OperationID)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
ids, err := json.Marshal(batch.DetailIDs)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
request, err := json.Marshal(batch.Request)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if _, err = tx.Ctx(ctx).Model(subTable).Data(g.Map{"batch_no": batch.BatchNo, "batch_type": batch.BatchType, "chan_key": batch.ChanKey, "operation_id": batch.OperationID, "detail_ids": string(ids), "request_data": string(request), "status": "prepared", "next_check_at": time.Now()}).Insert(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for i, detailID := range batch.DetailIDs {
|
||||||
|
if _, err := tx.Ctx(ctx).Model("data_operation_sub_detail").Data(g.Map{"detail_id": detailID, "operation_id": batch.OperationID, "batch_no": batch.BatchNo, "iccid": batch.Request.SimKeys[i].ICCID}).Insert(); err != nil {
|
||||||
|
return fmt.Errorf("明细已归属其他冻结意图: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Repository) GetSubBatch(ctx context.Context, operationID string) (*data.SubBatch, error) {
|
||||||
|
var row subRow
|
||||||
|
if err := noRows(r.database().Model(subTable).Ctx(ctx).Where("operation_id", operationID).Scan(&row)); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if row.ID == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
sub, err := decodeSub(row)
|
||||||
|
return &sub, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Repository) DelaySubBatchPrecheck(ctx context.Context, snapshot data.SubBatch, next time.Time, reason string) error {
|
||||||
|
if snapshot.Status == "done" || snapshot.Status == "review" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
q := r.database().Model(subTable).Ctx(ctx).Where("operation_id", snapshot.OperationID).Where("status", snapshot.Status).Where("COALESCE(lease_token,'')=?", snapshot.LeaseToken).Where("lease_until IS NULL OR lease_until<=?", time.Now())
|
||||||
|
if snapshot.NextCheckAt.IsZero() {
|
||||||
|
q = q.WhereNull("next_check_at")
|
||||||
|
} else {
|
||||||
|
q = q.Where("next_check_at", snapshot.NextCheckAt)
|
||||||
|
}
|
||||||
|
if snapshot.LeaseUntil.IsZero() {
|
||||||
|
q = q.WhereNull("lease_until")
|
||||||
|
} else {
|
||||||
|
q = q.Where("lease_until", snapshot.LeaseUntil)
|
||||||
|
}
|
||||||
|
_, err := q.Data(g.Map{"next_check_at": next, "last_reason": truncateReason("只读预检未就绪:"+reason, 1024)}).Update()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Repository) ReleaseUnsentSubBatch(ctx context.Context, batch data.SubBatch, reason string) error {
|
||||||
|
return r.durableTx(ctx, func(ctx context.Context, tx gdb.TX) error {
|
||||||
|
var owner struct {
|
||||||
|
TenantID string `orm:"tenant_id"`
|
||||||
|
}
|
||||||
|
if err := tx.Ctx(ctx).Model(dao.BatchMain.Table()).Fields("tenant_id").Where("batch_no", batch.BatchNo).Scan(&owner); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if _, err := r.lockSubCards(ctx, tx, owner.TenantID, batch.Request.SimKeys); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var row subRow
|
||||||
|
if err := r.subLock(tx.Ctx(ctx).Model(subTable).Where("id", batch.ID)).Scan(&row); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if row.LeaseToken != batch.LeaseToken || row.LeaseUntil == nil || !row.LeaseUntil.After(time.Now()) {
|
||||||
|
return fmt.Errorf("未发送回退租约失效")
|
||||||
|
}
|
||||||
|
changes := g.Map{"status": batch.Status, "last_reason": truncateReason("确定未发送:"+reason, 1024), "next_check_at": time.Now().Add(15 * time.Second), "lease_token": gdb.Raw("NULL"), "lease_until": gdb.Raw("NULL")}
|
||||||
|
if batch.Status == "prepared" {
|
||||||
|
changes["submitted_at"] = gdb.Raw("NULL")
|
||||||
|
var main struct {
|
||||||
|
TenantID string `orm:"tenant_id"`
|
||||||
|
}
|
||||||
|
if err := tx.Ctx(ctx).Model(dao.BatchMain.Table()).Fields("tenant_id").Where("batch_no", batch.BatchNo).Scan(&main); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
keys := append([]*model.SimKey(nil), batch.Request.SimKeys...)
|
||||||
|
sort.Slice(keys, func(i, j int) bool { return keys[i].ICCID < keys[j].ICCID })
|
||||||
|
for _, key := range keys {
|
||||||
|
if _, err := tx.Ctx(ctx).Model(claimTable).Where("tenant_id", main.TenantID).Where("iccid", key.ICCID).Where("active_operation_id", batch.OperationID).Data(g.Map{"active_operation_id": "", "active_batch_id": 0}).Update(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else if batch.PollCount > 0 {
|
||||||
|
changes["poll_count"] = batch.PollCount - 1
|
||||||
|
}
|
||||||
|
_, err := tx.Ctx(ctx).Model(subTable).Where("id", batch.ID).Where("lease_token", batch.LeaseToken).Data(changes).Update()
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Repository) SubBatchChannel(ctx context.Context, operationID string) (string, error) {
|
||||||
|
row, err := r.database().GetOne(ctx, "SELECT chan_key FROM data_operation_sub_batch WHERE operation_id=?", operationID)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
if len(row) == 0 {
|
||||||
|
return "", fmt.Errorf("APN子批不存在")
|
||||||
|
}
|
||||||
|
return row["chan_key"].String(), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Repository) ListDueSubBatches(ctx context.Context, now time.Time, limit int) ([]string, error) {
|
||||||
|
var rows []struct {
|
||||||
|
OperationID string `orm:"operation_id"`
|
||||||
|
}
|
||||||
|
err := r.database().Model(subTable).Ctx(ctx).Fields("operation_id").WhereIn("status", []string{"prepared", "submitting", "accepted", "uncertain"}).WhereLTE("next_check_at", now).Where("lease_until IS NULL OR lease_until <= ?", now).Where("EXISTS (SELECT 1 FROM data_operation_parent p WHERE p.batch_no=data_operation_sub_batch.batch_no AND p.ready=1)").OrderAsc("next_check_at").Limit(limit).Scan(&rows)
|
||||||
|
ids := make([]string, 0, len(rows))
|
||||||
|
for _, row := range rows {
|
||||||
|
ids = append(ids, row.OperationID)
|
||||||
|
}
|
||||||
|
return ids, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Repository) ClaimSubBatch(ctx context.Context, operationID, token string, now time.Time) (*data.SubBatch, error) {
|
||||||
|
var out *data.SubBatch
|
||||||
|
err := r.durableTx(ctx, func(ctx context.Context, tx gdb.TX) error {
|
||||||
|
out = nil
|
||||||
|
var hint subRow
|
||||||
|
if err := noRows(tx.Ctx(ctx).Model(subTable).Where("operation_id", operationID).Scan(&hint)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if hint.ID == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var parent struct {
|
||||||
|
Ready int `orm:"ready"`
|
||||||
|
}
|
||||||
|
if err := noRows(r.subLock(tx.Ctx(ctx).Model(parentTable).Where("batch_no", hint.BatchNo)).Scan(&parent)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if parent.Ready != 1 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var main struct {
|
||||||
|
ID int64 `orm:"id"`
|
||||||
|
TenantID string `orm:"tenant_id"`
|
||||||
|
Status int `orm:"batch_async_status"`
|
||||||
|
}
|
||||||
|
if err := tx.Ctx(ctx).Model(dao.BatchMain.Table()).Fields("id,tenant_id,batch_async_status").Where("batch_no", hint.BatchNo).Scan(&main); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if main.Status != consts.BatchAsyncStatusProcessing {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
frozen, err := decodeSub(hint)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
claims, err := r.lockSubCards(ctx, tx, main.TenantID, frozen.Request.SimKeys)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var row subRow
|
||||||
|
if err := r.subLock(tx.Ctx(ctx).Model(subTable).Where("id", hint.ID)).Scan(&row); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if row.Status == "done" || row.Status == "review" || row.Status == "review_query" || (row.NextCheckAt != nil && row.NextCheckAt.After(now)) || (row.LeaseUntil != nil && row.LeaseUntil.After(now)) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
sub, err := decodeSub(row)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if row.Status == "prepared" {
|
||||||
|
keys := append([]*model.SimKey(nil), sub.Request.SimKeys...)
|
||||||
|
sort.Slice(keys, func(i, j int) bool { return keys[i].ICCID < keys[j].ICCID })
|
||||||
|
for _, claim := range claims {
|
||||||
|
if claim.ActiveOperationID != "" && claim.ActiveOperationID != operationID {
|
||||||
|
var previous subRow
|
||||||
|
if err := noRows(tx.Ctx(ctx).Model(subTable).Fields("status").Where("operation_id", claim.ActiveOperationID).Scan(&previous)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if previous.Status == "review" || (row.CreatedAt != nil && !row.CreatedAt.Add(15*time.Minute).After(now)) {
|
||||||
|
reason := "未提交:同卡前序办理仍在途或结果未知,须人工核对前序后重新发起"
|
||||||
|
if _, err := tx.Ctx(ctx).Model(subTable).Where("id", row.ID).Data(g.Map{"status": "review", "last_reason": reason, "next_check_at": gdb.Raw("NULL")}).Update(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
_, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()).WhereIn("id", sub.DetailIDs).Where("detail_status", consts.BatchDetailStatusRunning).Data(g.Map{"detail_status": consts.BatchDetailStatusUnknown, "error_code": "NOT_SUBMITTED", "error_reason": reason}).Update()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
_, err := tx.Ctx(ctx).Model(subTable).Where("id", row.ID).Data(g.Map{"last_reason": "同卡前序办理在途或结果未知,等待核对后才能提交", "next_check_at": now.Add(15 * time.Second)}).Update()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// 父批创建ID决定已冻结意图的队序;不能用可能交错的冻结ID替代。
|
||||||
|
for _, key := range keys {
|
||||||
|
query := "SELECT s.operation_id FROM batch_main m JOIN data_operation_sub_detail sd ON sd.batch_no=m.batch_no AND sd.iccid=? JOIN data_operation_sub_batch s ON s.operation_id=sd.operation_id JOIN batch_detail d ON d.id=sd.detail_id WHERE m.tenant_id=? AND m.id<? AND d.detail_status=? AND s.status IN ('prepared','submitting','accepted','uncertain') LIMIT 1"
|
||||||
|
if !strings.EqualFold(r.database().GetConfig().Type, "sqlite") {
|
||||||
|
// 父ID先限tenant和队序,再锁对应冻结映射;不锁父表,保留RR当前读。
|
||||||
|
query = strings.Replace(query, "SELECT s.operation_id FROM batch_main m", "SELECT STRAIGHT_JOIN s.operation_id FROM batch_main m FORCE INDEX (PRIMARY)", 1)
|
||||||
|
query += " FOR UPDATE OF sd,s,d"
|
||||||
|
}
|
||||||
|
previous, err := tx.GetAll(query, key.ICCID, main.TenantID, main.ID, consts.BatchDetailStatusRunning)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if len(previous) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if row.CreatedAt != nil && !row.CreatedAt.Add(15*time.Minute).After(now) {
|
||||||
|
reason := "未提交:等待同卡更早冻结意图超过窗口,须核对前序后重新发起"
|
||||||
|
if _, err := tx.Ctx(ctx).Model(subTable).Where("id", row.ID).Data(g.Map{"status": "review", "last_reason": reason, "next_check_at": gdb.Raw("NULL")}).Update(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
_, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()).WhereIn("id", sub.DetailIDs).Where("detail_status", consts.BatchDetailStatusRunning).Data(g.Map{"detail_status": consts.BatchDetailStatusUnknown, "error_code": "NOT_SUBMITTED", "error_reason": reason}).Update()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
_, err = tx.Ctx(ctx).Model(subTable).Where("id", row.ID).Data(g.Map{"last_reason": "同卡存在更早冻结的办理意图,按先后顺序等待提交", "next_check_at": now.Add(15 * time.Second)}).Update()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for _, key := range keys {
|
||||||
|
if _, err := tx.Ctx(ctx).Model(claimTable).Where("tenant_id", main.TenantID).Where("iccid", key.ICCID).Where("active_operation_id", "").Data(g.Map{"active_batch_id": main.ID, "active_operation_id": operationID}).Update(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else if row.PollCount >= 4 {
|
||||||
|
if _, err := tx.Ctx(ctx).Model(subTable).Where("id", row.ID).Data(g.Map{"status": "review", "last_reason": "补查窗口耗尽,结果未知,禁止重发", "next_check_at": gdb.Raw("NULL")}).Update(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
_, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()).WhereIn("id", sub.DetailIDs).Where("detail_status", consts.BatchDetailStatusRunning).Data(g.Map{"detail_status": consts.BatchDetailStatusUnknown, "error_code": "UNKNOWN", "error_reason": "补查窗口耗尽,结果未知,禁止重发"}).Update()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
changes := g.Map{"lease_token": token, "lease_until": now.Add(subLease), "next_check_at": now.Add(subLease)}
|
||||||
|
if row.Status == "prepared" {
|
||||||
|
changes["status"] = "submitting"
|
||||||
|
changes["submitted_at"] = now
|
||||||
|
sub.SubmittedAt = now
|
||||||
|
} else {
|
||||||
|
changes["poll_count"] = row.PollCount + 1
|
||||||
|
sub.PollCount++
|
||||||
|
}
|
||||||
|
result, err := tx.Ctx(ctx).Model(subTable).Where("id", row.ID).Where("status", row.Status).Where("lease_until IS NULL OR lease_until <= ?", now).Data(changes).Update()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
n, err := result.RowsAffected()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if n != 1 {
|
||||||
|
return fmt.Errorf("APN租约CAS失败")
|
||||||
|
}
|
||||||
|
sub.LeaseToken = token
|
||||||
|
out = &sub
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
return out, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Repository) ClaimReviewSubBatch(ctx context.Context, operationID, token string, now time.Time) (*data.SubBatch, error) {
|
||||||
|
var out *data.SubBatch
|
||||||
|
err := r.durableTx(ctx, func(ctx context.Context, tx gdb.TX) error {
|
||||||
|
out = nil
|
||||||
|
var hint subRow
|
||||||
|
if err := noRows(tx.Ctx(ctx).Model(subTable).Where("operation_id", operationID).Scan(&hint)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if hint.ID == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var main struct {
|
||||||
|
Status int `orm:"batch_async_status"`
|
||||||
|
}
|
||||||
|
if err := noRows(r.subLock(tx.Ctx(ctx).Model(dao.BatchMain.Table()).Fields("batch_async_status").Where("batch_no", hint.BatchNo)).Scan(&main)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if main.Status != consts.BatchAsyncStatusReview && main.Status != consts.BatchAsyncStatusProcessing {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var parent struct {
|
||||||
|
Ready int `orm:"ready"`
|
||||||
|
FinalizeUntil *time.Time `orm:"finalize_until"`
|
||||||
|
}
|
||||||
|
if err := noRows(r.subLock(tx.Ctx(ctx).Model(parentTable).Where("batch_no", hint.BatchNo)).Scan(&parent)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if parent.Ready != 1 || (parent.FinalizeUntil != nil && parent.FinalizeUntil.After(now)) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var row subRow
|
||||||
|
if err := r.subLock(tx.Ctx(ctx).Model(subTable).Where("id", hint.ID)).Scan(&row); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if (row.Status != "review" && row.Status != "review_query") || row.SubmittedAt == nil || (row.LeaseUntil != nil && row.LeaseUntil.After(now)) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
sub, err := decodeSub(row)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
result, err := tx.Ctx(ctx).Model(subTable).Where("id", row.ID).Where("status", row.Status).Where("lease_until IS NULL OR lease_until <= ?", now).Data(g.Map{"status": "review_query", "lease_token": token, "lease_until": now.Add(subLease)}).Update()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
n, err := result.RowsAffected()
|
||||||
|
if err != nil || n != 1 {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
sub.Status = "review_query"
|
||||||
|
sub.LeaseToken = token
|
||||||
|
sub.LeaseUntil = now.Add(subLease)
|
||||||
|
out = &sub
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
return out, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListReviewSubBatches 只列出该父批下已冻结、仍待核查的子批.
|
||||||
|
// 手动同步据此按既有 operationId 查询上游,不重发办理,也不重开自动补查窗口。
|
||||||
|
func (r *Repository) ListReviewSubBatches(ctx context.Context, batchNo string) ([]string, error) {
|
||||||
|
var rows []struct {
|
||||||
|
OperationID string `orm:"operation_id"`
|
||||||
|
}
|
||||||
|
if err := r.database().Model(subTable).Ctx(ctx).
|
||||||
|
Fields("operation_id").
|
||||||
|
Where("batch_no", batchNo).
|
||||||
|
WhereIn("status", []string{"review", "review_query"}).
|
||||||
|
Where("lease_until IS NULL OR lease_until <= ?", time.Now()).
|
||||||
|
Order("id").
|
||||||
|
Scan(&rows); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out := make([]string, 0, len(rows))
|
||||||
|
for _, row := range rows {
|
||||||
|
if id := strings.TrimSpace(row.OperationID); id != "" {
|
||||||
|
out = append(out, id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Repository) UpdateSubBatch(ctx context.Context, batch data.SubBatch, status, jobID, reason string, next time.Time, pollCount int) error {
|
||||||
|
return r.durableTx(ctx, func(ctx context.Context, tx gdb.TX) error {
|
||||||
|
var nextAt any = gdb.Raw("NULL")
|
||||||
|
if !next.IsZero() {
|
||||||
|
nextAt = next
|
||||||
|
}
|
||||||
|
result, err := tx.Ctx(ctx).Model(subTable).Where("id", batch.ID).Where("lease_token", batch.LeaseToken).WhereGT("lease_until", time.Now()).WhereIn("status", []string{"submitting", "accepted", "uncertain", "review_query"}).Data(g.Map{"status": status, "job_id": jobID, "last_reason": truncateReason(reason, 1024), "next_check_at": nextAt, "poll_count": pollCount, "lease_token": gdb.Raw("NULL"), "lease_until": gdb.Raw("NULL")}).Update()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
n, err := result.RowsAffected()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if n != 1 {
|
||||||
|
return fmt.Errorf("APN租约已失效: %s", batch.OperationID)
|
||||||
|
}
|
||||||
|
if status == "review" {
|
||||||
|
if _, err = tx.Ctx(ctx).Model(dao.BatchDetail.Table()).Where("batch_no", batch.BatchNo).WhereIn("id", batch.DetailIDs).WhereIn("detail_status", []int{consts.BatchDetailStatusRunning, consts.BatchDetailStatusUnknown}).Data(g.Map{"detail_status": consts.BatchDetailStatusUnknown, "error_code": "UNKNOWN", "error_reason": batchfields.DetailReason(reason)}).Update(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if status != "review" && status != "done" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
// 手动补查把 review_query 子批改成终态时父批可能已是待核查,需重新纳入汇总。
|
||||||
|
return r.releaseReviewParentWhenSettled(ctx, tx, batch.BatchNo)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
func truncateReason(value string, n int) string {
|
||||||
|
runes := []rune(value)
|
||||||
|
if len(runes) > n {
|
||||||
|
return string(runes[:n])
|
||||||
|
}
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Repository) SaveSubBatchProgress(ctx context.Context, batch data.SubBatch, outcomes []data.Outcome) error {
|
||||||
|
return r.saveProgress(ctx, batch, outcomes, false)
|
||||||
|
}
|
||||||
|
func (r *Repository) CompleteSubBatch(ctx context.Context, batch data.SubBatch, outcomes []data.Outcome) error {
|
||||||
|
return r.saveProgress(ctx, batch, outcomes, true)
|
||||||
|
}
|
||||||
|
func (r *Repository) saveProgress(ctx context.Context, batch data.SubBatch, outcomes []data.Outcome, complete bool) error {
|
||||||
|
if len(outcomes) != len(batch.DetailIDs) {
|
||||||
|
return fmt.Errorf("APN明细结果数量不符")
|
||||||
|
}
|
||||||
|
return r.durableTx(ctx, func(ctx context.Context, tx gdb.TX) error {
|
||||||
|
var owner struct {
|
||||||
|
TenantID string `orm:"tenant_id"`
|
||||||
|
}
|
||||||
|
if err := tx.Ctx(ctx).Model(dao.BatchMain.Table()).Fields("tenant_id").Where("batch_no", batch.BatchNo).Scan(&owner); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if _, err := r.lockSubCards(ctx, tx, owner.TenantID, batch.Request.SimKeys); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var current subRow
|
||||||
|
if err := r.subLock(tx.Ctx(ctx).Model(subTable).Where("id", batch.ID)).Scan(¤t); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if current.LeaseToken != batch.LeaseToken || current.LeaseUntil == nil || !current.LeaseUntil.After(time.Now()) || current.Status == "done" || current.Status == "review" {
|
||||||
|
return fmt.Errorf("APN进度租约失效")
|
||||||
|
}
|
||||||
|
if batch.JobID != "" {
|
||||||
|
if _, err := tx.Ctx(ctx).Model(subTable).Where("id", batch.ID).Where("lease_token", batch.LeaseToken).Data(g.Map{"job_id": batch.JobID}).Update(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
task, err := r.loadTaskDetails(ctx, batch.BatchNo, batch.DetailIDs)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
byID := map[int64]*data.Detail{}
|
||||||
|
for _, detail := range task.Details {
|
||||||
|
byID[detail.ID] = detail
|
||||||
|
}
|
||||||
|
order := make([]int, len(batch.DetailIDs))
|
||||||
|
for i := range order {
|
||||||
|
order[i] = i
|
||||||
|
}
|
||||||
|
sort.Slice(order, func(i, j int) bool {
|
||||||
|
return batch.Request.SimKeys[order[i]].ICCID < batch.Request.SimKeys[order[j]].ICCID
|
||||||
|
})
|
||||||
|
for _, i := range order {
|
||||||
|
outcome := outcomes[i]
|
||||||
|
if outcome.Pending {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
detail := byID[batch.DetailIDs[i]]
|
||||||
|
if detail == nil {
|
||||||
|
return fmt.Errorf("缺少APN明细")
|
||||||
|
}
|
||||||
|
if detail.DetailStatus != consts.BatchDetailStatusRunning && !(batch.Status == "review_query" && detail.DetailStatus == consts.BatchDetailStatusUnknown) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var claim cardClaim
|
||||||
|
if err := r.subLock(tx.Ctx(ctx).Model(claimTable).Where("tenant_id", task.Batch.TenantID).Where("iccid", detail.ICCID)).Scan(&claim); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
previousStatus := detail.DetailStatus
|
||||||
|
detail.ApplyOutcome(outcome)
|
||||||
|
detail.Reason = batchfields.DetailReason(detail.Reason)
|
||||||
|
if detail.Card != nil {
|
||||||
|
detail.StatusBefore = detail.Card.DataServiceStatus
|
||||||
|
detail.StatusAfter = detail.StatusBefore
|
||||||
|
}
|
||||||
|
status := consts.BatchDetailStatusFailed
|
||||||
|
if outcome.Success {
|
||||||
|
status = consts.BatchDetailStatusSuccess
|
||||||
|
detail.StatusAfter = task.Operation.TargetStatus
|
||||||
|
}
|
||||||
|
if outcome.Success && detail.Card != nil && claim.ActiveOperationID == batch.OperationID {
|
||||||
|
if _, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()).Where("id", detail.Card.ID).Where("tenant_id", task.Batch.TenantID).Data(dataServiceUpdateData(task, detail)).Update(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()).Where("id", detail.ID).Where("detail_status", previousStatus).Data(g.Map{"detail_status": status, "error_code": fmt.Sprint(outcome.Code), "error_reason": detail.Reason, "snapshot": detail.Snapshot(task.Operation, task.ApnName).String(), "updated_by": task.Batch.CreatedBy}).Update()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
n, err := result.RowsAffected()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if n != 1 {
|
||||||
|
return fmt.Errorf("APN明细终态CAS失败")
|
||||||
|
}
|
||||||
|
if _, err := tx.Ctx(ctx).Model(claimTable).Where("tenant_id", task.Batch.TenantID).Where("iccid", detail.ICCID).Where("active_operation_id", batch.OperationID).Data(g.Map{"active_operation_id": "", "active_batch_id": 0}).Update(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if complete {
|
||||||
|
n, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()).WhereIn("id", batch.DetailIDs).WhereIn("detail_status", []int{consts.BatchDetailStatusRunning, consts.BatchDetailStatusUnknown}).Count()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if n != 0 {
|
||||||
|
return fmt.Errorf("APN子批仍有未确认明细,不能完成")
|
||||||
|
}
|
||||||
|
if _, err = tx.Ctx(ctx).Model(subTable).Where("id", batch.ID).Where("lease_token", batch.LeaseToken).Data(g.Map{"status": "done", "job_id": batch.JobID, "next_check_at": gdb.Raw("NULL"), "lease_token": gdb.Raw("NULL"), "lease_until": gdb.Raw("NULL")}).Update(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return r.releaseReviewParentWhenSettled(ctx, tx, batch.BatchNo)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// releaseReviewParentWhenSettled 在同一事务里:当子批转入终态、且该父批已无未终结子批时,
|
||||||
|
// 把父批从“待核查”退回“处理中”,交给既有恢复扫描重新触发汇总。
|
||||||
|
// 修复“子批与明细都已终态,但汇总失败或进程退出后父批永久挂在待核查”的死角。
|
||||||
|
// 只在确有终态转移处调用,因此合法的待核查批次不会被反复捞起空转。
|
||||||
|
// 状态读取不加锁,最后用带状态的 CAS 更新收口,避免在已持有卡/子批锁的事务里再锁父批造成锁序反转。
|
||||||
|
func (r *Repository) releaseReviewParentWhenSettled(ctx context.Context, tx gdb.TX, batchNo string) error {
|
||||||
|
var main struct {
|
||||||
|
Status int `orm:"batch_async_status"`
|
||||||
|
}
|
||||||
|
if err := noRows(tx.Ctx(ctx).Model(dao.BatchMain.Table()).Fields("batch_async_status").Where("batch_no", batchNo).Scan(&main)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if main.Status != consts.BatchAsyncStatusReview {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
// 只有已完成准备(ready=1)的父批才允许回到处理中,避免恢复扫描重跑准备阶段产生重复提交。
|
||||||
|
var parent struct {
|
||||||
|
Ready int `orm:"ready"`
|
||||||
|
}
|
||||||
|
if err := noRows(tx.Ctx(ctx).Model(parentTable).Fields("ready").Where("batch_no", batchNo).Scan(&parent)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if parent.Ready != 1 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
// 待核查父批的子批只可能是终态或在途手动补查:在途补查不阻塞退回,
|
||||||
|
// 因为 FinalizeReadyBatch 会再校验一次;不依赖一致读快照,避免并发补查互相看不见对方的终态。
|
||||||
|
pending, err := tx.Ctx(ctx).Model(subTable).Where("batch_no", batchNo).WhereNotIn("status", []string{"done", "review", "review_query"}).Count()
|
||||||
|
if err != nil || pending != 0 {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
_, err = tx.Ctx(ctx).Model(dao.BatchMain.Table()).Where("batch_no", batchNo).Where("batch_async_status", consts.BatchAsyncStatusReview).Data(g.Map{"batch_async_status": consts.BatchAsyncStatusProcessing}).Update()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Repository) FinalizeReadyBatch(ctx context.Context, batchNo string) error {
|
||||||
|
token := uuid.NewString()
|
||||||
|
claimed := false
|
||||||
|
err := r.durableTx(ctx, func(ctx context.Context, tx gdb.TX) error {
|
||||||
|
claimed = false
|
||||||
|
var main struct {
|
||||||
|
Status int `orm:"batch_async_status"`
|
||||||
|
}
|
||||||
|
if err := noRows(r.subLock(tx.Ctx(ctx).Model(dao.BatchMain.Table()).Fields("batch_async_status").Where("batch_no", batchNo)).Scan(&main)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if main.Status != consts.BatchAsyncStatusProcessing && main.Status != consts.BatchAsyncStatusReview {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var parent struct {
|
||||||
|
Ready int `orm:"ready"`
|
||||||
|
}
|
||||||
|
if err := noRows(r.subLock(tx.Ctx(ctx).Model(parentTable).Where("batch_no", batchNo)).Scan(&parent)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if parent.Ready != 1 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
n, err := tx.Ctx(ctx).Model(subTable).Where("batch_no", batchNo).WhereNotIn("status", []string{"done", "review"}).Count()
|
||||||
|
if err != nil || n != 0 {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
n, err = tx.Ctx(ctx).Model(dao.BatchDetail.Table()).Where("batch_no", batchNo).WhereIn("detail_status", []int{consts.BatchDetailStatusPending, consts.BatchDetailStatusRunning}).Count()
|
||||||
|
if err != nil || n != 0 {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
n, err = tx.Ctx(ctx).Model(dao.BatchMain.Table()).Where("batch_no", batchNo).WhereIn("batch_async_status", []int{consts.BatchAsyncStatusProcessing, consts.BatchAsyncStatusReview}).Count()
|
||||||
|
if err != nil || n == 0 {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
result, err := tx.Ctx(ctx).Model(parentTable).Where("batch_no", batchNo).Where("finalize_until IS NULL OR finalize_until <= ?", time.Now()).Data(g.Map{"finalize_token": token, "finalize_until": time.Now().Add(subLease)}).Update()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
count, err := result.RowsAffected()
|
||||||
|
claimed = count == 1
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
if err != nil || !claimed {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
finalCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 4*time.Minute)
|
||||||
|
defer cancel()
|
||||||
|
task, err := r.loadTask(finalCtx, batchNo, true)
|
||||||
|
if err == nil {
|
||||||
|
// 冻结归属决定有效数:只有通过校验的明细才会进入子批。
|
||||||
|
if err = r.markFrozenValidation(finalCtx, task); err == nil {
|
||||||
|
task.ValidationOK = true
|
||||||
|
err = r.Finalize(context.WithValue(finalCtx, finalizeTokenKey{}, token), task)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
_, _ = r.database().Model(parentTable).Ctx(finalCtx).Where("batch_no", batchNo).Where("finalize_token", token).Data(g.Map{"finalize_token": gdb.Raw("NULL"), "finalize_until": gdb.Raw("NULL"), "last_reason": truncateReason(err.Error(), 1024)}).Update()
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
func (r *Repository) ListRecoverableBatches(ctx context.Context, limit int) ([]string, error) {
|
||||||
|
var rows []struct {
|
||||||
|
BatchNo string `orm:"batch_no"`
|
||||||
|
}
|
||||||
|
// 未准备父批与可最终化父批轮转领取;在途父批交给子任务扫描,不挤占准备恢复窗口。
|
||||||
|
err := r.durableTx(ctx, func(ctx context.Context, tx gdb.TX) error {
|
||||||
|
rows = nil
|
||||||
|
q := tx.Ctx(ctx).Model(dao.BatchMain.Table()+" m").InnerJoin(parentTable+" p", "p.batch_no=m.batch_no").Fields("m.batch_no").WhereIn("m.batch_type", []int{107, 109, 110}).Where("m.batch_async_status", consts.BatchAsyncStatusProcessing).Where("m.start_time IS NULL OR m.start_time <= ?", time.Now().Add(-30*time.Second)).Where("p.ready=0 OR (NOT EXISTS (SELECT 1 FROM data_operation_sub_batch s WHERE s.batch_no=m.batch_no AND s.status NOT IN ('done','review')) AND NOT EXISTS (SELECT 1 FROM batch_detail d WHERE d.batch_no=m.batch_no AND d.detail_status IN (0,1)))").Order("COALESCE(p.recovery_attempt_at,'1970-01-01'),m.id").Limit(limit)
|
||||||
|
if err := q.Scan(&rows); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for _, row := range rows {
|
||||||
|
if _, err := tx.Ctx(ctx).Model(parentTable).Where("batch_no", row.BatchNo).Data(g.Map{"recovery_attempt_at": time.Now()}).Update(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
out := make([]string, 0, len(rows))
|
||||||
|
for _, row := range rows {
|
||||||
|
out = append(out, row.BatchNo)
|
||||||
|
}
|
||||||
|
return out, err
|
||||||
|
}
|
||||||
|
func (r *Repository) ProtectExecutionFailure(ctx context.Context, batchNo, reason string) error {
|
||||||
|
return r.durableTx(ctx, func(ctx context.Context, tx gdb.TX) error {
|
||||||
|
var main struct {
|
||||||
|
Status int `orm:"batch_async_status"`
|
||||||
|
}
|
||||||
|
if err := noRows(r.subLock(tx.Ctx(ctx).Model(dao.BatchMain.Table()).Fields("batch_async_status").Where("batch_no", batchNo)).Scan(&main)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if main.Status != consts.BatchAsyncStatusProcessing {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
_, err := tx.Ctx(ctx).Model(parentTable).Where("batch_no", batchNo).Data(g.Map{"last_reason": truncateReason("执行中断,交持久化调度恢复:"+reason, 1024)}).Update()
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
}
|
||||||
@@ -0,0 +1,523 @@
|
|||||||
|
package data
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"sync"
|
||||||
|
"sync/atomic"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
service "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/data"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestSubBatchPrecheckDelayCASNeverOverwritesClaimOrCompletion(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
b := testSub("S-1", "precheck-cas", "a", []int64{1}, []string{"A"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
snapshot, err := repo.GetSubBatch(ctx, b.OperationID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
next := time.Now().Add(30 * time.Second)
|
||||||
|
if err := repo.DelaySubBatchPrecheck(ctx, *snapshot, next, "能力拒绝"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
stored, err := repo.GetSubBatch(ctx, b.OperationID)
|
||||||
|
if err != nil || stored.NextCheckAt.Sub(next).Abs() > time.Millisecond {
|
||||||
|
t.Fatalf("退避未保存: %+v %v", stored, err)
|
||||||
|
}
|
||||||
|
if stored.Status != "prepared" || stored.PollCount != 0 || !stored.SubmittedAt.IsZero() || stored.LeaseToken != "" {
|
||||||
|
t.Fatalf("退避改变状态: %+v", stored)
|
||||||
|
}
|
||||||
|
if _, err := repo.database().Exec(ctx, "UPDATE data_operation_sub_batch SET next_check_at=NULL WHERE operation_id=?", b.OperationID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
snapshot, _ = repo.GetSubBatch(ctx, b.OperationID)
|
||||||
|
claimed, err := repo.ClaimSubBatch(ctx, b.OperationID, "owner", time.Now())
|
||||||
|
if err != nil || claimed == nil {
|
||||||
|
t.Fatalf("领取: %+v %v", claimed, err)
|
||||||
|
}
|
||||||
|
if err := repo.DelaySubBatchPrecheck(ctx, *snapshot, time.Now().Add(time.Hour), "陈旧拒绝"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
stored, _ = repo.GetSubBatch(ctx, b.OperationID)
|
||||||
|
if stored.Status != "submitting" || stored.LeaseToken != "owner" || stored.NextCheckAt.After(time.Now().Add(6*time.Minute)) {
|
||||||
|
t.Fatalf("陈旧预检覆盖领取: %+v", stored)
|
||||||
|
}
|
||||||
|
if _, err := repo.database().Exec(ctx, "UPDATE batch_detail SET item_id='A' WHERE id=1"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := repo.CompleteSubBatch(ctx, *claimed, []service.Outcome{{Code: 9999, Message: "明确失败"}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := repo.DelaySubBatchPrecheck(ctx, *snapshot, time.Now().Add(time.Hour), "更晚拒绝"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
stored, _ = repo.GetSubBatch(ctx, b.OperationID)
|
||||||
|
if stored.Status != "done" || !stored.NextCheckAt.IsZero() {
|
||||||
|
t.Fatalf("陈旧预检覆盖完成: %+v", stored)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchConcurrentPredecessorCompletionAndNewClaim(t *testing.T) {
|
||||||
|
repo, base := testSubBatchDB(t)
|
||||||
|
ctx, cancel := context.WithTimeout(base, 20*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
if _, err := repo.database().Exec(ctx, "UPDATE batch_detail SET item_id='A',batch_type=CASE WHEN batch_no='S-2' THEN 109 ELSE 110 END"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
older := testSub("S-1", "concurrent-old", "a", []int64{1}, []string{"A"})
|
||||||
|
newer := testSub("S-2", "concurrent-new", "a", []int64{3}, []string{"A"})
|
||||||
|
newer.BatchType = 109
|
||||||
|
for _, b := range []service.SubBatch{older, newer} {
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
first, err := repo.ClaimSubBatch(ctx, older.OperationID, "old", time.Now())
|
||||||
|
if err != nil || first == nil {
|
||||||
|
t.Fatalf("旧批领取: %+v %v", first, err)
|
||||||
|
}
|
||||||
|
start := make(chan struct{})
|
||||||
|
errs := make(chan error, 2)
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
wg.Add(2)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
<-start
|
||||||
|
errs <- repo.CompleteSubBatch(ctx, *first, []service.Outcome{{Success: true}})
|
||||||
|
}()
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
<-start
|
||||||
|
_, err := repo.ClaimSubBatch(ctx, newer.OperationID, "new", time.Now())
|
||||||
|
errs <- err
|
||||||
|
}()
|
||||||
|
close(start)
|
||||||
|
wg.Wait()
|
||||||
|
close(errs)
|
||||||
|
for err := range errs {
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
stored, _ := repo.GetSubBatch(ctx, newer.OperationID)
|
||||||
|
if stored.LeaseToken == "" {
|
||||||
|
if _, err := repo.database().Exec(ctx, "UPDATE data_operation_sub_batch SET next_check_at=NULL WHERE operation_id=?", newer.OperationID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
got, err := repo.ClaimSubBatch(ctx, newer.OperationID, "after", time.Now())
|
||||||
|
if err != nil || got == nil {
|
||||||
|
t.Fatal(fmt.Errorf("旧批完成后新批不可领取: %+v %v", got, err))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
row, err := repo.database().GetOne(ctx, "SELECT active_operation_id FROM data_operation_card_claim WHERE iccid='A'")
|
||||||
|
if err != nil || row["active_operation_id"].String() != newer.OperationID {
|
||||||
|
t.Fatalf("顺序屏障结果: %v %v", row, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchPrecheckBackoffMakes129thDatabaseTaskDue(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
base := time.Now().Add(-time.Minute)
|
||||||
|
for i := 0; i < 129; i++ {
|
||||||
|
id := int64(1000 + i)
|
||||||
|
if _, err := repo.database().Exec(ctx, "INSERT INTO batch_detail(id,batch_no,detail_status)VALUES(?,'S-1',1)", id); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
channel := "removed"
|
||||||
|
if i == 128 {
|
||||||
|
channel = "healthy"
|
||||||
|
}
|
||||||
|
b := testSub("S-1", fmt.Sprintf("db-fair-%03d", i), channel, []int64{id}, []string{fmt.Sprintf("fake-%03d", i)})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := repo.database().Exec(ctx, "UPDATE data_operation_sub_batch SET next_check_at=? WHERE operation_id=?", base.Add(time.Duration(i)*time.Millisecond), b.OperationID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ids, err := repo.ListDueSubBatches(ctx, time.Now(), 128)
|
||||||
|
if err != nil || len(ids) != 128 {
|
||||||
|
t.Fatalf("首窗口: %d %v", len(ids), err)
|
||||||
|
}
|
||||||
|
for _, id := range ids {
|
||||||
|
snapshot, err := repo.GetSubBatch(ctx, id)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if snapshot.ChanKey != "removed" {
|
||||||
|
t.Fatal("健康任务不应处于首128")
|
||||||
|
}
|
||||||
|
if err := repo.DelaySubBatchPrecheck(ctx, *snapshot, time.Now().Add(30*time.Second), "通道已移除"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ids, err = repo.ListDueSubBatches(ctx, time.Now(), 128)
|
||||||
|
if err != nil || len(ids) != 1 || ids[0] != "db-fair-128" {
|
||||||
|
t.Fatalf("健康第129仍被饿死: %v %v", ids, err)
|
||||||
|
}
|
||||||
|
if got, err := repo.ClaimSubBatch(ctx, ids[0], "healthy", time.Now()); err != nil || got == nil {
|
||||||
|
t.Fatalf("健康任务无法领取: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
n, err := repo.database().Model(subTable).Ctx(ctx).Where("chan_key", "removed").Where("status", "prepared").Where("poll_count", 0).WhereNull("submitted_at").WhereNull("lease_token").Count()
|
||||||
|
if err != nil || n != 128 {
|
||||||
|
t.Fatalf("预检失败意图被消耗: %d %v", n, err)
|
||||||
|
}
|
||||||
|
n, err = repo.database().Model(claimTable).Ctx(ctx).Where("active_operation_id != ''").Where("active_operation_id != ?", "db-fair-128").Count()
|
||||||
|
if err != nil || n != 0 {
|
||||||
|
t.Fatalf("预检退避错误占active: %d %v", n, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchMySQLCardWaitThenConcurrentReleaseAndReverseClaim(t *testing.T) {
|
||||||
|
if os.Getenv("APN_WORKER_MYSQL_TEST") != "1" {
|
||||||
|
t.Skip("仅隔离MySQL验证真实锁等待和两卡反向顺序")
|
||||||
|
}
|
||||||
|
repo, base := testSubBatchDB(t)
|
||||||
|
ctx, cancel := context.WithTimeout(base, 20*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
if _, err := repo.database().Exec(ctx, "INSERT INTO batch_detail(id,batch_no,detail_status)VALUES(4,'S-2',1)"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
older := testSub("S-1", "wait-old-BA", "a", []int64{1, 2}, []string{"B", "A"})
|
||||||
|
newer := testSub("S-2", "wait-new-AB", "a", []int64{3, 4}, []string{"A", "B"})
|
||||||
|
for _, b := range []service.SubBatch{older, newer} {
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
claimed, err := repo.ClaimSubBatch(ctx, older.OperationID, "old-owner", time.Now())
|
||||||
|
if err != nil || claimed == nil {
|
||||||
|
t.Fatalf("旧批领取: %+v %v", claimed, err)
|
||||||
|
}
|
||||||
|
holder, err := repo.database().Begin(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer holder.Rollback()
|
||||||
|
if _, err := holder.GetAll("SELECT iccid FROM data_operation_card_claim WHERE tenant_id='tenant' AND iccid='A' FOR UPDATE"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
start := make(chan struct{})
|
||||||
|
errs := make(chan error, 2)
|
||||||
|
go func() {
|
||||||
|
<-start
|
||||||
|
errs <- repo.ReleaseUnsentSubBatch(ctx, *claimed, "确定未发送")
|
||||||
|
}()
|
||||||
|
go func() {
|
||||||
|
<-start
|
||||||
|
_, err := repo.ClaimSubBatch(ctx, newer.OperationID, "new-owner", time.Now())
|
||||||
|
errs <- err
|
||||||
|
}()
|
||||||
|
close(start)
|
||||||
|
// 实际观察InnoDB锁等待,而非只靠两goroutine恰好同时运行。
|
||||||
|
waiting := false
|
||||||
|
deadline := time.After(5 * time.Second)
|
||||||
|
tick := time.NewTicker(10 * time.Millisecond)
|
||||||
|
defer tick.Stop()
|
||||||
|
observe:
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-deadline:
|
||||||
|
break observe
|
||||||
|
case <-tick.C:
|
||||||
|
value, err := repo.database().GetValue(ctx, "SELECT COUNT(*) FROM performance_schema.data_lock_waits w JOIN performance_schema.data_locks l ON l.ENGINE_LOCK_ID=w.REQUESTING_ENGINE_LOCK_ID WHERE l.OBJECT_SCHEMA=DATABASE() AND l.OBJECT_NAME='data_operation_card_claim'")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if value.Int() >= 2 {
|
||||||
|
waiting = true
|
||||||
|
break observe
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := holder.Commit(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for i := 0; i < 2; i++ {
|
||||||
|
if err := <-errs; err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !waiting {
|
||||||
|
t.Fatal("未实际观察到两条同卡交易的InnoDB锁等待")
|
||||||
|
}
|
||||||
|
stored, err := repo.GetSubBatch(ctx, older.OperationID)
|
||||||
|
if err != nil || stored.Status != "prepared" || !stored.SubmittedAt.IsZero() {
|
||||||
|
t.Fatalf("确定未发送回退状态: %+v %v", stored, err)
|
||||||
|
}
|
||||||
|
n, err := repo.database().Model(claimTable).Ctx(ctx).Where("active_operation_id != ''").Count()
|
||||||
|
if err != nil || n != 0 {
|
||||||
|
t.Fatalf("旧prepared未完成而新批抢先或残留部分active: %d %v", n, err)
|
||||||
|
}
|
||||||
|
t.Log("已观察两条事务等待同卡锁,释放后均提交成功,两卡输入BA/AB未形成反向环")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchMySQLClaimDoesNotLockBusinessParent(t *testing.T) {
|
||||||
|
if os.Getenv("APN_WORKER_MYSQL_TEST") != "1" {
|
||||||
|
t.Skip("仅隔离MySQL验证前序JOIN不锁业务父批")
|
||||||
|
}
|
||||||
|
repo, base := testSubBatchDB(t)
|
||||||
|
ctx, cancel := context.WithTimeout(base, 10*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
older := testSub("S-1", "main-lock-old", "a", []int64{1}, []string{"A"})
|
||||||
|
newer := testSub("S-2", "main-lock-new", "a", []int64{3}, []string{"A"})
|
||||||
|
for _, b := range []service.SubBatch{older, newer} {
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
holder, err := repo.database().Begin(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer holder.Rollback()
|
||||||
|
if _, err := holder.GetAll("SELECT id FROM batch_main WHERE batch_no='S-1' FOR UPDATE"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
done := make(chan error, 1)
|
||||||
|
go func() {
|
||||||
|
got, err := repo.ClaimSubBatch(ctx, newer.OperationID, "new-owner", time.Now())
|
||||||
|
if err == nil && got != nil {
|
||||||
|
err = fmt.Errorf("新批越过旧批")
|
||||||
|
}
|
||||||
|
done <- err
|
||||||
|
}()
|
||||||
|
select {
|
||||||
|
case err := <-done:
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal("Claim持card等待业务main,存在main→card反向环")
|
||||||
|
}
|
||||||
|
t.Log("旧业务main保持FOR UPDATE期间,新批前序当前读完成且未越序")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchMySQLCrossTenantSameCardsDoNotLockOtherSub(t *testing.T) {
|
||||||
|
if os.Getenv("APN_WORKER_MYSQL_TEST") != "1" {
|
||||||
|
t.Skip("仅隔离MySQL验证跨租户同ICCID双卡并发")
|
||||||
|
}
|
||||||
|
repo, base := testSubBatchDB(t)
|
||||||
|
ctx, cancel := context.WithTimeout(base, 15*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
for _, q := range []string{
|
||||||
|
"UPDATE batch_main SET tenant_id='other-tenant' WHERE batch_no='S-2'",
|
||||||
|
"INSERT INTO batch_detail(id,batch_no,detail_status)VALUES(4,'S-2',1)",
|
||||||
|
} {
|
||||||
|
if _, err := repo.database().Exec(ctx, q); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
first := testSub("S-1", "tenant-one-BA", "a", []int64{1, 2}, []string{"B", "A"})
|
||||||
|
second := testSub("S-2", "tenant-two-AB", "a", []int64{3, 4}, []string{"A", "B"})
|
||||||
|
for _, b := range []service.SubBatch{first, second} {
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
start := make(chan struct{})
|
||||||
|
errs := make(chan error, 2)
|
||||||
|
for _, b := range []service.SubBatch{first, second} {
|
||||||
|
go func(b service.SubBatch) {
|
||||||
|
<-start
|
||||||
|
got, err := repo.ClaimSubBatch(ctx, b.OperationID, b.OperationID, time.Now())
|
||||||
|
if err == nil && got == nil {
|
||||||
|
err = fmt.Errorf("跨租户意图不应互相阻断: %s", b.OperationID)
|
||||||
|
}
|
||||||
|
errs <- err
|
||||||
|
}(b)
|
||||||
|
}
|
||||||
|
close(start)
|
||||||
|
for i := 0; i < 2; i++ {
|
||||||
|
if err := <-errs; err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
n, err := repo.database().Model(claimTable).Ctx(ctx).Where("active_operation_id != ''").Count()
|
||||||
|
if err != nil || n != 4 {
|
||||||
|
t.Fatalf("跨tenant同ICCID active归属丢失: %d %v", n, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchMySQLCrossTenantForeignDetailLockDoesNotBlock(t *testing.T) {
|
||||||
|
if os.Getenv("APN_WORKER_MYSQL_TEST") != "1" {
|
||||||
|
t.Skip("仅隔离MySQL验证异租户相同ICCID不锁候选范围")
|
||||||
|
}
|
||||||
|
repo, base := testSubBatchDB(t)
|
||||||
|
ctx, cancel := context.WithTimeout(base, 15*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
if _, err := repo.database().Exec(ctx, "UPDATE batch_main SET tenant_id='other-tenant' WHERE batch_no='S-2'"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
foreign := testSub("S-1", "foreign-A", "a", []int64{1}, []string{"A"})
|
||||||
|
local := testSub("S-2", "local-A", "a", []int64{3}, []string{"A"})
|
||||||
|
for _, b := range []service.SubBatch{foreign, local} {
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
holder, err := repo.database().Begin(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer holder.Rollback()
|
||||||
|
if _, err := holder.GetAll("SELECT detail_id FROM data_operation_sub_detail WHERE operation_id=? FOR UPDATE", foreign.OperationID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
result := make(chan error, 1)
|
||||||
|
go func() {
|
||||||
|
got, err := repo.ClaimSubBatch(ctx, local.OperationID, "other-tenant-owner", time.Now())
|
||||||
|
if err == nil && got == nil {
|
||||||
|
err = fmt.Errorf("异租户独立意图未领取")
|
||||||
|
}
|
||||||
|
result <- err
|
||||||
|
}()
|
||||||
|
blocked := false
|
||||||
|
var claimErr error
|
||||||
|
ticker := time.NewTicker(20 * time.Millisecond)
|
||||||
|
defer ticker.Stop()
|
||||||
|
wait := time.After(2 * time.Second)
|
||||||
|
loop:
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case claimErr = <-result:
|
||||||
|
break loop
|
||||||
|
case <-wait:
|
||||||
|
blocked = true
|
||||||
|
break loop
|
||||||
|
case <-ticker.C:
|
||||||
|
value, err := repo.database().GetValue(ctx, "SELECT COUNT(*) FROM performance_schema.data_lock_waits w JOIN performance_schema.data_locks l ON l.ENGINE_LOCK_ID=w.REQUESTING_ENGINE_LOCK_ID WHERE l.OBJECT_SCHEMA=DATABASE() AND l.OBJECT_NAME='data_operation_sub_detail'")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if value.Int() > 0 {
|
||||||
|
blocked = true
|
||||||
|
break loop
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := holder.Commit(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if blocked {
|
||||||
|
claimErr = <-result
|
||||||
|
}
|
||||||
|
if claimErr != nil {
|
||||||
|
t.Fatal(claimErr)
|
||||||
|
}
|
||||||
|
if blocked {
|
||||||
|
t.Fatal("异租户旧明细持锁无谓阻塞新租户同ICCID Claim")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchMySQLRepeatableReadStillSeesCommittedPredecessor(t *testing.T) {
|
||||||
|
if os.Getenv("APN_WORKER_MYSQL_TEST") != "1" {
|
||||||
|
t.Skip("仅隔离MySQL验证RR快照之后的前序当前读")
|
||||||
|
}
|
||||||
|
repo, base := testSubBatchDB(t)
|
||||||
|
ctx, cancel := context.WithTimeout(base, 15*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
older := testSub("S-1", "rr-old-stop", "a", []int64{1}, []string{"A"})
|
||||||
|
newer := testSub("S-2", "rr-new-open", "a", []int64{3}, []string{"A"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, newer); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// 外层事务先建立无旧意图的RR快照,再由独立连接提交旧父批的冻结意图。
|
||||||
|
tx, err := repo.database().Begin(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer tx.Rollback()
|
||||||
|
if _, err := tx.GetAll("SELECT operation_id FROM data_operation_sub_batch"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
prepared := make(chan error, 1)
|
||||||
|
go func() { prepared <- repo.PrepareSubBatch(base, older) }()
|
||||||
|
if err := <-prepared; err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rows, err := tx.GetAll("SELECT operation_id FROM data_operation_sub_batch WHERE operation_id=?", older.OperationID)
|
||||||
|
if err != nil || len(rows) != 0 {
|
||||||
|
t.Fatalf("未实际建立看不到新提交前序的RR快照: %v %v", rows, err)
|
||||||
|
}
|
||||||
|
if gdb.TXFromCtx(tx.GetCtx(), repo.database().GetGroup()) == nil {
|
||||||
|
t.Fatal("外层RR事务未绑定上下文")
|
||||||
|
}
|
||||||
|
got, err := repo.ClaimSubBatch(tx.GetCtx(), newer.OperationID, "rr-owner", time.Now())
|
||||||
|
if err != nil || got != nil {
|
||||||
|
t.Fatalf("当前读遗漏已提交前序: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
t.Log("普通RR快照仍看不到旧意图,Claim的锁定当前读已阻止新批越序")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchMySQLDeadlockRetriesWholeTransaction(t *testing.T) {
|
||||||
|
if os.Getenv("APN_WORKER_MYSQL_TEST") != "1" {
|
||||||
|
t.Skip("仅隔离MySQL验证真实1213回滚重试")
|
||||||
|
}
|
||||||
|
repo, base := testSubBatchDB(t)
|
||||||
|
ctx, cancel := context.WithTimeout(base, 15*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
if _, err := repo.database().Exec(ctx, "INSERT INTO data_operation_card_claim(tenant_id,iccid,batch_id,operation_id)VALUES('tenant','dead-A',0,''),('tenant','dead-B',0,'')"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
locked := make(chan struct{}, 2)
|
||||||
|
proceed := make(chan struct{})
|
||||||
|
errs := make(chan error, 2)
|
||||||
|
var total atomic.Int32
|
||||||
|
for _, keys := range [][2]string{{"dead-A", "dead-B"}, {"dead-B", "dead-A"}} {
|
||||||
|
go func(keys [2]string) {
|
||||||
|
attempt := 0
|
||||||
|
errs <- repo.durableTx(ctx, func(ctx context.Context, tx gdb.TX) error {
|
||||||
|
attempt++
|
||||||
|
total.Add(1)
|
||||||
|
if _, err := tx.Exec("UPDATE data_operation_card_claim SET batch_id=batch_id+1 WHERE tenant_id='tenant' AND iccid=?", keys[0]); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if attempt == 1 {
|
||||||
|
locked <- struct{}{}
|
||||||
|
select {
|
||||||
|
case <-proceed:
|
||||||
|
case <-ctx.Done():
|
||||||
|
return ctx.Err()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_, err := tx.Exec("UPDATE data_operation_card_claim SET batch_id=batch_id+1 WHERE tenant_id='tenant' AND iccid=?", keys[1])
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
}(keys)
|
||||||
|
}
|
||||||
|
for i := 0; i < 2; i++ {
|
||||||
|
select {
|
||||||
|
case <-locked:
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal(ctx.Err())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
close(proceed)
|
||||||
|
for i := 0; i < 2; i++ {
|
||||||
|
if err := <-errs; err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if total.Load() < 3 {
|
||||||
|
t.Fatal("未触发真实死锁重试")
|
||||||
|
}
|
||||||
|
rows, err := repo.database().GetAll(ctx, "SELECT batch_id FROM data_operation_card_claim WHERE iccid IN ('dead-A','dead-B')")
|
||||||
|
if err != nil || len(rows) != 2 {
|
||||||
|
t.Fatalf("结果: %v %v", rows, err)
|
||||||
|
}
|
||||||
|
for _, row := range rows {
|
||||||
|
if row["batch_id"].Int() != 2 {
|
||||||
|
t.Fatalf("回滚后重复或遗漏更新: %v", rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
t.Logf("两事务成功提交,实际执行轮数=%d", total.Load())
|
||||||
|
}
|
||||||
@@ -0,0 +1,232 @@
|
|||||||
|
package data
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"runtime"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/os/gtime"
|
||||||
|
|
||||||
|
mysql "github.com/go-sql-driver/mysql"
|
||||||
|
_ "github.com/gogf/gf/contrib/drivers/mysql/v2"
|
||||||
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
"gopkg.in/yaml.v3"
|
||||||
|
)
|
||||||
|
|
||||||
|
// 仅显式启用时使用已核验的本机专用测试库;认证始终仅在内存中处理。
|
||||||
|
func testSubBatchMySQL(t *testing.T) gdb.DB {
|
||||||
|
t.Helper()
|
||||||
|
if os.Getenv("APN_WORKER_MYSQL_TEST") != "1" {
|
||||||
|
t.Fatal("未显式启用隔离MySQL测试")
|
||||||
|
}
|
||||||
|
_, file, _, _ := runtime.Caller(0)
|
||||||
|
root := filepath.Clean(filepath.Join(filepath.Dir(file), "../../../.."))
|
||||||
|
raw, err := os.ReadFile(filepath.Join(root, "iotlcm-sms/deploy/config.yaml"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal("读取本机认证配置失败,未打印凭据")
|
||||||
|
}
|
||||||
|
var config map[string]any
|
||||||
|
if err := yaml.Unmarshal(raw, &config); err != nil {
|
||||||
|
t.Fatal("解析本机认证配置失败,未打印内容")
|
||||||
|
}
|
||||||
|
var dsn string
|
||||||
|
var visit func(map[string]any)
|
||||||
|
visit = func(m map[string]any) {
|
||||||
|
for key, value := range m {
|
||||||
|
if key == "sceneDbDsn" {
|
||||||
|
if text, ok := value.(string); ok {
|
||||||
|
dsn = text
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if nested, ok := value.(map[string]any); ok {
|
||||||
|
visit(nested)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
visit(config)
|
||||||
|
parsed, err := mysql.ParseDSN(dsn)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal("本机DSN格式不正确,未打印内容")
|
||||||
|
}
|
||||||
|
if parsed.Net != "tcp" || (parsed.Addr != "127.0.0.1:3306" && parsed.Addr != "localhost:3306") {
|
||||||
|
t.Fatal("认证配置并非允许的本机3306,拒绝连接")
|
||||||
|
}
|
||||||
|
const database = "iotdb_apn_worker_test_20261015"
|
||||||
|
parsed.DBName = database
|
||||||
|
connection, err := sql.Open("mysql", parsed.FormatDSN())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal("创建隔离连接失败")
|
||||||
|
}
|
||||||
|
defer connection.Close()
|
||||||
|
ctx := context.Background()
|
||||||
|
var uuid, datadir, name, version string
|
||||||
|
var port int
|
||||||
|
if err := connection.QueryRowContext(ctx, "SELECT @@server_uuid,@@datadir,@@port,DATABASE(),VERSION()").Scan(&uuid, &datadir, &port, &name, &version); err != nil {
|
||||||
|
t.Fatal("只读核验本机实例身份失败")
|
||||||
|
}
|
||||||
|
normalized := strings.TrimRight(strings.ReplaceAll(datadir, "\\", "/"), "/")
|
||||||
|
if uuid != "4fa545af-ab36-11f1-be02-040e3c9de361" || port != 3306 || name != database || !strings.EqualFold(normalized, "D:/MySQL/MySQL Server 8.0/Data") {
|
||||||
|
t.Fatal("实例身份不匹配,拒绝任何写入")
|
||||||
|
}
|
||||||
|
t.Logf("隔离MySQL身份:version=%s uuid=%s port=%d database=%s", version, uuid, port, name)
|
||||||
|
db, err := gdb.New(gdb.ConfigNode{Type: "mysql", Host: "127.0.0.1", Port: "3306", User: parsed.User, Pass: parsed.Passwd, Name: database, Charset: "utf8mb4", Timezone: "Asia/Shanghai", Extra: "parseTime=true"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal("创建GoFrame隔离连接失败,未打印认证")
|
||||||
|
}
|
||||||
|
// 仅重建此专用库中的固定fixture表,不读取或修改任何业务库。
|
||||||
|
for _, table := range []string{"data_operation_sub_detail", "data_operation_sub_batch", "data_operation_card_claim", "data_operation_parent", "batch_detail", "batch_main", "t_sim_card_info", "sys_oss", "t_resource_batch_download"} {
|
||||||
|
if _, err := db.Exec(ctx, "DROP TABLE IF EXISTS `"+database+"`.`"+table+"`"); err != nil {
|
||||||
|
t.Fatalf("重建专用fixture失败:%s", table)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
migration, err := os.ReadFile(filepath.Join(root, "iotlcm-database/SQL/20261015_data_operation_sub_batch.sql"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal("读取worker迁移失败")
|
||||||
|
}
|
||||||
|
for _, statement := range strings.Split(string(migration), ";") {
|
||||||
|
statement = strings.TrimSpace(statement)
|
||||||
|
if statement == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, err := db.Exec(ctx, statement); err != nil {
|
||||||
|
t.Fatalf("执行专用库worker迁移失败:%v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return db
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchMySQLMigrationRepeatPreservesAcceptedIntent(t *testing.T) {
|
||||||
|
if os.Getenv("APN_WORKER_MYSQL_TEST") != "1" {
|
||||||
|
t.Skip("仅隔离MySQL验证真实迁移重复执行")
|
||||||
|
}
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
b := testSub("S-1", "migration-accepted", "a", []int64{1}, []string{"fake-A"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claimed, err := repo.ClaimSubBatch(ctx, b.OperationID, "owner", time.Now())
|
||||||
|
if err != nil || claimed == nil {
|
||||||
|
t.Fatalf("领取: %+v %v", claimed, err)
|
||||||
|
}
|
||||||
|
if err := repo.UpdateSubBatch(ctx, *claimed, "accepted", "preserved-job", "已受理", time.Now(), 2); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
query, err := repo.ClaimSubBatch(ctx, b.OperationID, "preserved-query-owner", time.Now().Add(time.Second))
|
||||||
|
if err != nil || query == nil {
|
||||||
|
t.Fatalf("领取在途查询租约: %+v %v", query, err)
|
||||||
|
}
|
||||||
|
before, err := repo.database().GetOne(ctx, "SELECT request_data,detail_ids,job_id,poll_count,submitted_at,status,lease_token,lease_until,next_check_at FROM data_operation_sub_batch WHERE operation_id=?", b.OperationID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if before["status"].String() != "accepted" || before["job_id"].String() != "preserved-job" || before["poll_count"].Int() != 3 || before["lease_token"].String() != "preserved-query-owner" || before["lease_until"].IsNil() {
|
||||||
|
t.Fatalf("未实际构造accepted在途租约: %v", before)
|
||||||
|
}
|
||||||
|
_, file, _, _ := runtime.Caller(0)
|
||||||
|
path := filepath.Join(filepath.Dir(file), "../../../../iotlcm-database/SQL/20261015_data_operation_sub_batch.sql")
|
||||||
|
migration, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for repeat := 0; repeat < 2; repeat++ {
|
||||||
|
for _, statement := range strings.Split(string(migration), ";") {
|
||||||
|
if strings.TrimSpace(statement) == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, err := repo.database().Exec(ctx, statement); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
after, err := repo.database().GetOne(ctx, "SELECT request_data,detail_ids,job_id,poll_count,submitted_at,status,lease_token,lease_until,next_check_at FROM data_operation_sub_batch WHERE operation_id=?", b.OperationID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for key, value := range before {
|
||||||
|
if value.String() != after[key].String() {
|
||||||
|
t.Fatalf("重复迁移改变受理字段: %s", key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
n, err := repo.database().Model("data_operation_sub_detail").Ctx(ctx).Where("operation_id", b.OperationID).Count()
|
||||||
|
if err != nil || n != 1 {
|
||||||
|
t.Fatalf("重复迁移丢明细归属: %d %v", n, err)
|
||||||
|
}
|
||||||
|
n, err = repo.database().Model(claimTable).Ctx(ctx).Where("active_operation_id", b.OperationID).Count()
|
||||||
|
if err != nil || n != 1 {
|
||||||
|
t.Fatalf("重复迁移丢active: %d %v", n, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchMySQLDifferentDatabaseDefaultCollation(t *testing.T) {
|
||||||
|
if os.Getenv("APN_WORKER_MYSQL_TEST") != "1" {
|
||||||
|
t.Skip("仅隔离MySQL验证不同默认排序规则")
|
||||||
|
}
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
value, err := repo.database().GetValue(ctx, "SELECT DEFAULT_COLLATION_NAME FROM information_schema.SCHEMATA WHERE SCHEMA_NAME=DATABASE()")
|
||||||
|
if err != nil || value.String() != "utf8mb4_0900_ai_ci" {
|
||||||
|
t.Fatalf("未实际覆盖不同DB默认排序规则: %v %v", value, err)
|
||||||
|
}
|
||||||
|
rows, err := repo.database().GetAll(ctx, "SELECT TABLE_NAME,COLLATION_NAME FROM information_schema.COLUMNS WHERE TABLE_SCHEMA=DATABASE() AND COLUMN_NAME='batch_no' AND TABLE_NAME IN ('batch_main','batch_detail','data_operation_parent','data_operation_sub_batch','data_operation_sub_detail')")
|
||||||
|
if err != nil || len(rows) != 5 {
|
||||||
|
t.Fatalf("关联列元数据: %v %v", rows, err)
|
||||||
|
}
|
||||||
|
for _, row := range rows {
|
||||||
|
if row["COLLATION_NAME"].String() != "utf8mb4_general_ci" {
|
||||||
|
t.Fatalf("关联列未匹配旧基线: %v", rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
b := testSub("S-1", "collation-real-join", "a", []int64{1}, []string{"fake-A"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if ids, err := repo.ListDueSubBatches(ctx, time.Now(), 128); err != nil || len(ids) != 1 {
|
||||||
|
t.Fatalf("跨新旧表due查询: %v %v", ids, err)
|
||||||
|
}
|
||||||
|
if ids, err := repo.ListRecoverableBatches(ctx, 128); err != nil {
|
||||||
|
t.Fatalf("跨新旧表恢复查询: %v %v", ids, err)
|
||||||
|
}
|
||||||
|
if got, err := repo.ClaimSubBatch(ctx, b.OperationID, "collation-owner", time.Now()); err != nil || got == nil {
|
||||||
|
t.Fatalf("跨新旧表前序锁定JOIN: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
indexRows, err := repo.database().GetAll(ctx, "EXPLAIN SELECT STRAIGHT_JOIN s.operation_id FROM batch_main m FORCE INDEX(PRIMARY) JOIN data_operation_sub_detail sd ON sd.batch_no=m.batch_no AND sd.iccid='fake-A' JOIN data_operation_sub_batch s ON s.operation_id=sd.operation_id JOIN batch_detail d ON d.id=sd.detail_id WHERE m.tenant_id='tenant' AND m.id<11 AND d.detail_status=1 AND s.status IN ('prepared','submitting','accepted','uncertain') LIMIT 1 FOR UPDATE OF sd,s,d")
|
||||||
|
if err != nil || len(indexRows) < 2 || indexRows[0]["table"].String() != "m" || indexRows[1]["table"].String() != "sd" || indexRows[1]["key"].String() != "uk_data_sub_batch_card" {
|
||||||
|
t.Fatalf("前序锁定查询未按父批→同卡映射驱动: %v %v", indexRows, err)
|
||||||
|
}
|
||||||
|
t.Logf("前序EXPLAIN首表=%s key=%s;第二表=%s key=%s", indexRows[0]["table"].String(), indexRows[0]["key"].String(), indexRows[1]["table"].String(), indexRows[1]["key"].String())
|
||||||
|
t.Logf("DB默认=%s,五张新旧表batch_no均general_ci,真实due/recovery/claim JOIN成功", value.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
func logMySQLLease(t *testing.T, repo *Repository, batchID int64, token string) {
|
||||||
|
t.Helper()
|
||||||
|
if os.Getenv("APN_WORKER_MYSQL_TEST") != "1" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ctx := context.Background()
|
||||||
|
raw, err := repo.database().GetOne(ctx, "SELECT CAST(lease_until AS CHAR) AS raw_lease, CAST(NOW(6) AS CHAR) AS db_now, CAST(UTC_TIMESTAMP(6) AS CHAR) AS db_utc, @@session.time_zone AS session_zone FROM data_operation_sub_batch WHERE id=?", batchID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var row subRow
|
||||||
|
if err := repo.database().Model(subTable).Ctx(ctx).Where("id", batchID).Scan(&row); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var alternate struct {
|
||||||
|
LeaseUntil *gtime.Time `orm:"lease_until"`
|
||||||
|
}
|
||||||
|
if err := repo.database().Model(subTable).Ctx(ctx).Where("id", batchID).Scan(&alternate); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
now := time.Now()
|
||||||
|
t.Logf("租约原值 raw=%s dbNow=%s dbUTC=%s sessionZone=%s tokenMatch=%t status=%s nil=%t", raw["raw_lease"].String(), raw["db_now"].String(), raw["db_utc"].String(), raw["session_zone"].String(), row.LeaseToken == token, row.Status, row.LeaseUntil == nil)
|
||||||
|
if row.LeaseUntil != nil {
|
||||||
|
t.Logf("time扫描=%s location=%s unixNano=%d; now=%s location=%s unixNano=%d", row.LeaseUntil.Format(time.RFC3339Nano), row.LeaseUntil.Location(), row.LeaseUntil.UnixNano(), now.Format(time.RFC3339Nano), now.Location(), now.UnixNano())
|
||||||
|
}
|
||||||
|
if alternate.LeaseUntil != nil {
|
||||||
|
t.Logf("gtime扫描=%s location=%s unixNano=%d", alternate.LeaseUntil.Format("c"), alternate.LeaseUntil.Time.Location(), alternate.LeaseUntil.UnixNano())
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,123 @@
|
|||||||
|
package data
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts"
|
||||||
|
service "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/data"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestSubBatchFrozenOrderNewClaimFirstDoesNotCancelOlderCards(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
for _, q := range []string{
|
||||||
|
`UPDATE batch_detail SET item_id=CASE id WHEN 2 THEN 'B' ELSE 'A' END,snapshot='{"authorized":true}'`,
|
||||||
|
`UPDATE batch_detail SET batch_type=109 WHERE batch_no='S-2'`,
|
||||||
|
`INSERT INTO t_sim_card_info(id,tenant_id,iccid,data_service_status)VALUES(1,'tenant','A',1),(2,'tenant','B',1)`,
|
||||||
|
} {
|
||||||
|
if _, err := repo.database().Exec(ctx, q); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
older := testSub("S-1", "frozen-stop-AB", "a", []int64{1, 2}, []string{"A", "B"})
|
||||||
|
newer := testSub("S-2", "frozen-open-A", "a", []int64{3}, []string{"A"})
|
||||||
|
newer.BatchType = 109
|
||||||
|
newer.Request.ReqData.OperType = "0"
|
||||||
|
for _, sub := range []service.SubBatch{older, newer} {
|
||||||
|
if err := repo.PrepareSubBatch(ctx, sub); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got, err := repo.ClaimSubBatch(ctx, newer.OperationID, "new-first", time.Now()); err != nil || got != nil {
|
||||||
|
t.Fatalf("newer overtook frozen predecessor: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
row, err := repo.database().GetOne(ctx, "SELECT status FROM data_operation_sub_batch WHERE operation_id=?", older.OperationID)
|
||||||
|
if err != nil || row["status"].String() != "prepared" {
|
||||||
|
t.Fatalf("older work cancelled: %v %v", row, err)
|
||||||
|
}
|
||||||
|
first, err := repo.ClaimSubBatch(ctx, older.OperationID, "old-first", time.Now())
|
||||||
|
if err != nil || first == nil {
|
||||||
|
t.Fatalf("older not claimable: %+v %v", first, err)
|
||||||
|
}
|
||||||
|
if err := repo.CompleteSubBatch(ctx, *first, []service.Outcome{{Success: true}, {Success: true}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rows, err := repo.database().GetAll(ctx, "SELECT iccid,data_service_status FROM t_sim_card_info ORDER BY iccid")
|
||||||
|
if err != nil || len(rows) != 2 || rows[0]["data_service_status"].Int() != 0 || rows[1]["data_service_status"].Int() != 0 {
|
||||||
|
t.Fatalf("stop A/B not persisted: %v %v", rows, err)
|
||||||
|
}
|
||||||
|
if _, err := repo.database().Exec(ctx, "UPDATE data_operation_sub_batch SET next_check_at=NULL WHERE operation_id=?", newer.OperationID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
second, err := repo.ClaimSubBatch(ctx, newer.OperationID, "new-after-old", time.Now())
|
||||||
|
if err != nil || second == nil {
|
||||||
|
t.Fatalf("newer not released after old terminal: %+v %v", second, err)
|
||||||
|
}
|
||||||
|
if err := repo.CompleteSubBatch(ctx, *second, []service.Outcome{{Success: true}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rows, err = repo.database().GetAll(ctx, "SELECT iccid,data_service_status FROM t_sim_card_info ORDER BY iccid")
|
||||||
|
if err != nil || rows[0]["data_service_status"].Int() != 1 || rows[1]["data_service_status"].Int() != 0 {
|
||||||
|
t.Fatalf("reverse should reopen only A, keep B stopped: %v %v", rows, err)
|
||||||
|
}
|
||||||
|
details, err := repo.database().GetAll(ctx, "SELECT detail_status FROM batch_detail ORDER BY id")
|
||||||
|
if err != nil || len(details) != 3 {
|
||||||
|
t.Fatalf("details: %v %v", details, err)
|
||||||
|
}
|
||||||
|
for _, d := range details {
|
||||||
|
if d["detail_status"].Int() != consts.BatchDetailStatusSuccess {
|
||||||
|
t.Fatalf("card was omitted or cancelled: %v", details)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchParentOrderSurvivesReverseFreezeOrder(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
older := testSub("S-1", "parent-old-freeze-second", "a", []int64{1, 2}, []string{"A", "B"})
|
||||||
|
newer := testSub("S-2", "parent-new-freeze-first", "a", []int64{3}, []string{"A"})
|
||||||
|
for _, b := range []service.SubBatch{newer, older} {
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
oldStored, _ := repo.GetSubBatch(ctx, older.OperationID)
|
||||||
|
newStored, _ := repo.GetSubBatch(ctx, newer.OperationID)
|
||||||
|
if oldStored.ID <= newStored.ID {
|
||||||
|
t.Fatal("未实际复现父批与冻结ID相反顺序")
|
||||||
|
}
|
||||||
|
if got, err := repo.ClaimSubBatch(ctx, newer.OperationID, "new-first", time.Now()); err != nil || got != nil {
|
||||||
|
t.Fatalf("新批越过已冻结的旧父批: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
if got, err := repo.ClaimSubBatch(ctx, older.OperationID, "old-first", time.Now()); err != nil || got == nil {
|
||||||
|
t.Fatalf("旧停AB应完整可领取: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
n, err := repo.database().Model(claimTable).Ctx(ctx).Where("active_operation_id", older.OperationID).Count()
|
||||||
|
if err != nil || n != 2 {
|
||||||
|
t.Fatalf("旧批不能遗漏B: %d %v", n, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchPreparedPredecessorWaitHasFiniteUnsubmittedOutcome(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
older := testSub("S-1", "prepared-old", "a", []int64{1}, []string{"A"})
|
||||||
|
newer := testSub("S-2", "prepared-new", "a", []int64{3}, []string{"A"})
|
||||||
|
for _, sub := range []service.SubBatch{older, newer} {
|
||||||
|
if err := repo.PrepareSubBatch(ctx, sub); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := repo.database().Exec(ctx, "UPDATE data_operation_sub_batch SET created_at=?,next_check_at=NULL WHERE operation_id=?", time.Now().Add(-16*time.Minute), newer.OperationID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got, err := repo.ClaimSubBatch(ctx, newer.OperationID, "expired", time.Now()); err != nil || got != nil {
|
||||||
|
t.Fatalf("expired later intent was submitted: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
row, err := repo.database().GetOne(ctx, "SELECT status,submitted_at,poll_count FROM data_operation_sub_batch WHERE operation_id=?", newer.OperationID)
|
||||||
|
if err != nil || row["status"].String() != "review" || !row["submitted_at"].IsNil() || row["poll_count"].Int() != 0 {
|
||||||
|
t.Fatalf("later intent not finitely closed unsent: %v %v", row, err)
|
||||||
|
}
|
||||||
|
row, err = repo.database().GetOne(ctx, "SELECT status FROM data_operation_sub_batch WHERE operation_id=?", older.OperationID)
|
||||||
|
if err != nil || row["status"].String() != "prepared" {
|
||||||
|
t.Fatalf("older intent cancelled: %v %v", row, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,781 @@
|
|||||||
|
package data
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"sync/atomic"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult"
|
||||||
|
service "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/data"
|
||||||
|
)
|
||||||
|
|
||||||
|
func testSubBatchDB(t *testing.T) (*Repository, context.Context) {
|
||||||
|
t.Helper()
|
||||||
|
ctx := context.Background()
|
||||||
|
var db gdb.DB
|
||||||
|
var err error
|
||||||
|
mysqlTest := os.Getenv("APN_WORKER_MYSQL_TEST") == "1"
|
||||||
|
if mysqlTest {
|
||||||
|
db = testSubBatchMySQL(t)
|
||||||
|
} else {
|
||||||
|
db, err = gdb.New(gdb.ConfigNode{Type: "sqlite", Link: "sqlite::@file(" + t.TempDir() + "/apn.db)"})
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
if err := db.Close(ctx); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
if !mysqlTest {
|
||||||
|
if _, err := db.Exec(ctx, "PRAGMA journal_mode=WAL"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, statement := range []string{
|
||||||
|
"CREATE TABLE batch_main (id INTEGER PRIMARY KEY, batch_no TEXT, batch_type INTEGER, batch_async_status INTEGER, tenant_id TEXT, start_time DATETIME, updated_at DATETIME, created_by INTEGER, created_dept INTEGER, valid_count INTEGER, success_count INTEGER, fail_count INTEGER, run_result INTEGER, fail_reason TEXT, batch_info TEXT, end_time DATETIME, updated_by INTEGER)",
|
||||||
|
"CREATE TABLE data_operation_parent (batch_no TEXT PRIMARY KEY, ready INTEGER DEFAULT 0, finalize_token TEXT, finalize_until DATETIME, recovery_attempt_at DATETIME, last_reason TEXT)",
|
||||||
|
"INSERT INTO data_operation_parent (batch_no,ready) VALUES ('S-1',1),('S-2',1)",
|
||||||
|
"CREATE TABLE t_sim_card_info (id INTEGER PRIMARY KEY, tenant_id TEXT, iccid TEXT, msisdn TEXT, imsi TEXT, chan_key TEXT, data_service_status INTEGER, status_sync_time DATETIME, traffic_cutoff_triggered_at DATETIME, updated_by INTEGER)",
|
||||||
|
"CREATE TABLE batch_detail (id INTEGER PRIMARY KEY, batch_no TEXT, batch_type INTEGER DEFAULT 110, row_no INTEGER, item_id TEXT, detail_status INTEGER, error_code TEXT, error_reason TEXT, snapshot TEXT, updated_by INTEGER)",
|
||||||
|
"CREATE TABLE sys_oss (id INTEGER PRIMARY KEY AUTOINCREMENT,tenant_id TEXT,file_name TEXT,original_name TEXT,file_suffix TEXT,path TEXT,url TEXT,created_dept INTEGER,created_by INTEGER,updated_by INTEGER,service TEXT,md5 TEXT,file_size INTEGER)",
|
||||||
|
"CREATE TABLE t_resource_batch_download (id INTEGER PRIMARY KEY AUTOINCREMENT,batch_no TEXT,task_name TEXT,resource_type INTEGER,export_scene INTEGER,total_count INTEGER,success_count INTEGER,fail_count INTEGER,download_status INTEGER,oss_id INTEGER,file_name TEXT,file_ext TEXT,file_url TEXT,file_path TEXT,file_size INTEGER,file_md5 TEXT,download_count INTEGER,max_download_count INTEGER,dept_id INTEGER,start_time DATETIME,finish_time DATETIME,tenant_id TEXT,created_dept INTEGER,created_by INTEGER,updated_by INTEGER)",
|
||||||
|
"CREATE TABLE data_operation_sub_batch (id INTEGER PRIMARY KEY AUTOINCREMENT, batch_no TEXT, batch_type INTEGER, chan_key TEXT, operation_id TEXT UNIQUE, detail_ids TEXT, request_data TEXT, job_id TEXT, status TEXT, poll_count INTEGER DEFAULT 0, next_check_at DATETIME, submitted_at DATETIME, lease_token TEXT, lease_until DATETIME, created_at DATETIME DEFAULT CURRENT_TIMESTAMP, last_reason TEXT)",
|
||||||
|
"CREATE TABLE data_operation_sub_detail (detail_id INTEGER PRIMARY KEY, operation_id TEXT, batch_no TEXT, iccid TEXT, UNIQUE(batch_no,iccid))",
|
||||||
|
"CREATE TABLE data_operation_card_claim (tenant_id TEXT, iccid TEXT, batch_id INTEGER, operation_id TEXT, active_batch_id INTEGER DEFAULT 0, active_operation_id TEXT DEFAULT '', PRIMARY KEY(tenant_id,iccid))",
|
||||||
|
"INSERT INTO batch_main (id,batch_no,batch_type,batch_async_status,tenant_id) VALUES (10,'S-1',110,2,'tenant'),(11,'S-2',109,2,'tenant')",
|
||||||
|
"INSERT INTO batch_detail (id,batch_no,detail_status) VALUES (1,'S-1',1),(2,'S-1',1),(3,'S-2',1)",
|
||||||
|
} {
|
||||||
|
if mysqlTest {
|
||||||
|
if strings.HasPrefix(statement, "CREATE TABLE data_operation_") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
statement = strings.ReplaceAll(statement, "INTEGER PRIMARY KEY AUTOINCREMENT", "BIGINT PRIMARY KEY AUTO_INCREMENT")
|
||||||
|
statement = strings.ReplaceAll(statement, " TEXT", " VARCHAR(512)")
|
||||||
|
statement = strings.ReplaceAll(statement, "snapshot VARCHAR(512)", "snapshot JSON")
|
||||||
|
statement = strings.ReplaceAll(statement, "batch_info VARCHAR(512)", "batch_info JSON")
|
||||||
|
if strings.HasPrefix(statement, "CREATE TABLE ") {
|
||||||
|
statement += " DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := db.Exec(ctx, statement); err != nil {
|
||||||
|
t.Fatalf("setup %q: %v", statement, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return &Repository{db: db}, ctx
|
||||||
|
}
|
||||||
|
|
||||||
|
func testSub(batchNo, operationID, chanKey string, detailIDs []int64, iccids []string) service.SubBatch {
|
||||||
|
keys := make([]*model.SimKey, 0, len(detailIDs))
|
||||||
|
for i, id := range detailIDs {
|
||||||
|
_ = id
|
||||||
|
keys = append(keys, &model.SimKey{ICCID: iccids[i], ChanKey: chanKey, MSISDN: fmt.Sprint(i + 1)})
|
||||||
|
}
|
||||||
|
return service.SubBatch{BatchNo: batchNo, BatchType: 110, ChanKey: chanKey, OperationID: operationID, DetailIDs: detailIDs, Request: model.SimCardApnChangeInput{SimKeys: keys, ReqData: model.SimCardApnChangeInputItem{OperType: "1", OperationID: operationID, SubmitOnly: true}}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchDurableClaimCASAndReview(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
b := testSub("S-1", "intent-1", "a", []int64{1, 2}, []string{"A", "B"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatalf("幂等准备失败: %v", err)
|
||||||
|
}
|
||||||
|
first, err := repo.ClaimSubBatch(ctx, b.OperationID, "owner1", time.Now())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if first == nil || first.Status != "prepared" {
|
||||||
|
t.Fatalf("claim=%+v", first)
|
||||||
|
}
|
||||||
|
second, err := repo.ClaimSubBatch(ctx, b.OperationID, "owner2", time.Now())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if second != nil {
|
||||||
|
t.Fatal("并发claim不应成功")
|
||||||
|
}
|
||||||
|
if err := repo.UpdateSubBatch(ctx, *first, "accepted", "job-1", "已受理", time.Now().Add(time.Minute), 0); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := repo.UpdateSubBatch(ctx, *first, "done", "", "stale", time.Time{}, 0); err == nil {
|
||||||
|
t.Fatal("旧lease竟能覆盖")
|
||||||
|
}
|
||||||
|
row, err := repo.database().GetOne(ctx, "SELECT status,job_id FROM data_operation_sub_batch WHERE operation_id=?", b.OperationID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if row["status"].String() != "accepted" || row["job_id"].String() != "job-1" {
|
||||||
|
t.Fatalf("jobId未立即落库: %+v", row)
|
||||||
|
}
|
||||||
|
if _, err := repo.ClaimSubBatch(ctx, b.OperationID, "owner3", time.Now()); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if second, err = repo.ClaimSubBatch(ctx, b.OperationID, "owner3", time.Now().Add(2*time.Minute)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if second == nil || second.JobID != "job-1" {
|
||||||
|
t.Fatalf("无法恢复已受理jobId: %+v", second)
|
||||||
|
}
|
||||||
|
if err := repo.UpdateSubBatch(ctx, *second, "review", "job-1", "结果未确认", time.Time{}, 4); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, id := range b.DetailIDs {
|
||||||
|
row, err := repo.database().GetOne(ctx, "SELECT detail_status,error_reason FROM batch_detail WHERE id=?", id)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if row["detail_status"].Int() != consts.BatchDetailStatusUnknown || row["error_reason"].String() != "结果未确认" {
|
||||||
|
t.Fatalf("未知状态落库失败 id=%d row=%+v", id, row)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ids, err := repo.ListDueSubBatches(ctx, time.Now().Add(time.Hour), 10); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
} else if len(ids) != 0 {
|
||||||
|
t.Fatalf("review不应自动重试: %+v", ids)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchPreparationRollbackAndReadyBarrier(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
b := testSub("S-1", "barrier", "a", []int64{1}, []string{"A"})
|
||||||
|
_, _ = repo.database().Exec(ctx, "UPDATE data_operation_parent SET ready=0 WHERE batch_no='S-1'")
|
||||||
|
interrupted := fmt.Errorf("模拟准备中断")
|
||||||
|
err := repo.ResumeOrPrepare(ctx, "S-1", func(prep context.Context) error {
|
||||||
|
if err := repo.PrepareSubBatch(prep, b); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return interrupted
|
||||||
|
})
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("中断应回滚")
|
||||||
|
}
|
||||||
|
n, _ := repo.database().Model(subTable).Ctx(ctx).Count()
|
||||||
|
if n != 0 {
|
||||||
|
t.Fatal("部分准备未回滚")
|
||||||
|
}
|
||||||
|
if err := repo.ResumeOrPrepare(ctx, "S-1", func(prep context.Context) error { return repo.PrepareSubBatch(prep, b) }); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
_, _ = repo.database().Exec(ctx, "UPDATE data_operation_parent SET ready=0 WHERE batch_no='S-1'")
|
||||||
|
if got, err := repo.ClaimSubBatch(ctx, b.OperationID, "premature", time.Now()); err != nil || got != nil {
|
||||||
|
t.Fatalf("ready前可领取: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
ids, err := repo.ListDueSubBatches(ctx, time.Now().Add(time.Minute), 10)
|
||||||
|
if err != nil || len(ids) != 0 {
|
||||||
|
t.Fatalf("ready前可扫描: %v %v", ids, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchActiveBlocksReverseAndReviewTerminatesUnsubmitted(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
older := testSub("S-1", "active-old", "a", []int64{1}, []string{"A"})
|
||||||
|
newer := testSub("S-2", "queued-new", "a", []int64{3}, []string{"A"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, older); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
first, err := repo.ClaimSubBatch(ctx, older.OperationID, "owner1", time.Now())
|
||||||
|
if err != nil || first == nil {
|
||||||
|
t.Fatalf("%+v %v", first, err)
|
||||||
|
}
|
||||||
|
if err := repo.PrepareSubBatch(ctx, newer); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got, err := repo.ClaimSubBatch(ctx, newer.OperationID, "reverse", time.Now()); err != nil || got != nil {
|
||||||
|
t.Fatalf("在途前序未阻断反向: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
row, _ := repo.database().GetOne(ctx, "SELECT active_operation_id,operation_id FROM data_operation_card_claim WHERE iccid='A'")
|
||||||
|
if row["active_operation_id"].String() != older.OperationID || row["operation_id"].String() != newer.OperationID {
|
||||||
|
t.Fatalf("queued抢占active: %v", row)
|
||||||
|
}
|
||||||
|
if err := repo.UpdateSubBatch(ctx, *first, "review", "", "前序未知", time.Time{}, 4); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
_, _ = repo.database().Exec(ctx, "UPDATE data_operation_sub_batch SET next_check_at=NULL WHERE operation_id=?", newer.OperationID)
|
||||||
|
if got, err := repo.ClaimSubBatch(ctx, newer.OperationID, "reverse", time.Now()); err != nil || got != nil {
|
||||||
|
t.Fatalf("未知前序未阻断: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
row, _ = repo.database().GetOne(ctx, "SELECT status,submitted_at,poll_count FROM data_operation_sub_batch WHERE operation_id=?", newer.OperationID)
|
||||||
|
if row["status"].String() != "review" || !row["submitted_at"].IsNil() || row["poll_count"].Int() != 0 {
|
||||||
|
t.Fatalf("未提交后继应有限收尾: %v", row)
|
||||||
|
}
|
||||||
|
row, _ = repo.database().GetOne(ctx, "SELECT active_operation_id FROM data_operation_card_claim WHERE iccid='A'")
|
||||||
|
if row["active_operation_id"].String() != older.OperationID {
|
||||||
|
t.Fatal("未知active被释放")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchBlockedGroupNeverAcquiresPartialActive(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
old := testSub("S-1", "old-B", "a", []int64{1}, []string{"B"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, old); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := repo.ClaimSubBatch(ctx, old.OperationID, "old", time.Now()); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
_, _ = repo.database().Exec(ctx, "INSERT INTO batch_detail(id,batch_no,detail_status)VALUES(4,'S-2',1)")
|
||||||
|
newer := testSub("S-2", "AB", "a", []int64{3, 4}, []string{"A", "B"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, newer); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got, err := repo.ClaimSubBatch(ctx, newer.OperationID, "new", time.Now()); err != nil || got != nil {
|
||||||
|
t.Fatalf("阻塞组竟领取: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
row, _ := repo.database().GetOne(ctx, "SELECT active_operation_id FROM data_operation_card_claim WHERE iccid='A'")
|
||||||
|
if row["active_operation_id"].String() != "" {
|
||||||
|
t.Fatal("部分卡占active产生死锁")
|
||||||
|
}
|
||||||
|
_, _ = repo.database().Exec(ctx, "UPDATE data_operation_sub_batch SET created_at=?,next_check_at=NULL WHERE operation_id=?", time.Now().Add(-16*time.Minute), newer.OperationID)
|
||||||
|
if _, err := repo.ClaimSubBatch(ctx, newer.OperationID, "deadline", time.Now()); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
row, _ = repo.database().GetOne(ctx, "SELECT status FROM data_operation_sub_batch WHERE operation_id=?", newer.OperationID)
|
||||||
|
if row["status"].String() != "review" {
|
||||||
|
t.Fatal("在途阻塞超过窗口仍永久Running")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchRecoveryWindowIsFair(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
for i := 0; i < 35; i++ {
|
||||||
|
no := fmt.Sprintf("R-%02d", i)
|
||||||
|
if _, err := repo.database().Exec(ctx, "INSERT INTO batch_main(id,batch_no,batch_type,batch_async_status,tenant_id)VALUES(?,?,109,2,'tenant')", 100+i, no); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := repo.database().Exec(ctx, "INSERT INTO data_operation_parent(batch_no,ready)VALUES(?,0)", no); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for i := 0; i < 3; i++ {
|
||||||
|
ids, err := repo.ListRecoverableBatches(ctx, 16)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, id := range ids {
|
||||||
|
seen[id] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for i := 0; i < 35; i++ {
|
||||||
|
if !seen[fmt.Sprintf("R-%02d", i)] {
|
||||||
|
t.Fatalf("固定恢复窗口饥饿: %d", i)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type durableGenerator struct {
|
||||||
|
calls atomic.Int32
|
||||||
|
create func(context.Context, batchresult.Request) (batchresult.Artifact, error)
|
||||||
|
request batchresult.Request
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *durableGenerator) Create(ctx context.Context, request batchresult.Request) (batchresult.Artifact, error) {
|
||||||
|
n := f.calls.Add(1)
|
||||||
|
f.request = request
|
||||||
|
if f.create != nil {
|
||||||
|
return f.create(ctx, request)
|
||||||
|
}
|
||||||
|
return batchresult.Artifact{FileName: fmt.Sprintf("%s-%d.xlsx", request.BatchNo, n), OriginalName: request.BatchNo + ".xlsx"}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchProgressIsMonotonicAndMissingCardSafe(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
_, _ = repo.database().Exec(ctx, "UPDATE batch_detail SET item_id=CASE id WHEN 1 THEN 'A' ELSE 'B' END,snapshot='{\"authorized\":true,\"statusBefore\":1}' WHERE batch_no='S-1'")
|
||||||
|
_, _ = repo.database().Exec(ctx, "INSERT INTO t_sim_card_info(id,tenant_id,iccid,data_service_status)VALUES(1,'tenant','A',1)")
|
||||||
|
b := testSub("S-1", "partial", "a", []int64{1, 2}, []string{"A", "B"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claimed, err := repo.ClaimSubBatch(ctx, b.OperationID, "first", time.Now())
|
||||||
|
if err != nil || claimed == nil {
|
||||||
|
t.Fatalf("claim: %v %v", claimed, err)
|
||||||
|
}
|
||||||
|
claimed.JobID = "job-partial"
|
||||||
|
if err := repo.SaveSubBatchProgress(ctx, *claimed, []service.Outcome{{Success: true, OrderNum: "order-A"}, {Pending: true}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
row, _ := repo.database().GetOne(ctx, "SELECT job_id FROM data_operation_sub_batch WHERE operation_id=?", b.OperationID)
|
||||||
|
if row["job_id"].String() != "job-partial" {
|
||||||
|
t.Fatal("jobId未随部分结果落库")
|
||||||
|
}
|
||||||
|
// A已明确成功,后续缺失不得覆盖;B无卡实体也不能panic丢掉整组结果。
|
||||||
|
if err := repo.SaveSubBatchProgress(ctx, *claimed, []service.Outcome{{Pending: true}, {Success: false, Code: 42, Message: "明确拒绝"}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := repo.UpdateSubBatch(ctx, *claimed, "review", claimed.JobID, "窗口结束", time.Time{}, 4); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rows, err := repo.database().GetAll(ctx, "SELECT detail_status,error_reason FROM batch_detail WHERE batch_no='S-1' ORDER BY id")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if rows[0]["detail_status"].Int() != consts.BatchDetailStatusSuccess || rows[1]["detail_status"].Int() != consts.BatchDetailStatusFailed || rows[1]["error_reason"].String() != "明确拒绝" {
|
||||||
|
t.Fatalf("review覆盖逐卡终态: %v", rows)
|
||||||
|
}
|
||||||
|
gen := &durableGenerator{}
|
||||||
|
repo.resultFiles = gen
|
||||||
|
if err := repo.FinalizeReadyBatch(ctx, "S-1"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(gen.request.Rows) != 2 || !gen.request.Rows[0].Success || gen.request.Rows[1].Success {
|
||||||
|
t.Fatalf("真实DB导出接线错误: %+v", gen.request)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchUnknownOnlyChangesUnconfirmedRows(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
_, _ = repo.database().Exec(ctx, "UPDATE batch_detail SET item_id=CASE id WHEN 1 THEN 'A' ELSE 'B' END,snapshot='{\"authorized\":true}' WHERE batch_no='S-1'")
|
||||||
|
b := testSub("S-1", "unknown", "a", []int64{1, 2}, []string{"A", "B"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claimed, err := repo.ClaimSubBatch(ctx, b.OperationID, "owner", time.Now())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := repo.SaveSubBatchProgress(ctx, *claimed, []service.Outcome{{Success: true, OrderNum: "confirmed"}, {Pending: true}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := repo.UpdateSubBatch(ctx, *claimed, "review", "job", "具体超时原因", time.Time{}, 4); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
gen := &durableGenerator{}
|
||||||
|
repo.resultFiles = gen
|
||||||
|
if err := repo.FinalizeReadyBatch(ctx, "S-1"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
main, _ := repo.database().GetOne(ctx, "SELECT batch_async_status,success_count,fail_count,valid_count,batch_info FROM batch_main WHERE batch_no='S-1'")
|
||||||
|
if main["batch_async_status"].Int() != consts.BatchAsyncStatusReview || main["success_count"].Int() != 1 || main["fail_count"].Int() != 0 || main["valid_count"].Int() != 2 {
|
||||||
|
t.Fatalf("未知误计失败或清零有效数: %v", main)
|
||||||
|
}
|
||||||
|
if len(gen.request.Rows) != 2 || gen.request.Rows[1].DetailStatus != consts.BatchDetailStatusUnknown || gen.request.Rows[1].Reason != "具体超时原因" {
|
||||||
|
t.Fatalf("待核查导出原因丢失: %+v", gen.request)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchZeroChildrenFinalizeLeaseRecoveryAndConcurrency(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
_, _ = repo.database().Exec(ctx, "UPDATE batch_detail SET detail_status=3,error_reason='校验不通过' WHERE batch_no='S-1'")
|
||||||
|
started := make(chan struct{})
|
||||||
|
release := make(chan struct{})
|
||||||
|
gen := &durableGenerator{create: func(context.Context, batchresult.Request) (batchresult.Artifact, error) {
|
||||||
|
close(started)
|
||||||
|
<-release
|
||||||
|
return batchresult.Artifact{FileName: "unique.xlsx"}, nil
|
||||||
|
}}
|
||||||
|
repo.resultFiles = gen
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
wg.Add(1)
|
||||||
|
var firstErr error
|
||||||
|
go func() { defer wg.Done(); firstErr = repo.FinalizeReadyBatch(ctx, "S-1") }()
|
||||||
|
<-started
|
||||||
|
if err := repo.FinalizeReadyBatch(ctx, "S-1"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
close(release)
|
||||||
|
wg.Wait()
|
||||||
|
if firstErr != nil {
|
||||||
|
t.Fatal(firstErr)
|
||||||
|
}
|
||||||
|
if gen.calls.Load() != 1 {
|
||||||
|
t.Fatal("并发最终化重复生成")
|
||||||
|
}
|
||||||
|
n, _ := repo.database().Model("sys_oss").Ctx(ctx).Count()
|
||||||
|
if n != 1 {
|
||||||
|
t.Fatal("重复发布OSS")
|
||||||
|
}
|
||||||
|
n, _ = repo.database().Model("t_resource_batch_download").Ctx(ctx).Count()
|
||||||
|
if n != 1 {
|
||||||
|
t.Fatal("重复下载记录")
|
||||||
|
}
|
||||||
|
// 真正零明细父批,以及前任已过期租约应能恢复。
|
||||||
|
_, _ = repo.database().Exec(ctx, "DELETE FROM batch_detail WHERE batch_no='S-2'")
|
||||||
|
_, _ = repo.database().Exec(ctx, "UPDATE data_operation_parent SET finalize_token='crashed',finalize_until=? WHERE batch_no='S-2'", time.Now().Add(-time.Minute))
|
||||||
|
repo.resultFiles = &durableGenerator{}
|
||||||
|
if err := repo.FinalizeReadyBatch(ctx, "S-2"); err != nil {
|
||||||
|
t.Fatalf("零明细/过期租约未恢复: %v", err)
|
||||||
|
}
|
||||||
|
row, _ := repo.database().GetOne(ctx, "SELECT batch_async_status FROM batch_main WHERE batch_no='S-2'")
|
||||||
|
if row["batch_async_status"].Int() == consts.BatchAsyncStatusProcessing {
|
||||||
|
t.Fatal("零子批不收尾")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchFinalizeFailureReleasesLease(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
_, _ = repo.database().Exec(ctx, "UPDATE batch_detail SET detail_status=3 WHERE batch_no='S-1'")
|
||||||
|
repo.resultFiles = &durableGenerator{create: func(context.Context, batchresult.Request) (batchresult.Artifact, error) {
|
||||||
|
return batchresult.Artifact{}, errors.New("模拟文件生成失败")
|
||||||
|
}}
|
||||||
|
if err := repo.FinalizeReadyBatch(ctx, "S-1"); err == nil {
|
||||||
|
t.Fatal("应暴露生成失败")
|
||||||
|
}
|
||||||
|
repo.resultFiles = &durableGenerator{}
|
||||||
|
if err := repo.FinalizeReadyBatch(ctx, "S-1"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchExplicitFailureReleasesActiveForReverse(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
_, _ = repo.database().Exec(ctx, "UPDATE batch_detail SET item_id='A' WHERE id=1")
|
||||||
|
old := testSub("S-1", "failed-old", "a", []int64{1}, []string{"A"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, old); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claimed, err := repo.ClaimSubBatch(ctx, old.OperationID, "owner", time.Now())
|
||||||
|
if err != nil || claimed == nil {
|
||||||
|
t.Fatalf("claim: %+v %v", claimed, err)
|
||||||
|
}
|
||||||
|
logMySQLLease(t, repo, claimed.ID, claimed.LeaseToken)
|
||||||
|
if err := repo.CompleteSubBatch(ctx, *claimed, []service.Outcome{{Code: 9999, Message: "任务状态4,已完成且明确失败"}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
row, _ := repo.database().GetOne(ctx, "SELECT detail_status FROM batch_detail WHERE id=1")
|
||||||
|
if row["detail_status"].Int() != consts.BatchDetailStatusFailed {
|
||||||
|
t.Fatal("明确9999未持久终态")
|
||||||
|
}
|
||||||
|
row, _ = repo.database().GetOne(ctx, "SELECT active_operation_id FROM data_operation_card_claim WHERE iccid='A'")
|
||||||
|
if row["active_operation_id"].String() != "" {
|
||||||
|
t.Fatal("明确失败仍保留active")
|
||||||
|
}
|
||||||
|
reverse := testSub("S-2", "reverse-after-failed", "a", []int64{3}, []string{"A"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, reverse); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got, err := repo.ClaimSubBatch(ctx, reverse.OperationID, "reverse", time.Now()); err != nil || got == nil {
|
||||||
|
t.Fatalf("明确失败后反向被阻塞: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchTypedUnsentReleasesOnlyOwnActiveAndRestoresPrepared(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
b := testSub("S-1", "unsent", "a", []int64{1, 2}, []string{"A", "B"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claimed, err := repo.ClaimSubBatch(ctx, b.OperationID, "first", time.Now())
|
||||||
|
if err != nil || claimed == nil {
|
||||||
|
t.Fatalf("%+v %v", claimed, err)
|
||||||
|
}
|
||||||
|
if err := repo.ReleaseUnsentSubBatch(ctx, *claimed, "第二次能力探测拒绝"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
row, _ := repo.database().GetOne(ctx, "SELECT status,submitted_at,poll_count,lease_token FROM data_operation_sub_batch WHERE operation_id=?", b.OperationID)
|
||||||
|
if row["status"].String() != "prepared" || !row["submitted_at"].IsNil() || row["poll_count"].Int() != 0 || row["lease_token"].String() != "" {
|
||||||
|
t.Fatalf("回退prepared失败: %v", row)
|
||||||
|
}
|
||||||
|
n, _ := repo.database().Model(claimTable).Ctx(ctx).Where("active_operation_id", b.OperationID).Count()
|
||||||
|
if n != 0 {
|
||||||
|
t.Fatal("未发送仍占active")
|
||||||
|
}
|
||||||
|
if err := repo.ReleaseUnsentSubBatch(ctx, *claimed, "陈旧回退"); err == nil {
|
||||||
|
t.Fatal("陈旧lease可重放回退")
|
||||||
|
}
|
||||||
|
_, _ = repo.database().Exec(ctx, "UPDATE data_operation_sub_batch SET next_check_at=NULL WHERE operation_id=?", b.OperationID)
|
||||||
|
retry, err := repo.ClaimSubBatch(ctx, b.OperationID, "retry", time.Now())
|
||||||
|
if err != nil || retry == nil || retry.Status != "prepared" {
|
||||||
|
t.Fatalf("回退后无法唯一首发: %+v %v", retry, err)
|
||||||
|
}
|
||||||
|
if err := repo.UpdateSubBatch(ctx, *retry, "accepted", "job", "已受理", time.Now(), 0); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
query, err := repo.ClaimSubBatch(ctx, b.OperationID, "query", time.Now().Add(time.Second))
|
||||||
|
if err != nil || query == nil {
|
||||||
|
t.Fatalf("%+v %v", query, err)
|
||||||
|
}
|
||||||
|
submitted := query.SubmittedAt
|
||||||
|
if err := repo.ReleaseUnsentSubBatch(ctx, *query, "查询探测拒绝"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
stored, err := repo.GetSubBatch(ctx, b.OperationID)
|
||||||
|
if err != nil || stored.PollCount != 0 || stored.Status != "accepted" || !stored.SubmittedAt.Equal(submitted) {
|
||||||
|
t.Fatalf("未发查询消耗轮次或丢首发时点: %+v %v", stored, err)
|
||||||
|
}
|
||||||
|
n, _ = repo.database().Model(claimTable).Ctx(ctx).Where("active_operation_id", b.OperationID).Count()
|
||||||
|
if n != 2 {
|
||||||
|
t.Fatal("已真实提交的active被查询门禁释放")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type prepareOnlyAPN struct{}
|
||||||
|
|
||||||
|
func (prepareOnlyAPN) ChangeSimApnFunction(context.Context, *model.SimCardApnChangeInput) ([]model.SimBaseRes[model.SimCardApnChangeOutputItem], error) {
|
||||||
|
return nil, errors.New("准备阶段不得调用上游")
|
||||||
|
}
|
||||||
|
func (prepareOnlyAPN) SubmitSimApnFunction(context.Context, *model.SimCardApnChangeInput) ([]model.SimBaseRes[model.SimCardApnChangeOutputItem], error) {
|
||||||
|
return nil, errors.New("准备阶段不得首发")
|
||||||
|
}
|
||||||
|
func (prepareOnlyAPN) QuerySimApnFunctionResult(context.Context, *model.SimCardApnChangeInput) ([]model.SimBaseRes[model.SimCardApnChangeOutputItem], error) {
|
||||||
|
return nil, errors.New("准备阶段不得补查")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchCutoffDoesNotSkipDisabledCardWithActiveOpen(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
if _, err := repo.database().Exec(ctx, "UPDATE batch_main SET batch_type=109 WHERE batch_no='S-1'"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := repo.database().Exec(ctx, "UPDATE batch_main SET batch_type=107 WHERE batch_no='S-2'"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := repo.database().Exec(ctx, "UPDATE batch_detail SET batch_type=107,item_id='A',snapshot='{\"authorized\":true}' WHERE id=3"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := repo.database().Exec(ctx, "INSERT INTO t_sim_card_info(id,tenant_id,iccid,chan_key,data_service_status)VALUES(1,'tenant','A','a',?)", consts.SimDataServiceStatusDisabled); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
old := testSub("S-1", "opening", "a", []int64{1}, []string{"A"})
|
||||||
|
old.BatchType = 109
|
||||||
|
old.Request.ReqData.OperType = consts.SimApnOperTypeOpen
|
||||||
|
if err := repo.PrepareSubBatch(ctx, old); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got, err := repo.ClaimSubBatch(ctx, old.OperationID, "open-owner", time.Now()); err != nil || got == nil {
|
||||||
|
t.Fatalf("开通领取: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
if _, err := repo.database().Exec(ctx, "UPDATE data_operation_parent SET ready=0 WHERE batch_no='S-2'"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := service.NewExecutor(repo, prepareOnlyAPN{}).Execute(ctx, "S-2"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
row, err := repo.database().GetOne(ctx, "SELECT operation_id,status FROM data_operation_sub_batch WHERE batch_no='S-2'")
|
||||||
|
if err != nil || len(row) == 0 {
|
||||||
|
t.Fatalf("本地disabled绕过冻结: %v %v", row, err)
|
||||||
|
}
|
||||||
|
detail, _ := repo.database().GetOne(ctx, "SELECT detail_status FROM batch_detail WHERE id=3")
|
||||||
|
if detail["detail_status"].Int() != consts.BatchDetailStatusRunning {
|
||||||
|
t.Fatal("开在途时达量停被提前成功")
|
||||||
|
}
|
||||||
|
if got, err := repo.ClaimSubBatch(ctx, row["operation_id"].String(), "cutoff", time.Now()); err != nil || got != nil {
|
||||||
|
t.Fatalf("达量停绕过前序active: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchLegacyProcessingWithoutParentNeverCreatesIntent(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
if _, err := repo.database().Exec(ctx, "DELETE FROM data_operation_parent WHERE batch_no='S-1'"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ids, err := repo.ListRecoverableBatches(ctx, 128)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, id := range ids {
|
||||||
|
if id == "S-1" {
|
||||||
|
t.Fatal("恢复扫描捡起无凭据旧批")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
called := false
|
||||||
|
if err := repo.ResumeOrPrepare(ctx, "S-1", func(context.Context) error { called = true; return nil }); err == nil || called {
|
||||||
|
t.Fatalf("旧批进入准备/提交: called=%t err=%v", called, err)
|
||||||
|
}
|
||||||
|
if err := repo.ProtectExecutionFailure(ctx, "S-1", "旧MQ重投失败"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if exists, err := repo.HasDurableParent(ctx, "S-1"); err != nil || exists {
|
||||||
|
t.Fatalf("失败保护或扫描凭空补凭据: exists=%t err=%v", exists, err)
|
||||||
|
}
|
||||||
|
n, err := repo.database().Model(subTable).Ctx(ctx).Where("batch_no", "S-1").Count()
|
||||||
|
if err != nil || n != 0 {
|
||||||
|
t.Fatalf("旧批产生新提交意图: %d %v", n, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubBatchDetailAndCardClaimProtectNewerOperation(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
older := testSub("S-1", "intent-old", "a", []int64{1}, []string{"A"})
|
||||||
|
newer := testSub("S-2", "intent-new", "a", []int64{3}, []string{"A"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, older); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := repo.PrepareSubBatch(ctx, newer); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claim, err := repo.database().GetOne(ctx, "SELECT batch_id,operation_id FROM data_operation_card_claim WHERE tenant_id='tenant' AND iccid='A'")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if claim["batch_id"].Int() != 11 || claim["operation_id"].String() != newer.OperationID {
|
||||||
|
t.Fatalf("后续反向操作claim未生效: %+v", claim)
|
||||||
|
}
|
||||||
|
// Even with a differently named intent, the same detail cannot join another submission.
|
||||||
|
duplicate := testSub("S-1", "intent-duplicate", "a", []int64{1}, []string{"A"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, duplicate); err == nil {
|
||||||
|
t.Fatal("同一明细重新归属另一提交意图应失败")
|
||||||
|
}
|
||||||
|
count, err := repo.database().GetOne(ctx, "SELECT COUNT(*) AS n FROM data_operation_sub_batch WHERE batch_no='S-1'")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if count["n"].Int() != 1 {
|
||||||
|
t.Fatalf("失败准备未回滚: %+v", count)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 并发的两次手动补查(不同 operationId)不得互相阻塞退回:
|
||||||
|
// 在途 review_query 子批不是“未终结”,否则两边都会以为对方还没完成而同时放弃退回。
|
||||||
|
func TestSubBatchReviewParentReturnsDespiteInFlightManualQuery(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
inflight := testSub("S-1", "inflight", "a", []int64{1}, []string{"A"})
|
||||||
|
settle := testSub("S-1", "settle", "a", []int64{2}, []string{"B"})
|
||||||
|
for _, sub := range []service.SubBatch{inflight, settle} {
|
||||||
|
if err := repo.PrepareSubBatch(ctx, sub); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
first, err := repo.ClaimSubBatch(ctx, inflight.OperationID, "owner-a", time.Now())
|
||||||
|
if err != nil || first == nil {
|
||||||
|
t.Fatalf("claim A: %+v %v", first, err)
|
||||||
|
}
|
||||||
|
if err := repo.UpdateSubBatch(ctx, *first, "review", "job-a", "补查窗口耗尽", time.Time{}, 4); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
second, err := repo.ClaimSubBatch(ctx, settle.OperationID, "owner-b", time.Now())
|
||||||
|
if err != nil || second == nil {
|
||||||
|
t.Fatalf("claim B: %+v %v", second, err)
|
||||||
|
}
|
||||||
|
if _, err := repo.database().Exec(ctx, "UPDATE batch_main SET batch_async_status=? WHERE batch_no='S-1'", consts.BatchAsyncStatusReview); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// A 处于已提交的在途补查状态;B 此时写回终态,仍必须能退回处理中。
|
||||||
|
if manual, err := repo.ClaimReviewSubBatch(ctx, inflight.OperationID, "manual", time.Now()); err != nil || manual == nil {
|
||||||
|
t.Fatalf("A 在途补查领取失败: %+v %v", manual, err)
|
||||||
|
}
|
||||||
|
if err := repo.UpdateSubBatch(ctx, *second, "done", "job-b", "", time.Time{}, second.PollCount); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
main, _ := repo.database().GetOne(ctx, "SELECT batch_async_status FROM batch_main WHERE batch_no='S-1'")
|
||||||
|
if main["batch_async_status"].Int() != consts.BatchAsyncStatusProcessing {
|
||||||
|
t.Fatalf("在途补查阻塞了退回,父批将永久挂待核查: %v", main)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 子批全部终结后父批必须能回到恢复扫描重新汇总:
|
||||||
|
// 修复“手动补查已把子批写成终态,但汇总失败/进程退出后父批永久挂待核查”的死角,
|
||||||
|
// 同时保证合法停查的批次不会因此被反复捞起空转。
|
||||||
|
func TestSubBatchReviewParentReentersRecoveryOnceSettled(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
_, _ = repo.database().Exec(ctx, "UPDATE batch_detail SET item_id=CASE id WHEN 1 THEN 'A' ELSE 'B' END,snapshot='{\"authorized\":true}' WHERE batch_no='S-1'")
|
||||||
|
b := testSub("S-1", "settle", "a", []int64{1, 2}, []string{"A", "B"})
|
||||||
|
if err := repo.PrepareSubBatch(ctx, b); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claimed, err := repo.ClaimSubBatch(ctx, b.OperationID, "owner", time.Now())
|
||||||
|
if err != nil || claimed == nil {
|
||||||
|
t.Fatalf("claim: %+v %v", claimed, err)
|
||||||
|
}
|
||||||
|
if err := repo.UpdateSubBatch(ctx, *claimed, "review", "job-1", "补查窗口耗尽", time.Time{}, 4); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// 模拟此前一次汇总已把父批停在待核查。
|
||||||
|
if _, err := repo.database().Exec(ctx, "UPDATE batch_main SET batch_async_status=? WHERE batch_no='S-1'", consts.BatchAsyncStatusReview); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// 手动同步领取并写回终态:这正是汇总失败/进程退出前最后一步。
|
||||||
|
manual, err := repo.ClaimReviewSubBatch(ctx, b.OperationID, "manual", time.Now())
|
||||||
|
if err != nil || manual == nil {
|
||||||
|
t.Fatalf("手动补查领取失败: %+v %v", manual, err)
|
||||||
|
}
|
||||||
|
if err := repo.UpdateSubBatch(ctx, *manual, "done", "job-1", "", time.Time{}, manual.PollCount); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
main, _ := repo.database().GetOne(ctx, "SELECT batch_async_status FROM batch_main WHERE batch_no='S-1'")
|
||||||
|
if main["batch_async_status"].Int() != consts.BatchAsyncStatusProcessing {
|
||||||
|
t.Fatalf("终态后未退回处理中,父批将永久挂待核查: %v", main)
|
||||||
|
}
|
||||||
|
if !recoverableContains(t, repo, ctx, "S-1") {
|
||||||
|
t.Fatal("退回处理中后仍未被恢复扫描领取,无法自动汇总")
|
||||||
|
}
|
||||||
|
// 汇总成功后若仍有未知明细,父批回到待核查且不再被捞起,避免每 5 秒空转生成结果文件。
|
||||||
|
repo.resultFiles = &durableGenerator{}
|
||||||
|
if err := repo.FinalizeReadyBatch(ctx, "S-1"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
main, _ = repo.database().GetOne(ctx, "SELECT batch_async_status FROM batch_main WHERE batch_no='S-1'")
|
||||||
|
if main["batch_async_status"].Int() != consts.BatchAsyncStatusReview {
|
||||||
|
t.Fatalf("待核查明细应仍停在待核查: %v", main)
|
||||||
|
}
|
||||||
|
if recoverableContains(t, repo, ctx, "S-1") {
|
||||||
|
t.Fatal("合法待核查批次被反复纳入恢复扫描,将导致空转汇总")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func recoverableContains(t *testing.T, repo *Repository, ctx context.Context, batchNo string) bool {
|
||||||
|
t.Helper()
|
||||||
|
ids, err := repo.ListRecoverableBatches(ctx, 128)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, id := range ids {
|
||||||
|
if id == batchNo {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// 手动同步只能看到该父批下仍待核查的子批,且不会因为已完成的子批而误触发。
|
||||||
|
func TestSubBatchListReviewSubBatchesForManualSync(t *testing.T) {
|
||||||
|
repo, ctx := testSubBatchDB(t)
|
||||||
|
review := testSub("S-1", "intent-review", "a", []int64{1}, []string{"A"})
|
||||||
|
done := testSub("S-1", "intent-done", "a", []int64{2}, []string{"B"})
|
||||||
|
other := testSub("S-2", "intent-other", "a", []int64{3}, []string{"C"})
|
||||||
|
for _, sub := range []service.SubBatch{review, done, other} {
|
||||||
|
if err := repo.PrepareSubBatch(ctx, sub); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claimed, err := repo.ClaimSubBatch(ctx, sub.OperationID, "owner-"+sub.OperationID, time.Now())
|
||||||
|
if err != nil || claimed == nil {
|
||||||
|
t.Fatalf("claim失败 operationId=%s err=%v", sub.OperationID, err)
|
||||||
|
}
|
||||||
|
status, jobID, reason := "review", "", "补查窗口耗尽,结果未知"
|
||||||
|
if sub.OperationID == done.OperationID {
|
||||||
|
status, jobID, reason = "done", "job-done", ""
|
||||||
|
}
|
||||||
|
if err := repo.UpdateSubBatch(ctx, *claimed, status, jobID, reason, time.Time{}, 4); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
ids, err := repo.ListReviewSubBatches(ctx, "S-1")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(ids) != 1 || ids[0] != review.OperationID {
|
||||||
|
t.Fatalf("手动同步候选错误: %+v", ids)
|
||||||
|
}
|
||||||
|
// 租约未过期的在途补查不能被第二个请求抢走。
|
||||||
|
if _, err := repo.database().Exec(ctx,
|
||||||
|
"UPDATE data_operation_sub_batch SET status='review_query', lease_token='owner', lease_until=? WHERE operation_id=?",
|
||||||
|
time.Now().Add(time.Minute), review.OperationID,
|
||||||
|
); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if ids, err = repo.ListReviewSubBatches(ctx, "S-1"); err != nil || len(ids) != 0 {
|
||||||
|
t.Fatalf("在途补查不应进入手动同步: %+v err=%v", ids, err)
|
||||||
|
}
|
||||||
|
// 租约过期后允许恢复,避免卡在 review_query 永不收敛。
|
||||||
|
if _, err := repo.database().Exec(ctx,
|
||||||
|
"UPDATE data_operation_sub_batch SET lease_until=? WHERE operation_id=?",
|
||||||
|
time.Now().Add(-time.Minute), review.OperationID,
|
||||||
|
); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if ids, err = repo.ListReviewSubBatches(ctx, "S-1"); err != nil || len(ids) != 1 || ids[0] != review.OperationID {
|
||||||
|
t.Fatalf("过期租约应可恢复: %+v err=%v", ids, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -127,8 +127,8 @@ func TestApplyBatchMixedGroups(t *testing.T) {
|
|||||||
repo := NewRepository()
|
repo := NewRepository()
|
||||||
if err := repo.ApplyBatch(ctx, []Result{
|
if err := repo.ApplyBatch(ctx, []Result{
|
||||||
{CardID: 1, Synced: true, ValueChanged: true, Imei: "865466070817587"}, // 值变更
|
{CardID: 1, Synced: true, ValueChanged: true, Imei: "865466070817587"}, // 值变更
|
||||||
{CardID: 2, Synced: true, ValueChanged: false}, // 心跳
|
{CardID: 2, Synced: true, ValueChanged: false}, // 心跳
|
||||||
{CardID: 3, Synced: false}, // 失败跳过
|
{CardID: 3, Synced: false}, // 失败跳过
|
||||||
{CardID: 4, Synced: true, ValueChanged: true, Imei: "111222333444555"}, // 从空到有值
|
{CardID: 4, Synced: true, ValueChanged: true, Imei: "111222333444555"}, // 从空到有值
|
||||||
}); err != nil {
|
}); err != nil {
|
||||||
t.Fatalf("ApplyBatch: %v", err)
|
t.Fatalf("ApplyBatch: %v", err)
|
||||||
@@ -156,12 +156,12 @@ func TestApplyBatchRejectsInvalidImeiValue(t *testing.T) {
|
|||||||
|
|
||||||
repo := NewRepository()
|
repo := NewRepository()
|
||||||
malicious := []string{
|
malicious := []string{
|
||||||
"865466070817587'", // 引号逃逸
|
"865466070817587'", // 引号逃逸
|
||||||
"865466070817587'; DROP TABLE x; --", // 注入尝试
|
"865466070817587'; DROP TABLE x; --", // 注入尝试
|
||||||
"865466070817587 OR 1=1", // 逻辑注入
|
"865466070817587 OR 1=1", // 逻辑注入
|
||||||
"abc", // 非数字
|
"abc", // 非数字
|
||||||
"865466070817587;999999999999999", // 分号拼接(本链路不支持多值)
|
"865466070817587;999999999999999", // 分号拼接(本链路不支持多值)
|
||||||
strings.Repeat("9", 33), // 超长
|
strings.Repeat("9", 33), // 超长
|
||||||
}
|
}
|
||||||
for _, value := range malicious {
|
for _, value := range malicious {
|
||||||
err := repo.ApplyBatch(ctx, []Result{
|
err := repo.ApplyBatch(ctx, []Result{
|
||||||
@@ -185,9 +185,9 @@ func TestApplyBatchAcceptsBoundaryLengths(t *testing.T) {
|
|||||||
|
|
||||||
repo := NewRepository()
|
repo := NewRepository()
|
||||||
values := []string{
|
values := []string{
|
||||||
"865466070817587", // 15 位
|
"865466070817587", // 15 位
|
||||||
"86546607081758712", // 17 位
|
"86546607081758712", // 17 位
|
||||||
strings.Repeat("9", 32), // varchar(32) 上限
|
strings.Repeat("9", 32), // varchar(32) 上限
|
||||||
}
|
}
|
||||||
for i, value := range values {
|
for i, value := range values {
|
||||||
cardID := int64(i + 1)
|
cardID := int64(i + 1)
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strings"
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
"github.com/gogf/gf/v2/database/gdb"
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
"github.com/gogf/gf/v2/encoding/gjson"
|
"github.com/gogf/gf/v2/encoding/gjson"
|
||||||
@@ -14,6 +15,7 @@ import (
|
|||||||
|
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/dao"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/batchfields"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/do"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity"
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult"
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/batchresult"
|
||||||
@@ -385,10 +387,23 @@ func (r *Repository) findExistingStocksByField(ctx context.Context, field, label
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (r *Repository) SaveValidationFailure(ctx context.Context, task *inbound.Task) error {
|
func (r *Repository) SaveValidationFailure(ctx context.Context, task *inbound.Task) error {
|
||||||
return r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error {
|
err := r.database().Transaction(ctx, func(ctx context.Context, tx gdb.TX) error {
|
||||||
batchInfo := buildInboundBatchInfo(task)
|
batchInfo := buildInboundBatchInfo(task)
|
||||||
task.Batch.BatchInfo = batchInfo
|
task.Batch.BatchInfo = batchInfo
|
||||||
details := buildBatchDetails(task, consts.BatchDetailStatusFailed)
|
details := buildBatchDetails(task, consts.BatchDetailStatusFailed)
|
||||||
|
failCount := 0
|
||||||
|
detailIndex := 0
|
||||||
|
for _, row := range task.Rows {
|
||||||
|
if row == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if inbound.IsCascadeFailure(row) {
|
||||||
|
details[detailIndex].DetailStatus = consts.BatchDetailStatusNotExecuted
|
||||||
|
} else {
|
||||||
|
failCount++
|
||||||
|
}
|
||||||
|
detailIndex++
|
||||||
|
}
|
||||||
if err := insertChunkedRows(ctx, tx, dao.BatchDetail.Table(), "inbound detail", details, inboundWriteBatchSize(ctx)); err != nil {
|
if err := insertChunkedRows(ctx, tx, dao.BatchDetail.Table(), "inbound detail", details, inboundWriteBatchSize(ctx)); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -397,14 +412,38 @@ func (r *Repository) SaveValidationFailure(ctx context.Context, task *inbound.Ta
|
|||||||
TotalCount: len(task.Rows),
|
TotalCount: len(task.Rows),
|
||||||
ValidCount: task.ValidCount,
|
ValidCount: task.ValidCount,
|
||||||
SuccessCount: 0,
|
SuccessCount: 0,
|
||||||
FailCount: len(task.Rows),
|
FailCount: failCount,
|
||||||
RunResult: consts.BatchRunResultFailed,
|
RunResult: consts.BatchRunResultFailed,
|
||||||
FailReason: task.FailureReason,
|
FailReason: batchfields.FailureReason(task.FailureReason),
|
||||||
BatchInfo: batchInfo,
|
BatchInfo: batchInfo,
|
||||||
EndTime: gtime.Now(),
|
EndTime: gtime.Now(),
|
||||||
UpdatedBy: task.Batch.CreatedBy,
|
UpdatedBy: task.Batch.CreatedBy,
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
if err == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
g.Log().Errorf(ctx, "save inbound validation details failed, batchNo:%s, err:%s", task.Batch.BatchNo, batchfields.FailureReason(err.Error()))
|
||||||
|
finalizationCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 30*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
fallbackErr := r.database().Transaction(finalizationCtx, func(ctx context.Context, tx gdb.TX) error {
|
||||||
|
return updateBatchMain(ctx, tx, task, do.BatchMain{
|
||||||
|
BatchAsyncStatus: consts.BatchAsyncStatusFailed,
|
||||||
|
TotalCount: len(task.Rows),
|
||||||
|
ValidCount: task.ValidCount,
|
||||||
|
SuccessCount: 0,
|
||||||
|
FailCount: len(task.Rows),
|
||||||
|
RunResult: consts.BatchRunResultFailed,
|
||||||
|
FailReason: batchfields.FailureReason("错误明细保存失败,本次整批未入库"),
|
||||||
|
BatchInfo: task.Batch.BatchInfo,
|
||||||
|
EndTime: gtime.Now(),
|
||||||
|
UpdatedBy: task.Batch.CreatedBy,
|
||||||
|
})
|
||||||
|
})
|
||||||
|
if fallbackErr != nil {
|
||||||
|
return gerror.Wrapf(fallbackErr, "save inbound failure details failed: %v; finalize batch failed", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (r *Repository) SaveSuccess(ctx context.Context, task *inbound.Task) error {
|
func (r *Repository) SaveSuccess(ctx context.Context, task *inbound.Task) error {
|
||||||
@@ -481,8 +520,8 @@ func batchDetailRow(task *inbound.Task, row *inbound.Row, status int) do.BatchDe
|
|||||||
DetailStatus: status,
|
DetailStatus: status,
|
||||||
BlankOutStatus: 1,
|
BlankOutStatus: 1,
|
||||||
RowNo: row.RowNo,
|
RowNo: row.RowNo,
|
||||||
ItemId: row.ICCID,
|
ItemId: batchfields.ItemID(row.ICCID),
|
||||||
ErrorReason: row.Reason,
|
ErrorReason: batchfields.DetailReason(row.Reason),
|
||||||
Snapshot: rowSnapshot(row),
|
Snapshot: rowSnapshot(row),
|
||||||
TenantId: task.Batch.TenantID,
|
TenantId: task.Batch.TenantID,
|
||||||
CreatedDept: task.Batch.CreatedDept,
|
CreatedDept: task.Batch.CreatedDept,
|
||||||
@@ -526,7 +565,7 @@ func (r *Repository) SaveResultFile(ctx context.Context, task *inbound.Task, req
|
|||||||
ExportScene: task.Batch.BatchType,
|
ExportScene: task.Batch.BatchType,
|
||||||
TotalCount: len(task.Rows),
|
TotalCount: len(task.Rows),
|
||||||
SuccessCount: successRows(task.Rows),
|
SuccessCount: successRows(task.Rows),
|
||||||
FailCount: len(task.Rows) - successRows(task.Rows),
|
FailCount: failedRows(task.Rows),
|
||||||
DownloadStatus: consts.DownloadStatusGenerated,
|
DownloadStatus: consts.DownloadStatusGenerated,
|
||||||
OssId: ossID,
|
OssId: ossID,
|
||||||
FileName: artifact.FileName,
|
FileName: artifact.FileName,
|
||||||
@@ -863,3 +902,13 @@ func successRows(rows []*inbound.Row) int {
|
|||||||
}
|
}
|
||||||
return count
|
return count
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func failedRows(rows []*inbound.Row) int {
|
||||||
|
count := 0
|
||||||
|
for _, row := range rows {
|
||||||
|
if row != nil && row.Reason != "" && !inbound.IsCascadeFailure(row) {
|
||||||
|
count++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return count
|
||||||
|
}
|
||||||
|
|||||||
@@ -1,16 +1,96 @@
|
|||||||
package inbound
|
package inbound
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts"
|
_ "github.com/gogf/gf/contrib/drivers/sqlite/v2"
|
||||||
serviceinbound "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/inbound"
|
"github.com/gogf/gf/v2/database/gdb"
|
||||||
|
|
||||||
"github.com/gogf/gf/v2/encoding/gjson"
|
"github.com/gogf/gf/v2/encoding/gjson"
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
"github.com/gogf/gf/v2/os/gcfg"
|
||||||
|
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/consts"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/batchfields"
|
||||||
|
serviceinbound "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/inbound"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
func TestBatchDetailRowClampsInvalidIdentifiersAndReasons(t *testing.T) {
|
||||||
|
task := &serviceinbound.Task{Batch: serviceinbound.Batch{BatchNo: "INB-TEST", BatchType: 103}}
|
||||||
|
row := &serviceinbound.Row{RowNo: 2, ICCID: strings.Repeat("9", 100), Reason: strings.Repeat("错", 1000)}
|
||||||
|
detail := batchDetailRow(task, row, consts.BatchDetailStatusFailed)
|
||||||
|
if detail.ItemId != "" || len([]rune(detail.ErrorReason.(string))) != batchfields.DetailReasonLimit {
|
||||||
|
t.Fatalf("invalid detail: itemID=%v, reason=%v", detail.ItemId, detail.ErrorReason)
|
||||||
|
}
|
||||||
|
if row.ICCID != strings.Repeat("9", 100) || row.Reason != strings.Repeat("错", 1000) {
|
||||||
|
t.Fatal("source row was mutated; full value must remain available to result export")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSaveValidationFailureHandlesOversizedDetailsAndFallback(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
oldAdapter := g.Cfg().GetAdapter()
|
||||||
|
adapter, err := gcfg.NewAdapterContent("batch:\n inbound:\n writeBatchSize: 1000\n")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
g.Cfg().SetAdapter(adapter)
|
||||||
|
defer g.Cfg().SetAdapter(oldAdapter)
|
||||||
|
for _, test := range []struct {
|
||||||
|
name string
|
||||||
|
constraint string
|
||||||
|
wantFallback bool
|
||||||
|
}{
|
||||||
|
{name: "oversized invalid row is saved", constraint: "CHECK(length(item_id) <= 64 AND length(error_reason) <= 512)"},
|
||||||
|
{name: "detail write fails", constraint: "CHECK(error_reason = 'allowed')", wantFallback: true},
|
||||||
|
} {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
db, err := gdb.New(gdb.ConfigNode{Type: "sqlite", Link: "sqlite::@file(:memory:)", MaxOpenConnCount: 1})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { _ = db.Close(ctx) })
|
||||||
|
statements := []string{
|
||||||
|
"CREATE TABLE batch_main (batch_no TEXT PRIMARY KEY, batch_type INTEGER, batch_async_status INTEGER, total_count INTEGER, valid_count INTEGER, success_count INTEGER, fail_count INTEGER, run_result INTEGER, fail_reason TEXT, batch_info TEXT, end_time DATETIME, updated_by INTEGER)",
|
||||||
|
"CREATE TABLE batch_detail (id INTEGER PRIMARY KEY AUTOINCREMENT, batch_no TEXT, batch_type INTEGER, detail_status INTEGER, blank_out_status INTEGER, row_no INTEGER, item_id TEXT, error_reason TEXT " + test.constraint + ", snapshot TEXT, tenant_id TEXT, created_dept INTEGER, created_by INTEGER, updated_by INTEGER)",
|
||||||
|
"INSERT INTO batch_main (batch_no, batch_type, batch_async_status) VALUES ('INB-TEST', 103, 2)",
|
||||||
|
}
|
||||||
|
for _, statement := range statements {
|
||||||
|
if _, err := db.Exec(ctx, statement); err != nil {
|
||||||
|
t.Fatalf("fixture failed: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
task := &serviceinbound.Task{
|
||||||
|
Batch: serviceinbound.Batch{BatchNo: "INB-TEST", BatchType: 103},
|
||||||
|
Rows: []*serviceinbound.Row{{RowNo: 2, ICCID: strings.Repeat("9", 100), Reason: strings.Repeat("错", 1000)}},
|
||||||
|
FailureReason: "输入格式不正确",
|
||||||
|
}
|
||||||
|
if err := (&Repository{db: db}).SaveValidationFailure(ctx, task); err != nil {
|
||||||
|
t.Fatalf("failed batch must be finalized: %v", err)
|
||||||
|
}
|
||||||
|
status, err := db.Model("batch_main").Ctx(ctx).Where("batch_no", "INB-TEST").Value("batch_async_status")
|
||||||
|
if err != nil || status.Int() != consts.BatchAsyncStatusFailed {
|
||||||
|
t.Fatalf("status = %v, err = %v", status, err)
|
||||||
|
}
|
||||||
|
reason, err := db.Model("batch_main").Ctx(ctx).Where("batch_no", "INB-TEST").Value("fail_reason")
|
||||||
|
if err != nil || strings.Contains(reason.String(), "错误明细保存失败") != test.wantFallback {
|
||||||
|
t.Fatalf("fallback reason = %v, err = %v", reason, err)
|
||||||
|
}
|
||||||
|
detailCount, err := db.Model("batch_detail").Ctx(ctx).Where("batch_no", "INB-TEST").Count()
|
||||||
|
wantDetails := 1
|
||||||
|
if test.wantFallback {
|
||||||
|
wantDetails = 0
|
||||||
|
}
|
||||||
|
if err != nil || detailCount != wantDetails {
|
||||||
|
t.Fatalf("details = %d, want %d, err = %v", detailCount, wantDetails, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestResolveLocalOssPath(t *testing.T) {
|
func TestResolveLocalOssPath(t *testing.T) {
|
||||||
uploadRoot := t.TempDir()
|
uploadRoot := t.TempDir()
|
||||||
expectedPath := filepath.Join(uploadRoot, "000000", "user", "file", "20260721", "source.xlsx")
|
expectedPath := filepath.Join(uploadRoot, "000000", "user", "file", "20260721", "source.xlsx")
|
||||||
|
|||||||
@@ -0,0 +1,70 @@
|
|||||||
|
package lifecycle
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity"
|
||||||
|
lifecycle "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/lifecycle"
|
||||||
|
)
|
||||||
|
|
||||||
|
// batchFailureSummary 生成中文失败汇总,替代原先的英文「N SIM cards failed」。
|
||||||
|
// 形如:共50张,成功28张,失败22张;主要失败原因:当前卡状态不支持该操作
|
||||||
|
func batchFailureSummary(total, success, fail int, reasons []string) string {
|
||||||
|
var builder strings.Builder
|
||||||
|
fmt.Fprintf(&builder, "共%d张,成功%d张,失败%d张", total, success, fail)
|
||||||
|
if reason := dominantReason(reasons); reason != "" {
|
||||||
|
builder.WriteString(";主要失败原因:")
|
||||||
|
builder.WriteString(reason)
|
||||||
|
}
|
||||||
|
return builder.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
// dominantReason 取出现次数最多的失败原因;次数相同时按字典序,保证结果稳定可测。
|
||||||
|
func dominantReason(reasons []string) string {
|
||||||
|
counts := make(map[string]int, len(reasons))
|
||||||
|
for _, reason := range reasons {
|
||||||
|
trimmed := strings.TrimSpace(reason)
|
||||||
|
if trimmed == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
counts[trimmed]++
|
||||||
|
}
|
||||||
|
if len(counts) == 0 {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
keys := make([]string, 0, len(counts))
|
||||||
|
for key := range counts {
|
||||||
|
keys = append(keys, key)
|
||||||
|
}
|
||||||
|
sort.Slice(keys, func(i, j int) bool {
|
||||||
|
if counts[keys[i]] != counts[keys[j]] {
|
||||||
|
return counts[keys[i]] > counts[keys[j]]
|
||||||
|
}
|
||||||
|
return keys[i] < keys[j]
|
||||||
|
})
|
||||||
|
return keys[0]
|
||||||
|
}
|
||||||
|
|
||||||
|
// lifecycleFailureSummary 依据内存中的任务明细生成汇总。
|
||||||
|
func lifecycleFailureSummary(details []*lifecycle.Detail, success, fail int) string {
|
||||||
|
reasons := make([]string, 0, fail)
|
||||||
|
for _, detail := range details {
|
||||||
|
if detail != nil && strings.TrimSpace(detail.Reason) != "" {
|
||||||
|
reasons = append(reasons, detail.Reason)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return batchFailureSummary(len(details), success, fail, reasons)
|
||||||
|
}
|
||||||
|
|
||||||
|
// storedFailureSummary 依据已落库的批次明细生成汇总(重启批次的复机阶段使用)。
|
||||||
|
func storedFailureSummary(details []*entity.BatchDetail, success, fail int) string {
|
||||||
|
reasons := make([]string, 0, fail)
|
||||||
|
for _, detail := range details {
|
||||||
|
if detail != nil && strings.TrimSpace(detail.ErrorReason) != "" {
|
||||||
|
reasons = append(reasons, detail.ErrorReason)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return batchFailureSummary(len(details), success, fail, reasons)
|
||||||
|
}
|
||||||
@@ -0,0 +1,73 @@
|
|||||||
|
package lifecycle
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model/entity"
|
||||||
|
lifecycle "git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/service/lifecycle"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestBatchFailureSummaryUsesChineseAndCounts 固化用户要求的失败文案:
|
||||||
|
// 必须是中文、带成功失败张数,并附带最主要的失败原因。
|
||||||
|
func TestBatchFailureSummaryUsesChineseAndCounts(t *testing.T) {
|
||||||
|
got := batchFailureSummary(50, 28, 22, []string{
|
||||||
|
"当前卡状态不支持该操作",
|
||||||
|
"当前卡状态不支持该操作",
|
||||||
|
"请求上游超时",
|
||||||
|
})
|
||||||
|
want := "共50张,成功28张,失败22张;主要失败原因:当前卡状态不支持该操作"
|
||||||
|
if got != want {
|
||||||
|
t.Fatalf("失败汇总=%q,期望%q", got, want)
|
||||||
|
}
|
||||||
|
if strings.Contains(got, "SIM cards failed") {
|
||||||
|
t.Fatal("仍包含英文文案")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestBatchFailureSummaryWithoutReason 覆盖没有具体原因的情况,不能拼出空的原因段。
|
||||||
|
func TestBatchFailureSummaryWithoutReason(t *testing.T) {
|
||||||
|
if got := batchFailureSummary(3, 1, 2, nil); got != "共3张,成功1张,失败2张" {
|
||||||
|
t.Fatalf("无原因时=%q", got)
|
||||||
|
}
|
||||||
|
if got := batchFailureSummary(3, 1, 2, []string{"", " "}); got != "共3张,成功1张,失败2张" {
|
||||||
|
t.Fatalf("空白原因应被忽略,实际=%q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestDominantReasonStableOnTie 同频时按字节序取最小值,保证同一批次文案稳定。
|
||||||
|
// 注意这里是字节序而非拼音序:「乙」的 UTF-8 编码小于「甲」,所以同频时取「乙原因」。
|
||||||
|
func TestDominantReasonStableOnTie(t *testing.T) {
|
||||||
|
if got := dominantReason([]string{"甲原因", "乙原因"}); got != "乙原因" {
|
||||||
|
t.Fatalf("同频应按字节序取最小,实际=%q", got)
|
||||||
|
}
|
||||||
|
if got := dominantReason([]string{"少数", "多数", "多数"}); got != "多数" {
|
||||||
|
t.Fatalf("应取出现次数最多,实际=%q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestStoredFailureSummarySkipsNilAndEmpty 覆盖重启复机阶段:落库明细里的空原因与空指针不能参与统计。
|
||||||
|
func TestStoredFailureSummarySkipsNilAndEmpty(t *testing.T) {
|
||||||
|
details := []*entity.BatchDetail{
|
||||||
|
{DetailStatus: 2},
|
||||||
|
nil,
|
||||||
|
{DetailStatus: 3, ErrorReason: "当前卡状态不支持该操作"},
|
||||||
|
{DetailStatus: 3, ErrorReason: ""},
|
||||||
|
}
|
||||||
|
got := storedFailureSummary(details, 1, 2)
|
||||||
|
if !strings.HasPrefix(got, "共4张,成功1张,失败2张") || !strings.Contains(got, "当前卡状态不支持该操作") {
|
||||||
|
t.Fatalf("落库汇总=%q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestLifecycleFailureSummaryUsesDetailReason 覆盖停复机首阶段:内存明细的 Reason 作为主要失败原因。
|
||||||
|
func TestLifecycleFailureSummaryUsesDetailReason(t *testing.T) {
|
||||||
|
details := []*lifecycle.Detail{
|
||||||
|
{Reason: ""},
|
||||||
|
{Reason: "停机或预销户卡,不能办理该业务"},
|
||||||
|
}
|
||||||
|
got := lifecycleFailureSummary(details, 1, 1)
|
||||||
|
if got != "共2张,成功1张,失败1张;主要失败原因:停机或预销户卡,不能办理该业务" {
|
||||||
|
t.Fatalf("首阶段汇总=%q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -211,7 +211,7 @@ func (r *Repository) SaveResult(ctx context.Context, task *lifecycle.Task, detai
|
|||||||
if resolved.Success {
|
if resolved.Success {
|
||||||
if detail.Card == nil {
|
if detail.Card == nil {
|
||||||
resolved.Success = false
|
resolved.Success = false
|
||||||
resolved.Message = "SIM card record is missing"
|
resolved.Message = "卡记录不存在,未办理"
|
||||||
} else {
|
} else {
|
||||||
result, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()).
|
result, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()).
|
||||||
Where(dao.TSimCardInfo.Columns().Id, detail.Card.ID).
|
Where(dao.TSimCardInfo.Columns().Id, detail.Card.ID).
|
||||||
@@ -231,7 +231,7 @@ func (r *Repository) SaveResult(ctx context.Context, task *lifecycle.Task, detai
|
|||||||
}
|
}
|
||||||
if affected != 1 {
|
if affected != 1 {
|
||||||
resolved.Success = false
|
resolved.Success = false
|
||||||
resolved.Message = "SIM card local status changed"
|
resolved.Message = "卡本地状态已变更"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -276,7 +276,7 @@ func (r *Repository) Finalize(ctx context.Context, task *lifecycle.Task) error {
|
|||||||
failCount := len(task.Details) - successCount
|
failCount := len(task.Details) - successCount
|
||||||
batchAsyncStatus, runResult := batchResult(successCount, failCount)
|
batchAsyncStatus, runResult := batchResult(successCount, failCount)
|
||||||
if task.FailureReason == "" && failCount > 0 {
|
if task.FailureReason == "" && failCount > 0 {
|
||||||
task.FailureReason = strconv.Itoa(failCount) + " SIM cards failed"
|
task.FailureReason = lifecycleFailureSummary(task.Details, successCount, failCount)
|
||||||
}
|
}
|
||||||
validCount := 0
|
validCount := 0
|
||||||
if task.ValidationOK {
|
if task.ValidationOK {
|
||||||
|
|||||||
@@ -135,7 +135,7 @@ func (r *Repository) SaveRestartValidationFailure(ctx context.Context, task *lif
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if strings.TrimSpace(detail.Reason) == "" {
|
if strings.TrimSpace(detail.Reason) == "" {
|
||||||
detail.Reason = "restart validation failed; upstream request was skipped"
|
detail.Reason = "重启校验未通过,未发起上游请求"
|
||||||
}
|
}
|
||||||
outcome := lifecycle.Outcome{Message: detail.Reason}
|
outcome := lifecycle.Outcome{Message: detail.Reason}
|
||||||
result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()).
|
result, err := tx.Ctx(ctx).Model(dao.BatchDetail.Table()).
|
||||||
@@ -175,7 +175,7 @@ func (r *Repository) SaveRestartResult(ctx context.Context, task *lifecycle.Task
|
|||||||
if resolved.Success {
|
if resolved.Success {
|
||||||
if detail.Card == nil {
|
if detail.Card == nil {
|
||||||
resolved.Success = false
|
resolved.Success = false
|
||||||
resolved.Message = "SIM card record is missing"
|
resolved.Message = "卡记录不存在,未办理"
|
||||||
} else {
|
} else {
|
||||||
result, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()).
|
result, err := tx.Ctx(ctx).Model(dao.TSimCardInfo.Table()).
|
||||||
Where(dao.TSimCardInfo.Columns().Id, detail.Card.ID).
|
Where(dao.TSimCardInfo.Columns().Id, detail.Card.ID).
|
||||||
@@ -195,7 +195,7 @@ func (r *Repository) SaveRestartResult(ctx context.Context, task *lifecycle.Task
|
|||||||
}
|
}
|
||||||
if affected != 1 {
|
if affected != 1 {
|
||||||
resolved.Success = false
|
resolved.Success = false
|
||||||
resolved.Message = "SIM card local status changed"
|
resolved.Message = "卡本地状态已变更"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -327,7 +327,7 @@ func (r *Repository) FinalizeRestart(ctx context.Context, batchNo string) error
|
|||||||
batchAsyncStatus, runResult := batchResult(successCount, failCount)
|
batchAsyncStatus, runResult := batchResult(successCount, failCount)
|
||||||
failReason := ""
|
failReason := ""
|
||||||
if failCount > 0 {
|
if failCount > 0 {
|
||||||
failReason = strconv.Itoa(failCount) + " SIM cards failed"
|
failReason = storedFailureSummary(details, successCount, failCount)
|
||||||
}
|
}
|
||||||
validCount := 0
|
validCount := 0
|
||||||
if restartValidationOK(main.BatchInfo) {
|
if restartValidationOK(main.BatchInfo) {
|
||||||
|
|||||||
@@ -77,7 +77,7 @@ type changeCardUpdateDO struct {
|
|||||||
DataServiceStatus any `orm:"data_service_status"`
|
DataServiceStatus any `orm:"data_service_status"`
|
||||||
PackageTotalFlowMb any `orm:"package_total_flow_mb"`
|
PackageTotalFlowMb any `orm:"package_total_flow_mb"`
|
||||||
UsedFlowMb any `orm:"used_flow_mb"`
|
UsedFlowMb any `orm:"used_flow_mb"`
|
||||||
TrafficCutoffStatus any `orm:"traffic_cutoff_status"`
|
TrafficCutoffStatus any `orm:"traffic_cutoff_status"`
|
||||||
TrafficCutoffThreshold any `orm:"traffic_cutoff_threshold"`
|
TrafficCutoffThreshold any `orm:"traffic_cutoff_threshold"`
|
||||||
SmsServiceStatus any `orm:"sms_service_status"`
|
SmsServiceStatus any `orm:"sms_service_status"`
|
||||||
PackageTotalSms any `orm:"package_total_sms"`
|
PackageTotalSms any `orm:"package_total_sms"`
|
||||||
@@ -344,7 +344,7 @@ func (r *Repository) UpdateChangeCard(ctx context.Context, tx gdb.TX, iccid, pac
|
|||||||
DataServiceStatus: 1,
|
DataServiceStatus: 1,
|
||||||
PackageTotalFlowMb: packageFlowMb,
|
PackageTotalFlowMb: packageFlowMb,
|
||||||
UsedFlowMb: 0,
|
UsedFlowMb: 0,
|
||||||
TrafficCutoffStatus: 0,
|
TrafficCutoffStatus: 0,
|
||||||
TrafficCutoffThreshold: 0,
|
TrafficCutoffThreshold: 0,
|
||||||
SmsServiceStatus: 0,
|
SmsServiceStatus: 0,
|
||||||
PackageTotalSms: 0,
|
PackageTotalSms: 0,
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ const (
|
|||||||
resultFileStatusSuccess = "success"
|
resultFileStatusSuccess = "success"
|
||||||
resultFileStatusFailed = "failed"
|
resultFileStatusFailed = "failed"
|
||||||
|
|
||||||
validationFailureReason = "batch validation failed; traffic cutoff update skipped"
|
validationFailureReason = "批次校验未通过,已跳过流量断网更新"
|
||||||
)
|
)
|
||||||
|
|
||||||
type Repository struct {
|
type Repository struct {
|
||||||
@@ -338,7 +338,7 @@ func (r *Repository) lockCards(ctx context.Context, tx gdb.TX, tenantID string,
|
|||||||
func validateLockedTask(task *trafficcutoff.Task) bool {
|
func validateLockedTask(task *trafficcutoff.Task) bool {
|
||||||
if task == nil || len(task.Details) == 0 {
|
if task == nil || len(task.Details) == 0 {
|
||||||
if task != nil {
|
if task != nil {
|
||||||
task.FailureReason = "traffic cutoff batch has no valid details"
|
task.FailureReason = "流量断网批次无有效明细"
|
||||||
}
|
}
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
@@ -360,15 +360,15 @@ func validateLockedTask(task *trafficcutoff.Task) bool {
|
|||||||
}
|
}
|
||||||
detail.Reason = ""
|
detail.Reason = ""
|
||||||
if detail.Card == nil {
|
if detail.Card == nil {
|
||||||
detail.Reason = "ICCID does not exist in SIM card records"
|
detail.Reason = "ICCID不存在或不属于当前租户"
|
||||||
} else if !detail.Authorized {
|
} else if !detail.Authorized {
|
||||||
detail.Reason = "ICCID is outside the current operator authorization scope"
|
detail.Reason = "ICCID不在当前操作员授权范围内"
|
||||||
} else if detail.InputMSISDN != "" && detail.InputMSISDN != detail.Card.MSISDN {
|
} else if detail.InputMSISDN != "" && detail.InputMSISDN != detail.Card.MSISDN {
|
||||||
detail.Reason = "MSISDN does not match the SIM card record"
|
detail.Reason = "MSISDN与卡档案不一致"
|
||||||
} else if detail.InputIMSI != "" && detail.InputIMSI != detail.Card.IMSI {
|
} else if detail.InputIMSI != "" && detail.InputIMSI != detail.Card.IMSI {
|
||||||
detail.Reason = "IMSI does not match the SIM card record"
|
detail.Reason = "IMSI与卡档案不一致"
|
||||||
} else if detail.Card.BossCardStatus == trafficcutoff.BossCardStatusCanceled {
|
} else if detail.Card.BossCardStatus == trafficcutoff.BossCardStatusCanceled {
|
||||||
detail.Reason = "ICCID is already canceled"
|
detail.Reason = "ICCID已销户"
|
||||||
}
|
}
|
||||||
if detail.Reason != "" {
|
if detail.Reason != "" {
|
||||||
hasFailure = true
|
hasFailure = true
|
||||||
@@ -379,7 +379,7 @@ func validateLockedTask(task *trafficcutoff.Task) bool {
|
|||||||
if len(statusSet) > 1 {
|
if len(statusSet) > 1 {
|
||||||
for _, detail := range task.Details {
|
for _, detail := range task.Details {
|
||||||
if detail != nil && detail.Reason == "" {
|
if detail != nil && detail.Reason == "" {
|
||||||
detail.Reason = "same batch contains different card statuses"
|
detail.Reason = "同一批次内卡状态不一致"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
hasFailure = true
|
hasFailure = true
|
||||||
@@ -390,7 +390,7 @@ func validateLockedTask(task *trafficcutoff.Task) bool {
|
|||||||
}
|
}
|
||||||
for _, detail := range task.Details {
|
for _, detail := range task.Details {
|
||||||
if detail != nil && detail.Reason == "" {
|
if detail != nil && detail.Reason == "" {
|
||||||
detail.Reason = "same batch has invalid traffic cutoff rows; update skipped"
|
detail.Reason = "同一批次存在无效流量断网明细,已跳过更新"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
task.FailureReason = validationFailureReason
|
task.FailureReason = validationFailureReason
|
||||||
|
|||||||
@@ -0,0 +1,84 @@
|
|||||||
|
package trafficusage
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"database/sql/driver"
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/gogf/gf/v2/frame/g"
|
||||||
|
"github.com/gogf/gf/v2/os/gcfg"
|
||||||
|
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/library/tdengine"
|
||||||
|
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
type routingDriver struct{ statements []string }
|
||||||
|
type routingConnection struct{ owner *routingDriver }
|
||||||
|
|
||||||
|
func (recorder *routingDriver) Open(string) (driver.Conn, error) {
|
||||||
|
return &routingConnection{owner: recorder}, nil
|
||||||
|
}
|
||||||
|
func (connection *routingConnection) Prepare(string) (driver.Stmt, error) {
|
||||||
|
return nil, errors.New("unexpected prepare")
|
||||||
|
}
|
||||||
|
func (connection *routingConnection) Close() error { return nil }
|
||||||
|
func (connection *routingConnection) Begin() (driver.Tx, error) {
|
||||||
|
return nil, errors.New("unexpected transaction")
|
||||||
|
}
|
||||||
|
func (connection *routingConnection) Ping(context.Context) error { return nil }
|
||||||
|
func (connection *routingConnection) ExecContext(_ context.Context, statement string, _ []driver.NamedValue) (driver.Result, error) {
|
||||||
|
connection.owner.statements = append(connection.owner.statements, statement)
|
||||||
|
return driver.RowsAffected(1), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var routingRecorder = &routingDriver{}
|
||||||
|
|
||||||
|
func init() {
|
||||||
|
sql.Register("traffic_usage_split_routing", routingRecorder)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRepositoryWritesToConfiguredSplitDatabase(t *testing.T) {
|
||||||
|
recorder := routingRecorder
|
||||||
|
recorder.statements = nil
|
||||||
|
previous := g.Cfg().GetAdapter()
|
||||||
|
t.Cleanup(func() { _ = tdengine.Close(); g.Cfg().SetAdapter(previous) })
|
||||||
|
adapter, err := gcfg.NewAdapterContent("tsdb:\n database: TdEngine\n tdengine:\n type: traffic_usage_split_routing\n dsn: test\n dbName: legacy\n dailyDbName: iotlcm_daily\n monthlyDbName: iotlcm_monthly\n")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
g.Cfg().SetAdapter(adapter)
|
||||||
|
_ = tdengine.Close()
|
||||||
|
repository := New()
|
||||||
|
period := time.Date(2026, 10, 21, 0, 0, 0, 0, time.FixedZone("CST", 8*60*60))
|
||||||
|
base := model.TrafficUsageBase{Iccid: "8986112520609988658", UsageMb: 12.5, PackageCode: "test"}
|
||||||
|
if err := repository.UpdateDaily(context.Background(), []model.TrafficUsageDailyItem{{TrafficUsageBase: base, Date: period}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := repository.UpdateMonthly(context.Background(), []model.TrafficUsageMonthlyItem{{TrafficUsageBase: base, Month: period}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(recorder.statements) != 2 {
|
||||||
|
t.Fatalf("statements=%v", recorder.statements)
|
||||||
|
}
|
||||||
|
for index, expected := range []string{
|
||||||
|
"INSERT INTO iotlcm_daily.sim_card_usage_d_8986112520609988658 USING iotlcm_daily.t_sim_card_usage_daily",
|
||||||
|
"INSERT INTO iotlcm_monthly.sim_card_usage_m_8986112520609988658 USING iotlcm_monthly.t_sim_card_usage_monthly",
|
||||||
|
} {
|
||||||
|
if !strings.HasPrefix(recorder.statements[index], expected) {
|
||||||
|
t.Errorf("statement=%s want prefix=%s", recorder.statements[index], expected)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := repository.UpdateDaily(context.Background(), nil); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := repository.UpdateMonthly(context.Background(), nil); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(recorder.statements) != 2 {
|
||||||
|
t.Fatal("empty updates issued SQL")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -332,7 +332,7 @@ func (r *Repository) SaveValidationFailure(ctx context.Context, task *unsub.Task
|
|||||||
}
|
}
|
||||||
reason := task.FailureReason
|
reason := task.FailureReason
|
||||||
if reason == "" {
|
if reason == "" {
|
||||||
reason = "batch validation failed; unsub request was skipped"
|
reason = "批次校验未通过,已跳过退订请求"
|
||||||
}
|
}
|
||||||
now := gtime.Now()
|
now := gtime.Now()
|
||||||
successCount := 0
|
successCount := 0
|
||||||
|
|||||||
@@ -46,7 +46,7 @@ func (r *Repository) loadParentTaskTx(ctx context.Context, tx gdb.TX, parent *en
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
detail := &Detail{OriginalSnapshot: batchDetail.Snapshot, ID: batchDetail.Id, BatchNo: batchDetail.BatchNo, RowNo: batchDetail.RowNo, DetailStatus: batchDetail.DetailStatus, ICCID: strings.TrimSpace(batchDetail.ItemId), Reason: strings.TrimSpace(batchDetail.ErrorReason)}
|
detail := &Detail{OriginalSnapshot: batchDetail.Snapshot, ID: batchDetail.Id, BatchNo: batchDetail.BatchNo, RowNo: batchDetail.RowNo, DetailStatus: batchDetail.DetailStatus, ICCID: strings.TrimSpace(batchDetail.ItemId), Reason: strings.TrimSpace(batchDetail.ErrorReason)}
|
||||||
if detail.DetailStatus != consts.BatchDetailStatusSuccess && detail.Reason == "" {
|
if detail.DetailStatus != consts.BatchDetailStatusSuccess && detail.DetailStatus != consts.BatchDetailStatusRunning && detail.Reason == "" {
|
||||||
detail.Reason = "子批次明细尚未全部验证成功"
|
detail.Reason = "子批次明细尚未全部验证成功"
|
||||||
}
|
}
|
||||||
if snapshot := batchDetail.Snapshot; snapshot != nil {
|
if snapshot := batchDetail.Snapshot; snapshot != nil {
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user