Compare commits

...
32 Commits
Author SHA1 Message Date
richjohnrambo 407842db91 fix: prepare native cluster deployment 2026-09-21 13:52:34 +08:00
richjohnrambo 5f717c2162 refactor: read user profile fields from profile table 2026-09-21 13:44:04 +08:00
Austin d398467fb9 docs: add IM contact us interface docking documentation
Complete the 4251-line docking document for IM contact us interface, including function description, interface definition, client rendering rules, local debugging example, operation configuration steps and reference implementation
2026-09-21 13:28:12 +08:00
Austin 9748c7a478 Merge branch 'dev' of https://git.118im.com/im_team/im_server into dev 2026-09-21 13:14:30 +08:00
Austin 7829990180 feat: 添加联系我们配置接口和字典配置缓存
1. 新增DictionaryItem字典配置结构体
2. 添加字典配置缓存及失效逻辑
3. 实现联系我们接口的业务逻辑和测试用例
4. 注册联系我们API路由
5. 集成字典配置到admin内部客户端
6. 处理字典配置更新的kafka事件
2026-09-21 13:11:42 +08:00
richjohnrambo 05ad37ba80 fix: persist settings for websocket groups 2026-09-21 11:58:36 +08:00
richjohnrambo c891afba73 test: 更新登录锁定表初始化断言 2026-09-21 09:55:29 +08:00
richjohnrambo e3b93113f1 feat: 接入消息加密与实名认证能力 2026-09-21 09:53:39 +08:00
Austin 841c0c77cd feat: 实现登录失败锁定功能
新增了登录失败次数统计和临时锁定机制,包括:
1. 定义登录锁定数据结构和数据库表结构
2. 实现租户级别的锁定状态持久化接口
3. 添加登录速率限制安全策略配置
4. 在登录流程中加入锁定前置检查和失败计数逻辑
5. 提供管理员手动解锁的内部API接口
6. 新增中文本地化的锁定提示文案
2026-09-20 17:08:38 +08:00
Austin 21063d4af0 Merge branch 'dev' of https://git.118im.com/im_team/im_server into dev 2026-09-20 16:39:46 +08:00
Austin 532ff49737 fix(im): parse Drafty entities for message type classification (#31)
Favorites from Drafty messages (link/file/image/video/voice) always fell back to text because imMessageType only recognized flat file_id/url content. Derive the type from Drafty ent[] entities: LN->link, IM->image, VD->video, AU->voice, EX refined by mime. Attachments win over inline links.
2026-09-20 16:38:51 +08:00
richjohnrambo 5c276596b8 fix: support removing group members 2026-09-20 15:37:56 +08:00
Austin 48b3369799 Merge branch 'dev' of https://git.118im.com/im_team/im_server into dev 2026-09-20 14:52:08 +08:00
Austin b2e8b64c0b refactor(mysql): 适配PXC严格模式,替换全局锁方案
重构MySQL迁移逻辑,新增acquireMigrationLock适配Percona XtraDB Cluster的pxc_strict_mode限制,将原有的GET_LOCK迁移锁替换为更兼容的方案,同时为各迁移流程添加快速检查路径减少锁竞争
2026-09-20 14:52:01 +08:00
richjohnrambo db66a965fa feat: 支持养成后台调用IM内部接口 2026-09-19 17:07:39 +08:00
richjohnrambo de01a95c96 Merge remote-tracking branch 'origin/dev' into dev 2026-09-19 15:25:40 +08:00
richjohnrambo a08c0bd921 feat: 提供养成服务内部 IM 能力 2026-09-19 15:24:55 +08:00
richjohnrambo 0fcbbb9318 fix: publish member joined after group invitation 2026-09-18 20:58:00 +08:00
Austin 753a04ed12 fix: broadcast offline presence as soon as the last session drops
Logout left contacts seeing the user online until the me topic idle timeout
fired presUsersOfInterest(off), and even that frame was silently dropped
for p2p routes by handlePresence's xoriginal check. MarkUserOffline now
returns the remaining session count and cleanUp notifies users of interest
immediately once it hits zero; p2p presence is delivered straight to the
contact's me topic, mirroring presSingleUserOffline. Zentao #22.
2026-09-18 17:42:18 +08:00
Austin f4cedc7d75 fix: push realtime del presence to topic sessions on message recall
The REST recall response reached only the caller, so peers kept showing the
recalled message until a full resync. broadcastIMMessageRecalled now pushes
the native {pres what:del delseq} tombstone through the hub for both single
and batch recall (empty pres topic for p2p, required by handlePresence's
xoriginal check); clients flush the message and render the recalled
placeholder. Zentao #16, #2.
2026-09-18 17:42:09 +08:00
richjohnrambo 8b20118570 feat: enforce youth mode policy and improve search 2026-09-18 14:51:40 +08:00
richjohnrambo 9482bb19f6 fix: wake app for call pushes 2026-09-18 13:51:22 +08:00
richjohnrambo 153f541078 fix: add call push category 2026-09-18 13:38:57 +08:00
richjohnrambo 192bdede33 fix: route call accept across pods 2026-09-18 11:15:19 +08:00
Austin 4c809129eb Merge branch 'dev' of https://git.118im.com/im_team/im_server into dev 2026-09-18 10:56:00 +08:00
richjohnrambo 3a92d81fca feat: enforce friend add and curfew policies 2026-09-18 10:55:53 +08:00
Austin 7a52db3437 fix: auto-provision im_user_privacy_settings columns on startup
Zentao #24/#25: commit 50f8581 added friend_request_verification and group_invite_verification to userPrivacyByUID, but existing databases lack those columns, so GetPrivacy failed with Error 1054 and every cross-user profile/qrcodes/friend-request endpoint returned 500.

Add user_privacy_schema.go which, under GET_LOCK, checks information_schema and ALTERs in any missing privacy columns, wired into the root adapter Open() and per-tenant TenantAdapterFor so tenant-scoped databases are provisioned too.
2026-09-18 10:55:13 +08:00
richjohnrambo c6e7f6b923 feat: improve group member events and push delivery 2026-09-18 01:00:41 +08:00
richjohnrambo 0e2f4e73d1 refactor: remove workbench code from im server 2026-09-17 23:38:44 +08:00
richjohnrambo 69a777c81f feat: apply configured default user avatars 2026-09-17 21:36:19 +08:00
richjohnrambo b9aaccc830 fix: update uid encryption key defaults 2026-09-17 16:59:47 +08:00
richjohnrambo ac75caba86 fix: load uid key from manager config 2026-09-17 16:30:47 +08:00
111 changed files with 6774 additions and 9715 deletions
+1 -1
View File
@@ -224,7 +224,7 @@ You can specify the following environment variables when issuing `docker run` co
| `TLS_DOMAIN_NAME`[^2] | string | | If non-empty, enables TLS (http**s**) and configures domain name of your container, e.g. `www.example.com`. In order for TLS to work you have to expose your HTTPS port to the Internet and correctly configure DNS. It WILL FAIL with `localhost` or unroutable IPs. | | `TLS_DOMAIN_NAME`[^2] | string | | If non-empty, enables TLS (http**s**) and configures domain name of your container, e.g. `www.example.com`. In order for TLS to work you have to expose your HTTPS port to the Internet and correctly configure DNS. It WILL FAIL with `localhost` or unroutable IPs. |
| `TNPG_AUTH_TOKEN` | string | | Tinode Push Gateway authentication token. | | `TNPG_AUTH_TOKEN` | string | | Tinode Push Gateway authentication token. |
| `TNPG_ORG`[^2] | string | | Tinode Push Gateway organization name as registered at https://console.tinode.co | | `TNPG_ORG`[^2] | string | | Tinode Push Gateway organization name as registered at https://console.tinode.co |
| `UID_ENCRYPTION_KEY`[^2] | string | `la6YsO+bNX/+XIkOqc5Svw==` | base64-encoded 16 random bytes used as an encryption key for user IDs. | | `UID_ENCRYPTION_KEY`[^2] | string | `dkXxWLwqUNRfYmdXRzOkZw==` | base64-encoded 16 random bytes used as an encryption key for user IDs. |
| `UPGRADE_DB` | bool | `false` | Upgrade database schema, if necessary. | | `UPGRADE_DB` | bool | `false` | Upgrade database schema, if necessary. |
| `WAIT_FOR` | string | | If non-empty, waits for the specified database `host:port` to be available before starting the server. | | `WAIT_FOR` | string | | If non-empty, waits for the specified database `host:port` to be available before starting the server. |
+1 -1
View File
@@ -153,7 +153,7 @@ ENV API_KEY_SALT=T713/rYYgW7g4m3vG6zGRh7+FM1t0T8j13koXScOAj4=
ENV AUTH_TOKEN_KEY=wfaY2RgF2S1OQI/ZlK+LSrp1KB2jwAdGAIHQ7JZn+Kc= ENV AUTH_TOKEN_KEY=wfaY2RgF2S1OQI/ZlK+LSrp1KB2jwAdGAIHQ7JZn+Kc=
# Key to initialize UID generator # Key to initialize UID generator
ENV UID_ENCRYPTION_KEY=la6YsO+bNX/+XIkOqc5Svw== ENV UID_ENCRYPTION_KEY=dkXxWLwqUNRfYmdXRzOkZw==
# Disable TLS by default. # Disable TLS by default.
ENV TLS_ENABLED=false ENV TLS_ENABLED=false
+1 -5
View File
@@ -271,11 +271,7 @@
"cluster_config": { "cluster_config": {
"self": "", "self": "",
"nodes": [ "nodes": $CLUSTER_NODES_JSON,
{"name": "tinode-0", "addr": "tinode-0:12000"},
{"name": "tinode-1", "addr": "tinode-1:12001"},
{"name": "tinode-2", "addr": "tinode-2:12002"}
],
"failover": { "failover": {
"enabled": true, "enabled": true,
"heartbeat": 100, "heartbeat": 100,
+3
View File
@@ -13,6 +13,9 @@ else
CONFIG=working.config CONFIG=working.config
: "${REDIS_KEY_PREFIX:=im}" : "${REDIS_KEY_PREFIX:=im}"
if [ -z "$CLUSTER_NODES_JSON" ] ; then
CLUSTER_NODES_JSON='[{"name":"tinode-0","addr":"tinode-0:12000"},{"name":"tinode-1","addr":"tinode-1:12001"},{"name":"tinode-2","addr":"tinode-2:12002"}]'
fi
# Remove the old config. # Remove the old config.
rm -f working.config rm -f working.config
@@ -2,11 +2,12 @@
## 1. 适用范围 ## 1. 适用范围
本文档用于 iOS、Android 客户端检查新版本,以及执行推荐更新和强制更新。 本文档用于 iOS、Android、Web 客户端检查新版本,以及执行推荐更新和强制更新。
- 版本策略为全局配置,存储在 `hotgo.im_app_releases`,不区分租户。 - 版本策略为全局配置,存储在 `hotgo.im_app_releases`,不区分租户。
- iOS 安装包由 App Store 托管,客户端通过 `store_url` 跳转。 - iOS 安装包由 App Store 托管,客户端通过 `store_url` 跳转。
- Android APK 由发布人员上传到专用 OSS 发布目录,客户端通过 `package_url` 下载。 - Android APK 由发布人员上传到专用 OSS 发布目录,客户端通过 `package_url` 下载。
- Web 端没有安装包时,`store_url` 和 `package_url` 可以为空,使用 `update_type`、`min_supported_version` 和 `release_notes` 控制强刷、兼容提醒或维护提示。
- 客户端不直接访问数据库,也不自行拼接 OSS 地址。 - 客户端不直接访问数据库,也不自行拼接 OSS 地址。
## 2. 检查版本接口 ## 2. 检查版本接口
@@ -22,14 +23,14 @@ GET https://api.118im.com/v1/im/app/releases/latest
| Header | 必填 | 示例 | 说明 | | Header | 必填 | 示例 | 说明 |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| `X-IM-Schema` | 是 | `1` | MeetingIM HTTP 协议版本 | | `X-IM-Schema` | 是 | `1` | MeetingIM HTTP 协议版本 |
| `X-Client-Platform` | 建议 | `ios` / `android` | 客户端平台 | | `X-Client-Platform` | 建议 | `ios` / `android` / `web` | 客户端平台 |
### 2.2 Query 参数 ### 2.2 Query 参数
| 参数 | 必填 | 示例 | 说明 | | 参数 | 必填 | 示例 | 说明 |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| `apikey` | 是 | `AQAAAA...` | 应用 API Key | | `apikey` | 是 | `AQAAAA...` | 应用 API Key |
| `platform` | 是 | `android` | 仅支持 `ios`、`android` | | `platform` | 是 | `android` | 仅支持 `ios`、`android`、`web` |
| `current_version` | 是 | `1.2.0` | 当前版本,使用数字点分格式 | | `current_version` | 是 | `1.2.0` | 当前版本,使用数字点分格式 |
| `channel` | 否 | `stable` | 发布渠道,默认 `stable` | | `channel` | 否 | `stable` | 发布渠道,默认 `stable` |
@@ -190,7 +191,7 @@ shasum -a 256 meetingim-1.3.0.apk
| --- | --- | | --- | --- |
| 400 | 缺少 `X-IM-Schema: 1` | | 400 | 缺少 `X-IM-Schema: 1` |
| 400 | 缺少或错误的 `current_version` | | 400 | 缺少或错误的 `current_version` |
| 400 | `platform` 不是 `ios` 或 `android` | | 400 | `platform` 不是 `ios`、`android` 或 `web` |
| 500 | Admin 版本配置缺失、格式错误或内部配置接口不可用 | | 500 | Admin 版本配置缺失、格式错误或内部配置接口不可用 |
## 9. 当前测试环境 ## 9. 当前测试环境
@@ -203,4 +204,4 @@ API Key: AQAAAAABAAAoeOI7tA3HsYvdzDhYhZJy
默认渠道: stable 默认渠道: stable
``` ```
当前测试库已配置 iOS、Android `1.0.0` 默认策略。正式验证 Android 更新前,需要先上传测试 APK,并在 Admin 填写对应 `package_url` 与 SHA-256。 当前测试库已配置 iOS、Android、Web `1.0.0` 默认策略。正式验证 Android 更新前,需要先上传测试 APK,并在 Admin 填写对应 `package_url` 与 SHA-256。
+28 -14
View File
@@ -438,7 +438,9 @@ X-MeetingIM-APIKey: <预置的普通 API Key>
"event": "group.member.joined", "event": "group.member.joined",
"payload": { "payload": {
"topic": "grp_xxx", "topic": "grp_xxx",
"user_id": "usr_xxx" "user_id": "usr_xxx",
"member_sync": "single",
"visibility": "all_members"
}, },
"ts": "2026-08-25T10:00:00Z" "ts": "2026-08-25T10:00:00Z"
} }
@@ -456,7 +458,9 @@ X-MeetingIM-APIKey: <预置的普通 API Key>
| 消息撤回 | Topic 在线成员 | `{"info":{"topic":"p2p_or_grp","from":"usr_actor","what":"im","event":"message.recalled","payload":{"topic":"p2p_or_grp","seq":12,"version":2}}}` | 按 `topic+seq+version` 重新拉取或本地标记已撤回 | | 消息撤回 | Topic 在线成员 | `{"info":{"topic":"p2p_or_grp","from":"usr_actor","what":"im","event":"message.recalled","payload":{"topic":"p2p_or_grp","seq":12,"version":2}}}` | 按 `topic+seq+version` 重新拉取或本地标记已撤回 |
| 消息编辑 | Topic 在线成员 | `{"info":{"topic":"p2p_or_grp","from":"usr_actor","what":"im","event":"message.edited","payload":{"topic":"p2p_or_grp","seq":12,"version":2}}}` | 按 `topic+seq+version` 重新拉取消息详情或修订记录 | | 消息编辑 | Topic 在线成员 | `{"info":{"topic":"p2p_or_grp","from":"usr_actor","what":"im","event":"message.edited","payload":{"topic":"p2p_or_grp","seq":12,"version":2}}}` | 按 `topic+seq+version` 重新拉取消息详情或修订记录 |
| 单聊消息变更脏标记 | P2P 双方在线会话 | `{"info":{"topic":"usr_target","from":"usr_actor","what":"im","event":"message.mutation","payload":{"topic":"p2p...","seq":12,"version":2,"event":"message.recalled"}}}` | 当前未订阅该 P2P 也要标记会话脏,并调用 `message-mutations` 做补偿同步 | | 单聊消息变更脏标记 | P2P 双方在线会话 | `{"info":{"topic":"usr_target","from":"usr_actor","what":"im","event":"message.mutation","payload":{"topic":"p2p...","seq":12,"version":2,"event":"message.recalled"}}}` | 当前未订阅该 P2P 也要标记会话脏,并调用 `message-mutations` 做补偿同步 |
| 群业务变化 | 群 Topic 在线成员 | `{"info":{"topic":"grp...","from":"usr_actor","what":"group","event":"group.member.joined","payload":{"topic":"grp...","user_id":"usr_xxx"}}}` | 按事件刷新群详情、成员列表、入群申请、公告、禁言或角色权限;除操作人外当前会写系统通知 | | 成员加入 | 群 Topic 在线成员 | `{"info":{"topic":"grp...","from":"usr_actor","what":"group","event":"group.member.joined","payload":{"user_id":"usr_xxx","member_sync":"single","visibility":"all_members"}}}` | 全体成员可见;调用 `GET /v1/im/groups/{topic}/members/{user_id}` 刷新单个成员。加入事件同时写入群消息,离线后可按 Topic seq 补拉 |
| 成员退出/移除、角色或禁言变化 | 群主、管理员和被操作人 | `{"info":{"topic":"grp...","from":"usr_actor","what":"group","event":"group.member.left","payload":{"user_id":"usr_xxx","member_sync":"remove","visibility":"admins_and_affected"}}}` | 普通成员不可见;`single` 拉取单成员,`remove` 本地删除,`full` 重新分页拉取全部成员 |
| 其他群业务变化 | 按具体事件授权范围 | `info.what="group"` | 刷新群详情、入群申请、公告或治理权限 |
| 会议变化 | 会议参会人 | `{"info":{"topic":"usr_target","from":"usr_actor","what":"im","event":"meeting.updated","payload":{"meeting_id":"mtg_1","version":2,"state":"scheduled"}}}` | 刷新会议列表/详情;除操作者外当前会写系统通知 | | 会议变化 | 会议参会人 | `{"info":{"topic":"usr_target","from":"usr_actor","what":"im","event":"meeting.updated","payload":{"meeting_id":"mtg_1","version":2,"state":"scheduled"}}}` | 刷新会议列表/详情;除操作者外当前会写系统通知 |
| 安全风险确认 | 操作人本人 | `{"info":{"topic":"usr_actor","from":"usr_actor","what":"im","event":"security.risk.continue","payload":{"risk_challenge_id":"risk_1","decision":"continue"}}}` | 刷新安全验证页面;当前会写系统通知 | | 安全风险确认 | 操作人本人 | `{"info":{"topic":"usr_actor","from":"usr_actor","what":"im","event":"security.risk.continue","payload":{"risk_challenge_id":"risk_1","decision":"continue"}}}` | 刷新安全验证页面;当前会写系统通知 |
@@ -526,7 +530,7 @@ X-MeetingIM-APIKey: <预置的普通 API Key>
| --- | --- | --- | | --- | --- | --- |
| `im_message` | `tenant_code,topic,seq` | 打开会话,按本地最后 seq 补齐消息并定位。 | | `im_message` | `tenant_code,topic,seq` | 打开会话,按本地最后 seq 补齐消息并定位。 |
| `group_mention` | `tenant_code,topic,seq,sender_id` | 打开群会话并定位被 @ 消息。 | | `group_mention` | `tenant_code,topic,seq,sender_id` | 打开群会话并定位被 @ 消息。 |
| `call_invite` | `tenant_code,topic,seq,call_id,call_type,caller_id` | 打开来电页前先登录 IM 并确认通话仍为有效来电。 | | `call_invite` | `tenant_code,topic,seq,call_id,call_type,caller_id,caller_display,expires_at` | 打开来电页前先登录 IM 并确认通话仍为有效来电。 |
| `call_missed` | `tenant_code,topic,call_id` | 打开通话记录或对应会话。 | | `call_missed` | `tenant_code,topic,call_id` | 打开通话记录或对应会话。 |
| `system_notification` | `tenant_code,notification_id` | 打开系统消息列表并刷新详情。 | | `system_notification` | `tenant_code,notification_id` | 打开系统消息列表并刷新详情。 |
| `friend_request` | `tenant_code,request_id` | 打开新朋友列表并刷新未处理数。 | | `friend_request` | `tenant_code,request_id` | 打开新朋友列表并刷新未处理数。 |
@@ -580,6 +584,8 @@ X-MeetingIM-APIKey: <预置的普通 API Key>
}, },
"sound": "default", "sound": "default",
"badge": "+1", "badge": "+1",
"category": "im_call",
"content-available": 1,
"extras": { "extras": {
"type": "call_invite", "type": "call_invite",
"tenant_code": "default", "tenant_code": "default",
@@ -587,18 +593,22 @@ X-MeetingIM-APIKey: <预置的普通 API Key>
"seq": "30", "seq": "30",
"call_id": "call_01K...", "call_id": "call_01K...",
"call_type": "audio", "call_type": "audio",
"caller_id": "usr_caller" "caller_id": "usr_caller",
"caller_display": "张三",
"expires_at": "2026-09-18T10:00:30Z"
} }
} }
}, },
"options": { "options": {
"time_to_live": 60, "time_to_live": 30,
"apns_production": false, "apns_production": false,
"classification": 1 "classification": 1
} }
} }
``` ```
来电推送必须走普通 APNs/极光 alert 通知,不使用 PushKit VoIP topic;iOS `notification.ios.category` 固定为 `im_call`,用于客户端通知按钮,同时带 `content-available: 1` 唤醒后台 App 拉取 `topic/seq` 并启动 Live Activity。`time_to_live` 和 `expires_at` 以 30 秒左右短有效期为准,过期后服务端不再投递。
#### 0.5.5 Long Polling 与文件基础示例 #### 0.5.5 Long Polling 与文件基础示例
##### 0.5.5.1 WebSocket 与文件基础接口 ##### 0.5.5.1 WebSocket 与文件基础接口
@@ -988,7 +998,7 @@ X-MeetingIM-APIKey: <预置的普通 API Key>
| `POST /v1/im/auth/password-reset/confirm` | `{"reset_token":"reset_token_123","new_password":"NewPass123"}` | `{"status":"ok"}` | | `POST /v1/im/auth/password-reset/confirm` | `{"reset_token":"reset_token_123","new_password":"NewPass123"}` | `{"status":"ok"}` |
| `POST /admin/v1/tenant/users/{user_id}/reset-password` 管理后台 | `{"new_password":"AdminReset123","reason":"用户无手机号无法自助找回"}` | `{"user_id":"usr...","status":"reset","auth_version":2}` | | `POST /admin/v1/tenant/users/{user_id}/reset-password` 管理后台 | `{"new_password":"AdminReset123","reason":"用户无手机号无法自助找回"}` | `{"user_id":"usr...","status":"reset","auth_version":2}` |
| `POST /v1/im/auth/logout` | `{}` | `{"status":"ok"}` | | `POST /v1/im/auth/logout` | `{}` | `{"status":"ok"}` |
| `GET /v1/im/app/releases/latest?platform=ios\|android&current_version=1.0.0&channel=stable` | Query only | `{"current_version":"1.0.0","has_update":true,"force_update":false,"platform":"ios","channel":"stable","latest_version":"1.1.0","min_supported_version":"1.0.0","update_type":"recommended","package_url":"","store_url":"https://apps.apple.com/app/id...","release_notes":"修复已知问题。","remind_after_seconds":86400}` | | `GET /v1/im/app/releases/latest?platform=ios\|android\|web&current_version=1.0.0&channel=stable` | Query only | `{"current_version":"1.0.0","has_update":true,"force_update":false,"platform":"web","channel":"stable","latest_version":"1.1.0","min_supported_version":"1.0.0","update_type":"recommended","package_url":"","store_url":"","release_notes":"修复已知问题。","remind_after_seconds":86400}` |
## 2. 消息首页、全局导航与系统通知 ## 2. 消息首页、全局导航与系统通知
@@ -2173,7 +2183,9 @@ GET /v1/im/topics/{topic}/message-mutations?since_version={localMutationVersion}
| `PATCH /v1/im/groups/{topic}/member-settings` | `{"pinned":true,"mute_notifications":false,"version":1}` | `{"topic":"grp...","pinned":true,"mute_notifications":false,"version":2,"updated_at":"2026-08-20T09:00:00Z"}` | | `PATCH /v1/im/groups/{topic}/member-settings` | `{"pinned":true,"mute_notifications":false,"version":1}` | `{"topic":"grp...","pinned":true,"mute_notifications":false,"version":2,"updated_at":"2026-08-20T09:00:00Z"}` |
| `PUT /v1/im/groups/{topic}/members/me/profile` | `{"nickname":"我在群里的昵称","version":1}` | `{"nickname":"我在群里的昵称","version":2,"updated_at":"2026-08-20T09:00:00Z"}` | | `PUT /v1/im/groups/{topic}/members/me/profile` | `{"nickname":"我在群里的昵称","version":1}` | `{"nickname":"我在群里的昵称","version":2,"updated_at":"2026-08-20T09:00:00Z"}` |
| `POST /v1/im/groups/{topic}/leave` | `{}` | `{"topic":"grp...","status":"left"}` | | `POST /v1/im/groups/{topic}/leave` | `{}` | `{"topic":"grp...","status":"left"}` |
| `GET /v1/im/groups/{topic}/members?query=&cursor=&mentionable=true` | Query only | `{"topic":"grp...","items":[{"user_id":"usr...","display_name":"张三","group_nickname":"小张","avatar_file_id":"","role":"member","mentionable":true,"acs":{"want":"JRWP","given":"JRWP","mode":"JRWP"}}],"next_cursor":"","permissions":{"can_mention_all":true}}` | | `GET /v1/im/groups/{topic}/members?query=&cursor=&mentionable=true` | Query only | `{"topic":"grp...","items":[{"user_id":"usr...","display_name":"张三","group_nickname":"小张","avatar_file_id":"","role":"member","member_status":"active","version":1,"joined_at":"2026-08-20T09:00:00Z","updated_at":"2026-08-20T09:00:00Z","mentionable":true,"acs":{"want":"JRWP","given":"JRWP","mode":"JRWP"}}],"next_cursor":"","sync_mode":"full","permissions":{"can_mention_all":true}}` |
| `GET /v1/im/groups/{topic}/members/{user_id}` | Path topic/user | `{"topic":"grp...","user_id":"usr...","display_name":"张三","group_nickname":"小张","role":"member","member_status":"active","version":1,"joined_at":"2026-08-20T09:00:00Z","updated_at":"2026-08-20T09:00:00Z","sync_mode":"single","acs":{"want":"JRWP","given":"JRWP","mode":"JRWP"}}` |
| `DELETE /v1/im/groups/{topic}/members/{user_id}` | Path topic/user;管理员可移除普通成员,群主可移除管理员和普通成员;不能移除群主或自己 | `{"topic":"grp...","user_id":"usr...","status":"left"}` |
| `GET /v1/im/groups/{topic}/members/{user_id}/profile` | Path topic/user;`topic` 必须是 `grp...`,不能传 `usr...` | `{"topic":"grp...","user_id":"usr...","display_name":"张三","group_nickname":"小张","role":"member","relationship":"friend","capabilities":{"can_send":true,"can_call":true}}` | | `GET /v1/im/groups/{topic}/members/{user_id}/profile` | Path topic/user;`topic` 必须是 `grp...`,不能传 `usr...` | `{"topic":"grp...","user_id":"usr...","display_name":"张三","group_nickname":"小张","role":"member","relationship":"friend","capabilities":{"can_send":true,"can_call":true}}` |
| `POST /v1/im/groups/{topic}/qrcodes` | `{"expires_in_seconds":86400}` | `{"qr_token":"token...","qr_payload":"im://qr?token=...&purpose=join_group","expires_at":"2026-08-21T09:00:00Z","group":{"topic":"grp...","name":"项目群","member_count":2},"group_snapshot":{"topic":"grp...","name":"项目群","member_count":2}}` | | `POST /v1/im/groups/{topic}/qrcodes` | `{"expires_in_seconds":86400}` | `{"qr_token":"token...","qr_payload":"im://qr?token=...&purpose=join_group","expires_at":"2026-08-21T09:00:00Z","group":{"topic":"grp...","name":"项目群","member_count":2},"group_snapshot":{"topic":"grp...","name":"项目群","member_count":2}}` |
| `POST /v1/im/groups/{topic}/invitations` | `{"user_ids":["usr..."],"message":"进群一起讨论"}` | `{"topic":"grp...","items":[{"user_id":"usr...","status":"joined","invitation_id":"ginv_1"}]}` | | `POST /v1/im/groups/{topic}/invitations` | `{"user_ids":["usr..."],"message":"进群一起讨论"}` | `{"topic":"grp...","items":[{"user_id":"usr...","status":"joined","invitation_id":"ginv_1"}]}` |
@@ -2320,6 +2332,7 @@ GET /v1/im/topics/{topic}/message-mutations?since_version={localMutationVersion}
- `GET /v1/im/topics/{topic}/messages/search?q=&type=text&cursor=` 返回 `seq,from,ts,snippet,matches[]`。 - `GET /v1/im/topics/{topic}/messages/search?q=&type=text&cursor=` 返回 `seq,from,ts,snippet,matches[]`。
- 点击结果后 `GET /v1/im/topics/{topic}/messages/{seq}/context?before=20&after=20`,返回锚点及上下文;客户端定位 `seq` 并高亮。 - 点击结果后 `GET /v1/im/topics/{topic}/messages/{seq}/context?before=20&after=20`,返回锚点及上下文;客户端定位 `seq` 并高亮。
- 服务端只搜索当前用户可见且未硬删除的消息,关键词做长度/频率限制,搜索日志不得记录原文。 - 服务端只搜索当前用户可见且未硬删除的消息,关键词做长度/频率限制,搜索日志不得记录原文。
- 搜索匹配字段包括文本正文、`content.text/txt/content/title/summary/name`、文件名、链接标题/摘要、卡片/转发快照等可展示文本;排序按 `seq desc`,`next_cursor` 继续向更早消息翻页。
### 7.13 查找聊天记录-文件.png ### 7.13 查找聊天记录-文件.png
@@ -2400,11 +2413,12 @@ GET /v1/im/topics/{topic}/message-mutations?since_version={localMutationVersion}
<img src="images/对方状态.png" alt="对方状态" width="360"> <img src="images/对方状态.png" alt="对方状态" width="360">
**状态:在线邀请已实现;Push 与产品鉴权需改造。** **状态:在线邀请、短有效期普通 Push 与产品鉴权已实现。**
- 来电是 Topic 下行的初始 `{data}`:`head.webrtc="started"`、`head.x-im-type="call"`,`content` 至少含 `callId,callType,audio,video`。客户端保存该消息 `seq` 作为后续所有 call note 的关联 ID。 - 来电是 Topic 下行的初始 `{data}`:`head.webrtc="started"`、`head.x-im-type="call"`,`content` 至少含 `callId,callType,audio,video`。客户端保存该消息 `seq` 作为后续所有 call note 的关联 ID。
- 来电端开始响铃后发送 `{note:{topic,what:"call",event:"ringing",seq,payload:{"callId":"..."}}}`;接听发送 `event:"accept"`;拒绝发送 `event:"hang-up"`。 - 来电端开始响铃后发送 `{note:{topic,what:"call",event:"ringing",seq,payload:{"callId":"..."}}}`;接听发送 `event:"accept"`;拒绝发送 `event:"hang-up"`。
- 后台 Push 只携带 `callId,topic,seq,caller_display,callType,expires_at`,不含 SDP、ICE 或 Token;用户操作后必须回 WebSocket 鉴权。 - 后台 Push 只携带 `callId,topic,seq,caller_id,caller_display,callType,expires_at`,不含 SDP、ICE 或 Token;iOS 来电 Push 使用普通 alert,`category=im_call`,`content-available=1`,短 TTL 约 30 秒,不使用 PushKit VoIP。
- 被叫在未发送 `ringing/accept` 前也可以直接发送 `hang-up`,服务端按 `topic+seq+callId` 校验后立即结束主叫侧振铃,并把原通话消息替换为 `declined`;主叫主动取消则替换为 `missed`。
### 8.3 语音通话-对方待接受状态.png ### 8.3 语音通话-对方待接受状态.png
@@ -2990,7 +3004,7 @@ GET /v1/im/topics/{topic}/message-mutations?since_version={localMutationVersion}
完整接口与 Android OSS APK 对接流程见:[IM-App版本更新前后端对接文档](./IM-App版本更新前后端对接文档.md)。 完整接口与 Android OSS APK 对接流程见:[IM-App版本更新前后端对接文档](./IM-App版本更新前后端对接文档.md)。
- `GET /v1/im/app/releases/latest?platform=ios|android&current_version=1.0.0&channel=stable`,全局公共接口,不依赖租户解析。`current_version` 必传,使用数字点分版本号(如 `1.2.0`)。 - `GET /v1/im/app/releases/latest?platform=ios|android|web&current_version=1.0.0&channel=stable`,全局公共接口,不依赖租户解析。`current_version` 必传,使用数字点分版本号(如 `1.2.0`)。
- 服务端返回 `has_update`、`force_update` 和 `update_type=none|recommended|force`;iOS 客户端使用 `store_url` 跳转 App Store。 - 服务端返回 `has_update`、`force_update` 和 `update_type=none|recommended|force`;iOS 客户端使用 `store_url` 跳转 App Store。
- Android 客户端使用 `package_url` 下载 OSS/CDN 上的 APK,并在安装前使用 `checksum` 完成 SHA-256 校验;强制更新时不得绕过进入主业务。 - Android 客户端使用 `package_url` 下载 OSS/CDN 上的 APK,并在安装前使用 `checksum` 完成 SHA-256 校验;强制更新时不得绕过进入主业务。
- 版本策略由 Admin 在 `hotgo.im_app_releases` 统一管理,IM Server 通过 HMAC 内部配置接口读取,不从租户库查询。 - 版本策略由 Admin 在 `hotgo.im_app_releases` 统一管理,IM Server 通过 HMAC 内部配置接口读取,不从租户库查询。
@@ -3032,7 +3046,7 @@ GET /v1/im/topics/{topic}/message-mutations?since_version={localMutationVersion}
**状态:新增。** **状态:新增。**
- `GET /v1/im/settings/youth-mode` 返回 `enabled,policy_version,restrictions,restriction_items,password_set,can_enable`。当前默认策略项为 `night_quiet`、`content_filter`、`payment_guard` 三条;后端现阶段完成状态持久化、开启/关闭校验和策略返回,业务接口层的深夜免打扰、内容过滤、支付守护尚未实际强制执行,`restriction_items[].enforced=false` 用于明确表达这一点。 - `GET /v1/im/settings/youth-mode` 返回 `enabled,policy_version,restrictions,restriction_items,password_set,can_enable`。当前默认策略项为 `night_quiet`、`content_filter`、`payment_guard` 三条;后端现阶段完成状态持久化、开启/关闭校验和策略返回;`night_quiet` 已接入音视频呼叫发起限制,`content_filter` 已接入消息发送前内容安全审核,二者返回 `enforced=true`;`payment_guard` 仍为预留项,返回 `enforced=false`。
- 点击开启先展示 10.15,再按 1.7 完成身份验证和 1.11 设置独立密码;最后调用 `POST /v1/im/settings/youth-mode/enable`,Body `{"policy_version":n,"action_token":"..."}`,其中 `action_token` 使用设置独立密码响应里的 `enable_action_token`。 - 点击开启先展示 10.15,再按 1.7 完成身份验证和 1.11 设置独立密码;最后调用 `POST /v1/im/settings/youth-mode/enable`,Body `{"policy_version":n,"action_token":"..."}`,其中 `action_token` 使用设置独立密码响应里的 `enable_action_token`。
- 开启链路中,短信校验返回的 `set_independent_password` token 只用于设置独立密码;设置密码成功响应中的 `enable_action_token` 才用于 `youth-mode/enable`。 - 开启链路中,短信校验返回的 `set_independent_password` token 只用于设置独立密码;设置密码成功响应中的 `enable_action_token` 才用于 `youth-mode/enable`。
- 服务端持久化到用户租户范围并推送所有设备,不能仅设置本地标志。 - 服务端持久化到用户租户范围并推送所有设备,不能仅设置本地标志。
@@ -3053,7 +3067,7 @@ GET /v1/im/topics/{topic}/message-mutations?since_version={localMutationVersion}
**状态:新增。** **状态:新增。**
- 页面调用 10.14,`enabled=true` 时展示 `enabled_at,restrictions`。`bootstrap` 同时返回受限模块和动作。 - 页面调用 10.14,`enabled=true` 时展示 `enabled_at,restrictions`。`bootstrap` 同时返回受限模块和动作。
- 后续某条规则真正接入业务拦截后,受限接口服务端返回 `YOUTH_MODE_RESTRICTED`,并把对应 `restriction_items[].enforced` 改为 `true`;前端隐藏只是体验优化,不能作为安全控制。 - 受限接口服务端返回 `YOUTH_MODE_RESTRICTED`,当前已覆盖夜间音视频呼叫发起;前端隐藏只是体验优化,不能作为安全控制。
- 登录、App 重启和另一设备均从服务端恢复状态;收到 `youth_mode.updated` 即刷新。 - 登录、App 重启和另一设备均从服务端恢复状态;收到 `youth_mode.updated` 即刷新。
### 10.17 关闭青少年模式(安卓).png ### 10.17 关闭青少年模式(安卓).png
@@ -3122,10 +3136,10 @@ GET /v1/im/topics/{topic}/message-mutations?since_version={localMutationVersion}
| `PATCH /v1/im/me/profile` | `{"nickname":"新昵称","signature":"新签名","version":1}` | `{"avatar_file_id":"","nickname":"新昵称","signature":"新签名","version":2,"updated_at":"2026-08-20T09:00:00Z"}` | | `PATCH /v1/im/me/profile` | `{"nickname":"新昵称","signature":"新签名","version":1}` | `{"avatar_file_id":"","nickname":"新昵称","signature":"新签名","version":2,"updated_at":"2026-08-20T09:00:00Z"}` |
| `GET /v1/im/me/privacy` | Query only | `{"add_me_methods":{"account":true,"phone":true,"qr":true,"group":true},"profile_visibility":"friends","last_seen_visibility":"friends","read_receipts":true,"friend_request_verification":true,"group_invite_verification":false,"version":1,"editable":{"add_me_methods":true,"profile_visibility":true,"last_seen_visibility":true,"read_receipts":true,"friend_request_verification":true,"group_invite_verification":true}}` | | `GET /v1/im/me/privacy` | Query only | `{"add_me_methods":{"account":true,"phone":true,"qr":true,"group":true},"profile_visibility":"friends","last_seen_visibility":"friends","read_receipts":true,"friend_request_verification":true,"group_invite_verification":false,"version":1,"editable":{"add_me_methods":true,"profile_visibility":true,"last_seen_visibility":true,"read_receipts":true,"friend_request_verification":true,"group_invite_verification":true}}` |
| `PATCH /v1/im/me/privacy` | `{"add_me_methods":{"phone":false},"read_receipts":false,"friend_request_verification":false,"group_invite_verification":true,"version":1}` | `{"add_me_methods":{"account":true,"phone":false,"qr":true,"group":true},"read_receipts":false,"friend_request_verification":false,"group_invite_verification":true,"version":2,"updated_at":"2026-08-20T09:00:00Z"}` | | `PATCH /v1/im/me/privacy` | `{"add_me_methods":{"phone":false},"read_receipts":false,"friend_request_verification":false,"group_invite_verification":true,"version":1}` | `{"add_me_methods":{"account":true,"phone":false,"qr":true,"group":true},"read_receipts":false,"friend_request_verification":false,"group_invite_verification":true,"version":2,"updated_at":"2026-08-20T09:00:00Z"}` |
| `GET /v1/im/settings/youth-mode` | Query only | `{"enabled":false,"policy_version":1,"restrictions":["night_quiet","content_filter","payment_guard"],"restriction_items":[{"code":"night_quiet","title":"深夜免打扰","description":"夜间减少消息提醒打扰,帮助用户保持健康作息。","enforced":false},{"code":"content_filter","title":"内容过滤","description":"对不适合青少年的内容进行识别和限制展示。","enforced":false},{"code":"payment_guard","title":"支付守护","description":"限制或确认支付、充值等敏感消费行为。","enforced":false}],"password_set":false,"can_enable":false,"updated_at":"2026-08-20T09:00:00Z"}` | | `GET /v1/im/settings/youth-mode` | Query only | `{"enabled":false,"policy_version":1,"restrictions":["night_quiet","content_filter","payment_guard"],"restriction_items":[{"code":"night_quiet","title":"深夜免打扰","description":"夜间减少消息提醒打扰,帮助用户保持健康作息。","enforced":true},{"code":"content_filter","title":"内容过滤","description":"对不适合青少年的内容进行识别和限制展示。","enforced":true},{"code":"payment_guard","title":"支付守护","description":"限制或确认支付、充值等敏感消费行为。","enforced":false}],"password_set":false,"can_enable":false,"updated_at":"2026-08-20T09:00:00Z"}` |
| `POST /v1/im/settings/independent-password` | `{"password":"1234","purpose":"youth_mode","action_token":"action_token"}` | `{"status":"ok","password_set":true,"purpose":"youth_mode","enable_action_token":"enable_token","enable_action_expires_at":"2026-08-20T09:05:00Z","next_action":"enable_youth_mode"}` | | `POST /v1/im/settings/independent-password` | `{"password":"1234","purpose":"youth_mode","action_token":"action_token"}` | `{"status":"ok","password_set":true,"purpose":"youth_mode","enable_action_token":"enable_token","enable_action_expires_at":"2026-08-20T09:05:00Z","next_action":"enable_youth_mode"}` |
| `POST /v1/im/settings/independent-password/verify` | `{"password":"1234","purpose":"close_youth_mode"}` | `{"action_token":"action_token","action":"close_youth_mode","expires_at":"2026-08-20T09:05:00Z"}` | | `POST /v1/im/settings/independent-password/verify` | `{"password":"1234","purpose":"close_youth_mode"}` | `{"action_token":"action_token","action":"close_youth_mode","expires_at":"2026-08-20T09:05:00Z"}` |
| `POST /v1/im/settings/youth-mode/enable` | `{"policy_version":1,"action_token":"enable_action_token"}` | `{"enabled":true,"policy_version":1,"password_set":true,"restrictions":["night_quiet","content_filter","payment_guard"],"restriction_items":[{"code":"night_quiet","title":"深夜免打扰","description":"夜间减少消息提醒打扰,帮助用户保持健康作息。","enforced":false},{"code":"content_filter","title":"内容过滤","description":"对不适合青少年的内容进行识别和限制展示。","enforced":false},{"code":"payment_guard","title":"支付守护","description":"限制或确认支付、充值等敏感消费行为。","enforced":false}],"updated_at":"2026-08-20T09:00:00Z"}` | | `POST /v1/im/settings/youth-mode/enable` | `{"policy_version":1,"action_token":"enable_action_token"}` | `{"enabled":true,"policy_version":1,"password_set":true,"restrictions":["night_quiet","content_filter","payment_guard"],"restriction_items":[{"code":"night_quiet","title":"深夜免打扰","description":"夜间减少消息提醒打扰,帮助用户保持健康作息。","enforced":true},{"code":"content_filter","title":"内容过滤","description":"对不适合青少年的内容进行识别和限制展示。","enforced":true},{"code":"payment_guard","title":"支付守护","description":"限制或确认支付、充值等敏感消费行为。","enforced":false}],"updated_at":"2026-08-20T09:00:00Z"}` |
| `POST /v1/im/settings/youth-mode/disable` | Header `X-Action-Token: action_token`;Body `{}` | `{"enabled":false,"policy_version":1,"password_set":true,"updated_at":"2026-08-20T09:00:00Z"}` | | `POST /v1/im/settings/youth-mode/disable` | Header `X-Action-Token: action_token`;Body `{}` | `{"enabled":false,"policy_version":1,"password_set":true,"updated_at":"2026-08-20T09:00:00Z"}` |
| `POST /v1/im/security/challenges` 开启青少年模式 | Header `Idempotency-Key`;Body `{"scene":"set_independent_password","channel":"sms"}` | `{"challenge_id":"sec_chg_1","phone_mask":"+861****000","expires_in_seconds":300,"resend_after_seconds":60}` | | `POST /v1/im/security/challenges` 开启青少年模式 | Header `Idempotency-Key`;Body `{"scene":"set_independent_password","channel":"sms"}` | `{"challenge_id":"sec_chg_1","phone_mask":"+861****000","expires_in_seconds":300,"resend_after_seconds":60}` |
| `POST /v1/im/security/challenges` 关闭青少年模式 | Header `Idempotency-Key`;Body `{"scene":"close_youth_mode","channel":"sms"}` | `{"challenge_id":"sec_chg_2","phone_mask":"+861****000","expires_in_seconds":300,"resend_after_seconds":60}` | | `POST /v1/im/security/challenges` 关闭青少年模式 | Header `Idempotency-Key`;Body `{"scene":"close_youth_mode","channel":"sms"}` | `{"challenge_id":"sec_chg_2","phone_mask":"+861****000","expires_in_seconds":300,"resend_after_seconds":60}` |
+227
View File
@@ -0,0 +1,227 @@
# IM 剩余问题修复方案
整理时间:2026-09-18
本文记录当前暂未完成的三个后端问题:青少年模式业务限制、Web 平台版本检查、消息搜索召回稳定性。
更新记录:
- 2026-09-18:P1 第一阶段已处理 Web 平台版本检查与消息搜索召回兜底;青少年模式已完成第一阶段策略层、夜间通话限制和消息内容过滤接入,剩余支付守护、好友/群邀请、Push 等全链路限制继续按 P2 增强。
## 1. 青少年模式业务限制未强制执行
### 现状
当前后端已经具备青少年模式的基础状态能力:
- `GET /v1/im/settings/youth-mode` 可返回 `enabled`、`policy_version`、`restrictions`、`restriction_items`。
- `POST /v1/im/settings/youth-mode/enable` 和 `disable` 已完成开启、关闭和独立密码校验链路。
- `im_youth_mode_settings` 已保存用户维度的启用状态、策略版本、限制项和密码状态。
第一阶段已将 `night_quiet` 接入音视频呼叫发起限制,将 `content_filter` 接入消息发送前内容安全审核,并把这两项 `restriction_items[].enforced` 改为 `true`。`payment_guard` 以及好友申请、群邀请、系统通知/Push 等更大范围入口仍未完整接入。
### 问题影响
如果只保存开关、不做服务端限制,客户端隐藏入口不能形成安全边界。用户可以通过旧客户端、重新安装、直接调用接口或构造 WebSocket 消息绕过限制。
### 建议修复方案
建议将青少年模式拆成一个统一策略检查层,例如 `imYouthPolicy` 或 `impolicy/youth`,所有关键入口调用同一套判断逻辑。
第一阶段建议先落这几个限制:
| 限制项 | 建议拦截点 | 修复方式 |
| --- | --- | --- |
| `night_quiet` 深夜免打扰 | 发消息、音视频呼叫、系统通知/Push | 服务端读取用户青少年状态和当前时间段,限制指定时段内发起呼叫或降低通知等级。 |
| `content_filter` 内容过滤 | 发文本、发链接、上传后发布文件、资料/群资料修改 | 复用现有内容安全/风险审核能力;命中高风险时拒绝或进入审核状态。 |
| `payment_guard` 支付守护 | 充值、购买、付费功能入口 | 当前 IM 暂无支付主链路,先预留统一错误码和策略接口,后续支付接口接入时强制调用。 |
建议新增统一返回:
```json
{
"code": 30005,
"reason": "YOUTH_MODE_RESTRICTED",
"message": "restricted by youth mode",
"details": {
"restriction": "night_quiet",
"policy_version": 1
}
}
```
### 建议改造点
- `server/im_http.go`
- HTTP 发消息、转发、收藏转发、好友申请、群邀请、音视频相关接口前置检查。
- `youthModeRestrictionItems` 中已真正接入的规则将 `enforced` 改为 `true`。
- `server/topic.go`
- 原生 WebSocket `{pub}` 保存消息前执行相同策略,避免绕过 HTTP。
- `server/store/store.go`
- 增加按用户读取青少年模式的轻量缓存,短 TTL 即可。
- `docs/IM-青少年模式前后端对接文档.md`
- 补充哪些接口会返回 `YOUTH_MODE_RESTRICTED`。
### 验收建议
- 开启青少年模式后,旧客户端直接发消息不能绕过限制。
- HTTP 和 WebSocket 两条链路限制结果一致。
- 关闭青少年模式后限制即时解除。
- `restriction_items[].enforced` 只对已经服务端强制执行的规则返回 `true`。
### 当前处理状态
已新增统一策略判断 `imYouthModePolicyDecision`。当前行为:
- `night_quiet`:青少年模式开启且带该限制时,22:00-06:00 发起音视频呼叫返回 `YOUTH_MODE_RESTRICTED`。
- `content_filter`:普通 WS 消息发送前进入内容安全审核,文本、文件名、链接 URL 等可展示内容会参与提取。
- `payment_guard`:仍为预留项,返回 `enforced=false`,后续支付或付费功能接入时再强制调用。
## 2. Web 平台版本检查接口缺失
### 现状
当前版本检查接口为:
```http
GET /v1/im/app/releases/latest?platform=ios|android&current_version=1.0.0&channel=stable
```
当前实现和文档主要覆盖 `ios`、`android`。Web 客户端如果调用 `platform=web`,会因为平台枚举未支持而无法得到可用版本配置。
### 问题影响
Web 端无法统一读取版本策略、升级提示、最低支持版本和发布说明。后续如果 Web 端需要灰度、强刷、维护提示或版本兼容控制,只能靠前端硬编码。
### 建议修复方案
将 `web` 加入 App Release 平台枚举。平台建议统一为:
```text
ios, android, web
```
Web 响应建议保持与移动端同结构:
```json
{
"current_version": "1.0.0",
"has_update": false,
"force_update": false,
"platform": "web",
"channel": "stable",
"latest_version": "1.0.0",
"min_supported_version": "1.0.0",
"update_type": "none",
"package_url": "",
"store_url": "",
"release_notes": "当前已是最新版本。",
"remind_after_seconds": 0
}
```
Web 没有安装包时,`package_url` 和 `store_url` 可以为空;如果需要强刷或维护提示,可通过 `update_type=required` 和 `release_notes` 传达。
### 建议改造点
- `server/im_http.go`
- 放开 `platform=web` 参数校验。
- 保持 `current_version` 必传。
- `server/db/mysql/adapter.go` 或 App Release 相关存储实现
- 确认 `im_app_releases.platform` 没有限制 `web`。
- 如数据库通过字典/配置初始化版本记录,需要补默认 Web 记录。
- `docs/IM-App版本更新前后端对接文档.md`
- 补充 Web 平台示例。
- `docs/IM-前后端对接文档.md`
- 将 `platform=ios|android` 更新为 `ios|android|web`。
### 验收建议
- `platform=web&current_version=1.0.0&channel=stable` 返回 200。
- `platform=invalid` 仍返回参数错误。
- Web 无更新、推荐更新、强制更新三类数据都能按同一结构返回。
### 当前处理状态
已在 `serveIMAppReleaseLatest` 放开 `platform=web`,并补充 Web 平台单元测试。对接文档已同步更新为 `ios|android|web`。
## 3. 消息搜索召回稳定性待复核
### 现状
当前接口为:
```http
GET /v1/im/topics/{topic}/messages/search?q=&type=text&cursor=
GET /v1/im/topics/{topic}/messages/search?type=file&q=&cursor=
```
现有后端已经能返回 `seq`、`from`、`ts`、`snippet`、`matches`、`snapshot`、`availability` 等字段。但问题清单中仍记录了“部分明显应该命中的关键词搜索返回空结果”的现象。该问题目前属于待复核项,可能来源包括:
- 搜索范围只覆盖部分消息内容字段,没有覆盖所有文本来源。
- 结构化消息中正文可能存在于 `content.text`、`content.txt`、`content.title`、`content.summary`、Drafty 或文件快照字段中,索引/过滤口径不一致。
- 中文分词、大小写、空格、标点归一化不一致。
- 分页游标与过滤顺序不一致,导致命中项被跳过。
- 热消息、分表消息和历史消息的查询路径不一致。
### 问题影响
搜索结果不稳定会直接影响聊天记录检索、文件检索、收藏定位和问题复盘。用户明知道某条消息存在,但搜索不到,会被认为是数据丢失。
### 建议修复方案
第一阶段先做“确定性兜底”,不急于引入复杂搜索引擎:
1. 明确搜索字段规范。
- 文本消息:统一提取 `content.text`、`content.txt`、`content.content`、`content.title`、`content.summary`、字符串 content。
- 文件消息:统一提取文件名、mime、文件说明、消息文本。
- 链接/名片/合并转发:提取标题、摘要、展示文本。
2. 增加 `message_search_text` 生成函数。
- 写入消息时生成可搜索文本。
- 老数据可查询时动态 fallback。
3. 查询顺序固定。
- 先按用户权限和 topic 过滤可见消息。
- 再按搜索文本匹配。
- 最后按 `seq desc` 或 `created_at desc` 分页。
4. 中文和符号归一化。
- 至少做大小写归一、前后空格、连续空白压缩、全角半角基础处理。
5. 增加回归测试。
- 普通文本、中文关键词、文件名、链接标题、转发快照、撤回/删除消息、分页边界都要覆盖。
第二阶段如果数据量和体验要求更高,再考虑引入专用搜索索引:
- MySQL FULLTEXT/ngram。
- OpenSearch/Elasticsearch。
- 独立 `im_message_search_index` 表,按 topic、seq、search_text 存储轻量索引。
### 建议改造点
- `server/im_http.go`
- `messageText` 和 `messageSearchPayload` 的提取逻辑继续收口,保证响应与查询使用同一文本口径。
- `server/db/mysql/adapter.go` 或消息查询实现
- 检查 `TenantMessageSearch` 的 SQL 过滤条件和分页顺序。
- 确认群消息分表 `im_group_messages_YYYYMM` 与私聊消息表搜索路径一致。
- `server/store/types`
- 如新增搜索索引表,补结构体和迁移 SQL。
- `docs/IM-前后端对接文档.md`
- 明确搜索匹配字段、排序规则和 `availability` 行为。
### 验收建议
- 同一 topic 下创建 20 条混合消息,逐条用关键词可搜到。
- 搜索中文短词、英文大小写、文件名、链接标题均有稳定结果。
- 翻页时不丢边界记录、不重复。
- 被删除/撤回消息按 `availability` 返回或按产品规则隐藏,行为一致。
- 私聊和群聊搜索结果字段一致。
### 当前处理状态
已完成第一阶段确定性兜底:私聊和群聊搜索不再只依赖 SQL `LIKE` 命中消息 JSON 原文,改为先按权限、Topic、类型和游标读取固定窗口,再对结构化文本做统一提取和归一化匹配。当前覆盖文本、文件名、标题、摘要、链接、卡片和转发快照等字段,并补充结构化内容召回测试。
## 建议排期
| 优先级 | 项目 | 原因 |
| --- | --- | --- |
| P1 | Web 平台版本检查 | 改动小,边界清楚,适合快速补齐。 |
| P1 | 消息搜索召回复核 | 影响用户感知,需要先补测试锁定问题。 |
| P2 | 青少年模式业务限制 | 涉及多个入口和策略口径,需要独立设计和联调,不能只改一个接口。 |
@@ -0,0 +1,256 @@
# IM 原生 Cluster 与 Redis Fanout 职责边界
## 1. 背景
IM Server 当前同时存在两套跨 Pod 能力:
- Tinode 原生 Cluster:以 Topic master 为中心,负责把 Topic 请求路由到唯一 master 节点,由 master 统一处理 Topic 状态机。
- Redis Fanout:以 Redis Pub/Sub 为跨 Pod 实时总线,负责把服务端生成的实时事件广播到其他 Pod,再由各 Pod 投递给本机 WebSocket session。
测试环境目前是多 Pod standalone 部署,日志表现为 `Cluster: running as a standalone server.`,因此当前实际跨 Pod 主链路是 Redis Fanout。目标架构需要启用原生 Cluster,把 Topic 热路径、强顺序状态机和实时通话状态迁回 Topic master;Redis Fanout 保留为大数据量、缓存、补偿和用户级控制能力。
本文定义两套机制的职责边界,避免后续同时启用后出现重复投递、重复状态变更或状态源不一致。
## 2. 总原则
1. 同一类 Topic 事件只能有一个实时主链路。
2. 原生 Cluster 启用时,Topic 状态机相关事件优先由 Cluster 处理。
3. Redis Fanout 不作为 Topic master 的替代品;它是跨 Pod 投递、通知和 session control 总线。
4. 关键业务状态必须以 MySQL、Redis 原子状态或 Kafka 顺序消费作为权威,WebSocket fanout 只负责实时通知。
5. 可丢的临时状态可以只走 fanout;不可丢的状态必须有历史补拉或条件更新兜底。
6. 能利用原生 Cluster 内存态、高性能、强一致和强顺序的 Topic 热路径,应优先交给原生 Cluster。
7. 数据量大、需要广播、缓存、聚合、补偿或跨业务模块共享的数据,应优先交给 Redis/MySQL/Kafka。
## 3. 分工原则
两套机制不是互斥关系,而是按数据特征分工:
```text
强状态、强顺序、小范围热路径 -> 原生 Cluster
大数据量、跨节点广播、缓存、补偿 -> Redis / MySQL / Kafka
```
### 3.1 优先使用原生 Cluster 的场景
以下场景应尽量使用原生 Cluster,发挥 Topic master 的内存态和串行状态机能力:
- 单个 Topic 内的订阅、离开、权限变更和在线 session 状态。
- 单聊/群聊 Topic 内的 `pub`、`read`、`recv`、`get/set meta`。
- 通话实时状态机:`ringing`、`accept`、`hang-up`、`offer/answer/ice-candidate`。
- 对顺序敏感的 Topic 内临时事件,例如正在输入、同 Topic 内 presence 派发。
- 需要立即反馈客户端、且依赖 Topic 当前内存状态的操作。
这些能力的共同特点是:
- 数据范围集中在单个 Topic。
- 需要强顺序或强一致。
- 更适合由唯一 Topic master 串行处理。
- 不适合广播给所有 Pod 后各自判断。
### 3.2 优先使用 Redis / MySQL / Kafka 的场景
以下场景应优先使用 Redis、MySQL 或 Kafka,不强行塞进原生 Cluster:
- 全局用户在线状态查询、好友列表在线状态展示。
- 群成员在线节点集合、大群在线成员筛选。
- 大群 fanout、热点消息缓存、未读聚合、离线补偿。
- 用户级 session control:踢端、全端下线、同平台挤旧端、密码重置清退。
- 限流、短状态、幂等、分布式去重。
- 历史消息、通话记录、审计、后台统计。
- Push、文件安全、后台管理命令等异步任务。
这些能力的共同特点是:
- 数据量大或跨 Topic。
- 需要被 HTTP、Kafka、后台任务等多入口共享。
- 可以最终一致,或可以通过历史补拉/条件更新补偿。
- 不适合放在某个 Topic master 的内存里长期维护。
## 4. 目标启用模式
目标模式为:
```text
原生 Cluster Topic master + Redis 大数据量支撑 + MySQL/Kafka 权威状态
```
目标分工:
- 原生 Cluster 负责单个 Topic 内的实时请求路由、内存状态机、强顺序处理和低延迟下发。
- Redis 负责全局在线状态、大群在线成员集合、热点缓存、大数据量 fanout、限流、幂等和 session control。
- MySQL 负责最终历史、通话记录、关系状态和可补偿查询。
- Kafka 负责后台异步任务、Push、审计、文件安全和跨服务事件。
启用原生 Cluster 后的核心原则:
```text
Topic 内强状态、强顺序、小范围热路径,必须优先走原生 Cluster
跨 Topic、大数据量、可最终一致或需补偿的数据,继续走 Redis/MySQL/Kafka
```
当前 standalone + Redis Fanout 是过渡状态。切换到原生 Cluster 前,需要先完成部署形态、职责收敛、重复链路关闭和压测验收。
## 5. 职责边界表
| 能力 | 当前 standalone + Redis Fanout 过渡模式 | 目标原生 Cluster 模式 | 说明 |
| --- | --- | --- | --- |
| Topic master 路由 | 不启用 | Cluster 负责 | 原生 Cluster 通过一致性哈希确定 Topic master。 |
| `{sub}` 订阅 | 本 Pod Hub/Topic 处理,跨 Pod 投递依赖 Redis | Cluster `ProxyReqJoin` | 启 Cluster 后不要再用 Redis 模拟订阅路由。 |
| `{leave}` 离开 | 本 Pod 处理 | Cluster `ProxyReqLeave` | 与订阅同源。 |
| `{pub}` 消息写入 | 本 Pod 处理,生成 data 后 Redis fanout | Cluster `ProxyReqBroadcast` 到 Topic master | 启 Cluster 后 Topic data 不应再额外 Redis topic fanout。 |
| `{get/set}` meta | 本 Pod 处理 | Cluster `ProxyReqMeta` | 强依赖 Topic 状态时应由 master 处理。 |
| `read/recv` 回执 | Redis topic/user fanout 下发结果 | Cluster 路由 note 到 master | 启 Cluster 后回执状态由 master 生成,避免双份 info。 |
| 通话 `ringing/accept/hang-up` | Redis `call_event` fanout + DB 条件更新兜底 | Cluster `ProxyReqCall` | 启 Cluster 后 Redis `call_event` 必须跳过。 |
| `offer/answer/ice-candidate` | 当前可由现有通话状态机处理;声网模式下不走 SDP/ICE | Cluster 由 Topic master 转发 | 首版声网主要关注 accept/hang-up。 |
| Topic 内 data/info/pres 下发 | Redis realtime fanout | Cluster proxy response | 两者不能同时作为主链路。 |
| 用户在线状态查询 | Redis presence | 可继续 Redis presence | 即使启 Cluster,也建议 Redis 保持全局在线查询缓存。 |
| 群在线成员缓存 | Redis topic member presence | 可继续 Redis 缓存 | 用于大群本节点在线成员查找和展示。 |
| 踢端/全端下线 | Redis session control | Redis session control | 用户级控制不属于单个 Topic,建议继续 Redis。 |
| 密码重置清退 | Redis session control | Redis session control | 与 Cluster 不冲突。 |
| 同平台新登录挤旧端 | 本 Pod + Redis platform evict | 本 Pod + Redis platform evict | 继续用 Redis 保证跨 Pod 生效。 |
| 好友/群业务事件 | 本地投递 + Redis user/topic fanout | 可继续 Redis user fanout | 业务事件通常不是 Topic 状态机请求。 |
| 系统消息通知 | 本地投递 + Redis fanout | 视消息写入入口决定 | 如果写入会话 Topic,应避免和 Cluster topic 下发重复。 |
| 群消息 seq/热消息缓存 | Redis / MySQL | 可继续 Redis / MySQL | 大数据量缓存和历史权威不依赖原生 Cluster。 |
| Kafka 后台事件 | Kafka 负责异步持久化、Push、审计 | Kafka 继续负责 | Kafka 不做实时 WS fanout 主链路。 |
## 6. Redis Fanout 主链路范围
在当前 standalone 过渡模式下,Redis Fanout 是跨 Pod 实时投递主链路。启用原生 Cluster 后,Redis Fanout 需要收敛为配套能力,负责:
1. `message/user`:用户维度的系统事件、业务事件、me presence 下发。
2. `session_control/user|device|platform`:踢端、清退、同平台挤旧端。
3. presence 缓存:用户在线状态、节点 session 信息、群成员在线节点信息。
4. 大群在线成员筛选、热点消息缓存、未读聚合和历史补偿辅助。
5. 限流、短状态、幂等、分布式去重。
仅在 standalone 过渡模式下,Redis Fanout 额外负责:
1. `message/topic`:Topic 维度的 `data`、`info`、`pres` 跨 Pod下发。
2. `call_event/topic`:通话 `ringing`、`accept`、`hang-up` 进入 Topic 状态机。
这些 Topic 状态机相关 fanout 在原生 Cluster 启用后必须关闭或跳过。
Redis Fanout 不负责:
- 作为消息历史权威。
- 作为通话最终状态权威。
- 保证同一 Topic 的全局严格顺序。
- 替代 MySQL/Kafka 的审计、补偿和持久化。
## 7. 原生 Cluster 主链路范围
启用原生 Cluster 后,它应负责:
1. Topic owner/master 分配。
2. Topic 请求路由:`sub`、`leave`、`pub`、`get/set`、`note`。
3. Topic 内状态机:订阅、权限、回执、通话、presence 派发。
4. Topic master 到 proxy session 的响应和广播。
5. 节点故障后的 proxy session 清理与订阅失效通知。
原生 Cluster 不建议负责:
- 用户级 session control。
- 全局在线状态查询缓存。
- 业务后台事件可靠投递。
- Kafka 异步消费、Push、审计。
- Redis 热消息缓存、限流、短状态。
- 大群全量成员 fanout、全局未读聚合、历史数据扫描等大数据量任务。
## 8. 双开时必须避免的重复链路
如果 `globals.cluster != nil`,以下 Redis Fanout 应跳过或降级:
| Redis 入口 | Cluster 启用时处理 |
| --- | --- |
| `publishRealtimeTopicMessage` | 跳过。Topic data/info/pres 由 Cluster proxy response 下发。 |
| `publishRealtimeCallEvent` | 跳过。通话 note 由 `ProxyReqCall` 路由到 Topic master。 |
| 群消息在线投递 fanout | 需要专项确认;如果同一 data 已由 Cluster 下发,不得再广播。 |
| 会话 Topic 系统消息 fanout | 如果消息进入 Topic 状态机,应避免再次 topic fanout。 |
以下 Redis Fanout 可继续保留:
| Redis 入口 | 保留原因 |
| --- | --- |
| `publishRealtimeEvictUser` | 用户级清退,不归属单一 Topic。 |
| `publishRealtimeEvictDevice` | 设备级清退,不归属单一 Topic。 |
| `publishRealtimeEvictUserPlatform` | 同平台挤旧端,需要跨 Pod 控制 session。 |
| 用户 presence 缓存 | 全局在线查询和好友列表展示仍需要。 |
| 业务 user event | 多数业务事件不是 Topic 状态机请求。 |
## 9. 状态权威与顺序规则
### 9.1 消息顺序
- 权威顺序来自 `topic + seq`。
- Redis Fanout 到达乱序时,客户端必须按 seq 去重和重排。
- 服务端历史读取按 seq 返回,作为重连补偿来源。
### 9.2 通话状态
通话状态必须使用条件更新保证单向流转:
```text
started -> accepted -> finished
started -> missed
started -> declined
accepted -> disconnected
```
规则:
- `accept` 只能从 `started` 写入 `accepted`。
- `missed` 只能从 `started` 且 `accepted_at IS NULL` 写入。
- `finished` 只能结束未结束通话。
- 已 `accepted` 的通话不得被 `missed` 或 `declined` 覆盖。
- WebSocket 下发失败不能改变最终状态,客户端可通过通话记录或会话消息补偿。
### 9.3 Session Control
session control 以 Redis 短状态和 token/auth_version 为兜底:
- Redis 实时踢端用于立即关闭在线 WS。
- token/auth_version 用于防止旧 token 重新登录。
- stale session-control fanout 必须丢弃,避免旧事件踢掉新 session。
## 10. 启用原生 Cluster 的前置条件
启用原生 Cluster 前必须满足:
1. K8s 部署从普通 Deployment 改为适合稳定网络身份的形态,例如 StatefulSet 或稳定 Pod DNS。
2. 每个 Pod 配置唯一且稳定的 `cluster_self`。
3. 配置完整 `cluster_config.nodes`,包括节点名和可达地址。
4. 开放并验证 gRPC/cluster 节点间通信。
5. 明确 Redis Fanout 哪些保留、哪些关闭。
6. 压测 `sub/pub/read/recv/call/group` 跨节点链路。
7. 验证节点重启、扩缩容、Topic rehash、proxy subscription 失效和客户端重订阅。
8. 验证不会重复投递 data/info/pres/call event。
## 11. 观测与验收
必须保留以下观测指标:
- Redis fanout publish/receive/deliver 计数。
- Redis fanout `skip_native_cluster` 计数。
- Redis fanout `no_local_topic`、`no_local_session`、`hub_full` 计数。
- 通话状态转换计数:started、accepted、finished、missed、declined。
- DB 条件更新失败计数,用于识别迟到事件和乱序事件。
- Cluster 模式下 proxy queue、TopicMaster RPC 失败、cluster unreachable、rehash 次数。
验收用例至少覆盖:
1. A、B 连接到不同 Pod,P2P 消息实时到达。
2. 接收方未订阅当前会话,仅在线 me,能收到未读提醒或补偿。
3. 通话 accept/hang-up 跨 Pod 到达,且不会被 missed 覆盖。
4. 多端登录同一账号,所有端收到业务事件。
5. 密码重置或后台踢端能清退所有 Pod 上的 session。
6. 大群在线成员分布在多 Pod,消息只投递在线成员,离线靠 seq 补拉。
7. Redis 短暂不可用时,关键状态不写坏,客户端可补偿。
## 12. 当前结论
当前阶段职责划分如下:
- 当前测试环境仍是 standalone + Redis Fanout,这是过渡状态。
- 目标是启用原生 Cluster,让 Topic master 承担内存高性能、强一致、强顺序的实时状态机。
- Redis 不退出架构,而是转为大数据量、缓存、补偿、全局 presence、session control 和异步支撑能力。
- 启用原生 Cluster 时,Topic 状态机相关的 Redis topic/call fanout 必须关闭或降级,避免 data/info/pres/call event 双路投递。
+430
View File
@@ -0,0 +1,430 @@
# IM 原生 Cluster 运维部署说明
## 1. 目标
本文给运维说明 IM Server 如何部署才能真正启用 Tinode 原生 Cluster。
目标效果:
- 多个 IM Pod 组成同一个 Tinode Cluster,不再是 standalone。
- 每个 Pod 拥有稳定、唯一的节点名。
- 节点之间通过 `cluster_config.nodes.addr` 建立内部 TCP RPC 通信。
- Topic 请求可以路由到唯一 Topic master,由 master 处理 Topic 内状态机、顺序和实时下发。
如果部署后日志仍出现:
```text
Cluster: running as a standalone server.
```
说明原生 Cluster 没有启用,跨 Pod Topic 状态机不会生效。
## 2. 原生 Cluster 启用条件
必须同时满足以下条件:
1. `cluster_self` 非空。
2. `cluster_self` 的值必须等于 `cluster_config.nodes` 中某个节点的 `name`。
3. `cluster_config.nodes` 必须包含所有 IM 节点,包括当前节点。
4. 每个节点的 `addr` 必须是其他 Pod 可访问的内部地址。
5. 每个节点必须监听自己的 `addr` 端口。
6. 节点之间网络策略、安全组、Service 必须放通 cluster 内部端口。
7. 至少 2 个节点才算有效 Cluster,生产建议 3 个节点起。
当前代码启动参数由 `docker/tinode/entrypoint.sh` 传入:
```bash
./tinode --config=${CONFIG} --static_data=$STATIC_DIR --cluster_self=$CLUSTER_SELF --pprof_url=$PPROF_URL
```
所以运维侧必须确保每个 Pod 的环境变量 `CLUSTER_SELF` 有值,并且值稳定。
## 3. 推荐部署形态
推荐使用 Kubernetes `StatefulSet`,不要用普通 `Deployment` 直接滚动多个副本。
原因:
- 原生 Cluster 依赖稳定节点名,例如 `im-server-0`、`im-server-1`、`im-server-2`。
- 原生 Cluster 依赖稳定 DNS,例如 `im-server-0.im-server-cluster.default.svc.cluster.local`。
- 普通 Deployment 的 Pod 名会变化,不适合作为 `cluster_self`。
- StatefulSet 配合 Headless Service 可以给每个 Pod 提供固定网络身份。
推荐结构:
```text
Headless Service: im-server-cluster
StatefulSet: im-server
Pod DNS:
im-server-0.im-server-cluster.<namespace>.svc.cluster.local
im-server-1.im-server-cluster.<namespace>.svc.cluster.local
im-server-2.im-server-cluster.<namespace>.svc.cluster.local
```
## 4. 端口规划
至少需要区分三类端口:
| 端口 | 用途 | 是否对公网暴露 | 说明 |
| --- | --- | --- | --- |
| `6060` | HTTP/WebSocket/API | 通过 Ingress/SLB 暴露 | 客户端连接入口。 |
| `16060` | Tinode gRPC 客户端端口 | 通常不对公网暴露 | 管理或内部 gRPC 客户端使用。 |
| `12000` | 原生 Cluster 内部 RPC | 不对公网暴露,只在 Pod 间放通 | `cluster_config.nodes.addr` 使用。 |
注意:`cluster_config.nodes.addr` 使用的是原生 Cluster 内部 TCP RPC 地址,不是 `grpc_listen` 的 `16060`。
建议统一所有 Pod 的 cluster 监听端口为 `12000`,通过 Pod DNS 区分节点:
```json
{
"name": "im-server-0",
"addr": "im-server-0.im-server-cluster.default.svc.cluster.local:12000"
}
```
不要沿用 docker compose 示例里的 `tinode-0:12000`、`tinode-1:12001`、`tinode-2:12002`,除非是在同一主机或 docker compose 网络里部署。
## 5. 配置文件要求
`server/tinode.conf` 默认配置是:
```json
"cluster_config": {
"self": "",
"nodes": [],
"failover": {
"enabled": false
}
}
```
这个配置不会启用 Cluster。
生产或测试环境需要提供完整配置。建议 3 节点配置如下:
```json
"cluster_config": {
"self": "",
"nodes": [
{
"name": "im-server-0",
"addr": "im-server-0.im-server-cluster.default.svc.cluster.local:12000"
},
{
"name": "im-server-1",
"addr": "im-server-1.im-server-cluster.default.svc.cluster.local:12000"
},
{
"name": "im-server-2",
"addr": "im-server-2.im-server-cluster.default.svc.cluster.local:12000"
}
],
"failover": {
"enabled": true,
"heartbeat": 100,
"vote_after": 8,
"node_fail_after": 16
}
}
```
说明:
- `self` 可以保持为空,由启动参数 `--cluster_self=$CLUSTER_SELF` 覆盖。
- `CLUSTER_SELF` 必须分别为 `im-server-0`、`im-server-1`、`im-server-2`。
- `nodes` 必须所有 Pod 完全一致。
- `addr` 必须能被其他 Pod 访问。
- 3 节点建议开启 failover;2 节点不建议开启 failover。
## 6. Kubernetes 示例
以下示例只展示 Cluster 相关关键配置,实际部署时需要合并现有镜像、环境变量、Secret、资源限制、探针、Ingress 和业务配置。
### 6.1 Headless Service
```yaml
apiVersion: v1
kind: Service
metadata:
name: im-server-cluster
namespace: default
spec:
clusterIP: None
selector:
app: im-server
ports:
- name: cluster
port: 12000
targetPort: 12000
- name: http
port: 6060
targetPort: 6060
- name: grpc
port: 16060
targetPort: 16060
```
### 6.2 客户端入口 Service
客户端入口可以继续使用普通 Service 或 Ingress,负载到所有 Pod:
```yaml
apiVersion: v1
kind: Service
metadata:
name: im-server
namespace: default
spec:
selector:
app: im-server
ports:
- name: http
port: 6060
targetPort: 6060
```
### 6.3 StatefulSet
```yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: im-server
namespace: default
spec:
serviceName: im-server-cluster
replicas: 3
selector:
matchLabels:
app: im-server
template:
metadata:
labels:
app: im-server
spec:
containers:
- name: im-server
image: <im-server-image>
ports:
- name: http
containerPort: 6060
- name: grpc
containerPort: 16060
- name: cluster
containerPort: 12000
env:
- name: CLUSTER_SELF
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: EXT_CONFIG
value: /etc/tinode/tinode.conf
volumeMounts:
- name: tinode-config
mountPath: /etc/tinode
readOnly: true
volumes:
- name: tinode-config
configMap:
name: im-server-tinode-conf
```
重点:
- `serviceName` 必须指向 Headless Service。
- `CLUSTER_SELF` 直接取 Pod 名,StatefulSet 下会稳定为 `im-server-0`、`im-server-1`、`im-server-2`。
- ConfigMap 中的 `cluster_config.nodes.name` 必须和 Pod 名一致。
- ConfigMap 中的 `cluster_config.nodes.addr` 建议使用 StatefulSet Pod DNS。
## 7. NetworkPolicy / 安全组要求
如果集群启用了 NetworkPolicy,需要允许 IM Pod 之间访问 cluster 端口:
```yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: im-server-cluster-allow
namespace: default
spec:
podSelector:
matchLabels:
app: im-server
policyTypes:
- Ingress
- Egress
ingress:
- from:
- podSelector:
matchLabels:
app: im-server
ports:
- protocol: TCP
port: 12000
egress:
- to:
- podSelector:
matchLabels:
app: im-server
ports:
- protocol: TCP
port: 12000
```
如果没有 NetworkPolicy,可以跳过此项,但仍要确认云厂商安全组、服务网格策略没有拦截 Pod 间 TCP `12000`。
## 8. 启动顺序
建议:
1. 先部署 ConfigMap、Headless Service、普通入口 Service。
2. 再部署 StatefulSet。
3. 第一个 Pod 可以执行数据库初始化和升级。
4. 其他 Pod 设置 `NO_DB_INIT=true`,避免多 Pod 同时初始化数据库。
5. 等所有 Pod Ready 后再接入客户端流量。
如果当前镜像沿用 `docker/tinode/entrypoint.sh` 的逻辑:
- 第一个 Pod 可以保留 `NO_DB_INIT=false`。
- 其他 Pod 建议设置 `NO_DB_INIT=true`。
- 如果所有 Pod 同时启动,要确认数据库初始化脚本具备幂等能力,否则建议改成 init Job 单独执行。
## 9. 发布步骤建议
### 9.1 灰度前检查
执行:
```bash
kubectl -n default get pod -l app=im-server -o wide
kubectl -n default get svc im-server-cluster
kubectl -n default exec im-server-0 -- sh -c 'echo $CLUSTER_SELF'
kubectl -n default exec im-server-1 -- sh -c 'echo $CLUSTER_SELF'
kubectl -n default exec im-server-2 -- sh -c 'echo $CLUSTER_SELF'
```
期望:
```text
im-server-0
im-server-1
im-server-2
```
验证节点 DNS:
```bash
kubectl -n default exec im-server-0 -- sh -c 'nc -vz im-server-1.im-server-cluster.default.svc.cluster.local 12000'
kubectl -n default exec im-server-0 -- sh -c 'nc -vz im-server-2.im-server-cluster.default.svc.cluster.local 12000'
```
### 9.2 启动后日志检查
查看日志:
```bash
kubectl -n default logs im-server-0 | grep -i cluster
kubectl -n default logs im-server-1 | grep -i cluster
kubectl -n default logs im-server-2 | grep -i cluster
```
不能出现:
```text
Cluster: running as a standalone server.
```
如果出现该日志,优先检查:
- `CLUSTER_SELF` 是否为空。
- `CLUSTER_SELF` 是否不在 `cluster_config.nodes.name` 中。
- 配置文件是否确实被容器读取。
- `EXT_CONFIG` 是否指向了正确文件。
- `cluster_config.nodes` 是否为空。
如果出现节点连接失败,优先检查:
- `cluster_config.nodes.addr` DNS 是否能解析。
- 端口 `12000` 是否监听。
- Pod 间 NetworkPolicy 是否放通。
- Service `targetPort` 是否正确。
### 9.3 功能验收
至少验证:
1. A、B 两个账号分别连到不同 Pod,P2P 消息实时互达。
2. A、B 分别连到不同 Pod,语音通话 `ringing`、`accept`、`hang-up` 状态实时互达。
3. 同一个账号多端连接不同 Pod,`recv/read` 回执正常。
4. 群聊成员分布在不同 Pod,群消息实时下发正常。
5. 重启任意一个 Pod 后,客户端能重连,Topic 可重新订阅。
6. 日志无持续 cluster unreachable、proxy timeout、dial timeout。
## 10. 与 Redis Fanout 的关系
启用原生 Cluster 后,Redis 仍然保留,但职责要收敛:
- 保留:全局 presence、session control、踢端、同平台挤旧端、大群在线缓存、热点缓存、限流、幂等、补偿。
- 收敛或关闭:Topic data/info/pres 的 Redis topic fanout。
- 收敛或关闭:通话 `call_event/topic` 的 Redis fanout。
否则会出现同一个 Topic 事件既由 Cluster 下发、又由 Redis Fanout 下发,导致重复 data/info/pres/call event。
对应职责边界参考:
- `docs/IM-原生Cluster与RedisFanout职责边界.md`
## 11. 常见问题
### 11.1 日志仍然是 standalone
原因通常是 `cluster_self` 为空或配置文件里的 `nodes` 为空。
处理:
1. 确认 Pod 环境变量 `CLUSTER_SELF`。
2. 确认启动参数包含 `--cluster_self=$CLUSTER_SELF`。
3. 确认容器读取的是带完整 `cluster_config.nodes` 的配置。
### 11.2 Pod 能启动,但跨 Pod 消息还是不通
原因通常是 cluster 内部端口不通,或节点地址写错。
处理:
1. 从任意 Pod `nc -vz <peer-pod-dns> 12000`。
2. 检查 Headless Service selector 是否匹配 Pod label。
3. 检查 NetworkPolicy 是否放通 egress/ingress。
4. 检查 `cluster_config.nodes.addr` 是否写成了只在 docker compose 里可用的短域名。
### 11.3 扩缩容后部分 Topic 异常
原生 Cluster 的 Topic master 基于节点集合计算。扩缩容会改变节点集合,可能触发 Topic 重新路由。
建议:
- 生产先固定 3 节点。
- 不要频繁自动扩缩容 IM Server。
- 扩缩容必须配合完整压测和重连验证。
- 如果要从 3 节点扩到 5 节点,先更新所有 Pod 的 `cluster_config.nodes`,再按发布计划滚动。
### 11.4 2 节点能不能用
2 节点可以启用基础 Cluster,但不建议开启 failover,也不建议生产使用。
生产推荐 3 节点起,便于 leader 心跳和故障判断。
## 12. 运维上线检查清单
- [ ] 使用 StatefulSet,不使用普通 Deployment 承载 Cluster 节点身份。
- [ ] 创建 Headless Service。
- [ ] 每个 Pod 的 `CLUSTER_SELF` 等于 Pod 名。
- [ ] `cluster_config.nodes` 包含所有 Pod。
- [ ] 所有 Pod 使用同一份 `cluster_config.nodes`。
- [ ] 每个 `nodes.addr` 可从其他 Pod 访问。
- [ ] 放通 cluster 内部 TCP 端口 `12000`。
- [ ] 普通 HTTP/WebSocket 入口仍通过 Service/Ingress 负载到所有 Pod。
- [ ] 只有一个入口负责数据库初始化,或数据库初始化已改为独立 Job。
- [ ] 启动日志不再出现 `Cluster: running as a standalone server.`。
- [ ] 跨 Pod P2P、群聊、read/recv、通话状态已验收。
- [ ] 原生 Cluster 启用后,Topic 相关 Redis Fanout 已按职责边界关闭或降级。
+10 -8
View File
@@ -121,19 +121,21 @@
| --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- |
| 创建群 | 群成员 | `info.what="group"`, `event="group.created"` | `topic`、群基础信息 | 除操作人外写入 | | 创建群 | 群成员 | `info.what="group"`, `event="group.created"` | `topic`、群基础信息 | 除操作人外写入 |
| 更新群设置 | 群成员 | `event="group.settings.updated"` | `topic` | 除操作人外写入 | | 更新群设置 | 群成员 | `event="group.settings.updated"` | `topic` | 除操作人外写入 |
| 成员资料/权限更新 | 群成员 | `event="group.member.updated"` | `user_id` | 除操作人外写入 | | 成员资料/权限更新 | 群主、管理员、本人 | `event="group.member.updated"` | `user_id`、`member_sync=single` | 不写群系统消息 |
| 成员加入 | 群成员 | `event="group.member.joined"` | `user_id`、`topic` | 除操作人外写入 | | 成员加入 | 群成员 | `event="group.member.joined"` | `user_id`、`topic`、`member_sync=single` | 写入群消息并参与个人未读与离线 Push |
| 入群申请创建 | 群成员/管理员可见链路 | `event="group.join_request.created"` | `request_id`、`applicant_user_id`、`topic` | 除操作人外写入 | | 入群申请创建 | 群主、管理员、申请人 | `event="group.join_request.created"` | `request_id`、`applicant_user_id`、`topic` | 普通成员不可见 |
| 入群申请处理 | 群成员 | `event="group.join_request.accepted/rejected"` | `request_id`、`status`、`topic` | 除操作人外写入 | | 入群申请处理 | 群主、管理员、申请人 | `event="group.join_request.accepted/rejected"` | `request_id`、`status`、`topic` | 普通成员不可见;接受时另发全群加入消息 |
| 群公告更新 | 群成员 | `event="group.announcement.updated"` | `topic`、公告信息 | 除操作人外写入 | | 群公告更新 | 群成员 | `event="group.announcement.updated"` | `topic`、公告信息 | 除操作人外写入 |
| 群禁言更新 | 群成员 | `event="group.mute.updated"` | `topic`、禁言信息 | 除操作人外写入 | | 群禁言更新 | 群主、管理员、被操作人 | `event="group.mute.updated"` | `topic`、禁言信息、`member_sync` | 普通成员不可见 |
| 成员离群 | 群成员 | `event="group.member.left"` | `user_id` | 除操作人外写入 | | 成员离群/移除 | 群主、管理员、离群成员 | `event="group.member.left"` | `user_id`、`member_sync=remove` | 普通成员不可见 |
| 群角色更新 | 群成员 | `event="group.role.updated"` | `user_id`、角色信息 | 除操作人外写入 | | 群角色更新 | 群主、管理员、被操作人 | `event="group.role.updated"` | `user_id`、角色信息、`member_sync` | 普通成员不可见 |
| 群主转让 | 群成员 | `event="group.owner.transferred"` | `previous_owner`、`new_owner` | 除操作人外写入 | | 群主转让 | 群成员 | `event="group.owner.transferred"` | `previous_owner`、`new_owner` | 除操作人外写入 |
前端建议动作: 前端建议动作:
- 群成员、角色、禁言、群主变化:刷新群详情、成员列表、当前用户群权限。 - 首次进入、断线重连、本地无快照或收到 `member_sync=full`:分页拉取完整群成员。
- 收到 `member_sync=single`:调用 `GET /v1/im/groups/{topic}/members/{user_id}` 只刷新一个成员。
- 收到 `member_sync=remove`:从本地成员快照删除 `user_id`;若本地状态无法确认,再全量分页同步。
- 入群申请变化:管理员刷新入群申请列表和群 summary 待处理数。 - 入群申请变化:管理员刷新入群申请列表和群 summary 待处理数。
- 群公告变化:刷新公告区。 - 群公告变化:刷新公告区。
- 成员加入/离开:刷新群成员数、成员列表和会话系统提示。 - 成员加入/离开:刷新群成员数、成员列表和会话系统提示。
+122
View File
@@ -0,0 +1,122 @@
# IM「联系我们」移动端接口对接文档
> 版本:v1.0
> 日期:2026-09-21
> 状态:**已实现**,可按本文直接联调
> 适用端:iOS / Android / Web(im_webapp)
## 1. 功能说明
移动端「设置 / 关于 / 支持」页面的**联系我们**入口,展示管理员配置的客服邮箱。
数据链路:
```text
管理后台「系统管理 → 字典管理 → 联系我们」
→ IM 管理端数据库(字典类型 im_contact_us,条目即 label/value)
→ im-server 缓存(内存 TTL 1 分钟兜底 + 管理端变更事件主动失效)
→ 移动端 GET /v1/im/contact-us
```
要点:
- 条目内容由运营在管理后台维护,客户端不需要硬编码任何联系方式。
- 管理后台修改后**近实时生效**(事件失效,秒级);事件通道不可用时最长 1 分钟内收敛(缓存 TTL 兜底)。
- 条目数量不固定,客户端必须按列表渲染,不能假设只有一个条目。
- 接口返回为空列表时,客户端应**隐藏联系我们入口**,不要渲染空区块。
## 2. 接口定义
### 2.1 获取联系我们配置
```text
GET {api_path}v1/im/contact-us
```
**请求头**(公共约定同《IM-前后端对接文档.md》§0.2):
| Header | 必填 | 说明 |
| --- | --- | --- |
| `X-MeetingIM-APIKey` | 是 | 应用级接入标识 |
| `X-IM-Schema` | 是 | 固定 `1` |
| `X-Request-ID` | 是 | 请求追踪 ID |
| `Authorization` | 是 | `Bearer <token>`,登录态必填 |
**请求参数**:无
**成功响应**(HTTP 200):
```json
{
"code": 0,
"data": {
"items": [
{ "label": "客服邮箱", "value": "support@example.com" }
]
},
"request_id": "req-contactus-2",
"server_time": "2026-09-21T04:58:19.521Z"
}
```
`data` 字段:
| 字段 | 类型 | 说明 |
| --- | --- | --- |
| `items` | `array` | 联系方式条目列表,按管理后台配置的排序返回;可能为空数组 `[]` |
| `items[].label` | `string` | 条目名称,如「客服邮箱」,客户端原样展示 |
| `items[].value` | `string` | 条目值,邮箱地址 |
**失败响应**:
| HTTP | code | reason | 场景 |
| --- | --- | --- | --- |
| 403 | 30003 | `API_KEY_REQUIRED` | API Key 缺失或无效 |
| 401 | 30043 | `AUTH_REQUIRED` | 未登录 / token 过期,引导重新登录 |
| 405 | 30002 | `METHOD_NOT_ALLOWED` | 非 GET 请求 |
| 500 | 90000 | `INTERNAL_ERROR` | 服务端内部错误,可重试 |
## 3. 客户端渲染规则
按条目值的形态渲染(客户端本地判断,与服务端无关):
| value 形态 | 判断 | 交互 |
| --- | --- | --- |
| 邮箱 | 包含 `@` 且形如 `x@y.z` | `mailto:{value}`,图标用 `email` |
| 其他 | 不满足邮箱格式 | 纯文本展示,不可点击 |
补充约定:
- 每个条目的展示文案使用 `label`,不要自行翻译或改写。
- 接口失败(网络/500)时静默降级:若客户端保留了旧版本静态联系方式可继续展示,否则隐藏入口,不弹 toast。
- 不需要本地缓存该接口结果;进入页面时请求即可,服务端已有缓存兜底。
## 4. 本地联调示例
```bash
curl -s "http://{host}/{api_path}v1/im/contact-us" \
-H "X-MeetingIM-APIKey: <api_key>" \
-H "X-IM-Schema: 1" \
-H "X-Request-ID: req-contactus-2" \
-H "Authorization: Bearer <token>"
```
实测返回:
```json
{"code":0,"data":{"items":[{"label":"客服邮箱","value":"support@example.com"}]},"request_id":"req-contactus-2","server_time":"2026-09-21T04:58:19.521Z"}
```
## 5. 配置维护(运营侧)
- 入口:管理后台「系统管理 → 字典管理」→ 字典类型 `联系我们`(type=`im_contact_us`)。
- 新增/修改/删除条目即配置生效,`状态=停用` 的条目不会返回。
- 测试环境初始化 SQL:`im_manager/docs/im/IM-联系我们字典配置.sql`(im_manager 仓库)。
- 首次部署后如接口返回空列表,先确认管理后台该字典类型下存在 `状态=启用` 的条目。
## 6. 参考实现
Web 端(im_webapp)已完成对接,可直接参考:
- API 封装:`src/lib/im-api.js` 中 `getContactUs()`
- 页面渲染:`src/views/acc-support-view.jsx`(邮箱 `mailto:`、空列表回退静态链接)
@@ -23,7 +23,7 @@ Idempotency-Key: <uuid>
测试环境验证码固定为 `123456`。`action_token` 为 5 分钟有效、单次消费,只能用于指定动作,不能在多个接口之间重复使用。 测试环境验证码固定为 `123456`。`action_token` 为 5 分钟有效、单次消费,只能用于指定动作,不能在多个接口之间重复使用。
当前默认策略项为 `night_quiet`、`content_filter`、`payment_guard` 三条。现阶段后端已完成状态持久化、开启/关闭校验和策略返回;业务接口层的深夜消息免打扰、内容过滤、支付拦截尚未实际强制执行,响应中的 `restriction_items[].enforced=false` 用于明确表达这一点。后续某条规则真正接入后台策略和业务拦截后,应把对应项改为 `true`。 当前默认策略项为 `night_quiet`、`content_filter`、`payment_guard` 三条。现阶段后端已完成状态持久化、开启/关闭校验和策略返回;`night_quiet` 已接入音视频呼叫发起限制,`content_filter` 已接入消息发送前内容安全审核,响应中的这两项 `restriction_items[].enforced=true`。`payment_guard` 当前仍为预留项,`enforced=false`。
## 2. 开启青少年模式 ## 2. 开启青少年模式
@@ -47,13 +47,13 @@ GET /v1/im/settings/youth-mode
"code": "night_quiet", "code": "night_quiet",
"title": "深夜免打扰", "title": "深夜免打扰",
"description": "夜间减少消息提醒打扰,帮助用户保持健康作息。", "description": "夜间减少消息提醒打扰,帮助用户保持健康作息。",
"enforced": false "enforced": true
}, },
{ {
"code": "content_filter", "code": "content_filter",
"title": "内容过滤", "title": "内容过滤",
"description": "对不适合青少年的内容进行识别和限制展示。", "description": "对不适合青少年的内容进行识别和限制展示。",
"enforced": false "enforced": true
}, },
{ {
"code": "payment_guard", "code": "payment_guard",
@@ -202,13 +202,13 @@ POST /v1/im/settings/youth-mode/enable
"code": "night_quiet", "code": "night_quiet",
"title": "深夜免打扰", "title": "深夜免打扰",
"description": "夜间减少消息提醒打扰,帮助用户保持健康作息。", "description": "夜间减少消息提醒打扰,帮助用户保持健康作息。",
"enforced": false "enforced": true
}, },
{ {
"code": "content_filter", "code": "content_filter",
"title": "内容过滤", "title": "内容过滤",
"description": "对不适合青少年的内容进行识别和限制展示。", "description": "对不适合青少年的内容进行识别和限制展示。",
"enforced": false "enforced": true
}, },
{ {
"code": "payment_guard", "code": "payment_guard",
@@ -966,7 +966,7 @@ X-MeetingIM-APIKey: <预置的普通 API Key>
| `POST /v1/im/auth/password-reset/confirm` | `{"reset_token":"reset_token_123","new_password":"NewPass123"}` | `{"status":"ok"}` | | `POST /v1/im/auth/password-reset/confirm` | `{"reset_token":"reset_token_123","new_password":"NewPass123"}` | `{"status":"ok"}` |
| `POST /admin/v1/tenant/users/{user_id}/reset-password` 管理后台 | `{"new_password":"AdminReset123","reason":"用户无手机号无法自助找回"}` | `{"user_id":"usr...","status":"reset","auth_version":2}` | | `POST /admin/v1/tenant/users/{user_id}/reset-password` 管理后台 | `{"new_password":"AdminReset123","reason":"用户无手机号无法自助找回"}` | `{"user_id":"usr...","status":"reset","auth_version":2}` |
| `POST /v1/im/auth/logout` 待实现 | `{}` | `{"status":"ok"}` | | `POST /v1/im/auth/logout` 待实现 | `{}` | `{"status":"ok"}` |
| `GET /v1/im/app/releases/latest?platform=ios\|android&current_version=1.0.0&channel=stable` | Query only | `{"platform":"ios","channel":"stable","latest_version":"1.0.0","min_supported_version":"1.0.0","update_type":"none","package_url":"","store_url":"","release_notes":"当前已是最新版本。","remind_after_seconds":0}` | | `GET /v1/im/app/releases/latest?platform=ios\|android\|web&current_version=1.0.0&channel=stable` | Query only | `{"platform":"web","channel":"stable","latest_version":"1.0.0","min_supported_version":"1.0.0","update_type":"none","package_url":"","store_url":"","release_notes":"当前已是最新版本。","remind_after_seconds":0}` |
## 2. 消息首页、全局导航与系统通知 ## 2. 消息首页、全局导航与系统通知
@@ -2271,6 +2271,7 @@ GET /v1/im/topics/{topic}/message-mutations?since_version={localMutationVersion}
- `GET /v1/im/topics/{topic}/messages/search?q=&type=text&cursor=` 返回 `seq,from,ts,snippet,matches[]`。 - `GET /v1/im/topics/{topic}/messages/search?q=&type=text&cursor=` 返回 `seq,from,ts,snippet,matches[]`。
- 点击结果后 `GET /v1/im/topics/{topic}/messages/{seq}/context?before=20&after=20`,返回锚点及上下文;客户端定位 `seq` 并高亮。 - 点击结果后 `GET /v1/im/topics/{topic}/messages/{seq}/context?before=20&after=20`,返回锚点及上下文;客户端定位 `seq` 并高亮。
- 服务端只搜索当前用户可见且未硬删除的消息,关键词做长度/频率限制,搜索日志不得记录原文。 - 服务端只搜索当前用户可见且未硬删除的消息,关键词做长度/频率限制,搜索日志不得记录原文。
- 搜索匹配字段包括文本正文、`content.text/txt/content/title/summary/name`、文件名、链接标题/摘要、卡片/转发快照等可展示文本;排序按 `seq desc`,`next_cursor` 继续向更早消息翻页。
### 7.13 查找聊天记录-文件.png ### 7.13 查找聊天记录-文件.png
@@ -2924,7 +2925,7 @@ GET /v1/im/topics/{topic}/message-mutations?since_version={localMutationVersion}
**状态:新增 App 配置接口。** **状态:新增 App 配置接口。**
- `GET /v1/im/app/releases/latest?platform=ios|android&current_version=1.0.0&channel=stable`,未登录可在企业解析后调用。 - `GET /v1/im/app/releases/latest?platform=ios|android|web&current_version=1.0.0&channel=stable`,未登录可在企业解析后调用。
- 返回 `latest_version,min_supported_version,update_type=none|optional|required,release_notes,store_url,package_url,checksum,published_at`。 - 返回 `latest_version,min_supported_version,update_type=none|optional|required,release_notes,store_url,package_url,checksum,published_at`。
- 客户端按平台只打开受信任商店/签名下载地址;Android 包必须校验签名和 checksum,服务端 URL 使用允许域名。 - 客户端按平台只打开受信任商店/签名下载地址;Android 包必须校验签名和 checksum,服务端 URL 使用允许域名。
@@ -2963,7 +2964,7 @@ GET /v1/im/topics/{topic}/message-mutations?since_version={localMutationVersion}
**状态:新增。** **状态:新增。**
- `GET /v1/im/settings/youth-mode` 返回 `enabled,policy_version,restrictions,restriction_items,password_set,can_enable`。当前默认策略项为 `night_quiet`、`content_filter`、`payment_guard` 三条;后端现阶段完成状态持久化、开启/关闭校验和策略返回,业务接口层的深夜免打扰、内容过滤、支付守护尚未实际强制执行,`restriction_items[].enforced=false` 用于明确表达这一点。 - `GET /v1/im/settings/youth-mode` 返回 `enabled,policy_version,restrictions,restriction_items,password_set,can_enable`。当前默认策略项为 `night_quiet`、`content_filter`、`payment_guard` 三条;后端现阶段完成状态持久化、开启/关闭校验和策略返回;`night_quiet` 已接入音视频呼叫发起限制,`content_filter` 已接入消息发送前内容安全审核,二者返回 `enforced=true`;`payment_guard` 仍为预留项,返回 `enforced=false`。
- 点击开启先展示 10.15,再按 1.7 完成身份验证和 1.11 设置独立密码;最后调用 `POST /v1/im/settings/youth-mode/enable`,Body `{"policy_version":n,"action_token":"..."}`,其中 `action_token` 使用设置独立密码响应里的 `enable_action_token`。 - 点击开启先展示 10.15,再按 1.7 完成身份验证和 1.11 设置独立密码;最后调用 `POST /v1/im/settings/youth-mode/enable`,Body `{"policy_version":n,"action_token":"..."}`,其中 `action_token` 使用设置独立密码响应里的 `enable_action_token`。
- 开启链路中,短信校验返回的 `set_independent_password` token 只用于设置独立密码;设置密码成功响应中的 `enable_action_token` 才用于 `youth-mode/enable`。 - 开启链路中,短信校验返回的 `set_independent_password` token 只用于设置独立密码;设置密码成功响应中的 `enable_action_token` 才用于 `youth-mode/enable`。
- 服务端持久化到用户租户范围并推送所有设备,不能仅设置本地标志。 - 服务端持久化到用户租户范围并推送所有设备,不能仅设置本地标志。
@@ -2984,7 +2985,7 @@ GET /v1/im/topics/{topic}/message-mutations?since_version={localMutationVersion}
**状态:新增。** **状态:新增。**
- 页面调用 10.14,`enabled=true` 时展示 `enabled_at,restrictions`。`bootstrap` 同时返回受限模块和动作。 - 页面调用 10.14,`enabled=true` 时展示 `enabled_at,restrictions`。`bootstrap` 同时返回受限模块和动作。
- 后续某条规则真正接入业务拦截后,受限接口服务端返回 `YOUTH_MODE_RESTRICTED`,并把对应 `restriction_items[].enforced` 改为 `true`;前端隐藏只是体验优化,不能作为安全控制。 - 受限接口服务端返回 `YOUTH_MODE_RESTRICTED`,当前已覆盖夜间音视频呼叫发起;前端隐藏只是体验优化,不能作为安全控制。
- 登录、App 重启和另一设备均从服务端恢复状态;收到 `youth_mode.updated` 即刷新。 - 登录、App 重启和另一设备均从服务端恢复状态;收到 `youth_mode.updated` 即刷新。
### 10.17 关闭青少年模式(安卓).png ### 10.17 关闭青少年模式(安卓).png
@@ -3053,10 +3054,10 @@ GET /v1/im/topics/{topic}/message-mutations?since_version={localMutationVersion}
| `PATCH /v1/im/me/profile` | `{"nickname":"新昵称","signature":"新签名","version":1}` | `{"avatar_file_id":"","nickname":"新昵称","signature":"新签名","version":2,"updated_at":"2026-08-20T09:00:00Z"}` | | `PATCH /v1/im/me/profile` | `{"nickname":"新昵称","signature":"新签名","version":1}` | `{"avatar_file_id":"","nickname":"新昵称","signature":"新签名","version":2,"updated_at":"2026-08-20T09:00:00Z"}` |
| `GET /v1/im/me/privacy` | Query only | `{"add_me_methods":{"account":true,"phone":true,"qr":true,"group":true},"profile_visibility":"friends","last_seen_visibility":"friends","read_receipts":true,"friend_request_verification":true,"group_invite_verification":false,"version":1,"editable":{"add_me_methods":true,"profile_visibility":true,"last_seen_visibility":true,"read_receipts":true,"friend_request_verification":true,"group_invite_verification":true}}` | | `GET /v1/im/me/privacy` | Query only | `{"add_me_methods":{"account":true,"phone":true,"qr":true,"group":true},"profile_visibility":"friends","last_seen_visibility":"friends","read_receipts":true,"friend_request_verification":true,"group_invite_verification":false,"version":1,"editable":{"add_me_methods":true,"profile_visibility":true,"last_seen_visibility":true,"read_receipts":true,"friend_request_verification":true,"group_invite_verification":true}}` |
| `PATCH /v1/im/me/privacy` | `{"add_me_methods":{"phone":false},"read_receipts":false,"friend_request_verification":false,"group_invite_verification":true,"version":1}` | `{"add_me_methods":{"account":true,"phone":false,"qr":true,"group":true},"read_receipts":false,"friend_request_verification":false,"group_invite_verification":true,"version":2,"updated_at":"2026-08-20T09:00:00Z"}` | | `PATCH /v1/im/me/privacy` | `{"add_me_methods":{"phone":false},"read_receipts":false,"friend_request_verification":false,"group_invite_verification":true,"version":1}` | `{"add_me_methods":{"account":true,"phone":false,"qr":true,"group":true},"read_receipts":false,"friend_request_verification":false,"group_invite_verification":true,"version":2,"updated_at":"2026-08-20T09:00:00Z"}` |
| `GET /v1/im/settings/youth-mode` | Query only | `{"enabled":false,"policy_version":1,"restrictions":["night_quiet","content_filter","payment_guard"],"restriction_items":[{"code":"night_quiet","title":"深夜免打扰","description":"夜间减少消息提醒打扰,帮助用户保持健康作息。","enforced":false},{"code":"content_filter","title":"内容过滤","description":"对不适合青少年的内容进行识别和限制展示。","enforced":false},{"code":"payment_guard","title":"支付守护","description":"限制或确认支付、充值等敏感消费行为。","enforced":false}],"password_set":false,"can_enable":false,"updated_at":"2026-08-20T09:00:00Z"}` | | `GET /v1/im/settings/youth-mode` | Query only | `{"enabled":false,"policy_version":1,"restrictions":["night_quiet","content_filter","payment_guard"],"restriction_items":[{"code":"night_quiet","title":"深夜免打扰","description":"夜间减少消息提醒打扰,帮助用户保持健康作息。","enforced":true},{"code":"content_filter","title":"内容过滤","description":"对不适合青少年的内容进行识别和限制展示。","enforced":true},{"code":"payment_guard","title":"支付守护","description":"限制或确认支付、充值等敏感消费行为。","enforced":false}],"password_set":false,"can_enable":false,"updated_at":"2026-08-20T09:00:00Z"}` |
| `POST /v1/im/settings/independent-password` | `{"password":"1234","purpose":"youth_mode","action_token":"action_token"}` | `{"status":"ok","password_set":true,"purpose":"youth_mode","enable_action_token":"enable_token","enable_action_expires_at":"2026-08-20T09:05:00Z","next_action":"enable_youth_mode"}` | | `POST /v1/im/settings/independent-password` | `{"password":"1234","purpose":"youth_mode","action_token":"action_token"}` | `{"status":"ok","password_set":true,"purpose":"youth_mode","enable_action_token":"enable_token","enable_action_expires_at":"2026-08-20T09:05:00Z","next_action":"enable_youth_mode"}` |
| `POST /v1/im/settings/independent-password/verify` | `{"password":"1234","purpose":"close_youth_mode"}` | `{"action_token":"action_token","action":"close_youth_mode","expires_at":"2026-08-20T09:05:00Z"}` | | `POST /v1/im/settings/independent-password/verify` | `{"password":"1234","purpose":"close_youth_mode"}` | `{"action_token":"action_token","action":"close_youth_mode","expires_at":"2026-08-20T09:05:00Z"}` |
| `POST /v1/im/settings/youth-mode/enable` | `{"policy_version":1,"action_token":"enable_action_token"}` | `{"enabled":true,"policy_version":1,"password_set":true,"restrictions":["night_quiet","content_filter","payment_guard"],"restriction_items":[{"code":"night_quiet","title":"深夜免打扰","description":"夜间减少消息提醒打扰,帮助用户保持健康作息。","enforced":false},{"code":"content_filter","title":"内容过滤","description":"对不适合青少年的内容进行识别和限制展示。","enforced":false},{"code":"payment_guard","title":"支付守护","description":"限制或确认支付、充值等敏感消费行为。","enforced":false}],"updated_at":"2026-08-20T09:00:00Z"}` | | `POST /v1/im/settings/youth-mode/enable` | `{"policy_version":1,"action_token":"enable_action_token"}` | `{"enabled":true,"policy_version":1,"password_set":true,"restrictions":["night_quiet","content_filter","payment_guard"],"restriction_items":[{"code":"night_quiet","title":"深夜免打扰","description":"夜间减少消息提醒打扰,帮助用户保持健康作息。","enforced":true},{"code":"content_filter","title":"内容过滤","description":"对不适合青少年的内容进行识别和限制展示。","enforced":true},{"code":"payment_guard","title":"支付守护","description":"限制或确认支付、充值等敏感消费行为。","enforced":false}],"updated_at":"2026-08-20T09:00:00Z"}` |
| `POST /v1/im/settings/youth-mode/disable` | Header `X-Action-Token: action_token`;Body `{}` | `{"enabled":false,"policy_version":1,"password_set":true,"updated_at":"2026-08-20T09:00:00Z"}` | | `POST /v1/im/settings/youth-mode/disable` | Header `X-Action-Token: action_token`;Body `{}` | `{"enabled":false,"policy_version":1,"password_set":true,"updated_at":"2026-08-20T09:00:00Z"}` |
| `POST /v1/im/security/challenges` 开启青少年模式 | Header `Idempotency-Key`;Body `{"scene":"set_independent_password","channel":"sms"}` | `{"challenge_id":"sec_chg_1","phone_mask":"+861****000","expires_in_seconds":300,"resend_after_seconds":60}` | | `POST /v1/im/security/challenges` 开启青少年模式 | Header `Idempotency-Key`;Body `{"scene":"set_independent_password","channel":"sms"}` | `{"challenge_id":"sec_chg_1","phone_mask":"+861****000","expires_in_seconds":300,"resend_after_seconds":60}` |
| `POST /v1/im/security/challenges` 关闭青少年模式 | Header `Idempotency-Key`;Body `{"scene":"close_youth_mode","channel":"sms"}` | `{"challenge_id":"sec_chg_2","phone_mask":"+861****000","expires_in_seconds":300,"resend_after_seconds":60}` | | `POST /v1/im/security/challenges` 关闭青少年模式 | Header `Idempotency-Key`;Body `{"scene":"close_youth_mode","channel":"sms"}` | `{"challenge_id":"sec_chg_2","phone_mask":"+861****000","expires_in_seconds":300,"resend_after_seconds":60}` |
+10 -1
View File
@@ -5,9 +5,11 @@ go 1.26.0
require ( require (
firebase.google.com/go v3.13.0+incompatible firebase.google.com/go v3.13.0+incompatible
github.com/AgoraIO/Tools/DynamicKey/AgoraDynamicKey/go/src v0.0.0-20260726132416-da06bfb83556 github.com/AgoraIO/Tools/DynamicKey/AgoraDynamicKey/go/src v0.0.0-20260726132416-da06bfb83556
github.com/alibabacloud-go/cloudauth-20190307/v3 v3.9.2
github.com/alibabacloud-go/darabonba-openapi/v2 v2.2.4 github.com/alibabacloud-go/darabonba-openapi/v2 v2.2.4
github.com/alibabacloud-go/green-20220302/v3 v3.5.4 github.com/alibabacloud-go/green-20220302/v3 v3.5.4
github.com/alibabacloud-go/tea v1.5.2 github.com/alibabacloud-go/tea v1.5.2
github.com/alibabacloud-go/tea-utils/v2 v2.0.9
github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible
github.com/aws/aws-sdk-go-v2 v1.42.1 github.com/aws/aws-sdk-go-v2 v1.42.1
github.com/aws/aws-sdk-go-v2/config v1.32.27 github.com/aws/aws-sdk-go-v2/config v1.32.27
@@ -52,7 +54,14 @@ require (
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.57.0 // indirect github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.57.0 // indirect
github.com/alibabacloud-go/alibabacloud-gateway-spi v0.0.5 // indirect github.com/alibabacloud-go/alibabacloud-gateway-spi v0.0.5 // indirect
github.com/alibabacloud-go/debug v1.0.1 // indirect github.com/alibabacloud-go/debug v1.0.1 // indirect
github.com/alibabacloud-go/tea-utils/v2 v2.0.9 // indirect github.com/alibabacloud-go/endpoint-util v1.1.0 // indirect
github.com/alibabacloud-go/openapi-util v0.1.0 // indirect
github.com/alibabacloud-go/openplatform-20191219/v2 v2.0.1 // indirect
github.com/alibabacloud-go/tea-fileform v1.1.1 // indirect
github.com/alibabacloud-go/tea-oss-sdk v1.1.3 // indirect
github.com/alibabacloud-go/tea-oss-utils v1.1.0 // indirect
github.com/alibabacloud-go/tea-utils v1.3.6 // indirect
github.com/alibabacloud-go/tea-xml v1.1.3 // indirect
github.com/aliyun/credentials-go v1.4.5 // indirect github.com/aliyun/credentials-go v1.4.5 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.14 // indirect github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.14 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.30 // indirect github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.30 // indirect
+27 -1
View File
@@ -44,12 +44,16 @@ github.com/alibabacloud-go/alibabacloud-gateway-pop v0.0.6/go.mod h1:4EUIoxs/do2
github.com/alibabacloud-go/alibabacloud-gateway-spi v0.0.4/go.mod h1:sCavSAvdzOjul4cEqeVtvlSaSScfNsTQ+46HwlTL1hc= github.com/alibabacloud-go/alibabacloud-gateway-spi v0.0.4/go.mod h1:sCavSAvdzOjul4cEqeVtvlSaSScfNsTQ+46HwlTL1hc=
github.com/alibabacloud-go/alibabacloud-gateway-spi v0.0.5 h1:zE8vH9C7JiZLNJJQ5OwjU9mSi4T9ef9u3BURT6LCLC8= github.com/alibabacloud-go/alibabacloud-gateway-spi v0.0.5 h1:zE8vH9C7JiZLNJJQ5OwjU9mSi4T9ef9u3BURT6LCLC8=
github.com/alibabacloud-go/alibabacloud-gateway-spi v0.0.5/go.mod h1:tWnyE9AjF8J8qqLk645oUmVUnFybApTQWklQmi5tY6g= github.com/alibabacloud-go/alibabacloud-gateway-spi v0.0.5/go.mod h1:tWnyE9AjF8J8qqLk645oUmVUnFybApTQWklQmi5tY6g=
github.com/alibabacloud-go/cloudauth-20190307/v3 v3.9.2 h1:y4s0WQ1jrBtOJfXGgsv/83brJvkkHbFdORp0WDyVAuw=
github.com/alibabacloud-go/cloudauth-20190307/v3 v3.9.2/go.mod h1:kD75qqMQyjCiz6lssjRzYGTumcli8STLXQstVe6ytxk=
github.com/alibabacloud-go/darabonba-array v0.1.0 h1:vR8s7b1fWAQIjEjWnuF0JiKsCvclSRTfDzZHTYqfufY= github.com/alibabacloud-go/darabonba-array v0.1.0 h1:vR8s7b1fWAQIjEjWnuF0JiKsCvclSRTfDzZHTYqfufY=
github.com/alibabacloud-go/darabonba-array v0.1.0/go.mod h1:BLKxr0brnggqOJPqT09DFJ8g3fsDshapUD3C3aOEFaI= github.com/alibabacloud-go/darabonba-array v0.1.0/go.mod h1:BLKxr0brnggqOJPqT09DFJ8g3fsDshapUD3C3aOEFaI=
github.com/alibabacloud-go/darabonba-encode-util v0.0.2 h1:1uJGrbsGEVqWcWxrS9MyC2NG0Ax+GpOM5gtupki31XE= github.com/alibabacloud-go/darabonba-encode-util v0.0.2 h1:1uJGrbsGEVqWcWxrS9MyC2NG0Ax+GpOM5gtupki31XE=
github.com/alibabacloud-go/darabonba-encode-util v0.0.2/go.mod h1:JiW9higWHYXm7F4PKuMgEUETNZasrDM6vqVr/Can7H8= github.com/alibabacloud-go/darabonba-encode-util v0.0.2/go.mod h1:JiW9higWHYXm7F4PKuMgEUETNZasrDM6vqVr/Can7H8=
github.com/alibabacloud-go/darabonba-map v0.0.2 h1:qvPnGB4+dJbJIxOOfawxzF3hzMnIpjmafa0qOTp6udc= github.com/alibabacloud-go/darabonba-map v0.0.2 h1:qvPnGB4+dJbJIxOOfawxzF3hzMnIpjmafa0qOTp6udc=
github.com/alibabacloud-go/darabonba-map v0.0.2/go.mod h1:28AJaX8FOE/ym8OUFWga+MtEzBunJwQGceGQlvaPGPc= github.com/alibabacloud-go/darabonba-map v0.0.2/go.mod h1:28AJaX8FOE/ym8OUFWga+MtEzBunJwQGceGQlvaPGPc=
github.com/alibabacloud-go/darabonba-openapi/v2 v2.0.0/go.mod h1:5JHVmnHvGzR2wNdgaW1zDLQG8kOC4Uec8ubkMogW7OQ=
github.com/alibabacloud-go/darabonba-openapi/v2 v2.0.10/go.mod h1:26a14FGhZVELuz2cc2AolvW4RHmIO3/HRwsdHhaIPDE=
github.com/alibabacloud-go/darabonba-openapi/v2 v2.2.4 h1:o6veen0IZ/Fe1JawwhwQMZcbw67CVDY1pQwXcNWPyQo= github.com/alibabacloud-go/darabonba-openapi/v2 v2.2.4 h1:o6veen0IZ/Fe1JawwhwQMZcbw67CVDY1pQwXcNWPyQo=
github.com/alibabacloud-go/darabonba-openapi/v2 v2.2.4/go.mod h1:eHjVxrT9g8uVYN/nyAwOFQEfkVA154ChiqRc2XnNKYU= github.com/alibabacloud-go/darabonba-openapi/v2 v2.2.4/go.mod h1:eHjVxrT9g8uVYN/nyAwOFQEfkVA154ChiqRc2XnNKYU=
github.com/alibabacloud-go/darabonba-signature-util v0.0.7 h1:UzCnKvsjPFzApvODDNEYqBHMFt1w98wC7FOo0InLyxg= github.com/alibabacloud-go/darabonba-signature-util v0.0.7 h1:UzCnKvsjPFzApvODDNEYqBHMFt1w98wC7FOo0InLyxg=
@@ -64,27 +68,47 @@ github.com/alibabacloud-go/endpoint-util v1.1.0 h1:r/4D3VSw888XGaeNpP994zDUaxdgT
github.com/alibabacloud-go/endpoint-util v1.1.0/go.mod h1:O5FuCALmCKs2Ff7JFJMudHs0I5EBgecXXxZRyswlEjE= github.com/alibabacloud-go/endpoint-util v1.1.0/go.mod h1:O5FuCALmCKs2Ff7JFJMudHs0I5EBgecXXxZRyswlEjE=
github.com/alibabacloud-go/green-20220302/v3 v3.5.4 h1:ZnlHri7tTLl28JWB5S2J6MYvD0ewJFI7hedK4E7+gSk= github.com/alibabacloud-go/green-20220302/v3 v3.5.4 h1:ZnlHri7tTLl28JWB5S2J6MYvD0ewJFI7hedK4E7+gSk=
github.com/alibabacloud-go/green-20220302/v3 v3.5.4/go.mod h1:PQTtoEuWOsuq7H7sgTRcXDw4Ut/Hekgy5z2mWGtO9OQ= github.com/alibabacloud-go/green-20220302/v3 v3.5.4/go.mod h1:PQTtoEuWOsuq7H7sgTRcXDw4Ut/Hekgy5z2mWGtO9OQ=
github.com/alibabacloud-go/openapi-util v0.0.11/go.mod h1:sQuElr4ywwFRlCCberQwKRFhRzIyG4QTP/P4y1CJ6Ws=
github.com/alibabacloud-go/openapi-util v0.1.0 h1:0z75cIULkDrdEhkLWgi9tnLe+KhAFE/r5Pb3312/eAY= github.com/alibabacloud-go/openapi-util v0.1.0 h1:0z75cIULkDrdEhkLWgi9tnLe+KhAFE/r5Pb3312/eAY=
github.com/alibabacloud-go/openapi-util v0.1.0/go.mod h1:sQuElr4ywwFRlCCberQwKRFhRzIyG4QTP/P4y1CJ6Ws= github.com/alibabacloud-go/openapi-util v0.1.0/go.mod h1:sQuElr4ywwFRlCCberQwKRFhRzIyG4QTP/P4y1CJ6Ws=
github.com/alibabacloud-go/openplatform-20191219/v2 v2.0.1 h1:L0TIjr9Qh/SLVc1yPhFkcB9+9SbCNK/jPq4ZKB5zmnc=
github.com/alibabacloud-go/openplatform-20191219/v2 v2.0.1/go.mod h1:EKxBRDLcMzwl4VLF/1WJwlByZZECJawPXUvinKMsTTs=
github.com/alibabacloud-go/tea v1.1.0/go.mod h1:IkGyUSX4Ba1V+k4pCtJUc6jDpZLFph9QMy2VUPTwukg= github.com/alibabacloud-go/tea v1.1.0/go.mod h1:IkGyUSX4Ba1V+k4pCtJUc6jDpZLFph9QMy2VUPTwukg=
github.com/alibabacloud-go/tea v1.1.7/go.mod h1:/tmnEaQMyb4Ky1/5D+SE1BAsa5zj/KeGOFfwYm3N/p4= github.com/alibabacloud-go/tea v1.1.7/go.mod h1:/tmnEaQMyb4Ky1/5D+SE1BAsa5zj/KeGOFfwYm3N/p4=
github.com/alibabacloud-go/tea v1.1.8/go.mod h1:/tmnEaQMyb4Ky1/5D+SE1BAsa5zj/KeGOFfwYm3N/p4= github.com/alibabacloud-go/tea v1.1.8/go.mod h1:/tmnEaQMyb4Ky1/5D+SE1BAsa5zj/KeGOFfwYm3N/p4=
github.com/alibabacloud-go/tea v1.1.10/go.mod h1:/tmnEaQMyb4Ky1/5D+SE1BAsa5zj/KeGOFfwYm3N/p4=
github.com/alibabacloud-go/tea v1.1.11/go.mod h1:/tmnEaQMyb4Ky1/5D+SE1BAsa5zj/KeGOFfwYm3N/p4= github.com/alibabacloud-go/tea v1.1.11/go.mod h1:/tmnEaQMyb4Ky1/5D+SE1BAsa5zj/KeGOFfwYm3N/p4=
github.com/alibabacloud-go/tea v1.1.12/go.mod h1:/tmnEaQMyb4Ky1/5D+SE1BAsa5zj/KeGOFfwYm3N/p4=
github.com/alibabacloud-go/tea v1.1.17/go.mod h1:nXxjm6CIFkBhwW4FQkNrolwbfon8Svy6cujmKFUq98A= github.com/alibabacloud-go/tea v1.1.17/go.mod h1:nXxjm6CIFkBhwW4FQkNrolwbfon8Svy6cujmKFUq98A=
github.com/alibabacloud-go/tea v1.1.19/go.mod h1:nXxjm6CIFkBhwW4FQkNrolwbfon8Svy6cujmKFUq98A=
github.com/alibabacloud-go/tea v1.1.20/go.mod h1:nXxjm6CIFkBhwW4FQkNrolwbfon8Svy6cujmKFUq98A= github.com/alibabacloud-go/tea v1.1.20/go.mod h1:nXxjm6CIFkBhwW4FQkNrolwbfon8Svy6cujmKFUq98A=
github.com/alibabacloud-go/tea v1.2.2/go.mod h1:CF3vOzEMAG+bR4WOql8gc2G9H3EkH3ZLAQdpmpXMgwk= github.com/alibabacloud-go/tea v1.2.2/go.mod h1:CF3vOzEMAG+bR4WOql8gc2G9H3EkH3ZLAQdpmpXMgwk=
github.com/alibabacloud-go/tea v1.5.2 h1:DUzV/cRTbJC++rmADfqVu//GuWbaD64ZJ3yW3v2ukNU= github.com/alibabacloud-go/tea v1.5.2 h1:DUzV/cRTbJC++rmADfqVu//GuWbaD64ZJ3yW3v2ukNU=
github.com/alibabacloud-go/tea v1.5.2/go.mod h1:hgSs82CkOiehSQMoiFN79dL6zsGX7pVGvnn9SIEs8/0= github.com/alibabacloud-go/tea v1.5.2/go.mod h1:hgSs82CkOiehSQMoiFN79dL6zsGX7pVGvnn9SIEs8/0=
github.com/alibabacloud-go/tea-utils v1.3.1 h1:iWQeRzRheqCMuiF3+XkfybB3kTgUXkXX+JMrqfLeB2I= github.com/alibabacloud-go/tea-fileform v1.1.1 h1:1YG6erAP3joQ0XdCXYIotuD7zyOM6qCR49xkp5FZDeU=
github.com/alibabacloud-go/tea-fileform v1.1.1/go.mod h1:ZeCV91o4ISmxidd686f0ebdS5EDHWU+vW+TkjLhrsFE=
github.com/alibabacloud-go/tea-oss-sdk v1.1.3 h1:EhAHI6edMeqgkZEqP7r4nc9iMWAUBKGxJHoBsOSKTtU=
github.com/alibabacloud-go/tea-oss-sdk v1.1.3/go.mod h1:yUnodpR3Bf2rudLE7V/Gft5txjJF30Pk+hH77K/Eab0=
github.com/alibabacloud-go/tea-oss-utils v1.1.0 h1:y65crjjcZ2Pbb6UZtC2deuIZHDVTS3IaDWE7M9nVLRc=
github.com/alibabacloud-go/tea-oss-utils v1.1.0/go.mod h1:PFCF12e9yEKyBUIn7X1IrF/pNjvxgkHy0CgxX4+xRuY=
github.com/alibabacloud-go/tea-utils v1.3.1/go.mod h1:EI/o33aBfj3hETm4RLiAxF/ThQdSngxrpF8rKUDJjPE= github.com/alibabacloud-go/tea-utils v1.3.1/go.mod h1:EI/o33aBfj3hETm4RLiAxF/ThQdSngxrpF8rKUDJjPE=
github.com/alibabacloud-go/tea-utils v1.3.6 h1:bVjrxHztM8hAs6nOfLWCgxQfAtKb9RgFFMV6J3rdvB4=
github.com/alibabacloud-go/tea-utils v1.3.6/go.mod h1:EI/o33aBfj3hETm4RLiAxF/ThQdSngxrpF8rKUDJjPE=
github.com/alibabacloud-go/tea-utils/v2 v2.0.0/go.mod h1:U5MTY10WwlquGPS34DOeomUGBB0gXbLueiq5Trwu0C4=
github.com/alibabacloud-go/tea-utils/v2 v2.0.5/go.mod h1:dL6vbUT35E4F4bFTHL845eUloqaerYBYPsdWR2/jhe4= github.com/alibabacloud-go/tea-utils/v2 v2.0.5/go.mod h1:dL6vbUT35E4F4bFTHL845eUloqaerYBYPsdWR2/jhe4=
github.com/alibabacloud-go/tea-utils/v2 v2.0.6/go.mod h1:qxn986l+q33J5VkialKMqT/TTs3E+U9MJpd001iWQ9I=
github.com/alibabacloud-go/tea-utils/v2 v2.0.9 h1:y6pUIlhjxbZl9ObDAcmA1H3c21eaAxADHTDQmBnAIgA= github.com/alibabacloud-go/tea-utils/v2 v2.0.9 h1:y6pUIlhjxbZl9ObDAcmA1H3c21eaAxADHTDQmBnAIgA=
github.com/alibabacloud-go/tea-utils/v2 v2.0.9/go.mod h1:qxn986l+q33J5VkialKMqT/TTs3E+U9MJpd001iWQ9I= github.com/alibabacloud-go/tea-utils/v2 v2.0.9/go.mod h1:qxn986l+q33J5VkialKMqT/TTs3E+U9MJpd001iWQ9I=
github.com/alibabacloud-go/tea-xml v1.1.1/go.mod h1:Rq08vgCcCAjHyRi/M7xlHKUykZCEtyBy9+DPF6GgEu8=
github.com/alibabacloud-go/tea-xml v1.1.2/go.mod h1:Rq08vgCcCAjHyRi/M7xlHKUykZCEtyBy9+DPF6GgEu8=
github.com/alibabacloud-go/tea-xml v1.1.3 h1:7LYnm+JbOq2B+T/B0fHC4Ies4/FofC4zHzYtqw7dgt0=
github.com/alibabacloud-go/tea-xml v1.1.3/go.mod h1:Rq08vgCcCAjHyRi/M7xlHKUykZCEtyBy9+DPF6GgEu8=
github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible h1:8psS8a+wKfiLt1iVDX79F7Y6wUM49Lcha2FMXt4UM8g= github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible h1:8psS8a+wKfiLt1iVDX79F7Y6wUM49Lcha2FMXt4UM8g=
github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible/go.mod h1:T/Aws4fEfogEE9v+HPhhw+CntffsBHJ8nXQCwKr0/g8= github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible/go.mod h1:T/Aws4fEfogEE9v+HPhhw+CntffsBHJ8nXQCwKr0/g8=
github.com/aliyun/credentials-go v1.1.2/go.mod h1:ozcZaMR5kLM7pwtCMEpVmQ242suV6qTJya2bDq4X1Tw= github.com/aliyun/credentials-go v1.1.2/go.mod h1:ozcZaMR5kLM7pwtCMEpVmQ242suV6qTJya2bDq4X1Tw=
github.com/aliyun/credentials-go v1.3.1/go.mod h1:8jKYhQuDawt8x2+fusqa1Y6mPxemTsBEN04dgcAcYz0= github.com/aliyun/credentials-go v1.3.1/go.mod h1:8jKYhQuDawt8x2+fusqa1Y6mPxemTsBEN04dgcAcYz0=
github.com/aliyun/credentials-go v1.3.6/go.mod h1:1LxUuX7L5YrZUWzBrRyk0SwSdH4OmPrib8NVePL3fxM= github.com/aliyun/credentials-go v1.3.6/go.mod h1:1LxUuX7L5YrZUWzBrRyk0SwSdH4OmPrib8NVePL3fxM=
github.com/aliyun/credentials-go v1.3.10/go.mod h1:Jm6d+xIgwJVLVWT561vy67ZRP4lPTQxMbEYRuT2Ti1U=
github.com/aliyun/credentials-go v1.4.5 h1:O76WYKgdy1oQYYiJkERjlA2dxGuvLRrzuO2ScrtGWSk= github.com/aliyun/credentials-go v1.4.5 h1:O76WYKgdy1oQYYiJkERjlA2dxGuvLRrzuO2ScrtGWSk=
github.com/aliyun/credentials-go v1.4.5/go.mod h1:Jm6d+xIgwJVLVWT561vy67ZRP4lPTQxMbEYRuT2Ti1U= github.com/aliyun/credentials-go v1.4.5/go.mod h1:Jm6d+xIgwJVLVWT561vy67ZRP4lPTQxMbEYRuT2Ti1U=
github.com/aws/aws-sdk-go-v2 v1.42.1 h1:9eOTgu1z/dVtYpNZ3/8/XbbaX0x/BqE3HUzAzs6K0ek= github.com/aws/aws-sdk-go-v2 v1.42.1 h1:9eOTgu1z/dVtYpNZ3/8/XbbaX0x/BqE3HUzAzs6K0ek=
@@ -138,6 +162,7 @@ github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/clbanning/mxj/v2 v2.5.5/go.mod h1:hNiWqW14h+kc+MdF9C6/YoRfjEJoR3ou6tn/Qo+ve2s=
github.com/clbanning/mxj/v2 v2.7.0 h1:WA/La7UGCanFe5NpHF0Q3DNtnCsVoxbPKuyBNHWRyME= github.com/clbanning/mxj/v2 v2.7.0 h1:WA/La7UGCanFe5NpHF0Q3DNtnCsVoxbPKuyBNHWRyME=
github.com/clbanning/mxj/v2 v2.7.0/go.mod h1:hNiWqW14h+kc+MdF9C6/YoRfjEJoR3ou6tn/Qo+ve2s= github.com/clbanning/mxj/v2 v2.7.0/go.mod h1:hNiWqW14h+kc+MdF9C6/YoRfjEJoR3ou6tn/Qo+ve2s=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
@@ -494,6 +519,7 @@ golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI= golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI=
golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE= golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4= golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
golang.org/x/time v0.0.0-20200416051211-89c76fbcd5d1/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U= golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno= golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+6
View File
@@ -721,6 +721,8 @@ CREATE TABLE IF NOT EXISTS `im_group_messages` (
`msg_type` varchar(32) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'text' COMMENT 'text/image/file/audio/video/system 等', `msg_type` varchar(32) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'text' COMMENT 'text/image/file/audio/video/system 等',
`status` varchar(32) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'normal' COMMENT 'normal/recalled/deleted/blocked', `status` varchar(32) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'normal' COMMENT 'normal/recalled/deleted/blocked',
`content` json DEFAULT NULL COMMENT '消息正文或 Drafty 内容', `content` json DEFAULT NULL COMMENT '消息正文或 Drafty 内容',
`content_nonce` varbinary(12) DEFAULT NULL COMMENT '每条消息独立随机Nonce',
`key_version` int NOT NULL DEFAULT '0' COMMENT '商户消息DEK版本',
`head` json DEFAULT NULL COMMENT '消息头,包含 @、引用、客户端能力等', `head` json DEFAULT NULL COMMENT '消息头,包含 @、引用、客户端能力等',
`mentioned_user_ids` json DEFAULT NULL COMMENT '@ 用户列表,便于提醒和检索', `mentioned_user_ids` json DEFAULT NULL COMMENT '@ 用户列表,便于提醒和检索',
`mention_all` tinyint(1) NOT NULL DEFAULT '0' COMMENT '是否 @ 全体', `mention_all` tinyint(1) NOT NULL DEFAULT '0' COMMENT '是否 @ 全体',
@@ -1088,6 +1090,8 @@ CREATE TABLE IF NOT EXISTS `im_message_revisions` (
`actor` bigint NOT NULL, `actor` bigint NOT NULL,
`head` json DEFAULT NULL, `head` json DEFAULT NULL,
`content` json DEFAULT NULL, `content` json DEFAULT NULL,
`content_nonce` varbinary(12) DEFAULT NULL,
`key_version` int NOT NULL DEFAULT '0',
`reason` varchar(256) NOT NULL DEFAULT '', `reason` varchar(256) NOT NULL DEFAULT '',
PRIMARY KEY (`id`), PRIMARY KEY (`id`),
UNIQUE KEY `uk_im_message_revisions_version` (`topic`,`seqid`,`version`) UNIQUE KEY `uk_im_message_revisions_version` (`topic`,`seqid`,`version`)
@@ -1108,6 +1112,8 @@ CREATE TABLE IF NOT EXISTS `im_messages` (
`recalled_by` bigint DEFAULT NULL, `recalled_by` bigint DEFAULT NULL,
`head` json DEFAULT NULL, `head` json DEFAULT NULL,
`content` json DEFAULT NULL, `content` json DEFAULT NULL,
`content_nonce` varbinary(12) DEFAULT NULL,
`key_version` int NOT NULL DEFAULT '0',
PRIMARY KEY (`id`), PRIMARY KEY (`id`),
UNIQUE KEY `messages_sender_client_mid` (`from`,`client_msg_id`), UNIQUE KEY `messages_sender_client_mid` (`from`,`client_msg_id`),
UNIQUE KEY `uk_im_messages_topic_seq` (`topic`,`seqid`) UNIQUE KEY `uk_im_messages_topic_seq` (`topic`,`seqid`)
+2
View File
@@ -383,6 +383,7 @@ func main() {
profile_visibility VARCHAR(32) NOT NULL DEFAULT 'friends', profile_visibility VARCHAR(32) NOT NULL DEFAULT 'friends',
last_seen_visibility VARCHAR(32) NOT NULL DEFAULT 'friends', last_seen_visibility VARCHAR(32) NOT NULL DEFAULT 'friends',
read_receipts TINYINT NOT NULL DEFAULT 1, read_receipts TINYINT NOT NULL DEFAULT 1,
friend_request_verification TINYINT NOT NULL DEFAULT 1,
group_invite_verification TINYINT NOT NULL DEFAULT 0, group_invite_verification TINYINT NOT NULL DEFAULT 0,
version INT NOT NULL DEFAULT 1, version INT NOT NULL DEFAULT 1,
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3), created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
@@ -540,6 +541,7 @@ func main() {
"ALTER TABLE tinode.im_youth_mode_settings ADD COLUMN enabled_at DATETIME(3) AFTER restrictions", "ALTER TABLE tinode.im_youth_mode_settings ADD COLUMN enabled_at DATETIME(3) AFTER restrictions",
"ALTER TABLE tinode.im_users ADD COLUMN phone VARCHAR(32) NULL AFTER useragent", "ALTER TABLE tinode.im_users ADD COLUMN phone VARCHAR(32) NULL AFTER useragent",
"ALTER TABLE tinode.im_user_profiles ADD COLUMN email VARCHAR(128) NOT NULL DEFAULT '' AFTER region", "ALTER TABLE tinode.im_user_profiles ADD COLUMN email VARCHAR(128) NOT NULL DEFAULT '' AFTER region",
"ALTER TABLE tinode.im_user_privacy_settings ADD COLUMN friend_request_verification TINYINT NOT NULL DEFAULT 1 AFTER read_receipts",
"ALTER TABLE tinode.im_user_privacy_settings ADD COLUMN group_invite_verification TINYINT NOT NULL DEFAULT 0 AFTER read_receipts", "ALTER TABLE tinode.im_user_privacy_settings ADD COLUMN group_invite_verification TINYINT NOT NULL DEFAULT 0 AFTER read_receipts",
} { } {
if _, err = db.Exec(stmt); err != nil && !strings.Contains(err.Error(), "Duplicate column name") { if _, err = db.Exec(stmt); err != nil && !strings.Contains(err.Error(), "Duplicate column name") {
+86
View File
@@ -0,0 +1,86 @@
-- 在当前商户业务库增加消息加密字段。
-- 可重复执行,覆盖单聊、消息修订、群聊主表及全部现有群消息月分片。
DELIMITER $$
DROP PROCEDURE IF EXISTS add_message_crypto_column$$
CREATE PROCEDURE add_message_crypto_column(
IN p_table VARCHAR(64),
IN p_column VARCHAR(64),
IN p_definition TEXT
)
BEGIN
IF EXISTS (
SELECT 1 FROM information_schema.TABLES
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = p_table
) AND NOT EXISTS (
SELECT 1 FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = p_table AND COLUMN_NAME = p_column
) THEN
SET @ddl = CONCAT('ALTER TABLE `', REPLACE(p_table, '`', '``'), '` ADD COLUMN `',
REPLACE(p_column, '`', '``'), '` ', p_definition);
PREPARE stmt FROM @ddl;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END IF;
END$$
DROP PROCEDURE IF EXISTS drop_message_crypto_column$$
CREATE PROCEDURE drop_message_crypto_column(IN p_table VARCHAR(64), IN p_column VARCHAR(64))
BEGIN
IF EXISTS (
SELECT 1 FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = p_table AND COLUMN_NAME = p_column
) THEN
SET @ddl = CONCAT('ALTER TABLE `', REPLACE(p_table, '`', '``'), '` DROP COLUMN `',
REPLACE(p_column, '`', '``'), '`');
PREPARE stmt FROM @ddl;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END IF;
END$$
DROP PROCEDURE IF EXISTS migrate_message_crypto_table$$
CREATE PROCEDURE migrate_message_crypto_table(IN p_table VARCHAR(64))
BEGIN
CALL add_message_crypto_column(p_table, 'content_nonce', 'VARBINARY(12) NULL COMMENT ''每条消息独立随机Nonce'' AFTER `content`');
CALL add_message_crypto_column(p_table, 'key_version', 'INT NOT NULL DEFAULT 0 COMMENT ''商户消息DEK版本'' AFTER `content_nonce`');
CALL drop_message_crypto_column(p_table, 'content_ciphertext');
CALL drop_message_crypto_column(p_table, 'encryption_alg');
CALL drop_message_crypto_column(p_table, 'encrypted');
END$$
DROP PROCEDURE IF EXISTS migrate_message_crypto_schema$$
CREATE PROCEDURE migrate_message_crypto_schema()
BEGIN
DECLARE done INT DEFAULT 0;
DECLARE table_name_value VARCHAR(64);
DECLARE shard_cursor CURSOR FOR
SELECT TABLE_NAME
FROM information_schema.TABLES
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME REGEXP '^im_group_messages_[0-9]{6}$';
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
CALL migrate_message_crypto_table('im_messages');
CALL migrate_message_crypto_table('im_message_revisions');
CALL migrate_message_crypto_table('im_group_messages');
OPEN shard_cursor;
shard_loop: LOOP
FETCH shard_cursor INTO table_name_value;
IF done = 1 THEN
LEAVE shard_loop;
END IF;
CALL migrate_message_crypto_table(table_name_value);
END LOOP;
CLOSE shard_cursor;
END$$
CALL migrate_message_crypto_schema()$$
DROP PROCEDURE migrate_message_crypto_schema$$
DROP PROCEDURE migrate_message_crypto_table$$
DROP PROCEDURE drop_message_crypto_column$$
DROP PROCEDURE add_message_crypto_column$$
DELIMITER ;
+82 -8
View File
@@ -60,15 +60,15 @@ type adminInternalTenantDTO struct {
UpdatedBy int64 `json:"updated_by"` UpdatedBy int64 `json:"updated_by"`
} }
func initAdminInternalAPI(raw json.RawMessage) error { func initAdminInternalAPI(raw json.RawMessage) (*adminInternalClient, error) {
store.SetAdminInternalConfigProvider(nil) store.SetAdminInternalConfigProvider(nil)
loginIPWhitelistProvider = nil loginIPWhitelistProvider = nil
if len(raw) == 0 || string(raw) == "null" { if len(raw) == 0 || string(raw) == "null" {
return nil return nil, nil
} }
var cfg adminInternalAPIConfig var cfg adminInternalAPIConfig
if err := json.Unmarshal(raw, &cfg); err != nil { if err := json.Unmarshal(raw, &cfg); err != nil {
return err return nil, err
} }
cfg.BaseURL = strings.TrimRight(strings.TrimSpace(cfg.BaseURL), "/") cfg.BaseURL = strings.TrimRight(strings.TrimSpace(cfg.BaseURL), "/")
cfg.ServiceName = strings.TrimSpace(cfg.ServiceName) cfg.ServiceName = strings.TrimSpace(cfg.ServiceName)
@@ -79,10 +79,10 @@ func initAdminInternalAPI(raw json.RawMessage) error {
cfg.TimeoutMS = 3000 cfg.TimeoutMS = 3000
} }
if !cfg.Enabled { if !cfg.Enabled {
return nil return nil, nil
} }
if cfg.BaseURL == "" || strings.TrimSpace(cfg.Secret) == "" { if cfg.BaseURL == "" || strings.TrimSpace(cfg.Secret) == "" {
return errors.New("admin_internal_api enabled but base_url or secret is empty") return nil, errors.New("admin_internal_api enabled but base_url or secret is empty")
} }
client := &adminInternalClient{ client := &adminInternalClient{
config: cfg, config: cfg,
@@ -90,8 +90,9 @@ func initAdminInternalAPI(raw json.RawMessage) error {
} }
store.SetAdminInternalConfigProvider(client) store.SetAdminInternalConfigProvider(client)
loginIPWhitelistProvider = client loginIPWhitelistProvider = client
go store.WarmMessageEncryptionKeys()
logs.Info.Printf("Admin internal config client enabled: base_url=%s service=%s", cfg.BaseURL, cfg.ServiceName) logs.Info.Printf("Admin internal config client enabled: base_url=%s service=%s", cfg.BaseURL, cfg.ServiceName)
return nil return client, nil
} }
func (c *adminInternalClient) GetTenantByCode(code string) (*types.Tenant, error) { func (c *adminInternalClient) GetTenantByCode(code string) (*types.Tenant, error) {
@@ -167,6 +168,19 @@ func (c *adminInternalClient) ThirdPartyConfig(group string, keys ...string) (ma
return wrapper.Values, nil return wrapper.Values, nil
} }
func (c *adminInternalClient) DictionaryConfig(dictType string) ([]types.DictionaryItem, error) {
var wrapper struct {
Items []types.DictionaryItem `json:"items"`
}
if err := c.getJSON("/internal/im-config/dictionary/"+url.PathEscape(strings.TrimSpace(dictType)), nil, &wrapper); err != nil {
return nil, err
}
if wrapper.Items == nil {
wrapper.Items = []types.DictionaryItem{}
}
return wrapper.Items, nil
}
func (c *adminInternalClient) ModerationProvider() (*types.ModerationProviderConfig, error) { func (c *adminInternalClient) ModerationProvider() (*types.ModerationProviderConfig, error) {
var result types.ModerationProviderConfig var result types.ModerationProviderConfig
if err := c.getJSON("/internal/im-config/moderation/provider", nil, &result); err != nil { if err := c.getJSON("/internal/im-config/moderation/provider", nil, &result); err != nil {
@@ -196,6 +210,55 @@ func (c *adminInternalClient) SensitiveWords(tenantCode string) (*types.Sensitiv
return &result, nil return &result, nil
} }
func (c *adminInternalClient) SecurityPolicy(tenantCode string) (*types.SecurityPolicy, error) {
var result types.SecurityPolicy
path := "/internal/im-config/tenants/" + url.PathEscape(strings.TrimSpace(tenantCode)) + "/security-policy"
if err := c.getJSON(path, nil, &result); err != nil {
return nil, err
}
if result.FriendDisabledPolicy == nil {
result.FriendDisabledPolicy = map[string]any{}
}
if result.CurfewRules == nil {
result.CurfewRules = []*types.MessageCurfewRule{}
}
return &result, nil
}
func (c *adminInternalClient) MessageKey(tenantCode string, version int) (*types.MessageKey, error) {
path := "/internal/im-config/tenants/" + url.PathEscape(strings.TrimSpace(tenantCode)) + "/message-keys/active"
if version > 0 {
path = "/internal/im-config/tenants/" + url.PathEscape(strings.TrimSpace(tenantCode)) + "/message-keys/" + strconv.Itoa(version)
}
var result types.MessageKey
if err := c.getJSON(path, nil, &result); err != nil {
return nil, err
}
return &result, nil
}
func (c *adminInternalClient) MessageKeyAck(tenantCode string, version int) error {
if version <= 0 {
return types.ErrMalformed
}
path := "/internal/im-config/tenants/" + url.PathEscape(strings.TrimSpace(tenantCode)) + "/message-keys/ack"
return c.doJSON(http.MethodPost, path, nil, map[string]int{"version": version}, nil)
}
func (c *adminInternalClient) RealNamePolicy(id types.TenantID) (*types.RealNamePolicy, error) {
if !id.IsValid() {
return nil, types.ErrMalformed
}
var result types.RealNamePolicy
path := "/internal/im-config/tenants/by-id/" + strconv.FormatInt(int64(id), 10) + "/realname-policy"
if err := c.getJSON(path, nil, &result); err != nil {
return nil, err
}
result.TenantID = id
result.Normalize()
return &result, nil
}
func (c *adminInternalClient) TenantLoginIPWhitelist(id types.TenantID) (*tenantLoginIPWhitelistConfig, error) { func (c *adminInternalClient) TenantLoginIPWhitelist(id types.TenantID) (*tenantLoginIPWhitelistConfig, error) {
if !id.IsValid() { if !id.IsValid() {
return nil, types.ErrMalformed return nil, types.ErrMalformed
@@ -252,6 +315,10 @@ func (c *adminInternalClient) LatestAppRelease(platform, channel string) (*types
} }
func (c *adminInternalClient) getJSON(path string, query url.Values, out any) error { func (c *adminInternalClient) getJSON(path string, query url.Values, out any) error {
return c.doJSON(http.MethodGet, path, query, nil, out)
}
func (c *adminInternalClient) doJSON(method, path string, query url.Values, payload, out any) error {
if c == nil || c.http == nil { if c == nil || c.http == nil {
return types.ErrUnsupported return types.ErrUnsupported
} }
@@ -263,11 +330,18 @@ func (c *adminInternalClient) getJSON(path string, query url.Values, out any) er
if len(query) > 0 { if len(query) > 0 {
u.RawQuery = query.Encode() u.RawQuery = query.Encode()
} }
req, err := http.NewRequest(http.MethodGet, u.String(), nil) var requestBody []byte
if payload != nil {
requestBody, err = json.Marshal(payload)
if err != nil { if err != nil {
return err return err
} }
if err = c.signRequest(req, nil); err != nil { }
req, err := http.NewRequest(method, u.String(), nil)
if err != nil {
return err
}
if err = c.signRequest(req, requestBody); err != nil {
return err return err
} }
resp, err := c.http.Do(req) resp, err := c.http.Do(req)
+8 -1
View File
@@ -55,7 +55,7 @@ func recordSessionLoginSuccess(s *Session, rec *auth.Rec, scheme, suppliedAccoun
} }
} }
func recordSessionLoginFailure(s *Session, scheme string, secret []byte, rec *auth.Rec, requestID string, cause error, at time.Time) { func recordSessionLoginFailure(s *Session, scheme string, secret []byte, rec *auth.Rec, requestID string, cause error, at time.Time, riskActionOverride ...string) {
if s == nil || !s.tenantID.IsValid() { if s == nil || !s.tenantID.IsValid() {
return return
} }
@@ -71,6 +71,13 @@ func recordSessionLoginFailure(s *Session, scheme string, secret []byte, rec *au
} }
accountType, account := loginAuditAccount(scheme, suppliedAccount, credential) accountType, account := loginAuditAccount(scheme, suppliedAccount, credential)
failureType, reason, riskLevel, riskAction := classifyLoginFailure(scheme, cause) failureType, reason, riskLevel, riskAction := classifyLoginFailure(scheme, cause)
for _, override := range riskActionOverride {
if strings.TrimSpace(override) == "" {
continue
}
riskAction = strings.TrimSpace(override)
riskLevel = "high"
}
event := &types.LoginFailureEvent{ event := &types.LoginFailureEvent{
UserID: uid, AccountType: accountType, AccountHash: hashLoginAccount(account), AccountMask: maskLoginAccount(account), UserID: uid, AccountType: accountType, AccountHash: hashLoginAccount(account), AccountMask: maskLoginAccount(account),
FailureType: failureType, Reason: reason, Detail: truncateAuditValue(loginFailureDetail(cause), 500), FailureType: failureType, Reason: reason, Detail: truncateAuditValue(loginFailureDetail(cause), 500),
+2 -3
View File
@@ -136,9 +136,8 @@ func TestUserQuerySupportsKeywordAndUserIDs(t *testing.T) {
"AS register_type", "AS register_type",
"im_registration_invite_uses", "im_registration_invite_uses",
"im_fileuploads AS avatar", "im_fileuploads AS avatar",
"JSON_EXTRACT(u.public,'$.avatar')", "p.avatar_file_id",
"JSON_EXTRACT(u.public,'$.photo.data')", "im_user_profiles AS p",
"FROM im_org_members AS om",
} { } {
if !strings.Contains(usersInternalQuerySpec().Select+usersInternalQuerySpec().From, field) { if !strings.Contains(usersInternalQuerySpec().Select+usersInternalQuerySpec().From, field) {
t.Fatalf("user query missing list field %q", field) t.Fatalf("user query missing list field %q", field)
+30 -3
View File
@@ -330,6 +330,23 @@ func (call *videoCall) isStaleUnanswered(now time.Time, timeoutSeconds int) bool
return now.Sub(call.startedAt) > time.Duration(timeoutSeconds)*time.Second return now.Sub(call.startedAt) > time.Duration(timeoutSeconds)*time.Second
} }
func (call *videoCall) hasPartySessionOrUser(sid string, uid types.Uid) bool {
if call == nil {
return false
}
if sid != "" {
if _, ok := call.parties[sid]; ok {
return true
}
}
for _, party := range call.parties {
if party.uid == uid {
return true
}
}
return false
}
func (t *Topic) auditCallCreate(call *videoCall, caller types.Uid) { func (t *Topic) auditCallCreate(call *videoCall, caller types.Uid) {
if store.Calls == nil || call == nil || call.callID == "" || caller.IsZero() { if store.Calls == nil || call == nil || call.callID == "" || caller.IsZero() {
return return
@@ -645,8 +662,14 @@ func (t *Topic) handleCallEvent(msg *ClientComMessage) {
case constCallEventHangUp: case constCallEventHangUp:
switch len(t.currentCall.parties) { switch len(t.currentCall.parties) {
case 2: case 2:
// If it's a call in progress, hangup may arrive only from a call participant session. // If it's a call in progress, hangup may arrive from any current
if _, ok := t.currentCall.parties[msg.sess.sid]; !ok { // session of a call participant. This also covers rtc-join
// auto-accept, where the accepting HTTP request has no WebSocket sid.
sid := ""
if msg.sess != nil {
sid = msg.sess.sid
}
if !t.currentCall.hasPartySessionOrUser(sid, asUid) {
return return
} }
case 1: case 1:
@@ -654,7 +677,11 @@ func (t *Topic) handleCallEvent(msg *ClientComMessage) {
originatorUid, originator := t.getCallOriginator() originatorUid, originator := t.getCallOriginator()
// Hangup may come from either the originating session or // Hangup may come from either the originating session or
// any callee user session. // any callee user session.
if asUid == originatorUid && originator.sid != msg.sess.sid { sid := ""
if msg.sess != nil {
sid = msg.sess.sid
}
if asUid == originatorUid && originator.sid != sid {
return return
} }
default: default:
+1 -1
View File
@@ -594,7 +594,7 @@ func (c *Cluster) TopicMaster(msg *ClusterReq, rejected *bool) error {
logs.Warn.Println("cluster: meta request for unknown topic", msg.RcptTo) logs.Warn.Println("cluster: meta request for unknown topic", msg.RcptTo)
} }
case ProxyReqBroadcast: case ProxyReqBroadcast, ProxyReqCall:
select { select {
case globals.hub.routeCli <- msg.CliMsg: case globals.hub.routeCli <- msg.CliMsg:
default: default:
+16
View File
@@ -1293,6 +1293,22 @@ func ErrAuthFailed(id, topic string, serverTs, incomingReqTs time.Time) *ServerC
} }
} }
// ErrAuthLocked account is temporarily locked after too many login failures
// (429); message carries the remaining lock time for the client to display.
func ErrAuthLocked(id string, ts time.Time, message string, params map[string]any) *ServerComMessage {
return &ServerComMessage{
Ctrl: &MsgServerCtrl{
Id: id,
Code: http.StatusTooManyRequests, // 429
Text: message,
Params: params,
Timestamp: ts,
},
Id: id,
Timestamp: ts,
}
}
// ErrAuthUnknownScheme authentication scheme is unrecognized or invalid (401). // ErrAuthUnknownScheme authentication scheme is unrecognized or invalid (401).
func ErrAuthUnknownScheme(id, topic string, ts time.Time) *ServerComMessage { func ErrAuthUnknownScheme(id, topic string, ts time.Time) *ServerComMessage {
return &ServerComMessage{ return &ServerComMessage{
+16 -38
View File
@@ -228,6 +228,12 @@ type ThirdPartyConfigAdapter interface {
ThirdPartyConfig(group string, keys ...string) (map[string]string, error) ThirdPartyConfig(group string, keys ...string) (map[string]string, error)
} }
// RealNamePolicyConfigAdapter reads tenant-level real-name review policy from
// the control database.
type RealNamePolicyConfigAdapter interface {
RealNamePolicy(t.TenantID) (*t.RealNamePolicy, error)
}
// TenantEntryAdapter stores public tenant entry data before a user is authenticated. // TenantEntryAdapter stores public tenant entry data before a user is authenticated.
type TenantEntryAdapter interface { type TenantEntryAdapter interface {
PublicUploadCreate(*t.PublicUpload) (*t.PublicUpload, error) PublicUploadCreate(*t.PublicUpload) (*t.PublicUpload, error)
@@ -285,6 +291,16 @@ type TenantRegistrationInviteAdapter interface {
TenantRegistrationInviteSyncUsage(t.TenantID, int64) error TenantRegistrationInviteSyncUsage(t.TenantID, int64) error
} }
// TenantLoginLockoutAdapter persists per-account login failure counters and
// lock state in each tenant business database.
type TenantLoginLockoutAdapter interface {
TenantLoginLockoutGet(t.TenantID, string) (*t.LoginLockout, error)
TenantLoginLockoutRecordFailure(t.TenantID, string, *int64, time.Time, time.Time) (*t.LoginLockout, error)
TenantLoginLockoutLock(t.TenantID, string, time.Time, time.Time) error
TenantLoginLockoutReset(t.TenantID, string) error
TenantLoginLockoutRelease(t.TenantID, string, time.Time) error
}
// TenantAuthAdapter is the tenant-aware authentication and credential capability. // TenantAuthAdapter is the tenant-aware authentication and credential capability.
// It is separate from Adapter while the remaining persistence domains are migrated. // It is separate from Adapter while the remaining persistence domains are migrated.
type TenantAuthAdapter interface { type TenantAuthAdapter interface {
@@ -403,44 +419,6 @@ type TenantBusinessAdapter interface {
TenantGroupOwnerTransfer(t.TenantID, string, t.Uid, t.Uid, int, time.Time) error TenantGroupOwnerTransfer(t.TenantID, string, t.Uid, t.Uid, int, time.Time) error
TenantGroupCanPost(t.TenantID, string, t.Uid, bool, time.Time) (bool, error) TenantGroupCanPost(t.TenantID, string, t.Uid, bool, time.Time) (bool, error)
TenantWorkbenchModules(t.TenantID, t.Uid) ([]t.WorkbenchModule, int, error)
TenantOrganizationDepartments(t.TenantID, t.Uid, string, string, int) ([]t.OrganizationDepartment, string, error)
TenantOrganizationMembers(t.TenantID, t.Uid, string, string, int) ([]t.OrganizationMember, string, error)
TenantOrganizationMembersAll(t.TenantID, t.Uid, string, string, string, int) ([]t.OrganizationMember, string, error)
TenantOrganizationSearch(t.TenantID, t.Uid, string, string, string, int) ([]t.OrganizationSearchItem, string, error)
TenantAttendanceConfig(t.TenantID, t.Uid, time.Time) (*t.AttendanceConfig, error)
TenantAttendanceRecordList(t.TenantID, t.Uid, t.AttendanceRecordListOptions) ([]t.AttendanceRecord, string, *t.AttendanceRecordStats, error)
TenantAttendanceCheckin(t.TenantID, t.Uid, string, t.AttendanceCheckinRequest) (*t.AttendanceRecord, *t.AttendanceCheckin, error)
TenantAttendanceAppealCreate(t.TenantID, t.Uid, *t.AttendanceAppeal, string) (*t.AttendanceAppeal, error)
TenantAttendanceAppealList(t.TenantID, t.Uid, t.AttendanceAppealListOptions) ([]t.AttendanceAppeal, string, error)
TenantMeetingCreate(t.TenantID, t.Uid, *t.Meeting, []t.Uid) (*t.Meeting, error)
TenantMeetingList(t.TenantID, t.Uid, string, int) ([]t.Meeting, string, error)
TenantMeetingGet(t.TenantID, t.Uid, string) (*t.Meeting, error)
TenantMeetingUpdate(t.TenantID, t.Uid, string, t.MeetingPatch, string) (*t.Meeting, error)
TenantMeetingCancel(t.TenantID, t.Uid, string, string) (*t.Meeting, error)
TenantMeetingRemind(t.TenantID, t.Uid, string, string) ([]t.MeetingParticipant, error)
TenantMeetingRespond(t.TenantID, t.Uid, string, string, int) (*t.Meeting, error)
TenantMeetingNoteList(t.TenantID, t.Uid, string, string, int) ([]t.Meeting, string, error)
TenantMeetingNoteFiles(t.TenantID, t.Uid, string) ([]t.MeetingNoteFile, error)
TenantMeetingNoteFileCreate(t.TenantID, t.Uid, string, string, string, string) (*t.MeetingNoteFile, error)
TenantMeetingNoteFileReplace(t.TenantID, t.Uid, string, string, string, string, int) (*t.MeetingNoteFile, error)
TenantMeetingNoteFileDelete(t.TenantID, t.Uid, string, string, string, int) (*t.MeetingNoteFile, error)
TenantDailyReportList(t.TenantID, t.Uid, string, string, string, string, string, int) ([]t.DailyReport, string, error)
TenantDailyReportCreateDraft(t.TenantID, t.Uid, string, time.Time) (*t.DailyReport, error)
TenantDailyReportGet(t.TenantID, t.Uid, string) (*t.DailyReport, error)
TenantDailyReportSaveDraft(t.TenantID, t.Uid, string, string, []string, int) (*t.DailyReport, error)
TenantDailyReportSubmit(t.TenantID, t.Uid, string, int) (*t.DailyReport, error)
TenantDailyReportStats(t.TenantID, t.Uid, time.Time, string) (*t.DailyReportStats, error)
TenantDailyReportRemind(t.TenantID, t.Uid, time.Time, []t.Uid, string) ([]t.Uid, error)
TenantMonthlyReportList(t.TenantID, t.Uid, string, string, string, string, string, int) ([]t.MonthlyReport, string, error)
TenantMonthlyReportCreateDraft(t.TenantID, t.Uid, string, string) (*t.MonthlyReport, error)
TenantMonthlyReportGet(t.TenantID, t.Uid, string) (*t.MonthlyReport, error)
TenantMonthlyReportSaveDraft(t.TenantID, t.Uid, string, string, string, []string, int) (*t.MonthlyReport, error)
TenantMonthlyReportSubmit(t.TenantID, t.Uid, string, int) (*t.MonthlyReport, error)
TenantMonthlyReportReopen(t.TenantID, t.Uid, string, int) (*t.MonthlyReport, error)
TenantMonthlyReportStats(t.TenantID, t.Uid, string, string) (*t.MonthlyReportStats, error)
TenantMailboxConfigGet(t.TenantID, t.Uid) (*t.MailboxConfig, error)
TenantMailboxSubmissionCreate(t.TenantID, t.Uid, *t.MailboxSubmission, string) (*t.MailboxSubmission, error)
TenantUserProfileGet(t.TenantID, t.Uid) (*t.UserProfile, error) TenantUserProfileGet(t.TenantID, t.Uid) (*t.UserProfile, error)
TenantUserProfileUpdate(t.TenantID, t.Uid, t.UserProfilePatch) (*t.UserProfile, error) TenantUserProfileUpdate(t.TenantID, t.Uid, t.UserProfilePatch) (*t.UserProfile, error)
TenantUserPrivacyGet(t.TenantID, t.Uid) (*t.UserPrivacySettings, error) TenantUserPrivacyGet(t.TenantID, t.Uid) (*t.UserPrivacySettings, error)
+37 -547
View File
@@ -284,6 +284,9 @@ func (a *adapter) Open(jsonconfig json.RawMessage) error {
return errors.New("mysql adapter failed to open tenant registry database: " + err.Error()) return errors.New("mysql adapter failed to open tenant registry database: " + err.Error())
} }
} }
if err = ensureUserPrivacySchema(a.db); err != nil {
return errors.New("mysql adapter failed to ensure user privacy schema: " + err.Error())
}
} }
return err return err
} }
@@ -494,6 +497,10 @@ func (a *adapter) TenantAdapterFor(tenantID t.TenantID) (dbadapter.TenantBusines
db.Close() db.Close()
return nil, nil, err return nil, nil, err
} }
if err = ensureUserPrivacySchema(db); err != nil {
db.Close()
return nil, nil, errors.New("mysql adapter failed to ensure user privacy schema for tenant: " + err.Error())
}
registryDB := a.registryDB registryDB := a.registryDB
if registryDB == nil { if registryDB == nil {
registryDB = a.db registryDB = a.db
@@ -587,6 +594,11 @@ func createTenantRegistry(tx *sql.Tx, registryDB, businessDB string) error {
max_users BIGINT UNSIGNED NOT NULL DEFAULT 0 COMMENT '0 means unlimited', max_users BIGINT UNSIGNED NOT NULL DEFAULT 0 COMMENT '0 means unlimited',
max_groups BIGINT UNSIGNED NOT NULL DEFAULT 0 COMMENT '0 means unlimited', max_groups BIGINT UNSIGNED NOT NULL DEFAULT 0 COMMENT '0 means unlimited',
max_group_members INT UNSIGNED NOT NULL DEFAULT 0 COMMENT '0 means unlimited', max_group_members INT UNSIGNED NOT NULL DEFAULT 0 COMMENT '0 means unlimited',
realname_review_policy TINYINT(1) NOT NULL DEFAULT 0 COMMENT 'Real-name review policy: 0 none, 1 auto then manual',
realname_provider_mode VARCHAR(32) NOT NULL DEFAULT 'id_card_2factor' COMMENT 'id_card_2factor or smart_verify',
realname_required TINYINT(1) NOT NULL DEFAULT 0 COMMENT 'Require real-name verification',
realname_allow_skip TINYINT(1) NOT NULL DEFAULT 1 COMMENT 'Allow skipping real-name reminder',
realname_remind_after VARCHAR(32) NOT NULL DEFAULT '24h' COMMENT 'Real-name reminder interval',
aliyun_moderation_enabled TINYINT(1) NOT NULL DEFAULT 0 COMMENT 'Enable Aliyun content moderation', aliyun_moderation_enabled TINYINT(1) NOT NULL DEFAULT 0 COMMENT 'Enable Aliyun content moderation',
local_moderation_enabled TINYINT(1) NOT NULL DEFAULT 0 COMMENT 'Enable local content moderation', local_moderation_enabled TINYINT(1) NOT NULL DEFAULT 0 COMMENT 'Enable local content moderation',
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3), created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
@@ -844,495 +856,6 @@ func createIMCallTables(tx *sql.Tx) error {
return err return err
} }
func createIMWorkbenchTables(tx *sql.Tx) error {
if _, err := tx.Exec(
`CREATE TABLE im_workbench_apps(
id BIGINT NOT NULL AUTO_INCREMENT,
code VARCHAR(64) NOT NULL,
name VARCHAR(128) NOT NULL,
icon_file_id VARCHAR(64) NOT NULL DEFAULT '',
route VARCHAR(128) NOT NULL,
sort_order INT NOT NULL DEFAULT 0,
unread_key VARCHAR(64) NOT NULL DEFAULT '',
enabled TINYINT NOT NULL DEFAULT 1,
permission VARCHAR(64) NOT NULL DEFAULT 'enabled',
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_workbench_apps_code(code),
INDEX idx_im_workbench_apps_enabled_order(enabled, sort_order)
)`); err != nil {
return err
}
_, err := tx.Exec(
`INSERT INTO im_workbench_apps(code,name,route,sort_order,unread_key,enabled,permission)
VALUES
('attendance','打卡','/attendance',10,'attendance',1,'enabled'),
('monthly_reports','月报','/reports/monthly',20,'monthly_reports',1,'enabled'),
('daily_reports','日报','/reports/daily',30,'daily_reports',1,'enabled'),
('organization','部门框架','/organization',40,'organization',1,'enabled'),
('executive_mailbox','举报信箱','/executive-mailbox',50,'executive_mailbox',1,'enabled'),
('meetings','会议通知','/meetings',60,'meetings',1,'enabled'),
('meeting_notes','会议笔记','/meeting-notes',70,'',1,'enabled')
ON DUPLICATE KEY UPDATE
name=VALUES(name),
route=VALUES(route),
sort_order=VALUES(sort_order),
unread_key=VALUES(unread_key),
updated_at=CURRENT_TIMESTAMP(3)`)
return err
}
func createIMOrganizationTables(tx *sql.Tx) error {
statements := []string{
`CREATE TABLE im_org_departments(
id BIGINT NOT NULL AUTO_INCREMENT,
department_id VARCHAR(64) NOT NULL,
name VARCHAR(128) NOT NULL,
parent_id VARCHAR(64) NOT NULL DEFAULT 'root',
sort_order INT NOT NULL DEFAULT 0,
version INT NOT NULL DEFAULT 1,
state VARCHAR(16) NOT NULL DEFAULT 'active',
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_org_departments_department(department_id),
INDEX idx_im_org_departments_parent(parent_id, state, sort_order, id),
INDEX idx_im_org_departments_name(name)
)`,
`CREATE TABLE im_org_members(
id BIGINT NOT NULL AUTO_INCREMENT,
department_id VARCHAR(64) NOT NULL,
user_id BIGINT NOT NULL,
display_name VARCHAR(128) NOT NULL DEFAULT '',
position VARCHAR(128) NOT NULL DEFAULT '',
avatar_file_id VARCHAR(64) NOT NULL DEFAULT '',
mobile VARCHAR(32) NOT NULL DEFAULT '',
email VARCHAR(128) NOT NULL DEFAULT '',
employment_state VARCHAR(16) NOT NULL DEFAULT 'active',
contactable TINYINT NOT NULL DEFAULT 1,
sort_order INT NOT NULL DEFAULT 0,
version INT NOT NULL DEFAULT 1,
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_org_members_department_user(department_id, user_id),
INDEX idx_im_org_members_department(department_id, employment_state, sort_order, id),
INDEX idx_im_org_members_user(user_id),
INDEX idx_im_org_members_display_name(display_name),
CONSTRAINT fk_im_org_members_department FOREIGN KEY(department_id) REFERENCES im_org_departments(department_id) ON DELETE CASCADE
)`,
`CREATE TABLE im_org_visibility_overrides(
id BIGINT NOT NULL AUTO_INCREMENT,
user_id BIGINT NOT NULL,
visibility_scope VARCHAR(16) NOT NULL,
state VARCHAR(16) NOT NULL DEFAULT 'active',
remark VARCHAR(256) NOT NULL DEFAULT '',
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_org_visibility_user(user_id),
INDEX idx_im_org_visibility_scope(visibility_scope,state)
)`,
}
for _, stmt := range statements {
if _, err := tx.Exec(stmt); err != nil {
return err
}
}
return nil
}
func ensureIMOrganizationVisibilityTables(exec sqlExecutor) error {
_, err := exec.Exec(`CREATE TABLE IF NOT EXISTS im_org_visibility_overrides(
id BIGINT NOT NULL AUTO_INCREMENT,
user_id BIGINT NOT NULL,
visibility_scope VARCHAR(16) NOT NULL,
state VARCHAR(16) NOT NULL DEFAULT 'active',
remark VARCHAR(256) NOT NULL DEFAULT '',
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_org_visibility_user(user_id),
INDEX idx_im_org_visibility_scope(visibility_scope,state)
)`)
return err
}
func createIMAttendanceTables(tx *sql.Tx) error {
for _, stmt := range imAttendanceTableStatements(false) {
if _, err := tx.Exec(stmt); err != nil {
return err
}
}
return seedIMAttendanceDefaults(tx)
}
func ensureIMAttendanceTables(db *sqlx.DB) error {
tx, err := db.Begin()
if err != nil {
return err
}
defer tx.Rollback()
for _, stmt := range imAttendanceTableStatements(true) {
if _, err = tx.Exec(stmt); err != nil {
return err
}
}
if err = seedIMAttendanceDefaults(tx); err != nil {
return err
}
return tx.Commit()
}
func imAttendanceTableStatements(ifNotExists bool) []string {
statements := []string{
`CREATE TABLE im_attendance_rules(
id BIGINT NOT NULL AUTO_INCREMENT,
rule_id VARCHAR(64) NOT NULL,
name VARCHAR(128) NOT NULL,
workday TINYINT NOT NULL DEFAULT 1,
location_required TINYINT NOT NULL DEFAULT 0,
wifi_required TINYINT NOT NULL DEFAULT 0,
allow_outside_checkin TINYINT NOT NULL DEFAULT 1,
appeal_allowed TINYINT NOT NULL DEFAULT 1,
effective_from DATE,
effective_to DATE,
state VARCHAR(16) NOT NULL DEFAULT 'active',
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_attendance_rules_rule(rule_id),
INDEX idx_im_attendance_rules_state(state,effective_from,effective_to)
)`,
`CREATE TABLE im_attendance_shifts(
id BIGINT NOT NULL AUTO_INCREMENT,
shift_id VARCHAR(64) NOT NULL,
rule_id VARCHAR(64) NOT NULL,
name VARCHAR(128) NOT NULL,
work_start TIME NOT NULL,
work_end TIME NOT NULL,
checkin_start TIME NOT NULL,
checkout_end TIME NOT NULL,
state VARCHAR(16) NOT NULL DEFAULT 'active',
sort_order INT NOT NULL DEFAULT 0,
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_attendance_shifts_shift(shift_id),
INDEX idx_im_attendance_shifts_rule(rule_id,state,sort_order)
)`,
`CREATE TABLE im_attendance_records(
id BIGINT NOT NULL AUTO_INCREMENT,
record_id VARCHAR(64) NOT NULL,
user_id BIGINT NOT NULL,
record_date DATE NOT NULL,
rule_id VARCHAR(64) NOT NULL,
shift_id VARCHAR(64) NOT NULL,
status VARCHAR(32) NOT NULL DEFAULT 'not_checked',
exceptions JSON,
appeal_status VARCHAR(32) NOT NULL DEFAULT '',
version INT NOT NULL DEFAULT 1,
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_attendance_records_record(record_id),
UNIQUE INDEX uk_im_attendance_records_user_date_shift(user_id,record_date,shift_id),
INDEX idx_im_attendance_records_user_id(user_id,id DESC),
INDEX idx_im_attendance_records_date_status(record_date,status,id DESC)
)`,
`CREATE TABLE im_attendance_checkins(
id BIGINT NOT NULL AUTO_INCREMENT,
checkin_id VARCHAR(64) NOT NULL,
record_id VARCHAR(64) NOT NULL,
user_id BIGINT NOT NULL,
record_date DATE NOT NULL,
action VARCHAR(32) NOT NULL,
rule_id VARCHAR(64) NOT NULL,
shift_id VARCHAR(64) NOT NULL,
client_time DATETIME(3) NOT NULL,
server_time DATETIME(3) NOT NULL,
result VARCHAR(32) NOT NULL,
location JSON,
wifi JSON,
device JSON,
idempotency_key VARCHAR(128) NOT NULL,
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_attendance_checkins_checkin(checkin_id),
UNIQUE INDEX uk_im_attendance_checkins_idempotency(user_id,idempotency_key),
INDEX idx_im_attendance_checkins_record(record_id,id)
)`,
`CREATE TABLE im_attendance_appeals(
id BIGINT NOT NULL AUTO_INCREMENT,
appeal_id VARCHAR(64) NOT NULL,
record_id VARCHAR(64) NOT NULL,
user_id BIGINT NOT NULL,
appeal_type VARCHAR(64) NOT NULL,
reason VARCHAR(512) NOT NULL,
target_action VARCHAR(32) NOT NULL DEFAULT '',
target_time DATETIME(3),
attachment_file_ids JSON,
status VARCHAR(32) NOT NULL DEFAULT 'pending',
version INT NOT NULL DEFAULT 1,
idempotency_key VARCHAR(128) NOT NULL,
submitted_at DATETIME(3) NOT NULL,
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_attendance_appeals_appeal(appeal_id),
UNIQUE INDEX uk_im_attendance_appeals_idempotency(user_id,idempotency_key),
INDEX idx_im_attendance_appeals_record(record_id),
INDEX idx_im_attendance_appeals_user_status(user_id,status,id DESC)
)`,
}
if ifNotExists {
for i, stmt := range statements {
statements[i] = strings.Replace(stmt, "CREATE TABLE ", "CREATE TABLE IF NOT EXISTS ", 1)
}
}
return statements
}
func seedIMAttendanceDefaults(tx *sql.Tx) error {
if _, err := tx.Exec(`INSERT INTO im_attendance_rules(rule_id,name,workday,location_required,wifi_required,allow_outside_checkin,appeal_allowed)
VALUES('att_rule_default','默认考勤规则',1,0,0,1,1)
ON DUPLICATE KEY UPDATE updated_at=updated_at`); err != nil {
return err
}
_, err := tx.Exec(`INSERT INTO im_attendance_shifts(shift_id,rule_id,name,work_start,work_end,checkin_start,checkout_end,sort_order)
VALUES('shift_default','att_rule_default','默认班次','09:00:00','18:00:00','08:30:00','20:00:00',10)
ON DUPLICATE KEY UPDATE updated_at=updated_at`)
return err
}
func createIMMeetingTables(tx *sql.Tx) error {
statements := []string{
`CREATE TABLE im_meetings(
id BIGINT NOT NULL AUTO_INCREMENT,
meeting_id VARCHAR(64) NOT NULL,
organizer_id BIGINT NOT NULL,
recorder_user_id BIGINT NOT NULL,
title VARCHAR(200) NOT NULL,
start_at DATETIME(3) NOT NULL,
end_at DATETIME(3) NOT NULL,
location VARCHAR(256) NOT NULL DEFAULT '',
state VARCHAR(16) NOT NULL DEFAULT 'scheduled',
reminder_minutes JSON,
idempotency_key VARCHAR(128),
version INT NOT NULL DEFAULT 1,
canceled_at DATETIME(3),
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_meetings_meeting(meeting_id),
UNIQUE INDEX uk_im_meetings_idempotency(organizer_id, idempotency_key),
INDEX idx_im_meetings_organizer(organizer_id, start_at),
INDEX idx_im_meetings_recorder(recorder_user_id, end_at),
INDEX idx_im_meetings_state_time(state, start_at)
)`,
`CREATE TABLE im_meeting_participants(
id BIGINT NOT NULL AUTO_INCREMENT,
meeting_id VARCHAR(64) NOT NULL,
user_id BIGINT NOT NULL,
response VARCHAR(16) NOT NULL DEFAULT 'pending',
response_at DATETIME(3),
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_meeting_participants_user(meeting_id, user_id),
INDEX idx_im_meeting_participants_user_time(user_id, meeting_id),
INDEX idx_im_meeting_participants_response(meeting_id, response),
CONSTRAINT fk_im_meeting_participants_meeting FOREIGN KEY(meeting_id) REFERENCES im_meetings(meeting_id) ON DELETE CASCADE
)`,
`CREATE TABLE im_meeting_note_files(
id BIGINT NOT NULL AUTO_INCREMENT,
note_file_id VARCHAR(64) NOT NULL,
meeting_id VARCHAR(64) NOT NULL,
file_id VARCHAR(64) NOT NULL,
uploader_id BIGINT NOT NULL,
file_name VARCHAR(255) NOT NULL DEFAULT '',
mime_type VARCHAR(128) NOT NULL DEFAULT '',
file_size BIGINT NOT NULL DEFAULT 0,
file_etag VARCHAR(128) NOT NULL DEFAULT '',
status VARCHAR(16) NOT NULL DEFAULT 'active',
preview_status VARCHAR(16) NOT NULL DEFAULT 'ready',
preview_mode VARCHAR(16) NOT NULL DEFAULT 'native',
preview_file_id VARCHAR(64) NOT NULL DEFAULT '',
idempotency_key VARCHAR(128) NOT NULL DEFAULT '',
version INT NOT NULL DEFAULT 1,
deleted_by BIGINT,
deleted_at DATETIME(3),
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_meeting_note_files_id(note_file_id),
INDEX idx_im_meeting_note_files_file(meeting_id,file_id,status),
UNIQUE INDEX uk_im_meeting_note_files_idempotency(meeting_id,uploader_id,idempotency_key),
INDEX idx_im_meeting_note_files_meeting(meeting_id,status,id DESC),
INDEX idx_im_meeting_note_files_uploader(uploader_id,id DESC)
)`,
}
for _, stmt := range statements {
if _, err := tx.Exec(stmt); err != nil {
return err
}
}
return nil
}
func ensureIMMeetingNotesTables(db *sqlx.DB) error {
if db == nil {
return nil
}
var recorderExists bool
if err := db.QueryRow(`SELECT EXISTS(
SELECT 1 FROM information_schema.columns
WHERE table_schema=DATABASE() AND table_name='im_meetings' AND column_name='recorder_user_id'
)`).Scan(&recorderExists); err != nil {
return err
}
if !recorderExists {
if _, err := db.Exec(`ALTER TABLE im_meetings ADD COLUMN recorder_user_id BIGINT NOT NULL DEFAULT 0 AFTER organizer_id`); err != nil {
return err
}
if _, err := db.Exec(`UPDATE im_meetings SET recorder_user_id=organizer_id WHERE recorder_user_id=0`); err != nil {
return err
}
if _, err := db.Exec(`CREATE INDEX idx_im_meetings_recorder ON im_meetings(recorder_user_id,end_at)`); err != nil && !isDupe(err) {
return err
}
}
if _, err := db.Exec(`CREATE TABLE IF NOT EXISTS im_meeting_note_files(
id BIGINT NOT NULL AUTO_INCREMENT,
note_file_id VARCHAR(64) NOT NULL,
meeting_id VARCHAR(64) NOT NULL,
file_id VARCHAR(64) NOT NULL,
uploader_id BIGINT NOT NULL,
file_name VARCHAR(255) NOT NULL DEFAULT '',
mime_type VARCHAR(128) NOT NULL DEFAULT '',
file_size BIGINT NOT NULL DEFAULT 0,
file_etag VARCHAR(128) NOT NULL DEFAULT '',
status VARCHAR(16) NOT NULL DEFAULT 'active',
preview_status VARCHAR(16) NOT NULL DEFAULT 'ready',
preview_mode VARCHAR(16) NOT NULL DEFAULT 'native',
preview_file_id VARCHAR(64) NOT NULL DEFAULT '',
idempotency_key VARCHAR(128) NOT NULL DEFAULT '',
version INT NOT NULL DEFAULT 1,
deleted_by BIGINT,
deleted_at DATETIME(3),
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_meeting_note_files_id(note_file_id),
INDEX idx_im_meeting_note_files_file(meeting_id,file_id,status),
UNIQUE INDEX uk_im_meeting_note_files_idempotency(meeting_id,uploader_id,idempotency_key),
INDEX idx_im_meeting_note_files_meeting(meeting_id,status,id DESC),
INDEX idx_im_meeting_note_files_uploader(uploader_id,id DESC)
)`); err != nil {
return err
}
_, err := db.Exec(`INSERT INTO im_workbench_apps(code,name,route,sort_order,unread_key,enabled,permission)
VALUES('meeting_notes','会议笔记','/meeting-notes',70,'',1,'enabled')
ON DUPLICATE KEY UPDATE name=VALUES(name),route=VALUES(route),sort_order=VALUES(sort_order),unread_key=VALUES(unread_key),updated_at=CURRENT_TIMESTAMP(3)`)
return err
}
func createIMReportTables(tx *sql.Tx) error {
if _, err := tx.Exec(
`CREATE TABLE im_daily_reports(
id BIGINT NOT NULL AUTO_INCREMENT,
report_id VARCHAR(64) NOT NULL,
author_id BIGINT NOT NULL,
report_date DATE NOT NULL,
content TEXT NOT NULL,
attachments JSON,
status VARCHAR(16) NOT NULL DEFAULT 'draft',
read_state VARCHAR(16) NOT NULL DEFAULT 'unread',
version INT NOT NULL DEFAULT 1,
submitted_at DATETIME(3),
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_daily_reports_report(report_id),
UNIQUE INDEX uk_im_daily_reports_author_date(author_id, report_date),
INDEX idx_im_daily_reports_author_id(author_id, id DESC),
INDEX idx_im_daily_reports_date_status(report_date, status, id DESC)
)`); err != nil {
return err
}
_, err := tx.Exec(
`CREATE TABLE im_monthly_reports(
id BIGINT NOT NULL AUTO_INCREMENT,
report_id VARCHAR(64) NOT NULL,
author_id BIGINT NOT NULL,
report_month CHAR(7) NOT NULL,
work_summary TEXT NOT NULL,
next_plan TEXT NOT NULL,
attachments JSON,
status VARCHAR(16) NOT NULL DEFAULT 'draft',
read_state VARCHAR(16) NOT NULL DEFAULT 'unread',
version INT NOT NULL DEFAULT 1,
submitted_at DATETIME(3),
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_monthly_reports_report(report_id),
UNIQUE INDEX uk_im_monthly_reports_author_month(author_id, report_month),
INDEX idx_im_monthly_reports_author_id(author_id, id DESC),
INDEX idx_im_monthly_reports_month_status(report_month, status, id DESC)
)`)
return err
}
func createIMMailboxTables(tx *sql.Tx) error {
if _, err := tx.Exec(
`CREATE TABLE im_mailbox_config(
id BIGINT NOT NULL AUTO_INCREMENT,
enabled TINYINT NOT NULL DEFAULT 1,
allow_anonymous TINYINT NOT NULL DEFAULT 1,
required_fields JSON,
max_length INT NOT NULL DEFAULT 5000,
attachment_policy JSON,
notice_title VARCHAR(128) NOT NULL DEFAULT '',
notice_content TEXT NOT NULL,
notice_updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id)
)`); err != nil {
return err
}
if _, err := tx.Exec(
`INSERT INTO im_mailbox_config(id,enabled,allow_anonymous,required_fields,max_length,attachment_policy,notice_title,notice_content)
VALUES(1,1,1,'["subject","content"]',5000,'{"max_files":5,"purpose":"executive_mailbox"}','总经理信箱','请如实填写诉求,匿名提交将按企业政策保护展示身份。')
ON DUPLICATE KEY UPDATE updated_at=updated_at`); err != nil {
return err
}
_, err := tx.Exec(
`CREATE TABLE im_mailbox_submissions(
id BIGINT NOT NULL AUTO_INCREMENT,
submission_id VARCHAR(64) NOT NULL,
sender_id BIGINT NOT NULL,
subject VARCHAR(200) NOT NULL,
content TEXT NOT NULL,
anonymous TINYINT NOT NULL DEFAULT 0,
attachment_file_ids JSON,
notice_version VARCHAR(64) NOT NULL DEFAULT '',
idempotency_key VARCHAR(128) NOT NULL,
status VARCHAR(32) NOT NULL DEFAULT 'received',
submitted_at DATETIME(3) NOT NULL,
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),
UNIQUE INDEX uk_im_mailbox_submissions_submission(submission_id),
UNIQUE INDEX uk_im_mailbox_submissions_idempotency(sender_id, idempotency_key),
INDEX idx_im_mailbox_submissions_sender(sender_id, id DESC),
INDEX idx_im_mailbox_submissions_status(status, id DESC)
)`)
return err
}
func createIMUserProductTables(tx *sql.Tx) error { func createIMUserProductTables(tx *sql.Tx) error {
statements := []string{ statements := []string{
`CREATE TABLE im_user_profiles( `CREATE TABLE im_user_profiles(
@@ -1346,6 +869,9 @@ func createIMUserProductTables(tx *sql.Tx) error {
region VARCHAR(128) NOT NULL DEFAULT '', region VARCHAR(128) NOT NULL DEFAULT '',
email VARCHAR(128) NOT NULL DEFAULT '', email VARCHAR(128) NOT NULL DEFAULT '',
public_account VARCHAR(64) NOT NULL DEFAULT '', public_account VARCHAR(64) NOT NULL DEFAULT '',
friend_add_enabled TINYINT NOT NULL DEFAULT 1,
friend_add_disabled_reason VARCHAR(255) NOT NULL DEFAULT '',
friend_add_policy_version INT NOT NULL DEFAULT 1,
version INT NOT NULL DEFAULT 1, version INT NOT NULL DEFAULT 1,
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3), created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3), updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
@@ -1826,13 +1352,14 @@ func (a *adapter) UserCreate(user *t.User) error {
if err != nil { if err != nil {
return err return err
} }
insert := "INSERT INTO im_users(id,username,createdat,updatedat,state,access,phone,public,trusted) VALUES(?,?,?,?,?,?,?,?,?)" nickname, avatarFileID := profileSeedFromPublic(user.Public)
insert := "INSERT INTO im_users(id,username,createdat,updatedat,state,access,phone,trusted) VALUES(?,?,?,?,?,?,?,?)"
args := []any{decoded_uid, user.Username, user.CreatedAt, user.UpdatedAt, user.State, user.Access, phone, args := []any{decoded_uid, user.Username, user.CreatedAt, user.UpdatedAt, user.State, user.Access, phone,
common.ToJSON(user.Public), common.ToJSON(user.Trusted)} common.ToJSON(user.Trusted)}
if hasAccountNo { if hasAccountNo {
insert = "INSERT INTO im_users(id,account_no,username,createdat,updatedat,state,access,phone,public,trusted) VALUES(?,?,?,?,?,?,?,?,?,?)" insert = "INSERT INTO im_users(id,account_no,username,createdat,updatedat,state,access,phone,trusted) VALUES(?,?,?,?,?,?,?,?,?)"
args = []any{decoded_uid, accountNo, user.Username, user.CreatedAt, user.UpdatedAt, user.State, user.Access, phone, args = []any{decoded_uid, accountNo, user.Username, user.CreatedAt, user.UpdatedAt, user.State, user.Access, phone,
common.ToJSON(user.Public), common.ToJSON(user.Trusted)} common.ToJSON(user.Trusted)}
} }
if _, err = tx.Exec(insert, args...); err != nil { if _, err = tx.Exec(insert, args...); err != nil {
if isDupe(err) { if isDupe(err) {
@@ -1845,7 +1372,6 @@ func (a *adapter) UserCreate(user *t.User) error {
securityID, decoded_uid, 1, user.CreatedAt, user.UpdatedAt); err != nil { securityID, decoded_uid, 1, user.CreatedAt, user.UpdatedAt); err != nil {
return err return err
} }
nickname, avatarFileID := profileSeedFromPublic(user.Public)
if _, err = tx.Exec(`INSERT IGNORE INTO im_user_profiles(id,user_id,avatar_file_id,nickname,public_account,created_at,updated_at) if _, err = tx.Exec(`INSERT IGNORE INTO im_user_profiles(id,user_id,avatar_file_id,nickname,public_account,created_at,updated_at)
VALUES(?,?,?,?,?,?,?)`, profileID, decoded_uid, avatarFileID, nickname, user.Uid().UserId(), user.CreatedAt, user.UpdatedAt); err != nil { VALUES(?,?,?,?,?,?,?)`, profileID, decoded_uid, avatarFileID, nickname, user.Uid().UserId(), user.CreatedAt, user.UpdatedAt); err != nil {
return err return err
@@ -2163,7 +1689,7 @@ func (a *adapter) UserGet(uid t.Uid) (*t.User, error) {
} }
var row profileUserRow var row profileUserRow
err := a.db.GetContext(ctx, &row, `SELECT u.id,u.username,u.createdat,u.updatedat,u.state,u.stateat,u.access,u.lastseen,u.useragent, err := a.db.GetContext(ctx, &row, `SELECT u.id,u.username,u.createdat,u.updatedat,u.state,u.stateat,u.access,u.lastseen,u.useragent,
COALESCE(u.phone,'') AS phone,u.public,u.trusted, COALESCE(u.phone,'') AS phone,u.trusted,
p.nickname AS profile_nickname,p.avatar_file_id AS profile_avatar_file_id,p.signature AS profile_signature, p.nickname AS profile_nickname,p.avatar_file_id AS profile_avatar_file_id,p.signature AS profile_signature,
p.gender AS profile_gender,p.region AS profile_region,p.public_account AS profile_public_account,p.updated_at AS profile_updated_at p.gender AS profile_gender,p.region AS profile_region,p.public_account AS profile_public_account,p.updated_at AS profile_updated_at
FROM im_users AS u LEFT JOIN im_user_profiles AS p ON p.user_id=u.id FROM im_users AS u LEFT JOIN im_user_profiles AS p ON p.user_id=u.id
@@ -2197,7 +1723,7 @@ func (a *adapter) UserGetAll(ids ...t.Uid) ([]t.User, error) {
defer cancel() defer cancel()
} }
q, uids, _ := sqlx.In(`SELECT u.id,u.username,u.createdat,u.updatedat,u.state,u.stateat,u.access,u.lastseen,u.useragent, q, uids, _ := sqlx.In(`SELECT u.id,u.username,u.createdat,u.updatedat,u.state,u.stateat,u.access,u.lastseen,u.useragent,
COALESCE(u.phone,'') AS phone,u.public,u.trusted, COALESCE(u.phone,'') AS phone,u.trusted,
p.nickname AS profile_nickname,p.avatar_file_id AS profile_avatar_file_id,p.signature AS profile_signature, p.nickname AS profile_nickname,p.avatar_file_id AS profile_avatar_file_id,p.signature AS profile_signature,
p.gender AS profile_gender,p.region AS profile_region,p.public_account AS profile_public_account,p.updated_at AS profile_updated_at p.gender AS profile_gender,p.region AS profile_region,p.public_account AS profile_public_account,p.updated_at AS profile_updated_at
FROM im_users AS u LEFT JOIN im_user_profiles AS p ON p.user_id=u.id FROM im_users AS u LEFT JOIN im_user_profiles AS p ON p.user_id=u.id
@@ -2420,7 +1946,7 @@ func (a *adapter) UserUpdate(uid t.Uid, update map[string]any) error {
} }
}() }()
update = withoutUserTags(update) update = sanitizeUserUpdate(update)
cols, args := common.UpdateByMap(update) cols, args := common.UpdateByMap(update)
if len(cols) == 0 { if len(cols) == 0 {
return tx.Commit() return tx.Commit()
@@ -2961,7 +2487,7 @@ func (a *adapter) TopicsForUser(uid t.Uid, keepDeleted bool, opts *t.QueryOpt) (
// Fetch p2p users and join to p2p subscriptions. // Fetch p2p users and join to p2p subscriptions.
if len(usrq) > 0 { if len(usrq) > 0 {
q = `SELECT u.id,u.updatedat,u.state,u.access,u.lastseen,u.useragent,u.public,u.trusted, q = `SELECT u.id,u.updatedat,u.state,u.access,u.lastseen,u.useragent,u.trusted,
p.nickname AS profile_nickname,p.avatar_file_id AS profile_avatar_file_id,p.signature AS profile_signature, p.nickname AS profile_nickname,p.avatar_file_id AS profile_avatar_file_id,p.signature AS profile_signature,
p.gender AS profile_gender,p.region AS profile_region,p.public_account AS profile_public_account,p.updated_at AS profile_updated_at, p.gender AS profile_gender,p.region AS profile_region,p.public_account AS profile_public_account,p.updated_at AS profile_updated_at,
fs.remark AS friendship_remark fs.remark AS friendship_remark
@@ -3035,7 +2561,7 @@ func (a *adapter) UsersForTopic(topic string, keepDeleted bool, opts *t.QueryOpt
// Fetch all subscribed users. The number of users is not large. // Fetch all subscribed users. The number of users is not large.
q := `SELECT s.createdat,s.updatedat,s.deletedat,s.userid,s.topic,s.delid,s.recvseqid, q := `SELECT s.createdat,s.updatedat,s.deletedat,s.userid,s.topic,s.delid,s.recvseqid,
s.readseqid,s.modewant,s.modegiven,u.public,u.trusted,u.lastseen,u.useragent,s.private, s.readseqid,s.modewant,s.modegiven,u.trusted,u.lastseen,u.useragent,s.private,
p.nickname AS profile_nickname,p.avatar_file_id AS profile_avatar_file_id,p.signature AS profile_signature, p.nickname AS profile_nickname,p.avatar_file_id AS profile_avatar_file_id,p.signature AS profile_signature,
p.gender AS profile_gender,p.region AS profile_region,p.public_account AS profile_public_account p.gender AS profile_gender,p.region AS profile_region,p.public_account AS profile_public_account
FROM im_subscriptions AS s JOIN im_users AS u ON s.userid=u.id FROM im_subscriptions AS s JOIN im_users AS u ON s.userid=u.id
@@ -3091,14 +2617,14 @@ func (a *adapter) UsersForTopic(topic string, keepDeleted bool, opts *t.QueryOpt
var subs []t.Subscription var subs []t.Subscription
var lastSeen sql.NullTime var lastSeen sql.NullTime
var userAgent string var userAgent string
var public, trusted any var trusted any
var profileNickname, profileAvatarFileID, profileSignature, profileGender, profileRegion, profilePublicAccount sql.NullString var profileNickname, profileAvatarFileID, profileSignature, profileGender, profileRegion, profilePublicAccount sql.NullString
for rows.Next() { for rows.Next() {
if err = rows.Scan( if err = rows.Scan(
&sub.CreatedAt, &sub.UpdatedAt, &sub.DeletedAt, &sub.CreatedAt, &sub.UpdatedAt, &sub.DeletedAt,
&sub.User, &sub.Topic, &sub.DelId, &sub.RecvSeqId, &sub.User, &sub.Topic, &sub.DelId, &sub.RecvSeqId,
&sub.ReadSeqId, &sub.ModeWant, &sub.ModeGiven, &sub.ReadSeqId, &sub.ModeWant, &sub.ModeGiven,
&public, &trusted, &lastSeen, &userAgent, &sub.Private, &trusted, &lastSeen, &userAgent, &sub.Private,
&profileNickname, &profileAvatarFileID, &profileSignature, &profileNickname, &profileAvatarFileID, &profileSignature,
&profileGender, &profileRegion, &profilePublicAccount); err != nil { &profileGender, &profileRegion, &profilePublicAccount); err != nil {
break break
@@ -3106,7 +2632,7 @@ func (a *adapter) UsersForTopic(topic string, keepDeleted bool, opts *t.QueryOpt
sub.User = common.EncodeUidString(sub.User).String() sub.User = common.EncodeUidString(sub.User).String()
sub.Private = common.FromJSON(sub.Private) sub.Private = common.FromJSON(sub.Private)
sub.SetPublic(profilePublicFromFields(common.FromJSON(public), profileNickname, profileAvatarFileID, profileSignature, profileGender, profileRegion, profilePublicAccount)) sub.SetPublic(profilePublicFromFields(profileNickname, profileAvatarFileID, profileSignature, profileGender, profileRegion, profilePublicAccount))
sub.SetTrusted(common.FromJSON(trusted)) sub.SetTrusted(common.FromJSON(trusted))
sub.SetLastSeenAndUA(&lastSeen.Time, userAgent) sub.SetLastSeenAndUA(&lastSeen.Time, userAgent)
subs = append(subs, sub) subs = append(subs, sub)
@@ -3439,7 +2965,7 @@ func (a *adapter) SubsForTopic(topic string, keepDeleted bool, opts *t.QueryOpt)
// Filter out deleted rows. // Filter out deleted rows.
q += " AND deletedat IS NULL" q += " AND deletedat IS NULL"
} }
limit := a.maxResults limit := 0
if opts != nil { if opts != nil {
// Ignore IfModifiedSince - we must return all entries // Ignore IfModifiedSince - we must return all entries
// Those unmodified will be stripped of Public & Private. // Those unmodified will be stripped of Public & Private.
@@ -3448,13 +2974,16 @@ func (a *adapter) SubsForTopic(topic string, keepDeleted bool, opts *t.QueryOpt)
q += " AND userid=?" q += " AND userid=?"
args = append(args, store.DecodeUid(opts.User)) args = append(args, store.DecodeUid(opts.User))
} }
limit = a.maxResults
if opts.Limit > 0 && opts.Limit < limit { if opts.Limit > 0 && opts.Limit < limit {
limit = opts.Limit limit = opts.Limit
} }
} }
if limit > 0 {
q += " LIMIT ?" q += " LIMIT ?"
args = append(args, limit) args = append(args, limit)
}
ctx, cancel := a.getContext() ctx, cancel := a.getContext()
if cancel != nil { if cancel != nil {
@@ -3785,9 +3314,9 @@ func (a *adapter) MessageSave(msg *t.Message) error {
// Using a sequential ID provided by the database. // Using a sequential ID provided by the database.
clientMsgID := sql.NullString{String: msg.ClientMsgID, Valid: msg.ClientMsgID != ""} clientMsgID := sql.NullString{String: msg.ClientMsgID, Valid: msg.ClientMsgID != ""}
res, err := a.db.ExecContext(ctx, res, err := a.db.ExecContext(ctx,
"INSERT INTO im_messages(createdAt,updatedAt,seqid,topic,`from`,client_msg_id,version,head,content) VALUES(?,?,?,?,?,?,?,?,?)", "INSERT INTO im_messages(createdAt,updatedAt,seqid,topic,`from`,client_msg_id,version,head,content,content_nonce,key_version) VALUES(?,?,?,?,?,?,?,?,?,?,?)",
msg.CreatedAt, msg.UpdatedAt, msg.SeqId, msg.Topic, msg.CreatedAt, msg.UpdatedAt, msg.SeqId, msg.Topic,
store.DecodeUid(t.ParseUid(msg.From)), clientMsgID, messageVersion(msg.Version), msg.Head, common.ToJSON(msg.Content)) store.DecodeUid(t.ParseUid(msg.From)), clientMsgID, messageVersion(msg.Version), msg.Head, common.ToJSON(msg.Content), msg.ContentNonce, msg.KeyVersion)
if isDupe(err) { if isDupe(err) {
return t.ErrDuplicate return t.ErrDuplicate
} }
@@ -3840,7 +3369,7 @@ func (a *adapter) MessageGetAll(topic string, forUser t.Uid, opts *t.QueryOpt) (
rows, err := a.db.QueryxContext( rows, err := a.db.QueryxContext(
ctx, ctx,
"SELECT m.createdat,m.updatedat,m.deletedat,m.delid,m.seqid,m.topic,m.`from`,m.head,m.content"+ "SELECT m.createdat,m.updatedat,m.deletedat,m.delid,m.seqid,m.topic,m.`from`,m.head,m.content,m.content_nonce,m.key_version"+
" FROM im_messages AS m LEFT JOIN im_dellog AS d"+ " FROM im_messages AS m LEFT JOIN im_dellog AS d"+
" ON d.topic=m.topic AND m.seqid BETWEEN d.low AND d.hi-1 AND d.deletedfor=?"+ " ON d.topic=m.topic AND m.seqid BETWEEN d.low AND d.hi-1 AND d.deletedfor=?"+
" WHERE m.delid=0 AND m.topic=? "+seqIdConstraint+" AND d.deletedfor IS NULL"+ " WHERE m.delid=0 AND m.topic=? "+seqIdConstraint+" AND d.deletedfor IS NULL"+
@@ -4789,14 +4318,6 @@ func (a *adapter) FileDeleteUnused(olderThan time.Time, limit int) ([]string, er
var locations []string var locations []string
var ids []any var ids []any
for _, candidate := range candidates { for _, candidate := range candidates {
var noteRefs int
fileID := store.EncodeUid(int64(candidate.id)).String()
if err = tx.QueryRow(`SELECT COUNT(*) FROM im_meeting_note_files WHERE file_id=? AND status=? LIMIT 1`, fileID, t.MeetingNoteFileStatusActive).Scan(&noteRefs); err != nil {
break
}
if noteRefs > 0 {
continue
}
if candidate.location != "" { if candidate.location != "" {
locations = append(locations, candidate.location) locations = append(locations, candidate.location)
} }
@@ -4914,23 +4435,13 @@ func (a *adapter) FileUserHasAccess(fid string, uid t.Uid) (bool, error) {
defer cancel() defer cancel()
} }
var allowed int var allowed int
buildQuery := func(includeRecorder bool) (string, []any) { buildQuery := func() (string, []any) {
recorderClause := ""
if includeRecorder {
recorderClause = `
OR mtg.recorder_user_id=?`
}
args := []any{ args := []any{
store.DecodeUid(fileID), fileID.String32(), "%/" + fileID.String32(), t.UploadCompleted, store.DecodeUid(fileID), fileID.String32(), "%/" + fileID.String32(), t.UploadCompleted,
store.DecodeUid(uid), store.DecodeUid(uid), store.DecodeUid(uid), t.RelationshipFriend, store.DecodeUid(uid), store.DecodeUid(uid), store.DecodeUid(uid), t.RelationshipFriend,
store.DecodeUid(uid), store.DecodeUid(uid), t.GroupMemberStatusActive, store.DecodeUid(uid), store.DecodeUid(uid), store.DecodeUid(uid), store.DecodeUid(uid), t.GroupMemberStatusActive, store.DecodeUid(uid), store.DecodeUid(uid),
store.DecodeUid(uid), fid, fid, store.DecodeUid(uid), fid, fid,
fid, t.MeetingStateCanceled, store.DecodeUid(uid),
} }
if includeRecorder {
args = append(args, store.DecodeUid(uid))
}
args = append(args, store.DecodeUid(uid))
return ` return `
SELECT COUNT(*) FROM im_fileuploads AS fu SELECT COUNT(*) FROM im_fileuploads AS fu
WHERE (fu.id=? OR fu.location=? OR fu.location LIKE ?) AND fu.status=? AND ( WHERE (fu.id=? OR fu.location=? OR fu.location LIKE ?) AND fu.status=? AND (
@@ -4980,34 +4491,13 @@ func (a *adapter) FileUserHasAccess(fid string, uid t.Uid) (bool, error) {
WHERE (gf.file_id=? OR gf.file_id LIKE CONCAT(?,'.%')) AND gf.status='active' WHERE (gf.file_id=? OR gf.file_id LIKE CONCAT(?,'.%')) AND gf.status='active'
AND s.deletedat IS NULL AND INSTR(s.modewant,'R')>0 AND INSTR(s.modegiven,'R')>0 AND s.deletedat IS NULL AND INSTR(s.modewant,'R')>0 AND INSTR(s.modegiven,'R')>0
) )
OR EXISTS (
SELECT 1 FROM im_meeting_note_files AS mnf
INNER JOIN im_meetings AS mtg
ON mtg.meeting_id=mnf.meeting_id
WHERE mnf.file_id=? AND mnf.status='active'
AND mtg.state<>?
AND mtg.end_at<=CURRENT_TIMESTAMP(3)
AND (
mtg.organizer_id=?
` + recorderClause + `
OR EXISTS (
SELECT 1 FROM im_meeting_participants AS mp
WHERE mp.meeting_id=mtg.meeting_id AND mp.user_id=?
)
)
)
) LIMIT 1`, args ) LIMIT 1`, args
} }
includeRecorder := true
run := func() error { run := func() error {
query, args := buildQuery(includeRecorder) query, args := buildQuery()
return a.db.GetContext(ctx, &allowed, query, args...) return a.db.GetContext(ctx, &allowed, query, args...)
} }
err := run() err := run()
if isUnknownColumn(err) && strings.Contains(strings.ToLower(err.Error()), "recorder_user_id") {
includeRecorder = false
err = run()
}
for attempt := 0; isRetryableMySQLReadErr(err) && attempt < 2; attempt++ { for attempt := 0; isRetryableMySQLReadErr(err) && attempt < 2; attempt++ {
time.Sleep(time.Duration(attempt+1) * 25 * time.Millisecond) time.Sleep(time.Duration(attempt+1) * 25 * time.Millisecond)
err = run() err = run()
@@ -38,6 +38,16 @@ func ensureIMAdminGovernanceTables(exec sqlExecutor) error {
INDEX idx_im_login_failure_type_time(failure_type, attempt_at DESC, id DESC), INDEX idx_im_login_failure_type_time(failure_type, attempt_at DESC, id DESC),
INDEX idx_im_login_failure_request(request_id) INDEX idx_im_login_failure_request(request_id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci`, ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci`,
`CREATE TABLE IF NOT EXISTS im_login_lockouts (
id BIGINT NOT NULL, user_id BIGINT,
account_hash CHAR(64) NOT NULL,
failed_attempts INT NOT NULL DEFAULT 0,
window_started_at DATETIME(3), locked_until DATETIME(3), last_attempt_at DATETIME(3),
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id), UNIQUE INDEX uk_im_login_lockouts_account(account_hash),
INDEX idx_im_login_lockouts_user(user_id), INDEX idx_im_login_lockouts_locked(locked_until)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci`,
`CREATE TABLE IF NOT EXISTS im_message_admin_actions ( `CREATE TABLE IF NOT EXISTS im_message_admin_actions (
id BIGINT NOT NULL AUTO_INCREMENT, action_id VARCHAR(96) NOT NULL, id BIGINT NOT NULL AUTO_INCREMENT, action_id VARCHAR(96) NOT NULL,
command_id VARCHAR(96) NOT NULL, conversation_type VARCHAR(16) NOT NULL, command_id VARCHAR(96) NOT NULL, conversation_type VARCHAR(16) NOT NULL,
@@ -13,11 +13,11 @@ func TestEnsureIMAdminGovernanceTablesUsesSelectedTenantDatabase(t *testing.T) {
if err := ensureIMAdminGovernanceTables(exec); err != nil { if err := ensureIMAdminGovernanceTables(exec); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if len(exec.statements) != 4 { if len(exec.statements) != 5 {
t.Fatalf("created %d admin governance tables, want 4", len(exec.statements)) t.Fatalf("created %d admin governance tables, want 5", len(exec.statements))
} }
joined := strings.ToLower(strings.Join(exec.statements, "\n")) joined := strings.ToLower(strings.Join(exec.statements, "\n"))
for _, table := range []string{"im_user_login_logs", "im_login_failure_logs", "im_message_admin_actions", "im_user_ip_whitelists"} { for _, table := range []string{"im_user_login_logs", "im_login_failure_logs", "im_login_lockouts", "im_message_admin_actions", "im_user_ip_whitelists"} {
if !strings.Contains(joined, table) { if !strings.Contains(joined, table) {
t.Fatalf("missing table %s", table) t.Fatalf("missing table %s", table)
} }
+2 -1
View File
@@ -97,7 +97,8 @@ func ensureGroupMessageShard(exec sqlExecutor, start time.Time, status string) e
message_id BIGINT NOT NULL, topic VARCHAR(64) NOT NULL, group_id BIGINT NOT NULL, message_id BIGINT NOT NULL, topic VARCHAR(64) NOT NULL, group_id BIGINT NOT NULL,
seqid BIGINT NOT NULL, from_user BIGINT NOT NULL, client_msg_id VARCHAR(64), seqid BIGINT NOT NULL, from_user BIGINT NOT NULL, client_msg_id VARCHAR(64),
msg_type VARCHAR(32) NOT NULL DEFAULT 'text', status VARCHAR(32) NOT NULL DEFAULT 'normal', msg_type VARCHAR(32) NOT NULL DEFAULT 'text', status VARCHAR(32) NOT NULL DEFAULT 'normal',
content JSON, head JSON, mentioned_user_ids JSON, mention_all TINYINT(1) NOT NULL DEFAULT 0, content JSON, content_nonce VARBINARY(12), key_version INT NOT NULL DEFAULT 0,
head JSON, mentioned_user_ids JSON, mention_all TINYINT(1) NOT NULL DEFAULT 0,
reply_seqid BIGINT, root_seqid BIGINT, file_count INT NOT NULL DEFAULT 0, reply_seqid BIGINT, root_seqid BIGINT, file_count INT NOT NULL DEFAULT 0,
content_size INT NOT NULL DEFAULT 0, has_risk TINYINT(1) NOT NULL DEFAULT 0, content_size INT NOT NULL DEFAULT 0, has_risk TINYINT(1) NOT NULL DEFAULT 0,
audit_status VARCHAR(32) NOT NULL DEFAULT 'pass', version INT NOT NULL DEFAULT 1, audit_status VARCHAR(32) NOT NULL DEFAULT 'pass', version INT NOT NULL DEFAULT 1,
+2 -1
View File
@@ -27,7 +27,8 @@ func ensureIMGroupBusinessTables(exec sqlExecutor) error {
id BIGINT NOT NULL AUTO_INCREMENT, topic VARCHAR(64) NOT NULL, group_id BIGINT NOT NULL, id BIGINT NOT NULL AUTO_INCREMENT, topic VARCHAR(64) NOT NULL, group_id BIGINT NOT NULL,
seqid BIGINT NOT NULL, from_user BIGINT NOT NULL, client_msg_id VARCHAR(64), seqid BIGINT NOT NULL, from_user BIGINT NOT NULL, client_msg_id VARCHAR(64),
msg_type VARCHAR(32) NOT NULL DEFAULT 'text', status VARCHAR(32) NOT NULL DEFAULT 'normal', msg_type VARCHAR(32) NOT NULL DEFAULT 'text', status VARCHAR(32) NOT NULL DEFAULT 'normal',
content JSON, head JSON, mentioned_user_ids JSON, mention_all TINYINT(1) NOT NULL DEFAULT 0, content JSON, content_nonce VARBINARY(12), key_version INT NOT NULL DEFAULT 0,
head JSON, mentioned_user_ids JSON, mention_all TINYINT(1) NOT NULL DEFAULT 0,
reply_seqid BIGINT, root_seqid BIGINT, file_count INT NOT NULL DEFAULT 0, reply_seqid BIGINT, root_seqid BIGINT, file_count INT NOT NULL DEFAULT 0,
content_size INT NOT NULL DEFAULT 0, has_risk TINYINT(1) NOT NULL DEFAULT 0, content_size INT NOT NULL DEFAULT 0, has_risk TINYINT(1) NOT NULL DEFAULT 0,
audit_status VARCHAR(32) NOT NULL DEFAULT 'pass', version INT NOT NULL DEFAULT 1, audit_status VARCHAR(32) NOT NULL DEFAULT 'pass', version INT NOT NULL DEFAULT 1,
+206
View File
@@ -0,0 +1,206 @@
//go:build mysql
// +build mysql
package mysql
import (
"database/sql"
"errors"
"time"
t "github.com/tinode/chat/server/store/types"
)
func (a *adapter) routedLoginLockoutAdapter(id t.TenantID) (*adapter, error) {
if err := validTenant(id); err != nil {
return nil, err
}
if a.tenantScopedDB {
return a, nil
}
tenantAdp, err := a.routedTenantAdapter(id)
if err != nil {
return nil, err
}
return tenantAdp, nil
}
type loginLockoutRow struct {
ID int64 `db:"id"`
UserID sql.NullInt64 `db:"user_id"`
AccountHash string `db:"account_hash"`
FailedAttempts int `db:"failed_attempts"`
WindowStartAt sql.NullTime `db:"window_started_at"`
LockedUntil sql.NullTime `db:"locked_until"`
LastAttemptAt sql.NullTime `db:"last_attempt_at"`
UpdatedAt sql.NullTime `db:"updated_at"`
}
func loginLockoutFromRow(id t.TenantID, row loginLockoutRow) *t.LoginLockout {
out := &t.LoginLockout{
TenantID: id,
ID: row.ID,
AccountHash: row.AccountHash,
FailedAttempts: row.FailedAttempts,
}
if row.UserID.Valid {
out.UserID = row.UserID.Int64
}
if row.WindowStartAt.Valid {
v := row.WindowStartAt.Time
out.WindowStartAt = &v
}
if row.LockedUntil.Valid {
v := row.LockedUntil.Time
out.LockedUntil = &v
}
if row.LastAttemptAt.Valid {
v := row.LastAttemptAt.Time
out.LastAttemptAt = &v
}
if row.UpdatedAt.Valid {
v := row.UpdatedAt.Time
out.UpdatedAt = &v
}
return out
}
const loginLockoutColumns = `id,user_id,account_hash,failed_attempts,window_started_at,locked_until,last_attempt_at,updated_at`
// TenantLoginLockoutGet reads the lockout counter for the given account hash.
func (a *adapter) TenantLoginLockoutGet(id t.TenantID, accountHash string) (*t.LoginLockout, error) {
tenantAdp, err := a.routedLoginLockoutAdapter(id)
if err != nil {
return nil, err
}
if tenantAdp != a {
return tenantAdp.TenantLoginLockoutGet(id, accountHash)
}
if accountHash == "" {
return nil, t.ErrMalformed
}
ctx, cancel := a.getContext()
if cancel != nil {
defer cancel()
}
var row loginLockoutRow
err = a.db.GetContext(ctx, &row,
`SELECT `+loginLockoutColumns+` FROM im_login_lockouts WHERE account_hash=?`, accountHash)
if errors.Is(err, sql.ErrNoRows) {
return nil, t.ErrNotFound
}
if err != nil {
return nil, err
}
return loginLockoutFromRow(id, row), nil
}
// TenantLoginLockoutRecordFailure atomically inserts or increments the failure
// counter. windowCutoff is "now minus window seconds": an older window start
// resets the counter to 1. The updated row is returned to the caller.
func (a *adapter) TenantLoginLockoutRecordFailure(id t.TenantID, accountHash string, userID *int64, windowCutoff, now time.Time) (*t.LoginLockout, error) {
tenantAdp, err := a.routedLoginLockoutAdapter(id)
if err != nil {
return nil, err
}
if tenantAdp != a {
return tenantAdp.TenantLoginLockoutRecordFailure(id, accountHash, userID, windowCutoff, now)
}
if accountHash == "" || now.IsZero() {
return nil, t.ErrMalformed
}
ctx, cancel := a.getContext()
if cancel != nil {
defer cancel()
}
rowID, err := nextSnowflakeID()
if err != nil {
return nil, err
}
// MySQL evaluates ON DUPLICATE KEY UPDATE assignments left to right, so
// failed_attempts is recomputed before window_started_at is overwritten.
_, err = a.db.ExecContext(ctx, `INSERT INTO im_login_lockouts
(id,user_id,account_hash,failed_attempts,window_started_at,last_attempt_at)
VALUES(?,?,?,1,?,?)
ON DUPLICATE KEY UPDATE
failed_attempts = IF(window_started_at IS NULL OR window_started_at < ?, 1, failed_attempts + 1),
window_started_at = IF(window_started_at IS NULL OR window_started_at < ?, ?, window_started_at),
last_attempt_at = ?,
user_id = COALESCE(user_id, ?)`,
rowID, userID, accountHash, now, now,
windowCutoff, windowCutoff, now, now, userID)
if err != nil {
return nil, err
}
var row loginLockoutRow
err = a.db.GetContext(ctx, &row,
`SELECT `+loginLockoutColumns+` FROM im_login_lockouts WHERE account_hash=?`, accountHash)
if err != nil {
return nil, err
}
return loginLockoutFromRow(id, row), nil
}
// TenantLoginLockoutLock sets locked_until unless a newer lock already covers now.
func (a *adapter) TenantLoginLockoutLock(id t.TenantID, accountHash string, lockedUntil, now time.Time) error {
tenantAdp, err := a.routedLoginLockoutAdapter(id)
if err != nil {
return err
}
if tenantAdp != a {
return tenantAdp.TenantLoginLockoutLock(id, accountHash, lockedUntil, now)
}
if accountHash == "" || lockedUntil.IsZero() || now.IsZero() {
return t.ErrMalformed
}
ctx, cancel := a.getContext()
if cancel != nil {
defer cancel()
}
_, err = a.db.ExecContext(ctx,
`UPDATE im_login_lockouts SET locked_until=?, updated_at=? WHERE account_hash=? AND (locked_until IS NULL OR locked_until < ?)`,
lockedUntil, now, accountHash, now)
return err
}
// TenantLoginLockoutReset removes the counter after a successful login.
func (a *adapter) TenantLoginLockoutReset(id t.TenantID, accountHash string) error {
tenantAdp, err := a.routedLoginLockoutAdapter(id)
if err != nil {
return err
}
if tenantAdp != a {
return tenantAdp.TenantLoginLockoutReset(id, accountHash)
}
if accountHash == "" {
return t.ErrMalformed
}
ctx, cancel := a.getContext()
if cancel != nil {
defer cancel()
}
_, err = a.db.ExecContext(ctx, `DELETE FROM im_login_lockouts WHERE account_hash=?`, accountHash)
return err
}
// TenantLoginLockoutRelease clears the lock and zeroes the counter (admin unlock).
func (a *adapter) TenantLoginLockoutRelease(id t.TenantID, accountHash string, now time.Time) error {
tenantAdp, err := a.routedLoginLockoutAdapter(id)
if err != nil {
return err
}
if tenantAdp != a {
return tenantAdp.TenantLoginLockoutRelease(id, accountHash, now)
}
if accountHash == "" || now.IsZero() {
return t.ErrMalformed
}
ctx, cancel := a.getContext()
if cancel != nil {
defer cancel()
}
_, err = a.db.ExecContext(ctx,
`UPDATE im_login_lockouts SET failed_attempts=0, window_started_at=NULL, locked_until=NULL, updated_at=? WHERE account_hash=?`,
now, accountHash)
return err
}
+36
View File
@@ -0,0 +1,36 @@
//go:build mysql
// +build mysql
package mysql
import (
"context"
"database/sql"
"errors"
"github.com/jmoiron/sqlx"
)
// acquireMigrationLock wraps SELECT GET_LOCK so that schema migrations keep
// working on Percona XtraDB Cluster where pxc_strict_mode=ENFORCING prohibits
// named locks: instead of failing, locked=false is returned and the caller
// proceeds without the lock. This is safe because Galera's total order
// isolation serializes DDL across the cluster, and every migration using this
// helper is idempotent: steps are guarded by existence checks, so concurrent
// runs at worst fail with duplicate-key errors and are retried on the next
// startup.
func acquireMigrationLock(ctx context.Context, conn *sqlx.Conn, lockExpr string) (locked bool, unlock func(), err error) {
var held sql.NullInt64
if err = conn.QueryRowContext(ctx, "SELECT GET_LOCK("+lockExpr+", 60)").Scan(&held); err != nil {
if isPXCStrictModeNamedLockError(err) {
return false, nil, nil
}
return false, nil, err
}
if !held.Valid || held.Int64 != 1 {
return false, nil, errors.New("mysql: timed out waiting for schema migration lock")
}
return true, func() {
conn.ExecContext(ctx, "SELECT RELEASE_LOCK("+lockExpr+")")
}, nil
}
+57
View File
@@ -0,0 +1,57 @@
//go:build mysql
// +build mysql
package mysql
import (
"database/sql"
"errors"
t "github.com/tinode/chat/server/store/types"
)
func (a *adapter) RealNamePolicy(tenantID t.TenantID) (*t.RealNamePolicy, error) {
if !tenantID.IsValid() {
return nil, t.ErrMalformed
}
db := a.registryDB
if db == nil {
db = a.db
}
if db == nil {
return nil, t.ErrInternal
}
ctx, cancel := a.getContext()
if cancel != nil {
defer cancel()
}
var policy t.RealNamePolicy
err := db.GetContext(ctx, &policy, `SELECT tenant_id, realname_review_policy, realname_provider_mode,
realname_required, realname_allow_skip, realname_remind_after
FROM im_tenant_config WHERE tenant_id=? LIMIT 1`, tenantID)
if errors.Is(err, sql.ErrNoRows) {
policy = *defaultRealNamePolicy(tenantID)
return &policy, nil
}
if isUnknownColumn(err) {
policy = *defaultRealNamePolicy(tenantID)
return &policy, nil
}
if err != nil {
return nil, err
}
policy.Normalize()
return &policy, nil
}
func defaultRealNamePolicy(tenantID t.TenantID) *t.RealNamePolicy {
policy := &t.RealNamePolicy{
TenantID: tenantID,
ReviewPolicy: t.RealNameReviewPolicyNone,
ProviderMode: "id_card_2factor",
AllowSkip: true,
RemindAfter: "24h",
}
policy.Normalize()
return policy
}
+22 -27
View File
@@ -10,31 +10,6 @@ import (
"github.com/tinode/chat/server/store/types" "github.com/tinode/chat/server/store/types"
) )
func TestCanUseManagedReportsRequiresAllOrganizationVisibility(t *testing.T) {
if !canUseManagedReports(organizationVisibility{Scope: orgVisibilityAll}) {
t.Fatal("all visibility should be allowed to use managed report scope")
}
for _, scope := range []string{orgVisibilitySelf, orgVisibilityFriends, ""} {
if canUseManagedReports(organizationVisibility{Scope: scope}) {
t.Fatalf("scope %q should not be allowed to use managed report scope", scope)
}
}
}
func TestReportMemberScopeWhereIncludesDepartmentAndVisibility(t *testing.T) {
where, args := reportMemberScopeWhere(organizationVisibility{Scope: orgVisibilityFriends, UserID: 42}, "dep_ops")
joined := strings.Join(where, " AND ")
if !strings.Contains(joined, "m.department_id=?") {
t.Fatalf("department filter missing from where: %s", joined)
}
if !strings.Contains(joined, "im_friendships") {
t.Fatalf("friend visibility filter missing from where: %s", joined)
}
if len(args) != 4 || args[0] != "dep_ops" || args[1] != int64(42) || args[2] != int64(42) || args[3] != types.RelationshipFriend {
t.Fatalf("unexpected args: %#v", args)
}
}
func TestGroupAdminAccessModeDoesNotGrantOwner(t *testing.T) { func TestGroupAdminAccessModeDoesNotGrantOwner(t *testing.T) {
mode := groupAdminAccessMode() mode := groupAdminAccessMode()
if !mode.IsAdmin() { if !mode.IsAdmin() {
@@ -86,8 +61,6 @@ func TestRuntimeOpenDoesNotAutoCreateOrMigrateTables(t *testing.T) {
"ensureIMAdminGovernanceTables", "ensureIMAdminGovernanceTables",
"ensureIMFileReviewTables", "ensureIMFileReviewTables",
"ensureIMOrganizationVisibilityTables", "ensureIMOrganizationVisibilityTables",
"ensureIMAttendanceTables",
"ensureIMMeetingNotesTables",
"ensureSnowflakePrimaryKeys", "ensureSnowflakePrimaryKeys",
"ensureUserIdentitySchema", "ensureUserIdentitySchema",
"ensureIMRegistrationInviteUseTable", "ensureIMRegistrationInviteUseTable",
@@ -156,6 +129,28 @@ func TestSearchPaginationCursorUsesLastReturnedRow(t *testing.T) {
} }
} }
func TestMessageSearchTextMatchesStructuredContent(t *testing.T) {
msg := types.Message{
Content: types.KVMap{
"file": map[string]any{
"file_name": "项目方案.pdf",
},
"link": map[string]any{
"title": "季度复盘",
"summary": "增长分析",
},
},
}
for _, query := range []string{"项目方案", "季度", "增长 分析"} {
if !messageMatchesSearch(msg, query) {
t.Fatalf("structured message should match query %q, text=%q", query, messageSearchText(msg.Content))
}
}
if messageMatchesSearch(msg, "不存在") {
t.Fatal("structured message matched unrelated query")
}
}
func TestTenantFileAccessIncludesAvatarAudienceRules(t *testing.T) { func TestTenantFileAccessIncludesAvatarAudienceRules(t *testing.T) {
source, err := os.ReadFile("tenant_adapter.go") source, err := os.ReadFile("tenant_adapter.go")
if err != nil { if err != nil {
+7 -6
View File
@@ -58,15 +58,16 @@ func ensureSnowflakePrimaryKeys(db *sqlx.DB) error {
} }
defer conn.Close() defer conn.Close()
var locked sql.NullInt64 // PXC strict-mode clusters prohibit GET_LOCK; acquireMigrationLock falls back
if err = conn.QueryRowContext(ctx, // to lock-free execution in that case. migrateSnowflakePrimaryKey checks the
"SELECT GET_LOCK("+snowflakeMigrationLock+", 60)").Scan(&locked); err != nil { // current state of every table before touching it, so it is idempotent.
_, unlock, err := acquireMigrationLock(ctx, conn, snowflakeMigrationLock)
if err != nil {
return err return err
} }
if !locked.Valid || locked.Int64 != 1 { if unlock != nil {
return errors.New("mysql: timed out waiting for snowflake primary-key migration lock") defer unlock()
} }
defer conn.ExecContext(ctx, "SELECT RELEASE_LOCK("+snowflakeMigrationLock+")")
for _, spec := range snowflakePrimaryKeySpecs { for _, spec := range snowflakePrimaryKeySpecs {
if err = migrateSnowflakePrimaryKey(ctx, conn, spec); err != nil { if err = migrateSnowflakePrimaryKey(ctx, conn, spec); err != nil {
File diff suppressed because it is too large Load Diff
+3 -7
View File
@@ -322,11 +322,8 @@ func (a *adapter) TenantGroupMemberList(id t.TenantID, topic string, opts t.Grou
if opts.Query != "" { if opts.Query != "" {
like := "%" + opts.Query + "%" like := "%" + opts.Query + "%"
where = append(where, `(gm.nickname LIKE ? OR p.nickname LIKE ? OR p.public_account LIKE ? where = append(where, `(gm.nickname LIKE ? OR p.nickname LIKE ? OR p.public_account LIKE ?
OR JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.fn')) LIKE ?
OR JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.name')) LIKE ?
OR JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.display_name')) LIKE ?
OR u.username LIKE ?)`) OR u.username LIKE ?)`)
args = append(args, like, like, like, like, like, like, like) args = append(args, like, like, like, like)
} }
if opts.Cursor != "" { if opts.Cursor != "" {
parsed, err := strconv.ParseInt(opts.Cursor, 10, 64) parsed, err := strconv.ParseInt(opts.Cursor, 10, 64)
@@ -346,7 +343,7 @@ func (a *adapter) TenantGroupMemberList(id t.TenantID, topic string, opts t.Grou
if cancel != nil { if cancel != nil {
defer cancel() defer cancel()
} }
rows, err := a.db.QueryxContext(ctx, `SELECT STRAIGHT_JOIN gm.*,s.modewant,s.modegiven,u.public,u.trusted, rows, err := a.db.QueryxContext(ctx, `SELECT STRAIGHT_JOIN gm.*,s.modewant,s.modegiven,u.trusted,
p.nickname AS profile_nickname,p.avatar_file_id AS profile_avatar_file_id,p.signature AS profile_signature, p.nickname AS profile_nickname,p.avatar_file_id AS profile_avatar_file_id,p.signature AS profile_signature,
p.gender AS profile_gender,p.region AS profile_region,p.public_account AS profile_public_account p.gender AS profile_gender,p.region AS profile_region,p.public_account AS profile_public_account
FROM im_group_members AS gm FORCE INDEX (`+memberIndex+`) FROM im_group_members AS gm FORCE INDEX (`+memberIndex+`)
@@ -363,7 +360,6 @@ func (a *adapter) TenantGroupMemberList(id t.TenantID, topic string, opts t.Grou
for rows.Next() { for rows.Next() {
row := struct { row := struct {
t.GroupMember t.GroupMember
Public any
Trusted any Trusted any
ProfileNickname sql.NullString `db:"profile_nickname"` ProfileNickname sql.NullString `db:"profile_nickname"`
ProfileAvatarFileID sql.NullString `db:"profile_avatar_file_id"` ProfileAvatarFileID sql.NullString `db:"profile_avatar_file_id"`
@@ -383,7 +379,7 @@ func (a *adapter) TenantGroupMemberList(id t.TenantID, topic string, opts t.Grou
} }
continue continue
} }
item.Public = profilePublicFromFields(common.FromJSON(row.Public), row.ProfileNickname, row.ProfileAvatarFileID, row.ProfileSignature, row.ProfileGender, row.ProfileRegion, row.ProfilePublicAccount) item.Public = profilePublicFromFields(row.ProfileNickname, row.ProfileAvatarFileID, row.ProfileSignature, row.ProfileGender, row.ProfileRegion, row.ProfilePublicAccount)
item.Trusted = common.FromJSON(row.Trusted) item.Trusted = common.FromJSON(row.Trusted)
items = append(items, item) items = append(items, item)
} }
+16 -8
View File
@@ -289,8 +289,8 @@ func TestTenantAuthIsolation(t *testing.T) {
userBID := userAID + 1 userBID := userAID + 1
for index, userID := range []int64{userAID, userBID} { for index, userID := range []int64{userAID, userBID} {
tenantDB := quoteIdent(dbNames[index]) tenantDB := quoteIdent(dbNames[index])
if _, err := db.Exec("INSERT INTO "+tenantDB+".im_users(id,account_no,username,createdat,updatedat,state,access,phone,public,trusted) "+ if _, err := db.Exec("INSERT INTO "+tenantDB+".im_users(id,account_no,username,createdat,updatedat,state,access,phone,trusted) "+
"VALUES(?,?,?,?,?,?,?,?,?,?)", userID, 1000+index, "alice", now, now, types.StateOK, `{}`, "13800000000", nil, nil); err != nil { "VALUES(?,?,?,?,?,?,?,?,?)", userID, 1000+index, "alice", now, now, types.StateOK, `{}`, "13800000000", nil); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if _, err := db.Exec("INSERT INTO "+tenantDB+".im_auth(userid,authlvl,secret) VALUES(?,?,?)", if _, err := db.Exec("INSERT INTO "+tenantDB+".im_auth(userid,authlvl,secret) VALUES(?,?,?)",
@@ -354,17 +354,25 @@ func TestTenantUserSearchMatchesUsernamePhoneAndPublicAccount(t *testing.T) {
id int64 id int64
username string username string
phone any phone any
public string nickname string
}{ }{
{viewerID, "viewerx", nil, `{"fn":"Viewer"}`}, {viewerID, "viewerx", nil, "Viewer"},
{targetID, "potter", "+8615500000002", `{"fn":"Potter"}`}, {targetID, "potter", "+8615500000002", "Potter"},
} { } {
if _, err := db.Exec( if _, err := db.Exec(
`INSERT INTO im_users(id,account_no,username,createdat,updatedat,state,access,phone,public,trusted) `INSERT INTO im_users(id,account_no,username,createdat,updatedat,state,access,phone,trusted)
VALUES(?,?,?,?,?,?,?,?,?,?)`, VALUES(?,?,?,?,?,?,?,?,?)`,
user.id, user.id%1000000, user.username, now, now, types.StateOK, `{"Auth":"JRWPAS","Anon":"N"}`, user.phone, user.public, `{}`); err != nil { user.id, user.id%1000000, user.username, now, now, types.StateOK, `{"Auth":"JRWPAS","Anon":"N"}`, user.phone, `{}`); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if user.id != targetID {
if _, err := db.Exec(
`INSERT INTO im_user_profiles(id,user_id,nickname,public_account,created_at,updated_at)
VALUES(?,?,?,?,?,?)`,
user.id+2000, user.id, user.nickname, store.EncodeUid(user.id).UserId(), now, now); err != nil {
t.Fatal(err)
}
}
if _, err := db.Exec( if _, err := db.Exec(
`INSERT INTO im_user_privacy_settings(id,user_id,add_me_methods,created_at,updated_at) `INSERT INTO im_user_privacy_settings(id,user_id,add_me_methods,created_at,updated_at)
VALUES(?,?,?,?,?)`, VALUES(?,?,?,?,?)`,
+52 -97
View File
@@ -5,7 +5,6 @@ package mysql
import ( import (
"context" "context"
"database/sql"
"errors" "errors"
"fmt" "fmt"
"strings" "strings"
@@ -32,14 +31,35 @@ func ensureUserIdentitySchema(db *sqlx.DB) error {
} }
defer conn.Close() defer conn.Close()
var locked sql.NullInt64 // Fast path: the last two steps of this migration drop the legacy im_usertags
if err = conn.QueryRowContext(ctx, "SELECT GET_LOCK("+userIdentityMigrationLock+", 60)").Scan(&locked); err != nil { // table and the im_users.tags column. If both are already gone, the migration
// has fully run (or the database was created fresh) and no lock is needed.
stale := false
if exists, err := identityTableExists(ctx, conn, "im_usertags"); err != nil {
return err
} else if exists {
stale = true
}
if !stale {
if exists, err := identityColumnExists(ctx, conn, "im_users", "tags"); err != nil {
return err
} else if exists {
stale = true
}
}
if !stale {
return nil
}
// PXC strict-mode clusters prohibit GET_LOCK; acquireMigrationLock falls back
// to lock-free execution in that case. The migration body is idempotent.
_, unlock, err := acquireMigrationLock(ctx, conn, userIdentityMigrationLock)
if err != nil {
return err return err
} }
if !locked.Valid || locked.Int64 != 1 { if unlock != nil {
return errors.New("mysql: timed out waiting for user identity migration lock") defer unlock()
} }
defer conn.ExecContext(ctx, "SELECT RELEASE_LOCK("+userIdentityMigrationLock+")")
usersExist, err := identityTableExists(ctx, conn, "im_users") usersExist, err := identityTableExists(ctx, conn, "im_users")
if err != nil || !usersExist { if err != nil || !usersExist {
@@ -65,7 +85,11 @@ func ensureUserIdentitySchema(db *sqlx.DB) error {
avatar_file_id VARCHAR(64) NOT NULL DEFAULT '',nickname VARCHAR(64) NOT NULL DEFAULT '', avatar_file_id VARCHAR(64) NOT NULL DEFAULT '',nickname VARCHAR(64) NOT NULL DEFAULT '',
birth_date DATE,signature VARCHAR(256) NOT NULL DEFAULT '',gender VARCHAR(16) NOT NULL DEFAULT '', birth_date DATE,signature VARCHAR(256) NOT NULL DEFAULT '',gender VARCHAR(16) NOT NULL DEFAULT '',
region VARCHAR(128) NOT NULL DEFAULT '',email VARCHAR(128) NOT NULL DEFAULT '', region VARCHAR(128) NOT NULL DEFAULT '',email VARCHAR(128) NOT NULL DEFAULT '',
public_account VARCHAR(64) NOT NULL DEFAULT '',version INT NOT NULL DEFAULT 1, public_account VARCHAR(64) NOT NULL DEFAULT '',
friend_add_enabled TINYINT NOT NULL DEFAULT 1,
friend_add_disabled_reason VARCHAR(255) NOT NULL DEFAULT '',
friend_add_policy_version INT NOT NULL DEFAULT 1,
version INT NOT NULL DEFAULT 1,
created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3), created_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3), updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY(id),UNIQUE INDEX uk_im_user_profiles_user(`+profileUniqueKey+`))`); err != nil { PRIMARY KEY(id),UNIQUE INDEX uk_im_user_profiles_user(`+profileUniqueKey+`))`); err != nil {
@@ -99,6 +123,27 @@ func ensureUserIdentitySchema(db *sqlx.DB) error {
return err return err
} }
} }
if exists, err := identityColumnExists(ctx, conn, "im_user_profiles", "friend_add_enabled"); err != nil {
return err
} else if !exists {
if _, err = conn.ExecContext(ctx, "ALTER TABLE im_user_profiles ADD COLUMN friend_add_enabled TINYINT NOT NULL DEFAULT 1 AFTER public_account"); err != nil {
return err
}
}
if exists, err := identityColumnExists(ctx, conn, "im_user_profiles", "friend_add_disabled_reason"); err != nil {
return err
} else if !exists {
if _, err = conn.ExecContext(ctx, "ALTER TABLE im_user_profiles ADD COLUMN friend_add_disabled_reason VARCHAR(255) NOT NULL DEFAULT '' AFTER friend_add_enabled"); err != nil {
return err
}
}
if exists, err := identityColumnExists(ctx, conn, "im_user_profiles", "friend_add_policy_version"); err != nil {
return err
} else if !exists {
if _, err = conn.ExecContext(ctx, "ALTER TABLE im_user_profiles ADD COLUMN friend_add_policy_version INT NOT NULL DEFAULT 1 AFTER friend_add_disabled_reason"); err != nil {
return err
}
}
} }
if err = backfillUserPhones(ctx, conn, usersScoped); err != nil { if err = backfillUserPhones(ctx, conn, usersScoped); err != nil {
@@ -111,12 +156,6 @@ func ensureUserIdentitySchema(db *sqlx.DB) error {
if err = backfillProfileEmails(ctx, conn, usersScoped); err != nil { if err = backfillProfileEmails(ctx, conn, usersScoped); err != nil {
return err return err
} }
if err = backfillProfileDisplay(ctx, conn, usersScoped); err != nil {
return err
}
if err = backfillUserPublicFromProfile(ctx, conn, usersScoped); err != nil {
return err
}
} }
indexName := "users_phone" indexName := "users_phone"
@@ -445,90 +484,6 @@ func backfillProfileEmails(ctx context.Context, conn *sqlx.Conn, usersScoped boo
return nil return nil
} }
func backfillProfileDisplay(ctx context.Context, conn *sqlx.Conn, usersScoped bool) error {
profilesScoped, err := identityColumnExists(ctx, conn, "im_user_profiles", "tenant_id")
if err != nil {
return err
}
if profilesScoped != usersScoped {
return errors.New("mysql: im_users and im_user_profiles tenant scope mismatch")
}
nicknameExpr := `COALESCE(
NULLIF(JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.display_name')), ''),
NULLIF(JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.nickname')), ''),
NULLIF(JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.fn')), ''),
NULLIF(JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.name')), '')
)`
avatarExpr := `COALESCE(
NULLIF(JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.avatar_file_id')), '')
)`
if usersScoped {
if _, err = conn.ExecContext(ctx, `INSERT IGNORE INTO im_user_profiles(id,tenant_id,user_id,nickname,avatar_file_id,public_account)
SELECT u.id,u.tenant_id,u.id,COALESCE(`+nicknameExpr+`,''),COALESCE(`+avatarExpr+`,''),''
FROM im_users AS u
WHERE `+nicknameExpr+` IS NOT NULL OR `+avatarExpr+` IS NOT NULL`); err != nil {
return err
}
_, err = conn.ExecContext(ctx, `UPDATE im_user_profiles AS p
JOIN im_users AS u ON u.tenant_id=p.tenant_id AND u.id=p.user_id
SET p.nickname=IF(p.nickname='' AND `+nicknameExpr+` IS NOT NULL, `+nicknameExpr+`, p.nickname),
p.avatar_file_id=IF(p.avatar_file_id='' AND `+avatarExpr+` IS NOT NULL, `+avatarExpr+`, p.avatar_file_id)
WHERE (p.nickname='' AND `+nicknameExpr+` IS NOT NULL) OR (p.avatar_file_id='' AND `+avatarExpr+` IS NOT NULL)`)
return err
}
if _, err = conn.ExecContext(ctx, `INSERT IGNORE INTO im_user_profiles(id,user_id,nickname,avatar_file_id,public_account)
SELECT u.id,u.id,COALESCE(`+nicknameExpr+`,''),COALESCE(`+avatarExpr+`,''),''
FROM im_users AS u
WHERE `+nicknameExpr+` IS NOT NULL OR `+avatarExpr+` IS NOT NULL`); err != nil {
return err
}
_, err = conn.ExecContext(ctx, `UPDATE im_user_profiles AS p
JOIN im_users AS u ON u.id=p.user_id
SET p.nickname=IF(p.nickname='' AND `+nicknameExpr+` IS NOT NULL, `+nicknameExpr+`, p.nickname),
p.avatar_file_id=IF(p.avatar_file_id='' AND `+avatarExpr+` IS NOT NULL, `+avatarExpr+`, p.avatar_file_id)
WHERE (p.nickname='' AND `+nicknameExpr+` IS NOT NULL) OR (p.avatar_file_id='' AND `+avatarExpr+` IS NOT NULL)`)
return err
}
func backfillUserPublicFromProfile(ctx context.Context, conn *sqlx.Conn, usersScoped bool) error {
profilesScoped, err := identityColumnExists(ctx, conn, "im_user_profiles", "tenant_id")
if err != nil {
return err
}
if profilesScoped != usersScoped {
return errors.New("mysql: im_users and im_user_profiles tenant scope mismatch")
}
publicDoc := "COALESCE(NULLIF(u.public,''), JSON_OBJECT())"
nicknamePatch := `u.public=JSON_SET(` + publicDoc + `,
'$.display_name',p.nickname,'$.nickname',p.nickname,'$.fn',p.nickname,'$.name',p.nickname)`
avatarPatch := `u.public=JSON_SET(` + publicDoc + `,'$.avatar_file_id',p.avatar_file_id)`
accountPatch := `u.public=JSON_SET(` + publicDoc + `,'$.public_account',p.public_account)`
join := "JOIN im_users AS u ON u.id=p.user_id"
if usersScoped {
join = "JOIN im_users AS u ON u.tenant_id=p.tenant_id AND u.id=p.user_id"
}
for _, query := range []string{
`UPDATE im_user_profiles AS p ` + join + `
SET ` + nicknamePatch + `
WHERE p.nickname<>'' AND (
COALESCE(JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.display_name')),'')<>p.nickname OR
COALESCE(JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.nickname')),'')<>p.nickname OR
COALESCE(JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.fn')),'')<>p.nickname OR
COALESCE(JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.name')),'')<>p.nickname)`,
`UPDATE im_user_profiles AS p ` + join + `
SET ` + avatarPatch + `
WHERE p.avatar_file_id<>'' AND COALESCE(JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.avatar_file_id')),'')<>p.avatar_file_id`,
`UPDATE im_user_profiles AS p ` + join + `
SET ` + accountPatch + `
WHERE p.public_account<>'' AND COALESCE(JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.public_account')),'')<>p.public_account`,
} {
if _, err = conn.ExecContext(ctx, query); err != nil {
return err
}
}
return nil
}
func identityCandidates(ctx context.Context, conn *sqlx.Conn, scoped bool, method string) ([]identityValue, error) { func identityCandidates(ctx context.Context, conn *sqlx.Conn, scoped bool, method string) ([]identityValue, error) {
var out []identityValue var out []identityValue
appendQuery := func(query string, args ...any) error { appendQuery := func(query string, args ...any) error {
+2 -18
View File
@@ -28,18 +28,7 @@ func TestNormalizeUserPhone(t *testing.T) {
} }
func TestProfilePublicFromFields(t *testing.T) { func TestProfilePublicFromFields(t *testing.T) {
public := profilePublicFromFields(map[string]any{ public := profilePublicFromFields(sql.NullString{String: "新昵称", Valid: true}, sql.NullString{String: "file_avatar_1", Valid: true},
"fn": "旧昵称",
"name": "旧昵称",
"nickname": "旧昵称",
"display_name": "旧昵称",
"avatar_file_id": "old_avatar",
"signature": "旧签名",
"gender": "old_gender",
"region": "old_region",
"photo": "old_photo",
"public_account": "usrAccount",
}, sql.NullString{String: "新昵称", Valid: true}, sql.NullString{String: "file_avatar_1", Valid: true},
sql.NullString{String: "新签名", Valid: true}, sql.NullString{String: "male", Valid: true}, sql.NullString{String: "新签名", Valid: true}, sql.NullString{String: "male", Valid: true},
sql.NullString{String: "上海", Valid: true}, sql.NullString{String: "usrProfile", Valid: true}).(map[string]any) sql.NullString{String: "上海", Valid: true}, sql.NullString{String: "usrProfile", Valid: true}).(map[string]any)
@@ -53,12 +42,7 @@ func TestProfilePublicFromFields(t *testing.T) {
t.Fatalf("profile public not overlaid: %#v", public) t.Fatalf("profile public not overlaid: %#v", public)
} }
public = profilePublicFromFields(map[string]any{ public = profilePublicFromFields(sql.NullString{Valid: true}, sql.NullString{Valid: true}, sql.NullString{Valid: true},
"fn": "旧昵称",
"avatar_file_id": "old_avatar",
"public_account": "usrAccount",
"signature": "旧签名",
}, sql.NullString{Valid: true}, sql.NullString{Valid: true}, sql.NullString{Valid: true},
sql.NullString{Valid: true}, sql.NullString{Valid: true}, sql.NullString{Valid: true}).(map[string]any) sql.NullString{Valid: true}, sql.NullString{Valid: true}, sql.NullString{Valid: true}).(map[string]any)
for _, key := range []string{"fn", "name", "nickname", "display_name", "avatar_file_id", "photo", "avatar", "signature", "gender", "region", "public_account"} { for _, key := range []string{"fn", "name", "nickname", "display_name", "avatar_file_id", "photo", "avatar", "signature", "gender", "region", "public_account"} {
if _, ok := public[key]; ok { if _, ok := public[key]; ok {
+83
View File
@@ -0,0 +1,83 @@
//go:build mysql
// +build mysql
package mysql
import (
"context"
"github.com/jmoiron/sqlx"
)
const userPrivacyMigrationLock = "CONCAT('im_user_privacy:', LEFT(SHA2(DATABASE(), 256), 40))"
// ensureUserPrivacySchema adds columns of im_user_privacy_settings that were
// introduced after the table's initial release. Databases created fresh by
// createIMUserProductTables already have them; legacy databases are upgraded
// in place. A missing table is skipped: it is created by the fresh-install
// path and there is nothing to upgrade yet.
func ensureUserPrivacySchema(db *sqlx.DB) error {
ctx := context.Background()
conn, err := db.Connx(ctx)
if err != nil {
return err
}
defer conn.Close()
// Fast path: check for missing columns without taking the migration lock.
// Databases created by createIMUserProductTables already have every column,
// so this is the common case and avoids GET_LOCK entirely (prohibited on
// Percona XtraDB Cluster with pxc_strict_mode=ENFORCING).
added, err := ensureUserPrivacyColumns(ctx, conn)
if err != nil || added {
return err
}
_, unlock, err := acquireMigrationLock(ctx, conn, userPrivacyMigrationLock)
if err != nil {
return err
}
if unlock != nil {
defer unlock()
}
// Double-check under the lock: another instance may have migrated while we
// were waiting. Without a lock (PXC fallback) this simply runs idempotently.
_, err = ensureUserPrivacyColumns(ctx, conn)
return err
}
// ensureUserPrivacyColumns adds any columns of im_user_privacy_settings that
// are missing and reports whether nothing was missing. A missing table is
// skipped: it is created by the fresh-install path and there is nothing to
// upgrade yet. Column definitions mirror createIMUserProductTables; the SELECT
// in userPrivacyByUID references every one of them.
func ensureUserPrivacyColumns(ctx context.Context, conn *sqlx.Conn) (bool, error) {
tableExists, err := identityTableExists(ctx, conn, "im_user_privacy_settings")
if err != nil || !tableExists {
return true, err
}
columns := []struct{ name, ddl string }{
{"profile_visibility", "VARCHAR(32) NOT NULL DEFAULT 'friends'"},
{"last_seen_visibility", "VARCHAR(32) NOT NULL DEFAULT 'friends'"},
{"read_receipts", "TINYINT NOT NULL DEFAULT 1"},
{"friend_request_verification", "TINYINT NOT NULL DEFAULT 1"},
{"group_invite_verification", "TINYINT NOT NULL DEFAULT 0"},
}
missing := false
for _, col := range columns {
exists, err := identityColumnExists(ctx, conn, "im_user_privacy_settings", col.name)
if err != nil {
return false, err
}
if exists {
continue
}
missing = true
if _, err = conn.ExecContext(ctx, "ALTER TABLE im_user_privacy_settings ADD COLUMN "+col.name+" "+col.ddl); err != nil {
return false, err
}
}
return !missing, nil
}
+77
View File
@@ -0,0 +1,77 @@
package main
import (
"encoding/json"
"hash/fnv"
"strings"
"github.com/tinode/chat/server/store"
"github.com/tinode/chat/server/store/types"
)
const (
imAssetsConfigGroup = "im_assets"
imDefaultMaleAvatarURLsConfig = "default_user_avatar_male_urls"
imDefaultFemaleAvatarURLsConfig = "default_user_avatar_female_urls"
)
func applyDefaultUserAvatar(data map[string]any, userID, gender string) {
if data == nil || strings.TrimSpace(stringFromAny(data["avatar_file_id"])) != "" {
return
}
if avatarURL := defaultUserAvatarURL(userID, gender); avatarURL != "" {
data["avatar_file_id"] = avatarURL
}
}
func defaultUserAvatarURL(userID, gender string) string {
if store.ThirdPartyConfig == nil {
return ""
}
values, err := store.ThirdPartyConfig.Get(imAssetsConfigGroup, imDefaultMaleAvatarURLsConfig, imDefaultFemaleAvatarURLsConfig)
if err != nil {
return ""
}
key := imDefaultMaleAvatarURLsConfig
if strings.EqualFold(strings.TrimSpace(gender), "female") {
key = imDefaultFemaleAvatarURLsConfig
}
urls := parseDefaultAvatarURLs(values[key])
if len(urls) == 0 && key == imDefaultFemaleAvatarURLsConfig {
urls = parseDefaultAvatarURLs(values[imDefaultMaleAvatarURLsConfig])
}
if len(urls) == 0 {
return ""
}
return urls[defaultAvatarIndex(userID, len(urls))]
}
func parseDefaultAvatarURLs(raw string) []string {
var urls []string
if err := json.Unmarshal([]byte(strings.TrimSpace(raw)), &urls); err != nil {
return nil
}
out := urls[:0]
for _, url := range urls {
if url = strings.TrimSpace(url); url != "" {
out = append(out, url)
}
}
return out
}
func defaultAvatarIndex(userID string, count int) int {
if count <= 1 {
return 0
}
uid := types.ParseUserId(strings.TrimSpace(userID))
if uid.IsZero() {
uid = types.ParseUid(strings.TrimSpace(userID))
}
if !uid.IsZero() {
return int(uint64(uid) % uint64(count))
}
h := fnv.New32a()
_, _ = h.Write([]byte(userID))
return int(h.Sum32() % uint32(count))
}
+60
View File
@@ -0,0 +1,60 @@
package main
import (
"fmt"
"testing"
"github.com/tinode/chat/server/store"
"github.com/tinode/chat/server/store/types"
)
type defaultAvatarConfigStore map[string]string
func (s defaultAvatarConfigStore) Get(_ string, keys ...string) (map[string]string, error) {
out := make(map[string]string, len(keys))
for _, key := range keys {
out[key] = s[key]
}
return out, nil
}
func TestUserSummaryPayloadAppliesDefaultAvatarByGenderAndUserID(t *testing.T) {
oldConfig := store.ThirdPartyConfig
store.ThirdPartyConfig = defaultAvatarConfigStore{
imDefaultMaleAvatarURLsConfig: `["m0","m1","m2","m3","m4"]`,
imDefaultFemaleAvatarURLsConfig: `["f0","f1","f2","f3","f4"]`,
}
defer func() { store.ThirdPartyConfig = oldConfig }()
for i, gender := range []string{"male", "female"} {
uid := types.Uid(10 + i)
payload := userSummaryPayload(&types.User{
ObjHeader: types.ObjHeader{Id: uid.String()},
Public: map[string]any{"display_name": "Alice", "gender": gender},
}, "")
wantPrefix := "m"
if gender == "female" {
wantPrefix = "f"
}
want := fmt.Sprintf("%s%d", wantPrefix, uint64(uid)%5)
if payload["avatar_file_id"] != want {
t.Fatalf("gender=%s avatar=%v want %s", gender, payload["avatar_file_id"], want)
}
}
}
func TestUserSummaryPayloadKeepsUploadedAvatar(t *testing.T) {
oldConfig := store.ThirdPartyConfig
store.ThirdPartyConfig = defaultAvatarConfigStore{
imDefaultMaleAvatarURLsConfig: `["m0","m1","m2","m3","m4"]`,
}
defer func() { store.ThirdPartyConfig = oldConfig }()
payload := userSummaryPayload(&types.User{
ObjHeader: types.ObjHeader{Id: types.Uid(10).String()},
Public: map[string]any{"display_name": "Alice", "gender": "male", "avatar_file_id": "uploaded.png"},
}, "")
if payload["avatar_file_id"] != "uploaded.png" {
t.Fatalf("avatar=%v", payload["avatar_file_id"])
}
}
-34
View File
@@ -38,8 +38,6 @@ import (
// See https://www.iana.org/assignments/media-types/media-types.xhtml // See https://www.iana.org/assignments/media-types/media-types.xhtml
var allowedMimeTypes = []string{"application/", "audio/", "font/", "image/", "text/", "video/"} var allowedMimeTypes = []string{"application/", "audio/", "font/", "image/", "text/", "video/"}
const fileUploadPurposeMeetingNote = "meeting_note"
// RequestPrincipal is a tenant-scoped identity authenticated for HTTP/gRPC file access. // RequestPrincipal is a tenant-scoped identity authenticated for HTTP/gRPC file access.
type RequestPrincipal struct { type RequestPrincipal struct {
TenantID types.TenantID TenantID types.TenantID
@@ -257,12 +255,6 @@ func largeFileReceiveHTTP(wrt http.ResponseWriter, req *http.Request) {
writeHttpResponse(ErrAuthRequired(msgID, "", now, now), nil) writeHttpResponse(ErrAuthRequired(msgID, "", now, now), nil)
return return
} }
meetingID, _, err := meetingNoteUploadParams(req, principal)
if err != nil {
writeHttpResponse(decodeStoreError(err, msgID, now, nil), err)
return
}
// Check if uploads are handled elsewhere. // Check if uploads are handled elsewhere.
headers, statusCode, err := mh.Headers(principal.TenantID, req.Method, req.URL, req.Header, true) headers, statusCode, err := mh.Headers(principal.TenantID, req.Method, req.URL, req.Header, true)
if err != nil { if err != nil {
@@ -374,36 +366,10 @@ func largeFileReceiveHTTP(wrt http.ResponseWriter, req *http.Request) {
// How long this file is guaranteed to exist without being attached to a message or a topic. // How long this file is guaranteed to exist without being attached to a message or a topic.
params["expires"] = now.Add(globals.mediaGcPeriod).Format(types.TimeFormatRFC3339) params["expires"] = now.Add(globals.mediaGcPeriod).Format(types.TimeFormatRFC3339)
} }
if meetingID != "" {
params["purpose"] = fileUploadPurposeMeetingNote
params["meeting_id"] = meetingID
}
writeHttpResponse(NoErrParams(msgID, "", now, params), nil) writeHttpResponse(NoErrParams(msgID, "", now, params), nil)
logs.Info.Println("media upload: ok", fdef.Id, fdef.Location) logs.Info.Println("media upload: ok", fdef.Id, fdef.Location)
} }
func meetingNoteUploadParams(req *http.Request, principal RequestPrincipal) (string, string, error) {
if strings.TrimSpace(req.FormValue("purpose")) != fileUploadPurposeMeetingNote {
return "", "", nil
}
if principal.UID.IsZero() {
return "", "", types.ErrPermissionDenied
}
meetingID := strings.TrimSpace(req.FormValue("meeting_id"))
if meetingID == "" {
return "", "", types.ErrMalformed
}
idempotencyKey := strings.TrimSpace(req.Header.Get("Idempotency-Key"))
if idempotencyKey == "" {
idempotencyKey = strings.TrimSpace(req.FormValue("id"))
}
if idempotencyKey == "" {
return "", "", types.ErrMalformed
}
return meetingID, idempotencyKey, nil
}
// LargeFileServe is the gRPC equivalent of largeFileServeHTTP. // LargeFileServe is the gRPC equivalent of largeFileServeHTTP.
func (*grpcNodeServer) LargeFileServe(req *pbx.FileDownReq, stream pbx.Node_LargeFileServeServer) error { func (*grpcNodeServer) LargeFileServe(req *pbx.FileDownReq, stream pbx.Node_LargeFileServeServer) error {
now := types.TimeNow() now := types.TimeNow()
-91
View File
@@ -1,11 +1,8 @@
package main package main
import ( import (
"bytes"
"encoding/base64" "encoding/base64"
"encoding/json"
"io" "io"
"mime/multipart"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"net/url" "net/url"
@@ -209,91 +206,3 @@ func TestLargeFileGarbageCollectTenantsScopesDeletes(t *testing.T) {
largeFileGarbageCollectTenants([]types.TenantID{7, 8}, olderThan, 25) largeFileGarbageCollectTenants([]types.TenantID{7, 8}, olderThan, 25)
} }
func TestLargeFileReceiveMeetingNoteUploadReturnsFileID(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
oldStore := store.Store
oldFiles := store.Files
oldMeetingNotes := store.MeetingNotes
oldKafkaBus := kafkaBus
defer func() {
store.Store = oldStore
store.Files = oldFiles
store.MeetingNotes = oldMeetingNotes
kafkaBus = oldKafkaBus
}()
handler := &testLargeFileMediaHandler{}
mockStore := mock_store.NewMockPersistentStorageInterface(ctrl)
tokenAuth := mock_auth.NewMockAuthHandler(ctrl)
files := mock_store.NewMockFilePersistenceInterface(ctrl)
store.Store = mockStore
store.Files = files
store.MeetingNotes = nil
kafkaBus = nil
fileID := types.Uid(101)
mockStore.EXPECT().GetMediaHandler().Return(handler).AnyTimes()
mockStore.EXPECT().GetLogicalAuthHandler("token").Return(tokenAuth)
mockStore.EXPECT().GetUidString().Return(fileID.String())
tokenAuth.EXPECT().Authenticate(auth.AuthContext{RemoteAddr: "192.0.2.1:12345"}, []byte("token-1")).
Return(&auth.Rec{TenantID: 7, Uid: 42, AuthLevel: auth.LevelAuth}, nil, nil)
files.EXPECT().FinishUpload(types.TenantID(7), gomock.Any(), true, int64(11)).DoAndReturn(
func(_ types.TenantID, fd *types.FileDef, _ bool, size int64) (*types.FileDef, error) {
fd.Status = types.UploadCompleted
fd.Size = size
return fd, nil
})
var body bytes.Buffer
writer := multipart.NewWriter(&body)
if err := writer.WriteField("id", "upload-meeting-note-1"); err != nil {
t.Fatal(err)
}
if err := writer.WriteField("purpose", fileUploadPurposeMeetingNote); err != nil {
t.Fatal(err)
}
if err := writer.WriteField("meeting_id", "mtg_test"); err != nil {
t.Fatal(err)
}
part, err := writer.CreateFormFile("file", "notes.txt")
if err != nil {
t.Fatal(err)
}
if _, err := part.Write([]byte("hello notes")); err != nil {
t.Fatal(err)
}
if err := writer.Close(); err != nil {
t.Fatal(err)
}
req := httptest.NewRequest(http.MethodPost, "/v0/file/u/", &body)
req.RemoteAddr = "192.0.2.1:12345"
req.Header.Set("Content-Type", writer.FormDataContentType())
req.Header.Set("X-MeetingIM-APIKey", makeTestAPIKey(t))
req.Header.Set("X-MeetingIM-Auth", "token "+base64.StdEncoding.EncodeToString([]byte("token-1")))
rec := httptest.NewRecorder()
largeFileReceiveHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String())
}
var resp ServerComMessage
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode response: %v body=%s", err, rec.Body.String())
}
params := resp.Ctrl.Params.(map[string]any)
if params["file_id"] != fileID.String() || params["purpose"] != fileUploadPurposeMeetingNote ||
params["meeting_id"] != "mtg_test" {
t.Fatalf("unexpected upload params: %#v", params)
}
if _, ok := params["note_file_id"]; ok {
t.Fatalf("upload should not bind meeting note file: %#v", params)
}
if handler.data != "hello notes" || handler.def.OriginalName != "notes.txt" {
t.Fatalf("unexpected side effects: handler=%#v", handler)
}
}
+60
View File
@@ -10,6 +10,8 @@ import (
"time" "time"
rtctokenbuilder "github.com/AgoraIO/Tools/DynamicKey/AgoraDynamicKey/go/src/rtctokenbuilder2" rtctokenbuilder "github.com/AgoraIO/Tools/DynamicKey/AgoraDynamicKey/go/src/rtctokenbuilder2"
"github.com/tinode/chat/server/auth"
"github.com/tinode/chat/server/logs"
"github.com/tinode/chat/server/store" "github.com/tinode/chat/server/store"
"github.com/tinode/chat/server/store/types" "github.com/tinode/chat/server/store/types"
) )
@@ -110,6 +112,9 @@ func serveIMCallRTCJoin(wrt http.ResponseWriter, req *http.Request) {
writeIMError(wrt, req, http.StatusConflict, 30064, imReasonCallStateInvalid, "call state invalid", map[string]any{"status": record.Status}) writeIMError(wrt, req, http.StatusConflict, 30064, imReasonCallStateInvalid, "call state invalid", map[string]any{"status": record.Status})
return return
} }
if body.Role == "callee" && record.Status == types.CallStatusStarted {
acceptIMCallFromRTCJoin(principal, record)
}
channelName := config.ChannelPrefix + record.CallID channelName := config.ChannelPrefix + record.CallID
uid, err := agoraUIDForUser(config, principal.Uid) uid, err := agoraUIDForUser(config, principal.Uid)
if err != nil { if err != nil {
@@ -267,6 +272,61 @@ func callStateAllowsRTCJoin(status string) bool {
} }
} }
func acceptIMCallFromRTCJoin(principal *imPrincipal, record *types.CallRecord) {
if principal == nil || record == nil || record.Status != types.CallStatusStarted || record.CalleeID != principal.Uid {
return
}
now := types.TimeNow()
if store.Calls != nil {
if err := store.Calls.Accept(principal.TenantID, record.CallID, record.SeqID, now); err != nil {
logs.Warn.Printf("rtc join: failed to mark call accepted tenant=%d topic=%s seq=%d call_id=%s: %v",
principal.TenantID, record.Topic, record.SeqID, record.CallID, err)
}
}
if globals.hub == nil {
return
}
payload, _ := json.Marshal(map[string]any{"callId": record.CallID})
msg := &ClientComMessage{
Note: &MsgClientNote{
Topic: record.Topic,
What: "call",
Event: constCallEventAccept,
SeqId: record.SeqID,
Payload: payload,
},
Original: record.Topic,
RcptTo: record.Topic,
AsUser: principal.Uid.UserId(),
AuthLvl: int(auth.LevelAuth),
Timestamp: now,
TenantID: principal.TenantID,
sess: &Session{
proto: NONE,
sid: "rtc-join:" + principal.Uid.UserId(),
uid: principal.Uid,
tenantID: principal.TenantID,
authLvl: auth.LevelAuth,
lastAction: now.UnixNano(),
},
init: true,
}
if globals.cluster != nil && globals.cluster.isRemoteTopic(types.TopicKey{TenantID: principal.TenantID, Topic: record.Topic}) {
if err := globals.cluster.routeToTopicMaster(ProxyReqCall, msg, record.Topic, msg.sess); err != nil {
logs.Warn.Printf("rtc join: failed to route call accept tenant=%d topic=%s seq=%d call_id=%s: %v",
principal.TenantID, record.Topic, record.SeqID, record.CallID, err)
}
return
}
publishRealtimeCallEvent(msg)
select {
case globals.hub.routeCli <- msg:
default:
logs.Err.Printf("rtc join: hub route queue full tenant=%d topic=%s seq=%d call_id=%s",
principal.TenantID, record.Topic, record.SeqID, record.CallID)
}
}
func agoraUIDForUser(config imAgoraConfig, uid types.Uid) (any, error) { func agoraUIDForUser(config imAgoraConfig, uid types.Uid) (any, error) {
switch config.UIDMode { switch config.UIDMode {
case "int": case "int":
-24
View File
@@ -74,30 +74,6 @@ func createIMSystemNotification(tenantID types.TenantID, uid types.Uid, eventID,
}) })
} }
func publishIMMeetingEvent(tenantID types.TenantID, actor types.Uid, meeting *types.Meeting, eventType string) error {
if meeting == nil {
return types.ErrMalformed
}
for _, participant := range meeting.Participants {
uid := participant.UserID
if uid.IsZero() {
continue
}
eventID := stableIMEventID("meeting", meeting.MeetingID, meeting.Version, eventType, uid.UserId())
if uid != actor {
if err := createIMSystemNotification(tenantID, uid, stableIMEventID(eventID, "notification"), eventType,
"会议通知", meeting.Title, types.KVMap{"meeting_id": meeting.MeetingID}); err != nil {
return err
}
}
if err := enqueueIMBusinessEvent(tenantID, eventID, "user", uid.UserId(), actor, eventType,
map[string]any{"meeting_id": meeting.MeetingID, "version": meeting.Version, "state": meeting.State}); err != nil {
return err
}
}
return nil
}
func publishIMRelationEvent(tenantID types.TenantID, actor, target types.Uid, stableID, eventType, title, summary string) error { func publishIMRelationEvent(tenantID types.TenantID, actor, target types.Uid, stableID, eventType, title, summary string) error {
if target.IsZero() { if target.IsZero() {
return types.ErrMalformed return types.ErrMalformed
+130
View File
@@ -2,10 +2,12 @@ package main
import ( import (
"encoding/json" "encoding/json"
"fmt"
"strings" "strings"
"time" "time"
"github.com/tinode/chat/server/logs" "github.com/tinode/chat/server/logs"
"github.com/tinode/chat/server/store"
"github.com/tinode/chat/server/store/types" "github.com/tinode/chat/server/store/types"
) )
@@ -15,6 +17,7 @@ func publishIMGroupRealtimeEvent(tenantID types.TenantID, topic string, actor ty
if tenantID.IsZero() || topic == "" || eventType == "" { if tenantID.IsZero() || topic == "" || eventType == "" {
return return
} }
payload = enrichIMGroupMemberEventPayload(eventType, payload)
encoded, err := json.Marshal(payload) encoded, err := json.Marshal(payload)
if err != nil { if err != nil {
logs.Warn.Printf("group realtime: encode event topic=%s event=%s: %v", topic, eventType, err) logs.Warn.Printf("group realtime: encode event topic=%s event=%s: %v", topic, eventType, err)
@@ -24,6 +27,10 @@ func publishIMGroupRealtimeEvent(tenantID types.TenantID, topic string, actor ty
if !actor.IsZero() { if !actor.IsZero() {
from = actor.UserId() from = actor.UserId()
} }
if isAdminOnlyIMGroupEvent(eventType) {
publishIMGroupEventToAdmins(tenantID, topic, actor, eventType, encoded, payload)
return
}
message := &ServerComMessage{ message := &ServerComMessage{
TenantID: tenantID, TenantID: tenantID,
RcptTo: topic, RcptTo: topic,
@@ -47,6 +54,129 @@ func publishIMGroupRealtimeEvent(tenantID types.TenantID, topic string, actor ty
publishRealtimeTopicMessage(topic, message) publishRealtimeTopicMessage(topic, message)
} }
func isAdminOnlyIMGroupEvent(eventType string) bool {
switch eventType {
case "group.member.updated", "group.member.left", "group.role.updated", "group.mute.updated",
"group.invitations.updated", "group.join_request.created", "group.join_request.accepted",
"group.join_request.rejected":
return true
default:
return false
}
}
func publishIMGroupEventToAdmins(tenantID types.TenantID, topic string, actor types.Uid, eventType string,
encoded json.RawMessage, payload any) {
recipients := map[types.Uid]struct{}{}
if !actor.IsZero() {
recipients[actor] = struct{}{}
}
collectIMGroupEventUserIDs(payload, recipients)
for _, role := range []string{types.GroupMemberRoleOwner, types.GroupMemberRoleAdmin} {
if store.GroupBusiness == nil {
break
}
cursor := ""
for {
members, next, err := store.GroupBusiness.ListMembers(tenantID, topic, types.GroupMemberListOptions{
Role: role, Cursor: cursor, Limit: 100,
})
if err != nil {
logs.Warn.Printf("group realtime: list %s recipients topic=%s event=%s: %v", role, topic, eventType, err)
break
}
for _, member := range members {
if !member.UserID.IsZero() {
recipients[member.UserID] = struct{}{}
}
}
if next == "" || next == cursor {
break
}
cursor = next
}
}
for uid := range recipients {
message := &ServerComMessage{
TenantID: tenantID, RcptTo: uid.UserId(), AsUser: actor.UserId(),
Timestamp: time.Now().UTC().Round(time.Millisecond),
Info: &MsgServerInfo{Topic: topic, From: actor.UserId(), What: "group", Event: eventType, Payload: encoded},
}
if globals.sessionStore != nil {
globals.sessionStore.DeliverToUser(tenantID, uid, "", message)
}
publishRealtimeUserMessage(uid, message)
}
}
func collectIMGroupEventUserIDs(value any, out map[types.Uid]struct{}) {
switch typed := value.(type) {
case map[string]any:
if ok, exists := typed["ok"].(bool); exists && !ok {
return
}
for key, item := range typed {
switch key {
case "user_id", "target_id", "target_user_id", "applicant_id", "applicant_user_id", "owner_id":
if uid := imGroupEventUID(item); !uid.IsZero() {
out[uid] = struct{}{}
}
}
collectIMGroupEventUserIDs(item, out)
}
case []map[string]any:
for _, item := range typed {
collectIMGroupEventUserIDs(item, out)
}
case []any:
for _, item := range typed {
collectIMGroupEventUserIDs(item, out)
}
}
}
func imGroupEventUID(value any) types.Uid {
if uid := types.ParseUserId(strings.TrimSpace(fmt.Sprint(value))); !uid.IsZero() {
return uid
}
switch typed := value.(type) {
case int64:
return store.EncodeUid(typed)
case int:
return store.EncodeUid(int64(typed))
case float64:
return store.EncodeUid(int64(typed))
}
return types.ZeroUid
}
func enrichIMGroupMemberEventPayload(eventType string, payload any) any {
data, ok := payload.(map[string]any)
if !ok {
return payload
}
copyPayload := make(map[string]any, len(data)+2)
for key, value := range data {
copyPayload[key] = value
}
switch eventType {
case "group.member.left":
copyPayload["member_sync"] = "remove"
case "group.member.joined", "group.member.updated", "group.role.updated", "group.mute.updated":
if _, ok := copyPayload["user_id"]; ok {
copyPayload["member_sync"] = "single"
} else {
copyPayload["member_sync"] = "full"
}
}
if isAdminOnlyIMGroupEvent(eventType) {
copyPayload["visibility"] = "admins_and_affected"
} else if eventType == "group.member.joined" {
copyPayload["visibility"] = "all_members"
}
return copyPayload
}
func userIDs(uids []types.Uid) []string { func userIDs(uids []types.Uid) []string {
ids := make([]string, 0, len(uids)) ids := make([]string, 0, len(uids))
for _, uid := range uids { for _, uid := range uids {
+149
View File
@@ -0,0 +1,149 @@
package main
import (
"encoding/json"
"testing"
"time"
"github.com/golang/mock/gomock"
"github.com/tinode/chat/server/store"
"github.com/tinode/chat/server/store/mock_store"
"github.com/tinode/chat/server/store/types"
)
func TestPublishIMGroupRealtimeEventRestrictsMemberRemoval(t *testing.T) {
oldGroupBusiness := store.GroupBusiness
oldSessions := globals.sessionStore
oldHub := globals.hub
admin := types.Uid(1)
affected := types.Uid(2)
regular := types.Uid(3)
actor := types.Uid(4)
store.GroupBusiness = &testGroupBusinessStore{listMembers: []types.GroupMember{
{UserID: admin, Role: types.GroupMemberRoleAdmin, MemberStatus: types.GroupMemberStatusActive},
}}
globals.sessionStore = NewSessionStore(time.Minute)
globals.hub = &Hub{routeSrv: make(chan *ServerComMessage, 2)}
t.Cleanup(func() {
store.GroupBusiness = oldGroupBusiness
globals.sessionStore = oldSessions
globals.hub = oldHub
})
sessions := map[types.Uid]*Session{}
for _, uid := range []types.Uid{admin, affected, regular} {
sess := &Session{
sid: uid.UserId(), uid: uid, tenantID: 7, proto: WEBSOCK,
send: make(chan any, 2), subs: make(map[string]*Subscription),
}
globals.sessionStore.sessCache[sess.sid] = sess
sessions[uid] = sess
}
publishIMGroupRealtimeEvent(7, "grpRestricted", actor, "group.member.left",
map[string]any{"user_id": affected.UserId()})
for _, uid := range []types.Uid{admin, affected} {
select {
case raw := <-sessions[uid].send:
msg, ok := raw.(*ServerComMessage)
if !ok || msg.Info == nil || msg.Info.Event != "group.member.left" || msg.Info.Topic != "grpRestricted" {
t.Fatalf("unexpected event for %s: %#v", uid.UserId(), raw)
}
var payload map[string]any
if err := json.Unmarshal(msg.Info.Payload, &payload); err != nil {
t.Fatal(err)
}
if payload["member_sync"] != "remove" || payload["visibility"] != "admins_and_affected" {
t.Fatalf("unexpected payload for %s: %#v", uid.UserId(), payload)
}
default:
t.Fatalf("expected event for %s", uid.UserId())
}
}
select {
case raw := <-sessions[regular].send:
t.Fatalf("regular member received restricted event: %#v", raw)
default:
}
if len(globals.hub.routeSrv) != 0 {
t.Fatalf("restricted event was broadcast to topic")
}
}
func TestPublishIMGroupRealtimeEventBroadcastsMemberJoin(t *testing.T) {
oldHub := globals.hub
globals.hub = &Hub{routeSrv: make(chan *ServerComMessage, 1)}
t.Cleanup(func() { globals.hub = oldHub })
publishIMGroupRealtimeEvent(7, "grpPublic", types.Uid(1), "group.member.joined",
map[string]any{"user_id": types.Uid(2).UserId()})
select {
case msg := <-globals.hub.routeSrv:
if msg.Info == nil || msg.Info.Event != "group.member.joined" {
t.Fatalf("unexpected joined event: %#v", msg)
}
var payload map[string]any
if err := json.Unmarshal(msg.Info.Payload, &payload); err != nil {
t.Fatal(err)
}
if payload["member_sync"] != "single" || payload["visibility"] != "all_members" {
t.Fatalf("unexpected joined payload: %#v", payload)
}
default:
t.Fatal("joined event was not broadcast")
}
}
func TestSendPushForStoredGroupMessageIncludesEverySubscription(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
oldTopics := store.Topics
oldUsersUpdate := globals.usersUpdate
oldCluster := globals.cluster
topics := mock_store.NewMockTopicsPersistenceInterface(ctrl)
store.Topics = topics
globals.usersUpdate = make(chan *UserCacheReq, 1)
globals.cluster = nil
t.Cleanup(func() {
store.Topics = oldTopics
globals.usersUpdate = oldUsersUpdate
globals.cluster = oldCluster
})
subs := make([]types.Subscription, 0, 1500)
for i := 1; i <= 1500; i++ {
sub := types.Subscription{
User: types.Uid(i).String(), Topic: "grpLarge", ModeWant: types.ModeCFull, ModeGiven: types.ModeCFull,
}
if i == 1500 {
sub.Private = map[string]any{"mute_notifications": true}
}
subs = append(subs, sub)
}
topics.EXPECT().GetSubs(types.TenantID(7), "grpLarge", nil).Return(subs, nil)
msg := &types.Message{
ObjHeader: types.ObjHeader{CreatedAt: time.Now().UTC()}, TenantID: 7,
Topic: "grpLarge", SeqId: 99, Content: map[string]any{"text": "成员加入了群聊"},
}
sendPushForStoredGroupMessage(7, msg)
select {
case req := <-globals.usersUpdate:
if req.PushRcpt == nil || len(req.PushRcpt.To) != len(subs) {
t.Fatalf("push recipients: got %d want %d", len(req.PushRcpt.To), len(subs))
}
for _, recipient := range req.PushRcpt.To {
if !recipient.ShouldIncrementUnreadCountInCache {
t.Fatal("system group message did not increment per-user unread")
}
}
if !req.PushRcpt.To[types.Uid(1500)].Silent {
t.Fatal("muted group recipient must receive a silent push")
}
default:
t.Fatal("group system message did not enqueue push")
}
}
+423 -2324
View File
File diff suppressed because it is too large Load Diff
+483 -1450
View File
File diff suppressed because it is too large Load Diff
+419
View File
@@ -0,0 +1,419 @@
package main
import (
"errors"
"fmt"
"net/http"
"strconv"
"strings"
"time"
"github.com/tinode/chat/server/logs"
"github.com/tinode/chat/server/store"
"github.com/tinode/chat/server/store/types"
)
const (
imReasonFriendAddDisabled = "FRIEND_ADD_DISABLED"
imReasonMessageCurfew = "MESSAGE_CURFEW_ACTIVE"
imReasonYouthRestricted = "YOUTH_MODE_RESTRICTED"
imYouthActionMessage = "message"
imYouthActionCall = "call"
)
type imPolicyDecision struct {
Allowed bool
Reason string
Message string
Details map[string]any
}
func imAllowDecision() imPolicyDecision {
return imPolicyDecision{Allowed: true}
}
func imDenyDecision(reason, message string) imPolicyDecision {
return imPolicyDecision{Allowed: false, Reason: reason, Message: strings.TrimSpace(message)}
}
func imDenyDecisionDetails(reason, message string, details map[string]any) imPolicyDecision {
decision := imDenyDecision(reason, message)
decision.Details = details
return decision
}
func imYouthModePolicyDecision(tenantID types.TenantID, actor types.Uid, action string, now time.Time) imPolicyDecision {
if store.Security == nil || actor.IsZero() {
return imAllowDecision()
}
settings, err := store.Security.GetYouthMode(tenantID, actor)
if err != nil {
if !errors.Is(err, types.ErrNotFound) && !errors.Is(err, types.ErrUnsupported) {
logs.Warn.Printf("youth mode policy lookup failed: tenant=%d user=%s action=%s err=%v", tenantID, actor.UserId(), action, err)
}
return imAllowDecision()
}
if settings == nil || !settings.Enabled {
return imAllowDecision()
}
switch strings.TrimSpace(action) {
case imYouthActionCall:
if youthModeHasRestriction(settings, "night_quiet") && imYouthNightQuietActive(now) {
return imDenyDecisionDetails(imReasonYouthRestricted, "restricted by youth mode", map[string]any{
"restriction": "night_quiet",
"policy_version": settings.PolicyVersion,
})
}
case imYouthActionMessage:
// content_filter is enforced by routing messages through content moderation
// before persistence; the decision remains allow unless moderation rejects it.
return imAllowDecision()
}
return imAllowDecision()
}
func youthModeHasRestriction(settings *types.YouthModeSettings, code string) bool {
if settings == nil {
return false
}
code = strings.TrimSpace(code)
for _, item := range settings.Restrictions {
if strings.TrimSpace(item) == code {
return true
}
}
return false
}
func imYouthNightQuietActive(now time.Time) bool {
if now.IsZero() {
now = time.Now()
}
local := now.In(time.Local)
minute := local.Hour()*60 + local.Minute()
start := 22 * 60
end := 6 * 60
return minute >= start || minute < end
}
func imTenantCodeForPolicy(tenantID types.TenantID) (string, error) {
tenant, err := store.Tenants.GetByID(tenantID)
if err != nil || tenant == nil {
return "", err
}
return strings.TrimSpace(tenant.Code), nil
}
func imSecurityPolicy(tenantID types.TenantID, tenantCode string) (*types.SecurityPolicy, error) {
tenantCode = strings.TrimSpace(tenantCode)
if tenantCode == "" {
code, err := imTenantCodeForPolicy(tenantID)
if err != nil {
return nil, err
}
tenantCode = code
}
if tenantCode == "" {
return nil, types.ErrNotFound
}
policy, err := store.ModerationConfig.SecurityPolicy(tenantCode)
if err != nil {
return nil, err
}
return policy, nil
}
func imFriendAddPolicyDecision(tenantID types.TenantID, tenantCode string, actor types.Uid, source string) imPolicyDecision {
policy, err := imSecurityPolicy(tenantID, tenantCode)
if err != nil {
if !errors.Is(err, types.ErrUnsupported) && !errors.Is(err, types.ErrNotFound) {
logs.Warn.Printf("friend add policy lookup failed: tenant=%d code=%s user=%s err=%v", tenantID, tenantCode, actor.UserId(), err)
}
return imAllowDecision()
}
if policy != nil && !policy.GlobalFriendEnabled {
entry := imNormalizeFriendAddEntry(source)
if imFriendEntryBlocked(policy.FriendDisabledPolicy, entry) {
return imDenyDecision(imReasonFriendAddDisabled, imPolicyString(policy.FriendDisabledPolicy, "message", "当前时段暂不允许添加好友"))
}
}
profile, err := store.Profiles.Get(tenantID, actor)
if err != nil {
logs.Warn.Printf("friend add user policy lookup failed: tenant=%d user=%s err=%v", tenantID, actor.UserId(), err)
return imAllowDecision()
}
if profile != nil && !profile.FriendAddEnabled {
message := strings.TrimSpace(profile.FriendAddDisabledReason)
if message == "" {
message = "该用户暂不允许发起加好友"
}
return imDenyDecision(imReasonFriendAddDisabled, message)
}
return imAllowDecision()
}
func imFriendGlobalDisabledDecision(tenantID types.TenantID, tenantCode, source string) imPolicyDecision {
policy, err := imSecurityPolicy(tenantID, tenantCode)
if err != nil {
if !errors.Is(err, types.ErrUnsupported) && !errors.Is(err, types.ErrNotFound) {
logs.Warn.Printf("friend global policy lookup failed: tenant=%d code=%s source=%s err=%v", tenantID, tenantCode, source, err)
}
return imAllowDecision()
}
if policy != nil && !policy.GlobalFriendEnabled && imFriendEntryBlocked(policy.FriendDisabledPolicy, imNormalizeFriendAddEntry(source)) {
return imDenyDecision(imReasonFriendAddDisabled, imPolicyString(policy.FriendDisabledPolicy, "message", "当前时段暂不允许添加好友"))
}
return imAllowDecision()
}
func imNormalizeFriendAddEntry(source string) string {
switch strings.TrimSpace(source) {
case "qr":
return "qrcode"
case "group":
return "group_member"
case "profile", "profile_card":
return "profile_card"
case "invite", "invite_auto_add":
return "invite_auto_add"
case "recommend", "recommendation":
return "recommendation"
default:
return "search"
}
}
func imFriendEntryBlocked(policy map[string]any, entry string) bool {
if len(policy) == 0 {
return true
}
raw, ok := policy["blocked_entries"]
if !ok {
return true
}
entries := imAnyStringSlice(raw)
if len(entries) == 0 {
return true
}
entry = strings.TrimSpace(entry)
for _, item := range entries {
if strings.TrimSpace(item) == entry {
return true
}
}
return false
}
func imMessageCurfewDecision(tenantID types.TenantID, tenantCode string, actor types.Uid, topic string, cat types.TopicCat) imPolicyDecision {
if cat != types.TopicCatP2P {
return imAllowDecision()
}
policy, err := imSecurityPolicy(tenantID, tenantCode)
if err != nil {
if !errors.Is(err, types.ErrUnsupported) && !errors.Is(err, types.ErrNotFound) {
logs.Warn.Printf("message curfew policy lookup failed: tenant=%d code=%s user=%s topic=%s err=%v", tenantID, tenantCode, actor.UserId(), topic, err)
}
return imAllowDecision()
}
if policy == nil || !policy.MessageCurfewEnabled {
return imAllowDecision()
}
scope := "private_message"
if cat == types.TopicCatGrp {
scope = "group_message"
}
for _, rule := range policy.CurfewRules {
if rule == nil || rule.Status != 1 || !imCurfewRuleScopeMatches(rule.Scope, scope) {
continue
}
if imCurfewWhitelisted(rule, actor, topic, cat) {
continue
}
if imCurfewRuleActive(rule, time.Now()) {
message := strings.TrimSpace(rule.Message)
if message == "" {
message = "当前时段暂不能发送消息"
}
return imDenyDecision(imReasonMessageCurfew, message)
}
}
return imAllowDecision()
}
func imCurfewRuleScopeMatches(ruleScope, scope string) bool {
ruleScope = strings.TrimSpace(ruleScope)
return ruleScope == "" || ruleScope == "all" || ruleScope == scope
}
func imCurfewWhitelisted(rule *types.MessageCurfewRule, actor types.Uid, topic string, cat types.TopicCat) bool {
uidValues := map[string]bool{
actor.UserId(): true,
actor.String(): true,
}
if raw := store.DecodeUid(actor); raw > 0 {
uidValues[strconv.FormatInt(raw, 10)] = true
}
for _, item := range rule.WhitelistUserIds {
if uidValues[strings.TrimSpace(item)] {
return true
}
}
if cat == types.TopicCatGrp {
for _, item := range rule.WhitelistGroupIds {
if strings.TrimSpace(item) == topic {
return true
}
}
}
return false
}
func imCurfewRuleActive(rule *types.MessageCurfewRule, now time.Time) bool {
loc := time.Local
if tz := strings.TrimSpace(rule.Timezone); tz != "" {
if loaded, err := time.LoadLocation(tz); err == nil {
loc = loaded
}
}
localNow := now.In(loc)
if !imCurfewDateMatches(rule, localNow) {
return false
}
start, okStart := imParseClock(rule.StartTime)
end, okEnd := imParseClock(rule.EndTime)
if !okStart || !okEnd {
return false
}
minute := localNow.Hour()*60 + localNow.Minute()
if start == end {
return true
}
if start < end {
return minute >= start && minute < end
}
return minute >= start || minute < end
}
func imCurfewDateMatches(rule *types.MessageCurfewRule, now time.Time) bool {
switch strings.TrimSpace(rule.DateType) {
case "", "daily":
return true
case "weekday":
return now.Weekday() >= time.Monday && now.Weekday() <= time.Friday
case "weekend":
return now.Weekday() == time.Saturday || now.Weekday() == time.Sunday
case "custom":
date := now.Format("2006-01-02")
for _, item := range rule.CustomDates {
if strings.TrimSpace(item) == date {
return true
}
}
return false
default:
if len(rule.Weekdays) == 0 {
return false
}
weekday := int(now.Weekday())
if weekday == 0 {
weekday = 7
}
for _, item := range rule.Weekdays {
if item == weekday {
return true
}
}
return false
}
}
func imParseClock(value string) (int, bool) {
value = strings.TrimSpace(value)
if value == "" {
return 0, false
}
parts := strings.Split(value, ":")
if len(parts) < 2 {
return 0, false
}
hour, errHour := strconv.Atoi(parts[0])
minute, errMinute := strconv.Atoi(parts[1])
if errHour != nil || errMinute != nil || hour < 0 || hour > 23 || minute < 0 || minute > 59 {
return 0, false
}
return hour*60 + minute, true
}
func imPolicyString(policy map[string]any, key, fallback string) string {
if value := strings.TrimSpace(fmt.Sprint(policy[key])); value != "" && value != "<nil>" {
return value
}
return fallback
}
func imAnyStringSlice(raw any) []string {
switch value := raw.(type) {
case []string:
return value
case []any:
out := make([]string, 0, len(value))
for _, item := range value {
if text := strings.TrimSpace(fmt.Sprint(item)); text != "" && text != "<nil>" {
out = append(out, text)
}
}
return out
default:
return nil
}
}
func writeIMPolicyDecision(wrt http.ResponseWriter, req *http.Request, decision imPolicyDecision) {
message := strings.TrimSpace(decision.Message)
if message == "" {
message = "permission denied"
}
reason := strings.TrimSpace(decision.Reason)
if reason == "" {
reason = imReasonPermissionDenied
}
details := map[string]any{
"reason": reason,
"message": message,
}
for key, value := range decision.Details {
details[key] = value
}
writeIMError(wrt, req, http.StatusForbidden, 30005, reason, message, details)
}
func imCurfewPolicyReply(msg *ClientComMessage, decision imPolicyDecision) *ServerComMessage {
reply := ErrPolicyReply(msg, types.TimeNow())
reply.Ctrl.Text = "message curfew active"
reply.Ctrl.Params = map[string]any{
"reason": decision.Reason,
"message": decision.Message,
}
return reply
}
func imYouthPolicyReply(msg *ClientComMessage, decision imPolicyDecision) *ServerComMessage {
reply := ErrPolicyReply(msg, types.TimeNow())
reply.Ctrl.Text = "restricted by youth mode"
params := map[string]any{
"reason": imReasonYouthRestricted,
"message": "restricted by youth mode",
}
if strings.TrimSpace(decision.Reason) != "" {
params["reason"] = decision.Reason
}
if strings.TrimSpace(decision.Message) != "" {
params["message"] = decision.Message
}
for key, value := range decision.Details {
params[key] = value
}
reply.Ctrl.Params = params
return reply
}
+35
View File
@@ -0,0 +1,35 @@
package main
import (
"testing"
"time"
"github.com/tinode/chat/server/store"
"github.com/tinode/chat/server/store/types"
)
func TestYouthModeNightQuietRestrictsCalls(t *testing.T) {
oldSecurity := store.Security
uid := types.Uid(42)
store.Security = &testSecurityStore{youthMode: &types.YouthModeSettings{
UserID: uid,
Enabled: true,
PolicyVersion: 3,
Restrictions: []string{"night_quiet", "content_filter", "payment_guard"},
}}
defer func() { store.Security = oldSecurity }()
quietTime := time.Date(2026, 9, 18, 23, 30, 0, 0, time.Local)
decision := imYouthModePolicyDecision(7, uid, imYouthActionCall, quietTime)
if decision.Allowed {
t.Fatal("night quiet should restrict call initiation")
}
if decision.Reason != imReasonYouthRestricted || decision.Details["restriction"] != "night_quiet" || decision.Details["policy_version"] != 3 {
t.Fatalf("unexpected decision: %#v", decision)
}
daytime := time.Date(2026, 9, 18, 10, 0, 0, 0, time.Local)
if decision = imYouthModePolicyDecision(7, uid, imYouthActionCall, daytime); !decision.Allowed {
t.Fatalf("daytime call should be allowed: %#v", decision)
}
}
+57 -2
View File
@@ -9,7 +9,9 @@
package main package main
import ( import (
"errors"
"strings" "strings"
"time"
"github.com/tinode/chat/server/auth" "github.com/tinode/chat/server/auth"
"github.com/tinode/chat/server/logs" "github.com/tinode/chat/server/logs"
@@ -602,9 +604,15 @@ func initTopicNewGrp(t *Topic, sreg *ClientComMessage, isChan bool) error {
Trusted: t.trusted, Trusted: t.trusted,
} }
// store.Topics.Create will add a subscription record for the topic creator // Persist the topic and its owner subscription together with product group metadata.
stopic.GiveAccess(t.owner, userData.modeWant, userData.modeGiven) stopic.GiveAccess(t.owner, userData.modeWant, userData.modeGiven)
err := store.Topics.Create(t.tenantID, stopic, t.owner, t.perUser[t.owner].private) var err error
if !isChan && store.GroupBusiness != nil {
err = store.GroupBusiness.CreateGroup(t.tenantID, stopic, t.owner, t.perUser[t.owner].private,
newWebsocketGroupSettings(stopic, t.owner, timestamp), nil, nil, timestamp)
} else {
err = store.Topics.Create(t.tenantID, stopic, t.owner, t.perUser[t.owner].private)
}
if err != nil { if err != nil {
return err return err
} }
@@ -626,6 +634,28 @@ func initTopicNewGrp(t *Topic, sreg *ClientComMessage, isChan bool) error {
return nil return nil
} }
func newWebsocketGroupSettings(topic *types.Topic, owner types.Uid, now time.Time) *types.GroupSettings {
public := groupPublicMap(topic.Public)
description, _ := imPublicString(public, "description")
return &types.GroupSettings{
Topic: topic.Id,
OwnerUserID: owner,
Name: groupName(public, topic.Id),
AvatarFileID: imAvatarFileIDFromPublic(public),
Description: description,
GroupStatus: types.GroupStatusActive,
MemberUploadFiles: true,
MemberPrivateChat: true,
MemberInvite: true,
MaxAdminCount: groupMaxAdminCount,
MemberCount: 1,
Version: 1,
CreatedBy: owner,
CreatedAt: now,
UpdatedAt: now,
}
}
// Initialize existing group topic. There is a race condition when two users attempt to load // Initialize existing group topic. There is a race condition when two users attempt to load
// the same topic at the same time. It's prevented at hub level. // the same topic at the same time. It's prevented at hub level.
func initTopicGrp(t *Topic) error { func initTopicGrp(t *Topic) error {
@@ -671,6 +701,11 @@ func initTopicGrp(t *Topic) error {
t.lastID = stopic.SeqId t.lastID = stopic.SeqId
t.delID = stopic.DelId t.delID = stopic.DelId
t.subCnt = stopic.SubCnt t.subCnt = stopic.SubCnt
if !t.isChan {
if err = ensureLegacyWebsocketGroupSettings(t.tenantID, stopic, t.owner, len(t.perUser)); err != nil {
return err
}
}
// Initialize channel for receiving session online updates. // Initialize channel for receiving session online updates.
t.supd = make(chan *sessionUpdate, 32) t.supd = make(chan *sessionUpdate, 32)
@@ -680,6 +715,26 @@ func initTopicGrp(t *Topic) error {
return nil return nil
} }
func ensureLegacyWebsocketGroupSettings(tenantID types.TenantID, topic *types.Topic, owner types.Uid, memberCount int) error {
if store.GroupBusiness == nil || topic == nil || owner.IsZero() {
return nil
}
if _, err := store.GroupBusiness.GetSettings(tenantID, topic.Id); err == nil {
return nil
} else if errors.Is(err, types.ErrUnsupported) {
return nil
} else if !errors.Is(err, types.ErrNotFound) {
return err
}
settings := newWebsocketGroupSettings(topic, owner, topic.CreatedAt)
settings.MemberCount = memberCount
if !topic.UpdatedAt.IsZero() {
settings.UpdatedAt = topic.UpdatedAt
}
_, err := store.GroupBusiness.CreateSettings(tenantID, settings)
return err
}
// Initialize system topic. System topic is a singleton, always in memory. // Initialize system topic. System topic is a singleton, always in memory.
func initTopicSys(t *Topic) error { func initTopicSys(t *Topic) error {
t.cat = types.TopicCatSys t.cat = types.TopicCatSys
+107
View File
@@ -0,0 +1,107 @@
package main
import (
"testing"
"time"
"github.com/tinode/chat/server/auth"
"github.com/tinode/chat/server/store"
"github.com/tinode/chat/server/store/types"
)
func TestInitTopicNewGrpCreatesBusinessSettings(t *testing.T) {
previous := store.GroupBusiness
groupStore := &testGroupBusinessStore{}
store.GroupBusiness = groupStore
defer func() { store.GroupBusiness = previous }()
owner := types.Uid(42)
topic := &Topic{
tenantID: 7,
name: "grpWebsocket",
xoriginal: "new",
perUser: make(map[types.Uid]perUserData),
}
request := &ClientComMessage{
RcptTo: topic.name,
AsUser: owner.UserId(),
AuthLvl: int(auth.LevelAuth),
Sub: &MsgClientSub{Topic: "new", Set: &MsgSetQuery{Desc: &MsgSetDesc{Public: map[string]any{
"fn": "WebSocket 测试群",
"description": "业务资料需要原子创建",
}}}},
}
if err := initTopicNewGrp(topic, request, false); err != nil {
t.Fatal(err)
}
if groupStore.settings == nil || groupStore.settings.Topic != topic.name || groupStore.settings.OwnerUserID != owner {
t.Fatalf("business settings were not created: %#v", groupStore.settings)
}
if !request.Sub.Created || !request.Sub.Newsub || topic.subCnt != 1 || topic.xoriginal != topic.name {
t.Fatalf("unexpected initialized topic: created=%v newsub=%v subcnt=%d original=%q",
request.Sub.Created, request.Sub.Newsub, topic.subCnt, topic.xoriginal)
}
}
func TestNewWebsocketGroupSettings(t *testing.T) {
now := types.TimeNow()
owner := types.Uid(42)
topic := &types.Topic{
ObjHeader: types.ObjHeader{Id: "grpWebsocket"},
Public: map[string]any{
"fn": "WebSocket 测试群",
"avatar_file_id": "avatar.png",
"description": "通过 sub topic:new 创建",
},
}
settings := newWebsocketGroupSettings(topic, owner, now)
if settings.Topic != topic.Id || settings.OwnerUserID != owner || settings.CreatedBy != owner {
t.Fatalf("unexpected group identity: %#v", settings)
}
if settings.Name != "WebSocket 测试群" || settings.AvatarFileID != "avatar.png" ||
settings.Description != "通过 sub topic:new 创建" {
t.Fatalf("unexpected group profile: %#v", settings)
}
if settings.GroupStatus != types.GroupStatusActive || !settings.MemberUploadFiles ||
!settings.MemberPrivateChat || !settings.MemberInvite || settings.MemberCount != 1 ||
settings.MaxAdminCount != groupMaxAdminCount || settings.Version != 1 {
t.Fatalf("unexpected group defaults: %#v", settings)
}
if !settings.CreatedAt.Equal(now) || !settings.UpdatedAt.Equal(now) {
t.Fatalf("unexpected group timestamps: %#v", settings)
}
}
func TestNewWebsocketGroupSettingsFallsBackToTopicName(t *testing.T) {
topic := &types.Topic{ObjHeader: types.ObjHeader{Id: "grpFallback"}}
settings := newWebsocketGroupSettings(topic, types.Uid(7), types.TimeNow())
if settings.Name != topic.Id {
t.Fatalf("name: got %q, want %q", settings.Name, topic.Id)
}
}
func TestEnsureLegacyWebsocketGroupSettingsRepairsMissingRecord(t *testing.T) {
previous := store.GroupBusiness
groupStore := &testGroupBusinessStore{getSettingsErr: types.ErrNotFound}
store.GroupBusiness = groupStore
defer func() { store.GroupBusiness = previous }()
created := types.TimeNow().Add(-time.Hour)
updated := types.TimeNow()
topic := &types.Topic{
ObjHeader: types.ObjHeader{Id: "grpLegacy", CreatedAt: created, UpdatedAt: updated},
Public: map[string]any{"fn": "历史 WebSocket 群"},
}
owner := types.Uid(42)
if err := ensureLegacyWebsocketGroupSettings(7, topic, owner, 3); err != nil {
t.Fatal(err)
}
if groupStore.settings == nil || groupStore.settings.Topic != topic.Id || groupStore.settings.MemberCount != 3 {
t.Fatalf("legacy settings were not repaired: %#v", groupStore.settings)
}
if !groupStore.settings.CreatedAt.Equal(created) || !groupStore.settings.UpdatedAt.Equal(updated) {
t.Fatalf("legacy timestamps were not preserved: %#v", groupStore.settings)
}
}
+94 -39
View File
@@ -1,6 +1,7 @@
package main package main
import ( import (
"encoding/json"
"fmt" "fmt"
"net/http" "net/http"
"strconv" "strconv"
@@ -68,7 +69,7 @@ var internalTenantResourceSpecs = map[string]*internalQuerySpec{
Filters: []internalQueryFilter{{"user_id", "user_id", "=", "uid"}, {"account", "account_mask", "LIKE", "string"}, {"login_type", "login_type", "=", "string"}, {"event_type", "event_type", "=", "string"}, {"platform", "platform", "=", "string"}, {"ip", "ip", "LIKE", "string"}, {"login_at_start", "login_at", ">=", "time"}, {"login_at_end", "login_at", "<=", "time"}}, Filters: []internalQueryFilter{{"user_id", "user_id", "=", "uid"}, {"account", "account_mask", "LIKE", "string"}, {"login_type", "login_type", "=", "string"}, {"event_type", "event_type", "=", "string"}, {"platform", "platform", "=", "string"}, {"ip", "ip", "LIKE", "string"}, {"login_at_start", "login_at", ">=", "time"}, {"login_at_end", "login_at", "<=", "time"}},
}, },
"login-failure-logs": { "login-failure-logs": {
Select: `SELECT id,CAST(user_id AS CHAR) AS user_id,account_type,account_mask,failure_type,reason,detail,ip,region,device_id,platform,app_version,user_agent,risk_level,risk_action,request_id,attempt_at,metadata`, Select: `SELECT id,CAST(user_id AS CHAR) AS user_id,account_type,account_mask,account_hash,failure_type,reason,detail,ip,region,device_id,platform,app_version,user_agent,risk_level,risk_action,request_id,attempt_at,metadata`,
From: "FROM im_login_failure_logs", DefaultSort: "ORDER BY id DESC", DetailKey: "id", From: "FROM im_login_failure_logs", DefaultSort: "ORDER BY id DESC", DetailKey: "id",
Filters: []internalQueryFilter{{"user_id", "user_id", "=", "uid"}, {"account", "account_mask", "LIKE", "string"}, {"failure_type", "failure_type", "=", "string"}, {"platform", "platform", "=", "string"}, {"ip", "ip", "LIKE", "string"}, {"attempt_at_start", "attempt_at", ">=", "time"}, {"attempt_at_end", "attempt_at", "<=", "time"}}, Filters: []internalQueryFilter{{"user_id", "user_id", "=", "uid"}, {"account", "account_mask", "LIKE", "string"}, {"failure_type", "failure_type", "=", "string"}, {"platform", "platform", "=", "string"}, {"ip", "ip", "LIKE", "string"}, {"attempt_at_start", "attempt_at", ">=", "time"}, {"attempt_at_end", "attempt_at", "<=", "time"}},
}, },
@@ -96,12 +97,7 @@ var internalTenantResourceSpecs = map[string]*internalQuerySpec{
"feedback-config": { "feedback-config": {
Select: "SELECT id,categories,max_length,attachment_policy,allow_contact,diagnostics_notice,created_at,updated_at", Select: "SELECT id,categories,max_length,attachment_policy,allow_contact,diagnostics_notice,created_at,updated_at",
From: "FROM im_feedback_config", DefaultSort: "ORDER BY id DESC", DetailKey: "id", From: "FROM im_feedback_config", DefaultSort: "ORDER BY id DESC", DetailKey: "id",
}, }, "abuse-reasons": {
"mailbox-config": {
Select: "SELECT id,enabled,allow_anonymous,required_fields,max_length,attachment_policy,notice_title,notice_content,notice_updated_at,created_at,updated_at",
From: "FROM im_mailbox_config", DefaultSort: "ORDER BY id DESC", DetailKey: "id",
},
"abuse-reasons": {
Select: "SELECT id,target_type,reason_code,label,requires_description,allows_evidence,enabled,sort_order,created_at,updated_at", Select: "SELECT id,target_type,reason_code,label,requires_description,allows_evidence,enabled,sort_order,created_at,updated_at",
From: "FROM im_abuse_reasons", DefaultSort: "ORDER BY sort_order ASC,id ASC", DetailKey: "id", From: "FROM im_abuse_reasons", DefaultSort: "ORDER BY sort_order ASC,id ASC", DetailKey: "id",
Filters: []internalQueryFilter{{"target_type", "target_type", "=", "string"}}, Filters: []internalQueryFilter{{"target_type", "target_type", "=", "string"}},
@@ -109,28 +105,10 @@ var internalTenantResourceSpecs = map[string]*internalQuerySpec{
"app-releases": { "app-releases": {
Select: "SELECT id,platform,channel,latest_version,min_supported_version,update_type,release_notes,store_url,package_url,checksum,remind_after_seconds,published_at,created_at,updated_at", Select: "SELECT id,platform,channel,latest_version,min_supported_version,update_type,release_notes,store_url,package_url,checksum,remind_after_seconds,published_at,created_at,updated_at",
From: "FROM im_app_releases", DefaultSort: "ORDER BY id DESC", DetailKey: "id", From: "FROM im_app_releases", DefaultSort: "ORDER BY id DESC", DetailKey: "id",
}, }}
"workbench/apps": {
Select: "SELECT id,code AS app_code,name,icon_file_id,route,sort_order,unread_key,enabled,permission,created_at,updated_at",
From: "FROM im_workbench_apps", DefaultSort: "ORDER BY sort_order ASC,id ASC", DetailKey: "id",
},
}
var internalOutboxResourceSpecs = map[string]*internalQuerySpec{} var internalOutboxResourceSpecs = map[string]*internalQuerySpec{}
var internalOrgResourceSpecs = map[string]*internalQuerySpec{
"departments": {
Select: "SELECT d.id,d.department_id,d.parent_id,d.name,d.sort_order,d.version,d.state,d.created_at,d.updated_at,(SELECT COUNT(*) FROM im_org_departments child WHERE child.parent_id=d.department_id AND child.state='active') AS child_count,(SELECT COUNT(*) FROM im_org_members m WHERE m.department_id=d.department_id) AS member_count",
From: "FROM im_org_departments d", DefaultSort: "ORDER BY d.sort_order ASC,d.id ASC",
Filters: []internalQueryFilter{{"department_id", "d.department_id", "=", "string"}, {"parent_id", "d.parent_id", "=", "string"}, {"name", "d.name", "LIKE", "string"}, {"state", "d.state", "=", "string"}},
},
"members": {
Select: "SELECT m.id,m.department_id,COALESCE(d.name,'') AS department_name,m.user_id,m.display_name,m.position,m.avatar_file_id,m.mobile,m.email,m.employment_state,m.contactable,m.sort_order,m.version,m.created_at,m.updated_at",
From: "FROM im_org_members m LEFT JOIN im_org_departments d ON d.department_id=m.department_id", DefaultSort: "ORDER BY m.sort_order ASC,m.id ASC",
Filters: []internalQueryFilter{{"department_id", "m.department_id", "=", "string"}, {"user_id", "m.user_id", "=", "uid"}, {"q", "m.display_name", "LIKE", "string"}, {"employment_state", "m.employment_state", "=", "string"}},
},
}
func serveInternalAdminResourceList(wrt http.ResponseWriter, req *http.Request, tenantID types.TenantID, resource, scope string) { func serveInternalAdminResourceList(wrt http.ResponseWriter, req *http.Request, tenantID types.TenantID, resource, scope string) {
spec, ok := internalAdminQuerySpec(resource) spec, ok := internalAdminQuerySpec(resource)
if !ok { if !ok {
@@ -150,11 +128,23 @@ func serveInternalAdminResourceList(wrt http.ResponseWriter, req *http.Request,
writeIMError(wrt, req, http.StatusBadRequest, 30000, imReasonMalformedRequest, "malformed request", nil) writeIMError(wrt, req, http.StatusBadRequest, 30000, imReasonMalformedRequest, "malformed request", nil)
return return
} }
result, err := store.InternalAdminQueries.Query(tenantID, query, countQuery, args, page, perPage) var result *types.InternalAdminQueryResult
keyword := strings.TrimSpace(req.URL.Query().Get("keyword"))
if (resource == "private-messages" || resource == "group-messages") && keyword != "" {
result, err = queryEncryptedAdminMessages(req, tenantID, spec, scope, keyword, page, perPage)
} else {
result, err = store.InternalAdminQueries.Query(tenantID, query, countQuery, args, page, perPage)
}
if err != nil { if err != nil {
writeInternalQueryError(wrt, req, err) writeInternalQueryError(wrt, req, err)
return return
} }
if resource == "private-messages" || resource == "group-messages" {
if err = decryptInternalAdminMessageRows(tenantID, result.Items); err != nil {
writeInternalQueryError(wrt, req, err)
return
}
}
writeIMSuccess(wrt, req, map[string]any{ writeIMSuccess(wrt, req, map[string]any{
"items": normalizeInternalAdminRowsForAPI(result.Items), "items": normalizeInternalAdminRowsForAPI(result.Items),
"total": result.Total, "total": result.Total,
@@ -163,6 +153,81 @@ func serveInternalAdminResourceList(wrt http.ResponseWriter, req *http.Request,
}) })
} }
func queryEncryptedAdminMessages(req *http.Request, tenantID types.TenantID, spec *internalQuerySpec, scope, keyword string, page, perPage int) (*types.InternalAdminQueryResult, error) {
requestCopy := req.Clone(req.Context())
urlCopy := *req.URL
queryValues := req.URL.Query()
queryValues.Del("keyword")
urlCopy.RawQuery = queryValues.Encode()
requestCopy.URL = &urlCopy
query, countQuery, args, _, _, err := buildInternalAdminListQuery(requestCopy, spec, scope)
if err != nil {
return nil, err
}
const batchSize = 200
start := (page - 1) * perPage
end := start + perPage
items := make([]map[string]any, 0, perPage)
matched := 0
for batchPage := 1; ; batchPage++ {
batch, queryErr := store.InternalAdminQueries.Query(tenantID, query, countQuery, args, batchPage, batchSize)
if queryErr != nil {
return nil, queryErr
}
if err = decryptInternalAdminMessageRows(tenantID, batch.Items); err != nil {
return nil, err
}
for _, row := range batch.Items {
if !strings.Contains(strings.ToLower(fmt.Sprint(row["content_excerpt"])), strings.ToLower(keyword)) {
continue
}
if matched >= start && matched < end {
items = append(items, row)
}
matched++
}
if len(batch.Items) < batchSize || int64(batchPage*batchSize) >= batch.Total {
break
}
}
return &types.InternalAdminQueryResult{Items: items, Total: int64(matched), Page: page, PerPage: perPage}, nil
}
func decryptInternalAdminMessageRows(tenantID types.TenantID, rows []map[string]any) error {
for _, row := range rows {
seq, ok := numericID(row["seq_id"])
if !ok || seq <= 0 {
continue
}
msg, err := store.Messages.GetOne(tenantID, fmt.Sprint(row["topic"]), seq)
if err != nil {
return err
}
row["content_excerpt"] = internalMessageExcerpt(msg.Content)
}
return nil
}
func internalMessageExcerpt(content any) string {
if value, ok := content.(map[string]any); ok {
for _, key := range []string{"txt", "text", "content", "title"} {
if text := strings.TrimSpace(fmt.Sprint(value[key])); text != "" && text != "<nil>" {
if len([]rune(text)) > 500 {
return string([]rune(text)[:500])
}
return text
}
}
}
raw, _ := json.Marshal(content)
text := string(raw)
if len([]rune(text)) > 500 {
text = string([]rune(text)[:500])
}
return text
}
func serveInternalAdminResourceDetail(wrt http.ResponseWriter, req *http.Request, tenantID types.TenantID, resource, id string) { func serveInternalAdminResourceDetail(wrt http.ResponseWriter, req *http.Request, tenantID types.TenantID, resource, id string) {
spec, ok := internalAdminQuerySpec(resource) spec, ok := internalAdminQuerySpec(resource)
if !ok || spec.DetailKey == "" { if !ok || spec.DetailKey == "" {
@@ -255,27 +320,17 @@ func internalAdminQuerySpec(resource string) (*internalQuerySpec, bool) {
spec := internalOutboxResourceSpecs[strings.TrimPrefix(resource, "outbox_")] spec := internalOutboxResourceSpecs[strings.TrimPrefix(resource, "outbox_")]
return spec, spec != nil return spec, spec != nil
} }
if strings.HasPrefix(resource, "org_") {
spec := internalOrgResourceSpecs[strings.TrimPrefix(resource, "org_")]
return spec, spec != nil
}
spec := internalTenantResourceSpecs[resource] spec := internalTenantResourceSpecs[resource]
return spec, spec != nil return spec, spec != nil
} }
func usersInternalQuerySpec() *internalQuerySpec { func usersInternalQuerySpec() *internalQuerySpec {
return &internalQuerySpec{ return &internalQuerySpec{
Select: `SELECT CAST(u.id AS CHAR) AS user_id,u.account_no AS display_no,u.username,u.phone,u.state,u.stateat AS state_at,u.access,u.lastseen AS last_seen,u.useragent AS user_agent,u.public,u.trusted, Select: `SELECT CAST(u.id AS CHAR) AS user_id,u.account_no AS display_no,u.username,u.phone,u.state,u.stateat AS state_at,u.access,u.lastseen AS last_seen,u.useragent AS user_agent,u.trusted,
p.nickname,p.public_account,p.gender,p.region,p.avatar_file_id, p.nickname,p.public_account,p.gender,p.region,p.avatar_file_id,COALESCE(p.friend_add_enabled,1) AS friend_add_enabled,
COALESCE( COALESCE(
NULLIF(avatar.location,''), NULLIF(avatar.location,''),
NULLIF(CASE WHEN p.avatar_file_id REGEXP '^(https?://|data:)' THEN p.avatar_file_id ELSE '' END,''), NULLIF(CASE WHEN p.avatar_file_id REGEXP '^(https?://|data:)' THEN p.avatar_file_id ELSE '' END,''),
NULLIF(JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.avatar')),''),
NULLIF(CASE WHEN JSON_TYPE(JSON_EXTRACT(u.public,'$.photo'))='STRING' THEN JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.photo')) ELSE '' END,''),
CASE WHEN JSON_TYPE(JSON_EXTRACT(u.public,'$.photo'))='OBJECT' AND NULLIF(JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.photo.data')),'') IS NOT NULL
THEN CONCAT('data:image/',COALESCE(NULLIF(JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.photo.type')),''),'jpeg'),';base64,',JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.photo.data'))) ELSE NULL END,
NULLIF(CASE WHEN JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.avatar_file_id')) REGEXP '^(https?://|data:)' THEN JSON_UNQUOTE(JSON_EXTRACT(u.public,'$.avatar_file_id')) ELSE '' END,''),
NULLIF((SELECT CASE WHEN om.avatar_file_id REGEXP '^(https?://|data:)' THEN om.avatar_file_id ELSE '' END FROM im_org_members AS om WHERE om.user_id=u.id AND om.avatar_file_id<>'' ORDER BY om.id LIMIT 1),''),
'' ''
) AS avatar, ) AS avatar,
p.signature,sec.auth_version,sec.password_updated_at,u.createdat AS created_at,u.updatedat AS updated_at, p.signature,sec.auth_version,sec.password_updated_at,u.createdat AS created_at,u.updatedat AS updated_at,
+44 -6
View File
@@ -26,6 +26,7 @@ import (
const ( const (
internalCapabilityAdminQuery = "admin_query" internalCapabilityAdminQuery = "admin_query"
internalCapabilityCultivate = "cultivate"
internalHeaderService = "X-Service-Name" internalHeaderService = "X-Service-Name"
internalHeaderTimestamp = "X-Timestamp" internalHeaderTimestamp = "X-Timestamp"
@@ -116,6 +117,8 @@ func serveInternalAdminQuery(wrt http.ResponseWriter, req *http.Request, prefix
serveInternalUserCreate(wrt, req, tenant.ID) serveInternalUserCreate(wrt, req, tenant.ID)
case req.Method == http.MethodPost && len(parts) == 5 && parts[2] == "users" && parts[4] == "action": case req.Method == http.MethodPost && len(parts) == 5 && parts[2] == "users" && parts[4] == "action":
serveInternalUserAction(wrt, req, tenant.ID, parts[3]) serveInternalUserAction(wrt, req, tenant.ID, parts[3])
case req.Method == http.MethodPost && len(parts) == 4 && parts[2] == "login-lockouts" && parts[3] == "release":
serveInternalLoginLockoutRelease(wrt, req, tenant.ID)
case req.Method == http.MethodGet && len(parts) == 3 && parts[2] == "users": case req.Method == http.MethodGet && len(parts) == 3 && parts[2] == "users":
serveInternalAdminResourceList(wrt, req, tenant.ID, "users", "") serveInternalAdminResourceList(wrt, req, tenant.ID, "users", "")
case len(parts) == 4 && parts[2] == "users": case len(parts) == 4 && parts[2] == "users":
@@ -150,10 +153,6 @@ func serveInternalAdminQuery(wrt http.ResponseWriter, req *http.Request, prefix
serveInternalAdminResourceList(wrt, req, tenant.ID, "outbox_"+parts[3], "") serveInternalAdminResourceList(wrt, req, tenant.ID, "outbox_"+parts[3], "")
case len(parts) == 5 && parts[2] == "outbox" && internalOutboxResourceSpecs[parts[3]] != nil: case len(parts) == 5 && parts[2] == "outbox" && internalOutboxResourceSpecs[parts[3]] != nil:
serveInternalAdminResourceDetail(wrt, req, tenant.ID, "outbox_"+parts[3], parts[4]) serveInternalAdminResourceDetail(wrt, req, tenant.ID, "outbox_"+parts[3], parts[4])
case len(parts) == 4 && parts[2] == "org" && internalOrgResourceSpecs[parts[3]] != nil:
serveInternalAdminResourceList(wrt, req, tenant.ID, "org_"+parts[3], "")
case len(parts) == 4 && parts[2] == "workbench" && parts[3] == "apps":
serveInternalAdminResourceList(wrt, req, tenant.ID, "workbench/apps", "")
case len(parts) == 4 && parts[2] == "dashboard" && parts[3] == "summary": case len(parts) == 4 && parts[2] == "dashboard" && parts[3] == "summary":
serveInternalDashboardSummary(wrt, req, tenant.ID) serveInternalDashboardSummary(wrt, req, tenant.ID)
case len(parts) == 4 && parts[2] == "kafka" && parts[3] == "dead-letters": case len(parts) == 4 && parts[2] == "kafka" && parts[3] == "dead-letters":
@@ -167,7 +166,8 @@ func serveInternalAdminQuery(wrt http.ResponseWriter, req *http.Request, prefix
func isInternalAdminPostRoute(parts []string) bool { func isInternalAdminPostRoute(parts []string) bool {
return (len(parts) == 3 && parts[2] == "users") || return (len(parts) == 3 && parts[2] == "users") ||
(len(parts) == 5 && parts[2] == "users" && parts[4] == "action") (len(parts) == 5 && parts[2] == "users" && parts[4] == "action") ||
(len(parts) == 4 && parts[2] == "login-lockouts" && parts[3] == "release")
} }
type internalUserCreateRequest struct { type internalUserCreateRequest struct {
@@ -332,6 +332,37 @@ func isInternalUserActionSupported(eventType string) bool {
} }
} }
type internalLoginLockoutReleaseRequest struct {
AccountHash string `json:"account_hash"`
}
// serveInternalLoginLockoutRelease clears the login failure lockout for one
// account so the user can sign in again immediately.
func serveInternalLoginLockoutRelease(wrt http.ResponseWriter, req *http.Request, tenantID types.TenantID) {
var body internalLoginLockoutReleaseRequest
if err := json.NewDecoder(io.LimitReader(req.Body, 1<<20)).Decode(&body); err != nil {
writeIMError(wrt, req, http.StatusBadRequest, 30000, imReasonMalformedRequest, "malformed request", nil)
return
}
accountHash := strings.ToLower(strings.TrimSpace(body.AccountHash))
if len(accountHash) != 64 {
writeIMError(wrt, req, http.StatusBadRequest, 30000, imReasonMalformedRequest, "account_hash 格式不正确", nil)
return
}
rec, err := store.LoginLockouts.Get(tenantID, accountHash)
if err != nil && !errors.Is(err, types.ErrNotFound) && !errors.Is(err, types.ErrUnsupported) {
writeInternalQueryError(wrt, req, err)
return
}
if err = store.LoginLockouts.Release(tenantID, accountHash, time.Now().UTC().Round(time.Millisecond)); err != nil {
writeInternalQueryError(wrt, req, err)
return
}
logs.Info.Printf("login lockout released tenant:%d account_hash:%s had_lock:%v", tenantID, accountHash,
rec != nil && rec.LockedUntil != nil)
writeIMSuccess(wrt, req, map[string]any{"account_hash": accountHash, "status": "released"})
}
func authorizeInternalAPI(wrt http.ResponseWriter, req *http.Request, capability string) bool { func authorizeInternalAPI(wrt http.ResponseWriter, req *http.Request, capability string) bool {
if !internalAPI.config.Enabled { if !internalAPI.config.Enabled {
writeIMError(wrt, req, http.StatusNotFound, 30004, imReasonMalformedRequest, "resource not found", nil) writeIMError(wrt, req, http.StatusNotFound, 30004, imReasonMalformedRequest, "resource not found", nil)
@@ -492,7 +523,7 @@ func serveInternalGroupMembersQuery(wrt http.ResponseWriter, req *http.Request,
return return
} }
spec := &internalQuerySpec{ spec := &internalQuerySpec{
Select: `SELECT gm.id,gm.topic,gm.user_id,u.state,u.public,p.nickname,p.public_account,s.modewant AS mode_want,s.modegiven AS mode_given,s.recvseqid AS recv_seq_id,s.readseqid AS read_seq_id,s.delid AS del_id, Select: `SELECT gm.id,gm.topic,gm.user_id,u.state,p.nickname,p.public_account,p.avatar_file_id,p.signature,p.gender,p.region,s.modewant AS mode_want,s.modegiven AS mode_given,s.recvseqid AS recv_seq_id,s.readseqid AS read_seq_id,s.delid AS del_id,
gm.role,gm.member_status,gm.nickname AS group_nickname,gm.pinned,gm.mute_notifications,gm.join_source,gm.joined_by,gm.joined_at,gm.left_at,gm.removed_by,gm.version,gm.created_at,gm.updated_at`, gm.role,gm.member_status,gm.nickname AS group_nickname,gm.pinned,gm.mute_notifications,gm.join_source,gm.joined_by,gm.joined_at,gm.left_at,gm.removed_by,gm.version,gm.created_at,gm.updated_at`,
From: "FROM im_group_members AS gm LEFT JOIN im_users AS u ON u.id=gm.user_id LEFT JOIN im_user_profiles AS p ON p.user_id=gm.user_id LEFT JOIN im_subscriptions AS s ON s.topic=gm.topic AND s.userid=gm.user_id", From: "FROM im_group_members AS gm LEFT JOIN im_users AS u ON u.id=gm.user_id LEFT JOIN im_user_profiles AS p ON p.user_id=gm.user_id LEFT JOIN im_subscriptions AS s ON s.topic=gm.topic AND s.userid=gm.user_id",
DefaultSort: "ORDER BY gm.id DESC", DefaultSort: "ORDER BY gm.id DESC",
@@ -576,6 +607,13 @@ func serveInternalMessageQuery(wrt http.ResponseWriter, req *http.Request, tenan
writeInternalQueryError(wrt, req, err) writeInternalQueryError(wrt, req, err)
return return
} }
msg, err := store.Messages.GetOne(tenantID, topic, seq)
if err != nil {
writeInternalQueryError(wrt, req, err)
return
}
row["content"] = msg.Content
row["content_excerpt"] = internalMessageExcerpt(msg.Content)
row["conversation_type"] = strings.TrimSuffix(resource, "-messages") row["conversation_type"] = strings.TrimSuffix(resource, "-messages")
writeIMSuccess(wrt, req, normalizeInternalAdminRowForAPI(row)) writeIMSuccess(wrt, req, normalizeInternalAdminRowForAPI(row))
} }
+275
View File
@@ -0,0 +1,275 @@
package main
import (
"errors"
"net/http"
"strconv"
"strings"
"time"
"github.com/tinode/chat/server/moderation"
"github.com/tinode/chat/server/push"
"github.com/tinode/chat/server/store"
"github.com/tinode/chat/server/store/types"
)
type cultivateUsersRequest struct {
UserIDs []int64 `json:"user_ids"`
}
type cultivateRelationsRequest struct {
ViewerID int64 `json:"viewer_id"`
PeerIDs []int64 `json:"peer_ids"`
}
type cultivateFilesRequest struct {
UserID int64 `json:"user_id"`
FileIDs []string `json:"file_ids"`
}
type cultivateReviewRequest struct {
ObjectType string `json:"object_type"`
ObjectID int64 `json:"object_id"`
ActorID int64 `json:"actor_id"`
Text string `json:"text"`
TraceID string `json:"trace_id"`
}
type cultivateEventRequest struct {
EventID string `json:"event_id"`
EventType string `json:"event_type"`
ActorID int64 `json:"actor_id"`
TargetUserID int64 `json:"target_user_id"`
Title string `json:"title"`
Summary string `json:"summary"`
Action map[string]any `json:"action"`
Payload map[string]any `json:"payload"`
}
func serveInternalCultivate(w http.ResponseWriter, r *http.Request, prefix string) {
if r.Method != http.MethodPost {
writeIMError(w, r, http.StatusMethodNotAllowed, 30002, imReasonMethodNotAllowed, "method not allowed", nil)
return
}
if !authorizeInternalAPI(w, r, internalCapabilityCultivate) {
return
}
parts := splitInternalPath(strings.Trim(strings.TrimPrefix(r.URL.Path, prefix), "/"))
if len(parts) != 4 || parts[0] != "tenants" {
writeIMError(w, r, http.StatusNotFound, 30004, imReasonMalformedRequest, "resource not found", nil)
return
}
tenant, ok := resolveCultivateTenant(w, r, parts[1])
if !ok {
return
}
switch parts[2] + "/" + parts[3] {
case "users/batch":
serveCultivateUsers(w, r, tenant.ID)
case "relations/check":
serveCultivateRelations(w, r, tenant.ID)
case "files/validate":
serveCultivateFiles(w, r, tenant.ID)
case "moderation/review":
serveCultivateReview(w, r, tenant)
case "events/publish":
serveCultivateEvent(w, r, tenant.ID)
default:
writeIMError(w, r, http.StatusNotFound, 30004, imReasonMalformedRequest, "resource not found", nil)
}
}
func resolveCultivateTenant(w http.ResponseWriter, r *http.Request, value string) (*types.Tenant, bool) {
if id, err := strconv.ParseInt(value, 10, 64); err == nil && id > 0 && store.Tenants != nil {
tenant, lookupErr := store.Tenants.GetByID(types.TenantID(id))
if lookupErr == nil && tenant != nil && tenant.IsActive() {
return tenant, true
}
writeIMError(w, r, http.StatusNotFound, 30010, imReasonTenantUnavailable, "tenant unavailable", nil)
return nil, false
}
return resolveInternalTenant(w, r, value)
}
func serveCultivateUsers(w http.ResponseWriter, r *http.Request, tenantID types.TenantID) {
var body cultivateUsersRequest
if !decodeIMJSONBody(w, r, &body) {
return
}
if len(body.UserIDs) == 0 || len(body.UserIDs) > 200 {
writeIMError(w, r, http.StatusBadRequest, 30000, imReasonMalformedRequest, "malformed request", nil)
return
}
items := make([]map[string]any, 0, len(body.UserIDs))
for _, rawID := range body.UserIDs {
if rawID <= 0 {
continue
}
uid := store.EncodeUid(rawID)
user, err := store.Users.Get(tenantID, uid)
if err != nil || user == nil {
continue
}
item := map[string]any{"user_id": rawID, "state": user.State.String(), "nickname": "", "avatar_file_id": "", "youth_mode": false}
if profile, err := store.Profiles.Get(tenantID, uid); err == nil && profile != nil {
item["nickname"], item["avatar_file_id"] = profile.Nickname, profile.AvatarFileID
}
if settings, err := store.Security.GetYouthMode(tenantID, uid); err == nil && settings != nil {
item["youth_mode"] = settings.Enabled
}
items = append(items, item)
}
writeIMSuccess(w, r, map[string]any{"items": items})
}
func serveCultivateRelations(w http.ResponseWriter, r *http.Request, tenantID types.TenantID) {
var body cultivateRelationsRequest
if !decodeIMJSONBody(w, r, &body) {
return
}
if body.ViewerID <= 0 || len(body.PeerIDs) > 500 {
writeIMError(w, r, http.StatusBadRequest, 30000, imReasonMalformedRequest, "malformed request", nil)
return
}
viewer := store.EncodeUid(body.ViewerID)
items := make([]map[string]any, 0, len(body.PeerIDs))
for _, rawID := range body.PeerIDs {
if rawID <= 0 || rawID == body.ViewerID {
continue
}
peer := store.EncodeUid(rawID)
ctx, err := store.Relations.GetRelationshipContext(tenantID, viewer, peer, viewer.P2PName(peer))
if err != nil || ctx == nil {
continue
}
items = append(items, map[string]any{"peer_id": rawID, "relationship": ctx.Relationship, "can_send": ctx.CanSend})
}
youth := false
if settings, err := store.Security.GetYouthMode(tenantID, viewer); err == nil && settings != nil {
youth = settings.Enabled
}
writeIMSuccess(w, r, map[string]any{"items": items, "viewer_youth_mode": youth})
}
func serveCultivateFiles(w http.ResponseWriter, r *http.Request, tenantID types.TenantID) {
var body cultivateFilesRequest
if !decodeIMJSONBody(w, r, &body) {
return
}
if body.UserID < 0 || len(body.FileIDs) == 0 || len(body.FileIDs) > 20 {
writeIMError(w, r, http.StatusBadRequest, 30000, imReasonMalformedRequest, "malformed request", nil)
return
}
uid := store.EncodeUid(body.UserID)
items := make([]map[string]any, 0, len(body.FileIDs))
for _, supplied := range body.FileIDs {
fileID := normalizeIMUploadedFileID(supplied)
file, err := store.Files.Get(tenantID, fileID)
owner := types.ParseUid(fileOwner(file))
if owner.IsZero() {
owner = types.ParseUserId(fileOwner(file))
}
if err != nil || file == nil || file.Status != types.UploadCompleted || (!uid.IsZero() && owner != uid) {
writeIMError(w, r, http.StatusUnprocessableEntity, 30000, imReasonMalformedRequest, "invalid file", map[string]any{"file_id": supplied})
return
}
items = append(items, map[string]any{"file_id": fileID, "mime_type": file.MimeType, "size": file.Size, "review_status": cultivateFileReviewStatus(tenantID, fileID)})
}
writeIMSuccess(w, r, map[string]any{"items": items})
}
func fileOwner(file *types.FileDef) string {
if file == nil {
return ""
}
return file.User
}
func cultivateFileReviewStatus(tenantID types.TenantID, fileID string) string {
uid := types.ParseUid(fileID)
if uid.IsZero() {
return "unconfigured"
}
row, err := store.InternalAdminQueries.QueryOne(tenantID, `SELECT COUNT(*) AS total,
SUM(status='risky') AS risky_count,
SUM(status IN ('pending','running','failed')) AS pending_count
FROM im_file_review_tasks WHERE file_id=?`, []any{store.DecodeUid(uid)})
if err != nil {
return "unconfigured"
}
total, _ := numericID(row["total"])
risky, _ := numericID(row["risky_count"])
pending, _ := numericID(row["pending_count"])
if risky > 0 {
return "risky"
}
if pending > 0 {
return "pending"
}
if total > 0 {
return "clean"
}
return "unconfigured"
}
func serveCultivateReview(w http.ResponseWriter, r *http.Request, tenant *types.Tenant) {
var body cultivateReviewRequest
if !decodeIMJSONBody(w, r, &body) {
return
}
if body.ActorID <= 0 || body.ObjectID <= 0 {
writeIMError(w, r, http.StatusBadRequest, 30000, imReasonMalformedRequest, "malformed request", nil)
return
}
scene := moderation.SceneFeedPost
if body.ObjectType == "comment" {
scene = moderation.SceneFeedComment
} else if body.ObjectType != "post" {
writeIMError(w, r, http.StatusBadRequest, 30000, imReasonMalformedRequest, "malformed request", map[string]any{"field": "object_type"})
return
}
result, err := moderation.Review(r.Context(), moderation.Request{TenantID: tenant.ID, TenantCode: tenant.Code, Scene: scene, ActorID: store.EncodeUid(body.ActorID), Text: body.Text, TraceID: body.TraceID})
if result == nil {
result = &moderation.Result{Action: moderation.ActionAuditOnly}
}
data := map[string]any{"request_id": result.RequestID, "action": result.Action, "risk_level": result.RiskLevel, "risk_labels": result.RiskLabels, "provider": result.Provider, "provider_request_id": result.ProviderRequestID}
if errors.Is(err, moderation.ErrUnavailable) {
writeIMError(w, r, http.StatusServiceUnavailable, 50001, imReasonInternal, "content review unavailable", data)
return
}
// A moderation rejection is a valid review decision, not a transport error.
writeIMSuccess(w, r, data)
}
func serveCultivateEvent(w http.ResponseWriter, r *http.Request, tenantID types.TenantID) {
var body cultivateEventRequest
if !decodeIMJSONBody(w, r, &body) {
return
}
if body.EventID == "" || body.EventType == "" || body.TargetUserID <= 0 {
writeIMError(w, r, http.StatusBadRequest, 30000, imReasonMalformedRequest, "malformed request", nil)
return
}
target, actor := store.EncodeUid(body.TargetUserID), store.EncodeUid(body.ActorID)
if body.Title != "" {
if err := createIMSystemNotification(tenantID, target, stableIMEventID(body.EventID, "notification"), body.EventType, body.Title, body.Summary, types.KVMap(body.Action)); err != nil {
if errors.Is(err, types.ErrDuplicate) {
writeIMSuccess(w, r, map[string]any{"event_id": body.EventID, "duplicate": true})
return
}
writeIMError(w, r, http.StatusInternalServerError, 90000, imReasonInternal, "internal error", nil)
return
}
}
if err := enqueueIMBusinessEvent(tenantID, body.EventID, "user", target.UserId(), actor, body.EventType, body.Payload); err != nil {
writeIMError(w, r, http.StatusInternalServerError, 90000, imReasonInternal, "internal error", nil)
return
}
if body.Title != "" {
sendPush(&push.Receipt{TenantID: tenantID, To: map[types.Uid]push.Recipient{target: {}}, Payload: push.Payload{
What: push.ActMsg, Topic: "sys", Timestamp: time.Now().UTC(), ContentType: "application/json",
Content: map[string]any{"event_id": body.EventID, "event_type": body.EventType, "title": body.Title, "summary": body.Summary, "action": body.Action},
}})
}
writeIMSuccess(w, r, map[string]any{"event_id": body.EventID})
}
+36
View File
@@ -0,0 +1,36 @@
package main
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestInternalCultivateRequiresSignedServiceRequest(t *testing.T) {
previous := internalAPI
defer func() { internalAPI = previous }()
internalAPI = internalAPIRuntime{config: internalAPIConfig{Enabled: true, Services: map[string]internalAPIServiceConfig{"cultivate-server": {Secret: "secret", Allow: []string{internalCapabilityCultivate}}}}}
req := httptest.NewRequest(http.MethodPost, "/internal/cultivate/tenants/default/users/batch", strings.NewReader(`{"user_ids":[1]}`))
rec := httptest.NewRecorder()
serveInternalCultivate(rec, req, "/internal/cultivate/")
if rec.Code != http.StatusUnauthorized {
t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String())
}
}
func TestInternalCultivateRejectsWrongCapability(t *testing.T) {
previous := internalAPI
defer func() { internalAPI = previous }()
internalAPI = internalAPIRuntime{config: internalAPIConfig{Enabled: true, Services: map[string]internalAPIServiceConfig{"cultivate-server": {Secret: "secret", Allow: []string{internalCapabilityAdminQuery}}}}}
req := httptest.NewRequest(http.MethodPost, "/internal/cultivate/tenants/default/users/batch", strings.NewReader(`{"user_ids":[1]}`))
req.Header.Set(internalHeaderService, "cultivate-server")
req.Header.Set(internalHeaderTimestamp, "1")
req.Header.Set(internalHeaderNonce, "nonce")
req.Header.Set(internalHeaderSignature, "signature")
rec := httptest.NewRecorder()
serveInternalCultivate(rec, req, "/internal/cultivate/")
if rec.Code != http.StatusForbidden {
t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String())
}
}
+64
View File
@@ -0,0 +1,64 @@
package main
import (
"context"
"encoding/json"
"time"
"github.com/tinode/chat/server/store/types"
)
type kafkaAuditPayload struct {
ActorType string `json:"actor_type"`
ActorID string `json:"actor_id"`
Action string `json:"action"`
Resource string `json:"resource"`
ResourceID string `json:"resource_id"`
Result string `json:"result"`
Reason string `json:"reason,omitempty"`
Data map[string]any `json:"data,omitempty"`
OccurredAt time.Time `json:"occurred_at"`
}
var publishKafkaAuditEventEnvelope = publishKafkaEnvelope
func enqueueKafkaAuditEvent(ctx context.Context, tenantID types.TenantID, actorType string, actor types.Uid,
action, resource, resourceID, result, reason string, data map[string]any) error {
if !kafkaEnabled() {
return nil
}
actorID := ""
if !actor.IsZero() {
actorID = actor.UserId()
}
payload := kafkaAuditPayload{
ActorType: actorType,
ActorID: actorID,
Action: action,
Resource: resource,
ResourceID: resourceID,
Result: result,
Reason: reason,
Data: data,
OccurredAt: time.Now().UTC().Round(time.Millisecond),
}
encoded, err := json.Marshal(payload)
if err != nil {
return err
}
envelope := kafkaEnvelope{
EventID: stableIMEventID("audit_event", tenantID, "audit."+action, resource+":"+resourceID, encoded),
TenantID: tenantID,
SourceService: "im-server",
TargetService: "hotgo-admin",
EventType: "audit." + action,
BizType: "audit",
BizID: resource + ":" + resourceID,
OccurredAt: payload.OccurredAt,
Payload: encoded,
}
if !actor.IsZero() {
envelope.Actor = &kafkaActor{Type: actorType, ID: actorID}
}
return publishKafkaAuditEventEnvelope(ctx, "audit_event", envelope.BizID, envelope)
}
@@ -23,7 +23,7 @@ func TestKafkaTopicsKeepOnlyMessageInternalAsyncTopics(t *testing.T) {
if topics["audit_event"] != "im.audit.event" { if topics["audit_event"] != "im.audit.event" {
t.Fatalf("unexpected audit event topic: %q", topics["audit_event"]) t.Fatalf("unexpected audit event topic: %q", topics["audit_event"])
} }
for _, removed := range []string{"file_lifecycle", "workbench_event"} { for _, removed := range []string{"file_lifecycle"} {
if _, ok := topics[removed]; ok { if _, ok := topics[removed]; ok {
t.Fatalf("internal sync topic %q must not be configured", removed) t.Fatalf("internal sync topic %q must not be configured", removed)
} }
@@ -116,30 +116,3 @@ func TestEnqueueKafkaFileAttachmentsBindRunsSynchronously(t *testing.T) {
t.Fatalf("enqueueKafkaFileAttachmentsBind returned error: %v", err) t.Fatalf("enqueueKafkaFileAttachmentsBind returned error: %v", err)
} }
} }
func TestEnqueueKafkaWorkbenchEventRunsSynchronously(t *testing.T) {
oldKafkaBus := kafkaBus
oldInfrastructure := store.HTTPInfrastructure
oldSecurity := store.Security
defer func() {
kafkaBus = oldKafkaBus
store.HTTPInfrastructure = oldInfrastructure
store.Security = oldSecurity
}()
kafkaBus = &kafkaRuntime{}
security := &testSecurityStore{}
store.Security = security
err := enqueueKafkaWorkbenchEvent(context.Background(), 7, types.Uid(42), "meeting", "meeting.created", "meeting-1",
[]types.Uid{42, 43}, "会议通知", "周会", map[string]any{"meeting_id": "meeting-1"})
if err != nil {
t.Fatalf("enqueueKafkaWorkbenchEvent returned error: %v", err)
}
if len(security.notifications) != 1 {
t.Fatalf("expected one notification for non-actor target, got %#v", security.notifications)
}
if security.notifications[0].UserID != types.Uid(43) || security.notifications[0].EventType != "meeting.created" {
t.Fatalf("unexpected notification: %#v", security.notifications[0])
}
}
+8 -5
View File
@@ -38,8 +38,10 @@ type cacheInvalidatePayload struct {
Ranges []types.Range `json:"ranges"` Ranges []types.Range `json:"ranges"`
} }
var kafkaCacheInvalidateConsumerGroup string
func startKafkaCacheInvalidateConsumer() { func startKafkaCacheInvalidateConsumer() {
startKafkaConsumer("cache_invalidate", handleKafkaCacheInvalidateEvent) kafkaCacheInvalidateConsumerGroup = startKafkaBroadcastConsumer("cache_invalidate", handleKafkaCacheInvalidateEvent)
} }
func handleKafkaCacheInvalidateEvent(ctx context.Context, envelope kafkaEnvelope, _ kafka.Message) error { func handleKafkaCacheInvalidateEvent(ctx context.Context, envelope kafkaEnvelope, _ kafka.Message) error {
@@ -50,17 +52,18 @@ func handleKafkaCacheInvalidateEvent(ctx context.Context, envelope kafkaEnvelope
if tenantID.IsZero() { if tenantID.IsZero() {
tenantID = types.TenantID(1) tenantID = types.TenantID(1)
} }
consume, claimed, err := beginKafkaConsumeForTopic(tenantID, envelope, "cache_invalidate") consumerGroup := kafkaCacheInvalidateConsumerGroup
consume, claimed, err := beginKafkaConsumeForTopicWithGroup(tenantID, envelope, "cache_invalidate", consumerGroup)
if err != nil { if err != nil {
return err return err
} }
if !claimed { if !claimed {
logs.Info.Printf("kafka cache invalidate skipped: event=%s group=%s status=%s", envelope.EventID, kafkaBus.config.ConsumerGroup, consume.Status) logs.Info.Printf("kafka cache invalidate skipped: event=%s group=%s status=%s", envelope.EventID, consumerGroup, consume.Status)
return nil return nil
} }
if err = executeKafkaCacheInvalidateEvent(envelope); err != nil { if err = executeKafkaCacheInvalidateEvent(envelope); err != nil {
reason := kafkaCommandErrorReason(err) reason := kafkaCommandErrorReason(err)
failed, markErr := store.HTTPInfrastructure.MarkKafkaConsumeFailed(tenantID, envelope.EventID, kafkaBus.config.ConsumerGroup, reason, err.Error(), time.Now().UTC().Round(time.Millisecond)) failed, markErr := store.HTTPInfrastructure.MarkKafkaConsumeFailed(tenantID, envelope.EventID, consumerGroup, reason, err.Error(), time.Now().UTC().Round(time.Millisecond))
if markErr != nil { if markErr != nil {
return markErr return markErr
} }
@@ -70,7 +73,7 @@ func handleKafkaCacheInvalidateEvent(ctx context.Context, envelope kafkaEnvelope
sleepKafkaRetryBackoff(failed.RetryCount) sleepKafkaRetryBackoff(failed.RetryCount)
return err return err
} }
return store.HTTPInfrastructure.MarkKafkaConsumeSucceeded(tenantID, envelope.EventID, kafkaBus.config.ConsumerGroup, time.Now().UTC().Round(time.Millisecond)) return store.HTTPInfrastructure.MarkKafkaConsumeSucceeded(tenantID, envelope.EventID, consumerGroup, time.Now().UTC().Round(time.Millisecond))
} }
func executeKafkaCacheInvalidateEvent(envelope kafkaEnvelope) error { func executeKafkaCacheInvalidateEvent(envelope kafkaEnvelope) error {
+33 -6
View File
@@ -23,6 +23,8 @@ const (
configEventAppReleaseUpdated = "app.release.updated" configEventAppReleaseUpdated = "app.release.updated"
configEventSecurityPolicyUpdated = "security.policy.updated" configEventSecurityPolicyUpdated = "security.policy.updated"
configEventSensitiveWordsUpdated = "sensitive_words.updated" configEventSensitiveWordsUpdated = "sensitive_words.updated"
configEventMessageKeysUpdated = "message.keys.updated"
configEventMessageKeysActivated = "message.keys.activated"
) )
type configEventPayload struct { type configEventPayload struct {
@@ -32,8 +34,10 @@ type configEventPayload struct {
Keys []string `json:"keys"` Keys []string `json:"keys"`
} }
var kafkaConfigEventConsumerGroup string
func startKafkaConfigEventConsumer() { func startKafkaConfigEventConsumer() {
startKafkaConsumer("config_event", handleKafkaConfigEvent) kafkaConfigEventConsumerGroup = startKafkaBroadcastConsumer("config_event", handleKafkaConfigEvent)
} }
func handleKafkaConfigEvent(ctx context.Context, envelope kafkaEnvelope, _ kafka.Message) error { func handleKafkaConfigEvent(ctx context.Context, envelope kafkaEnvelope, _ kafka.Message) error {
@@ -44,17 +48,18 @@ func handleKafkaConfigEvent(ctx context.Context, envelope kafkaEnvelope, _ kafka
if tenantID.IsZero() { if tenantID.IsZero() {
tenantID = types.TenantID(1) tenantID = types.TenantID(1)
} }
consume, claimed, err := beginKafkaConsumeForTopic(tenantID, envelope, "config_event") consumerGroup := kafkaConfigEventConsumerGroup
consume, claimed, err := beginKafkaConsumeForTopicWithGroup(tenantID, envelope, "config_event", consumerGroup)
if err != nil { if err != nil {
return err return err
} }
if !claimed { if !claimed {
logs.Info.Printf("kafka config event skipped: event=%s group=%s status=%s", envelope.EventID, kafkaBus.config.ConsumerGroup, consume.Status) logs.Info.Printf("kafka config event skipped: event=%s group=%s status=%s", envelope.EventID, consumerGroup, consume.Status)
return nil return nil
} }
if err = executeKafkaConfigEvent(envelope); err != nil { if err = executeKafkaConfigEvent(envelope); err != nil {
reason := kafkaCommandErrorReason(err) reason := kafkaCommandErrorReason(err)
failed, markErr := store.HTTPInfrastructure.MarkKafkaConsumeFailed(tenantID, envelope.EventID, kafkaBus.config.ConsumerGroup, reason, err.Error(), time.Now().UTC().Round(time.Millisecond)) failed, markErr := store.HTTPInfrastructure.MarkKafkaConsumeFailed(tenantID, envelope.EventID, consumerGroup, reason, err.Error(), time.Now().UTC().Round(time.Millisecond))
if markErr != nil { if markErr != nil {
return markErr return markErr
} }
@@ -64,7 +69,7 @@ func handleKafkaConfigEvent(ctx context.Context, envelope kafkaEnvelope, _ kafka
sleepKafkaRetryBackoff(failed.RetryCount) sleepKafkaRetryBackoff(failed.RetryCount)
return err return err
} }
return store.HTTPInfrastructure.MarkKafkaConsumeSucceeded(tenantID, envelope.EventID, kafkaBus.config.ConsumerGroup, time.Now().UTC().Round(time.Millisecond)) return store.HTTPInfrastructure.MarkKafkaConsumeSucceeded(tenantID, envelope.EventID, consumerGroup, time.Now().UTC().Round(time.Millisecond))
} }
func executeKafkaConfigEvent(envelope kafkaEnvelope) error { func executeKafkaConfigEvent(envelope kafkaEnvelope) error {
@@ -90,6 +95,9 @@ func executeKafkaConfigEvent(envelope kafkaEnvelope) error {
} }
case configEventDictionaryUpdated, configEventThirdPartyUpdated: case configEventDictionaryUpdated, configEventThirdPartyUpdated:
store.InvalidateThirdPartyConfigCache(payload.Group) store.InvalidateThirdPartyConfigCache(payload.Group)
if strings.TrimSpace(envelope.EventType) == configEventDictionaryUpdated {
store.InvalidateDictionaryConfigCache(payload.Group)
}
if payload.Group == "moderation" { if payload.Group == "moderation" {
store.InvalidateModerationConfigCache(tenantCode, true, false, false) store.InvalidateModerationConfigCache(tenantCode, true, false, false)
} }
@@ -99,14 +107,23 @@ func executeKafkaConfigEvent(envelope kafkaEnvelope) error {
store.InvalidateAppReleaseConfigCache() store.InvalidateAppReleaseConfigCache()
case configEventSecurityPolicyUpdated: case configEventSecurityPolicyUpdated:
store.InvalidateModerationConfigCache(tenantCode, false, true, false) store.InvalidateModerationConfigCache(tenantCode, false, true, false)
store.InvalidateSecurityPolicyCache(tenantCode)
case configEventSensitiveWordsUpdated: case configEventSensitiveWordsUpdated:
store.InvalidateModerationConfigCache(tenantCode, false, false, true) store.InvalidateModerationConfigCache(tenantCode, false, false, true)
case configEventMessageKeysUpdated, configEventMessageKeysActivated:
store.InvalidateMessageEncryptionKeys(tenantID)
if strings.TrimSpace(envelope.EventType) == configEventMessageKeysUpdated && envelope.Version > 0 {
if err := store.PrepareMessageEncryptionKey(tenantID, tenantCode, int(envelope.Version)); err != nil {
return err
}
}
default: default:
store.InvalidateTenantConfigCache(tenantCode, tenantID) store.InvalidateTenantConfigCache(tenantCode, tenantID)
store.InvalidateThirdPartyConfigCache(payload.Group) store.InvalidateThirdPartyConfigCache(payload.Group)
store.InvalidateRegistrationConfigCache() store.InvalidateRegistrationConfigCache()
store.InvalidateAppReleaseConfigCache() store.InvalidateAppReleaseConfigCache()
store.InvalidateModerationConfigCache(tenantCode, true, true, true) store.InvalidateModerationConfigCache(tenantCode, true, true, true)
store.InvalidateSecurityPolicyCache(tenantCode)
} }
logs.Info.Printf("kafka config event applied: event=%s type=%s tenant=%s group=%s", envelope.EventID, envelope.EventType, tenantCode, payload.Group) logs.Info.Printf("kafka config event applied: event=%s type=%s tenant=%s group=%s", envelope.EventID, envelope.EventType, tenantCode, payload.Group)
return nil return nil
@@ -127,10 +144,20 @@ func beginKafkaConsumeForTopic(tenantID types.TenantID, envelope kafkaEnvelope,
if kafkaBus == nil { if kafkaBus == nil {
return nil, false, types.ErrUnsupported return nil, false, types.ErrUnsupported
} }
return beginKafkaConsumeForTopicWithGroup(tenantID, envelope, topicName, kafkaBus.config.ConsumerGroup)
}
func beginKafkaConsumeForTopicWithGroup(tenantID types.TenantID, envelope kafkaEnvelope, topicName, consumerGroup string) (*types.KafkaConsumeRecord, bool, error) {
if kafkaBus == nil {
return nil, false, types.ErrUnsupported
}
if consumerGroup = strings.TrimSpace(consumerGroup); consumerGroup == "" {
consumerGroup = kafkaBus.config.ConsumerGroup
}
deadline := time.Now().UTC().Add(5 * time.Minute).Round(time.Millisecond) deadline := time.Now().UTC().Add(5 * time.Minute).Round(time.Millisecond)
return store.HTTPInfrastructure.BeginKafkaConsume(tenantID, &types.KafkaConsumeRecord{ return store.HTTPInfrastructure.BeginKafkaConsume(tenantID, &types.KafkaConsumeRecord{
EventID: envelope.EventID, EventID: envelope.EventID,
ConsumerGroup: kafkaBus.config.ConsumerGroup, ConsumerGroup: consumerGroup,
SourceTopic: kafkaTopic(topicName), SourceTopic: kafkaTopic(topicName),
EventType: envelope.EventType, EventType: envelope.EventType,
BizType: envelope.BizType, BizType: envelope.BizType,
+5 -5
View File
@@ -54,10 +54,10 @@ func saveMessageMaybeKafka(ctx context.Context, tenantID types.TenantID, msg *ty
if runtimeReady && !claimed { if runtimeReady && !claimed {
if claim != nil && claim.Seq > 0 { if claim != nil && claim.Seq > 0 {
msg.SeqId = claim.Seq msg.SeqId = claim.Seq
return &types.MessageSaveResult{SeqId: claim.Seq, Duplicate: claim.Status == "done", Pending: claim.Status == "pending"}, nil return &types.MessageSaveResult{SeqId: claim.Seq, Duplicate: true, Pending: claim.Status == "pending"}, nil
} }
if claim != nil && claim.Status == "pending" { if claim != nil && claim.Status == "pending" {
return &types.MessageSaveResult{Pending: true}, types.ErrDuplicate return &types.MessageSaveResult{Duplicate: true, Pending: true}, types.ErrDuplicate
} }
return nil, types.ErrDuplicate return nil, types.ErrDuplicate
} }
@@ -87,8 +87,8 @@ func saveMessageMaybeKafka(ctx context.Context, tenantID types.TenantID, msg *ty
return &types.MessageSaveResult{SeqId: msg.SeqId, Pending: true, MarkedReadBySender: readBySender && !fromUid.IsZero()}, err return &types.MessageSaveResult{SeqId: msg.SeqId, Pending: true, MarkedReadBySender: readBySender && !fromUid.IsZero()}, err
} }
if clientMsgID != "" { if clientMsgID != "" {
if err = store.MarkMessageUniqueDone(tenantID, msg.Topic, fromUid, clientMsgID, msg.SeqId, ""); err != nil { if err = store.MarkMessageUniquePending(tenantID, msg.Topic, fromUid, clientMsgID, msg.SeqId); err != nil {
logs.Warn.Printf("kafka message persist: failed to mark message unique key done topic=%s seq=%d: %v", msg.Topic, msg.SeqId, err) logs.Warn.Printf("kafka message persist: failed to mark message unique key pending topic=%s seq=%d: %v", msg.Topic, msg.SeqId, err)
} }
} }
if isGroupMessage { if isGroupMessage {
@@ -96,7 +96,7 @@ func saveMessageMaybeKafka(ctx context.Context, tenantID types.TenantID, msg *ty
logs.Warn.Printf("kafka message persist: failed to cache pending hot message after publish topic=%s seq=%d: %v", msg.Topic, msg.SeqId, cacheErr) logs.Warn.Printf("kafka message persist: failed to cache pending hot message after publish topic=%s seq=%d: %v", msg.Topic, msg.SeqId, cacheErr)
} }
} }
return &types.MessageSaveResult{SeqId: msg.SeqId, MarkedReadBySender: readBySender && !fromUid.IsZero()}, nil return &types.MessageSaveResult{SeqId: msg.SeqId, Pending: true, MarkedReadBySender: readBySender && !fromUid.IsZero()}, nil
} }
func enqueueKafkaMessagePersist(ctx context.Context, tenantID types.TenantID, msg *types.Message, attachmentURLs []string, readBySender bool) error { func enqueueKafkaMessagePersist(ctx context.Context, tenantID types.TenantID, msg *types.Message, attachmentURLs []string, readBySender bool) error {
+37 -4
View File
@@ -5,6 +5,7 @@ import (
"crypto/tls" "crypto/tls"
"encoding/json" "encoding/json"
"errors" "errors"
"regexp"
"strings" "strings"
"sync" "sync"
"time" "time"
@@ -24,7 +25,6 @@ const (
kafkaTopicMessagePersist = "im.message.persist" kafkaTopicMessagePersist = "im.message.persist"
kafkaTopicGroupMessagePersist = "im.group.message.persist" kafkaTopicGroupMessagePersist = "im.group.message.persist"
kafkaTopicMessagePush = "im.message.push" kafkaTopicMessagePush = "im.message.push"
kafkaTopicWorkbenchExternal = "im.workbench.event"
kafkaTopicFileSecurity = "im.file.security" kafkaTopicFileSecurity = "im.file.security"
kafkaTopicCacheInvalidate = "im.cache.invalidate" kafkaTopicCacheInvalidate = "im.cache.invalidate"
kafkaTopicAuditEvent = "im.audit.event" kafkaTopicAuditEvent = "im.audit.event"
@@ -89,6 +89,8 @@ type kafkaRuntime struct {
var kafkaBus *kafkaRuntime var kafkaBus *kafkaRuntime
var kafkaConsumerGroupUnsafeChars = regexp.MustCompile(`[^A-Za-z0-9._-]+`)
func initKafkaRuntime(raw json.RawMessage) error { func initKafkaRuntime(raw json.RawMessage) error {
if len(raw) == 0 { if len(raw) == 0 {
kafkaBus = nil kafkaBus = nil
@@ -211,7 +213,6 @@ func kafkaTopics(overrides map[string]string) map[string]string {
"message_persist": kafkaTopicMessagePersist, "message_persist": kafkaTopicMessagePersist,
"group_message_persist": kafkaTopicGroupMessagePersist, "group_message_persist": kafkaTopicGroupMessagePersist,
"message_push": kafkaTopicMessagePush, "message_push": kafkaTopicMessagePush,
"workbench_external": kafkaTopicWorkbenchExternal,
"file_security": kafkaTopicFileSecurity, "file_security": kafkaTopicFileSecurity,
"cache_invalidate": kafkaTopicCacheInvalidate, "cache_invalidate": kafkaTopicCacheInvalidate,
"audit_event": kafkaTopicAuditEvent, "audit_event": kafkaTopicAuditEvent,
@@ -274,6 +275,19 @@ func publishKafkaEnvelope(ctx context.Context, topicName string, key string, env
type kafkaMessageHandler func(context.Context, kafkaEnvelope, kafka.Message) error type kafkaMessageHandler func(context.Context, kafkaEnvelope, kafka.Message) error
func startKafkaConsumer(topicName string, handler kafkaMessageHandler) { func startKafkaConsumer(topicName string, handler kafkaMessageHandler) {
startKafkaConsumerWithGroup(topicName, "", handler)
}
func startKafkaBroadcastConsumer(topicName string, handler kafkaMessageHandler) string {
if kafkaBus == nil || handler == nil {
return ""
}
groupID := kafkaBroadcastConsumerGroup(topicName)
startKafkaConsumerWithGroup(topicName, groupID, handler)
return groupID
}
func startKafkaConsumerWithGroup(topicName, groupID string, handler kafkaMessageHandler) {
if kafkaBus == nil || handler == nil { if kafkaBus == nil || handler == nil {
return return
} }
@@ -281,8 +295,11 @@ func startKafkaConsumer(topicName string, handler kafkaMessageHandler) {
if topic == "" { if topic == "" {
topic = topicName topic = topicName
} }
groupID := kafkaBus.config.ConsumerGroup groupOverride := groupID != ""
if topicName != "" { if groupID == "" {
groupID = kafkaBus.config.ConsumerGroup
}
if topicName != "" && !groupOverride {
groupID += "." + topicName groupID += "." + topicName
} }
reader := kafka.NewReader(kafka.ReaderConfig{ reader := kafka.NewReader(kafka.ReaderConfig{
@@ -326,6 +343,22 @@ func startKafkaConsumer(topicName string, handler kafkaMessageHandler) {
logs.Info.Printf("Kafka consumer started: topic=%s group=%s", topic, groupID) logs.Info.Printf("Kafka consumer started: topic=%s group=%s", topic, groupID)
} }
func kafkaBroadcastConsumerGroup(topicName string) string {
if kafkaBus == nil {
return ""
}
base := strings.TrimSpace(kafkaBus.config.ConsumerGroup)
if base == "" {
base = "im-server"
}
nodeID := strings.TrimSpace(currentNodeID())
if nodeID == "" {
nodeID = "local"
}
nodeID = kafkaConsumerGroupUnsafeChars.ReplaceAllString(nodeID, "-")
return base + "." + topicName + "." + nodeID
}
func (rt *kafkaRuntime) dialer() *kafka.Dialer { func (rt *kafkaRuntime) dialer() *kafka.Dialer {
dialer := &kafka.Dialer{ dialer := &kafka.Dialer{
ClientID: rt.config.ClientID, ClientID: rt.config.ClientID,
-129
View File
@@ -1,129 +0,0 @@
package main
import (
"context"
"encoding/json"
"time"
"github.com/tinode/chat/server/store/types"
)
type kafkaWorkbenchPayload struct {
Module string `json:"module"`
Action string `json:"action"`
ResourceID string `json:"resource_id"`
ActorUserID string `json:"actor_user_id"`
TargetUserIDs []string `json:"target_user_ids,omitempty"`
Title string `json:"title,omitempty"`
Summary string `json:"summary,omitempty"`
Data map[string]any `json:"data,omitempty"`
OccurredAt time.Time `json:"occurred_at"`
}
type kafkaAuditPayload struct {
ActorType string `json:"actor_type"`
ActorID string `json:"actor_id"`
Action string `json:"action"`
Resource string `json:"resource"`
ResourceID string `json:"resource_id"`
Result string `json:"result"`
Reason string `json:"reason,omitempty"`
Data map[string]any `json:"data,omitempty"`
OccurredAt time.Time `json:"occurred_at"`
}
var publishKafkaAuditEventEnvelope = publishKafkaEnvelope
func enqueueKafkaWorkbenchEvent(ctx context.Context, tenantID types.TenantID, actor types.Uid, module, action, resourceID string,
targets []types.Uid, title, summary string, data map[string]any) error {
targetUserIDs := make([]string, 0, len(targets))
for _, target := range targets {
if !target.IsZero() {
targetUserIDs = append(targetUserIDs, target.UserId())
}
}
payload := kafkaWorkbenchPayload{
Module: module,
Action: action,
ResourceID: resourceID,
ActorUserID: actor.UserId(),
TargetUserIDs: targetUserIDs,
Title: title,
Summary: summary,
Data: data,
OccurredAt: time.Now().UTC().Round(time.Millisecond),
}
envelope := kafkaEnvelope{
EventID: stableIMEventID("workbench-sync", tenantID, "workbench."+action, resourceID, payload.OccurredAt),
TenantID: tenantID,
EventType: "workbench." + action,
BizType: "workbench",
BizID: resourceID,
}
return executeKafkaWorkbenchEvent(tenantID, envelope, payload)
}
func enqueueKafkaAuditEvent(ctx context.Context, tenantID types.TenantID, actorType string, actor types.Uid,
action, resource, resourceID, result, reason string, data map[string]any) error {
if !kafkaEnabled() {
return nil
}
actorID := ""
if !actor.IsZero() {
actorID = actor.UserId()
}
payload := kafkaAuditPayload{
ActorType: actorType,
ActorID: actorID,
Action: action,
Resource: resource,
ResourceID: resourceID,
Result: result,
Reason: reason,
Data: data,
OccurredAt: time.Now().UTC().Round(time.Millisecond),
}
encoded, err := json.Marshal(payload)
if err != nil {
return err
}
envelope := kafkaEnvelope{
EventID: stableIMEventID("audit_event", tenantID, "audit."+action, resource+":"+resourceID, encoded),
TenantID: tenantID,
SourceService: "im-server",
TargetService: "hotgo-admin",
EventType: "audit." + action,
BizType: "audit",
BizID: resource + ":" + resourceID,
OccurredAt: payload.OccurredAt,
Payload: encoded,
}
if !actor.IsZero() {
envelope.Actor = &kafkaActor{Type: actorType, ID: actorID}
}
return publishKafkaAuditEventEnvelope(ctx, "audit_event", envelope.BizID, envelope)
}
func executeKafkaWorkbenchEvent(tenantID types.TenantID, envelope kafkaEnvelope, payload kafkaWorkbenchPayload) error {
actor := types.ParseUserId(payload.ActorUserID)
if actor.IsZero() {
actor = types.ParseUid(payload.ActorUserID)
}
for _, raw := range payload.TargetUserIDs {
target := types.ParseUserId(raw)
if target.IsZero() {
continue
}
eventID := stableIMEventID("workbench", envelope.EventID, target.UserId())
if target != actor && payload.Title != "" {
if err := createIMSystemNotification(tenantID, target, stableIMEventID(eventID, "notification"), payload.Action,
payload.Title, payload.Summary, types.KVMap{"module": payload.Module, "resource_id": payload.ResourceID}); err != nil {
return err
}
}
if err := enqueueIMBusinessEvent(tenantID, eventID, "user", target.UserId(), actor, payload.Action, payload.Data); err != nil {
return err
}
}
return nil
}
-163
View File
@@ -1,163 +0,0 @@
package main
import (
"context"
"encoding/json"
"strings"
"time"
"github.com/segmentio/kafka-go"
"github.com/tinode/chat/server/logs"
"github.com/tinode/chat/server/store"
"github.com/tinode/chat/server/store/types"
)
type kafkaExternalWorkbenchEvent struct {
EventID string `json:"event_id"`
TraceID string `json:"trace_id,omitempty"`
TenantID types.TenantID `json:"tenant_id,omitempty"`
TenantCode string `json:"tenant_code,omitempty"`
ActorUserID int64 `json:"actor_user_id"`
Module string `json:"module"`
Action string `json:"action"`
ResourceID string `json:"resource_id"`
TargetUserIDs []int64 `json:"target_user_ids,omitempty"`
Title string `json:"title,omitempty"`
Summary string `json:"summary,omitempty"`
OccurredAt time.Time `json:"occurred_at"`
Data map[string]any `json:"data,omitempty"`
}
func startKafkaWorkbenchEventConsumer() {
startKafkaConsumer("workbench_external", handleKafkaWorkbenchEvent)
}
func handleKafkaWorkbenchEvent(ctx context.Context, envelope kafkaEnvelope, msg kafka.Message) error {
payload, normalized, err := decodeKafkaWorkbenchEvent(envelope, msg)
if err != nil {
logs.Warn.Printf("kafka workbench event malformed: topic=%s offset=%d err=%v", msg.Topic, msg.Offset, err)
return err
}
tenantID, err := kafkaTenantIDFromEnvelope(normalized)
if err != nil {
return err
}
consume, claimed, err := beginKafkaConsumeForTopic(tenantID, normalized, "workbench_external")
if err != nil {
return err
}
if !claimed {
logs.Info.Printf("kafka workbench event skipped: event=%s group=%s status=%s", normalized.EventID, kafkaBus.config.ConsumerGroup, consume.Status)
return nil
}
if err = executeKafkaExternalWorkbenchEvent(tenantID, normalized, payload); err != nil {
reason := kafkaCommandErrorReason(err)
failed, markErr := store.HTTPInfrastructure.MarkKafkaConsumeFailed(tenantID, normalized.EventID, kafkaBus.config.ConsumerGroup, reason, err.Error(), time.Now().UTC().Round(time.Millisecond))
if markErr != nil {
return markErr
}
if failed != nil && failed.RetryCount >= kafkaBus.config.MaxRetryCount {
return markKafkaDeadLetter(ctx, tenantID, normalized, failed, reason, err.Error())
}
if failed != nil {
sleepKafkaRetryBackoff(failed.RetryCount)
}
return err
}
return store.HTTPInfrastructure.MarkKafkaConsumeSucceeded(tenantID, normalized.EventID, kafkaBus.config.ConsumerGroup, time.Now().UTC().Round(time.Millisecond))
}
func decodeKafkaWorkbenchEvent(envelope kafkaEnvelope, msg kafka.Message) (kafkaExternalWorkbenchEvent, kafkaEnvelope, error) {
var payload kafkaExternalWorkbenchEvent
if len(envelope.Payload) > 0 {
if err := json.Unmarshal(envelope.Payload, &payload); err != nil {
return payload, envelope, err
}
} else if err := json.Unmarshal(msg.Value, &payload); err != nil {
return payload, envelope, err
}
if strings.TrimSpace(payload.EventID) == "" {
payload.EventID = envelope.EventID
}
if strings.TrimSpace(payload.Action) == "" && envelope.EventType != "" {
payload.Action = strings.TrimPrefix(envelope.EventType, "workbench.")
}
if strings.TrimSpace(payload.EventID) == "" || strings.TrimSpace(payload.Action) == "" || strings.TrimSpace(payload.ResourceID) == "" {
return payload, envelope, types.ErrMalformed
}
if payload.OccurredAt.IsZero() {
payload.OccurredAt = time.Now().UTC().Round(time.Millisecond)
}
normalized := kafkaEnvelope{
EventID: payload.EventID,
TraceID: firstNonEmpty(payload.TraceID, envelope.TraceID),
TenantCode: firstNonEmpty(payload.TenantCode, envelope.TenantCode),
TenantID: firstTenantID(payload.TenantID, envelope.TenantID),
SourceService: firstNonEmpty(envelope.SourceService, "im-workbench"),
TargetService: firstNonEmpty(envelope.TargetService, "im-server"),
EventType: "workbench." + payload.Action,
BizType: firstNonEmpty(payload.Module, envelope.BizType, "workbench"),
BizID: firstNonEmpty(payload.ResourceID, envelope.BizID),
OccurredAt: payload.OccurredAt,
Payload: msg.Value,
}
if payload.ActorUserID > 0 {
normalized.Actor = &kafkaActor{Type: "user", ID: types.Uid(payload.ActorUserID).UserId()}
}
return payload, normalized, nil
}
func executeKafkaExternalWorkbenchEvent(tenantID types.TenantID, envelope kafkaEnvelope, payload kafkaExternalWorkbenchEvent) error {
if store.Security == nil {
return types.ErrUnsupported
}
actor := types.Uid(payload.ActorUserID)
data := types.KVMap{
"module": payload.Module,
"resource_id": payload.ResourceID,
}
for k, v := range payload.Data {
data[k] = v
}
targets := compactWorkbenchTargets(payload.TargetUserIDs, actor)
for _, target := range targets {
targetID := target.UserId()
eventID := stableIMEventID("workbench", envelope.EventID, targetID)
if target != actor && payload.Title != "" {
if err := createIMSystemNotification(tenantID, target, stableIMEventID(eventID, "notification"), payload.Action,
payload.Title, payload.Summary, data); err != nil {
return err
}
}
if err := enqueueIMBusinessEvent(tenantID, eventID, "user", targetID, actor, payload.Action, data); err != nil {
return err
}
}
return nil
}
func compactWorkbenchTargets(raw []int64, actor types.Uid) []types.Uid {
seen := make(map[types.Uid]bool, len(raw)+1)
targets := make([]types.Uid, 0, len(raw)+1)
for _, value := range raw {
uid := types.Uid(value)
if uid.IsZero() || seen[uid] {
continue
}
seen[uid] = true
targets = append(targets, uid)
}
if len(targets) == 0 && !actor.IsZero() {
targets = append(targets, actor)
}
return targets
}
func firstTenantID(values ...types.TenantID) types.TenantID {
for _, value := range values {
if !value.IsZero() {
return value
}
}
return types.ZeroTenantID
}
-89
View File
@@ -1,89 +0,0 @@
package main
import (
"testing"
"time"
"github.com/segmentio/kafka-go"
"github.com/tinode/chat/server/store"
"github.com/tinode/chat/server/store/types"
)
func TestKafkaTopicsIncludeExternalWorkbenchEvent(t *testing.T) {
topics := kafkaTopics(nil)
if topics["workbench_external"] != "im.workbench.event" {
t.Fatalf("unexpected workbench external topic: %q", topics["workbench_external"])
}
if _, ok := topics["workbench_event"]; ok {
t.Fatal("legacy in-process workbench_event topic key must not be configured")
}
}
func TestDecodeKafkaWorkbenchEventDirectPayload(t *testing.T) {
msg := kafka.Message{Value: []byte(`{
"event_id":"wbe_1",
"trace_id":"trace-1",
"tenant_id":7,
"tenant_code":"default",
"actor_user_id":42,
"module":"meeting",
"action":"meeting.created",
"resource_id":"meeting-1",
"target_user_ids":[42,43],
"title":"会议通知",
"summary":"周会",
"occurred_at":"2026-09-13T10:00:00Z",
"data":{"meeting_id":"meeting-1"}
}`)}
payload, envelope, err := decodeKafkaWorkbenchEvent(kafkaEnvelope{}, msg)
if err != nil {
t.Fatalf("decodeKafkaWorkbenchEvent returned error: %v", err)
}
if payload.EventID != "wbe_1" || payload.ActorUserID != 42 || len(payload.TargetUserIDs) != 2 {
t.Fatalf("unexpected payload: %#v", payload)
}
if envelope.EventID != "wbe_1" || envelope.EventType != "workbench.meeting.created" ||
envelope.BizType != "meeting" || envelope.BizID != "meeting-1" {
t.Fatalf("unexpected normalized envelope: %#v", envelope)
}
if envelope.Actor == nil || envelope.Actor.ID != types.Uid(42).UserId() {
t.Fatalf("unexpected actor: %#v", envelope.Actor)
}
if envelope.OccurredAt.IsZero() || envelope.TenantID != types.TenantID(7) {
t.Fatalf("unexpected occurrence or tenant: %#v", envelope)
}
}
func TestExecuteKafkaExternalWorkbenchEventCreatesNotificationForNonActor(t *testing.T) {
oldSecurity := store.Security
defer func() { store.Security = oldSecurity }()
security := &testSecurityStore{}
store.Security = security
payload := kafkaExternalWorkbenchEvent{
EventID: "wbe_1",
TenantID: 7,
ActorUserID: 42,
Module: "meeting",
Action: "meeting.created",
ResourceID: "meeting-1",
TargetUserIDs: []int64{42, 43},
Title: "会议通知",
Summary: "周会",
OccurredAt: time.Date(2026, 9, 13, 10, 0, 0, 0, time.UTC),
Data: map[string]any{"meeting_id": "meeting-1"},
}
envelope := kafkaEnvelope{EventID: payload.EventID}
if err := executeKafkaExternalWorkbenchEvent(7, envelope, payload); err != nil {
t.Fatalf("executeKafkaExternalWorkbenchEvent returned error: %v", err)
}
if len(security.notifications) != 1 {
t.Fatalf("expected one notification for non-actor target, got %#v", security.notifications)
}
if security.notifications[0].UserID != types.Uid(43) || security.notifications[0].EventType != "meeting.created" {
t.Fatalf("unexpected notification: %#v", security.notifications[0])
}
}
+131
View File
@@ -0,0 +1,131 @@
package main
import (
"errors"
"fmt"
"strings"
"time"
"github.com/tinode/chat/server/logs"
"github.com/tinode/chat/server/store"
"github.com/tinode/chat/server/store/types"
)
// loginAuditRiskActionLocked marks failure-log rows recorded at the moment the
// account lockout is triggered; the admin console uses it to offer unlocking.
const loginAuditRiskActionLocked = "account_locked"
// loginLockoutAccount extracts the account name supplied with a basic-auth secret.
func loginLockoutAccount(secret []byte) string {
parts := strings.SplitN(string(secret), ":", 2)
if len(parts) != 2 || strings.TrimSpace(parts[0]) == "" {
return ""
}
return strings.ToLower(strings.TrimSpace(parts[0]))
}
// tenantLoginRateLimitPolicy loads the published login rate limit policy for
// the tenant. Any error or disabled policy yields ok=false (feature off, the
// login flow then behaves exactly as before).
func tenantLoginRateLimitPolicy(tenantCode string) (types.LoginRateLimitPolicy, bool) {
if strings.TrimSpace(tenantCode) == "" {
return types.LoginRateLimitPolicy{}, false
}
policy, err := store.ModerationConfig.SecurityPolicy(tenantCode)
if err != nil {
if !errors.Is(err, types.ErrUnsupported) {
logs.Warn.Printf("login lockout policy load failed tenant:%s err:%v", tenantCode, err)
}
return types.LoginRateLimitPolicy{}, false
}
return policy.LoginRateLimit.Normalized()
}
// loginLockedMessage renders the user-facing lockout message with the rounded
// down remaining minutes; sub-minute remainders are reported as "within 1 minute".
func loginLockedMessage(retryAfter time.Duration) string {
if minutes := int(retryAfter.Minutes()); minutes >= 1 {
return fmt.Sprintf("登录错误次数过多,账号已锁定,请 %d 分钟后再试", minutes)
}
return "登录错误次数过多,账号已锁定,请 1 分钟内再试"
}
// loginLockoutGuard enforces the configured login failure lockout for one
// login attempt of the "basic" scheme. A nil guard means the feature is not
// applicable (non-basic scheme, unknown tenant, or disabled policy).
type loginLockoutGuard struct {
session *Session
account string
accountHash string
policy types.LoginRateLimitPolicy
}
func newLoginLockoutGuard(s *Session, scheme string, secret []byte) *loginLockoutGuard {
if s == nil || !s.tenantID.IsValid() || strings.TrimSpace(s.tenantCode) == "" ||
!strings.EqualFold(strings.TrimSpace(scheme), "basic") {
return nil
}
account := loginLockoutAccount(secret)
if account == "" {
return nil
}
policy, ok := tenantLoginRateLimitPolicy(s.tenantCode)
if !ok {
return nil
}
return &loginLockoutGuard{session: s, account: account, accountHash: hashLoginAccount(account), policy: policy}
}
// preCheck rejects the attempt up front when the account is currently locked,
// without comparing the password.
func (g *loginLockoutGuard) preCheck() (time.Duration, bool) {
rec, err := store.LoginLockouts.Get(g.session.tenantID, g.accountHash)
if err != nil {
if !errors.Is(err, types.ErrNotFound) && !errors.Is(err, types.ErrUnsupported) {
logs.Warn.Printf("login lockout pre-check failed tenant:%d account_hash:%s err:%v", g.session.tenantID, g.accountHash, err)
}
return 0, false
}
now := time.Now()
if rec.LockedUntil == nil || !now.Before(*rec.LockedUntil) {
return 0, false
}
return time.Until(*rec.LockedUntil), true
}
// recordFailure counts one failed password attempt and, when the configured
// threshold is reached, persists the lock. It returns the remaining lock time
// and whether the account is now locked.
func (g *loginLockoutGuard) recordFailure() (time.Duration, bool) {
now := time.Now()
cutoff := now.Add(-time.Duration(g.policy.WindowSeconds) * time.Second)
rec, err := store.LoginLockouts.RecordFailure(g.session.tenantID, g.accountHash, nil, cutoff, now)
if err != nil {
if !errors.Is(err, types.ErrUnsupported) {
logs.Warn.Printf("login lockout record failed tenant:%d account_hash:%s err:%v", g.session.tenantID, g.accountHash, err)
}
return 0, false
}
if rec.FailedAttempts < g.policy.PasswordMaxAttempts {
return 0, false
}
until := now.Add(time.Duration(g.policy.LockSeconds) * time.Second)
if rec.LockedUntil != nil && rec.LockedUntil.After(now) {
// Lock already active (e.g. recorded within the same window): keep it.
until = *rec.LockedUntil
} else if err = store.LoginLockouts.Lock(g.session.tenantID, g.accountHash, until, now); err != nil {
if !errors.Is(err, types.ErrUnsupported) {
logs.Warn.Printf("login lockout persist failed tenant:%d account_hash:%s err:%v", g.session.tenantID, g.accountHash, err)
}
return 0, false
}
return time.Until(until), true
}
// clear resets the counter after a successful password check.
func (g *loginLockoutGuard) clear() {
if err := store.LoginLockouts.Reset(g.session.tenantID, g.accountHash); err != nil &&
!errors.Is(err, types.ErrNotFound) && !errors.Is(err, types.ErrUnsupported) {
logs.Warn.Printf("login lockout reset failed tenant:%d account_hash:%s err:%v", g.session.tenantID, g.accountHash, err)
}
}
+16 -25
View File
@@ -494,6 +494,17 @@ func main() {
logs.Info.Printf("Profiling info saved to '%s.(cpu|mem)'", *pprofFile) logs.Info.Printf("Profiling info saved to '%s.(cpu|mem)'", *pprofFile)
} }
adminInternalClient, err := initAdminInternalAPI(config.AdminAPI)
if err != nil {
logs.Err.Fatal("Failed to initialize Admin internal API client:", err)
}
if storeConfig, ok, err := storeConfigWithAdminUIDKey(config.Store, adminInternalClient); err != nil {
logs.Warn.Printf("Failed to load UID key from Admin internal config, falling back to store_config.uid_key: %v", err)
} else if ok {
config.Store = storeConfig
logs.Info.Println("Loaded store UID key from Admin internal config")
}
err = store.Store.Open(workerId, config.Store) err = store.Store.Open(workerId, config.Store)
logs.Info.Println("DB adapter", store.Store.GetAdapterName(), store.Store.GetAdapterVersion()) logs.Info.Println("DB adapter", store.Store.GetAdapterName(), store.Store.GetAdapterVersion())
if err != nil { if err != nil {
@@ -513,9 +524,6 @@ func main() {
if err = initInternalAPI(config.Internal); err != nil { if err = initInternalAPI(config.Internal); err != nil {
logs.Err.Fatal("Failed to initialize internal API:", err) logs.Err.Fatal("Failed to initialize internal API:", err)
} }
if err = initAdminInternalAPI(config.AdminAPI); err != nil {
logs.Err.Fatal("Failed to initialize Admin internal API client:", err)
}
// API key signing secret // API key signing secret
globals.apiKeySalt = config.APIKeySalt globals.apiKeySalt = config.APIKeySalt
@@ -750,7 +758,6 @@ func main() {
startKafkaConfigEventConsumer() startKafkaConfigEventConsumer()
startKafkaCacheInvalidateConsumer() startKafkaCacheInvalidateConsumer()
startKafkaMessagePersistConsumer() startKafkaMessagePersistConsumer()
startKafkaWorkbenchEventConsumer()
startKafkaDeadLetterConsumer() startKafkaDeadLetterConsumer()
// Start accepting cluster traffic. // Start accepting cluster traffic.
@@ -878,31 +885,11 @@ func main() {
mux.HandleFunc(config.ApiPath+"v1/im/security/realname/appeals", serveIMRealNameAppeals) mux.HandleFunc(config.ApiPath+"v1/im/security/realname/appeals", serveIMRealNameAppeals)
mux.HandleFunc(config.ApiPath+"v1/im/security/realname/submissions", serveIMRealNameSubmissions) mux.HandleFunc(config.ApiPath+"v1/im/security/realname/submissions", serveIMRealNameSubmissions)
mux.HandleFunc(config.ApiPath+"v1/im/bootstrap", serveIMBootstrap) mux.HandleFunc(config.ApiPath+"v1/im/bootstrap", serveIMBootstrap)
mux.HandleFunc(config.ApiPath+"v1/im/workbench", serveIMWorkbench)
mux.HandleFunc(config.ApiPath+"v1/im/organization/departments", serveIMOrganizationDepartments)
mux.HandleFunc(config.ApiPath+"v1/im/organization/departments/", serveIMOrganizationDepartmentResource)
mux.HandleFunc(config.ApiPath+"v1/im/organization/members", serveIMOrganizationMembers)
mux.HandleFunc(config.ApiPath+"v1/im/organization/search", serveIMOrganizationSearch)
mux.HandleFunc(config.ApiPath+"v1/im/attendance/config", serveIMAttendanceConfig)
mux.HandleFunc(config.ApiPath+"v1/im/attendance/records", serveIMAttendanceRecords)
mux.HandleFunc(config.ApiPath+"v1/im/attendance/checkins", serveIMAttendanceCheckins)
mux.HandleFunc(config.ApiPath+"v1/im/attendance/appeals", serveIMAttendanceAppeals)
mux.HandleFunc(config.ApiPath+"v1/im/meeting-notes", serveIMMeetingNotes)
mux.HandleFunc(config.ApiPath+"v1/im/meetings", serveIMMeetings)
mux.HandleFunc(config.ApiPath+"v1/im/meetings/", serveIMMeetingResource)
mux.HandleFunc(config.ApiPath+"v1/im/reports/abuse/reasons", serveIMAbuseReasons) mux.HandleFunc(config.ApiPath+"v1/im/reports/abuse/reasons", serveIMAbuseReasons)
mux.HandleFunc(config.ApiPath+"v1/im/reports/abuse", serveIMAbuseReports) mux.HandleFunc(config.ApiPath+"v1/im/reports/abuse", serveIMAbuseReports)
mux.HandleFunc(config.ApiPath+"v1/im/reports/daily/statistics", serveIMDailyReportStatistics)
mux.HandleFunc(config.ApiPath+"v1/im/reports/daily/reminders", serveIMDailyReportReminders)
mux.HandleFunc(config.ApiPath+"v1/im/reports/daily", serveIMDailyReports)
mux.HandleFunc(config.ApiPath+"v1/im/reports/daily/", serveIMDailyReportResource)
mux.HandleFunc(config.ApiPath+"v1/im/reports/monthly/statistics", serveIMMonthlyReportStatistics)
mux.HandleFunc(config.ApiPath+"v1/im/reports/monthly", serveIMMonthlyReports)
mux.HandleFunc(config.ApiPath+"v1/im/reports/monthly/", serveIMMonthlyReportResource)
mux.HandleFunc(config.ApiPath+"v1/im/executive-mailbox/config", serveIMMailboxConfig)
mux.HandleFunc(config.ApiPath+"v1/im/executive-mailbox/submissions", serveIMMailboxSubmissions)
mux.HandleFunc(config.ApiPath+"v1/im/feedback/config", serveIMFeedbackConfig) mux.HandleFunc(config.ApiPath+"v1/im/feedback/config", serveIMFeedbackConfig)
mux.HandleFunc(config.ApiPath+"v1/im/feedback", serveIMFeedback) mux.HandleFunc(config.ApiPath+"v1/im/feedback", serveIMFeedback)
mux.HandleFunc(config.ApiPath+"v1/im/contact-us", serveIMContactUs)
mux.HandleFunc(config.ApiPath+"v1/im/app/releases/latest", serveIMAppReleaseLatest) mux.HandleFunc(config.ApiPath+"v1/im/app/releases/latest", serveIMAppReleaseLatest)
mux.HandleFunc(config.ApiPath+"v1/im/me", serveIMMe) mux.HandleFunc(config.ApiPath+"v1/im/me", serveIMMe)
mux.HandleFunc(config.ApiPath+"v1/im/me/profile", serveIMMeProfile) mux.HandleFunc(config.ApiPath+"v1/im/me/profile", serveIMMeProfile)
@@ -942,6 +929,10 @@ func main() {
mux.HandleFunc(adminQueryPrefix, func(wrt http.ResponseWriter, req *http.Request) { mux.HandleFunc(adminQueryPrefix, func(wrt http.ResponseWriter, req *http.Request) {
serveInternalAdminQuery(wrt, req, adminQueryPrefix) serveInternalAdminQuery(wrt, req, adminQueryPrefix)
}) })
cultivatePrefix := config.ApiPath + "internal/cultivate/"
mux.HandleFunc(cultivatePrefix, func(wrt http.ResponseWriter, req *http.Request) {
serveInternalCultivate(wrt, req, cultivatePrefix)
})
// Handle long polling clients. Enable compression. // Handle long polling clients. Enable compression.
mux.Handle(config.ApiPath+"v0/channels/lp", gh.CompressHandler(http.HandlerFunc(serveLongPoll))) mux.Handle(config.ApiPath+"v0/channels/lp", gh.CompressHandler(http.HandlerFunc(serveLongPoll)))
if config.Media != nil { if config.Media != nil {
+6 -3
View File
@@ -76,7 +76,7 @@
}, },
"store_config": { "store_config": {
"uid_key": "la6YsO+bNX/+XIkOqc5Svw==", "uid_key": "dkXxWLwqUNRfYmdXRzOkZw==",
"max_results": 1024, "max_results": 1024,
"use_adapter": "mysql", "use_adapter": "mysql",
"adapters": { "adapters": {
@@ -137,7 +137,6 @@
"message_persist": "im.message.persist", "message_persist": "im.message.persist",
"group_message_persist": "im.group.message.persist", "group_message_persist": "im.group.message.persist",
"message_push": "im.message.push", "message_push": "im.message.push",
"workbench_external": "im.workbench.event",
"file_security": "im.file.security", "file_security": "im.file.security",
"cache_invalidate": "im.cache.invalidate", "cache_invalidate": "im.cache.invalidate",
"audit_event": "im.audit.event", "audit_event": "im.audit.event",
@@ -150,7 +149,11 @@
"services": { "services": {
"hotgo-admin": { "hotgo-admin": {
"secret": "8569f5626a54950539eece9f731c9e3bf073daa74af8d23735d19afacac40f4b", "secret": "8569f5626a54950539eece9f731c9e3bf073daa74af8d23735d19afacac40f4b",
"allow": ["admin_query"] "allow": ["admin_query", "cultivate"]
},
"cultivate-server": {
"secret": "cultivate-local-internal-secret-change-me",
"allow": ["cultivate"]
} }
} }
}, },
+1
View File
@@ -5,6 +5,7 @@ import "strings"
var visibleTextKeys = map[string]bool{ var visibleTextKeys = map[string]bool{
"txt": true, "text": true, "title": true, "caption": true, "txt": true, "text": true, "title": true, "caption": true,
"description": true, "file_name": true, "filename": true, "name": true, "description": true, "file_name": true, "filename": true, "name": true,
"url": true,
} }
func ExtractText(content any) string { func ExtractText(content any) string {
+9 -1
View File
@@ -2,6 +2,8 @@ package moderation
import ( import (
"context" "context"
"sort"
"strings"
"testing" "testing"
"time" "time"
@@ -19,6 +21,7 @@ func (c testConfig) Provider() (*types.ModerationProviderConfig, error) {
} }
func (c testConfig) Policy(string) (*types.ModerationPolicy, error) { return c.policy, nil } func (c testConfig) Policy(string) (*types.ModerationPolicy, error) { return c.policy, nil }
func (c testConfig) Words(string) (*types.SensitiveWordSet, error) { return c.words, nil } func (c testConfig) Words(string) (*types.SensitiveWordSet, error) { return c.words, nil }
func (c testConfig) SecurityPolicy(string) (*types.SecurityPolicy, error) { return nil, nil }
type testAudit struct{} type testAudit struct{}
@@ -32,7 +35,12 @@ func (testAudit) RecordContentModeration(types.TenantID, *types.ContentModeratio
func TestExtractText(t *testing.T) { func TestExtractText(t *testing.T) {
content := map[string]any{"txt": "正文", "ent": []any{map[string]any{"data": map[string]any{"title": "查看详情", "url": "https://example.com"}}}, "meta": "ignore"} content := map[string]any{"txt": "正文", "ent": []any{map[string]any{"data": map[string]any{"title": "查看详情", "url": "https://example.com"}}}, "meta": "ignore"}
if got := ExtractText(content); got != "正文\n查看详情" && got != "查看详情\n正文" { gotParts := strings.Split(ExtractText(content), "\n")
sort.Strings(gotParts)
wantParts := []string{"https://example.com", "查看详情", "正文"}
sort.Strings(wantParts)
if strings.Join(gotParts, "\n") != strings.Join(wantParts, "\n") {
got := ExtractText(content)
t.Fatalf("ExtractText()=%q", got) t.Fatalf("ExtractText()=%q", got)
} }
} }
+2
View File
@@ -16,6 +16,8 @@ const (
ScenePrivateMessage = "private_message" ScenePrivateMessage = "private_message"
SceneGroupMessage = "group_message" SceneGroupMessage = "group_message"
SceneGroupAnnouncement = "group_announcement" SceneGroupAnnouncement = "group_announcement"
SceneFeedPost = "feed_post"
SceneFeedComment = "feed_comment"
ActionAllow = "allow" ActionAllow = "allow"
ActionBlock = "block" ActionBlock = "block"
+32
View File
@@ -278,6 +278,7 @@ func (t *Topic) presUsersOfInterest(what, ua string) {
// Publish user's update to his/her users of interest on their 'me' topic while user's 'me' topic is offline // Publish user's update to his/her users of interest on their 'me' topic while user's 'me' topic is offline
// Case A: user is being deleted, "gone". // Case A: user is being deleted, "gone".
// Case B: user's last session just disconnected, "off".
func presUsersOfInterestOffline(tenantID types.TenantID, uid types.Uid, subs []types.Subscription, what string) { func presUsersOfInterestOffline(tenantID types.TenantID, uid types.Uid, subs []types.Subscription, what string) {
// Push update to subscriptions // Push update to subscriptions
for i := range subs { for i := range subs {
@@ -286,6 +287,37 @@ func presUsersOfInterestOffline(tenantID types.TenantID, uid types.Uid, subs []t
continue continue
} }
if types.GetTopicCat(subs[i].Topic) == types.TopicCatP2P {
// A presence request routed to the p2p topic would be silently
// dropped: the topic is loaded under its canonical p2p name while
// the request carries Topic="me" (notifyOnOrSkip), so the
// xoriginal check in handlePresence rejects it. Deliver to the
// contact's 'me' topic directly instead, the same way
// presSingleUserOffline does.
uid1, uid2, err := types.ParseP2P(subs[i].Topic)
if err != nil {
continue
}
contact := uid1
if contact == uid {
contact = uid2
}
if contact == uid || contact.IsZero() {
continue
}
routeUserServerMessage(contact, &ServerComMessage{
TenantID: tenantID,
Pres: &MsgServerPres{
Topic: notifyOn,
What: what,
Src: subs[i].Topic,
WantReply: false,
},
RcptTo: contact.UserId(),
})
continue
}
globals.hub.routeSrv <- &ServerComMessage{ globals.hub.routeSrv <- &ServerComMessage{
TenantID: tenantID, TenantID: tenantID,
Pres: &MsgServerPres{ Pres: &MsgServerPres{
+56
View File
@@ -91,6 +91,8 @@ func (t *Topic) pushForData(fromUid types.Uid, data *MsgServerData, msgMarkedAsR
// Number of attached sessions the data message will be delivered to. // Number of attached sessions the data message will be delivered to.
// Push notifications sent to users with non-zero online sessions will be marked silent. // Push notifications sent to users with non-zero online sessions will be marked silent.
Delivered: online, Delivered: online,
Silent: t.cat == types.TopicCatGrp &&
boolFromAny(groupPrivateMap(pud.private)["mute_notifications"]),
// Unread counts are incremented for all recipients, // Unread counts are incremented for all recipients,
// and for sender only if the message wasnt't marked 'read' by the sender // and for sender only if the message wasnt't marked 'read' by the sender
ShouldIncrementUnreadCountInCache: uid != fromUid || !msgMarkedAsReadBySender, ShouldIncrementUnreadCountInCache: uid != fromUid || !msgMarkedAsReadBySender,
@@ -104,6 +106,60 @@ func (t *Topic) pushForData(fromUid types.Uid, data *MsgServerData, msgMarkedAsR
return nil return nil
} }
// sendPushForStoredGroupMessage handles server-generated group messages which
// bypass Topic.handlePubBroadcast but must still update per-user unread counts
// and notify offline devices.
func sendPushForStoredGroupMessage(tenantID types.TenantID, msg *types.Message) {
if msg == nil || globals.usersUpdate == nil || types.GetTopicCat(msg.Topic) != types.TopicCatGrp || store.Topics == nil {
return
}
subs, err := store.Topics.GetSubs(tenantID, msg.Topic, nil)
if err != nil {
logs.Warn.Printf("group system push: load recipients topic=%s: %v", msg.Topic, err)
return
}
receipt := &push.Receipt{
TenantID: tenantID,
To: make(map[types.Uid]push.Recipient, len(subs)),
Payload: push.Payload{
What: push.ActMsg, Topic: msg.Topic, Timestamp: msg.CreatedAt, SeqId: msg.SeqId,
Content: msg.Content,
},
}
if mime, ok := msg.Head["mime"].(string); ok {
receipt.Payload.ContentType = mime
}
for _, sub := range subs {
uid := types.ParseUid(sub.User)
mode := sub.ModeWant & sub.ModeGiven
if uid.IsZero() || !mode.IsPresencer() || !mode.IsReader() {
continue
}
delivered := 0
if globals.sessionStore != nil {
globals.sessionStore.Range(func(_ string, sess *Session) bool {
if sess != nil && !sess.isCluster() && !sess.isProxy() && sess.tenantID == tenantID && sess.uid == uid {
delivered++
}
return true
})
}
if presence, presenceErr := store.GetUserPresence(tenantID, uid); presenceErr != nil {
logs.Warn.Printf("group system push: read presence topic=%s uid=%s: %v", msg.Topic, uid.UserId(), presenceErr)
} else if presence != nil && presence.Online && presence.SessionCount > delivered {
delivered = presence.SessionCount
}
receipt.To[uid] = push.Recipient{
Delivered: delivered,
Silent: boolFromAny(groupPrivateMap(sub.Private)["mute_notifications"]),
ShouldIncrementUnreadCountInCache: true,
}
}
if len(receipt.To) > 0 {
sendPush(receipt)
}
}
func (t *Topic) preparePushForSubReceipt(fromUid types.Uid, now time.Time) *push.Receipt { func (t *Topic) preparePushForSubReceipt(fromUid types.Uid, now time.Time) *push.Receipt {
// P2P push payloads use the canonical p2p... topic. usr... is accepted only // P2P push payloads use the canonical p2p... topic. usr... is accepted only
// as an input alias. // as an input alias.
+1 -1
View File
@@ -158,7 +158,7 @@ func PrepareV1Notifications(rcpt *push.Receipt, config *configType) ([]*fcmv1.Me
for uid, devList := range devices { for uid, devList := range devices {
topic := rcpt.Payload.Topic topic := rcpt.Payload.Topic
userData := data userData := data
if rcpt.To[uid].Delivered > 0 { if rcpt.To[uid].Delivered > 0 || rcpt.To[uid].Silent {
userData = clonePayload(data) userData = clonePayload(data)
// Silence the push for user who have received the data interactively. // Silence the push for user who have received the data interactively.
userData["silent"] = "true" userData["silent"] = "true"
+84 -1
View File
@@ -26,6 +26,7 @@ const (
defaultPushPath = "/v3/push" defaultPushPath = "/v3/push"
defaultTTL = 86400 defaultTTL = 86400
defaultCallTTL = 30 defaultCallTTL = 30
callCategory = "im_call"
) )
var handler Handler var handler Handler
@@ -81,6 +82,8 @@ type iosPayload struct {
Alert iosAlert `json:"alert"` Alert iosAlert `json:"alert"`
Sound string `json:"sound,omitempty"` Sound string `json:"sound,omitempty"`
Badge string `json:"badge,omitempty"` Badge string `json:"badge,omitempty"`
Category string `json:"category,omitempty"`
ContentAvailable int `json:"content-available,omitempty"`
Extras map[string]string `json:"extras,omitempty"` Extras map[string]string `json:"extras,omitempty"`
} }
@@ -327,6 +330,7 @@ func preparePayloads(rcpt *push.Receipt, config *configType) ([]*jpushPayload, [
logs.Warn.Println("jpush: could not build payload:", err) logs.Warn.Println("jpush: could not build payload:", err)
return nil, nil return nil, nil
} }
hydrateCallInviteData(rcpt.TenantID, data)
if pushExpired(data) { if pushExpired(data) {
return nil, nil return nil, nil
} }
@@ -351,7 +355,7 @@ func preparePayloads(rcpt *push.Receipt, config *configType) ([]*jpushPayload, [
for _, uid := range uids { for _, uid := range uids {
recipient := rcpt.To[uid] recipient := rcpt.To[uid]
userData := cloneData(data) userData := cloneData(data)
if recipient.Delivered > 0 { if recipient.Delivered > 0 || recipient.Silent {
userData["silent"] = "true" userData["silent"] = "true"
} }
topic := rcpt.Payload.Topic topic := rcpt.Payload.Topic
@@ -424,6 +428,9 @@ func pushData(pl push.Payload) (map[string]string, error) {
if pl.From != "" { if pl.From != "" {
data["caller_id"] = pl.From data["caller_id"] = pl.From
} }
if display := payloadContentFirstString(pl.Content, "caller_display", "callerDisplay", "caller_name", "callerName", "display_name", "from_display"); display != "" {
data["caller_display"] = display
}
} }
if pl.Replace != "" { if pl.Replace != "" {
data["replace"] = pl.Replace data["replace"] = pl.Replace
@@ -498,6 +505,8 @@ func buildPayload(audience map[string][]string, topic string, data map[string]st
Alert: iosAlert{Title: title, Body: alert}, Alert: iosAlert{Title: title, Body: alert},
Sound: "default", Sound: "default",
Badge: badge, Badge: badge,
Category: iosCategory(data),
ContentAvailable: iosContentAvailable(data),
Extras: data, Extras: data,
}, },
HMOS: &hmosPayload{ HMOS: &hmosPayload{
@@ -514,9 +523,16 @@ func alertText(data map[string]string, config *configType) (string, string) {
title := config.Title title := config.Title
switch data["type"] { switch data["type"] {
case "call_invite": case "call_invite":
caller := strings.TrimSpace(data["caller_display"])
if data["call_type"] == "audio" { if data["call_type"] == "audio" {
if caller != "" {
return caller + "邀请你语音通话", "语音通话"
}
return "收到一通语音通话", "语音通话" return "收到一通语音通话", "语音通话"
} }
if caller != "" {
return caller + "邀请你视频通话", "视频通话"
}
return "收到一通视频通话", "视频通话" return "收到一通视频通话", "视频通话"
case "call_missed": case "call_missed":
return "你有一通未接来电", "未接来电" return "你有一通未接来电", "未接来电"
@@ -527,6 +543,64 @@ func alertText(data map[string]string, config *configType) (string, string) {
} }
} }
func iosCategory(data map[string]string) string {
if data["type"] == "call_invite" {
return callCategory
}
return ""
}
func iosContentAvailable(data map[string]string) int {
if data["type"] == "call_invite" {
return 1
}
return 0
}
func hydrateCallInviteData(tenantID t.TenantID, data map[string]string) {
if data["type"] != "call_invite" || strings.TrimSpace(data["caller_display"]) != "" {
return
}
callerID := strings.TrimSpace(data["caller_id"])
if callerID == "" || store.Users == nil {
return
}
uid := t.ParseUserId(callerID)
if uid.IsZero() {
return
}
user, err := store.Users.Get(tenantID, uid)
if err != nil || user == nil {
return
}
if display := displayNameFromPublic(user.Public); display != "" {
data["caller_display"] = display
}
}
func displayNameFromPublic(public any) string {
for _, key := range []string{"display_name", "fn", "nickname", "name"} {
if value := publicMapString(public, key); value != "" {
return value
}
}
return ""
}
func publicMapString(public any, key string) string {
switch values := public.(type) {
case map[string]any:
value, ok := values[key]
if !ok || value == nil {
return ""
}
return strings.TrimSpace(fmt.Sprint(value))
case map[string]string:
return strings.TrimSpace(values[key])
}
return ""
}
func cidFor(topic string, data map[string]string) string { func cidFor(topic string, data map[string]string) string {
parts := []string{data["type"], topic, data["seq"], data["replace"]} parts := []string{data["type"], topic, data["seq"], data["replace"]}
return strings.Trim(strings.Join(parts, "_"), "_") return strings.Trim(strings.Join(parts, "_"), "_")
@@ -560,6 +634,15 @@ func payloadContentString(content any, key string) string {
return "" return ""
} }
func payloadContentFirstString(content any, keys ...string) string {
for _, key := range keys {
if value := payloadContentString(content, key); value != "" {
return value
}
}
return ""
}
func truncateRunes(value string, limit int) string { func truncateRunes(value string, limit int) string {
if limit <= 0 || len(value) <= limit { if limit <= 0 || len(value) <= limit {
return value return value
+14
View File
@@ -178,6 +178,7 @@ func TestBuildPayloadUsesShortTTLForCalls(t *testing.T) {
"topic": "p2pTopic", "topic": "p2pTopic",
"seq": "30", "seq": "30",
"call_type": "audio", "call_type": "audio",
"caller_display": "张三",
"expires_at": expiresAt, "expires_at": expiresAt,
}, 0, config) }, 0, config)
if payload.Options.TimeToLive > 30 { if payload.Options.TimeToLive > 30 {
@@ -186,6 +187,15 @@ func TestBuildPayloadUsesShortTTLForCalls(t *testing.T) {
if payload.Notification.Android.Title != "语音通话" { if payload.Notification.Android.Title != "语音通话" {
t.Fatalf("unexpected title: %s", payload.Notification.Android.Title) t.Fatalf("unexpected title: %s", payload.Notification.Android.Title)
} }
if payload.Notification.IOS.Category != callCategory {
t.Fatalf("unexpected iOS category: %s", payload.Notification.IOS.Category)
}
if payload.Notification.IOS.ContentAvailable != 1 {
t.Fatalf("unexpected iOS content-available: %d", payload.Notification.IOS.ContentAvailable)
}
if payload.Notification.IOS.Alert.Body != "张三邀请你语音通话" {
t.Fatalf("unexpected body: %s", payload.Notification.IOS.Alert.Body)
}
} }
func TestPushDataExtractsCallFields(t *testing.T) { func TestPushDataExtractsCallFields(t *testing.T) {
@@ -200,6 +210,7 @@ func TestPushDataExtractsCallFields(t *testing.T) {
Content: map[string]any{ Content: map[string]any{
"callId": "call_1", "callId": "call_1",
"callType": "audio", "callType": "audio",
"caller_display": "张三",
}, },
Timestamp: time.Date(2026, 9, 4, 10, 0, 0, 0, time.UTC), Timestamp: time.Date(2026, 9, 4, 10, 0, 0, 0, time.UTC),
}) })
@@ -212,6 +223,9 @@ func TestPushDataExtractsCallFields(t *testing.T) {
if data["expires_at"] != expiresAt.Format(time.RFC3339Nano) { if data["expires_at"] != expiresAt.Format(time.RFC3339Nano) {
t.Fatalf("unexpected expires_at: %#v", data) t.Fatalf("unexpected expires_at: %#v", data)
} }
if data["caller_display"] != "张三" {
t.Fatalf("unexpected caller_display: %#v", data)
}
} }
func TestPreparePayloadsDropsExpiredCallInvite(t *testing.T) { func TestPreparePayloadsDropsExpiredCallInvite(t *testing.T) {
+2
View File
@@ -28,6 +28,8 @@ type Recipient struct {
Delivered int `json:"delivered"` Delivered int `json:"delivered"`
// List of user's devices that the packet was delivered to (if known). Len(Devices) >= Delivered // List of user's devices that the packet was delivered to (if known). Len(Devices) >= Delivered
Devices []string `json:"devices,omitempty"` Devices []string `json:"devices,omitempty"`
// Deliver as a data-only notification without showing an alert.
Silent bool `json:"silent,omitempty"`
// Unread count to include in the push // Unread count to include in the push
Unread int `json:"unread"` Unread int `json:"unread"`
// Indicates whether unread counter in the cache should be incremented before sending the push. // Indicates whether unread counter in the cache should be incremented before sending the push.
+112
View File
@@ -0,0 +1,112 @@
package aliyun
import (
"errors"
"strconv"
"strings"
"time"
cloudauth "github.com/alibabacloud-go/cloudauth-20190307/v3/client"
openapi "github.com/alibabacloud-go/darabonba-openapi/v2/client"
util "github.com/alibabacloud-go/tea-utils/v2/service"
"github.com/alibabacloud-go/tea/dara"
)
type Config struct {
AccessKeyID string
AccessKeySecret string
RegionID string
Endpoint string
TimeoutMS int
}
type ID2MetaResult struct {
Passed bool
BizCode string
RequestID string
ProviderCode string
ProviderMessage string
ProviderStatusCode int32
Latency time.Duration
}
type ID2MetaClient struct {
client *cloudauth.Client
timeoutMS int
}
func NewID2MetaClient(config Config) (*ID2MetaClient, error) {
if strings.TrimSpace(config.AccessKeyID) == "" || strings.TrimSpace(config.AccessKeySecret) == "" {
return nil, errors.New("missing aliyun realname credentials")
}
if strings.TrimSpace(config.RegionID) == "" {
config.RegionID = "cn-hangzhou"
}
clientConfig := &openapi.Config{
AccessKeyId: dara.String(strings.TrimSpace(config.AccessKeyID)),
AccessKeySecret: dara.String(strings.TrimSpace(config.AccessKeySecret)),
RegionId: dara.String(strings.TrimSpace(config.RegionID)),
Protocol: dara.String("https"),
}
if endpoint := strings.TrimSpace(config.Endpoint); endpoint != "" {
clientConfig.Endpoint = dara.String(endpoint)
}
client, err := cloudauth.NewClient(clientConfig)
if err != nil {
return nil, err
}
if config.TimeoutMS <= 0 {
config.TimeoutMS = 1500
}
return &ID2MetaClient{client: client, timeoutMS: config.TimeoutMS}, nil
}
func (c *ID2MetaClient) Verify(name, idNumber string) (*ID2MetaResult, error) {
if c == nil || c.client == nil {
return nil, errors.New("nil aliyun realname client")
}
name = strings.TrimSpace(name)
idNumber = strings.TrimSpace(idNumber)
if name == "" || idNumber == "" {
return nil, errors.New("missing realname identity fields")
}
started := time.Now()
resp, err := c.client.Id2MetaVerifyWithOptions(&cloudauth.Id2MetaVerifyRequest{
UserName: dara.String(name),
IdentifyNum: dara.String(idNumber),
ParamType: dara.String("normal"),
}, (&util.RuntimeOptions{}).SetConnectTimeout(c.timeoutMS).SetReadTimeout(c.timeoutMS).SetAutoretry(false))
latency := time.Since(started)
if err != nil {
return nil, err
}
result := &ID2MetaResult{Latency: latency}
if resp != nil {
result.ProviderStatusCode = dara.Int32Value(resp.StatusCode)
}
if resp == nil || resp.Body == nil {
return result, errors.New("empty aliyun realname response")
}
result.RequestID = dara.StringValue(resp.Body.RequestId)
result.ProviderCode = dara.StringValue(resp.Body.Code)
result.ProviderMessage = dara.StringValue(resp.Body.Message)
if resp.Body.ResultObject != nil {
result.BizCode = strings.TrimSpace(dara.StringValue(resp.Body.ResultObject.BizCode))
}
if result.ProviderCode != "" && result.ProviderCode != "200" {
return result, errors.New("aliyun realname response code " + result.ProviderCode)
}
if result.BizCode == "" {
return result, errors.New("empty aliyun realname biz code")
}
result.Passed = result.BizCode == "1"
return result, nil
}
func ParseTimeoutMS(value string, fallback int) int {
parsed, err := strconv.Atoi(strings.TrimSpace(value))
if err != nil || parsed <= 0 {
return fallback
}
return parsed
}
@@ -0,0 +1,36 @@
package aliyun
import (
"os"
"testing"
)
func TestLiveID2MetaVerify(t *testing.T) {
accessKeyID := os.Getenv("ALIYUN_REALNAME_ACCESS_KEY_ID")
accessKeySecret := os.Getenv("ALIYUN_REALNAME_ACCESS_KEY_SECRET")
if accessKeyID == "" || accessKeySecret == "" {
t.Skip("ALIYUN_REALNAME_ACCESS_KEY_ID and ALIYUN_REALNAME_ACCESS_KEY_SECRET are required")
}
client, err := NewID2MetaClient(Config{
AccessKeyID: accessKeyID,
AccessKeySecret: accessKeySecret,
RegionID: envOrDefault("ALIYUN_REALNAME_REGION_ID", "cn-hangzhou"),
Endpoint: os.Getenv("ALIYUN_REALNAME_ENDPOINT"),
TimeoutMS: ParseTimeoutMS(os.Getenv("ALIYUN_REALNAME_TIMEOUT_MS"), 3000),
})
if err != nil {
t.Fatal(err)
}
result, err := client.Verify(envOrDefault("ALIYUN_REALNAME_TEST_NAME", "张三"), envOrDefault("ALIYUN_REALNAME_TEST_ID_NUMBER", "110101199003071234"))
if err != nil {
t.Fatalf("live id2meta verify failed: %v result=%+v", err, result)
}
t.Logf("live id2meta result: passed=%v biz_code=%s request_id=%s code=%s message=%s", result.Passed, result.BizCode, result.RequestID, result.ProviderCode, result.ProviderMessage)
}
func envOrDefault(key, fallback string) string {
if value := os.Getenv(key); value != "" {
return value
}
return fallback
}
+108
View File
@@ -7,6 +7,7 @@ import (
"strings" "strings"
"time" "time"
"github.com/tinode/chat/server/auth"
"github.com/tinode/chat/server/logs" "github.com/tinode/chat/server/logs"
immetrics "github.com/tinode/chat/server/metrics" immetrics "github.com/tinode/chat/server/metrics"
"github.com/tinode/chat/server/store" "github.com/tinode/chat/server/store"
@@ -16,6 +17,7 @@ import (
const ( const (
realtimeKindMessage = "message" realtimeKindMessage = "message"
realtimeKindSessionControl = "session_control" realtimeKindSessionControl = "session_control"
realtimeKindCallEvent = "call_event"
realtimeTargetTopic = "topic" realtimeTargetTopic = "topic"
realtimeTargetUser = "user" realtimeTargetUser = "user"
@@ -23,6 +25,16 @@ const (
realtimeTargetPlatf = "platform" realtimeTargetPlatf = "platform"
) )
type realtimeCallEventMessage struct {
Topic string `json:"topic"`
From string `json:"from"`
Sid string `json:"sid,omitempty"`
Event string `json:"event"`
Seq int64 `json:"seq"`
Payload json.RawMessage `json:"payload,omitempty"`
Timestamp time.Time `json:"ts,omitempty"`
}
func startRedisRealtimeFanout(ctx context.Context) { func startRedisRealtimeFanout(ctx context.Context) {
if err := store.StartRealtimeFanoutSubscriber(ctx, currentNodeID(), deliverRealtimeFanoutEnvelope); err != nil { if err := store.StartRealtimeFanoutSubscriber(ctx, currentNodeID(), deliverRealtimeFanoutEnvelope); err != nil {
immetrics.IncRealtimeFanout("subscribe", realtimeKindMessage, realtimeTargetTopic, "error") immetrics.IncRealtimeFanout("subscribe", realtimeKindMessage, realtimeTargetTopic, "error")
@@ -65,6 +77,35 @@ func publishRealtimeUserMessage(uid types.Uid, msg *ServerComMessage) {
immetrics.IncRealtimeFanout("publish", realtimeKindMessage, realtimeTargetUser, "ok") immetrics.IncRealtimeFanout("publish", realtimeKindMessage, realtimeTargetUser, "ok")
} }
func publishRealtimeCallEvent(msg *ClientComMessage) {
if msg == nil || msg.Note == nil || !msg.TenantID.IsValid() ||
msg.RcptTo == "" || msg.AsUser == "" || msg.Note.What != "call" ||
!isDetachedCallEvent(msg.Note.Event) || globals.cluster != nil {
return
}
sid := ""
if msg.sess != nil {
sid = msg.sess.sid
}
event := realtimeCallEventMessage{
Topic: msg.RcptTo,
From: msg.AsUser,
Sid: sid,
Event: normalizeCallEvent(msg.Note.Event),
Seq: msg.Note.SeqId,
Payload: msg.Note.Payload,
Timestamp: msg.Timestamp,
}
dedupKey := "call:" + msg.RcptTo + ":" + msg.AsUser + ":" + event.Event + ":" + strconv.FormatInt(msg.Note.SeqId, 10)
if err := store.PublishRealtimeFanout(currentNodeID(), msg.TenantID, realtimeKindCallEvent, realtimeTargetTopic, msg.RcptTo, sid, dedupKey, event); err != nil {
immetrics.IncRealtimeFanout("publish", realtimeKindCallEvent, realtimeTargetTopic, "error")
logs.Warn.Printf("redis realtime fanout: publish call event topic=%s from=%s event=%s seq=%d failed: %v",
msg.RcptTo, msg.AsUser, event.Event, msg.Note.SeqId, err)
return
}
immetrics.IncRealtimeFanout("publish", realtimeKindCallEvent, realtimeTargetTopic, "ok")
}
func publishRealtimeEvictUser(tenantID types.TenantID, uid types.Uid, reason string, skipSid string) { func publishRealtimeEvictUser(tenantID types.TenantID, uid types.Uid, reason string, skipSid string) {
if !tenantID.IsValid() || uid.IsZero() { if !tenantID.IsValid() || uid.IsZero() {
immetrics.IncRealtimeFanout("publish", realtimeKindSessionControl, realtimeTargetUser, "skip_invalid") immetrics.IncRealtimeFanout("publish", realtimeKindSessionControl, realtimeTargetUser, "skip_invalid")
@@ -175,12 +216,79 @@ func deliverRealtimeFanoutEnvelope(envelope *store.RealtimeFanoutEnvelope) {
switch envelope.Kind { switch envelope.Kind {
case realtimeKindSessionControl: case realtimeKindSessionControl:
deliverRealtimeSessionControl(envelope, &msg) deliverRealtimeSessionControl(envelope, &msg)
case realtimeKindCallEvent:
deliverRealtimeCallEvent(envelope)
default: default:
immetrics.ObserveRealtimeFanoutDelivery(envelope.Kind, envelope.TargetType, msg.Timestamp) immetrics.ObserveRealtimeFanoutDelivery(envelope.Kind, envelope.TargetType, msg.Timestamp)
deliverRealtimeMessage(envelope, &msg) deliverRealtimeMessage(envelope, &msg)
} }
} }
func deliverRealtimeCallEvent(envelope *store.RealtimeFanoutEnvelope) {
var event realtimeCallEventMessage
if err := json.Unmarshal(envelope.Message, &event); err != nil {
immetrics.IncRealtimeFanout("deliver", realtimeKindCallEvent, realtimeTargetTopic, "malformed")
logs.Warn.Println("redis realtime fanout: malformed call event", err)
return
}
topic := strings.TrimSpace(event.Topic)
if topic == "" {
topic = strings.TrimSpace(envelope.Target)
}
if topic == "" || event.From == "" || event.Seq <= 0 || !isDetachedCallEvent(event.Event) {
immetrics.IncRealtimeFanout("deliver", realtimeKindCallEvent, realtimeTargetTopic, "skip_invalid")
return
}
key := types.TopicKey{TenantID: envelope.TenantID, Topic: topic}
if globals.hub == nil || globals.hub.topicGet(key) == nil {
immetrics.IncRealtimeFanout("deliver", realtimeKindCallEvent, realtimeTargetTopic, "no_local_topic")
return
}
uid := types.ParseUserId(event.From)
if uid.IsZero() {
immetrics.IncRealtimeFanout("deliver", realtimeKindCallEvent, realtimeTargetTopic, "skip_invalid_user")
return
}
ts := event.Timestamp
if ts.IsZero() {
ts = types.TimeNow()
}
msg := &ClientComMessage{
Note: &MsgClientNote{
Topic: topic,
What: "call",
Event: normalizeCallEvent(event.Event),
SeqId: event.Seq,
Payload: event.Payload,
},
Original: topic,
RcptTo: topic,
AsUser: event.From,
AuthLvl: int(auth.LevelAuth),
Timestamp: ts,
TenantID: envelope.TenantID,
sess: &Session{
proto: NONE,
sid: "redis-call:" + strings.TrimSpace(event.Sid),
uid: uid,
tenantID: envelope.TenantID,
authLvl: auth.LevelAuth,
lastAction: ts.UnixNano(),
},
init: true,
}
if msg.sess.sid == "redis-call:" {
msg.sess.sid = "redis-call:" + event.From
}
select {
case globals.hub.routeCli <- msg:
immetrics.IncRealtimeFanout("deliver", realtimeKindCallEvent, realtimeTargetTopic, "hub_queued")
default:
immetrics.IncRealtimeFanout("deliver", realtimeKindCallEvent, realtimeTargetTopic, "hub_full")
logs.Err.Println("redis realtime fanout: hub routeCli queue full for call event", topic)
}
}
func deliverRealtimeMessage(envelope *store.RealtimeFanoutEnvelope, msg *ServerComMessage) { func deliverRealtimeMessage(envelope *store.RealtimeFanoutEnvelope, msg *ServerComMessage) {
if globals.sessionStore == nil { if globals.sessionStore == nil {
immetrics.IncRealtimeFanout("deliver", envelope.Kind, envelope.TargetType, "skip_no_session_store") immetrics.IncRealtimeFanout("deliver", envelope.Kind, envelope.TargetType, "skip_no_session_store")
+60
View File
@@ -2,6 +2,7 @@ package main
import ( import (
"encoding/json" "encoding/json"
"sync"
"testing" "testing"
"time" "time"
@@ -259,6 +260,65 @@ func TestRealtimeP2PTopicFallbackRewritesVisibleInfoTopic(t *testing.T) {
} }
} }
func TestRealtimeCallEventRoutesToLocalTopic(t *testing.T) {
oldHub := globals.hub
defer func() { globals.hub = oldHub }()
tenantID := types.TenantID(1)
caller := types.Uid(1001)
callee := types.Uid(1002)
topic := caller.P2PName(callee)
if topic == "" {
t.Fatal("failed to create p2p topic")
}
topics := &sync.Map{}
topics.Store(types.TopicKey{TenantID: tenantID, Topic: topic}, &Topic{name: topic, tenantID: tenantID})
globals.hub = &Hub{
topics: topics,
routeCli: make(chan *ClientComMessage, 1),
}
payload := json.RawMessage(`{"callId":"call-1"}`)
event := realtimeCallEventMessage{
Topic: topic,
From: callee.UserId(),
Sid: "callee-sid",
Event: constCallEventAccept,
Seq: 12,
Payload: payload,
}
raw, err := json.Marshal(event)
if err != nil {
t.Fatal(err)
}
deliverRealtimeFanoutEnvelope(&store.RealtimeFanoutEnvelope{
TenantID: tenantID,
Kind: realtimeKindCallEvent,
TargetType: realtimeTargetTopic,
Target: topic,
Message: raw,
})
select {
case got := <-globals.hub.routeCli:
if got.TenantID != tenantID || got.RcptTo != topic || got.AsUser != callee.UserId() {
t.Fatalf("unexpected routed message: tenant=%d rcpt=%q as=%q", got.TenantID, got.RcptTo, got.AsUser)
}
if got.Note == nil || got.Note.What != "call" || got.Note.Event != constCallEventAccept || got.Note.SeqId != 12 {
t.Fatalf("unexpected call note: %#v", got.Note)
}
if string(got.Note.Payload) != string(payload) {
t.Fatalf("payload = %s, want %s", got.Note.Payload, payload)
}
if got.sess == nil || got.sess.uid != callee || got.sess.sid != "redis-call:callee-sid" {
t.Fatalf("unexpected synthetic session: %#v", got.sess)
}
default:
t.Fatal("expected call event to be routed to local topic")
}
}
func TestRealtimeInfoDedupKeySeparatesReadRecv(t *testing.T) { func TestRealtimeInfoDedupKeySeparatesReadRecv(t *testing.T) {
read := realtimeMessageDedupKey("p2pABC", &ServerComMessage{ read := realtimeMessageDedupKey("p2pABC", &ServerComMessage{
Info: &MsgServerInfo{From: "usrA", What: "read", SeqId: 12}, Info: &MsgServerInfo{From: "usrA", What: "read", SeqId: 12},
+47 -1
View File
@@ -12,6 +12,7 @@ package main
import ( import (
"container/list" "container/list"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"net/http" "net/http"
"os" "os"
@@ -465,9 +466,19 @@ func (s *Session) cleanUp(expired bool) {
if err := store.SecurityAudit.CloseLogin(s.tenantID, s.sid, time.Now().UTC().Round(time.Millisecond)); err != nil { if err := store.SecurityAudit.CloseLogin(s.tenantID, s.sid, time.Now().UTC().Round(time.Millisecond)); err != nil {
logs.Warn.Println("s.cleanUp: failed to close login audit", s.uid, err, s.sid) logs.Warn.Println("s.cleanUp: failed to close login audit", s.uid, err, s.sid)
} }
if err := store.MarkUserOffline(s.tenantID, s.uid, s.sid, currentNodeID()); err != nil { remaining, err := store.MarkUserOffline(s.tenantID, s.uid, s.sid, currentNodeID())
if err != nil {
logs.Warn.Println("s.cleanUp: failed to mark user offline", s.uid, err, s.sid) logs.Warn.Println("s.cleanUp: failed to mark user offline", s.uid, err, s.sid)
} }
// 最后一个会话已断开(如主动登出、断网):立即通知关注者该用户离线,
// 不再等待 me topic 闲置超时后的 presUsersOfInterest("off")。
if err == nil && remaining == 0 && globals.hub != nil {
if uoi, uerr := store.Users.GetSubs(s.tenantID, s.uid); uerr == nil {
presUsersOfInterestOffline(s.tenantID, s.uid, uoi, "off")
} else {
logs.Warn.Println("s.cleanUp: failed to notify users of interest", s.uid, uerr, s.sid)
}
}
} }
s.purgeChannels() s.purgeChannels()
s.inflightReqs.Wait() s.inflightReqs.Wait()
@@ -1140,9 +1151,29 @@ func (s *Session) login(msg *ClientComMessage) {
return return
} }
// Login failure lockout: reject already-locked accounts up front without
// comparing the password. A nil guard means the feature is not applicable.
lockGuard := newLoginLockoutGuard(s, msg.Login.Scheme, msg.Login.Secret)
if lockGuard != nil {
if retryAfter, locked := lockGuard.preCheck(); locked {
logs.Info.Printf("s.login: account locked, rejecting tenant:%d account_hash:%s sid:%s", s.tenantID, lockGuard.accountHash, s.sid)
s.queueOut(ErrAuthLocked(msg.Id, msg.Timestamp, loginLockedMessage(retryAfter),
map[string]any{"what": "auth", "retry_after_seconds": int(retryAfter.Seconds()) + 1}))
return
}
}
rec, challenge, err := handler.Authenticate( rec, challenge, err := handler.Authenticate(
auth.AuthContext{TenantID: s.tenantID, RemoteAddr: s.remoteAddr}, msg.Login.Secret) auth.AuthContext{TenantID: s.tenantID, RemoteAddr: s.remoteAddr}, msg.Login.Secret)
if err != nil { if err != nil {
if lockGuard != nil && errors.Is(err, types.ErrFailed) {
if retryAfter, lockedNow := lockGuard.recordFailure(); lockedNow {
recordSessionLoginFailure(s, msg.Login.Scheme, msg.Login.Secret, rec, msg.Id, err, msg.Timestamp, loginAuditRiskActionLocked)
s.queueOut(ErrAuthLocked(msg.Id, msg.Timestamp, loginLockedMessage(retryAfter),
map[string]any{"what": "auth", "retry_after_seconds": int(retryAfter.Seconds()) + 1}))
return
}
}
recordSessionLoginFailure(s, msg.Login.Scheme, msg.Login.Secret, rec, msg.Id, err, msg.Timestamp) recordSessionLoginFailure(s, msg.Login.Scheme, msg.Login.Secret, rec, msg.Id, err, msg.Timestamp)
resp := decodeStoreError(err, msg.Id, msg.Timestamp, nil) resp := decodeStoreError(err, msg.Id, msg.Timestamp, nil)
if resp.Ctrl.Code >= 500 { if resp.Ctrl.Code >= 500 {
@@ -1160,6 +1191,11 @@ func (s *Session) login(msg *ClientComMessage) {
return return
} }
// Password accepted: clear the failure counter and any stale lock state.
if lockGuard != nil {
lockGuard.clear()
}
// If authenticator did not check user state, it returns state "undef". If so, check user state here. // If authenticator did not check user state, it returns state "undef". If so, check user state here.
if rec.State == types.StateUndefined { if rec.State == types.StateUndefined {
rec.State, err = userGetState(s.tenantID, rec.Uid) rec.State, err = userGetState(s.tenantID, rec.Uid)
@@ -1650,6 +1686,16 @@ func (s *Session) note(msg *ClientComMessage) {
// the user about the call. // the user about the call.
// //
// Hub will forward to topic, if appropriate. // Hub will forward to topic, if appropriate.
if msg.Note.What == "call" && globals.cluster != nil &&
globals.cluster.isRemoteTopic(types.TopicKey{TenantID: msg.TenantID, Topic: msg.RcptTo}) {
if err := globals.cluster.routeToTopicMaster(ProxyReqCall, msg, msg.RcptTo, s); err != nil {
logs.Warn.Printf("s.note: failed routing detached call event to topic master topic=%s sid=%s err=%v", msg.RcptTo, s.sid, err)
}
return
}
if msg.Note.What == "call" {
publishRealtimeCallEvent(msg)
}
select { select {
case globals.hub.routeCli <- msg: case globals.hub.routeCli <- msg:
default: default:
+33
View File
@@ -564,6 +564,39 @@ func TestDispatchAlreadySubscribed(t *testing.T) {
verifyResponseCodes(&r, []int{http.StatusNotModified}, t) verifyResponseCodes(&r, []int{http.StatusNotModified}, t)
} }
func TestSubscribeP2PUserAliasReachesTopic(t *testing.T) {
oldHub := globals.hub
globals.hub = &Hub{join: make(chan *ClientComMessage, 1)}
t.Cleanup(func() { globals.hub = oldHub })
self := types.Uid(1)
peer := types.Uid(2)
canonical := self.P2PName(peer)
s := test_makeSession(self)
s.subs = map[string]*Subscription{canonical: {}}
msg := &ClientComMessage{
Sub: &MsgClientSub{Id: "alias-sub", Topic: peer.UserId()},
Id: "alias-sub",
Original: peer.UserId(),
AsUser: self.UserId(),
TenantID: s.tenantID,
Timestamp: types.TimeNow(),
sess: s,
}
s.subscribe(msg)
select {
case routed := <-globals.hub.join:
if routed != msg || routed.RcptTo != canonical {
t.Fatalf("unexpected alias subscription route: %#v", routed)
}
s.inflightReqs.Done()
default:
t.Fatal("p2p user alias was incorrectly handled as an already-subscribed topic")
}
}
func TestDispatchSubscribeJoinChannelFull(t *testing.T) { func TestDispatchSubscribeJoinChannelFull(t *testing.T) {
uid := types.Uid(1) uid := types.Uid(1)
s := test_makeSession(uid) s := test_makeSession(uid)
+36
View File
@@ -60,13 +60,24 @@ var (
tenantByIDCache = newConfigCache[*types.Tenant]() tenantByIDCache = newConfigCache[*types.Tenant]()
registrationSMSCache = newConfigCache[bool]() registrationSMSCache = newConfigCache[bool]()
thirdPartyConfigCache = newConfigCache[map[string]string]() thirdPartyConfigCache = newConfigCache[map[string]string]()
dictConfigCache = newConfigCache[[]types.DictionaryItem]()
appReleaseConfigCache = newConfigCache[*types.AppRelease]() appReleaseConfigCache = newConfigCache[*types.AppRelease]()
moderationProviderCache = newConfigCache[*types.ModerationProviderConfig]() moderationProviderCache = newConfigCache[*types.ModerationProviderConfig]()
moderationPolicyCache = newConfigCache[*types.ModerationPolicy]() moderationPolicyCache = newConfigCache[*types.ModerationPolicy]()
moderationWordsCache = newConfigCache[*types.SensitiveWordSet]() moderationWordsCache = newConfigCache[*types.SensitiveWordSet]()
securityPolicyCache = newConfigCache[*types.SecurityPolicy]()
realNamePolicyCache = newConfigCache[*types.RealNamePolicy]()
thirdPartyConfigAllKeys = "*" thirdPartyConfigAllKeys = "*"
) )
func InvalidateSecurityPolicyCache(tenantCode string) {
if strings.TrimSpace(tenantCode) == "" {
securityPolicyCache.clear()
return
}
securityPolicyCache.delete(strings.TrimSpace(tenantCode))
}
func InvalidateModerationConfigCache(tenantCode string, provider, policy, words bool) { func InvalidateModerationConfigCache(tenantCode string, provider, policy, words bool) {
if provider { if provider {
moderationProviderCache.clear() moderationProviderCache.clear()
@@ -136,8 +147,10 @@ func InvalidateTenantConfigCache(code string, id types.TenantID) {
} }
if id.IsValid() { if id.IsValid() {
tenantByIDCache.delete(formatTenantIDForCache(id)) tenantByIDCache.delete(formatTenantIDForCache(id))
realNamePolicyCache.delete(formatTenantIDForCache(id))
} else if strings.TrimSpace(code) == "" { } else if strings.TrimSpace(code) == "" {
tenantByIDCache.clear() tenantByIDCache.clear()
realNamePolicyCache.clear()
} }
} }
@@ -162,6 +175,29 @@ func InvalidateThirdPartyConfigCache(group string) {
thirdPartyConfigCache.mu.Unlock() thirdPartyConfigCache.mu.Unlock()
} }
func dictConfigCacheKey(dictType string) string {
return strings.TrimSpace(dictType)
}
func cloneDictionaryItems(src []types.DictionaryItem) []types.DictionaryItem {
if src == nil {
return nil
}
dst := make([]types.DictionaryItem, len(src))
copy(dst, src)
return dst
}
// InvalidateDictionaryConfigCache clears cached dictionary config. An empty
// type clears all dictionary entries.
func InvalidateDictionaryConfigCache(dictType string) {
if strings.TrimSpace(dictType) == "" {
dictConfigCache.clear()
return
}
dictConfigCache.delete(strings.TrimSpace(dictType))
}
func formatTenantIDForCache(id types.TenantID) string { func formatTenantIDForCache(id types.TenantID) string {
return strconv.FormatInt(int64(id), 10) return strconv.FormatInt(int64(id), 10)
} }
+8 -8
View File
@@ -109,7 +109,7 @@ type groupMessageRuntime interface {
InvalidateHotMessages(tenantID types.TenantID, topic string, ranges []types.Range) error InvalidateHotMessages(tenantID types.TenantID, topic string, ranges []types.Range) error
MarkUserOnline(tenantID types.TenantID, uid types.Uid, sid, nodeID, device, userAgent string) error MarkUserOnline(tenantID types.TenantID, uid types.Uid, sid, nodeID, device, userAgent string) error
RefreshUserPresence(tenantID types.TenantID, uid types.Uid, sid, nodeID string) error RefreshUserPresence(tenantID types.TenantID, uid types.Uid, sid, nodeID string) error
MarkUserOffline(tenantID types.TenantID, uid types.Uid, sid, nodeID string) error MarkUserOffline(tenantID types.TenantID, uid types.Uid, sid, nodeID string) (int, error)
GetUserPresence(tenantID types.TenantID, uid types.Uid) (*UserPresence, error) GetUserPresence(tenantID types.TenantID, uid types.Uid) (*UserPresence, error)
MarkTopicMemberOnline(tenantID types.TenantID, topic string, uid types.Uid, nodeID string) error MarkTopicMemberOnline(tenantID types.TenantID, topic string, uid types.Uid, nodeID string) error
MarkTopicMemberOffline(tenantID types.TenantID, topic string, uid types.Uid, nodeID string) error MarkTopicMemberOffline(tenantID types.TenantID, topic string, uid types.Uid, nodeID string) error
@@ -300,9 +300,9 @@ func RefreshUserPresence(tenantID types.TenantID, uid types.Uid, sid, nodeID str
return groupMsgRuntime.RefreshUserPresence(tenantID, uid, sid, nodeID) return groupMsgRuntime.RefreshUserPresence(tenantID, uid, sid, nodeID)
} }
func MarkUserOffline(tenantID types.TenantID, uid types.Uid, sid, nodeID string) error { func MarkUserOffline(tenantID types.TenantID, uid types.Uid, sid, nodeID string) (int, error) {
if groupMsgRuntime == nil || !tenantID.IsValid() || uid.IsZero() || sid == "" { if groupMsgRuntime == nil || !tenantID.IsValid() || uid.IsZero() || sid == "" {
return nil return 0, nil
} }
return groupMsgRuntime.MarkUserOffline(tenantID, uid, sid, nodeID) return groupMsgRuntime.MarkUserOffline(tenantID, uid, sid, nodeID)
} }
@@ -939,7 +939,7 @@ func (r *redisGroupMessageRuntime) RefreshUserPresence(tenantID types.TenantID,
return err return err
} }
func (r *redisGroupMessageRuntime) MarkUserOffline(tenantID types.TenantID, uid types.Uid, sid, nodeID string) error { func (r *redisGroupMessageRuntime) MarkUserOffline(tenantID types.TenantID, uid types.Uid, sid, nodeID string) (int, error) {
if nodeID == "" { if nodeID == "" {
nodeID = "local" nodeID = "local"
} }
@@ -956,11 +956,11 @@ func (r *redisGroupMessageRuntime) MarkUserOffline(tenantID types.TenantID, uid
pipe.SRem(ctx, sessionsKey, member) pipe.SRem(ctx, sessionsKey, member)
pipe.SRem(ctx, nodeKey, uid.UserId()) pipe.SRem(ctx, nodeKey, uid.UserId())
if _, err := pipe.Exec(ctx); err != nil { if _, err := pipe.Exec(ctx); err != nil {
return err return 0, err
} }
count, err := r.client.SCard(ctx, sessionsKey).Result() count, err := r.client.SCard(ctx, sessionsKey).Result()
if err != nil { if err != nil {
return err return 0, err
} }
if count > 0 { if count > 0 {
pipe = r.client.Pipeline() pipe = r.client.Pipeline()
@@ -968,14 +968,14 @@ func (r *redisGroupMessageRuntime) MarkUserOffline(tenantID types.TenantID, uid
pipe.Expire(ctx, userKey, ttl) pipe.Expire(ctx, userKey, ttl)
pipe.Expire(ctx, sessionsKey, ttl) pipe.Expire(ctx, sessionsKey, ttl)
_, err = pipe.Exec(ctx) _, err = pipe.Exec(ctx)
return err return int(count), err
} }
pipe = r.client.Pipeline() pipe = r.client.Pipeline()
pipe.HSet(ctx, userKey, "online", "0", "session_count", 0, "node_id", nodeID, "last_seen_at", now.Format(time.RFC3339Nano)) pipe.HSet(ctx, userKey, "online", "0", "session_count", 0, "node_id", nodeID, "last_seen_at", now.Format(time.RFC3339Nano))
pipe.Expire(ctx, userKey, ttl) pipe.Expire(ctx, userKey, ttl)
pipe.Del(ctx, sessionsKey) pipe.Del(ctx, sessionsKey)
_, err = pipe.Exec(ctx) _, err = pipe.Exec(ctx)
return err return 0, err
} }
func (r *redisGroupMessageRuntime) GetUserPresence(tenantID types.TenantID, uid types.Uid) (*UserPresence, error) { func (r *redisGroupMessageRuntime) GetUserPresence(tenantID types.TenantID, uid types.Uid) (*UserPresence, error) {
+2 -2
View File
@@ -114,8 +114,8 @@ func (r *testGroupMessageRuntime) RefreshUserPresence(types.TenantID, types.Uid,
return nil return nil
} }
func (r *testGroupMessageRuntime) MarkUserOffline(types.TenantID, types.Uid, string, string) error { func (r *testGroupMessageRuntime) MarkUserOffline(types.TenantID, types.Uid, string, string) (int, error) {
return nil return 0, nil
} }
func (r *testGroupMessageRuntime) GetUserPresence(types.TenantID, types.Uid) (*UserPresence, error) { func (r *testGroupMessageRuntime) GetUserPresence(types.TenantID, types.Uid) (*UserPresence, error) {
@@ -0,0 +1,191 @@
//go:build mysql
package store_test
import (
"database/sql"
"encoding/base64"
"encoding/json"
"fmt"
"os"
"strings"
"testing"
"time"
_ "github.com/go-sql-driver/mysql"
_ "github.com/tinode/chat/server/db/mysql"
"github.com/tinode/chat/server/store"
"github.com/tinode/chat/server/store/types"
)
type localCryptoProvider struct {
key string
}
func (p localCryptoProvider) GetTenantByCode(string) (*types.Tenant, error) {
return nil, types.ErrUnsupported
}
func (p localCryptoProvider) GetTenantByID(types.TenantID) (*types.Tenant, error) {
return nil, types.ErrUnsupported
}
func (p localCryptoProvider) ListActiveTenants(int) ([]types.Tenant, error) {
return nil, types.ErrUnsupported
}
func (p localCryptoProvider) RegistrationSMSEnabled() (bool, error) {
return false, types.ErrUnsupported
}
func (p localCryptoProvider) ThirdPartyConfig(string, ...string) (map[string]string, error) {
return nil, types.ErrUnsupported
}
func (p localCryptoProvider) LatestAppRelease(string, string) (*types.AppRelease, error) {
return nil, types.ErrUnsupported
}
func (p localCryptoProvider) ModerationProvider() (*types.ModerationProviderConfig, error) {
return nil, types.ErrUnsupported
}
func (p localCryptoProvider) ModerationPolicy(string) (*types.ModerationPolicy, error) {
return nil, types.ErrUnsupported
}
func (p localCryptoProvider) SensitiveWords(string) (*types.SensitiveWordSet, error) {
return nil, types.ErrUnsupported
}
func (p localCryptoProvider) SecurityPolicy(string) (*types.SecurityPolicy, error) {
return nil, types.ErrUnsupported
}
func (p localCryptoProvider) RealNamePolicy(types.TenantID) (*types.RealNamePolicy, error) {
return nil, types.ErrUnsupported
}
func (p localCryptoProvider) MessageKey(tenantCode string, version int) (*types.MessageKey, error) {
if tenantCode != "default" || (version != 0 && version != 7) {
return nil, types.ErrNotFound
}
return &types.MessageKey{TenantID: 1, TenantCode: tenantCode, Enabled: true, Version: 7, Algorithm: "AES-256-GCM", Status: "active", Key: p.key}, nil
}
func (p localCryptoProvider) MessageKeyAck(string, int) error { return nil }
func TestLocalMySQLMessageCryptoFlow(t *testing.T) {
if os.Getenv("IM_LOCAL_CRYPTO_TEST") != "1" {
t.Skip("set IM_LOCAL_CRYPTO_TEST=1 to run against local MySQL")
}
config := json.RawMessage(`{
"uid_key":"dkXxWLwqUNRfYmdXRzOkZw==",
"max_results":1024,
"use_adapter":"mysql",
"adapters":{"mysql":{
"dsn":"mysql://root:root@tcp(127.0.0.1:3306)/tinode?parseTime=true&collation=utf8mb4_0900_ai_ci",
"tenant_registry_db":"hotgo",
"tenant_db_routing":true,
"sql_timeout":10
}}
}`)
if err := store.Store.Open(991, config); err != nil {
t.Fatal(err)
}
defer store.Store.Close()
defer store.SetAdminInternalConfigProvider(nil)
store.SetAdminInternalConfigProvider(localCryptoProvider{key: base64.StdEncoding.EncodeToString([]byte("0123456789abcdef0123456789abcdef"))})
store.InvalidateMessageEncryptionKeys(1)
db, err := sql.Open("mysql", "root:root@tcp(127.0.0.1:3306)/tinode?parseTime=true&collation=utf8mb4_0900_ai_ci")
if err != nil {
t.Fatal(err)
}
defer db.Close()
var topic string
var senderID, originalTopicSeq int64
err = db.QueryRow(`SELECT s.topic,s.userid,t.seqid FROM im_subscriptions s JOIN im_topics t ON t.name=s.topic
WHERE s.topic LIKE 'p2p%' AND s.deletedat IS NULL ORDER BY t.id DESC LIMIT 1`).Scan(&topic, &senderID, &originalTopicSeq)
if err != nil {
t.Fatal(err)
}
sender := store.EncodeUid(senderID)
marker := fmt.Sprintf("LOCAL-CRYPTO-%d", time.Now().UnixNano())
msg := &types.Message{
TenantID: 1,
SeqId: originalTopicSeq + 1,
Topic: topic,
From: sender.String(),
Head: types.KVMap{"x-im-client-mid": marker, "x-im-type": "text"},
Content: map[string]any{"txt": marker},
}
msg.InitTimes()
msg.SetUid(store.Store.GetUid())
fallbackSeq := int64(0)
defer func() {
_, _ = db.Exec("DELETE FROM im_message_revisions WHERE topic=? AND seqid=?", topic, msg.SeqId)
_, _ = db.Exec("DELETE FROM im_messages WHERE topic=? AND seqid=?", topic, msg.SeqId)
if fallbackSeq > 0 {
_, _ = db.Exec("DELETE FROM im_messages WHERE topic=? AND seqid=?", topic, fallbackSeq)
}
_, _ = db.Exec("UPDATE im_topics SET seqid=? WHERE name=? AND seqid IN (?,?)", originalTopicSeq, topic, msg.SeqId, fallbackSeq)
}()
if _, err = store.PersistPreparedMessage(1, msg, nil, false); err != nil {
t.Fatal(err)
}
var rawContent string
var nonce []byte
var keyVersion int
if err = db.QueryRow("SELECT JSON_UNQUOTE(content),content_nonce,key_version FROM im_messages WHERE topic=? AND seqid=?", topic, msg.SeqId).Scan(&rawContent, &nonce, &keyVersion); err != nil {
t.Fatal(err)
}
if keyVersion != 7 || len(nonce) != 12 || strings.Contains(rawContent, marker) {
t.Fatalf("message not encrypted at rest: key_version=%d nonce=%d content=%q", keyVersion, len(nonce), rawContent)
}
if _, err = base64.StdEncoding.DecodeString(rawContent); err != nil {
t.Fatalf("stored ciphertext is not base64: %v", err)
}
loaded, err := store.Messages.GetOne(1, topic, msg.SeqId)
if err != nil || fmt.Sprint(loaded.Content.(map[string]any)["txt"]) != marker {
t.Fatalf("GetOne plaintext mismatch: content=%#v err=%v", loaded, err)
}
editedMarker := marker + "-EDITED"
edited, err := store.Messages.Edit(1, topic, msg.SeqId, sender, loaded.Version, loaded.Head, map[string]any{"txt": editedMarker}, "local integration test", types.TimeNow())
if err != nil || fmt.Sprint(edited.Content.(map[string]any)["txt"]) != editedMarker {
t.Fatalf("Edit plaintext mismatch: content=%#v err=%v", edited, err)
}
var revisionVersion, currentVersion int
if err = db.QueryRow("SELECT key_version FROM im_message_revisions WHERE topic=? AND seqid=? ORDER BY version DESC LIMIT 1", topic, msg.SeqId).Scan(&revisionVersion); err != nil {
t.Fatal(err)
}
if err = db.QueryRow("SELECT key_version FROM im_messages WHERE topic=? AND seqid=?", topic, msg.SeqId).Scan(&currentVersion); err != nil {
t.Fatal(err)
}
if revisionVersion != 7 || currentVersion != 7 {
t.Fatalf("unexpected key versions: revision=%d current=%d", revisionVersion, currentVersion)
}
history, err := store.Messages.GetAll(1, topic, sender, &types.QueryOpt{Since: msg.SeqId, Before: msg.SeqId + 1, Limit: 10})
if err != nil || len(history) != 1 || fmt.Sprint(history[0].Content.(map[string]any)["txt"]) != editedMarker {
t.Fatalf("history plaintext mismatch: messages=%#v err=%v", history, err)
}
found, _, err := store.Messages.Search(1, topic, sender, editedMarker, "text", "", 10)
if err != nil || len(found) != 1 || found[0].SeqId != msg.SeqId {
t.Fatalf("search plaintext mismatch: messages=%#v err=%v", found, err)
}
store.SetAdminInternalConfigProvider(localCryptoProvider{})
store.InvalidateMessageEncryptionKeys(1)
fallback := &types.Message{
TenantID: 1,
SeqId: msg.SeqId + 1,
Topic: topic,
From: sender.String(),
Head: types.KVMap{"x-im-client-mid": marker + "-FALLBACK", "x-im-type": "text"},
Content: map[string]any{"txt": marker + "-PLAINTEXT"},
}
fallback.InitTimes()
fallback.SetUid(store.Store.GetUid())
fallbackSeq = fallback.SeqId
if _, err = store.PersistPreparedMessage(1, fallback, nil, false); err != nil {
t.Fatal(err)
}
if err = db.QueryRow("SELECT JSON_UNQUOTE(content),key_version FROM im_messages WHERE topic=? AND seqid=?", topic, fallback.SeqId).Scan(&rawContent, &keyVersion); err != nil {
t.Fatal(err)
}
if keyVersion != 0 || !strings.Contains(rawContent, marker+"-PLAINTEXT") {
t.Fatalf("missing-key fallback was not plaintext: key_version=%d content=%q", keyVersion, rawContent)
}
}
+260
View File
@@ -0,0 +1,260 @@
package store
import (
"crypto/aes"
"crypto/cipher"
"crypto/rand"
"encoding/base64"
"encoding/binary"
"encoding/json"
"errors"
"fmt"
"io"
"strconv"
"strings"
"sync"
"time"
"github.com/tinode/chat/server/logs"
"github.com/tinode/chat/server/store/types"
)
type cachedMessageKey struct {
enabled bool
version int
status string
key []byte
expires time.Time
}
const messageKeyCacheTTL = 5 * time.Minute
var messageKeyCache = struct {
sync.RWMutex
items map[string]cachedMessageKey
}{items: make(map[string]cachedMessageKey)}
func messageKeyCacheID(tenantID types.TenantID, version int) string {
return strconv.FormatInt(int64(tenantID), 10) + ":" + strconv.Itoa(version)
}
// WarmMessageEncryptionKeys preloads active keys. Missing keys leave the tenant in plaintext mode.
func WarmMessageEncryptionKeys() {
if adminInternalConfigProvider == nil || Tenants == nil {
return
}
tenants, err := Tenants.ListActive(10000)
if err != nil {
logs.Warn.Printf("message crypto: tenant preload failed: %v", err)
return
}
for i := range tenants {
if _, err = loadMessageKey(&tenants[i], 0); err != nil {
logs.Info.Printf("message crypto: plaintext mode tenant=%s reason=%v", tenants[i].Code, err)
}
}
}
func loadMessageKey(tenant *types.Tenant, version int) (cachedMessageKey, error) {
var zero cachedMessageKey
if tenant == nil || adminInternalConfigProvider == nil {
return zero, types.ErrNotFound
}
result, err := adminInternalConfigProvider.MessageKey(tenant.Code, version)
if err != nil || result == nil {
if err == nil {
err = types.ErrNotFound
}
return zero, err
}
decoded, err := base64.StdEncoding.DecodeString(result.Key)
if err != nil || len(decoded) != 32 {
return zero, errors.New("invalid tenant message key")
}
entry := cachedMessageKey{enabled: result.Enabled, version: result.Version, status: result.Status, key: decoded, expires: time.Now().Add(messageKeyCacheTTL)}
messageKeyCache.Lock()
messageKeyCache.items[messageKeyCacheID(tenant.ID, result.Version)] = entry
if version == 0 {
messageKeyCache.items[messageKeyCacheID(tenant.ID, 0)] = entry
}
messageKeyCache.Unlock()
return entry, nil
}
func getMessageKey(tenantID types.TenantID, version int) (cachedMessageKey, error) {
cacheID := messageKeyCacheID(tenantID, version)
messageKeyCache.RLock()
entry, ok := messageKeyCache.items[cacheID]
messageKeyCache.RUnlock()
if ok && time.Now().Before(entry.expires) {
return entry, nil
}
if ok {
messageKeyCache.Lock()
delete(messageKeyCache.items, cacheID)
messageKeyCache.Unlock()
}
tenant, err := Tenants.GetByID(tenantID)
if err != nil {
return cachedMessageKey{}, err
}
entry, err = loadMessageKey(tenant, version)
if err != nil && version == 0 {
entry = cachedMessageKey{expires: time.Now().Add(messageKeyCacheTTL)}
messageKeyCache.Lock()
messageKeyCache.items[cacheID] = entry
messageKeyCache.Unlock()
return entry, nil
}
return entry, err
}
// PrepareMessageEncryptionKey verifies a pending key in memory and acknowledges
// it to Admin Server. Admin activates the key only after this succeeds.
func PrepareMessageEncryptionKey(tenantID types.TenantID, tenantCode string, version int) error {
if version <= 0 || adminInternalConfigProvider == nil {
return types.ErrMalformed
}
var tenant *types.Tenant
var err error
if tenantID.IsValid() {
tenant, err = Tenants.GetByID(tenantID)
} else {
tenant, err = Tenants.GetByCode(strings.TrimSpace(tenantCode))
}
if err != nil || tenant == nil {
return err
}
entry, err := loadMessageKey(tenant, version)
if err != nil {
return err
}
if strings.EqualFold(entry.status, "pending") {
if err = adminInternalConfigProvider.MessageKeyAck(tenant.Code, version); err != nil {
return err
}
}
messageKeyCache.Lock()
messageKeyCache.items[messageKeyCacheID(tenant.ID, 0)] = entry
messageKeyCache.Unlock()
return nil
}
// InvalidateMessageEncryptionKeys forces the next access to reload tenant keys.
func InvalidateMessageEncryptionKeys(tenantID types.TenantID) {
prefix := strconv.FormatInt(int64(tenantID), 10) + ":"
messageKeyCache.Lock()
for key := range messageKeyCache.items {
if len(key) >= len(prefix) && key[:len(prefix)] == prefix {
delete(messageKeyCache.items, key)
}
}
messageKeyCache.Unlock()
}
func messageAAD(msg *types.Message, keyVersion int) []byte {
topic := []byte(msg.Topic)
aad := make([]byte, 8+8+4+len(topic))
binary.BigEndian.PutUint64(aad[0:8], uint64(msg.TenantID))
binary.BigEndian.PutUint64(aad[8:16], uint64(msg.SeqId))
binary.BigEndian.PutUint32(aad[16:20], uint32(keyVersion))
copy(aad[20:], topic)
return aad
}
func encryptMessageContent(msg *types.Message) (*types.Message, error) {
entry, err := getMessageKey(msg.TenantID, 0)
if err != nil || !entry.enabled || entry.version <= 0 {
if err != nil {
logs.Warn.Printf("message crypto: active key unavailable, storing plaintext tenant=%d reason=%v", msg.TenantID, err)
}
copy := *msg
copy.KeyVersion, copy.ContentNonce = 0, nil
return &copy, nil
}
plain, err := json.Marshal(msg.Content)
if err != nil {
return nil, err
}
block, err := aes.NewCipher(entry.key)
if err != nil {
return nil, err
}
gcm, err := cipher.NewGCM(block)
if err != nil {
return nil, err
}
nonce := make([]byte, gcm.NonceSize())
if _, err = io.ReadFull(rand.Reader, nonce); err != nil {
return nil, err
}
copy := *msg
copy.KeyVersion = entry.version
copy.ContentNonce = nonce
copy.Content = base64.StdEncoding.EncodeToString(gcm.Seal(nil, nonce, plain, messageAAD(&copy, entry.version)))
return &copy, nil
}
func decryptMessageContent(msg *types.Message) error {
if msg == nil || msg.KeyVersion <= 0 {
return nil
}
entry, err := getMessageKey(msg.TenantID, msg.KeyVersion)
if err != nil {
return fmt.Errorf("message key version %d unavailable: %w", msg.KeyVersion, err)
}
ciphertextText, ok := msg.Content.(string)
if !ok {
return errors.New("encrypted message content is not a string")
}
ciphertext, err := base64.StdEncoding.DecodeString(ciphertextText)
if err != nil {
return err
}
block, err := aes.NewCipher(entry.key)
if err != nil {
return err
}
gcm, err := cipher.NewGCM(block)
if err != nil {
return err
}
if len(msg.ContentNonce) != gcm.NonceSize() {
return fmt.Errorf("invalid message nonce length: got %d want %d", len(msg.ContentNonce), gcm.NonceSize())
}
plain, err := gcm.Open(nil, msg.ContentNonce, ciphertext, messageAAD(msg, msg.KeyVersion))
if err != nil {
return err
}
return json.Unmarshal(plain, &msg.Content)
}
func decryptMessages(messages []types.Message) error {
for i := range messages {
if err := decryptMessageContent(&messages[i]); err != nil {
return err
}
}
return nil
}
func messageContentMatchesSearch(content any, query string) bool {
query = strings.ToLower(strings.TrimSpace(query))
if query == "" {
return true
}
raw, err := json.Marshal(content)
if err != nil {
return false
}
text := strings.ToLower(string(raw))
if strings.Contains(text, query) {
return true
}
for _, term := range strings.Fields(query) {
if !strings.Contains(text, term) {
return false
}
}
return true
}
+93
View File
@@ -0,0 +1,93 @@
package store
import (
"bytes"
"testing"
"time"
"github.com/tinode/chat/server/store/types"
)
func TestMessageCryptoRoundTrip(t *testing.T) {
tenantID := types.TenantID(9001)
key := bytes.Repeat([]byte{7}, 32)
messageKeyCache.Lock()
expires := time.Now().Add(time.Minute)
messageKeyCache.items[messageKeyCacheID(tenantID, 0)] = cachedMessageKey{enabled: true, version: 3, key: key, expires: expires}
messageKeyCache.items[messageKeyCacheID(tenantID, 3)] = cachedMessageKey{enabled: true, version: 3, key: key, expires: expires}
messageKeyCache.Unlock()
defer InvalidateMessageEncryptionKeys(tenantID)
original := &types.Message{TenantID: tenantID, Topic: "p2p-test", SeqId: 42, Content: map[string]any{"txt": "hello"}}
stored, err := encryptMessageContent(original)
if err != nil {
t.Fatal(err)
}
if stored.KeyVersion != 3 || len(stored.ContentNonce) != 12 || stored.Content == original.Content {
t.Fatalf("message was not encrypted: %#v", stored)
}
if err = decryptMessageContent(stored); err != nil {
t.Fatal(err)
}
content, ok := stored.Content.(map[string]any)
if !ok || content["txt"] != "hello" {
t.Fatalf("unexpected plaintext: %#v", stored.Content)
}
}
func TestMessageCryptoPlaintextWhenDisabled(t *testing.T) {
tenantID := types.TenantID(9002)
messageKeyCache.Lock()
messageKeyCache.items[messageKeyCacheID(tenantID, 0)] = cachedMessageKey{expires: time.Now().Add(time.Minute)}
messageKeyCache.Unlock()
defer InvalidateMessageEncryptionKeys(tenantID)
original := &types.Message{TenantID: tenantID, Topic: "grp-test", SeqId: 7, Content: "plain"}
stored, err := encryptMessageContent(original)
if err != nil {
t.Fatal(err)
}
if stored.KeyVersion != 0 || stored.Content != "plain" || len(stored.ContentNonce) != 0 {
t.Fatalf("unexpected encrypted message: %#v", stored)
}
}
func TestMessageCryptoRejectsTampering(t *testing.T) {
tenantID := types.TenantID(9003)
key := bytes.Repeat([]byte{9}, 32)
expires := time.Now().Add(time.Minute)
messageKeyCache.Lock()
messageKeyCache.items[messageKeyCacheID(tenantID, 0)] = cachedMessageKey{enabled: true, version: 4, key: key, expires: expires}
messageKeyCache.items[messageKeyCacheID(tenantID, 4)] = cachedMessageKey{enabled: true, version: 4, key: key, expires: expires}
messageKeyCache.Unlock()
defer InvalidateMessageEncryptionKeys(tenantID)
original := &types.Message{TenantID: tenantID, Topic: "p2p-secure", SeqId: 8, Content: map[string]any{"txt": "secret"}}
stored, err := encryptMessageContent(original)
if err != nil {
t.Fatal(err)
}
stored.Topic = "p2p-tampered"
if err = decryptMessageContent(stored); err == nil {
t.Fatal("expected authenticated decryption to reject changed topic")
}
stored, err = encryptMessageContent(original)
if err != nil {
t.Fatal(err)
}
stored.ContentNonce = stored.ContentNonce[:len(stored.ContentNonce)-1]
if err = decryptMessageContent(stored); err == nil {
t.Fatal("expected invalid nonce length to be rejected")
}
}
func TestMessageCryptoSearchMatchesDecryptedContent(t *testing.T) {
content := map[string]any{"txt": "项目 验收通过", "nested": map[string]any{"title": "日报"}}
if !messageContentMatchesSearch(content, "验收") {
t.Fatal("expected plaintext keyword to match")
}
if messageContentMatchesSearch(content, "不存在") {
t.Fatal("unexpected keyword match")
}
}
+299 -614
View File
File diff suppressed because it is too large Load Diff
-153
View File
@@ -1,153 +0,0 @@
package types
import "time"
const (
AttendanceActionCheckIn = "check_in"
AttendanceActionCheckOut = "check_out"
AttendanceStatusNotChecked = "not_checked"
AttendanceStatusCheckedIn = "checked_in"
AttendanceStatusNormal = "normal"
AttendanceStatusLate = "late"
AttendanceStatusEarlyLeave = "early_leave"
AttendanceStatusAbnormal = "abnormal"
AttendanceCheckinNormal = "normal"
AttendanceCheckinLate = "late"
AttendanceCheckinEarlyLeave = "early_leave"
AttendanceCheckinOutsideRange = "outside_range"
AttendanceAppealPending = "pending"
AttendanceAppealApproved = "approved"
AttendanceAppealRejected = "rejected"
AttendanceAppealCanceled = "canceled"
)
type AttendanceRule struct {
ID int64
RuleID string
Name string
Workday bool
LocationRequired bool
WifiRequired bool
AllowOutsideCheckin bool
AppealAllowed bool
AppealDeadline *time.Time
CreatedAt time.Time
UpdatedAt time.Time
}
type AttendanceShift struct {
ID int64
ShiftID string
RuleID string
Name string
WorkStart string
WorkEnd string
CheckinStart string
CheckoutEnd string
SortOrder int
CreatedAt time.Time
UpdatedAt time.Time
}
type AttendanceCheckin struct {
ID int64
CheckinID string
RecordID string
UserID Uid
Date time.Time
Action string
RuleID string
ShiftID string
ClientTime time.Time
ServerTime time.Time
Result string
Location KVMap
Wifi KVMap
Device KVMap
IdempotencyKey string
CreatedAt time.Time
}
type AttendanceRecord struct {
ID int64
RecordID string
UserID Uid
Date time.Time
RuleID string
ShiftID string
Status string
Checkins []AttendanceCheckin
Exceptions []string
AppealStatus string
Version int
CreatedAt time.Time
UpdatedAt time.Time
}
type AttendanceConfig struct {
Date time.Time
Timezone string
Rule AttendanceRule
Shifts []AttendanceShift
AllowedActions []string
TodayRecord *AttendanceRecord
CanViewManaged bool
}
type AttendanceRecordListOptions struct {
Scope string
From string
To string
Date string
Status string
DepartmentID string
UserID string
Cursor string
Limit int
}
type AttendanceRecordStats struct {
ExpectedCount int
CheckedCount int
ExceptionCount int
NotCheckedCount int
}
type AttendanceCheckinRequest struct {
Date time.Time
Action string
RuleID string
ShiftID string
ClientTime time.Time
Location KVMap
Wifi KVMap
Device KVMap
IdempotencyKey string
}
type AttendanceAppeal struct {
ID int64
AppealID string
RecordID string
UserID Uid
AppealType string
Reason string
TargetAction string
TargetTime *time.Time
AttachmentFileIDs []string
Status string
Version int
SubmittedAt time.Time
CreatedAt time.Time
UpdatedAt time.Time
}
type AttendanceAppealListOptions struct {
Scope string
Status string
Cursor string
Limit int
}
+17
View File
@@ -0,0 +1,17 @@
package types
import "time"
// LoginLockout is the per-account login failure counter and lock state
// stored in each tenant business database (table im_login_lockouts).
type LoginLockout struct {
TenantID TenantID `json:"tenant_id"`
ID int64 `json:"id"`
UserID int64 `json:"user_id"`
AccountHash string `json:"account_hash"`
FailedAttempts int `json:"failed_attempts"`
WindowStartAt *time.Time `json:"window_started_at"`
LockedUntil *time.Time `json:"locked_until"`
LastAttemptAt *time.Time `json:"last_attempt_at"`
UpdatedAt *time.Time `json:"updated_at"`
}
-36
View File
@@ -1,36 +0,0 @@
package types
import "time"
const MailboxSubmissionStatusReceived = "received"
type MailboxNotice struct {
Title string
Content string
UpdatedAt time.Time
}
type MailboxConfig struct {
Enabled bool
AllowAnonymous bool
RequiredFields []string
MaxLength int
AttachmentPolicy map[string]any
Notice MailboxNotice
UpdatedAt time.Time
}
type MailboxSubmission struct {
ID int64
SubmissionID string
SenderID Uid
Subject string
Content string
Anonymous bool
AttachmentFileIDs []string
NoticeVersion string
Status string
SubmittedAt time.Time
CreatedAt time.Time
UpdatedAt time.Time
}
-83
View File
@@ -1,83 +0,0 @@
package types
import "time"
const (
MeetingStateScheduled = "scheduled"
MeetingStateEnded = "ended"
MeetingStateCanceled = "canceled"
MeetingResponsePending = "pending"
MeetingResponseAccepted = "accepted"
MeetingResponseDeclined = "declined"
MeetingResponseTentative = "tentative"
)
type Meeting struct {
ID int64
MeetingID string
OrganizerID Uid
RecorderID Uid
Title string
StartAt time.Time
EndAt time.Time
Location string
State string
ReminderMinutes []int
IdempotencyKey string
Version int
CanceledAt *time.Time
CreatedAt time.Time
UpdatedAt time.Time
Participants []MeetingParticipant
}
type MeetingParticipant struct {
ID int64
MeetingID string
UserID Uid
Response string
ResponseAt *time.Time
CreatedAt time.Time
UpdatedAt time.Time
}
type MeetingPatch struct {
Title *string
RecorderID *Uid
StartAt *time.Time
EndAt *time.Time
Location *string
ReminderMinutes []int
Version int
}
const (
MeetingNoteStatusEmpty = "empty"
MeetingNoteStatusAvailable = "available"
MeetingNoteFileStatusActive = "active"
MeetingNoteFileStatusDeleted = "deleted"
)
type MeetingNoteFile struct {
ID int64
NoteFileID string
MeetingID string
FileID string
UploaderID Uid
FileName string
MIMEType string
FileSize int64
FileETag string
Status string
PreviewStatus string
PreviewMode string
PreviewFileID string
IdempotencyKey string
Version int
DeletedBy Uid
DeletedAt *time.Time
CreatedAt time.Time
UpdatedAt time.Time
}
+19
View File
@@ -0,0 +1,19 @@
package types
// MessageKey is a tenant message data-encryption key returned by Admin Server.
type MessageKey struct {
TenantID TenantID `json:"tenantId"`
TenantCode string `json:"tenantCode"`
Enabled bool `json:"enabled"`
Version int `json:"version"`
Algorithm string `json:"algorithm"`
Status string `json:"status"`
Key string `json:"key"`
}
// StoredMessageContent carries encrypted content metadata through the edit adapter boundary.
type StoredMessageContent struct {
Content any
Nonce []byte
KeyVersion int
}
-40
View File
@@ -1,40 +0,0 @@
package types
import "time"
type OrganizationDepartment struct {
ID int64 `db:"id"`
DepartmentID string `db:"department_id"`
Name string `db:"name"`
ParentID string `db:"parent_id"`
SortOrder int `db:"sort_order"`
ChildCount int `db:"child_count"`
MemberCount int `db:"member_count"`
Version int `db:"version"`
State string `db:"state"`
CreatedAt time.Time `db:"created_at"`
UpdatedAt time.Time `db:"updated_at"`
}
type OrganizationMember struct {
ID int64 `db:"id"`
DepartmentID string `db:"department_id"`
UserID Uid `db:"-"`
DisplayName string `db:"display_name"`
Position string `db:"position"`
AvatarFileID string `db:"avatar_file_id"`
Mobile string `db:"mobile"`
Email string `db:"email"`
Status string `db:"employment_state"`
Contactable bool `db:"contactable"`
SortOrder int `db:"sort_order"`
Version int `db:"version"`
CreatedAt time.Time `db:"created_at"`
UpdatedAt time.Time `db:"updated_at"`
}
type OrganizationSearchItem struct {
Type string
Department *OrganizationDepartment
Member *OrganizationMember
}
+11
View File
@@ -12,6 +12,9 @@ type UserProfile struct {
Region string Region string
Email string Email string
PublicAccount string PublicAccount string
FriendAddEnabled bool
FriendAddDisabledReason string
FriendAddPolicyVersion int
Version int Version int
CreatedAt time.Time CreatedAt time.Time
UpdatedAt time.Time UpdatedAt time.Time
@@ -68,6 +71,14 @@ type AppRelease struct {
UpdatedAt time.Time UpdatedAt time.Time
} }
// DictionaryItem is a single entry of an admin-side dictionary config, such as
// the "contact us" email/phone entries shown in mobile clients.
type DictionaryItem struct {
Label string
Value string
Sort int
}
type YouthModeSettings struct { type YouthModeSettings struct {
UserID Uid UserID Uid
Enabled bool Enabled bool

Some files were not shown because too many files have changed in this diff Show More