feat: 2026-07-12~13 全量更新 - AI对话链路改造+H5 v4/v5+坐席端v5+上下文感知诊断+知识库迭代3
## H5 员工端 v4 (2026-07-13 00:48 已部署)
- 人工按钮三态文案统一为"人工坐席"
- 按钮位置移至发送键和语音按钮上方(垂直堆叠)
- 点按钮直接调 store.shakeAgent(),删除 CallAgentModal 弹窗动画
- 截图快捷键提示改为"截图->粘贴:Alt+Shift+A-Ctrl+V ---> Ctrl+V"
- 移动端隐藏截图提示(CSS 媒体查询)
- AI转人工提示改为"已为您呼叫人工坐席,请稍等!"
- 坐席接入提示改为"坐席正在查看您的信息,请等待处理回复!"
- 删除"摇铃呼叫坐席"入口和文案
- 删除孤儿组件 MessageList.vue + shake 动画 CSS
## H5 员工端 v5 (2026-07-13 02:08 已部署)
- RightPanel v2.1:删除"软件安装"和"资源权限"标签页
- 移除标签栏,智能推荐(DynamicRecommend)直接展示
- 删除 SoftwareDownloads/ApprovalLinks 引用和相关 CSS
## AI 对话链路全栈改造 Phase 1-6 (已部署)
- Phase 1: Dify JSON输出 + 后端blocking解析 + 双WS推送 + 错误降级
- Phase 2: 关键词收窄(~25强意图词) + 两级分类Prompt + 删除前端checkApprovalIntent
- Phase 3: WS扩展(ai_thinking+dynamic_recommend) + ai_structured气泡 + RightPanel v2 + 选项回传
- Phase 4: VisionService接入 + 图片消息融合(5秒窗口) + 降级策略
- Phase 5: 坐席端ai_thinking指示器 + ai_structured/byod_card渲染 + handleNewMessage修复
- Phase 6: diagnosis_stage(6值) + response_time_ms计时 + 慢响应告警(>10s)
## 坐席端 v5 (2026-07-13 01:38 已部署)
- ai_structured/byod_card 只读渲染
- AI思考指示器 UI
- handleNewMessage 透传 msg_type/extra_data 修复
- 布局优化v2.0: QuickReplyBar L1+L2悬浮 + ReplyBox左右分区 + 右栏260/560px切换
- 键盘快捷键v2.3: 纯数字路由 + ESC分层撤销 + Shift+Space用event.code
## 上下文感知智能诊断闭环 (2026-07-12 已部署)
- 三层诊断(API→Script→AI) + 三段排队(VIP→info_locked→not locked)
- 答题插队 + 五场景关闭
- 迁移052(6表+6列) + queue_service + quiz_service + closing_service
- H5前端: QueueWaiting + RightPanel双Tab + InputBar三态 + ResolveConfirmCard
- 坐席前端: pending_close结单流程 + 信息锁定(Dify步骤完成+有效回答率≥70%)
## 知识库迭代3 (2026-07-12 已部署)
- 分诊交互(H5+坐席+Dify独立应用)
- 拓扑预览(ECharts只读)
- 代答排除(4种匹配器: keyword/regex/intent/category)
- 迁移051 + 44文件43测试通过
## 后端变更
- 6个Python文件改造(h5_ai_task.py/h5.py/ai_service.py/closing_service.py等)
- funny_phrase_service.py: shake/connected/keyword 默认文案更新
- session_service.py: 企微消息文案同步
- 新增: queue.py/quiz.py/triage.py/exclusion_rules.py 等API端点
- 新增: diagnostic.py/quiz.py/triage_session.py 等模型
- 新增: closing_service/queue_service/quiz_service/triage_service 等服务
## 文档更新
- CHANGELOG.md: 新增 [未发布] 区全部变更记录
- 项目管理主文档 v2.5: 新增v0.7.3版本 + 已完成看板 + 最近搞定
- 版本记录: 新增v0.7.3条目
- AI对话链路实施计划: Phase 1-6 全部标记✅已实施
- 新增架构图/时序图/类图(mermaid)
## 部署路径修正
- 服务器项目根路径: /opt/wecom-it-desk/
- 所有前端dist均为ro bind mount,只能在宿主机源路径操作
- 服务器nginx /h5/ 是静态文件服务(非proxy_pass)
- elFinder上传二进制不可靠(MD5不匹配),改用base64分块上传
This commit is contained in:
+487
-57
@@ -14,8 +14,13 @@
|
||||
# (请求返回后该 session 会被关闭)。
|
||||
# =============================================================================
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from datetime import datetime
|
||||
import os
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.api.byod import _byod_keyword_prefilter
|
||||
from app.database import _get_session_factory
|
||||
@@ -30,11 +35,184 @@ from app.services.routing_service import (
|
||||
record_routing_event,
|
||||
_keyword_fallback_category,
|
||||
)
|
||||
from app.services.vision_service import VisionService
|
||||
from app.services.ws_manager import manager as ws_manager
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Phase 4A: VisionService 接入 — 图片消息视觉理解
|
||||
# =============================================================================
|
||||
|
||||
# 图片文件本地存储根目录(与 upload.py 中 UPLOAD_DIR 一致)
|
||||
_UPLOAD_DIR = Path(os.getenv("UPLOAD_DIR", "./uploads"))
|
||||
|
||||
# 视觉理解置信度阈值:低于此值不注入描述(避免错误描述误导 AI)
|
||||
_VISION_CONFIDENCE_THRESHOLD = 0.6
|
||||
|
||||
|
||||
def _media_url_to_local_path(media_url: str) -> Path:
|
||||
"""将媒体 URL 路径转换为本地文件系统路径。
|
||||
|
||||
做什么:把 "/api/media/2026/07/13/abc.png" 转换为
|
||||
"./uploads/2026/07/13/abc.png"
|
||||
为什么:VisionService 需要读取原始图片字节流,
|
||||
而媒体 URL 是 HTTP 访问路径,不是文件系统路径。
|
||||
|
||||
Args:
|
||||
media_url: 媒体文件 URL(如 /api/media/2026/07/13/abc.png)
|
||||
|
||||
Returns:
|
||||
Path: 本地文件路径对象
|
||||
"""
|
||||
# 去掉 URL 前缀 /api/media/,拼接到 UPLOAD_DIR
|
||||
# 例: "/api/media/2026/07/13/abc.png" → "2026/07/13/abc.png"
|
||||
relative = media_url.replace("/api/media/", "", 1)
|
||||
return _UPLOAD_DIR / relative
|
||||
|
||||
|
||||
async def _fetch_recent_employee_text(
|
||||
db, conversation_id: str, employee_id: str, within_seconds: int = 5
|
||||
) -> str:
|
||||
"""获取最近 N 秒内员工的文字消息(Phase 4B 消息融合)。
|
||||
|
||||
做什么:查询同一会话中,当前图片消息之前 within_seconds 秒内,
|
||||
员工发送的文本消息内容。
|
||||
为什么:用户经常先打字描述问题再发截图,或先发截图再补充文字。
|
||||
将文字与图片视觉描述融合后一次性传给 Dify,
|
||||
避免 AI 分别处理两条消息导致上下文割裂。
|
||||
|
||||
Args:
|
||||
db: 异步 DB session
|
||||
conversation_id: 会话 ID
|
||||
employee_id: 员工企微 UserID
|
||||
within_seconds: 时间窗口(秒),默认 5 秒
|
||||
|
||||
Returns:
|
||||
str: 最近的员工文字消息内容(多条用换行拼接),无则返回空字符串
|
||||
"""
|
||||
cutoff = datetime.now() - timedelta(seconds=within_seconds)
|
||||
stmt = (
|
||||
select(Message)
|
||||
.where(
|
||||
Message.conversation_id == conversation_id,
|
||||
Message.sender_type == "employee",
|
||||
Message.sender_id == employee_id,
|
||||
Message.msg_type == "text",
|
||||
Message.created_at >= cutoff,
|
||||
)
|
||||
.order_by(Message.created_at.desc())
|
||||
.limit(3) # 最多取 3 条,避免内容过长
|
||||
)
|
||||
result = await db.execute(stmt)
|
||||
messages = result.scalars().all()
|
||||
|
||||
if not messages:
|
||||
return ""
|
||||
|
||||
# 按时间正序拼接(先发的在前)
|
||||
texts = [m.content for m in reversed(messages) if m.content]
|
||||
return "\n".join(texts)
|
||||
|
||||
|
||||
async def _enrich_image_content(
|
||||
db,
|
||||
media_url: str,
|
||||
original_content: str,
|
||||
conversation_id: str,
|
||||
employee_id: str,
|
||||
) -> str:
|
||||
"""用 VisionService 分析图片,生成增强后的消息内容。
|
||||
|
||||
做什么:
|
||||
1. 从本地文件系统读取图片
|
||||
2. 调用 VisionService.analyze_screenshot() 获取视觉描述
|
||||
3. 查询最近 5 秒内的员工文字消息(消息融合)
|
||||
4. 拼接视觉描述 + 用户文字 → 传给 Dify
|
||||
|
||||
为什么:Dify 文本模型无法直接"看"图片,需要先将图片转为
|
||||
文字描述,再与用户输入融合后传给 Dify 推理。
|
||||
|
||||
降级策略:
|
||||
- 图片文件不存在 → 返回原始 content
|
||||
- VisionService 调用失败 → 返回 "我收到了您的截图,但暂时无法识别内容"
|
||||
- 置信度 < 0.6 → 不注入视觉描述,仅使用用户文字
|
||||
|
||||
Args:
|
||||
db: 异步 DB session
|
||||
media_url: 图片 URL(如 /api/media/2026/07/13/abc.png)
|
||||
original_content: 原始消息内容(如 "[图片] 截图")
|
||||
conversation_id: 会话 ID
|
||||
employee_id: 员工企微 UserID
|
||||
|
||||
Returns:
|
||||
str: 增强后的消息内容(视觉描述 + 用户文字)
|
||||
"""
|
||||
# 1. 读取本地图片文件
|
||||
local_path = _media_url_to_local_path(media_url)
|
||||
if not local_path.exists():
|
||||
logger.warning(f"图片文件不存在: {local_path} (media_url={media_url})")
|
||||
return original_content
|
||||
|
||||
try:
|
||||
image_bytes = local_path.read_bytes()
|
||||
except Exception as e:
|
||||
logger.error(f"读取图片文件失败: {local_path} - {e}")
|
||||
return original_content
|
||||
|
||||
# 2. 调用 VisionService 分析截图
|
||||
vision_service = VisionService()
|
||||
try:
|
||||
result = await vision_service.analyze_screenshot(
|
||||
image_bytes, conversation_id
|
||||
)
|
||||
description = result.get("description", "")
|
||||
confidence = result.get("confidence", 0.0)
|
||||
|
||||
logger.info(
|
||||
f"VisionService 分析完成: conversation={conversation_id}, "
|
||||
f"confidence={confidence:.2f}, desc_len={len(description)}"
|
||||
)
|
||||
|
||||
# 3. 注入视觉描述到会话上下文(供后续多轮对话使用)
|
||||
if description and confidence >= _VISION_CONFIDENCE_THRESHOLD:
|
||||
await vision_service.inject_to_conversation_context(
|
||||
description, conversation_id
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"VisionService 调用异常: {e}")
|
||||
description = ""
|
||||
confidence = 0.0
|
||||
finally:
|
||||
await vision_service.close()
|
||||
|
||||
# 4. 消息融合:查询最近 5 秒内员工的文字消息
|
||||
recent_text = await _fetch_recent_employee_text(
|
||||
db, conversation_id, employee_id, within_seconds=5
|
||||
)
|
||||
|
||||
# 5. 拼接增强内容
|
||||
# 格式:[视觉描述] + [用户最近文字] + [原始消息内容]
|
||||
parts = []
|
||||
|
||||
if description and confidence >= _VISION_CONFIDENCE_THRESHOLD:
|
||||
parts.append(f"[用户发送了截图,视觉理解结果] {description}")
|
||||
|
||||
if recent_text:
|
||||
parts.append(f"[用户最近的文字描述] {recent_text}")
|
||||
|
||||
# 原始内容如果不是纯占位符(如"[图片] 截图"),也加入
|
||||
if original_content and not original_content.startswith("[图片]"):
|
||||
parts.append(original_content)
|
||||
|
||||
if not parts:
|
||||
# 降级:视觉分析失败且无文字补充
|
||||
return "我收到了您的截图,但暂时无法识别内容,请描述一下您遇到的问题。"
|
||||
|
||||
return "\n".join(parts)
|
||||
|
||||
|
||||
async def _persist_and_push(
|
||||
db,
|
||||
conversation: Conversation,
|
||||
@@ -124,6 +302,174 @@ async def _persist_and_push(
|
||||
logger.warning(f"WS 广播 AI 回复给坐席失败(消息已存储): {ws_err}")
|
||||
|
||||
|
||||
async def _persist_and_push_structured(
|
||||
db,
|
||||
conversation: Conversation,
|
||||
employee_id: str,
|
||||
result: dict,
|
||||
):
|
||||
"""持久化结构化 AI 回复并推送给员工端 + 广播坐席端(v2.0 双 WS 通道)。
|
||||
|
||||
改造后的核心变化(2026-07-13):
|
||||
- Dify 返回 JSON {text, action, options},后端解析后同时发两条 WS:
|
||||
① ai_reply → 聊天气泡(text + options)
|
||||
② dynamic_recommend → 侧边栏推荐(action 卡片)
|
||||
- 两条消息同一时刻发出,零时间差到达
|
||||
- 文字明确引用侧边栏内容(如"右侧已为您准备好入口"),语义强关联
|
||||
|
||||
命中判断规则:
|
||||
- 结构化回复且有 action 或 options → 视为命中(AI 在主动引导)
|
||||
- 纯文本回复 → 走原有 _check_knowledge_hit 判断
|
||||
|
||||
Args:
|
||||
db: 异步 DB session
|
||||
conversation: 当前会话对象
|
||||
employee_id: 员工企微 UserID
|
||||
result: get_structured_reply() 返回的结构化结果
|
||||
"""
|
||||
text = result.get("text", "")
|
||||
action = result.get("action")
|
||||
options = result.get("options")
|
||||
hit = result.get("hit", False)
|
||||
is_structured = result.get("is_structured", False)
|
||||
dify_conv_id = result.get("conversation_id")
|
||||
# Phase 6A: 提取诊断阶段
|
||||
diagnosis_stage = result.get("diagnosis_stage")
|
||||
|
||||
# 结构化回复且有 action 或 options → 视为命中(AI 在主动引导/推荐)
|
||||
if is_structured and (action or options):
|
||||
hit = True
|
||||
|
||||
# Phase 6A: 基于 diagnosis_stage 调整会话状态
|
||||
# escalating → AI 建议转人工
|
||||
# resolved → AI 认为问题已解决
|
||||
if diagnosis_stage == "escalating":
|
||||
hit = False # 不计为有效回复,触发转人工
|
||||
elif diagnosis_stage == "resolved":
|
||||
hit = True # 计为有效回复
|
||||
|
||||
should_count = hit
|
||||
should_transfer = not hit
|
||||
|
||||
# 确定消息类型
|
||||
if is_structured and (options or action):
|
||||
msg_type = "ai_structured"
|
||||
else:
|
||||
msg_type = "text"
|
||||
|
||||
# 构建 extra_data(存储 options 和 action 供前端渲染)
|
||||
extra_data = {}
|
||||
if options:
|
||||
extra_data["options"] = options
|
||||
if action:
|
||||
extra_data["action"] = action
|
||||
|
||||
# 1. 存 AI 消息
|
||||
ai_message = Message(
|
||||
conversation_id=conversation.id,
|
||||
sender_type="ai",
|
||||
sender_id="ai_bot",
|
||||
sender_name="Duckula(达寇拉)",
|
||||
content=text,
|
||||
msg_type=msg_type,
|
||||
extra_data=extra_data if extra_data else None,
|
||||
is_read=True,
|
||||
)
|
||||
db.add(ai_message)
|
||||
await db.flush()
|
||||
|
||||
# 2. 更新会话状态
|
||||
if dify_conv_id:
|
||||
conversation.dify_conversation_id = dify_conv_id
|
||||
if should_count:
|
||||
conversation.ai_substantive_reply_count += 1
|
||||
if should_transfer:
|
||||
conversation.status = "queued"
|
||||
# Phase 6A: 将 diagnosis_stage 存入 tags(无需迁移,利用现有 JSON 字段)
|
||||
if diagnosis_stage:
|
||||
tags = conversation.tags or {}
|
||||
tags["diagnosis_stage"] = diagnosis_stage
|
||||
tags["diagnosis_updated_at"] = datetime.now().isoformat()
|
||||
conversation.tags = tags
|
||||
conversation.updated_at = datetime.now()
|
||||
db.add(conversation)
|
||||
await db.flush()
|
||||
await db.commit()
|
||||
|
||||
# 3. 推 ai_reply 给员工端(聊天气泡:text + options)
|
||||
await ws_manager.broadcast_to_employees([employee_id], {
|
||||
"type": "ai_reply",
|
||||
"data": {
|
||||
"message_id": str(ai_message.id),
|
||||
"conversation_id": str(conversation.id),
|
||||
"sender_type": "ai",
|
||||
"sender_id": "ai_bot",
|
||||
"sender_name": "Duckula(达寇拉)",
|
||||
"content": text,
|
||||
"msg_type": msg_type,
|
||||
"extra_data": extra_data if extra_data else None,
|
||||
"is_guidance": False,
|
||||
"ai_reply_count": conversation.ai_substantive_reply_count,
|
||||
"can_call_agent": conversation.ai_substantive_reply_count >= 3,
|
||||
"conversation_status": conversation.status,
|
||||
# Phase 6A: 诊断阶段(前端可据此调整 UI/提示)
|
||||
"diagnosis_stage": diagnosis_stage,
|
||||
},
|
||||
})
|
||||
|
||||
# 4. 推 dynamic_recommend 给员工端侧边栏(仅当 action 非空时)
|
||||
# 与 ai_reply 同一时刻发出 → 零时间差到达
|
||||
if action:
|
||||
recommend_data = {
|
||||
"recommend_id": f"rec_{ai_message.id}",
|
||||
"card_type": action.get("type", "approval_card"),
|
||||
"title": action.get("title", ""),
|
||||
"description": action.get("description", ""),
|
||||
"approval_type": action.get("approval_type"),
|
||||
"confidence": action.get("confidence", 0.85),
|
||||
"message_id": str(ai_message.id),
|
||||
"conversation_id": str(conversation.id),
|
||||
}
|
||||
await ws_manager.broadcast_to_employees([employee_id], {
|
||||
"type": "dynamic_recommend",
|
||||
"data": recommend_data,
|
||||
})
|
||||
logger.info(
|
||||
f"动态推荐已推送: employee={employee_id}, "
|
||||
f"card_type={recommend_data['card_type']}, "
|
||||
f"title={recommend_data['title']}"
|
||||
)
|
||||
|
||||
# 5. 广播坐席端(new_message + conversation_updated)
|
||||
try:
|
||||
await ws_manager.broadcast({
|
||||
"type": "new_message",
|
||||
"data": {
|
||||
"conversation_id": str(conversation.id),
|
||||
"message_id": str(ai_message.id),
|
||||
"sender_type": "ai",
|
||||
"sender_id": "ai_bot",
|
||||
"sender_name": "Duckula(达寇拉)",
|
||||
"content": text,
|
||||
"msg_type": msg_type,
|
||||
"extra_data": extra_data if extra_data else None,
|
||||
},
|
||||
})
|
||||
await ws_manager.broadcast({
|
||||
"type": "conversation_updated",
|
||||
"data": {
|
||||
"conversation_id": str(conversation.id),
|
||||
"status": conversation.status,
|
||||
"assigned_agent_id": (
|
||||
str(conversation.assigned_agent_id)
|
||||
if conversation.assigned_agent_id else None
|
||||
),
|
||||
},
|
||||
})
|
||||
except Exception as ws_err:
|
||||
logger.warning(f"WS 广播结构化 AI 回复给坐席失败(消息已存储): {ws_err}")
|
||||
|
||||
|
||||
async def _handle_byod_query(db, conversation, employee_id, content):
|
||||
"""处理 BYOD 自备电脑补贴查询。
|
||||
|
||||
@@ -370,13 +716,38 @@ async def process_h5_ai_reply(
|
||||
employee_id: str,
|
||||
content: str,
|
||||
dify_conversation_id=None,
|
||||
msg_type: str = "text",
|
||||
media_url: str = None,
|
||||
):
|
||||
"""H5 发送消息后的 AI 回复处理(asyncio.create_task 入口)。
|
||||
|
||||
v2.0 改造(2026-07-13):
|
||||
- AI 回复从流式 SSE 改为 blocking + JSON 结构化输出
|
||||
- Dify 返回 {text, action, options} JSON → 后端解析 → 双 WS 推送
|
||||
- 聊天气泡收到 ai_reply(text + options),侧边栏收到 dynamic_recommend(action)
|
||||
- 新增 ai_thinking 指示器,用户发送后立即看到"正在思考..."
|
||||
|
||||
v2.1 改造(2026-07-13 Phase 4):
|
||||
- 新增图片消息处理分支(msg_type=image)
|
||||
- 图片 → VisionService.analyze_screenshot() → 视觉描述 → 融合到用户文字
|
||||
- 消息融合:查询最近 5 秒内员工的文字消息,与图片描述合并后传给 Dify
|
||||
- 降级:VisionService 失败/低置信度 → 使用原始文字或提示用户描述问题
|
||||
|
||||
流程:
|
||||
- 本地快判断(打招呼 / 呼叫人工)→ 同步结果,整段推送(不调 Dify)
|
||||
- 否则流式调 Dify,逐 chunk 推 ai_reply_chunk,流结束推 ai_reply 终态
|
||||
- 任意异常 → 推 ai_reply_failed,不阻塞用户
|
||||
1. BYOD 关键词拦截 → byod_card 卡片
|
||||
2. 路由关键词拦截 → 名片推荐
|
||||
3. 本地快判断(打招呼/呼叫人工)→ 同步引导
|
||||
4. ★ 图片消息处理(Phase 4A)→ VisionService 分析 → 内容增强
|
||||
5. ★ 结构化 AI 回复(blocking + JSON 解析 + 双 WS 推送)
|
||||
6. 任意异常 → 推 ai_reply_failed
|
||||
|
||||
Args:
|
||||
conversation_id: 会话 ID
|
||||
employee_id: 员工企微 UserID
|
||||
content: 消息文本内容
|
||||
dify_conversation_id: Dify 会话 ID(用于多轮上下文)
|
||||
msg_type: 消息类型(text/image/file),默认 text
|
||||
media_url: 媒体文件 URL(图片消息时使用)
|
||||
"""
|
||||
ai_handler = get_shared_ai_handler()
|
||||
factory = _get_session_factory()
|
||||
@@ -387,31 +758,20 @@ async def process_h5_ai_reply(
|
||||
logger.warning(f"后台 AI 任务:会话不存在 {conversation_id}")
|
||||
return
|
||||
|
||||
is_guidance = False
|
||||
should_count = False
|
||||
should_transfer = False
|
||||
new_dify_conv_id = dify_conversation_id
|
||||
full_parts: list = []
|
||||
|
||||
# === BYOD 关键词拦截 ===
|
||||
# 在打招呼/呼叫人工判断之前,先检查是否为 BYOD(自备电脑补贴)意图。
|
||||
# 命中关键词 → 执行 BYOD 资格检查并推送 byod_card 卡片,不走正常 AI 流程。
|
||||
if _byod_keyword_prefilter(content):
|
||||
# === BYOD 关键词拦截(仅文本消息)===
|
||||
# 图片消息的 content 是占位符(如 "[图片] 截图"),跳过关键词拦截
|
||||
if msg_type == "text" and _byod_keyword_prefilter(content):
|
||||
await _handle_byod_query(db, conversation, employee_id, content)
|
||||
return # BYOD 处理完毕,直接返回
|
||||
return
|
||||
|
||||
# === 业务路由检测(新增)===
|
||||
# 在 BYOD 检测之后、打招呼/呼叫人工检测之前,检查是否为非IT业务路由。
|
||||
# 命中路由关键词 → 调用 Dify 统一意图识别 → non_it_routing && confidence≥0.7
|
||||
# → 发送名片三段式消息(路由文本 + contact_card + 系统提示)
|
||||
# 置信度不足或非路由意图 → 继续往下走正常 AI 流程
|
||||
if routing_keyword_prefilter(content):
|
||||
# === 业务路由检测(仅文本消息)===
|
||||
if msg_type == "text" and routing_keyword_prefilter(content):
|
||||
routed = await _handle_routing(db, conversation, employee_id, content)
|
||||
if routed:
|
||||
return # 路由名片已发送,直接返回
|
||||
return
|
||||
|
||||
# 本地快判断(不打 Dify):打招呼 / 呼叫人工 → 同步路径
|
||||
if ai_handler.is_greeting(content) or ai_handler.is_call_human(content):
|
||||
# === 本地快判断:打招呼 / 呼叫人工(仅文本消息)===
|
||||
if msg_type == "text" and (ai_handler.is_greeting(content) or ai_handler.is_call_human(content)):
|
||||
result = await ai_handler.handle_message(
|
||||
content=content,
|
||||
dify_conversation_id=dify_conversation_id,
|
||||
@@ -424,40 +784,110 @@ async def process_h5_ai_reply(
|
||||
)
|
||||
return
|
||||
|
||||
# 流式调 Dify(get_reply_stream 内部已处理真 SSE / 非流式 fallback)
|
||||
# 注意:首参是 message(用户文本),不是 content
|
||||
async for chunk in ai_handler.ai_service.get_reply_stream(
|
||||
message=content,
|
||||
conversation_id=dify_conversation_id,
|
||||
user_id=employee_id,
|
||||
):
|
||||
delta = chunk.get("delta", "")
|
||||
if delta:
|
||||
full_parts.append(delta)
|
||||
await ws_manager.broadcast_to_employees([employee_id], {
|
||||
"type": "ai_reply_chunk",
|
||||
"data": {
|
||||
"conversation_id": conversation_id,
|
||||
"chunk": delta,
|
||||
},
|
||||
})
|
||||
if chunk.get("finished"):
|
||||
new_dify_conv_id = chunk.get("conversation_id") or dify_conversation_id
|
||||
hit = chunk.get("hit")
|
||||
# 命中 → 计数;未命中 → 转人工
|
||||
should_count = bool(hit)
|
||||
should_transfer = not bool(hit)
|
||||
# === ★ v2.1 图片消息处理(Phase 4A/4B)===
|
||||
# 做什么:检测到图片消息 → 调用 VisionService 分析截图 →
|
||||
# 将视觉描述与用户文字融合 → 传给 Dify 推理
|
||||
# 为什么:Dify 文本模型无法"看"图片,需要先将图片转为文字描述
|
||||
# 降级:VisionService 失败 → 使用原始 content 继续流程
|
||||
enriched_content = content # 默认使用原始内容
|
||||
if msg_type == "image" and media_url:
|
||||
logger.info(
|
||||
f"图片消息检测: conversation={conversation_id}, "
|
||||
f"media_url={media_url}"
|
||||
)
|
||||
try:
|
||||
enriched_content = await _enrich_image_content(
|
||||
db=db,
|
||||
media_url=media_url,
|
||||
original_content=content,
|
||||
conversation_id=conversation_id,
|
||||
employee_id=employee_id,
|
||||
)
|
||||
logger.info(
|
||||
f"图片内容增强完成: original_len={len(content)}, "
|
||||
f"enriched_len={len(enriched_content)}"
|
||||
)
|
||||
except Exception as vision_err:
|
||||
logger.error(
|
||||
f"VisionService 处理失败,降级为纯文本: {vision_err}"
|
||||
)
|
||||
# 降级:使用原始 content,AI 会收到 "[图片] 截图" 这样的占位符
|
||||
# Dify 会回复"我收到了您的截图,请描述一下问题"
|
||||
|
||||
content_ai = "".join(full_parts)
|
||||
if not content_ai:
|
||||
# 流式无内容(极端情况),给降级提示,不转人工
|
||||
content_ai = "⚠️ AI 暂时没有返回内容,请输入「IT」转人工。"
|
||||
should_count = False
|
||||
should_transfer = False
|
||||
await _persist_and_push(
|
||||
db, conversation, employee_id, content_ai,
|
||||
is_guidance, should_count, should_transfer, new_dify_conv_id,
|
||||
# === ★ v2.0 结构化 AI 回复(替代流式)===
|
||||
# 1. 立即推送 "正在思考..." 指示器
|
||||
# 同时推给员工(气泡动画)和坐席(状态指示)
|
||||
await ws_manager.broadcast_to_employees([employee_id], {
|
||||
"type": "ai_thinking",
|
||||
"data": {
|
||||
"conversation_id": conversation_id,
|
||||
},
|
||||
})
|
||||
# 坐席端也通知:AI 正在处理此会话的消息
|
||||
try:
|
||||
await ws_manager.broadcast({
|
||||
"type": "ai_thinking",
|
||||
"data": {
|
||||
"conversation_id": conversation_id,
|
||||
"employee_id": employee_id,
|
||||
},
|
||||
})
|
||||
except Exception:
|
||||
pass # 坐席端通知失败不影响主流程
|
||||
|
||||
# 2. 启动延迟 "仍在思考" 后台任务(15 秒后触发)
|
||||
# 如果 Dify 在 15 秒内返回,此任务会被取消
|
||||
async def _push_still_thinking():
|
||||
"""15 秒后推送 "仍在思考" 提示,缓解用户等待焦虑。"""
|
||||
await asyncio.sleep(15)
|
||||
await ws_manager.broadcast_to_employees([employee_id], {
|
||||
"type": "ai_thinking",
|
||||
"data": {
|
||||
"conversation_id": conversation_id,
|
||||
"status": "still_thinking",
|
||||
},
|
||||
})
|
||||
|
||||
thinking_task = asyncio.create_task(_push_still_thinking())
|
||||
|
||||
# 3. 调用 Dify(blocking 模式 + JSON 解析 + 30 秒硬超时)
|
||||
# get_structured_reply 内部处理 HTTP 错误和 JSON 解析失败
|
||||
# asyncio.wait_for 处理 30 秒硬超时 → 建议转人工
|
||||
# 注意:图片消息使用 enriched_content(视觉描述+用户文字融合)
|
||||
try:
|
||||
result = await asyncio.wait_for(
|
||||
ai_handler.ai_service.get_structured_reply(
|
||||
message=enriched_content,
|
||||
conversation_id=dify_conversation_id,
|
||||
user_id=employee_id,
|
||||
),
|
||||
timeout=30,
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
# 30 秒硬超时 → 建议转人工
|
||||
thinking_task.cancel()
|
||||
logger.warning(f"Dify 30 秒超时: conversation={conversation_id}")
|
||||
await ws_manager.broadcast_to_employees([employee_id], {
|
||||
"type": "ai_reply_failed",
|
||||
"data": {
|
||||
"conversation_id": conversation_id,
|
||||
"message": "AI 响应时间较长,建议转人工坐席处理。",
|
||||
},
|
||||
})
|
||||
return
|
||||
|
||||
# 4. 取消 "仍在思考" 任务(Dify 已返回)
|
||||
thinking_task.cancel()
|
||||
try:
|
||||
await thinking_task # 等待 task 真正取消,避免 warning
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
|
||||
# 5. 持久化 + 双 WS 推送(ai_reply + dynamic_recommend)
|
||||
await _persist_and_push_structured(
|
||||
db, conversation, employee_id, result,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"后台 AI 任务异常: {e}", exc_info=True)
|
||||
try:
|
||||
@@ -465,7 +895,7 @@ async def process_h5_ai_reply(
|
||||
"type": "ai_reply_failed",
|
||||
"data": {
|
||||
"conversation_id": conversation_id,
|
||||
"message": "⚠️ AI 服务异常,请输入「IT」转人工或稍后重试。",
|
||||
"message": "AI 服务异常,请转人工坐席或稍后重试。",
|
||||
},
|
||||
})
|
||||
except Exception:
|
||||
|
||||
Reference in New Issue
Block a user