{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "49092bc2",
   "metadata": {},
   "source": [
    "# DataAnalyst - 智能数据分析助手\n",
    "\n",
    "> 基于 HelloAgents 框架的三阶段多智能体数据分析流水线：**规划 → 分析 → 报告**，一键把任意 CSV 变成图文并茂的数据分析报告。\n",
    "\n",
    "## 项目简介\n",
    "\n",
    "数据分析是业务决策的重要环节，但人工分析耗时长、容易遗漏数据中的关键模式。DataAnalyst 让你只需**替换一个 CSV 文件**，即可自动完成：数据探查 → 分析任务规划 → 多工具深度分析 → 自动生成图表 → 撰写 Markdown 分析报告。\n",
    "\n",
    "## 架构设计\n",
    "\n",
    "```\n",
    "                 ┌─────────────────────────────────────────────┐\n",
    "  sales_data.csv │                                             │\n",
    "  (任意CSV) ───► │  阶段1 规划师Planner(ReActAgent)             │\n",
    "                 │   └─ 调用 data_overview 探查数据             │\n",
    "                 │   └─ 输出 3~5 个分析任务(JSON)               │\n",
    "                 │                                             │\n",
    "                 │  阶段2 分析员Analyst(ReActAgent)             │\n",
    "                 │   └─ 逐任务调用6个分析工具(统计/相关性/      │\n",
    "                 │      异常检测/聚合/绘图)并给出数字结论       │\n",
    "                 │                                             │\n",
    "                 │  阶段3 撰写师Reporter(SimpleAgent)           │\n",
    "                 │   └─ 汇总结论 → Markdown报告 + 嵌入图表      │\n",
    "                 └─────────────────────────────────────────────┘\n",
    "                                │\n",
    "                                ▼\n",
    "              outputs/analysis_report.md + outputs/charts/*.png\n",
    "```\n",
    "\n",
    "## 作者信息\n",
    "- 姓名：夏明浩\n",
    "- GitHub：[@minghaoxia61-web](https://github.com/minghaoxia61-web)\n",
    "- 日期：2026-09-19"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "192fd6c0",
   "metadata": {},
   "source": [
    "## 第1部分：环境配置"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19abc4a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 导入必要的库\n",
    "import os\n",
    "import json\n",
    "import re\n",
    "import glob\n",
    "import warnings\n",
    "from typing import Any, Dict, List\n",
    "\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "from hello_agents import HelloAgentsLLM, SimpleAgent, ReActAgent, ToolRegistry, Config\n",
    "from hello_agents.tools import Tool, ToolParameter, ToolResponse, ToolErrorCode\n",
    "\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "# 加载 .env 中的 LLM 配置（LLM_MODEL_ID / LLM_API_KEY / LLM_BASE_URL，参考 .env.example）\n",
    "load_dotenv()\n",
    "\n",
    "# matplotlib 中文显示设置（Windows 用微软雅黑，macOS/Linux 自动回退）\n",
    "plt.rcParams[\"font.sans-serif\"] = [\"Microsoft YaHei\", \"SimHei\", \"PingFang SC\", \"Noto Sans CJK SC\", \"sans-serif\"]\n",
    "plt.rcParams[\"axes.unicode_minus\"] = False\n",
    "\n",
    "# 项目路径约定\n",
    "DATA_PATH = \"data/sales_data.csv\"      # 待分析的数据集：换成你自己的 CSV 即可，无需改任何代码\n",
    "OUTPUT_DIR = \"outputs\"                 # 分析报告与图表的输出目录\n",
    "CHART_DIR = os.path.join(OUTPUT_DIR, \"charts\")\n",
    "os.makedirs(CHART_DIR, exist_ok=True)\n",
    "\n",
    "if not os.getenv(\"LLM_API_KEY\"):\n",
    "    print(\"⚠️ 未检测到 LLM_API_KEY，请先复制 .env.example 为 .env 并填入你的 API 密钥\")\n",
    "else:\n",
    "    print(\"✅ 环境配置完成，LLM 模型:\", os.getenv(\"LLM_MODEL_ID\", \"未设置(将使用框架默认值)\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a806291",
   "metadata": {},
   "source": [
    "## 第2部分：数据准备\n",
    "\n",
    "本项目自带一份模拟电商销售数据（800 条订单，含季节性、地区差异、渠道差异、少量缺失值与异常大额订单）。\n",
    "**要分析你自己的数据，只需把 CSV 放到 `data/` 目录并修改上面的 `DATA_PATH`。**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b63e2d11",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 读取数据集\n",
    "df = pd.read_csv(DATA_PATH, encoding=\"utf-8-sig\")\n",
    "GLOBAL_DF = df   # 工具层共享的数据引用\n",
    "\n",
    "print(f\"数据集规模: {df.shape[0]} 行 × {df.shape[1]} 列\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1453ce2e",
   "metadata": {},
   "source": [
    "## 第3部分：数据分析工具定义\n",
    "\n",
    "基于 hello-agents 的 `Tool` 基类实现 6 个数据分析工具，每个工具负责一类原子分析能力，返回 `ToolResponse`（LLM 阅读的文本 + 结构化数据）：\n",
    "\n",
    "| 工具 | 功能 |\n",
    "|---|---|\n",
    "| `data_overview` | 数据概览：行列数、类型、缺失率、唯一值、数值列统计摘要 |\n",
    "| `column_profile` | 单列画像：数值列统计量 / 类别列频次 Top 榜 |\n",
    "| `correlation_analysis` | 数值列两两皮尔逊相关系数，按绝对值排序 |\n",
    "| `group_aggregate` | 按类别列分组聚合（sum/mean/count...），返回 Top N |\n",
    "| `detect_outliers` | IQR 异常值检测：阈值、数量、最大异常样本 |\n",
    "| `plot_chart` | 6 种统计图表（直方图/柱状/箱线/折线/散点/热力图），自动处理中文字体并保存 PNG |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "396b0874",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ========================================\n",
    "# 工具1-3：概览 / 列画像 / 相关性\n",
    "# ========================================\n",
    "class DataOverviewTool(Tool):\n",
    "    \"\"\"输出数据集整体概况，供规划智能体了解数据结构\"\"\"\n",
    "\n",
    "    def __init__(self):\n",
    "        super().__init__(\n",
    "            name=\"data_overview\",\n",
    "            description=\"获取数据集整体概况：行列数、每列类型/缺失/唯一值、数值列统计摘要。分析开始前应先调用本工具。\",\n",
    "        )\n",
    "\n",
    "    def get_parameters(self) -> List[ToolParameter]:\n",
    "        return []\n",
    "\n",
    "    def run(self, parameters: Dict[str, Any]) -> ToolResponse:\n",
    "        df = GLOBAL_DF\n",
    "        lines = [f\"数据规模: {df.shape[0]} 行 × {df.shape[1]} 列\", \"\", \"字段概况:\"]\n",
    "        for col in df.columns:\n",
    "            s = df[col]\n",
    "            missing = int(s.isna().sum())\n",
    "            miss_pct = missing / len(df) * 100\n",
    "            uniq = int(s.nunique(dropna=True))\n",
    "            line = f\"- {col} | 类型:{s.dtype} | 缺失:{missing}({miss_pct:.1f}%) | 唯一值:{uniq}\"\n",
    "            if uniq <= 8:\n",
    "                tops = s.value_counts().head(3)\n",
    "                line += \" | 高频值: \" + \", \".join(f\"{k}({v})\" for k, v in tops.items())\n",
    "            lines.append(line)\n",
    "        num_cols = df.select_dtypes(include=\"number\").columns.tolist()\n",
    "        if num_cols:\n",
    "            lines.append(\"\")\n",
    "            lines.append(\"数值列统计摘要:\")\n",
    "            lines.append(df[num_cols].describe().T.round(2).to_string())\n",
    "        return ToolResponse.success(text=\"\\n\".join(lines))\n",
    "\n",
    "\n",
    "class ColumnProfileTool(Tool):\n",
    "    \"\"\"深入分析单个指定列\"\"\"\n",
    "\n",
    "    def __init__(self):\n",
    "        super().__init__(\n",
    "            name=\"column_profile\",\n",
    "            description=\"深入分析单个指定列：数值列返回均值/标准差/分位数/偏度，类别列返回频次Top榜单，日期列返回时间范围。\",\n",
    "        )\n",
    "\n",
    "    def get_parameters(self) -> List[ToolParameter]:\n",
    "        return [ToolParameter(name=\"column\", type=\"string\",\n",
    "                              description=\"要分析的列名（须与数据集列名完全一致）\", required=True)]\n",
    "\n",
    "    def run(self, parameters: Dict[str, Any]) -> ToolResponse:\n",
    "        col = parameters.get(\"column\", \"\")\n",
    "        df = GLOBAL_DF\n",
    "        if col not in df.columns:\n",
    "            return ToolResponse.error(code=ToolErrorCode.INVALID_PARAM,\n",
    "                                      message=f\"列不存在: {col}。可用列: {list(df.columns)}\")\n",
    "        s = df[col]\n",
    "        lines = [f\"列 {col} 画像（非空 {int(s.notna().sum())} / {len(s)}）\"]\n",
    "        if pd.api.types.is_datetime64_any_dtype(s):\n",
    "            lines.append(f\"时间范围: {s.min()} ~ {s.max()}\")\n",
    "        elif pd.api.types.is_numeric_dtype(s):\n",
    "            lines.append(s.describe().round(2).to_string())\n",
    "            lines.append(f\"偏度: {s.skew():.2f}\")\n",
    "        else:\n",
    "            vc = s.value_counts().head(8)\n",
    "            lines.append(\"频次Top8:\")\n",
    "            for k, v in vc.items():\n",
    "                lines.append(f\"- {k}: {v} ({v / len(s) * 100:.1f}%)\")\n",
    "        return ToolResponse.success(text=\"\\n\".join(lines))\n",
    "\n",
    "\n",
    "class CorrelationTool(Tool):\n",
    "    \"\"\"数值列两两相关性分析\"\"\"\n",
    "\n",
    "    def __init__(self):\n",
    "        super().__init__(\n",
    "            name=\"correlation_analysis\",\n",
    "            description=\"计算所有数值列两两之间的皮尔逊相关系数，返回相关性最强的字段对（按绝对值降序）。用于发现字段间的线性关联。\",\n",
    "        )\n",
    "\n",
    "    def get_parameters(self) -> List[ToolParameter]:\n",
    "        return [ToolParameter(name=\"top_n\", type=\"integer\", description=\"返回相关性最强的前N对字段，默认10\", required=False)]\n",
    "\n",
    "    def run(self, parameters: Dict[str, Any]) -> ToolResponse:\n",
    "        num = GLOBAL_DF.select_dtypes(include=\"number\")\n",
    "        if num.shape[1] < 2:\n",
    "            return ToolResponse.error(code=ToolErrorCode.INVALID_PARAM, message=\"数值列不足2列，无法计算相关性\")\n",
    "        corr = num.corr().round(3)\n",
    "        pairs = []\n",
    "        cols = corr.columns\n",
    "        for i in range(len(cols)):\n",
    "            for j in range(i + 1, len(cols)):\n",
    "                pairs.append((cols[i], cols[j], corr.iloc[i, j]))\n",
    "        pairs.sort(key=lambda x: abs(x[2]), reverse=True)\n",
    "        top_n = int(parameters.get(\"top_n\") or 10)\n",
    "        lines = [\"相关性最强的字段对（皮尔逊系数）:\"]\n",
    "        for a, b, r in pairs[:top_n]:\n",
    "            strength = \"强\" if abs(r) >= 0.7 else (\"中等\" if abs(r) >= 0.4 else \"弱\")\n",
    "            lines.append(f\"- {a} × {b}: {r:+.3f}（{strength}{'负' if r < 0 else '正'}相关）\")\n",
    "        return ToolResponse.success(text=\"\\n\".join(lines), data={\"matrix\": corr.to_dict()})\n",
    "\n",
    "print(\"✅ 工具1-3定义完成：data_overview / column_profile / correlation_analysis\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "603fb52e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ========================================\n",
    "# 工具4-6：分组聚合 / 异常检测 / 绘图\n",
    "# ========================================\n",
    "class GroupAggregateTool(Tool):\n",
    "    \"\"\"按类别列分组聚合统计\"\"\"\n",
    "\n",
    "    def __init__(self):\n",
    "        super().__init__(\n",
    "            name=\"group_aggregate\",\n",
    "            description=\"按某个类别列分组，对某个数值列做聚合统计（sum/mean/count/max/min），返回Top N分组结果。适合对比不同类别/地区/渠道的指标。\",\n",
    "        )\n",
    "\n",
    "    def get_parameters(self) -> List[ToolParameter]:\n",
    "        return [\n",
    "            ToolParameter(name=\"group_col\", type=\"string\", description=\"分组列名（类别列，如：地区、销售渠道、产品类别）\", required=True),\n",
    "            ToolParameter(name=\"value_col\", type=\"string\", description=\"被聚合的数值列名（如：销售额）\", required=True),\n",
    "            ToolParameter(name=\"agg\", type=\"string\", description=\"聚合方式: sum/mean/count/max/min，默认sum\", required=False),\n",
    "            ToolParameter(name=\"top_n\", type=\"integer\", description=\"返回前N组，默认10\", required=False),\n",
    "        ]\n",
    "\n",
    "    def run(self, parameters: Dict[str, Any]) -> ToolResponse:\n",
    "        df = GLOBAL_DF\n",
    "        g, v = parameters.get(\"group_col\", \"\"), parameters.get(\"value_col\", \"\")\n",
    "        agg = (parameters.get(\"agg\") or \"sum\").lower()\n",
    "        top_n = int(parameters.get(\"top_n\") or 10)\n",
    "        if g not in df.columns or v not in df.columns:\n",
    "            return ToolResponse.error(code=ToolErrorCode.INVALID_PARAM,\n",
    "                                      message=f\"列不存在。可用列: {list(df.columns)}\")\n",
    "        if agg not in {\"sum\", \"mean\", \"count\", \"max\", \"min\"}:\n",
    "            return ToolResponse.error(code=ToolErrorCode.INVALID_PARAM, message=f\"不支持的聚合方式: {agg}\")\n",
    "        res = df.groupby(g)[v].agg(agg).sort_values(ascending=False)\n",
    "        share = (res / res.sum() * 100).round(1) if agg == \"sum\" else None\n",
    "        lines = [f\"按 {g} 分组对 {v} 做 {agg}（Top {min(top_n, len(res))}）:\"]\n",
    "        for i, (k, val) in enumerate(res.head(top_n).items(), 1):\n",
    "            s = f\" | 占比 {share[k]}%\" if share is not None else \"\"\n",
    "            lines.append(f\"{i}. {k}: {round(float(val), 2)}{s}\")\n",
    "        return ToolResponse.success(text=\"\\n\".join(lines), data={\"result\": res.head(top_n).to_dict()})\n",
    "\n",
    "\n",
    "class OutlierTool(Tool):\n",
    "    \"\"\"IQR 异常值检测\"\"\"\n",
    "\n",
    "    def __init__(self):\n",
    "        super().__init__(\n",
    "            name=\"detect_outliers\",\n",
    "            description=\"用IQR方法检测指定数值列的异常值：返回正常范围阈值、异常点数量与占比、最大的异常样本。\",\n",
    "        )\n",
    "\n",
    "    def get_parameters(self) -> List[ToolParameter]:\n",
    "        return [ToolParameter(name=\"column\", type=\"string\", description=\"要检测异常值的数值列名\", required=True)]\n",
    "\n",
    "    def run(self, parameters: Dict[str, Any]) -> ToolResponse:\n",
    "        col = parameters.get(\"column\", \"\")\n",
    "        df = GLOBAL_DF\n",
    "        if col not in df.columns:\n",
    "            return ToolResponse.error(code=ToolErrorCode.INVALID_PARAM,\n",
    "                                      message=f\"列不存在。可用列: {list(df.columns)}\")\n",
    "        s = pd.to_numeric(df[col], errors=\"coerce\").dropna()\n",
    "        q1, q3 = s.quantile(0.25), s.quantile(0.75)\n",
    "        iqr = q3 - q1\n",
    "        low, high = q1 - 1.5 * iqr, q3 + 1.5 * iqr\n",
    "        mask = (s < low) | (s > high)\n",
    "        out = df.loc[s[mask].index]\n",
    "        lines = [f\"列 {col} 的IQR异常检测:\",\n",
    "                 f\"- 正常范围: [{low:.2f}, {high:.2f}]（Q1={q1:.2f}, Q3={q3:.2f}）\",\n",
    "                 f\"- 异常点: {len(out)} 个，占比 {len(out) / len(df) * 100:.2f}%\"]\n",
    "        if len(out):\n",
    "            id_cols = [c for c in df.columns if (\"ID\" in c or \"日期\" in c) and c != col][:2]\n",
    "            top = out.sort_values(col, ascending=False).head(5)\n",
    "            lines.append(\"- 最大的异常样本:\")\n",
    "            lines.append(top[id_cols + [col]].to_string(index=False))\n",
    "        return ToolResponse.success(text=\"\\n\".join(lines))\n",
    "\n",
    "\n",
    "_CHART_SEQ = {\"n\": 0}   # 图表编号（避免文件名冲突）\n",
    "\n",
    "def _safe_name(x: str) -> str:\n",
    "    \"\"\"把列名转成安全的文件名片段\"\"\"\n",
    "    return \"\".join(c if c.isalnum() else \"_\" for c in str(x))[:20] or \"chart\"\n",
    "\n",
    "\n",
    "class PlotChartTool(Tool):\n",
    "    \"\"\"生成统计图表并保存 PNG，返回可嵌入 Markdown 的相对路径\"\"\"\n",
    "\n",
    "    def __init__(self):\n",
    "        super().__init__(\n",
    "            name=\"plot_chart\",\n",
    "            description=\"生成统计图表并保存为PNG（中文可正常显示）。类型: histogram/bar/box/line/scatter/heatmap。返回图片相对路径（相对outputs目录），可直接嵌入Markdown报告。\",\n",
    "        )\n",
    "\n",
    "    def get_parameters(self) -> List[ToolParameter]:\n",
    "        return [\n",
    "            ToolParameter(name=\"chart_type\", type=\"string\",\n",
    "                          description=\"图表类型: histogram/bar/box/line/scatter/heatmap\", required=True),\n",
    "            ToolParameter(name=\"x_column\", type=\"string\", description=\"X轴列名（heatmap可留空）\", required=False),\n",
    "            ToolParameter(name=\"y_column\", type=\"string\", description=\"Y轴数值列名（histogram/box/heatmap可留空）\", required=False),\n",
    "            ToolParameter(name=\"agg\", type=\"string\", description=\"bar/line图的聚合方式: sum/mean/count，默认sum\", required=False),\n",
    "            ToolParameter(name=\"title\", type=\"string\", description=\"图表标题（中文），默认自动生成\", required=False),\n",
    "        ]\n",
    "\n",
    "    def run(self, parameters: Dict[str, Any]) -> ToolResponse:\n",
    "        df = GLOBAL_DF\n",
    "        ctype = (parameters.get(\"chart_type\") or \"\").lower()\n",
    "        x, y = parameters.get(\"x_column\") or \"\", parameters.get(\"y_column\") or \"\"\n",
    "        agg = (parameters.get(\"agg\") or \"sum\").lower()\n",
    "        title = parameters.get(\"title\") or \"\"\n",
    "        num_cols = df.select_dtypes(include=\"number\").columns.tolist()\n",
    "\n",
    "        if ctype not in {\"histogram\", \"bar\", \"box\", \"line\", \"scatter\", \"heatmap\"}:\n",
    "            return ToolResponse.error(code=ToolErrorCode.INVALID_PARAM, message=f\"不支持的图表类型: {ctype}\")\n",
    "        if ctype != \"heatmap\":   # 校验列名\n",
    "            need = {\"histogram\": [], \"box\": [x or y], \"bar\": [x, y], \"line\": [x, y], \"scatter\": [x, y]}[ctype]\n",
    "            for c in [c for c in need if c]:\n",
    "                if c not in df.columns:\n",
    "                    return ToolResponse.error(code=ToolErrorCode.INVALID_PARAM,\n",
    "                                              message=f\"列不存在: {c}。可用列: {list(df.columns)}\")\n",
    "        if agg not in {\"sum\", \"mean\", \"count\"}:\n",
    "            agg = \"sum\"\n",
    "\n",
    "        fig, ax = plt.subplots(figsize=(8, 5))\n",
    "        if ctype == \"heatmap\":\n",
    "            corr = df[num_cols].corr()\n",
    "            im = ax.imshow(corr, cmap=\"coolwarm\", vmin=-1, vmax=1)\n",
    "            ax.set_xticks(range(len(corr.columns)), corr.columns, rotation=45, ha=\"right\")\n",
    "            ax.set_yticks(range(len(corr.columns)), corr.columns)\n",
    "            for i in range(len(corr.columns)):\n",
    "                for j in range(len(corr.columns)):\n",
    "                    ax.text(j, i, f\"{corr.iloc[i, j]:.2f}\", ha=\"center\", va=\"center\", fontsize=8)\n",
    "            fig.colorbar(im, ax=ax, shrink=0.8)\n",
    "            title = title or \"数值列相关性热力图\"\n",
    "        elif ctype == \"histogram\":\n",
    "            if not num_cols:\n",
    "                return ToolResponse.error(code=ToolErrorCode.EXECUTION_ERROR, message=\"数据集中没有数值列，无法绘制直方图\")\n",
    "            col = y or x or num_cols[0]\n",
    "            data = pd.to_numeric(df[col], errors=\"coerce\").dropna()\n",
    "            if data.empty:\n",
    "                return ToolResponse.error(code=ToolErrorCode.EXECUTION_ERROR,\n",
    "                                          message=f\"列 {col} 无法转换为数值，无法绘制直方图\")\n",
    "            ax.hist(data, bins=30, color=\"#4C72B0\", edgecolor=\"white\")\n",
    "            ax.set_xlabel(col)\n",
    "            ax.set_ylabel(\"频数\")\n",
    "            title = title or f\"{col} 的分布直方图\"\n",
    "        elif ctype == \"box\":\n",
    "            col = y or x\n",
    "            data = pd.to_numeric(df[col], errors=\"coerce\").dropna()\n",
    "            if data.empty:\n",
    "                return ToolResponse.error(code=ToolErrorCode.EXECUTION_ERROR,\n",
    "                                          message=f\"列 {col} 无法绘制箱线图（非数值列或无有效数据）\")\n",
    "            ax.boxplot(data, vert=True, tick_labels=[col])\n",
    "            ax.set_ylabel(col)\n",
    "            title = title or f\"{col} 的箱线图\"\n",
    "        elif ctype == \"bar\":\n",
    "            tmp = df.dropna(subset=[x])\n",
    "            if agg == \"count\" or not y:\n",
    "                res = tmp.groupby(x).size().sort_values(ascending=False)\n",
    "                ylab = \"数量\"\n",
    "            else:\n",
    "                res = tmp.groupby(x)[y].agg(agg).sort_values(ascending=False)\n",
    "                ylab = f\"{y}（{agg}）\"\n",
    "            res = res.head(10)\n",
    "            ax.barh([str(k) for k in res.index][::-1], list(res.values)[::-1], color=\"#4C72B0\")\n",
    "            ax.set_xlabel(ylab)\n",
    "            title = title or f\"各{x}的{ylab}对比（Top{len(res)}）\"\n",
    "        elif ctype == \"line\":\n",
    "            tmp = df.dropna(subset=[x, y]).copy()\n",
    "            xt = pd.to_datetime(tmp[x], errors=\"coerce\")\n",
    "            if xt.notna().sum() > len(tmp) * 0.8:      # 日期列：按月重采样\n",
    "                tmp[\"_t\"] = xt\n",
    "                res = tmp.set_index(\"_t\")[y].resample(\"ME\").agg(agg)\n",
    "                ax.plot(res.index, res.values, marker=\"o\", color=\"#4C72B0\")\n",
    "                ax.set_xlabel(x + \"（按月）\")\n",
    "            else:\n",
    "                res = tmp.groupby(x)[y].agg(agg).sort_index()\n",
    "                ax.plot([str(k) for k in res.index], res.values, marker=\"o\", color=\"#4C72B0\")\n",
    "                ax.set_xlabel(x)\n",
    "            ax.set_ylabel(f\"{y}（{agg}）\")\n",
    "            title = title or f\"{y}随{x}的变化趋势（{agg}）\"\n",
    "        else:                                          # scatter\n",
    "            tmp = df.dropna(subset=[x, y])\n",
    "            if tmp.empty:\n",
    "                return ToolResponse.error(code=ToolErrorCode.EXECUTION_ERROR,\n",
    "                                          message=f\"列 {x} 或 {y} 无有效成对数据，无法绘制散点图\")\n",
    "            if len(tmp) > 2000:\n",
    "                tmp = tmp.sample(2000, random_state=42)\n",
    "            ax.scatter(tmp[x], tmp[y], s=12, alpha=0.5, color=\"#4C72B0\")\n",
    "            ax.set_xlabel(x)\n",
    "            ax.set_ylabel(y)\n",
    "            title = title or f\"{x} 与 {y} 的散点图\"\n",
    "        ax.set_title(title)\n",
    "        ax.grid(alpha=0.25)\n",
    "\n",
    "        _CHART_SEQ[\"n\"] += 1\n",
    "        # 文件名使用实际绘制的列（箱线图/直方图取被统计列，热力图为corr）\n",
    "        name_col = {\"heatmap\": \"corr\", \"histogram\": (y or x or (num_cols[0] if num_cols else \"data\")),\n",
    "                    \"box\": (y or x), \"bar\": x, \"line\": x, \"scatter\": x}[ctype]\n",
    "        fname = f\"chart_{_CHART_SEQ['n']}_{ctype}_{_safe_name(name_col)}.png\"\n",
    "        rel = f\"charts/{fname}\"\n",
    "        fig.savefig(os.path.join(CHART_DIR, fname), dpi=150, bbox_inches=\"tight\")\n",
    "        plt.close(\"all\")\n",
    "        return ToolResponse.success(\n",
    "            text=(f\"图表已生成: {rel}（相对 {OUTPUT_DIR}/ 目录）\\n标题: {title}\\n\"\n",
    "                  f\"请在报告对应小节用 ![图表描述]({rel}) 嵌入该图片。\"),\n",
    "            data={\"path\": rel},\n",
    "        )\n",
    "\n",
    "print(\"✅ 工具4-6定义完成：group_aggregate / detect_outliers / plot_chart\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39ac9f19",
   "metadata": {},
   "source": [
    "## 第4部分：智能体构建\n",
    "\n",
    "| 智能体 | 范式 | 职责 | 配备工具 |\n",
    "|---|---|---|---|\n",
    "| 分析规划师 Planner | ReActAgent | 探查数据、规划 3~5 个分析任务（JSON） | data_overview、column_profile |\n",
    "| 数据分析员 Analyst | ReActAgent | 逐任务调用工具完成分析并给出数字结论 | 全部 6 个工具 |\n",
    "| 报告撰写师 Reporter | SimpleAgent | 汇总结论，撰写图文并茂的 Markdown 报告 | 无（纯生成） |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d87179c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ========================================\n",
    "# 工具注册表：规划智能体配轻量探查工具，分析智能体配全部分析工具\n",
    "# ========================================\n",
    "planner_registry = ToolRegistry()\n",
    "planner_registry.register_tool(DataOverviewTool())\n",
    "planner_registry.register_tool(ColumnProfileTool())\n",
    "\n",
    "analysis_registry = ToolRegistry()\n",
    "for t in [DataOverviewTool(), ColumnProfileTool(), CorrelationTool(),\n",
    "          GroupAggregateTool(), OutlierTool(), PlotChartTool()]:\n",
    "    analysis_registry.register_tool(t)\n",
    "\n",
    "# LLM 客户端（自动读取 .env 中的 LLM_MODEL_ID / LLM_API_KEY / LLM_BASE_URL）\n",
    "llm = HelloAgentsLLM()\n",
    "\n",
    "# 说明: hello-agents 1.0.0 的 TraceLogger 在每次 run() 结束时关闭轨迹文件，\n",
    "# 同一智能体第二次 run() 时会因写入已关闭的文件句柄而报错（I/O operation on closed file）。\n",
    "# 流水线需要对同一智能体多轮调用，因此这里关闭轨迹追踪（不影响分析与报告结果）。\n",
    "agent_config = Config(trace_enabled=False)\n",
    "\n",
    "PLANNER_PROMPT = \"\"\"你是一位资深数据分析规划师，负责为已加载到内存中的CSV数据集制定分析计划。\n",
    "\n",
    "你的工作流程：\n",
    "1. 先调用 data_overview 工具了解数据集的字段结构与数据质量（如有需要可再用 column_profile 查看关键列）\n",
    "2. 结合字段实际含义，规划 3~5 个有业务价值的分析任务，可覆盖：时间趋势、类别结构、分组对比、相关性分析、异常值检测等维度\n",
    "3. 每个任务都要具体可执行：写明使用哪个字段、做什么统计、需要什么图表（图表类型支持: histogram/bar/box/line/scatter/heatmap）\n",
    "\n",
    "最后，你必须以纯JSON数组作为最终答案输出分析计划（直接输出JSON本身，不要加代码块围栏，不要输出任何解释文字），格式如下:\n",
    "[\n",
    "  {\"task\": \"任务简短标题\", \"goal\": \"具体分析目标，写明字段、统计方式与所需图表\"}\n",
    "]\"\"\"\n",
    "\n",
    "ANALYST_PROMPT = \"\"\"你是一位严谨的数据分析员。针对交给你的每个分析任务：\n",
    "\n",
    "1. 选择合适的工具完成分析（group_aggregate / correlation_analysis / detect_outliers / column_profile / plot_chart）\n",
    "2. 需要展示分布、对比或趋势时，请调用 plot_chart 生成图表（标题用中文）\n",
    "3. 最终用 3~6 句话总结结论，结论中必须引用工具返回的具体数字，不要空泛\n",
    "4. 如果生成了图表，在结论最后单独一行列出图片路径，格式: 图表: charts/xxx.png\n",
    "\n",
    "注意：一个任务通常 1~3 次工具调用即可完成，不要重复调用完全相同的工具。\"\"\"\n",
    "\n",
    "REPORTER_PROMPT = \"\"\"你是一位资深数据分析师，负责把分析结论整理成一份专业的中文数据分析报告（Markdown格式）。\n",
    "\n",
    "报告结构要求：\n",
    "# 报告标题\n",
    "## 一、数据概况\n",
    "## 二、核心发现（每个分析任务一个小节，标题概括发现，正文给出结论与关键数字）\n",
    "## 三、业务建议（基于发现给出 3~5 条可落地的建议）\n",
    "## 四、分析方法说明（简述使用的工具与统计方法）\n",
    "\n",
    "撰写要求：\n",
    "1. 所有结论与数字必须来自输入内容，禁止编造数据\n",
    "2. 若某条结论中带有\"图表: charts/xxx.png\"路径，请在对应小节用Markdown图片语法嵌入: ![图表描述](charts/xxx.png)\n",
    "3. 语言专业、简洁，突出业务洞察\"\"\"\n",
    "\n",
    "planner_agent = ReActAgent(name=\"分析规划师\", llm=llm, tool_registry=planner_registry,\n",
    "                           system_prompt=PLANNER_PROMPT, config=agent_config, max_steps=5)\n",
    "analyst_agent = ReActAgent(name=\"数据分析员\", llm=llm, tool_registry=analysis_registry,\n",
    "                           system_prompt=ANALYST_PROMPT, config=agent_config, max_steps=6)\n",
    "reporter_agent = SimpleAgent(name=\"报告撰写师\", llm=llm, system_prompt=REPORTER_PROMPT,\n",
    "                             config=agent_config)\n",
    "\n",
    "print(\"✅ 三个智能体构建完成：分析规划师(ReAct) / 数据分析员(ReAct) / 报告撰写师(Simple)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1341d72",
   "metadata": {},
   "source": [
    "## 第5部分：三阶段分析流水线\n",
    "\n",
    "`run_pipeline()` 串起三个智能体：**规划 → 逐任务分析 → 汇总报告**，并把报告与图表落盘到 `outputs/` 目录。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc18ad8f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ========================================\n",
    "# 流水线实现\n",
    "# ========================================\n",
    "def parse_tasks(plan_text: str) -> List[Dict[str, str]]:\n",
    "    \"\"\"从规划智能体的输出中解析任务列表（JSON优先，正则兜底）\"\"\"\n",
    "    candidates = []\n",
    "    m = re.search(r\"```(?:json)?\\s*(\\[.*?\\])\\s*```\", plan_text, re.S)\n",
    "    if m:\n",
    "        candidates.append(m.group(1))\n",
    "    # 兜底1: 输出是裸JSON数组（没有代码块围栏）\n",
    "    if not candidates:\n",
    "        i, j = plan_text.find(\"[\"), plan_text.rfind(\"]\")\n",
    "        if 0 <= i < j:\n",
    "            candidates.append(plan_text[i:j + 1])\n",
    "    for cand in candidates:\n",
    "        try:\n",
    "            tasks = json.loads(cand)\n",
    "            return [\n",
    "                {\"task\": str(t.get(\"task\", \"\")).strip() or f\"任务{i}\",\n",
    "                 \"goal\": str(t.get(\"goal\", \"\")).strip()}\n",
    "                for i, t in enumerate(tasks, 1) if isinstance(t, dict)\n",
    "            ]\n",
    "        except json.JSONDecodeError:\n",
    "            continue\n",
    "    # 兜底2: 按\"任务N：描述\"格式的行提取\n",
    "    tasks = []\n",
    "    for line in plan_text.splitlines():\n",
    "        m2 = re.match(r\"\\s*[-*\\d.、)]*\\s*(?:\\*\\*)?任务\\d+[**:：]?\\s*(.+)\", line)\n",
    "        if m2 and len(m2.group(1).strip()) > 4:\n",
    "            tasks.append({\"task\": m2.group(1).strip(), \"goal\": \"\"})\n",
    "    return tasks[:6]\n",
    "\n",
    "\n",
    "def run_pipeline(data_path: str = DATA_PATH,\n",
    "                 report_path: str = os.path.join(OUTPUT_DIR, \"analysis_report.md\")):\n",
    "    \"\"\"三阶段分析流水线：规划 → 逐任务分析 → 汇总报告\"\"\"\n",
    "    print(\"=\" * 60)\n",
    "    print(\"【阶段1/3】分析规划：探查数据并生成分析任务\")\n",
    "    plan_text = planner_agent.run(\n",
    "        f\"请针对数据集 {data_path}（{GLOBAL_DF.shape[0]} 行 × {GLOBAL_DF.shape[1]} 列，字段: \"\n",
    "        f\"{', '.join(GLOBAL_DF.columns)}）制定分析计划。\"\n",
    "    )\n",
    "    tasks = parse_tasks(plan_text)\n",
    "    if not tasks:\n",
    "        tasks = [{\"task\": \"数据整体概览与质量检查\", \"goal\": \"使用 data_overview 了解数据\"}]\n",
    "    print(f\"\\n✅ 规划完成，共 {len(tasks)} 个分析任务:\")\n",
    "    for i, t in enumerate(tasks, 1):\n",
    "        print(f\"   任务{i}: {t['task']}\")\n",
    "\n",
    "    print(\"\\n\" + \"=\" * 60)\n",
    "    print(\"【阶段2/3】逐任务深度分析\")\n",
    "    conclusions = []\n",
    "    for i, t in enumerate(tasks, 1):\n",
    "        print(f\"\\n>>> 执行任务{i}: {t['task']}\")\n",
    "        out = analyst_agent.run(f\"分析任务: {t['task']}\\n分析目标: {t['goal'] or t['task']}\")\n",
    "        conclusions.append({\"task\": t[\"task\"], \"conclusion\": out})\n",
    "        print(f\"✅ 任务{i} 完成\")\n",
    "\n",
    "    print(\"\\n\" + \"=\" * 60)\n",
    "    print(\"【阶段3/3】撰写分析报告\")\n",
    "    chart_files = sorted(glob.glob(os.path.join(CHART_DIR, \"*.png\")))\n",
    "    chart_paths = [os.path.relpath(p, OUTPUT_DIR).replace(\"\\\\\\\\\", \"/\") for p in chart_files]\n",
    "    report_input = (\n",
    "        f\"数据集概况: {GLOBAL_DF.shape[0]} 行 × {GLOBAL_DF.shape[1]} 列，字段: {', '.join(GLOBAL_DF.columns)}\\n\\n\"\n",
    "        f\"已生成的图表文件（相对outputs目录）: {json.dumps(chart_paths, ensure_ascii=False)}\\n\\n\"\n",
    "        f\"各分析任务的结论（JSON）:\\n{json.dumps(conclusions, ensure_ascii=False, indent=2)}\\n\\n\"\n",
    "        f\"请撰写完整的数据分析报告。\"\n",
    "    )\n",
    "    report = reporter_agent.run(report_input)\n",
    "    with open(report_path, \"w\", encoding=\"utf-8\") as f:\n",
    "        f.write(report)\n",
    "    print(f\"\\n✅ 报告已保存: {report_path}\")\n",
    "    print(f\"✅ 共生成图表 {len(chart_paths)} 张，保存于 {CHART_DIR}/\")\n",
    "    return report, tasks, conclusions\n",
    "\n",
    "\n",
    "print(\"✅ 流水线函数定义完成\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66fc3483",
   "metadata": {},
   "source": [
    "## 第6部分：运行完整分析\n",
    "\n",
    "执行三阶段流水线（需要 `.env` 中配置好 LLM API 密钥）。运行结束后：\n",
    "- 报告：`outputs/analysis_report.md`\n",
    "- 图表：`outputs/charts/*.png`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d5699e62",
   "metadata": {},
   "outputs": [],
   "source": [
    "report, tasks, conclusions = run_pipeline()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "01a3593a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 查看生成的分析报告\n",
    "print(report)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fc78b04",
   "metadata": {},
   "source": [
    "## 第7部分：工具自检（不消耗 LLM 调用）\n",
    "\n",
    "直接调用两个核心工具验证工具层工作正常——即使没有配置 API 密钥，也可以运行本单元格体验工具层。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0229e7ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 自检1：分组聚合 —— 各产品类别的销售额贡献\n",
    "print(GroupAggregateTool().run({\"group_col\": \"产品类别\", \"value_col\": \"销售额\"}).text)\n",
    "\n",
    "print(\"\\n\" + \"=\" * 50 + \"\\n\")\n",
    "\n",
    "# 自检2：IQR异常值检测 —— 找出异常大额订单\n",
    "print(OutlierTool().run({\"column\": \"销售额\"}).text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2563b7c",
   "metadata": {},
   "source": [
    "## 第8部分：总结与展望\n",
    "\n",
    "### 实现的功能\n",
    "- ✅ 通用 CSV 数据分析：替换数据文件即可分析新数据集，无需改代码\n",
    "- ✅ 三阶段多智能体流水线：规划（Plan）→ 分析（Execute）→ 报告（Report）\n",
    "- ✅ 6 个原子分析工具：概览 / 列画像 / 相关性 / 分组聚合 / 异常检测 / 绘图\n",
    "- ✅ 自动中文图表生成，并在报告中以相对路径嵌入\n",
    "- ✅ 工具层可独立运行（第7部分自检），便于调试与评审\n",
    "\n",
    "### 遇到的挑战与解决方案\n",
    "- **规划输出不稳定**：LLM 偶尔不按 JSON 输出 → 采用「JSON 优先 + 正则兜底」双解析策略（`parse_tasks`）\n",
    "- **matplotlib 中文乱码**：统一配置 `font.sans-serif` 候选字体链，并处理负号显示\n",
    "- **LLM 传参错误**：所有工具对列名/参数做校验并返回可用列提示，智能体可自行纠错重试\n",
    "\n",
    "### 未来改进方向\n",
    "- [ ] 支持多 Sheet / Excel 与数据库数据源\n",
    "- [ ] 引入 ReflectionAgent 对报告质量自动复审\n",
    "- [ ] 增加 Gradio 交互界面，支持拖拽上传\n",
    "- [ ] 分析结果缓存，避免重复计算"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.12.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}