可执行模板包，包含：

1. ✅ OpenTelemetry Collector + ClickHouse 部署 YAML
2. ✅ MCP Server 最小可运行 Python 代码 + Dockerfile
3. ✅ LangChain Agent 核心逻辑（FastAPI 封装）
4. ✅ Grafana Dashboard JSON（用于展示 AI 分析结果）

---

1️⃣ OpenTelemetry Collector + ClickHouse 部署（Kubernetes）
clickhouse.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: clickhouse
spec:
  serviceName: clickhouse
  replicas: 1
  selector:
    matchLabels:
      app: clickhouse
  template:
    metadata:
      labels:
        app: clickhouse
    spec:
      containers:
      - name: clickhouse
        image: clickhouse/clickhouse-server:23.8
        ports:
        - containerPort: 9000
          name: tcp
        - containerPort: 8123
          name: http
        volumeMounts:
        - name: data
          mountPath: /var/lib/clickhouse
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 20Gi
---
apiVersion: v1
kind: Service
metadata:
  name: clickhouse
spec:
  ports:
  - port: 9000
    name: tcp
  - port: 8123
    name: http
  selector:
    app: clickhouse
otel-collector.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: otel-collector-config
data:
  config.yaml: |
    receivers:
      otlp:
        protocols:
          grpc: { endpoint: "0.0.0.0:4317" }
          http: { endpoint: "0.0.0.0:4318" }
      prometheus:
        config:
          scrape_configs:
            - job_name: 'k8s-pods'
              kubernetes_sd_configs:
                - role: pod
    exporters:
      clickhouse:
        endpoint: "tcp://clickhouse:9000"
        database: "observability"
        traces_table: "otel_traces"
        metrics_table: "otel_metrics"
        logs_table: "otel_logs"
    processors:
      batch: { timeout: 10s, send_batch_size: 1024 }

    service:
      pipelines:
        traces: { receivers: [otlp], processors: [batch], exporters: [clickhouse] }
        metrics: { receivers: [prometheus, otlp], processors: [batch], exporters: [clickhouse] }
        logs: { receivers: [otlp], processors: [batch], exporters: [clickhouse] }

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: otel-collector
spec:
  replicas: 1
  selector:
    matchLabels:
      app: otel-collector
  template:
    metadata:
      labels:
        app: otel-collector
    spec:
      containers:
      - name: otel-collector
        image: otel/opentelemetry-collector-contrib:latest
        args: ["--config=/conf/config.yaml"]
        ports:
        - containerPort: 4317
        - containerPort: 4318
        volumeMounts:
        - name: config
          mountPath: /conf
      volumes:
      - name: config
        configMap:
        name: otel-collector-config
---
apiVersion: v1
kind: Service
metadata:
  name: otel-collector
spec:
  ports:
  - port: 4317
    name: otlp-grpc
  - port: 4318
    name: otlp-http
  selector:
    app: otel-collector

💡 首次部署后，请手动执行建表 SQL（见下文）。
建表脚本（init-clickhouse.sql）

CREATE DATABASE IF NOT EXISTS observability;
CREATE TABLE observability.otel_metrics (
  timestamp DateTime64(9),
  service_name String,
  metric_name String,
  metric_value Float64,
  attributes Map(String, String)
) ENGINE = MergeTree()
PARTITION BY toYYYYMMDD(timestamp)
ORDER BY (service_name, metric_name, timestamp);

CREATE TABLE observability.otel_logs (
  timestamp DateTime64(9),
  service_name String,
  severity String,
  body String,
  attributes Map(String, String)
) ENGINE = MergeTree()
PARTITION BY toYYYYMMDD(timestamp)
ORDER BY (service_name, timestamp);

– Traces 表略（可根据 OTel Schema 扩展）
执行方式：

kubectl exec -it clickhouse-0 -- clickhouse-client --multiquery < init-clickhouse.sql

---

2️⃣ MCP Server（最小可运行版）
mcp_server.py

from mcp.server import Server
from mcp.types import Tool
from clickhouse_driver import Client
import json

初始化 ClickHouse 客户端

client = Client(host='clickhouse')

server = Server()

@server.tool()
def query_error_logs(service_name: str, minutes: int = 10) -> str:
    """查询指定服务近 N 分钟的错误日志"""
    query = """
    SELECT timestamp, body FROM observability.otel_logs
    WHERE service_name = %(svc)s AND severity IN ('ERROR', 'FATAL')
      AND timestamp > now() - INTERVAL %(min)s MINUTE
    ORDER BY timestamp DESC LIMIT 50
    """
    rows = client.execute(query, {'svc': service_name, 'min': minutes})
    return json.dumps([{'time': str(r[0]), 'log': r[1]} for r in rows])

@server.tool()
def query_metric(service_name: str, metric_name: str, minutes: int = 10) -> str:
    """查询指定服务的指标值"""
    query = """
    SELECT timestamp, metric_value FROM observability.otel_metrics
    WHERE service_name = %(svc)s AND metric_name = %(metric)s
      AND timestamp > now() - INTERVAL %(min)s MINUTE
    ORDER BY timestamp DESC LIMIT 100
    """
    rows = client.execute(query, {'svc': service_name, 'metric': metric_name, 'min': minutes})
    return json.dumps([{'time': str(r[0]), 'value': r[1]} for r in rows])

if __name__ == "__main__":
    import asyncio
    asyncio.run(server.run_stdin_stdout())

requirements.txt

mcp-server==0.1.0
clickhouse-driver==0.2.6

Dockerfile

FROM python:3.10-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY mcp_server.py .
CMD ["python", "mcp_server.py"]

构建 & 推送：

docker build -t your-registry/mcp-server:v1 .
docker push your-registry/mcp-server:v1

---

3️⃣ LangChain Agent（FastAPI 封装）
agent_service.py

from fastapi import FastAPI, HTTPException
from langchain_anthropic import ChatAnthropic
from langchain_mcp import McpToolNode
from langchain_core.messages import HumanMessage
import subprocess
import os

app = FastAPI()

启动 MCP Server 子进程（或连接远程）

mcp_process = subprocess.Popen(
    ["python", "mcp_server.py"],
    stdin=subprocess.PIPE,
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE
)

构建工具节点

tool_node = McpToolNode.from_subprocess(mcp_process)

初始化 LLM（替换 YOUR_API_KEY）

llm = ChatAnthropic(model="claude-3-haiku-20240307", api_key=os.getenv("ANTHROPIC_API_KEY"))

@app.post("/analyze")
def analyze(alert: dict):
    try:
        prompt = f"""
        你是一个资深 SRE。当前告警：服务 {alert['service']} 出现 {alert['alertname']}。
        请调用工具分析根因。如果数据不足，明确说“无法确定”。
        按以下格式回答：
        - 根因推测：<...>
        - 证据：<...>
        - 置信度：<高/中/低>
        """
        messages = [HumanMessage(content=prompt)]
        response = llm.bind_tools(tool_node.get_tools()).invoke(messages)
        
        # 执行工具调用（简化版，实际需循环）
        tool_response = tool_node.invoke(response.tool_calls)
        final_answer = llm.invoke([response, tool_response])
        
        return {"result": final_answer.content}
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

🔔 注意：生产环境建议将 MCP Server 独立部署，而非子进程。

---

4️⃣ Grafana Dashboard（AI 分析结果展示）
导入以下 JSON 到 Grafana（支持 Annotations 显示 AI 结论）：
aiops-dashboard.json（片段，完整版可扩展）

{
  "dashboard": {
    "id": null,
    "title": "AIOps Root Cause Analysis",
    "panels": [
      {
        "type": "text",
        "title": "Latest AI Diagnosis",
        "options": {
          "content": "${ai_result}",
          "mode": "markdown"
        }
      },
      {
        "type": "timeseries",
        "title": "Error Rate",
        "targets": [
          {
            "datasource": "ClickHouse",
            "rawSql": "SELECT timestamp, metric_value FROM observability.otel_metrics WHERE metric_name = 'http_server_errors' AND service_name = '$service'"
          }
        ]
      }
    ],
    "templating": {
      "list": [
        {
          "name": "service",
          "type": "query",
          "query": "SELECT DISTINCT service_name FROM observability.otel_metrics"
        }
      ]
    }
  }
}

💡 提示：可通过 Alertmanager webhook 调用 Grafana API 添加 Annotation：

curl -X POST http://grafana/api/annotations -H "Content-Type: application/json" -d '{"dashboardId":1,"text":"AI: DB connection pool exhausted","time":1700000000000}'

---

📦 使用说明

1. 部署顺序：

kubectl apply -f clickhouse.yaml
kubectl apply -f otel-collector.yaml

手动建表

kubectl apply -f mcp-server-deployment.yaml  # （基于上述 Dockerfile）
kubectl apply -f agent-service.yaml
1. 环境变量：
  - ANTHROPIC_API_KEY（或替换为 Qwen/OpenAI）
  - CLICKHOUSE_HOST=clickhouse
2. 测试流程：
curl -X POST http://agent-service/analyze -H "Content-Type: application/json" -d '{"service":"order-api","alertname":"HighLatency"}'