1.前言

​ 在大模型项目落地过程中，由于安全、合规和管控等原因不会直接使用原生开源大模型，一般会在开源大模型的基础上添加额外的认证信息或者修改接口访问方式。本次的目标是通过修改 dify源码能够对接不遵守标准 openai接口协议，在 header里面添加了额外认证信息的开源大模型。

环境信息：dify-0.8.3, docker-21

2.定位对接大模型代码

dify对接开源大模型的代码在 openai_api_compatible模块中，具体的路径是 api\core\model_runtime\model_providers\openai_api_compatible，工程结构如下：

其中 openai_api_compatible.yaml可以配置界面上填写的连接信息，如下：

provider: openai_api_compatible
label:
  en_US: OpenAI-API-compatible
description:
  en_US: Model providers compatible with OpenAI's API standard, such as LM Studio.
  zh_Hans: 兼容 OpenAI API 的模型供应商，例如 LM Studio 。
supported_model_types:
  - llm
  - text-embedding
  - speech2text
configurate_methods:
  - customizable-model
model_credential_schema:
  model:
    label:
      en_US: Model Name
      zh_Hans: 模型名称
    placeholder:
      en_US: Enter full model name
      zh_Hans: 输入模型全称
  credential_form_schemas:
    - variable: capability_name
      label:
        zh_Hans: Capability Name
        en_US: Capability Name
      type: text-input
      required: true
      placeholder:
        zh_Hans: Capability Name
        en_US: Capability Name
    - variable: app_key
      label:
        zh_Hans: APP Key
        en_US: APP Key
      type: text-input
      required: true
      placeholder:
        zh_Hans: APP Key
        en_US: APP Key
...

llm.py则是连接大模型和对话的核心代码，本次也是修改此文件。

3.修改大模型验证连接代码

3.1修改添加模型页面

原始页面：

修改后页面：

修改 openai_api_compatible.yaml文件，如下：

...
  credential_form_schemas:
    - variable: capability_name
      label:
        zh_Hans: Capability Name
        en_US: Capability Name
      type: text-input
      required: true
      placeholder:
        zh_Hans: Capability Name
        en_US: Capability Name
    - variable: app_key
      label:
        zh_Hans: APP Key
        en_US: APP Key
      type: text-input
      required: true
      placeholder:
        zh_Hans: APP Key
        en_US: APP Key
    - variable: checksum_key
      label:
        zh_Hans: Checksum 校验密钥
        en_US: Checksum validation key
      type: text-input
      required: true
      placeholder:
        zh_Hans: Checksum key
        en_US: Checksum key
...

text-input是文本输入框，required: true设置必填。

3.2修改模型验证后台代码

本次修改的内容是往 header里面添加额外的认证信息，需要修改 llm.py文件，需要修改的地方如下：

# 生成调用XX大模型需要的 header
def gen_xx_llm_headers(self,credentials):
		# 添加生成 header的逻辑
        ...
        
        return {
            "X-CurTime": cur_time,
            "appKey": app_key,
            "X-Server-Param": server_param,
            "X-CheckSum": checksum,
        }


def validate_credentials(self, model: str, credentials: dict) -> None:
        """
        Validate model credentials using requests to ensure compatibility with all providers following
         OpenAI's API standard.

        :param model: model name
        :param credentials: model credentials
        :return:
        """
        try:
            headers = {
                "Content-Type": "application/json",
                **self.gen_xx_llm_headers(credentials)
                }

            api_key = credentials.get("api_key")
            if api_key:
                headers["Authorization"] = f"Bearer {api_key}"
...

4.修改大模型对话接口代码

也是修改 llm.py文件，需要修改的地方如下：

    # validate_credentials method has been rewritten to use the requests library for compatibility with all providers
    # following OpenAI's API standard.
    def _generate(
        self,
        model: str,
        credentials: dict,
        prompt_messages: list[PromptMessage],
        model_parameters: dict,
        tools: Optional[list[PromptMessageTool]] = None,
        stop: Optional[list[str]] = None,
        stream: bool = True,
        user: Optional[str] = None,
    ) -> Union[LLMResult, Generator]:
        """
        Invoke llm completion model

        :param model: model name
        :param credentials: credentials
        :param prompt_messages: prompt messages
        :param model_parameters: model parameters
        :param stop: stop words
        :param stream: is stream response
        :param user: unique user id
        :return: full response or stream response chunk generator result
        """
        logger.info(f"start call llm, credentials: {credentials}")

        headers = {
            "Content-Type": "application/json",
            "Accept-Charset": "utf-8",
            **self.gen_xx_llm_headers(credentials),
        }
        extra_headers = credentials.get("extra_headers")
        if extra_headers is not None:
            headers = {
                **headers,
                **extra_headers,
            }
...

5.总结

本次源码改造只涉及连接大模型时的 header，并不复杂，仔细阅读 dify的源码，理解流程就能修改。