Merge pull request #104 from alipay/dev_request_tool

requst tool

docs/guidebook/en/2_2_3_Integrated_LangChain_Tools.md

@@ -0,0 +1,83 @@
+# Integrated LangChain Tool
+
+Based on the level of difficulty in initializing tool objects in LangChain, they can be divided into two categories:
+The first category involves simple initialization, where initialization can be completed with basic parameter configuration.
+The second category involves complex initialization with intricate internal objects that need to be set up.
+For the first category of tools, you can use configuration files in aU to directly perform initialization, such as initializing the DuDuckGo search tool.
+For the second category of tools, we have implemented a LangChainTool base class. You only need to implement the init_langchain_tool method of this class to initialize the corresponding LangChain tool objects, with reference to the initialization method of Wikipedia.
+
+Note: If you want to directly use the description from LangChain, the description in the configuration file must be set to empty.
+
+An Example of Tool Initialization:
+[Tool Address](../../../sample_standard_app/app/core/tool/langchain_tool/human_input_run.yaml)
+```yaml
+name: 'human_input_run'
+description: ''
+tool_type: 'api'
+input_keys: ['input']
+langchain:
+  module: langchain_community.tools
+  class_name: HumanInputRun
+metadata:
+  type: 'TOOL'
+  module: 'sample_standard_app.app.core.tool.langchain_tool'
+  class: 'LangChainTool'
+```
+Parameter Description:
+    langchain: The LangChain tool you intend to use, which requires the configuration of module and class_name.
+    langchain.module: The name of the LangChain module, e.g., langchain_community.tools.
+    langchain.class_name: The name of the LangChain class, e.g., HumanInputRun.
+    langchain.init_params: The initialization parameters for LangChain, such as:
+        ```yaml
+        langchain:
+          module: langchain_community.tools
+          class_name: HumanInputRun
+          init_params:
+            prompt: 'please Input your question'
+        ```
+    If you completely override the `init_langchain_tool` method, then you do not need to configure this part.
+
+## 1. Integrate the DuckDuckGo Tool from LangChain
+[Tool Address](../../../sample_standard_app/app/core/tool/langchain_tool/duckduckgo_search.yaml)
+```yaml
+name: 'duckduckgo_search'
+description: 'DuckDuckGo Search tool'
+tool_type: 'api'
+input_keys: ['input']
+langchain:
+  module: langchain.tools
+  class_name: DuckDuckGoSearchResults
+  init_params:
+    backend: news
+metadata:
+  type: 'TOOL'
+  module: 'sample_standard_app.app.core.tool.langchain_tool.langchain_tool'
+  class: 'LangChainTool'
+```
+This tool can be used directly without any keys.
+
+## 2.Integrate Wikipedia Search
+Since the definition of LangChain includes an api_wrapper object, define the object file and override the initialization method:
+```python
+from langchain_community.tools import WikipediaQueryRun
+from langchain_community.utilities import WikipediaAPIWrapper
+
+from sample_standard_app.app.core.tool.langchain_tool.langchain_tool import LangChainTool
+
+
+class WikipediaTool(LangChainTool):
+    def init_langchain_tool(self, component_configer):
+        wrapper = WikipediaAPIWrapper()
+        return WikipediaQueryRun(api_wrapper=wrapper)
+```
+Define Configuration:
+```yaml
+name: 'wikipedia_query'
+description: ''
+tool_type: 'api'
+input_keys: ['input']
+metadata:
+  type: 'TOOL'
+  module: 'sample_standard_app.app.core.tool.langchain_tool.wikipedia_query'
+  class: 'WikipediaTool'
+```

docs/guidebook/en/2_2_3_Integrated_Tools.md

@@ -104,7 +104,7 @@ SEARCHAPI_API_KEY="xxxxxx"
 ```
 
 
-## 2. 代码工具
+## 2. Code Tool
 
 ### 2.1 PythonRepl
 [Tool address](../../../sample_standard_app/app/core/tool/python_repl_tool.yaml)  
@@ -134,5 +134,56 @@ metadata:
 This tool can be used directly without any key, but for system security, please do not use this tool in production environments.
 
 
+## 3.HTTP Tool
 
+### 3.1 HTTP GET
+[Tool address](../../../sample_standard_app/app/core/tool/request_get_tool.yaml)
+The tool can send a GET request, with its configuration information being:
+```yaml
+name: 'requests_get'
+description: 'A portal to the internet. Use this when you need to get specific
+    content from a website. Input should be a  url (i.e. https://www.google.com).
+    The output will be the text response of the GET request.
+        ```'
+headers:
+  content-type: 'application/json'
+method: 'POST'
+json_parser: true
+response_content_type: json
+tool_type: 'api'
+input_keys: ['input']
+metadata:
+  type: 'TOOL'
+  module: 'sample_standard_app.app.core.tool.request_tool'
+  class: 'RequestTool'
+```
+Configuration to Refer to When Sending a POST Request：
+```yaml
+name: 'requests_post'
+# description copy from langchain RequestPOSTTool
+description: 'Use this when you want to POST to a website.
+    Input should be a json string with two keys: "url" and "data".
+    The value of "url" should be a string, and the value of "data" should be a dictionary of 
+    key-value pairs you want to POST to the url.
+    Be careful to always use double quotes for strings in the json string
+    The output will be the text response of the POST request.
+        ```'
+headers:
+  content-type: 'application/json'
+method: 'POST'
+json_parser: true
+response_content_type: json
+tool_type: 'api'
+input_keys: ['input']
+metadata:
+  type: 'TOOL'
+  module: 'sample_standard_app.app.core.tool.request_tool'
+  class: 'RequestTool'
+```
+Parameter Description:
+    method: The method of the request, such as GET, POST, PUT, etc.
+    headers: The HTTP headers required for sending the request.
+    json_parse: Specifies whether the input parameters need to be parsed by HTTP. It should be set to True for POST requests and False for GET requests.
+    response_content_type: The parsing method for the HTTP request result. If set to 'json', the result will be returned in JSON format; if set to 'text', the result will be returned as text.
+This tool can be used directly without any keys required.
 

docs/guidebook/zh/2_2_3_集成LangChain工具.md

@@ -0,0 +1,84 @@
+# 集成LangChain工具
+
+根据langchain中工具对象的初始化的难易程度，可以将其分为两类：
+第一类，简单初始化，只需要简单的参数配置即可完成初始化。
+第二类，复杂初始化，内部包含一些复杂的对象需要进行初始化。
+对于一类工具，你可以在aU中直接使用配置文件进行初始化，如DuDuckGo搜索工具的初始化。
+对于第二类工具，我们实现了一个LangChainTool基础类，你只需要实现该类的init_langchain_tool方法，初始化对应的langchain工具对象即可，参考维基百科的初始化方法。
+
+注意，如果你想直接使用LangChain中的description，在配置文件中description必须要配置为空
+
+一个工具初始化示例：
+[工具地址](../../../sample_standard_app/app/core/tool/langchain_tool/human_input_run.yaml)
+```yaml
+name: 'human_input_run'
+description: ''
+tool_type: 'api'
+input_keys: ['input']
+langchain:
+  module: langchain_community.tools
+  class_name: HumanInputRun
+metadata:
+  type: 'TOOL'
+  module: 'sample_standard_app.app.core.tool.langchain_tool'
+  class: 'LangChainTool'
+```
+参数说明：
+    langchain: 你打算使用的langchain工具，需要配置module和class_name
+    langchain.module: langchain的模块名，例如langchain_community.tools
+    langchain.class_name: langchain的类名，例如HumanInputRun
+    langchain.init_params： langchain的初始化参数，例如：
+        ```yaml
+        langchain:
+          module: langchain_community.tools
+          class_name: HumanInputRun
+          init_params:
+            prompt: '请输入你的问题'
+        ```
+    如果需要使用你完全重写了init_langchain_tool方法，那么你不需要配置该部分
+该工具可以直接使用，无需任何keys
+
+## 1. 集成LangChain中的DuckDuckGo工具
+[工具地址](../../../sample_standard_app/app/core/tool/langchain_tool/duckduckgo_search.yaml)
+```yaml
+name: 'duckduckgo_search'
+description: 'DuckDuckGo Search tool'
+tool_type: 'api'
+input_keys: ['input']
+langchain:
+  module: langchain.tools
+  class_name: DuckDuckGoSearchResults
+  init_params:
+    backend: news
+metadata:
+  type: 'TOOL'
+  module: 'sample_standard_app.app.core.tool.langchain_tool.langchain_tool'
+  class: 'LangChainTool'
+```
+该工具可以直接使用，无需任何keys
+
+## 2.集成维基百科搜索
+因为LangChain的定义当中，包含一个api_wrapper对象，因此定义对象文件，并重写初始化方法：
+```python
+from langchain_community.tools import WikipediaQueryRun
+from langchain_community.utilities import WikipediaAPIWrapper
+
+from sample_standard_app.app.core.tool.langchain_tool.langchain_tool import LangChainTool
+
+
+class WikipediaTool(LangChainTool):
+    def init_langchain_tool(self, component_configer):
+        wrapper = WikipediaAPIWrapper()
+        return WikipediaQueryRun(api_wrapper=wrapper)
+```
+定义配置
+```yaml
+name: 'wikipedia_query'
+description: ''
+tool_type: 'api'
+input_keys: ['input']
+metadata:
+  type: 'TOOL'
+  module: 'sample_standard_app.app.core.tool.langchain_tool.wikipedia_query'
+  class: 'WikipediaTool'
+```

docs/guidebook/zh/2_2_3_集成的工具.md

@@ -135,6 +135,54 @@ metadata:
 
 该工具可以直接使用，无需任何key，但是为了系统安全，请不要在生产环境使用该工具
 
-
-
-
+## 3.HTTP 工具
+[工具地址](../../../sample_standard_app/app/core/tool/request_get_tool.yaml)
+该工具可以发送一个GET请求，工具的配置信息 :  
+```yaml
+name: 'requests_get'
+# description copy from langchain RequestGetTool
+description: 'A portal to the internet. Use this when you need to get specific
+    content from a website. Input should be a  url (i.e. https://www.google.com).
+    The output will be the text response of the GET request.
+        ```'
+headers:
+  content-type: 'application/json'
+method: 'POST'
+json_parser: false
+response_content_type: json
+tool_type: 'api'
+input_keys: ['input']
+metadata:
+  type: 'TOOL'
+  module: 'sample_standard_app.app.core.tool.request_tool'
+  class: 'RequestTool'
+```
+发送POST请求时可以参考的配置：
+```yaml
+name: 'requests_post'
+# description copy from langchain RequestPOSTTool
+description: 'Use this when you want to POST to a website.
+    Input should be a json string with two keys: "url" and "data".
+    The value of "url" should be a string, and the value of "data" should be a dictionary of 
+    key-value pairs you want to POST to the url.
+    Be careful to always use double quotes for strings in the json string
+    The output will be the text response of the POST request.
+        ```'
+headers:
+  content-type: 'application/json'
+method: 'POST'
+json_parser: true
+response_content_type: json
+tool_type: 'api'
+input_keys: ['input']
+metadata:
+  type: 'TOOL'
+  module: 'sample_standard_app.app.core.tool.request_tool'
+  class: 'RequestTool'
+```
+参数说明：
+    method 请求的方式GET/POST/PUT等
+    headers 发送请求需要使用的 http的header,
+    json_parse 输入参数是否需要是要HTTP解析，POST请求时需要设置为True,GET请求需要设置为False
+    response_content_type http请求结果的解析方式，设置为json时，会返回json结果，设置为text时会返回text结果
+该工具可以直接使用，无需任何keys

pyproject.toml

@@ -54,6 +54,8 @@ langchain-anthropic = '^0.1.13'
 numpy = '^1.26.0'
 pandas = "^2.2.2"
 pyarrow = "^16.1.0"
+duckduckgo-search = "^6.1.7"
+wikipedia= "^1.4.0"
 
 [tool.poetry.extras]
 log_ext = ["aliyun-log-python-sdk"]

sample_standard_app/app/core/tool/langchain_tool/duckduckgo_search.yaml

@@ -0,0 +1,13 @@
+name: 'duckduckgo_search'
+description: ''
+tool_type: 'api'
+input_keys: ['input']
+langchain:
+  module: langchain.tools
+  class_name: DuckDuckGoSearchResults
+  init_params:
+    backend: news
+metadata:
+  type: 'TOOL'
+  module: 'sample_standard_app.app.core.tool.langchain_tool.langchain_tool'
+  class: 'LangChainTool'

sample_standard_app/app/core/tool/langchain_tool/human_input_run.yaml

@@ -0,0 +1,11 @@
+name: 'human_input_run'
+description: ''
+tool_type: 'api'
+input_keys: ['input']
+langchain:
+  module: langchain_community.tools
+  class_name: HumanInputRun
+metadata:
+  type: 'TOOL'
+  module: 'sample_standard_app.app.core.tool.langchain_tool.langchain_tool'
+  class: 'LangChainTool'

sample_standard_app/app/core/tool/langchain_tool/langchain_tool.py

@@ -0,0 +1,50 @@
+# !/usr/bin/env python3
+# -*- coding:utf-8 -*-
+import importlib
+import json
+# @Time    : 2024/6/24 11:42
+# @Author  : weizjajj 
+# @Email   : weizhongjie.wzj@antgroup.com
+# @FileName: langchain_tool.py
+
+from typing import Optional, Type
+
+from langchain_community.tools import DuckDuckGoSearchResults
+from langchain_core.tools import BaseTool
+
+from agentuniverse.agent.action.tool.tool import Tool, ToolInput
+from agentuniverse.base.config.component_configer.configers.tool_configer import ToolConfiger
+
+
+class LangChainTool(Tool):
+    name: Optional[str] = ""
+    description: Optional[str] = ""
+    tool: Optional[BaseTool] = None
+
+    def execute(self, tool_input: ToolInput):
+        input = tool_input.get_data("input")
+        callbacks = tool_input.get_data("callbacks", None)
+        return self.tool.run(input, callbacks=callbacks)
+
+    def initialize_by_component_configer(self, component_configer: ToolConfiger) -> 'Tool':
+        super().initialize_by_component_configer(component_configer)
+        self.tool = self.init_langchain_tool(component_configer)
+        if not component_configer.description:
+            self.description = self.tool.description
+        return self
+
+    def init_langchain_tool(self, component_configer):
+        langchain_info = component_configer.configer.value.get('langchain')
+        module = langchain_info.get("module")
+        class_name = langchain_info.get("class_name")
+        module = importlib.import_module(module)
+        clz = getattr(module, class_name)
+        init_params = langchain_info.get("init_params")
+        self.get_langchain_tool(init_params, clz)
+        return self.tool
+
+    def get_langchain_tool(self, init_params: dict, clz: Type[BaseTool]):
+        if init_params:
+            self.tool = clz(**init_params)
+        else:
+            self.tool = clz()

sample_standard_app/app/core/tool/langchain_tool/wikipedia_query.py

@@ -0,0 +1,19 @@
+# !/usr/bin/env python3
+# -*- coding:utf-8 -*-
+
+# @Time    : 2024/6/27 17:38
+# @Author  : weizjajj 
+# @Email   : weizhongjie.wzj@antgroup.com
+# @FileName: wikipedia_query.py
+
+
+from langchain_community.tools import WikipediaQueryRun
+from langchain_community.utilities import WikipediaAPIWrapper
+
+from sample_standard_app.app.core.tool.langchain_tool.langchain_tool import LangChainTool
+
+
+class WikipediaTool(LangChainTool):
+    def init_langchain_tool(self, component_configer):
+        wrapper = WikipediaAPIWrapper()
+        return WikipediaQueryRun(api_wrapper=wrapper)

sample_standard_app/app/core/tool/langchain_tool/wikipedia_query.yaml

@@ -0,0 +1,8 @@
+name: 'wikipedia_query'
+description: ''
+tool_type: 'api'
+input_keys: ['input']
+metadata:
+  type: 'TOOL'
+  module: 'sample_standard_app.app.core.tool.langchain_tool.wikipedia_query'
+  class: 'WikipediaTool'

sample_standard_app/app/core/tool/request_get_tool.yaml

@@ -0,0 +1,16 @@
+name: 'requests_get'
+description: 'A portal to the internet. Use this when you need to get specific
+    content from a website. Input should be a  url (i.e. https://www.google.com).
+    The output will be the text response of the GET request.
+        ```'
+headers:
+  content-type: 'application/json'
+method: 'GET'
+json_parser: false
+response_content_type: json
+tool_type: 'api'
+input_keys: ['input']
+metadata:
+  type: 'TOOL'
+  module: 'sample_standard_app.app.core.tool.request_tool'
+  class: 'RequestTool'

sample_standard_app/app/core/tool/request_post_tool.yaml

@@ -0,0 +1,19 @@
+name: 'requests_post'
+description: 'Use this when you want to POST to a website.
+    Input should be a json string with two keys: "url" and "data".
+    The value of "url" should be a string, and the value of "data" should be a dictionary of 
+    key-value pairs you want to POST to the url.
+    Be careful to always use double quotes for strings in the json string
+    The output will be the text response of the POST request.
+        ```'
+headers:
+  content-type: 'application/json'
+method: 'POST'
+json_parser: true
+response_content_type: json
+tool_type: 'api'
+input_keys: ['input']
+metadata:
+  type: 'TOOL'
+  module: 'sample_standard_app.app.core.tool.request_tool'
+  class: 'RequestTool'

sample_standard_app/app/core/tool/request_tool.py

@@ -0,0 +1,69 @@
+# !/usr/bin/env python3
+# -*- coding:utf-8 -*-
+
+# @Time    : 2024/6/24 10:19
+# @Author  : weizjajj 
+# @Email   : weizhongjie.wzj@antgroup.com
+# @FileName: request_tool.py
+
+
+from typing import Optional
+
+from langchain_community.utilities.requests import GenericRequestsWrapper
+from langchain_core.utils.json import parse_json_markdown
+
+from agentuniverse.agent.action.tool.tool import Tool, ToolInput
+from agentuniverse.base.config.component_configer.configers.tool_configer import ToolConfiger
+from agentuniverse.base.util.logging.logging_util import LOGGER
+
+
+class RequestTool(Tool):
+    method:Optional[str] = 'GET'
+    headers: Optional[dict]= {}
+    response_content_type:Optional[str] = 'text'
+    requests_wrapper: Optional[GenericRequestsWrapper] = None
+    json_parser: Optional[bool] = False
+
+    @staticmethod
+    def _clean_url(url: str) -> str:
+        """Strips quotes from the url."""
+        return url.strip("\"'")
+
+    def execute(self, tool_input: ToolInput):
+        input_params: str = tool_input.get_data('input')
+        if self.json_parser:
+            try:
+                parse_data = parse_json_markdown(input_params)
+                return self.execute_by_method(**parse_data)
+            except Exception as e:
+                LOGGER.error(f'execute request error input{input_params} error{e}')
+                return str(e)
+        else:
+            return self.execute_by_method(input_params)
+
+    def execute_by_method(self, url: str, data: dict = None, **kwargs):
+        url = self._clean_url(url)
+        if self.method == 'GET':
+            return self.requests_wrapper.get(url)
+        elif self.method == 'POST':
+            return self.requests_wrapper.post(url, data=data)
+        elif self.method == 'PUT':
+            return self.requests_wrapper.put(url, data=data)
+        elif self.method == 'DELETE':
+            return self.requests_wrapper.delete(url)
+        else:
+            raise ValueError(f"Unsupported method: {self.method}")
+
+    def initialize_by_component_configer(self, component_configer: ToolConfiger) -> 'Tool':
+        """
+        :param component_configer:
+        :return:
+        """
+        self.headers = component_configer.configer.value.get('headers')
+        self.method = component_configer.configer.value.get('method')
+        self.response_content_type = component_configer.configer.value.get('response_content_type')
+        self.requests_wrapper = GenericRequestsWrapper(
+            headers=self.headers,
+            response_content_type=self.response_content_type
+        )
+        return super().initialize_by_component_configer(component_configer)