docs: improve sample proj docs

This commit is contained in:
AmyBei
2024-12-05 14:42:28 +08:00
parent d11839f7d8
commit 338537fb0c
47 changed files with 500 additions and 493 deletions

View File

@@ -1,16 +1,18 @@
# Translation Assistant
Recently, Professor Andrew Ng of Stanford University open-sourced an AI agent for machine translation—an agent that employs a reflective workflow for translation. This project showcases an example of machine translation utilizing the reflective agent workflow. The primary steps of the agent include:
1. Initial translation, utilizing a large model (LLM) to translate text from the source language to the target language;
2. Reflection, wherein the large model identifies deficiencies in the translation and offers constructive improvement suggestions;
3. Revision of the translation, where the large model retranslates based on the suggestions derived from the reflection;
The reflective workflow employed in the project enables the model to identify its own deficiencies and implement targeted revisions, constituting a typical multi-agent collaborative working mechanism. This is also a domain in which agentUniverse has been actively engaged, prompting me to contemplate the migration of Andrew Ng's translation agent to agentUniverse.
This case is based on the Qianwen large model, and prior to use, you need to configure the `DASHSCOPE_API_KEY` in the environment variables.
Recently, Professor Andrew Ng of Stanford University open-sourced an AI agent for machine translation, which employs a reflective workflow for translation process. This project serve as an example of machine translation utilizing the reflective agent workflow. The primary steps of the agent are as follow:
1. Initial translation, where a large model (LLM) is utilized to translate text from the source language to the target language.
2. Reflection, in which the large model identifies deficiencies in the translation and proposes constructive improvement suggestions.
3. Revision of the translation, where the large model retranslates the text based on the suggestions derived from the reflection process.
The reflective workflow employed in this project enables the model to identify its own deficiencies and implement targeted revisions, constituting a typical multi-agent collaborative working mechanism. This is also an area in which agentUniverse has been actively engaged, prompting me to contemplate the migration of Andrew Ng's translation agent to agentUniverse framework.
This case is based on the Qianwen large model. Prior to use, you need to configure the `DASHSCOPE_API_KEY` in the environment variables.
## Collaborative Working Process of Multi-Agents
During the translation process, the priority is to ascertain whether the model's length surpasses the maximum number of tokens it can accommodate. For instances where the length exceeds the limit, the text is initially segmented, followed by the translation of each segment. Nonetheless, the entire translation process adheres to the initial translation -> reflection -> revision workflow.
![Multi-Agent Collaborative Working Process](../../_picture/translation_flow_graph.png)
During the translation process, the priority is to ascertain whether the model's length surpasses the maximum number of tokens it can accommodate. For instances where the length exceeds the limit, the text is initially segmented, followed by the translation of each segment. However, the entire translation process adheres to the initial translation -> reflection -> revision workflow.
![](assets/17332297960881.jpg)
Implementing this translation in aU involves the following steps:
Implementing this translation in agentUniverse involves the following steps:
1. Define prompts related to translation, with three for short text and three for long text. Relevant files include:
- [Short Text Init Prompt](../../../../sample_standard_app/app/core/prompt/translation/translation_init_en.yaml)
- [Short Text Reflection Prompt](../../../../sample_standard_app/app/core/prompt/translation/translation_reflection_en.yaml)
@@ -24,9 +26,12 @@ Implementing this translation in aU involves the following steps:
- [Short Text Translation Improvement Agent](../../../../sample_standard_app/app/core/agent/translation_agent_case/translation_improve_agent.yaml)
These agents switch their prompts based on whether the task involves translating long or short texts. The specific logic for this switching mechanism is detailed in the [agent file](../../../../sample_standard_app/app/core/agent/translation_agent_case/translation_agent.py).
3. Define the collaborative work process of the three agents
The collaborative process is illustrated in the earlier-mentioned flowchart of multi-agent collaboration.
![Collaborative Work](../../_picture/translation_execute_flow.png)
3. Define the collaborative work process of the three agents
The collaborative process is illustrated in the earlier-mentioned flowchart of multi-agent collaboration.
![](assets/17332298612184.jpg)
For a deeper dive into the processes, refer to the [detailed code file](../../../../sample_standard_app/app/core/agent/translation_agent_case/translation_by_token_agent.py).
The [configuration file](../../../../sample_standard_app/app/core/agent/translation_agent_case/translation_agent.yaml) for the collaborative agents.
@@ -39,13 +44,17 @@ The [configuration file](../../../../sample_standard_app/app/core/agent/translat
[Translation Of Long Text](../../../../sample_standard_app/app/test/translation_data/short_text_result.txt)
[Translation Of Short Text](../../../../sample_standard_app/app/test/translation_data/long_text_result.txt)
### 演示结果
We can see that using aU maintains consistency with the original translation_agent project results, successfully replicated.
aU Results:
![aU Long Text Result](../../_picture/long_translation_au.png)
### Demonstration Results
We can see that using agentUniverse maintains consistency with the results of the orginal translation_agent project, which hasa been successfully replicated.
agentUniverse Results:
![](assets/17332298883233.jpg)
translation_agent Results:
![translation Long Text Result](../../_picture/long_translation_wu.png)
![](assets/17332299736651.jpg)
### Others
In reference to Professor Andrew Ng's code, when managing segmented translations, to ensure the coherence of the translation, each segment is translated with the corresponding context. However, incorporating all translation content into the context within his code can undermine the original intent of segmentation, and in certain cases, it can result in an excess of tokens beyond the model's capacity. I have raised a related issue on the corresponding repository address: [Issue Link](https://github.com/andrewyng/translation-agent/issues/28)## Implementation in aU
In reference to Professor Andrew Ng's code, when managing segmented translations, to ensure the coherence of the translation, each segment is translated with its corresponding context. However, incorporating all translation content into the context within his code can undermine the original intent of segmentation, and in certain cases, it can result in exceeding the model's taken capacity. I have raised a related issue on the corresponding repository address: [Issue Link](https://github.com/andrewyng/translation-agent/issues/28)## Implementation in agentUniverse.