mirror of
https://github.com/openclaw/openclaw.git
synced 2026-02-09 05:19:32 +08:00
chore: apply formatter
This commit is contained in:
committed by
Vignesh
parent
edd6289f26
commit
9bef525944
@@ -4,6 +4,7 @@ read_when:
|
||||
- You want the memory file layout and workflow
|
||||
- You want to tune the automatic pre-compaction memory flush
|
||||
---
|
||||
|
||||
# Memory
|
||||
|
||||
Moltbot memory is **plain Markdown in the agent workspace**. The files are the
|
||||
@@ -38,7 +39,7 @@ These files live under the workspace (`agents.defaults.workspace`, default
|
||||
|
||||
When a session is **close to auto-compaction**, Moltbot triggers a **silent,
|
||||
agentic turn** that reminds the model to write durable memory **before** the
|
||||
context is compacted. The default prompts explicitly say the model *may reply*,
|
||||
context is compacted. The default prompts explicitly say the model _may reply_,
|
||||
but usually `NO_REPLY` is the correct response so the user never sees this turn.
|
||||
|
||||
This is controlled by `agents.defaults.compaction.memoryFlush`:
|
||||
@@ -53,15 +54,16 @@ This is controlled by `agents.defaults.compaction.memoryFlush`:
|
||||
enabled: true,
|
||||
softThresholdTokens: 4000,
|
||||
systemPrompt: "Session nearing compaction. Store durable memories now.",
|
||||
prompt: "Write any lasting notes to memory/YYYY-MM-DD.md; reply with NO_REPLY if nothing to store."
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
prompt: "Write any lasting notes to memory/YYYY-MM-DD.md; reply with NO_REPLY if nothing to store.",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Details:
|
||||
|
||||
- **Soft threshold**: flush triggers when the session token estimate crosses
|
||||
`contextWindow - reserveTokensFloor - softThresholdTokens`.
|
||||
- **Silent** by default: prompts include `NO_REPLY` so nothing is delivered.
|
||||
@@ -79,6 +81,7 @@ Moltbot can build a small vector index over `MEMORY.md` and `memory/*.md` so
|
||||
semantic queries can find related notes even when wording differs.
|
||||
|
||||
Defaults:
|
||||
|
||||
- Enabled by default.
|
||||
- Watches memory files for changes (debounced).
|
||||
- Uses remote embeddings by default. If `memorySearch.provider` is not set, Moltbot auto-selects:
|
||||
@@ -258,6 +261,7 @@ agents: {
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- `remote.baseUrl` is optional (defaults to the Gemini API base URL).
|
||||
- `remote.headers` lets you add extra headers if needed.
|
||||
- Default model: `gemini-embedding-001`.
|
||||
@@ -285,10 +289,12 @@ If you don't want to set an API key, use `memorySearch.provider = "local"` or se
|
||||
`memorySearch.fallback = "none"`.
|
||||
|
||||
Fallbacks:
|
||||
|
||||
- `memorySearch.fallback` can be `openai`, `gemini`, `local`, or `none`.
|
||||
- The fallback provider is only used when the primary embedding provider fails.
|
||||
|
||||
Batch indexing (OpenAI + Gemini):
|
||||
|
||||
- Enabled by default for OpenAI and Gemini embeddings. Set `agents.defaults.memorySearch.remote.batch.enabled = false` to disable.
|
||||
- Default behavior waits for batch completion; tune `remote.batch.wait`, `remote.batch.pollIntervalMs`, and `remote.batch.timeoutMinutes` if needed.
|
||||
- Set `remote.batch.concurrency` to control how many batch jobs we submit in parallel (default: 2).
|
||||
@@ -296,6 +302,7 @@ Batch indexing (OpenAI + Gemini):
|
||||
- Gemini batch jobs use the async embeddings batch endpoint and require Gemini Batch API availability.
|
||||
|
||||
Why OpenAI batch is fast + cheap:
|
||||
|
||||
- For large backfills, OpenAI is typically the fastest option we support because we can submit many embedding requests in a single batch job and let OpenAI process them asynchronously.
|
||||
- OpenAI offers discounted pricing for Batch API workloads, so large indexing runs are usually cheaper than sending the same requests synchronously.
|
||||
- See the OpenAI Batch API docs and pricing for details:
|
||||
@@ -321,10 +328,12 @@ agents: {
|
||||
```
|
||||
|
||||
Tools:
|
||||
|
||||
- `memory_search` — returns snippets with file + line ranges.
|
||||
- `memory_get` — read memory file content by path.
|
||||
|
||||
Local mode:
|
||||
|
||||
- Set `agents.defaults.memorySearch.provider = "local"`.
|
||||
- Provide `agents.defaults.memorySearch.local.modelPath` (GGUF or `hf:` URI).
|
||||
- Optional: set `agents.defaults.memorySearch.fallback = "none"` to avoid remote fallback.
|
||||
@@ -345,6 +354,7 @@ Local mode:
|
||||
### Hybrid search (BM25 + vector)
|
||||
|
||||
When enabled, Moltbot combines:
|
||||
|
||||
- **Vector similarity** (semantic match, wording can differ)
|
||||
- **BM25 keyword relevance** (exact tokens like IDs, env vars, code symbols)
|
||||
|
||||
@@ -353,10 +363,12 @@ If full-text search is unavailable on your platform, Moltbot falls back to vecto
|
||||
#### Why hybrid?
|
||||
|
||||
Vector search is great at “this means the same thing”:
|
||||
|
||||
- “Mac Studio gateway host” vs “the machine running the gateway”
|
||||
- “debounce file updates” vs “avoid indexing on every write”
|
||||
|
||||
But it can be weak at exact, high-signal tokens:
|
||||
|
||||
- IDs (`a828e60`, `b3b9895a…`)
|
||||
- code symbols (`memorySearch.query.hybrid`)
|
||||
- error strings (“sqlite-vec unavailable”)
|
||||
@@ -369,17 +381,21 @@ good results for both “natural language” queries and “needle in a haystack
|
||||
|
||||
Implementation sketch:
|
||||
|
||||
1) Retrieve a candidate pool from both sides:
|
||||
1. Retrieve a candidate pool from both sides:
|
||||
|
||||
- **Vector**: top `maxResults * candidateMultiplier` by cosine similarity.
|
||||
- **BM25**: top `maxResults * candidateMultiplier` by FTS5 BM25 rank (lower is better).
|
||||
|
||||
2) Convert BM25 rank into a 0..1-ish score:
|
||||
2. Convert BM25 rank into a 0..1-ish score:
|
||||
|
||||
- `textScore = 1 / (1 + max(0, bm25Rank))`
|
||||
|
||||
3) Union candidates by chunk id and compute a weighted score:
|
||||
3. Union candidates by chunk id and compute a weighted score:
|
||||
|
||||
- `finalScore = vectorWeight * vectorScore + textWeight * textScore`
|
||||
|
||||
Notes:
|
||||
|
||||
- `vectorWeight` + `textWeight` is normalized to 1.0 in config resolution, so weights behave as percentages.
|
||||
- If embeddings are unavailable (or the provider returns a zero-vector), we still run BM25 and return keyword matches.
|
||||
- If FTS5 can’t be created, we keep vector-only search (no hard failure).
|
||||
@@ -443,6 +459,7 @@ agents: {
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- Session indexing is **opt-in** (off by default).
|
||||
- Session updates are debounced and **indexed asynchronously** once they cross delta thresholds (best-effort).
|
||||
- `memory_search` never blocks on indexing; results can be slightly stale until background sync finishes.
|
||||
@@ -491,6 +508,7 @@ agents: {
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- `enabled` defaults to true; when disabled, search falls back to in-process
|
||||
cosine similarity over stored embeddings.
|
||||
- If the sqlite-vec extension is missing or fails to load, Moltbot logs the
|
||||
@@ -527,5 +545,6 @@ agents: {
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- `remote.*` takes precedence over `models.providers.openai.*`.
|
||||
- `remote.headers` merge with OpenAI headers; remote wins on key conflicts. Omit `remote.headers` to use the OpenAI defaults.
|
||||
|
||||
@@ -3,10 +3,10 @@ import type { AgentSession } from "@mariozechner/pi-coding-agent";
|
||||
import type { MemoryCitationsMode } from "../../config/types.memory.js";
|
||||
import type { ResolvedTimeFormat } from "../date-time.js";
|
||||
import type { EmbeddedContextFile } from "../pi-embedded-helpers.js";
|
||||
import { buildAgentSystemPrompt, type PromptMode } from "../system-prompt.js";
|
||||
import { buildToolSummaryMap } from "../tool-summaries.js";
|
||||
import type { EmbeddedSandboxInfo } from "./types.js";
|
||||
import type { ReasoningLevel, ThinkLevel } from "./utils.js";
|
||||
import { buildAgentSystemPrompt, type PromptMode } from "../system-prompt.js";
|
||||
import { buildToolSummaryMap } from "../tool-summaries.js";
|
||||
|
||||
export function buildEmbeddedSystemPrompt(params: {
|
||||
workspaceDir: string;
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
import type { ReasoningLevel, ThinkLevel } from "../auto-reply/thinking.js";
|
||||
import { SILENT_REPLY_TOKEN } from "../auto-reply/tokens.js";
|
||||
import { listDeliverableMessageChannels } from "../utils/message-channel.js";
|
||||
import type { MemoryCitationsMode } from "../config/types.memory.js";
|
||||
import type { ResolvedTimeFormat } from "./date-time.js";
|
||||
import type { EmbeddedContextFile } from "./pi-embedded-helpers.js";
|
||||
import { SILENT_REPLY_TOKEN } from "../auto-reply/tokens.js";
|
||||
import { listDeliverableMessageChannels } from "../utils/message-channel.js";
|
||||
|
||||
/**
|
||||
* Controls which hardcoded sections are included in the system prompt.
|
||||
|
||||
@@ -1,14 +1,13 @@
|
||||
import { Type } from "@sinclair/typebox";
|
||||
|
||||
import type { MoltbotConfig } from "../../config/config.js";
|
||||
import type { MemoryCitationsMode } from "../../config/types.memory.js";
|
||||
import type { MemorySearchResult } from "../../memory/types.js";
|
||||
import type { AnyAgentTool } from "./common.js";
|
||||
import { resolveMemoryBackendConfig } from "../../memory/backend-config.js";
|
||||
import { getMemorySearchManager } from "../../memory/index.js";
|
||||
import type { MemorySearchResult } from "../../memory/types.js";
|
||||
import { parseAgentSessionKey } from "../../routing/session-key.js";
|
||||
import { resolveSessionAgentId } from "../agent-scope.js";
|
||||
import { resolveMemorySearchConfig } from "../memory-search.js";
|
||||
import type { AnyAgentTool } from "./common.js";
|
||||
import { jsonResult, readNumberParam, readStringParam } from "./common.js";
|
||||
|
||||
const MemorySearchSchema = Type.Object({
|
||||
@@ -200,10 +199,7 @@ function deriveChatTypeFromSessionKey(sessionKey?: string): "direct" | "group" |
|
||||
if (!parsed?.rest) {
|
||||
return "direct";
|
||||
}
|
||||
const tokens = parsed.rest
|
||||
.toLowerCase()
|
||||
.split(":")
|
||||
.filter(Boolean);
|
||||
const tokens = parsed.rest.toLowerCase().split(":").filter(Boolean);
|
||||
if (tokens.includes("channel")) {
|
||||
return "channel";
|
||||
}
|
||||
|
||||
@@ -319,7 +319,12 @@ export async function runMemoryStatus(opts: MemoryCommandOptions) {
|
||||
) as MemorySourceName[];
|
||||
const workspaceDir = status.workspaceDir;
|
||||
const scan = workspaceDir
|
||||
? await scanMemorySources({ workspaceDir, agentId, sources, extraPaths: status.extraPaths })
|
||||
? await scanMemorySources({
|
||||
workspaceDir,
|
||||
agentId,
|
||||
sources,
|
||||
extraPaths: status.extraPaths,
|
||||
})
|
||||
: undefined;
|
||||
allResults.push({ agentId, status, embeddingProbe, indexError, scan });
|
||||
},
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import type { MemoryProviderStatus } from "../memory/types.js";
|
||||
import type { RuntimeEnv } from "../runtime.js";
|
||||
import { withProgress } from "../cli/progress.js";
|
||||
import { loadConfig } from "../config/config.js";
|
||||
import { buildGatewayConnectionDetails, callGateway } from "../gateway/call.js";
|
||||
@@ -7,14 +9,12 @@ import { collectChannelStatusIssues } from "../infra/channels-status-issues.js";
|
||||
import { resolveOsSummary } from "../infra/os-summary.js";
|
||||
import { getTailnetHostname } from "../infra/tailscale.js";
|
||||
import { getMemorySearchManager } from "../memory/index.js";
|
||||
import type { MemoryProviderStatus } from "../memory/types.js";
|
||||
import { runExec } from "../process/exec.js";
|
||||
import type { RuntimeEnv } from "../runtime.js";
|
||||
import { buildChannelsTable } from "./status-all/channels.js";
|
||||
import { getAgentLocalStatuses } from "./status.agent-local.js";
|
||||
import { pickGatewaySelfPresence, resolveGatewayProbeAuth } from "./status.gateway-probe.js";
|
||||
import { getStatusSummary } from "./status.summary.js";
|
||||
import { getUpdateCheckResult } from "./status.update.js";
|
||||
import { buildChannelsTable } from "./status-all/channels.js";
|
||||
|
||||
type MemoryStatusSnapshot = MemoryProviderStatus & {
|
||||
agentId: string;
|
||||
|
||||
@@ -12,6 +12,7 @@ import type {
|
||||
TalkConfig,
|
||||
} from "./types.gateway.js";
|
||||
import type { HooksConfig } from "./types.hooks.js";
|
||||
import type { MemoryConfig } from "./types.memory.js";
|
||||
import type {
|
||||
AudioConfig,
|
||||
BroadcastConfig,
|
||||
@@ -23,7 +24,6 @@ import type { NodeHostConfig } from "./types.node-host.js";
|
||||
import type { PluginsConfig } from "./types.plugins.js";
|
||||
import type { SkillsConfig } from "./types.skills.js";
|
||||
import type { ToolsConfig } from "./types.tools.js";
|
||||
import type { MemoryConfig } from "./types.memory.js";
|
||||
|
||||
export type OpenClawConfig = {
|
||||
meta?: {
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { z } from "zod";
|
||||
import { ToolsSchema } from "./zod-schema.agent-runtime.js";
|
||||
import { ApprovalsSchema } from "./zod-schema.approvals.js";
|
||||
import { AgentsSchema, AudioSchema, BindingsSchema, BroadcastSchema } from "./zod-schema.agents.js";
|
||||
import { ApprovalsSchema } from "./zod-schema.approvals.js";
|
||||
import { HexColorSchema, ModelsConfigSchema } from "./zod-schema.core.js";
|
||||
import { HookMappingSchema, HooksGmailSchema, InternalHooksSchema } from "./zod-schema.hooks.js";
|
||||
import { ChannelsSchema } from "./zod-schema.providers.js";
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
import { createSubsystemLogger } from "../logging/subsystem.js";
|
||||
import type { MoltbotConfig } from "../config/config.js";
|
||||
import { resolveMemoryBackendConfig } from "./backend-config.js";
|
||||
import type { ResolvedQmdConfig } from "./backend-config.js";
|
||||
import type {
|
||||
MemoryEmbeddingProbeResult,
|
||||
MemorySearchManager,
|
||||
MemorySyncProgressUpdate,
|
||||
} from "./types.js";
|
||||
import { createSubsystemLogger } from "../logging/subsystem.js";
|
||||
import { resolveMemoryBackendConfig } from "./backend-config.js";
|
||||
|
||||
const log = createSubsystemLogger("memory");
|
||||
const QMD_MANAGER_CACHE = new Map<string, MemorySearchManager>();
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import fs from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { resolveSessionTranscriptsDirForAgent } from "../config/sessions/paths.js";
|
||||
import { createSubsystemLogger } from "../logging/subsystem.js";
|
||||
import { redactSensitiveText } from "../logging/redact.js";
|
||||
import { createSubsystemLogger } from "../logging/subsystem.js";
|
||||
import { hashText } from "./internal.js";
|
||||
|
||||
const log = createSubsystemLogger("memory");
|
||||
|
||||
Reference in New Issue
Block a user