Daily Report — 2026-10-07
Daily Overview
- What was done: Conducted a large-scale closed-loop evaluation of 16 robotic tasks using a new constraint-based memory framework and GPT-6 Astra, while simultaneously implementing a system to share GPU lease registries between a host and a Hyper-V VM for sticker collection automation.
- How it was done: Utilized remote execution on the Tianhe3 cluster and Python-based failure attribution for robotics, and modified the ‘model_access’ library with directory junctions to support multi-host tracking for the VM automation project.
- Impact: Proved the new VLM memory framework approaches the theoretical oracle-guided upper bound (80.6%) and enabled vision-based sticker scanning in a virtualized environment without compromising host GPU resource protection.
Successfully executed a large-scale VLM evaluation achieving a 77.5% success rate and implemented a cross-host GPU lease sharing mechanism to enable vision-based sticker automation for a WeChat bot.
Tasks
Architecture & Strategy
- ✅ Main Experiment Execution — Running the 16-task evaluation suite with the new memory framework and GPT-6 Astra.
- ✅ GPU Lease Sharing (Option 2) — Modify model_access to support multi-host tracking and set up shared directory via junctions between host and VM.
- ✅ Failure Attribution Analysis — Analyzing the 36 failed episodes to distinguish between policy failures and memory errors.
- 🔄 Comparison and Ablation Study — Preparing and verifying ablation study code for further comparative experiments.
- 🔄 Automated WeChat Sticker Addition — Develop and debug a script to add downloaded stickers to WeChat via the emoji panel automation.
Implementation & Fixes
- ✅ Sticker Collection — Find and download ~100 AI-girl stickers and ~100 AI-related stickers, organized into separate folders.
- ✅ RouteStick Episode 6 Rerun — Rerunning a specific episode that failed due to a ‘Connection reset by peer’ network error.
- 🔄 Sticker Scanning Optimization — Improve the sticker scanning script to support full scrolling of the WeChat emoji panel and handle permission errors.
Problems & Solutions
Critical Issues
1. VM cannot access local vision models due to missing ‘model_access’ package and lack of shared GPU lease registry.
Solution: Modified ‘model_access’ to track ‘host’ names in leases and used directory junctions to share the lease JSON file between host and VM.
2. WeChat 4.x image right-click menu lacks the ‘Add to Stickers’ option, breaking the initial automation logic.
Solution: Rewrote the addition script to use the ‘Emoji Panel -> Custom Stickers -> +’ workflow instead of the context menu.
General Issues
3. The evaluation script automatically skips episodes marked as ’error’ in the log, preventing retries.
Solution: Analyzed the eval.py skip logic and implemented a way to launch a separate run to target only the errored episode without affecting the main results.
4. The remote connection to the LLM interface via the SSH tunnel was unstable.
Solution: Monitored the tunnel status and verified automatic reconnection once the proxy became reachable.
5. Episode 6 of RouteStick failed due to a network ‘Connection reset by peer’ error.
Solution: Identified it as an infrastructure issue rather than a policy failure and executed a targeted rerun for that specific episode.
6. PermissionError when running automation scripts from System32 directory due to uiautomation attempting to write logs.
Solution: Modified scripts to explicitly change the current working directory to the TEMP folder upon startup.
Human vs AI Approaches
Strategic Level
Drift Prevention Strategy
| Role | Approach |
|---|---|
| Human | Proposed adding a ‘settled-state’ check to ensure recording only starts when objects are fully stable. |
| AI | Proposed a more robust technical implementation: calculating the settled state on the server side and saving it in the distribution package to ensure bit-wise identical starting points across different OS/hardware. |
Difference Analysis: The human provided the intuitive physical requirement, while the AI provided the correct distributed systems solution to prevent floating-point drift.
GPU Lease Management Strategy
| Role | Approach |
|---|---|
| Human | Proposed a solution where the VM and host share the same registry (Option 2) to ensure the host’s GPU protection isn’t cleared by the VM. |
| AI | Identified the logic conflict where the library only checks local PIDs, which would cause cross-host lease corruption if not handled. |
Difference Analysis: The human provided the strategic direction (Option 2), while the AI performed the complex technical implementation of multi-host awareness and junction linking.
AI Limitations
General Limitations
- AI had to be reminded to check the specific code logic for how the status table counts ’error’ vs ‘success’ to ensure data integrity.
- Difficulty in accessing/seeing the VM desktop during automation due to window occlusion/black screenshots.
Learnings
Key Learnings
- The new approach achieves parity with handwritten rules, which is a strong claim for a VLM-based autonomous framework.
- 75% of failures in the current system are rooted in policy execution rather than memory errors, suggesting the next bottleneck is policy robustness.
- When implementing cross-host resource management, tracking the ‘host’ name is essential because PIDs are not unique or reachable across network shares.
Conversation Summaries
RoboMemory Harness
✅ Main Experiment Completion and Result Analysis The main evaluation run of 160 episodes was completed. The new framework achieved a 77.5% success rate, significantly outperforming FrameSamp (47.5%). Detailed failure attribution showed most errors were policy-related. Final results and PPT were updated.
ErrorRecoveryBenchmark
🔍 Drift Prevention via Settled-State Discussed handling object drift by waiting for a stable state before recording. Decided that stability must be pre-calculated on the server to ensure cross-platform consistency.
WeAgent Sticker System
🔄 AI Sticker Collection and WeChat Integration The user requested a collection of AI-themed stickers. The AI successfully scraped 183 stickers from GitHub repositories. Subsequent efforts focused on automating the addition of these stickers to WeChat via UI automation, encountering several hurdles including menu changes and directory permissions.
PersonalAgent Model Access
✅ Cross-host GPU Lease Synchronization Addressed the ‘model_access’ error in the VM by implementing a shared resource directory. Modified the core ‘model_access’ library to support multi-host lease tracking and successfully verified synchronization via a custom probe script.