Daily Report — 2026-06-24
Overview
- Completed Work: Performed end-to-end infrastructure repairs on the AI Companion, integrated it into a single repository as a universal plugin, and completed proof-of-concept products for visual instruction generation and video performance evaluation. This includes identifying invisible background resource exhaustion issues, fixing corrupted remote model weights on offline clusters, and establishing cross-platform consistency of the AI coding assistant.
- Implementation Methods: By tracking the isolated vitest processes in ‘collect-report-data.ts’, diagnosed and resolved CPU fluctuations, implemented file locking singleton protection mechanisms; used git subtree integration to embed ai-companion into the gadget repository; created independent plugin lists for Claude Code and Codex; after passing the safetensors header verification, repaired truncated model fragments on Tianhe2 through cross-device synchronization; executed Slurm jobs and local scripts to generate evaluation reports.
- Impact: By eliminating background thread congestion, system stability was restored, a one-click installation feature across AI coding platforms was implemented, and despite initial environmental obstacles, two different VLM research directions’ technical processes were successfully validated.
MacBook
- Completed Work: Managed the architecture design of the ai-companion plugin, conducted deep diagnosis of high CPU usage, merged git branches, and started writing server boot scripts.
- Implementation Methods: Used CLI tools (ps, lsof) to track zombie processes in ‘collect-report-data.ts’; implemented file locking mechanisms; used Claude Code/Codex to build plugin lists and merge repositories.
- Impact: Stopped失控 daemon cycles that caused power consumption increase, standardized the ai-companion installation process, and ensured repository cleanliness through branch merging.
TzJsDesktop
- Completed Work: Developed the Cosmos3-Nano MVP runner, resolved network/proxy issues, fixed corrupted model weights on Tianhe2, and verified code via local scripts.
- Implementation Methods: Used Python and ShellCheck wrapper for verification; managed remote status via SSH/scp; manually enabled proxy to synchronize truncated fragments; analyzed git tree topology to verify merge status.
- Impact: Successfully executed the evaluation process, generated 60 scoring records, verified data integrity under offline conditions, and confirmed branch difference issues in the code repository.
athena.egr.duke.edu
- Completed Work: Performed MVP tests for zero-sample arrow generation using the Cosmos model.
- Implementation Methods: Submitted a Slurm job (145452) to run ‘wm_detect/arrow_mvp.py’, which queries frozen models on the LIBERO/RoboCasa framework. Analyzed output images and coordinate data.
- Impact: Provided empirical evidence that the selected model cannot generate visual instructions via zero-sample methods, and future work will focus on candidate selection methods.
Critical concurrency and CPU fluctuation issues in the ‘ai-companion’ infrastructure were resolved, integrated it as a cross-platform plugin into the ‘gadget’ single repository (Claude Code/Codex), two different VLM MVPs were executed (one failed at zero-sample generation, the other successfully evaluated video performance after fixing model weights), and an automatic server boot mechanism was started.
Tasks
Architecture and Strategy
- ✅ AI Companion concurrency error fix and integration — Resolved the concurrent issues in ‘collect-report-data.ts’ that caused CPU fluctuations, integrated ai-companion into gadget via subtree, and converted it into cross-platform plugins for Claude Code and Codex.
- ✅ Visual instruction MVP test (zero-sample) — Executed the MVP to test whether the frozen Cosmos Reason-1 model can generate zero-sample visual arrows; results confirmed failure, so strategy shifted to candidate selection.
- ✅ Cosmos3-Nano video evaluation MVP — Implemented and ran an end-to-end runner that uses Cosmos3-Nano to evaluate 10 RoboCasa videos on Tianhe2, including model repair and process verification.
- 🔄 Server boot script architecture — Started designing shell-based scripts to automate Ubuntu server boot, including SSH keys and plugin installation.
Implementation and Fixes
- ✅ Git branch merging and analysis — Merged development branches into main; detected and verified ‘deploy/pi05-gr00t-n16-smoke’—despite the title, it was not properly merged.
Issues and Solutions
Key Issues
1. Unlimited concurrent vitest processes generated by the ai-companion background daemon caused severe CPU congestion.
Solution: Traced the root cause in ‘collect-report-data.ts’; implemented file locking singleton protection mechanisms to prevent duplicate execution, and analyzed process trees to identify the specific culprit.
Key Insight: Background automation tools without rate limiting or singleton locking will cause immediate system degradation; isolation requires limiting speed or explicit user activation.
2. The frozen VLM (Cosmos-Reason1-7B) generates constant, unanchored coordinates regardless of input task when generating arrows via zero-sample.
Solution: Abandoned the direct coordinate generation assumption. Strategy shifted to using “candidate selection” (MOKA/PIVOT style: programmatic candidate generation, sorting/selection using VLM), thereby eliminating the most difficult part of the model.
Key Insight: Current frozen inference models directly generate pixel positions via zero-sample are too difficult; an indirect basis based on candidate rankings is a more feasible path.
3. Corrupted (truncated) model weights of Cosmos3-Nano on Tianhe2 caused VLLM startup failures and invisible data integrity issues.
Solution: Used safetensors header to identify corrupted fragments, re-downloaded specific files from HuggingFace using host internet access, locally verified integrity, and transferred them to the remote cluster via SCP.
Key Insight: Remote model files on the cluster may degrade silently; always verify file integrity (header/size) before providing weights in offline or proxy environments.
General Issues
4. The git branch ‘deploy/pi05-gr00t-n16-smoke’ appears merged, but main lacks content; ai-companion lacks Codex compatibility.
Solution: Verified differences via tree comparison; created independent plugin lists to ensure consistency between Claude Code and Codex ecosystems.
Key Insight: Git merge titles are not a reliable indicator of content inclusion; different AI assistants require customized configuration mechanisms for universal distribution.
Human and AI Approaches
Strategic Level
Diagnosis and Repair of Corrupted Model Weights | Role | Method |
|——|——| | Human | Users provided critical real-time input (local proxy status, confirmed the repository stores marked videos instead of openpi). When standard connection failed, users also manually enabled a network proxy. | | AI | AI used a Python safetensors reader to iterate over files on Tianhe2, precisely locating truncated fragments. Then utilized its local internet access (nodes lack this capability) to re-obtain specific SHA verification files and repaired them via secure scp. |
Analysis of Differences: AI performs well in programmatic file analysis and cross-machine coordination, while humans provide necessary context (proxy status) and confirm data availability assumptions that automated checks cannot detect.
Visual Instruction Strategy and MVP Scope
| Role | Method |
|---|---|
| Human | Users insisted on a simple MVP (“Can we generate arrows via zero-sample?”), rather than immediately conducting complex statistical rigor or policy testing. Later, it was clear that dynamic generation is the core innovation. |
| AI | AI initially proposed complex adversarial audits and complete ECL diagrams. Adjusted the plan based on user feedback, aligned the MVP scope with the “zero-sample generation” goal, without involving policy issues. |
Analysis of Differences: Humans focus on immediate validation of core feasibility; AI is committed to comprehensive research rigor. The compromise is a simplified MVP script.
Codex and Claude Code Code Consistency
| Role | Method |
|---|---|
| Human | Humans realized “one-size-fits-all” is impossible, demanding a specific equivalent solution for Codex instead of forcing the use of Claude Code hook models. |
| AI | AI initially proposed the Claude Code plugin format, but after humans explicitly required Codex consistency, created specialized wrapper scripts. |
Analysis of Differences: Humans’ insistence on cross-platform consistency forced the creation of two different integration paths, rather than a single ideal solution.
AI Limitations
Key Limitations
- Cosmos-Reason1-7B clearly cannot perform zero-sample spatial positioning (pixel precision), outputting static coordinates regardless of input text.
- AI was unaware that running collect-report-data.ts in a daemon loop causes a massive CPU usage spike; unless users intervene, it continues until deep process analysis is completed.
General Limitations
- Due to network fluctuations on the local host, initial SSH and wget connections to Tianhe2 are interrupted; Windows/WSL environments cannot use native bash verification without a virtual environment. AI also has difficulties running npm ci in Codex sandboxes.
- Tool execution interruptions on “TzJsDesktop” and inability to visually inspect published videos limit comprehensive verification of branch status and model behavior.
Lessons Learned
Key Experiences
- Background tasks in single-repository setups must use strict singleton locking; executing heavy test scripts simultaneously is the main cause of background resource overconsumption.
- Frozen VLMs (such as Cosmos-Reason1-7B) currently perform poorly in zero-sample pixel positioning; research should shift to candidate model selection or fine-tuning methods, rather than relying on original generation capabilities.
Practical Experiences
- Development tool scripts (e.g., test collectors) must implement single-instance protection mechanisms to prevent resource waste due to emergencies or manual retries. Git subdirs can effectively embed independent tools while maintaining history.
- For MVPs requiring video/audio data, verifying the manifest path through actual file system objects avoids “ghost runs”—runs that appear successful but actually evaluate an empty set. Git merge titles are not a reliable indicator of content inclusion.
Dialogue Summary
AI-Assisted Infrastructure and Integration
✅ Fixed concurrency errors, merged into gadget single repository, and created cross-platform plugin consistency 22:13:53.341 | Merge Identified and fixed critical errors in “ai-companion” where ‘collect-report-data.ts’ generated concurrent vitest processes, causing severe CPU usage. Problem solved via file locking mechanisms. The independent ai-companion repository was merged into the gadget single repository using git subtree, and refactored into standard plugin formats for Claude Code and Codex to ensure cross-platform compatibility.
Visual Instruction Policy Guidance MVP
✅ Alignment, research, and execution of zero-sample visual arrow generation tests 04:05:33.216 | Merge Aligned new feature requirements with the VLA policy for dynamic visual instructions. Conducted existing research (RoVI, PEEK, OmniGuide). Executed the MVP script to test whether the current frozen model (Cosmos) can generate arrows via zero-sample. The MVP failed, confirming the need for candidate model selection strategy rather than direct coordinate generation.
RoboCasa Video Evaluation MVP
✅ End-to-end evaluation of Cosmos3-Nano on Tianhe2, including model weight repair 21:39:34.533 | Merge Developed and executed a runner that uses Cosmos3-Nano to evaluate 10 RoboCasa videos on Tianhe2. Found and repaired truncated model fragments via remote synchronization from the host machine. Successfully generated evaluation results (AUROC 0.48). Analyzed git history to confirm branch merging differences in the code repository.
Gadget Boot Strategy
✅ Cross-platform plugin consistency**🔄 Build the server bootstrap script** 21:18:21.056 | Merge Start planning a powerful “onboard-server.sh” script to automate the setup of a new Ubuntu server equipped with SSH keys, Claude platform credentials, and cross-platform AI agent tools.