Project Summary

Gadgets is a practical toolkit aimed at simplifying daily development processes. It integrates multiple functional modules such as AI assistants, academic research, performance benchmarks, automated blog publishing, and document translation.

Repository Structure

• tools/: The core functional area of the project, containing 5 independent tools:
  • summarize: Handles AI conversation records and generates daily/weekly/monthly summary reports.
  • research: Academic research tool, including paper discovery, researcher analysis, and citation graph.
  • benchmark: Cross-platform CPU/GPU performance benchmark suite.
  • website: Hugo-based blog system, including automated deployment and media compression.
  • translator: Gradio-based document translation tool, supporting multiple formats while preserving Markdown format.
• common/: Shared infrastructure package, containing LLM invocation, caching, translation engine, Hugo deployment, and other general functions.
• scripts/: Operations and maintenance scripts, handling data synchronization, environment initialization, language checks, etc.
• outputs/: Stores all generated content (logs, reports, cache, etc.), not counted in the Git repository.
• ai-companion (separate repository): Located in the same directory, providing development workflows, code change tracking, and a range of AI skills.

Usage Guide

This guide aims to help beginners get started with the Gadget project quickly.

1. Environment Preparation

Before starting, ensure your machine meets the following requirements:

• Python: 3.10+ (It is recommended to use the AI Conda environment).
• Node.js: 18+ (Primarily used for ai-companion modules).
• Basic dependencies: It is suggested to install the basic rclone tools first (for data synchronization).

2. Installation and Initialization

After cloning the repository, enter the project root directory and execute the following steps:

Basic installation (includes core tools and most functions):

pip install -e ".[all]"

If you need the translation function (requires additional dependency installation):

pip install -e ".[translator]"

Initialization for a new machine (automatic environment configuration): If this is your first time configuring this project on a machine, you can use the onboarding script:

1. Fill in onboard.yaml (including API Key and other information) in the tokens/ directory.
2. Run the following command to automatically complete the configuration (including SSH, tool configuration, rclone initialization, etc.):

   python scripts/onboard.py
   

3. Usage Guide for Core Tools

Each tool can be run independently in its corresponding directory or called through a unified module.

A. AI Conversation Summary (Summarize)

Used to organize your AI conversation records (Claude, ChatGPT, etc.) into structured reports.

• Fully automated process (recommended):

  python -m summarize auto --deploy
  

• Step-by-step operation:
  • Export unexported dates: python -m summarize daily export
  • Merge dates: python -m summarize daily merge --sync-all
  • Generate weekly report: python -m summarize weekly generate --week 2026-W12 --deploy
  • Generate monthly report: python -m summarize monthly generate --month 2026-02 --deploy

B. Academic Research (Research)

Provides paper search, deep analysis, researcher profiles, and citation graph analysis.

• Start the full research process:

  python tools/research/research_scout.py report --project my-project
  

• Natural language search:

  python tools/research/research_scout.py ask "Find Pieter Abbeel's latest robot operation papers"
  

• Get researcher profiles:

  python tools/research/research_scout.py profile "Sergey Levine"
  

• Analyze citation graphs of a specific literature:

  python tools/research/research_scout.py citations 2301.12597
  

C. Performance Benchmark (Benchmark)

Cross-platform CPU/GPU performance testing. Note: Please enter the corresponding directory before execution.

cd tools/benchmark
# 运行所有基准测试并将结果追加到 CSV
python -m benchmark.cli
# 仅测试 GPU
python -m benchmark.cli --gpu-only
# 运行测试并生成 HTML 报告
python -m benchmark.cli --report
# 生成报告并发布到 Hugo 网站
python -m benchmark.cli --report --deploy

D. Document Translation (Translator)

Start a Gradio interface to translate various documents (supports .md, .pdf, .docx, etc.).

• Start the translation interface:

  python -m translator
  

E. Blog Synchronization and Deployment (Website)

1. Project summary

Gadgets is a collection of practical tools accumulated by developers, aiming to automate repetitive tasks in development, research, and content publishing through AI capabilities. It not only includes multiple independent tools but also builds a complete automation workflow through a core shared package common.

Repository structure

• tools/: Core tool directory, containing five independent tool products. Each tool can run independently.
• common/: Shared infrastructure package. Provides unified underlying capabilities for all tools, such as LLM calls, JSON parsing, disk caching, atomic writing, translation engines, and Hugo deployment features.
• scripts/: Operations and maintenance scripts, used for data synchronization, new machine initialization configuration, etc.
• outputs/: Location of all generated outputs (reports, caches, logs, data, etc.).
• docs/: Design documents and archives.
• ai-companion/ (external repository): This project uses a linked brother directory via hooks, responsible for providing the development workflow engine (planning/execution/change tracking) and all Claude Code-related skills.

Overview of tool functions

1. Summarize: AI dialogue summarization tool. Automatically reads AI dialogue records from different devices (Claude/ChatGPT, etc.) and generates daily/weekly/monthly reports.
2. Research: Academic research tools. Supports paper discovery, deep paper insights (analyzing writing structure/OpenReview comments), researcher profiling, and citation graph analysis.
3. Benchmark: Performance testing suite. Cross-platform (NVIDIA/Apple/Intel) CPU/GPU floating-point operation performance tests.
4. Website: Hugo blog management. Includes media auto-compression, bilingual translation, and automated build and publishing processes.
5. Translator: Document translator. A localized translation web interface based on Gradio, supporting document translation in Markdown format, images, and files.

2. Usage guide

This guide is for users who are encountering Gadgets for the first time.

I. Environment preparation

Before starting, ensure your system has the following environment installed:

• Python 3.10+ (It is recommended to use Conda to manage the environment, e.g. conda activate AI)
• Node.js 18+ (Primarily used for the associated ai-companion repository)
• Ollama (Recommended, for running LLMs locally)

II. Installation and configuration

1. Install core packages and most tools: Enter the root directory of the project and run the following command to install the common package and dependencies required by most tools:

   pip install -e ".[all]"
   

2. Install the translator tool (requires additional installation): Since the translator depends on specific Gradio and model stack, it needs to be installed separately:

   pip install -e ".[translator]"
   

3. Initial configuration:

   • The project uses config.json for configuration. Please copy a copy from the template first:

     cp config.example.json config.json
     

   • One-click configuration for new machines: If you are deploying on a brand-new server, you can use the built-in onboarding script (prepare tokens/onboard.yaml first):

     python scripts/onboard.py
     

III. Typical usage of tools

1. Summarize (Dialogue summarization)

This tool operates through a two-stage workflow: first export data, then merge to generate reports.

• First stage: Export un-exported data:

  python -m summarize daily export
  

• Second stage: Sync and merge by day:

  python -m summarize daily merge --sync-all
  

• Generate weekly/monthly reports and deploy:

  python -m summarize weekly generate --week 2026-W12 --deploy
  python -m summarize monthly generate --month 2026-02 --deploy
  

• One-click full process (export $\rightarrow$ merge $\rightarrow$ weekly report $\rightarrow$ monthly report $\rightarrow$ deploy):

  python -m summarize auto --deploy
  

• Tip: You can switch the backend via the --api parameter, such as --api ollama or --api openai.

2. Research (Academic research)

Operate using the unified entry point research_scout.py.

• Execute the full research pipeline (search $\rightarrow$ evaluation $\rightarrow$ report):

  python tools/research/research_scout.py report --project my-project
  

• Search papers in natural language:

  python tools/research/research_scout.py ask "Find the latest robot operation paper by Pieter Abbeel"
  

• View researcher profiles:

  python tools/research/research_scout.py profile "Sergey Levine"
  

• Analyze citation graphs of papers (using arXiv ID):

  python tools/research/research_scout.py citations 2301.12597
  

3. Benchmark (Performance testing)

Note: Enter the tool directory first before running.

cd tools/benchmark
# 运行所有测试（结果追加至 CSV）
python -m benchmark.cli
# 仅运行 GPU 测试
python -m benchmark.cli --gpu-only
# 运行测试、生成 HTML 报告并发布到 Hugo 网站
python -m benchmark.cli --report --deploy

4. Website (Blog management)

• Local preview:

  cd tools/website && hugo server -D
  

• Update the website (compress media $\rightarrow$ build $\rightarrow$ push to GitHub Pages):
  • macOS/Linux: bash update.sh
  • Windows: powershell -ExecutionPolicy Bypass -File update.ps1

5. Translator (Document translation)

• Start the Gradio graphical interface:

  python -m translator
  

  After starting, open the displayed address in a browser to translate documents. The tencent/Hy-MT2-1.8B model will be automatically downloaded during the first run.

Complete Guide to Gadget Project

1. Project Summary

1.1 Project Overview

Gadget is an open-source toolset that gathers practical tools accumulated from daily development. It includes the following functions:

• AI daily report generation — Automatically count AI conversation records across multiple devices and generate weekly/monthly reports.
• Paper discovery — Search from academic databases, evaluate papers, and analyze researchers and citation networks.
• Performance benchmarking — Cross-platform FLOPS tests for CPU/GPU.
• Blog automation — Hugo static site with incremental media compression and automatic deployment.
• Document translation — Local inference Gradio translator that supports text and file formats.

1.2 Repository Structure

gadget/
├── tools/                    # 5个独立工具产品
│   ├── summarize/           # AI对话日/周/月报告
│   ├── research/            # 论文发现 + 研究者分析 + 引文图
│   ├── benchmark/           # CPU/GPU性能基准测试
│   ├── website/             # Hugo博客（增量压缩 + 自动发布）
│   └── translator/          # Gradio文档翻译器
├── common/                  # 共享基础设施包（所有工具依赖）
├── scripts/                 # 运维脚本（数据同步、机器配置等）
├── docs/                    # 设计文档和ECL计划
├── outputs/                 # 生成的文件（自动重建，已gitignore）
├── CLAUDE.md               # Claude Code 仓库指南
└── pyproject.toml          # Python打包配置

1.3 Functionality of each tool

1.4 Infrastructure

• common/ — Shared capabilities for LLM calls, disk caching, translation engine, Hugo deployment, etc.
• scripts/ — Data sync (sync.py), machine initialization (onboard.py), LLM services (serve_local_llm.sh), etc.
• ai-companion/ — Separate Node.js repository (in ../ai-companion/), provides code change tracking and all Claude Code skills

2. Detailed usage guide

2.1 Environment preparation

System requirements

• Python 3.10+ (recommended using conda environment)
• Node.js 18+ (only required for ai-companion repository)
• Git

Create a Python environment

# 使用conda创建Python 3.10环境
conda create -n AI python=3.10
conda activate AI

# 或者使用venv
python3.10 -m venv venv
source venv/bin/activate  # Linux/macOS
# 或在Windows上: venv\Scripts\activate

2.2 Installation steps

Step 1: Clone the repository

git clone <gadget-repository-url>
cd gadget

Step 2: Install public packages and all tools

# 安装common包 + summarize/research/benchmark/website的所有依赖
pip install -e ".[all]"

# 如果需要translator工具，需单独安装
pip install -e ".[translator]"

# 如果需要website工具的完整功能（包含torch+transformers用于翻译）
pip install -e ".[website]"

Step 3: Prepare API keys and configuration

# 创建token目录（存放API密钥等敏感信息，已gitignore）
mkdir -p tokens/

# 从示例配置创建主配置文件
cp config.example.json config.json

# 编辑config.json，填入以下信息：
# - LLM API密钥（如果使用anthropic或openai后端）
# - Hugo部署配置（如果使用website工具）
# - rclone数据同步配置（如果需要跨设备数据同步）

Step 4: One-time machine initialization (optional but recommended)

If using a new machine, use automated scripts to complete all configurations:

# 编辑初始化配置表
# 打开 tokens/onboard.yaml，填入SSH/API密钥/工具配置等

# 运行初始化脚本
python scripts/onboard.py

# 这个脚本会自动完成：
# - SSH配置
# - Claude Code / Codex CLI 登录
# - pip 依赖安装
# - 各工具配置
# - rclone 引导启动

Step 5: Verify installation

# 运行烟雾测试（只读，无LLM调用，无网络，无写入）
bash scripts/smoke.sh

2.3 Tool configuration

Global LLM backend configuration

All tools switch backends via unified --api parameter:

# 方式1：命令行指定
python -m summarize daily export --api ollama

# 方式2：环境变量全局设置
export GADGET_LLM_BACKEND=claude_cli
python -m summarize daily export  # 自动使用claude_cli

# 可用后端
# - ollama（默认，本地Ollama，无需密钥，模型：Qwen3.8-27B）
# - claude_cli（重用Claude Code CLI登录状态，无需API密钥）
# - anthropic（需要ANTHROPIC_API_KEY环境变量）
# - openai（需要OPENAI_API_KEY环境变量）

Translation engine configuration

Translation for Website and Translator tools uses local inference:

# 方式1：自动选择（推荐）
# - Ollama已安装且模型已拉取 → 使用ollama
# - Linux → vllm优先
# - Windows → transformers优先

# 方式2：指定后端
export GADGET_TRANSLATION_BACKEND=transformers

# 方式3：指定模型
export GADGET_TRANSLATION_MODEL=tencent/Hy-MT2-1.8B-GGUF

# 默认模型会在首次使用时自动下载

Ollama local LLM service (recommended)

# 启动Ollama服务并输出环境变量
eval "$(bash scripts/serve_local_llm.sh env)"

# 这会：
# 1. 拉取并启动 Qwen 模型（如果未启动）
# 2. 输出环境变量供后续命令使用
# 3. 持久化配置保存在repo根目录 config.json 的 summarize 字段

2.4 Detailed usage of each tool

2.4.1 Summarize — AI conversation daily/weekly/monthly report

Function: Automatically extract token usage and conversation content from AI conversation records across multiple devices, generating structured daily, weekly, and monthly reports.Workflow:

1. Export unexported conversation records on each device (Phase 1)
2. Sync via cloud drive or manually copy the aggregation (Phase 2)
3. Generate daily, weekly, and monthly reports

Typical Commands:

# === Phase 1: 导出所有未导出日期的对话记录 ===
python -m summarize daily export

# === Phase 2: 同步所有日期并逐日合并 ===
python -m summarize daily merge --sync-all

# === 生成周报（周号格式：2026-W12，表示2026年第12周）===
python -m summarize weekly generate --week 2026-W12

# 生成周报并自动发布到Hugo博客
python -m summarize weekly generate --week 2026-W12 --deploy

# === 生成月报（月份格式：2026-02）===
python -m summarize monthly generate --month 2026-02

# 生成月报并发布
python -m summarize monthly generate --month 2026-02 --deploy

# === 一键完整流程 ===
# 导出 → 合并 → 生成周报 → 生成月报 + 自动发布
python -m summarize auto --deploy

# === 指定LLM后端 ===
python -m summarize daily export --api anthropic
python -m summarize auto --deploy --api openai

Supported LLM backends:

• ollama (default, local, no key)
• claude_cli (reuse Claude Code CLI)
• anthropic (requires ANTHROPIC_API_KEY)
• openai (requires OPENAI_API_KEY)

Detailed step tutorial: See TUTORIAL.md — Summarize and tools/summarize/tutorial.md

2.4.2 Research — Paper discovery and researcher analysis

Function: A unified academic research toolkit providing paper discovery, in-depth paper analysis, researcher analysis, and citation graph analysis.

Entry: tools/research/research_scout.py

Typical Commands:

# === 论文发现完整流程 ===
# 搜索论文 → 三阶段评估（快速筛选→深度评估→引文影响）→ 生成周报 → 发布Hugo
python tools/research/research_scout.py report --project my-project

# === 自然语言搜索（自动路由源）===
python tools/research/research_scout.py ask "找 Pieter Abbeel 最近的机器人操作论文"

# === 研究者分析 ===
# 获取论文+引用数据，生成研究轨迹分析，计算分级分数
# 自动发现导师-学生关系
python tools/research/research_scout.py profile "Sergey Levine"

# === 引文图分析 ===
# 基于Semantic Scholar API的前向/后向引用分析
python tools/research/research_scout.py citations 2301.12597

# === 发布报告到Hugo ===
python tools/research/research_scout.py deploy

# === 指定LLM后端 ===
python tools/research/research_scout.py report --project my-project --api claude_cli
python tools/research/research_scout.py profile "Sergey Levine" --api anthropic

LLM backend options:

• ollama (default)
• claude_cli
• anthropic
• openai

Detailed step tutorial: See TUTORIAL.md — Research and tools/research/TUTORIAL.md

2.4.3 Benchmark — Performance benchmark testing

Function: Cross-platform CPU/GPU FLOPS benchmark testing, supporting various hardware and precision levels.

Prerequisites:

cd tools/benchmark  # 所有命令必须在此目录执行

Supported hardware:

• NVIDIA (CUDA)
• Apple Silicon (MPS)
• Intel (XPU)

Supported precisions: FP64, FP32, FP16, BF16, FP8 (experimental)

Typical Commands:

cd tools/benchmark

# === 运行所有基准测试（结果追加到CSV） ===
python -m benchmark.cli

# === 仅CPU测试 ===
python -m benchmark.cli --cpu-only

# === 仅GPU测试 ===
python -m benchmark.cli --gpu-only

# === 运行测试 + 生成HTML报告 ===
# 生成Plotly交互式图表、排行榜
python -m benchmark.cli --report

# === 生成报告 + 发布到Hugo /benchmark/ 页面 ===
python -m benchmark.cli --report --deploy

# === 提交到公共排行榜（如需要） ===
# 见详细文档

Result processing:

• Results accumulated into CSV (append mode, never overwritten)
• Supports generating interactive HTML reports (Plotly)
• Supports publishing to Hugo website
• Supports submitting to public leaderboards

Detailed step tutorial: See TUTORIAL.md — Benchmark and tools/benchmark/tutorial.md

2.4.4 Website — Hugo blog automation

Function: Hugo static blog website, supporting incremental media compression and automated deployment.

Website information:

• Website name: “TzJ’s Net”
• Theme: PaperMod
• Deployment: GitHub Pages https://tzj2006.github.io/

Features:

• Incremental image/video compression (only compress files changed after .last_build)
• Local bilingual translation (default Ollama, vLLM alternative for Linux, transformers alternative for Windows)
• Automatic translation, compression, building, and pushing

Prerequisite dependencies:

# 安装website依赖（包含torch + transformers用于本地翻译）
pip install -e ".[website]"

Typical Commands:

# === macOS / Linux ===
cd tools/website
bash update.sh  # 增量压缩 + Hugo构建 + 推送GitHub Pages

# === Windows ===
powershell -ExecutionPolicy Bypass -File tools/website/update.ps1
# 脚本自动cd到自己的目录

# === 本地预览（dev server） ===
cd tools/website
hugo server -D  # -D 表示包含草稿，打开 http://localhost:1313

Automatic content processing:

• Daily/weekly/monthly reports, research papers, benchmark pages, charts, etc. are directly written into tools/website/content|static by the release processes of each tool
• All automatically generated files are marked with gadget_generated
• Handwritten files (without marks) will never be overwritten

Deployment repository:

• tools/website/public/ is a separate deployment repository tzj2006/tzj2006.github.io
• Automatically committed and pushed by the build script
• Do not submit directly to this directory

Detailed step tutorial: See TUTORIAL.md — Website and tools/website/CLAUDE.md

2.4.5 Translator — Gradio document translator

Function: Local inference document translator, supporting text and various file formats.

Features:

• Local web pages with Google Translate-style translation
• Supported formats: .md / .txt / .pdf / .docx / images
• Preserves Markdown format (code blocks, URLs, Hugo shortcodes, etc. are protected)
• Local inference, no cloud API calls

Prerequisite dependencies:

pip install -e ".[translator]"

Start the application:

python -m translator
# 自动在浏览器打开 Gradio GUI

Configuration:

# 默认模型（自动在首次使用时下载）
# 模型：tencent/Hy-MT2-1.8B（GGUF变体：tencent/Hy-MT2-1.8B-GGUF）

# 方式1：环境变量指定模型
export GADGET_TRANSLATION_MODEL=tencent/Hy-MT2-1.8B-GGUF

# 方式2：在GUI中管理模型

# 方式3：环境变量指定后端
export GADGET_TRANSLATION_BACKEND=ollama

Function:

• Source language can be set to auto (automatically detected via text’s CJK ratio)
• Automatically switches between Chinese and English when target language is auto

Detailed step tutorial: See TUTORIAL.md — Translator

Code reference:

• UI configuration: tools/translator/app.py
• Translation and file logic: tools/translator/core.py
• Shared translation engine: common/engine.py, common/translation.py

2.5 Cross-device data synchronization

Function: Centralized rclone data synchronization, supporting push/pull/status/bootstrap/configuration.

# 查看帮助
python scripts/sync.py

# 推送到远程
python scripts/sync.py push

# 拉取从远程
python scripts/sync.py pull

# 查看同步状态
python scripts/sync.py status

# 初始化bootstrap（首次setup）
python scripts/sync.py bootstrap

# 配置同步
python scripts/sync.py config

Configuration:

• Saved in the sync field in the repo root directory config.json
• Overwrite configuration path: GADGET_CONFIG environment variable
• Coverage: summarize / website / research / benchmark / backups / dag (DAG website generation + publishing)

2.6 Output directory structure

All generated files are stored in the outputs/ directory (gitignoreed, can be automatically rebuilt):

outputs/
├── logs/         # 中间文件（导出日志、运行日志）
├── reports/      # 最终报告（Markdown、JSON、HTML）
├── cache/        # LLM缓存、搜索缓存
├── data/         # 结构化数据（CSV、JSON研究者数据）
├── images/       # 图表和生成的图像
└── backups/      # 强制覆盖备份（website-force等）

Hugo website content directly written into tools/website/content|static (no separate output hierarchy directory).

2.7 Common tips and considerations

Quick reference of environment variables

# LLM后端选择
export GADGET_LLM_BACKEND=claude_cli

# 翻译后端选择
export GADGET_TRANSLATION_BACKEND=transformers

# 翻译模型选择
export GADGET_TRANSLATION_MODEL=tencent/Hy-MT2-1.8B-GGUF

# 配置文件路径（默认./config.json）
export GADGET_CONFIG=/path/to/custom/config.json

Checklist for first use

• Environment preparation (Python 3.10+, conda/venv)
• Install dependencies pip install -e ".[all]"
• Create config.json (copied from config.example.json)
• Prepare API key (placed in config.json or environment variable)
• Run smoke test bash scripts/smoke.sh
• Select a tool to start using

Handling common issues

# 测试特定工具的导入（无网络、无LLM）
python -c "import summarize; print('OK')"
python -c "import research; print('OK')"

# 查看当前配置
cat config.json

# 重置缓存（如遇cache问题）
rm -rf outputs/cache/

# 检查GPU（仅benchmark）
cd tools/benchmark && python -c "import torch; print(torch.cuda.is_available())"

2.8 AI Companion integration (optional)

This project provides through the ai-companion repository ../ai-companion/:

• Code change tracking
• Claude Code planning/execution skills
• All domain skills (summarize, optimize, etc.)

No additional configuration is needed; automatically integrated via hooks. For deeper understanding, see ../ai-companion/scripts/install.ts.

2.9 Quick start example

Scenario 1: Generate the AI conversation weekly report for this week

# 激活环境
conda activate AI

# 启动本地Ollama（如未启动）
eval "$(bash scripts/serve_local_llm.sh env)"

# 导出所有记录并生成周报
python -m summarize daily export
python -m summarize daily merge --sync-all
python -m summarize weekly generate --week 2026-W35 --deploy

Scenario 2: Search for the latest machine learning papers

conda activate AI
python tools/research/research_scout.py report --project ml-papers --api claude_cli

Scenario 3: Test GPU performance and generate a report

conda activate AI
cd tools/benchmark
python -m benchmark.cli --report

Scenario 4: Start the local document translator

conda activate AI
python -m translator
# 浏览器自动打开 http://localhost:7860

This tutorial covers all key steps for using Gadget from scratch. According to your specific needs, choose the appropriate tool and use it with the above commands. If you have any questions, refer to TUTORIAL.md and source documentation for each tool.

1. Project summary

1. What is Gadget

Gadget is a set of practical tools accumulated from daily development, covering five types of tasks:

• Generation of daily, weekly, and monthly AI conversation records
• Paper discovery, in-depth paper analysis, and researcher profiling
• CPU/GPU floating-point performance testing
• Hugo blog building, translation, media compression, and automatic publishing
• Local document translation

The five tools under tools/ are independent and can be used separately. They all rely on the common/ package in the repository root directory. common/ provides general capabilities such as LLM invocation, JSON repair, disk caching, atomic writing, local translation, and Hugo deployment.

The development workflow and Claude Code skills of the project are not in this repository but in a separate repository ../ai-companion/ in adjacent directories, accessed via hooks under .claude/ and .codex/.

The project uses GPL-3 license.

2. Repository structure

gadget/
├── tools/
│   ├── summarize/      # AI 对话日报、周报、月报
│   ├── research/       # 论文发现、研究者分析、引用图谱
│   ├── benchmark/      # CPU/GPU 性能测试
│   ├── website/        # Hugo 博客构建和发布
│   └── translator/     # Gradio 文档翻译器
├── common/             # 所有工具共享的基础设施
├── scripts/            # 同步、初始化、维护脚本
├── docs/               # 设计文档、ECL 计划、审计和历史文档
├── outputs/            # 自动生成的输出，已被 gitignore
├── AGENTS.md           # AI agent 工作协议
├── CLAUDE.md           # Claude Code 仓库指南
└── pyproject.toml      # Python 包及 extras 配置

When modifying different types of functions, the corresponding locations are as follows:

3. What each tool does

Summarize

Reads AI conversation records in Claude Code, Codex, Cursor Agent, ChatGPT, or general JSON format, and generates structured daily, weekly, and monthly reports.

It uses a multi-device, two-stage process:

1. Export conversation records on each device.
2. Aggregate via cloud drive sync or manual copy, then generate reports daily.

When a certain number of reports accumulate, weekly and monthly trend summaries can also be generated further.

It uses ccusage 20.x command statistics for different agent CLI token usage and costs, such as ccusage claude, ccusage codex, ccusage gemini.

Four LLM backends are supported:

• ollama: Default, runs locally, no API key required
• claude_cli: Reuses Claude Code CLI login state, no API key required
• anthropic
• openai

Research

A unified academic research toolkit, entry is:

tools/research/research_scout.py

It has four main capabilities:- Research report generation: Search for papers on arXiv, bioRxiv, and PubMed, and generate a research weekly report through the three-stage LLM process of “quick filtering → deep evaluation → citation impact”, which can be published on Hugo.

• Deep paper insight: Download the full text of papers, analyze their writing structure, publication strategies, and reusable knowledge; match OpenReview comments, and analyze the consensus and disputes in reviews; finally, synthesize a cross-paper writing guide.
• Researcher profile: Obtain paper and citation data from ArXiv and Semantic Scholar, generate analysis of research trajectories and hierarchical scoring, and infer the mentor–student relationship through personal profiles and co-author patterns.
• Citation graph: Use the Semantic Scholar API to analyze forward citations and backward references, and then let the LLM interpret the impact of the papers.

It also supports four LLM backends: ollama, claude_cli, anthropic, and openai.

Benchmark

A cross-platform CPU/GPU FLOPS floating-point performance testing tool that supports:

• NVIDIA CUDA
• Apple Silicon MPS
• Intel XPU

OpenCL may appear in the --info detection results, but the README does not indicate that it can perform actual GPU tests.

Supported precision includes:

• FP64
• FP32
• FP16
• BF16
• FP8, experimental support

The testing process includes preheating, formal measurement, median statistics, IQR outlier removal, and explicit GPU synchronization.

Results are written to CSV in append mode and do not overwrite existing results, allowing data from multiple hardware devices to be accumulated and a leaderboard to be formed. The tool can also generate interactive HTML reports using Plotly and publish them on Hugo websites or submit them to public leaderboards.

Website

A Hugo static blog named “TzJ’s Net” using the PaperMod theme, with the target site being https://tzj2006.github.io/.

Main capabilities include:

• Compressing only media files that change after .last_build
• Compressing images with pngquant
• Compressing videos with HandBrakeCLI
• Generating bilingual content using local models
• Automatically building the Hugo website
• Automatically submitting and pushing to GitHub Pages deployment repositories

Daily reports, weekly reports, monthly reports, research reports, benchmark pages, and images will be written directly by the deployment process:

tools/website/content
tools/website/static

The automatically generated files carry the gadget_generated tag. Handwritten files without this tag will not be overwritten.

tools/website/public/ is an independent deployment repository corresponding to tzj2006/tzj2006.github.io; files should not be submitted manually there.

Translator

A local document translation web application based on Gradio, with a interface similar to Google Translate.

Translation is supported for:

• Plain text
• .md
• .txt
• .pdf
• .docx
• Images

It will preserve the Markdown format as much as possible and protect fragments such as code blocks, URLs, and Hugo shortcodes.

Translation is completely done locally; no cloud LLM API is used. It automatically selects a backend through common.engine.create_engine():

• Ollama is preferred when the corresponding model has been pulled
• Windows falls back to transformers
• vLLM is preferred on Linux
• llama.cpp/GGUF can be used in low memory scenarios

The default model is:

tencent/Hy-MT2-1.8B

The GGUF version is:

tencent/Hy-MT2-1.8B-GGUF

The model will be automatically downloaded the first time it is used. After loading, the model remains in memory, and loading is delayed when switching models as needed.

When the source language is set to auto, it is automatically detected based on the proportion of CJK characters in the text; when the target language is set to auto, it automatically switches between Chinese to English and English to Chinese.

4. Public modules and operation scripts

common/

A public Python package that provides:

• A unified two-layer LLM API
• JSON parsing and repair
• SHA-256 disk cache with namespace and TTL
• Atomic writing and content hashing
• Local translation engine
• Bilingual content generation
• Cross-platform Hugo deployment

scripts/

Main scripts include:

• sync.py: Sync summarize, website, research, benchmark, and backups data using rclone, and also supports generating and deploying DAG sites.
• onboard.py: One-time initialization for a new machine, responsible for SSH, Claude/Codex CLI, authentication, Python extras, AI Companion, tool configuration, and rclone bootstrap.
• smoke.sh: Read-only smoke test, checks help, hardware information, and import, without calling LLM, network, or performing writing.
• serve_local_llm.sh: Create an Ollama model variant suitable for summarize and output environment variables.
• language.py: Hugo bilingual content auditing and renaming of summarize reports.
• profile_translation.py: Translation engine GPU profiler.

AI Companion

../ai-companion/ is an independent Node.js/TypeScript repository that provides:

• Function-level code change records
• Testing and HTML report generation
• /idea → /ccplan → /ccedit planning and execution process
• Claude Code and Codex integration
• Skills such as optimization, debugging, auditing, cleaning, and paper writing

Node.js 18+ is only required when using this independent repository.

2. Tutorial for the first use of Gadget

The following tutorial is strictly limited to the information and commands given in the README. The more detailed configuration fields, data formats, and FAQ in the README are placed in TUTORIAL.md and the tutorials of each tool itself, so no ungiven configuration items will be guessed here.

1. Environment preparation

Python

Python 3.10 or higher is required.

The README recommends using a conda environment named AI:

conda activate AI

The README does not provide the command to create this conda environment, so a Python 3.10+ environment must be prepared in advance.

Node.js

Node.js 18+ is only required when using the independent AI Companion repository:

../ai-companion/

There is no requirement to install Node.js in the README for running only the Python tools of Gadget.

Other external components

Different tools may also require:

• rclone: Cross-device synchronization
• Hugo: Blog building and local preview
• pngquant: Image compression
• HandBrakeCLI: Video compression
• Ollama: Default LLM or translation backend
• PyTorch, NumPy, Plotly: Benchmarking
• Gradio, GGUF/transformers: Translator

Specific tool dependencies can also be checked in the requirements.txt of each tool directory.

2. Installation

The following commands should be executed in the root directory of the Gadget repository, unless explicitly instructed to switch directories before the command.

Installing public packages

pip install -e .

This is the installation method for the common/ base package that all tools depend on.

After installation, the root directory of the repository may generate:

build/
gadget.egg-info/

They are build artifacts and have been added to gitignore; they should not be committed.

Installing all extras for the main tools

pip install -e ".[all]"

This all includes:

• summarize
• research
• benchmark
• website

But does not include translator.

Installing Translator separately

pip install -e ".[translator]"

Installing only dependencies related to Website

pip install -e ".[website]"

This extra includes torch and transformers required for website translation.

3. Basic configuration

Root configuration file

The main settings of all tools are located in the root directory of the repository:

config.json

This file is not tracked by Git. The README requires copying from the following example:

config.example.json

The README does not provide specific copying commands or content for each field, so config.json should be created manually referring to the example file.

If you need to use a different configuration file path, you can set:

GADGET_CONFIG

LLM backend

Summarize and Research uniformly support the following --api values:

• ollama
• claude_cli
• anthropic
• openai

The default value is ollama.

The backend can also be selected through global environment variables:

GADGET_LLM_BACKEND

Note:

• ollama uses local Ollama and does not require an API key.
• claude_cli reuses the login status of Claude Code CLI and does not require an API key.
• The README does not provide the specific variable names or configuration fields for Anthropic and OpenAI keys, so do not guess on your own; refer to the project tutorial or configuration example.

Translation configuration

The translation model can be overridden through the following environment variables:

GADGET_TRANSLATION_MODEL

The translation backend can be selected through the following variables:

GADGET_TRANSLATION_BACKEND

Default model:

tencent/Hy-MT2-1.8B

The model will be automatically downloaded the first time it is used.

Token and initialization configuration

API keys and new machine initialization tables are placed in:

tokens/

The initialization table is:

tokens/onboard.yaml

tokens/ has been added to gitignore; do not commit its content at all.

The README states: fill in tokens/onboard.yaml and run scripts/onboard.py to complete the one-time machine configuration at the repository level, including:

• SSH configuration
• Claude/Codex CLI installation and authentication
• Python extras installation
• AI Companion installation
• Configuration of each tool
• rclone bootstrap

The README does not provide a complete call example for onboard.py in the command block, so its parameters are not expanded here.

4. Tutorial for using Summarize

Summarize recommends using a unified entry point:

python -m summarize

Step 1: Export conversation records on the current device

python -m summarize daily export

This command executes the first stage of the multi-device process, exporting all dates that have not been exported yet.

Each device with AI conversation records should execute this stage.

Step 2: Synchronization and daily merging

python -m summarize daily merge --sync-all

This command synchronizes data from all dates and merges them daily to generate a daily report.

Generating weekly reports for a specified week

python -m summarize weekly generate --week 2026-W12 --deploy

Where:

• 2026-W12 represents the 12th week of 2026.
• --deploy represents deployment after generation.

Generating monthly reports for a specified month

python -m summarize monthly generate --month 2026-02 --deploy

Where:- 2026-02 indicates February 2026.

• --deploy indicates generation followed by deployment.

One-click execution of the full process

python -m summarize auto --deploy

It will complete sequentially:

export → merge → weekly → monthly → deploy

Old entry point

The following old scripts are still compatible:

python tools/summarize/daily_summary.py ...
python tools/summarize/weekly_summary.py ...
python tools/summarize/monthly_summary.py ...

But the README recommends using:

python -m summarize

The README does not list the specific parameters of the old entry point, so the parameters of the new entry point should not be assumed as those of the old entry point.

5. Research usage tutorial

The unified entry point for Research is:

python tools/research/research_scout.py

Full paper search and report process

python tools/research/research_scout.py report --project my-project

This command executes:

搜索 → 三阶段评估 → 生成报告

my-project is the project name in the README example.

Searching papers with natural language

python tools/research/research_scout.py ask "找 Pieter Abbeel 最近的机器人操作论文"

The tool will automatically select search sources based on the query.

Generating researcher profiles

python tools/research/research_scout.py profile "Sergey Levine"

Researcher profiles may include:

• Paper data from ArXiv and Semantic Scholar
• Citation data
• Research trajectory analysis
• tier score
• mentor–student relationship discovery
• name ambiguity resolution
• reverse lookup

The specific output fields are not expanded in the README.

Analyzing paper citation graphs

Using arXiv ID:

python tools/research/research_scout.py citations 2301.12597

The README indicates that this location also supports DOI, but no example format for DOI is provided, so no examples are fabricated additionally.

The analysis includes:

• Forward citations
• Backward references
• Influence interpretation driven by LLM

Deploying reports to Hugo

python tools/research/research_scout.py deploy

Deep paper insights

The README states that Research supports --insight, and can:

• Download full paper texts
• Analyze writing structures
• Analyze publication strategies
• Extract reusable core knowledge
• Match OpenReview
• Analyze review consensus and disputes
• Generate a research writing guide comprehensively

But the README does not provide the complete --insight command combination, so it should not be guessed as which subcommand it should follow. When using it, refer to tools/research/TUTORIAL.md.

6. Benchmark usage tutorial

All Benchmark commands must first enter their directory:

cd tools/benchmark

Execute all benchmarks

python -m benchmark.cli

The results will be appended to CSV without overwriting existing results.

Only test CPU

python -m benchmark.cli --cpu-only

Only test GPU

python -m benchmark.cli --gpu-only

The tool will automatically detect:

• CUDA
• Apple MPS
• Intel XPU

Execute testing and generate HTML report

python -m benchmark.cli --report

The report uses Plotly to generate interactive charts.

Execute testing, generate report, and deploy

python -m benchmark.cli --report --deploy

The report will be published on the Hugo website:

/benchmark/

Testing uses preheating, formal measurement, median statistics, IQR outlier removal, and GPU synchronization. Since CSV uses append mode, it can be run repeatedly on different hardware and accumulate leaderboard data.

7. Website usage tutorial

Install website dependencies

Run in the root directory of the repository:

pip install -e ".[website]"

macOS/Linux: Build and publish

cd tools/website && bash update.sh

This process will execute:

• Incremental compression of modified images and videos
• Local bilingual translation of models
• Hugo build
• Submit and push to GitHub Pages deployment repository

Windows: Build and publish

Run in the root directory of the repository:

powershell -ExecutionPolicy Bypass -File tools/website/update.ps1

The Windows script will automatically switch to its own directory, and cd tools/website does not need to be executed in advance.

Local preview

cd tools/website && hugo server -D

-D will make the local development server contain draft content.

Notes on website files

Automatically generated content is written directly:

tools/website/content
tools/website/static

Files marked with gadget_generated may be automatically updated; handwritten files without this mark will not be overwritten.

Do not manually commit or directly modify the deployment repository:

tools/website/public/

It is automatically committed and pushed by the build script.

8. Translator usage tutorial

Install dependencies

pip install -e ".[translator]"

Start the translation interface

python -m translator

After starting, the Gradio GUI in the browser will open.

In the interface:

• Enter text for translation
• Upload .md, .txt, .pdf, .docx or images
• Select source and target languages
• Manage or switch translation models

The default model will be automatically downloaded during the first run.

When setting the source language to auto, the tool will detect the language based on the CJK character ratio; when setting the target language to auto, it will automatically translate inversely between Chinese and English.

9. Data synchronization

Synchronization is driven by rclone, and the configuration is in the sync section of the root directory config.json.

View the synchronization tool entry point:

python scripts/sync.py

Push and pull across devices use respectively:

python scripts/sync.py push

python scripts/sync.py pull

The README also states that sync.py supports:

status
bootstrap
config

as well as summarize, website, research, benchmark, backups, and special dag categories, but no complete command combinations for these operations are provided, so parameters are not expanded here.

10. Local Summarize Ollama environment

The startup and environment variable loading commands provided in the README are:

eval "$(bash scripts/serve_local_llm.sh env)"

This script creates an Ollama variant suitable for Summarize and outputs environment settings.

Long-term configuration items should be written in the summarize section of the root directory config.json, or can be pointed to other configurations using GADGET_CONFIG.

11. Running a read-only smoke test

bash scripts/smoke.sh

This check only executes:

• --help
• --info
• Python import check

It does not call LLM, access the network, or write files, making it suitable for basic verification after installation.

12. Where to place the output file

All reconstructable outputs are placed uniformly in:

outputs/

The structure is as follows:

outputs/
├── logs/         # 导出日志、运行日志等中间产物
├── reports/      # Markdown、JSON、HTML 最终报告
├── cache/        # LLM 和搜索缓存
├── data/         # CSV、JSON profile 等结构化数据
├── images/       # 图表和生成图片
└── backups/      # 强制覆盖前的备份

Hugo content is not temporarily stored via outputs/site, but written directly:

tools/website/content
tools/website/static

13. Things to note when using

• Do not submit the API key or onboarding configuration in tokens/.
• Do not submit the automatically generated files in outputs/.
• Do not submit build/ and gadget.egg-info/.
• Do not submit data synchronized by rclone.
• Do not directly submit the deployment repository or theme repository under tools/website/.
• config.json is the local configuration file, which should be created from config.example.json.
• Translator is not included in .[all]; .[translator] must be installed separately.
• Website and translation functions use local inference models; tencent/Hy-MT2-1.8B may need to be downloaded during the first run.
• More detailed configuration fields, data formats, troubleshooting, and --insight usage are not fully demonstrated and should be checked in the repository TUTORIAL.md and tool tutorials; this README alone cannot provide reliable information.

Project overview

Gadgets is a collection of daily developer tools covering five types of tasks:

• Generation of daily, weekly, and monthly reports for AI conversation records
• Searching academic papers, researcher analysis, and citation graphs
• CPU/GPU performance benchmark tests
• Automatic build, media compression, and publishing of Hugo blogs
• Document translation driven by local models

Each directory under tools/ is a standalone tool; the common/ in the root directory provides shared capabilities, such as LLM invocation, caching, JSON parsing, atomic writing, translation engines, and Hugo deployment. Development processes and Claude Code skills are not in this repository but in independent repositories ../ai-companion/ in the same directory.

Repository structure

gadget/
├── tools/
│   ├── summarize/   # AI 对话日报 / 周报 / 月报
│   ├── research/    # 论文发现、研究者分析、引用图谱
│   ├── benchmark/   # CPU/GPU 性能测试
│   ├── website/     # Hugo 博客及自动发布
│   └── translator/  # Gradio 文档翻译器
├── common/          # 所有工具共享的基础能力
├── scripts/         # 同步、初始化、检查等运维脚本
├── docs/            # 设计文档、ECL 计划、审计与历史资料
├── outputs/         # 可重新生成的输出文件（已忽略 Git）
├── AGENTS.md        # AI Agent 工作流程协议
├── CLAUDE.md        # Claude Code 仓库说明
└── pyproject.toml   # Python 打包与依赖配置

Generation results are uniformly placed in outputs/:

outputs/
├── logs/      # 导出及运行日志
├── reports/   # Markdown、JSON、HTML 报告
├── cache/     # LLM 与搜索缓存
├── data/      # CSV、JSON 等结构化数据
├── images/    # 图表和生成图片
└── backups/   # 强制覆盖时的备份

Hugo website content is written directly into tools/website/content and tools/website/static.

Purpose of each tool

Usage tutorial

1. Environment preparation

The basic environment specified in the README is:

• Python 3.10+
• It is recommended to use a Conda environment AI
• Node.js 18+: only when using the ../ai-companion/ repository of the same level is required
• Website and translation functions require a local inference engine: Ollama is preferred; vLLM can be used on Linux, and transformers can be used on Windows
• The translation model tencent/Hy-MT2-1.8B will be automatically downloaded during the first use

If using Conda:

conda activate AI

Tool settings are in the root directory config.json; the README states that they should be copied from config.example.json. This configuration file can be overridden by environment variables:

GADGET_CONFIG

Do not commit the following to Git:

1. Project Summary

1. What is Gadget

Gadget is a set of practical development tools composed of multiple independent tools, covering:

• Generation of AI dialogue daily reports, weekly reports, and monthly reports
• Research paper discovery and researcher analysis
• CPU/GPU performance benchmark tests
• Automatic build and publication of Hugo blogs
• Local model-driven document translation

Each subdirectory tools/ is a standalone tool. The common/ in the repository root directory provides shared capabilities, including:

• LLM calls
• JSON parsing and repair
• Disk caching
• Atomic writing
• Local model translation
• Hugo deployment

The development process, code change tracking, and Claude Code skills are provided by ../ai-companion/ outside the repository.

2. Repository Structure

gadget/
├── tools/
│   ├── summarize/      # AI 对话日报、周报、月报
│   ├── research/       # 论文发现、研究者分析、引用图谱
│   ├── benchmark/      # CPU/GPU 性能测试
│   ├── website/        # Hugo 博客
│   └── translator/     # Gradio 文档翻译器
├── common/             # 共享基础设施
├── scripts/            # 同步、初始化、维护脚本
├── docs/               # 设计文档、计划和历史资料
├── outputs/            # 生成的报告、缓存、数据和图片
├── AGENTS.md           # AI Agent 工作流协议
├── CLAUDE.md           # Claude Code 使用说明
└── pyproject.toml      # Python 打包配置

When modifications are needed:

3. Features of Each Tool

summarize

Read AI dialogue records from sources such as Claude Code, Codex, Cursor Agent, and ChatGPT, and call an LLM to generate:

• Daily reports
• Weekly reports
• Monthly reports and trend summaries

It uses a multi-device two-stage workflow:

1. Each device exports dialogue records
2. Aggregate data and generate reports by date

At the same time, ccusage statistics on the token usage and costs of different Agent CLI are performed.

Four LLM backends are supported:

• ollama: Default, local Ollama, no API key required
• claude_cli: Reuses Claude Code login status
• anthropic
• openai

research

Research tool, including four types of capabilities:

• Paper discovery: search arXiv, bioRxiv, PubMed, and perform a three-stage LLM evaluation
• Deep paper insights: analyze writing structure, publishing strategies, core knowledge, and OpenReview review comments
• Researcher profiles: combine ArXiv and Semantic Scholar data to analyze research trajectories, citation patterns, and mentor-student relationships
• Citation graphs: analyze forward citations and backward references in papers

benchmark

Cross-platform CPU/GPU floating-point performance testing tool.

Supported:

• NVIDIA CUDA
• Apple Silicon MPS
• Intel XPU

Multiple precision levels are supported, including FP64, FP32, FP16, BF16, and experimental FP8.

Results are written to CSV in append mode, without overwriting historical data; interactive HTML reports and leaderboards can also be generated.

website

A Hugo-based blog site using PaperMod theme, published on GitHub Pages.

Main functions:- Incremental compression of images and videos

• Automatic translation of bilingual content
• Automatic building of Hugo websites
• Automatic submission and push to deployment repository
• Receiving automatically generated content such as daily reports, weekly reports, research reports, and benchmark pages

Website content is written into:

tools/website/content
tools/website/static

tools/website/public/ is a separate deployment repository and should not be submitted directly to the current project.

translator

A local document translator based on Gradio, supporting:

• Text
• Markdown
• TXT
• PDF
• DOCX
• Images

It will protect Markdown code blocks, URLs, and Hugo shortcodes. The local model tencent/Hy-MT2-1.8B is used by default, and the GGUF version is also supported.

The translation backend is automatically selected by a shared translation engine:

• Ollama
• vLLM
• transformers
• llama.cpp/GGUF

4. common/, scripts/ and outputs/

common/

A shared Python package that is installed for all tools, including LLM, caching, IO, translation, and Hugo deployment functions.

scripts/

The main scripts include:

• sync.py: Use rclone to sync data
• onboard.py: Initialization of new machines
• smoke.sh: Perform read-only smoke checks for all tools
• serve_local_llm.sh: Create an Ollama model variant suitable for summarize
• language.py: Check Hugo bilingual content
• profile_translation.py: Analyze the GPU performance of the translation engine

outputs/

All generated outputs are centralized here:

outputs/
├── logs/
├── reports/
├── cache/
├── data/
├── images/
└── backups/

This directory is ignored by Git and can be automatically regenerated.

2. Chinese Usage Tutorial

1. Environment Preparation

Project requirements:

• Python 3.10+
• Node.js 18+, only ../ai-companion/ is required
• Additional dependencies for each tool can be found in the corresponding directory under requirements.txt

The README recommends using a conda environment named AI:

conda activate AI

If using AI Companion, it needs to be placed in a sibling directory of the current repository:

../ai-companion/

It is an independent repository and does not belong to Gadget itself.

2. Install Basic Dependencies

Execute in the root directory of the repository:

pip install -e .

This will install the shared common/ package from the root directory in an editable mode.

If you want to install dependencies for summarize, research, benchmark, and website, you can execute:

pip install -e ".[all]"

Note: all does not include the translator.

Install translator dependencies:

pip install -e ".[translator]"

If you need website functionality, you can execute:

pip install -e ".[website]"

Website dependencies include PyTorch and transformers required for translation.

3. Configuration Files and Keys

Tool settings in the project root directory are located at:

config.json

The README indicates that this file is ignored by Git; you should copy it from:

config.example.json

Then configure it later.

You can also specify other configuration file paths through environment variables:

GADGET_CONFIG

API Key and initialization configuration are placed at:

tokens/

This directory is also ignored by Git and cannot be committed to the repository.

The LLM backend can be uniformly selected through --api:

• ollama
• claude_cli
• anthropic
• openai

It can also be set through global environment variables:

GADGET_LLM_BACKEND

The translation backend can be controlled through:

GADGET_TRANSLATION_BACKEND

The translation model can be overridden through:

GADGET_TRANSLATION_MODEL

4. Initialization of New Machines

The project provides a one-time initialization script. The README requires filling in:

tokens/onboard.yaml

Then run:

python scripts/onboard.py

This script will automatically handle:

• SSH configuration
• Claude/Codex CLI installation and authentication
• pip extras installation
• ai-companion installation
• Tool configurations
• rclone initialization

The server-level Ubuntu 22.04/24.04 initialization script is located in the ai-companion repository:

bash ../ai-companion/scripts/onboard-server.sh --help

5. Data Sync and Basic Checks

The project uses rclone for cross-device data sync. Check the sync script for help:

python scripts/sync.py

Sync data:

python scripts/sync.py push
python scripts/sync.py pull

Run read-only smoke checks:

bash scripts/smoke.sh

This check will verify help information, information detection, and import for each tool; it does not call the LLM, access the network, or write data.

3. Usage Methods of Each Tool

1. Summarize: Generate AI Dialogue Reports

Daily Report Process

First stage: Export all dates that have not been exported yet:

python -m summarize daily export

Second stage: Sync all dates and merge them day by day:

python -m summarize daily merge --sync-all

Generate weekly reports for a specified week and deploy them to Hugo:

python -m summarize weekly generate --week 2026-W12 --deploy

Generate monthly reports for a specified month and deploy them to Hugo:

python -m summarize monthly generate --month 2026-02 --deploy

One-click execution of the full process

python -m summarize auto --deploy

This command executes:

导出 → 合并 → 周报 → 月报 → 部署

The old entry points are still compatible, for example:

python tools/summarize/daily_summary.py
python tools/summarize/weekly_summary.py
python tools/summarize/monthly_summary.py

But the README recommends using:

python -m summarize

LLM Backend

Summarize uses the local Ollama by default, and also supports:

ollama
claude_cli
anthropic
openai

The specific backend can be switched through the --api parameter.

2. Research: Paper Search and Researcher Analysis

Generate research project reports

python tools/research/research_scout.py report --project my-project

This process will perform paper search, three-stage evaluation, and generate a report.

Use natural language to search for papers

python tools/research/research_scout.py ask "找 Pieter Abbeel 最近的机器人操作论文"

The tool will automatically select data sources based on the question.

Generate researcher profiles

python tools/research/research_scout.py profile "Sergey Levine"

It will obtain paper and citation data and generate research trajectory analysis.

Analyze paper citation graphs

python tools/research/research_scout.py citations 2301.12597

The parameter can be arXiv ID or DOI.

Deploy research reports

python tools/research/research_scout.py deploy

The LLM backend of Research also supports:

ollama
claude_cli
anthropic
openai

It can be uniformly switched through --api.

3. Benchmark: Run CPU/GPU Performance Tests

The README specifically states: All Benchmark commands need to enter the directory first:

cd tools/benchmark

Run all benchmark tests:

python -m benchmark.cli

The results will be appended to CSV.

Run only CPU tests:

python -m benchmark.cli --cpu-only

Run only GPU tests:

python -m benchmark.cli --gpu-only

Run tests and generate an HTML report:

python -m benchmark.cli --report

Run tests, generate a report, and deploy to Hugo:

python -m benchmark.cli --report --deploy

The tools will automatically detect CUDA, Apple MPS, and Intel XPU. OpenCL may only appear in --info detection results, but it does not mean that an OpenCL benchmark test will be executed.

4. Website: Build and Preview Hugo Websites

Install website dependencies:

pip install -e ".[website]"

macOS/Linux: Build and publish

cd tools/website && bash update.sh

This process includes:

1. Incremental compression of media files
2. Execution of Hugo build
3. Push to GitHub Pages

Images are compressed using pngquant, and videos are processed using HandBrakeCLI.

Windows: Build and publish

powershell -ExecutionPolicy Bypass -File tools/website/update.ps1

This script will automatically switch to its own directory.

Local preview

cd tools/website && hugo server -D

-D indicates that the preview includes drafts.

Automatically generated content in the website is directly written into:

tools/website/content
tools/website/static

Automatically generated files marked with gadget_generated can be updated by the deployment process; handwritten files without this mark will not be overwritten.

5. Translator: Start the Local Document Translator

Install translator dependencies:

pip install -e ".[translator]"

Start the Gradio graphical interface:

python -m translator

After starting, a browser will open.

The translator supports text and the following files:

.md
.txt
.pdf
.docx
图片

The default model is:

tencent/Hy-MT2-1.8B

The GGUF version is:

tencent/Hy-MT2-1.8B-GGUF

The model will be automatically downloaded when used for the first time. The translator defaults to using the local Ollama; on Linux, vLLM can be used, and on Windows, transformers can be used back, and it also supports paths for GGUF/llama.cpp with low memory.

The source language can be set to auto, and the target language can be set to auto, and an automatic switch between Chinese and English will occur.

4. Common Precautions

• Files in outputs/ are automatically generated and should not be submitted.
• tokens/ contains keys and initialization information and cannot be submitted.
• build/ and gadget.egg-info/ are build files generated during installation.
• Do not submit tools/website/public/ or website themes, deployment repository directly.
• Cross-device sync requires configuring rclone first.
• Website and translation functions rely on local inference engines and use Ollama by default.
• ../ai-companion/ is an independent repository responsible for the development workflow and skills.
• The project uses the GPL-3 license.

The following content is based only on the README you provided; commands or parameters not mentioned in the tutorial will not be added.

1. Project Summary

Gadget is a collection of practical tools accumulated in daily development, covering scenarios such as AI dialogue daily/weekly/monthly reports generation, paper discovery and researcher analysis, performance benchmarking, Hugo blog automation, and document translation.

Its core design is that each directory under tools/ is an independent tool; the common/ in the repository root directory is a shared infrastructure package installed via pip install -e ., providing general capabilities such as LLM invocation, JSON parsing, disk caching, atomic writing, translation, and Hugo deployment for all tools. The development workflow and Claude Code skills are not in this repository but provided by an independent repository in a neighboring directory ../ai-companion/ and accessed through hooks.

Repository Structure

gadget/
├── tools/              # 五个独立工具
│   ├── summarize/      # AI 对话日报 / 周报 / 月报
│   ├── research/       # 论文发现、研究者分析、引用图
│   ├── benchmark/      # CPU/GPU 性能基准测试
│   ├── website/        # Hugo 博客，媒体压缩，自动构建发布
│   └── translator/     # Gradio 文档翻译器
├── common/             # 共享基础设施包
├── scripts/            # 运维与维护脚本
├── docs/               # 设计文档、ECL 计划、审计报告、历史归档
├── outputs/            # 所有生成产物，gitignored，可重新生成
├── AGENTS.md           # AI agent 工作流协议
├── CLAUDE.md           # Claude Code 仓库指南
└── pyproject.toml      # common 和各工具的 Python packaging 配置

outputs/ is used to store generated results collectively:

outputs/
├── logs/         # 中间日志和运行日志
├── reports/      # 最终报告，Markdown / JSON / HTML
├── cache/        # LLM 缓存、搜索缓存
├── data/         # CSV、JSON profiles 等结构化数据
├── images/       # 图表和生成图片
└── backups/      # 强制覆盖备份

Hugo website content is not written into outputs/site initially but directly written into tools/website/content|static by the deployment process.

The following is organized solely based on README.md you posted, without additional commands, parameters, or behaviors beyond the README.

What is the project?

Gadgets is a “set of practical tools accumulated during daily development”. The repository splits different functions into 5 independent tools, each directory can be used separately, while common capabilities are encapsulated in the common/ package in the root directory.

Its coverage mainly includes:

• Generation of AI dialogue daily/weekly/monthly reports
• Paper discovery and researcher analysis
• CPU/GPU performance benchmarking
• Automated Hugo blog publishing
• Document translation

There is also an important prerequisite: the development workflow and Claude Code skills are not in this repository, but in a separate repository ../ai-companion/ nearby, accessed via hooks.

Repository Structure

gadget/
├── tools/              # 5 个独立工具
│   ├── summarize/      # AI 对话日报 / 周报 / 月报
│   ├── research/       # 论文发现 + 研究者分析 + citation graph
│   ├── benchmark/      # CPU/GPU 性能基准测试
│   ├── website/        # Hugo 博客
│   └── translator/     # Gradio 文档翻译器
├── common/             # 通用基础能力：LLM / cache / IO / translation / Hugo
├── scripts/            # 运维和维护脚本
├── docs/               # 设计文档、计划、归档
├── outputs/            # 所有生成产物
├── AGENTS.md           # 给 AI agent 的工作协议
├── CLAUDE.md           # Claude Code 仓库指南
└── pyproject.toml      # 包装配置

What each tool does

• tools/summarize/: Reads AI dialogue records and generates daily/weekly/monthly summaries.
• tools/research/: Performs paper search, researcher profiling, and citation relationship analysis.
• tools/benchmark/: Conducts cross-platform CPU/GPU performance testing and generates reports.
• tools/website/: Automated Hugo blog building, compression, and publishing.
• tools/translator/: Gradio document translator, supporting text and file translation.

common/ is a basic library shared by all tools, providing:

• LLM invocation
• JSON parsing and fixing
• Disk caching
• Atomic writing
• Translation engine
• Hugo deployment

scripts/ contains some ops scripts:

• sync.py: rclone synchronization
• onboard.py: New machine initialization
• smoke.sh: Read-only smoke test
• serve_local_llm.sh: Create summarize-tuned Ollama variant
• Others: language.py, profile_translation.py

Newbie Usage Guide

The following is written according to “Environment preparation → Installation → Configuration → Usage of each tool”.

1. Environment Preparation

The README explicitly requires:

• Python 3.10+
• Recommended to use conda activate AI
• Node.js 18+ is only for the nearby ../ai-companion/ repository
• ../ai-companion/ is an independent repository, not within the current repository
• Specific dependencies needed are based on requirements.txt in each tool directory

If you only plan to use the Python tools in this repository, focus on the Python environment; if you also need to use ai-companion, prepare Node.js.

2. Installation

The installation methods provided in the README have three levels:

pip install -e .

This will install the common/ package and basic capabilities of the repository in the root directory.

If you need to install additional dependencies for most tools, you can use:

pip install -e ".[all]"

The README indicates:

• all includes summarize / research / benchmark / website
• all does not include translator

If you need a translator, install it separately:

pip install -e ".[translator]"

Both website translation function and translator depend on a local inference engine; the README mentions:

• Default is Ollama
• vLLM can be used on Linux
• transformers can be used on Windows
• The model tencent/Hy-MT2-1.8B will be automatically downloaded during the first run

3. Configuration

The configuration points mentioned in the README are as follows:

• There is config.json in the root directory
• config.json is gitignored by default
• Can be copied from config.example.json
• The configuration file path can be overridden using GADGET_CONFIG

In terms of environment variables:

• GADGET_LLM_BACKEND: Unified switching of LLM backend
• GADGET_TRANSLATION_MODEL: Specify translation model
• GADGET_TRANSLATION_BACKEND: Specify translation backend

tokens/ directory:

• Stores API keys and onboarding table
• Also gitignored
• Cannot be submitted

If you are initializing a new machine, the README also mentions:

• Fill in tokens/onboard.yaml
• Then run scripts/onboard.py once
• It will automatically handle SSH, Claude/Codex authentication, pip extras, ai-companion installation, tool configuration, and rclone bootstrap

The README does not expand on the full command details here, so I only keep the explicitly written content without adding additional commands.

4. Typical Usage of Each Tool

4.1 summarize: AI Dialogue Daily/Weekly/Monthly Report

This tool will automatically read your daily AI dialogue records and support:

• Claude Code
• Codex
• Cursor Agent
• ChatGPT
• General JSON

It will also combine ccusage 20.x multi-source commands to count tokens and costs.

The recommended commands given in the README are:

python -m summarize daily export
python -m summarize daily merge --sync-all
python -m summarize weekly generate --week 2026-W12 --deploy
python -m summarize monthly generate --month 2026-02 --deploy
python -m summarize auto --deploy

These commands represent:

• daily export: Export all unexported dates
• daily merge --sync-all: Sync all dates and merge by day
• weekly generate --week ... --deploy: Generate weekly report and deploy
• monthly generate --month ... --deploy: Generate monthly report and deploy
• auto --deploy: One-click full process, according to the README, it is export → merge → weekly → monthly + deploy

Swordable LLM backends:

• ollama, default
• claude_cli
• anthropic
• openai

The README also mentions that the old entry is still available, but the new form is recommended:

python tools/summarize/daily_summary.py ...
python tools/summarize/weekly_summary.py ...
python tools/summarize/monthly_summary.py ...

4.2 research: Paper Discovery, Researcher Analysis, Citation Graph

The unified entry for this tool is:

python tools/research/research_scout.py

The typical commands given in the README are:

python tools/research/research_scout.py report --project my-project
python tools/research/research_scout.py ask "找 Pieter Abbeel 最近的机器人操作论文"
python tools/research/research_scout.py profile "Sergey Levine"
python tools/research/research_scout.py citations 2301.12597
python tools/research/research_scout.py deploy

These commands perform respectively:

• report: Full process, search → three-stage evaluation → report
• ask: Natural language search, automatic routing of data sources
• profile: Researcher profiling
• citations: Citation graph analysis, supporting arXiv ID / DOI
• deploy: Deploy the report in Hugo

同样支持 switching LLM backends:

• ollama
• claude_cli
• anthropic
• openai

4.3 benchmark: Performance Benchmarking

The README emphasizes that you must enter the directory before running this tool.

cd tools/benchmark
python -m benchmark.cli
python -m benchmark.cli --cpu-only
python -m benchmark.cli --gpu-only
python -m benchmark.cli --report
python -m benchmark.cli --report --deploy

These commands represent respectively:

• Default: Run all benchmarks, results appended to CSV
• Only CPU
• Only GPU
• Run benchmarks and generate HTML report
• Generate report and publish in Hugo’s /benchmark/

The README also indicates that it supports:

• NVIDIA CUDA
• Apple Silicon MPS
• Intel XPU

And some precision levels:

• FP64
• FP32
• FP16
• BF16
• FP8 (experimental)

4.4 website: Hugo Blog

This tool is a Hugo static blog site; the README provides three common operations.

Install website-related dependencies:

pip install -e ".[website]"

Building and publishing:

• macOS / Linux:

cd tools/website && bash update.sh

• Windows:

powershell -ExecutionPolicy Bypass -File tools/website/update.ps1

Local preview:

cd tools/website && hugo server -D

The README also indicates:

• It will perform incremental image/video compression
• Images use pngquant
• Videos use HandBrakeCLI
• Automatically generated content will be directly written into tools/website/content or tools/website/static
• tools/website/public/ is a separate deployment repository, do not submit directly

4.5 translator: Gradio Document Translator

Install dependencies:

pip install -e ".[translator]"

Start the GUI:

python -m translator

This tool will open the local translation page in the browser, supporting:

• Text
• .md
• .txt
• .pdf
• .docx
• Images

The README specifically mentions:

• It preserves Markdown format
• Protects code blocks, URLs, Hugo shortcodes
• It is local inference, not cloud API
• The default model is tencent/Hy-MT2-1.8B
• It can also be modified via GADGET_TRANSLATION_MODEL or model management inside the GUI

—5. Operations and Auxiliary Scripts

These are also explicitly mentioned in the README and can be used directly.

sync.py:

python scripts/sync.py

The README states that it is responsible for rclone synchronization, covering:

• summarize
• website
• research
• benchmark
• backups
• There is also a dag category

smoke.sh:

bash scripts/smoke.sh

This is a read-only smoke test, without LLM, network, or writing functionality.

serve_local_llm.sh:

eval "$(bash scripts/serve_local_llm.sh env)"

The README indicates that it will create a Ollama variant optimized for summarize and output environment variables.

ai-companion-related server onboarding:

bash ../ai-companion/scripts/onboard-server.sh --help

The README explains that this script has been migrated to the ../ai-companion/ repository.

6. A Shortest Onboarding Process

If you just want to get it running initially, the shortest path supported by the README is:

pip install -e .
pip install -e ".[all]"

If using a translator, add:

pip install -e ".[translator]"

Then select the tool based on what you want to do:

python -m summarize auto --deploy
python tools/research/research_scout.py report --project my-project
cd tools/benchmark && python -m benchmark.cli --report
cd tools/website && hugo server -D
python -m translator

If you prefer, I can next organize this content into a “more formal document” style Chinese manual, in three versions:

1. A long-form version suitable for beginners
2. A concise version suitable for project documentation
3. A reordered version by “Installation / Configuration / Usage / Common Issues”

1. Project Summary

What is it

Gadgets is a collection of practical tools accumulated in daily development, covering AI daily report generation, paper discovery, performance benchmarking, blog automation, and document translation. Each directory under tools/ is a standalone tool; the shared infrastructure is placed in the common/ package at the root of the repository (installed via pip install -e .), providing general capabilities such as LLM invocation, JSON parsing, disk caching, atomic writing, translation, and Hugo deployment.

The development workflow (planning/execution/changelog tracking) and all Claude Code skills are provided by ai-companion — an independent repository([git@github.com:TzJ2006/ai-companion.git), checked out in the same directory ../ai-companion/, and integrated into this repository via hooks. The license is GPL-3.

Repository Structure

gadget/
├── tools/              # 五个独立工具产品(改工具来这里)
│   ├── summarize/      # AI 对话日报 / 周报 / 月报
│   ├── research/       # 论文发现 + 研究者分析 + 引用图谱
│   ├── benchmark/      # CPU/GPU 性能基准测试
│   ├── website/        # Hugo 博客(增量媒体压缩 + 自动构建发布)
│   └── translator/     # Gradio 文档翻译器
├── common/             # 共享基础设施包(LLM / 缓存 / IO / 翻译 / Hugo)—— 所有工具依赖它
├── scripts/            # 运维 + 维护脚本(sync.py, onboard.py, language.py, profile_translation.py, …)
├── docs/               # 设计文档、ECL 计划(docs/ecl/)、审计报告、历史归档(docs/archive/)
├── outputs/            # 所有生成产物(gitignored,可自动重建)
├── AGENTS.md           # 所有 AI agent 的工作流协议(行动前先读)
├── CLAUDE.md           # 给 Claude Code 的仓库指南
└── pyproject.toml      # common/ 与各工具的打包配置(pip install -e .)

build/ and gadget.egg-info/ are generated by pip install -e . (already in gitignore, always located at the repository root).

Where to go for changes:

What each tool does

tools/summarize/ —— AI dialogue daily/weekly/monthly reports

Automatically reads daily AI dialogue records (Claude Code / Codex / Cursor Agent / ChatGPT / general JSON), calls the LLM API to generate structured daily, weekly, and monthly reports. Multi-device workflow: each device exports dialogue logs → synchronized via cloud drive or manually copied summary → final daily report generated; after a certain amount of reports are accumulated, weekly and monthly trend summaries are generated. Using ccusage 20.x commands named by source namespace([ccusage claude, ccusage codex, ccusage gemini…) automatically discovers and counts the token usage and costs of all agent CLI commands. Four AI summarization backends, uniformly switched using --api. Key technologies: Claude/OpenAI API, ccusage 20.x (multi-source unification), matplotlib.

tools/research/ —— Paper discovery and researcher analysis

A unified academic research toolkit, providing four major capabilities through a single CLI entry tools/research/research_scout.py:

• Paper discovery: Search for papers from arXiv / bioRxiv / PubMed, follow a three-stage LLM pipeline (quick screening → deep evaluation → citation impact), generate weekly research reports and deploy them into Hugo. Supports search for conference papers and author searches.
• Deep paper insights([--insight): Download full paper texts, let the LLM analyze their writing structure, publishing strategy, and reusable core knowledge; automatically match OpenReview to obtain review comments, analyze consensus and disputes; synthesize research writing guidelines across papers.
• Researcher profiles: Fetch paper and citation data from ArXiv + Semantic Scholar, let the LLM generate research trajectory analysis, calculate tier scores, and automatically discover teacher-student relationships through主页 extraction + co-authorship patterns. Supports name-based disambiguation and reverse lookup.
• Citation graph analysis: Forward citation / backward reference analysis based on Semantic Scholar API, combined with LLM-driven impact interpretation.

tools/benchmark/ —— Performance benchmarking suite

Cross-platform CPU/GPU FLOPS benchmarking tools, uniformly measure floating-point performance across different hardware vendors and precision levels. Supports actual GPU operations on NVIDIA(CUDA), Apple Silicon(MPS), Intel(XPU) (OpenCL may only appear in --info detection), covering FP64 / FP32 / FP16 / BF16 / FP8 (experimental). Measurements use preheating + formal measurement + statistical analysis (median, IQR outlier removal), and explicit GPU synchronization to ensure accurate timing. Results are accumulated in append mode into CSV (never overwritten, naturally supports multi-hardware cumulative leaderboards), can generate interactive HTML reports with Plotly charts and leaderboards, supports deployment to Hugo websites and submission to public leaderboards.

tools/website/ —— Hugo blog

Hugo static blog site (“TzJ’s Net”, PaperMod theme), deployed to GitHub Pages([https://tzj2006.github.io/). Built-in incremental image/video compression pipeline (only compresses media changed since .last_build): images use pngquant, videos use HandBrakeCLI) and local model bilingual translation (default Ollama, vLLM on Linux / transformers on Windows as fallback; models tencent/Hy-MT2-1.8B automatically downloaded on first run, without relying on cloud LLM API). The automatically generated content (daily/weekly/monthly reports, research reports, benchmark pages, images) are directly written into tools/website/content|static(the only Hugo content root, files marked with gadget_generated; handwritten files without marks never overwrite), then translated, compressed, built, and pushed.

⚠️ tools/website/public/ is an independent deployment repository([tzj2006/tzj2006.github.io), automatically submitted + pushed by the build script — do not submit directly.

tools/translator/ —— Gradio document translator

Gradio document translator based on common translation engine: a local web page in Google Translate style for translating text and files([.md / .txt / .pdf / .docx / images), while preserving Markdown format (code blocks, URLs, Hugo shortcodes and other segments are protected). It runs local inference instead of cloud API: common.engine.create_engine() is reused to automatically select the backend (model tag pulled when ollama; transformers is Windows fallback, vllm, llamacpp is low memory GGUF option), model remains in memory (warm), and lazy loading occurs when switching models. Default model tencent/Hy-MT2-1.8B(GGUF variant tencent/Hy-MT2-1.8B-GGUF), automatically downloaded on first use, can be overridden via GADGET_TRANSLATION_MODEL or model management within the GUI. Source language can be set to auto(detected by text CJK ratio); when target language is auto, it flips between zh↔en.

common/ —— Shared infrastructure package

The shared layer relied on by all tools, installed as a Python package via pip install -e .:

• LLM invocation (unified two-layer API, supports ollama(default)/ claude_cli / anthropic / openai four backends)
• JSON parsing and repair
• SHA-256 disk caching (namespace + TTL)
• Atomic writing and content hash
• Local inference translation engine (Ollama / vLLM / transformers / GGUF) and bilingual content generation
• Cross-platform Hugo deployment

scripts/ —— Operations and maintenance scripts- sync.py —— Centralized rclone data synchronization (push / pull / status / bootstrap / config); there is a special dag category for generating and deploying DAG sites). Configured as the config.json section at the repository root sync (the GADGET_CONFIG can be used for path overriding).

• onboard.py —— Repository-level one-time machine initialization: fill a YAML table (tokens/onboard.yaml), run a script once, which automatically completes SSH configuration, Claude/Codex CLI installation and authentication, pip extras and ai-companion installation, tool configurations, and rclone bootstrap.
• smoke.sh —— Read-only smoke checks covering all tools (--help / --info / import); does not involve LLM, network, or writing.
• serve_local_llm.sh —— Create an Ollama variant optimized for summarize and print environment variables. Persistent switches are placed in the config.json section of the repository root summarize.
• Others: language.py(Hugo bilingual audit + renaming of summarize reports), profile_translation.py(translation engine GPU profiler).

../ai-companion/ —— Independent repository

An independent Node.js repository, checked out in the same directory ../ai-companion/, not within this repository. It records function-level code changes, generates tests and HTML reports, provides a planning-execution pipeline of /idea→/ccplan→/ccedit, and hosts all Claude Code skills (methodologies + domains). This repository is accessed via hooks under .claude/ and .codex/.

It is packaged as both a standard Claude Code plugin and an equivalent Codex integration — installed immediately, no build required (the change-tracking hook is a pre-built single-file product submitted with the repository). All skills (methodologies: ccplan planning, optimize optimization, cchypothesis debugging, repo-audit, repo-tidy; domains: summarize code overview, slurm-gpu cluster detection, nature-benchmark / NIPS-2025-paper paper writing advisor) are in that repository; the skills/ directory is no longer present in this repository. Installation can be found at ../ai-companion/scripts/install.ts.

outputs/ —— Output directory

All generated outputs are placed uniformly in outputs/(gitignore, can be automatically rebuilt):

outputs/
├── logs/         # 中间产物(导出日志、运行日志)
├── reports/      # 最终报告(Markdown、JSON、HTML)
├── cache/        # LLM 缓存、搜索缓存
├── data/         # 结构化数据(CSV、JSON profiles)
├── images/       # 图表和生成的图片
└── backups/      # 强制覆盖备份(website-force 等)

Hugo site content is directly written into tools/website/content|static(no independent outputs/site staging tree).

2. Usage Guide (for first-time users)

This guide covers only the content mentioned in the README. For step-by-step operations, configurations, data formats, and FAQs of each tool, please refer to TUTORIAL.md in the repository and the source documents of each tool.

Step 0: Environment Preparation

• Python 3.10+ (it is recommended to use a conda environment AI)
• Node.js 18+ (only required for the independent repository ../ai-companion/)
• Specific dependencies of each tool can be found in the corresponding directory under requirements.txt
• The website / translation functions use local inference engines (default: Ollama; fallback on Linux: vLLM, on Windows: transformers). The model tencent/Hy-MT2-1.8B is automatically downloaded during the first run.

Activate the recommended environment:

conda activate AI

Step 1: Installation

Install the common package plus the extras for summarize / research / benchmark / website:

pip install -e ".[all]"

⚠️ Note: all does not include translator; it needs to be installed separately:

pip install -e ".[translator]"

If you only want to use a specific tool, you can install only the corresponding extra:

pip install -e ".[website]"      # website(含 torch + transformers 用于翻译)
pip install -e ".[translator]"   # translator(gradio + GGUF 翻译栈)

After installation, build/ and gadget.egg-info/ will be generated at the repository root (gitignore, normal behavior).

Step 2: Configuration

Tool configuration file: The config.json section at the repository root (gitignore), copied from config.example.json:

cp config.example.json config.json

If you want a different path, set environment variables for overriding:

export GADGET_CONFIG=/path/to/your/config.json

The sync section is the configuration for scripts/sync.py, and the summarize section is the persistent switch for scripts/serve_local_llm.sh.

API key: Place it in the tokens/ directory (gitignore, do not commit its content).

LLM backend selection: All LLM functions are switched using the --api parameters, with four values:

You can also set it globally using environment variables:

export GADGET_LLM_BACKEND=ollama

Translation backend: The translation uses a local inference engine, selected by GADGET_TRANSLATION_BACKEND: ollama (default when the model is downloaded, uses local Ollama server) → llamacpp / vllm / transformers (within the process), with the model being tencent/Hy-MT2-1.8B. The translation model can be overridden:

export GADGET_TRANSLATION_BACKEND=ollama
export GADGET_TRANSLATION_MODEL=tencent/Hy-MT2-1.8B

Step 3 (optional): One-time initialization on a new machine

After filling in the tokens/onboard.yaml YAML table, run it once:

python scripts/onboard.py

It will automatically complete SSH configuration, Claude/Codex CLI installation and authentication, pip extras and ai-companion installation, tool configurations, and rclone bootstrap.

Server system-level installation (Ubuntu 22.04/24.04, one-click install of SSH public key, Claude Code, Codex, Superpowers, Ponytail, and AI Companion) has been migrated to the ai-companion repository:

bash ../ai-companion/scripts/onboard-server.sh --help

After the repository is split, --companion-repo can be directly sent to the ai-companion repository (git@github.com:TzJ2006/ai-companion.git), and --companion-ref is no longer needed.

Step 4: Run a smoke check first

Read-only, does not involve LLM/network/writing, used to confirm installation:

bash scripts/smoke.sh

Tool 1: Summarize —— AI dialogue daily/weekly/monthly reports

It is recommended to use the python -m summarize format. The typical process is two-stage: first export on each device, then aggregate and merge.

python -m summarize daily export                                   # 阶段 1:导出所有尚未导出的日期
python -m summarize daily merge --sync-all                         # 阶段 2:同步所有日期并逐日合并
python -m summarize weekly generate --week 2026-W12 --deploy       # 周报 + 部署
python -m summarize monthly generate --month 2026-02 --deploy      # 月报 + 部署
python -m summarize auto --deploy                                  # 一键全流程:导出 → 合并 → 周报 → 月报 + 部署

Multi-device usage: run daily export on each device, synchronize via cloud storage or manually copy logs together, then run daily merge --sync-all to generate the final daily report; after enough daily reports are collected, generate weekly reports and monthly trend summaries.

Switch the AI summary backend:

python -m summarize auto --deploy --api claude_cli

If you want to use the optimized variant of summarize from local Ollama:

eval "$(bash scripts/serve_local_llm.sh env)"

The old entry points python tools/summarize/daily_summary.py ... / weekly_summary.py / monthly_summary.py are still available (backward-compatible re-export shim), but the new format above is recommended.

See TUTORIAL.md#summarize and tools/summarize/tutorial.md for details.

Tool 2: Research —— Paper discovery and researcher analysis

A single CLI entry point tools/research/research_scout.py:

python tools/research/research_scout.py report --project my-project        # 全流程:搜索 → 三阶段评估 → 报告
python tools/research/research_scout.py ask "找 Pieter Abbeel 最近的机器人操作论文"  # 自然语言搜索(自动路由数据源)
python tools/research/research_scout.py profile "Sergey Levine"             # 研究者画像
python tools/research/research_scout.py citations 2301.12597               # 引用图谱(按 arXiv ID / DOI)
python tools/research/research_scout.py deploy                              # 把报告部署到 Hugo

For deep insights, use --insight; all LLM functions support switching backends via --api, for example:

python tools/research/research_scout.py report --project my-project --api claude_cli

See TUTORIAL.md#research and tools/research/TUTORIAL.md for details.

Tool 3: Benchmark —— Performance benchmark testing

⚠️ All commands need to first go through cd into tools/benchmark/.

cd tools/benchmark
python -m benchmark.cli                    # 跑全部基准测试(结果追加到 CSV)
python -m benchmark.cli --cpu-only         # 只跑 CPU
python -m benchmark.cli --gpu-only         # 只跑 GPU
python -m benchmark.cli --report           # 跑测试 + 生成 HTML 报告
python -m benchmark.cli --report --deploy  # 生成报告 + 发布到 Hugo /benchmark/

If you only want to view hardware detection results without actually running, use --info (note that OpenCL may only appear in --info detection, but does not mean it can run actually). GPU benchmark tests automatically detect CUDA / Apple MPS / Intel XPU. Results are accumulated in append mode and do not overwrite history, so they can be run on multiple machines and accumulated into a ranking.

See TUTORIAL.md#benchmark and tools/benchmark/tutorial.md for details.

Tool 4: Website —— Hugo blog

pip install -e ".[website]"                          # 安装依赖(含 torch + transformers 用于翻译)
cd tools/website && bash update.sh                   # macOS/Linux:增量压缩 + Hugo 构建 + 推送 Pages
powershell -ExecutionPolicy Bypass -File tools/website/update.ps1  # Windows(脚本会自己 cd 到所在目录)
cd tools/website && hugo server -D                   # 本地预览(dev server,包含草稿)

Bilingual audit and renaming of summarize reports:

python scripts/language.py

Two critical rules to remember:

1. tools/website/public/ is an independent deployment repository, automatically pushed by the build script — do not submit directly.
2. Automatically generated files are marked with gadget_generated; handwritten files without the mark will never be overwritten — so handwritten content is safe, but do not add that mark to handwritten files.

See TUTORIAL.md#website and tools/website/CLAUDE.md for details.

Tool 5: Translator —— Gradio document translator

pip install -e ".[translator]"   # 安装依赖(gradio + GGUF 翻译栈)
python -m translator             # 启动 Gradio GUI(在浏览器中打开)

After starting, you can translate text in the web page, or upload .md / .txt / .pdf / .docx / image files. Select auto for the source language, which will automatically detect the CJK proportion of the text; select auto for the target language, which will flip between Chinese and English. You can switch models in the model management within the GUI, or set environment variables before starting:

export GADGET_TRANSLATION_MODEL=tencent/Hy-MT2-1.8B-GGUF
python -m translator

The model is automatically downloaded during first use, and all operations are performed locally, without using cloud API.

--infoUI wiring can be found in tools/translator/app.py, translation and file logic in tools/translator/core.py, shared engine in common/engine.py and common/translation.py. For details, see TUTORIAL.md#translator.

Cross-device Data Synchronization

python scripts/sync.py           # 查看用法
python scripts/sync.py push      # 推送
python scripts/sync.py pull      # 拉取

Sub-commands include push / pull / status / bootstrap / config. Categories cover summarize / website / research / benchmark / backups. There is also a special dag category used to generate and deploy DAG sites. rclone must be configured first; the configuration is in the sync section of config.json.

Common Precautions (Must Follow)

• tokens/ stores API keys and onboarding tables, already in gitignore — never submit any content from it.
• All generated files are output to outputs/, already in gitignore.
• Never git add: automatically generated content, data synchronized by rclone, build artifacts (build/, gadget.egg-info/), and deployment and theme repositories under tools/website/.
• Read AGENTS.md(the workflow protocol for all AI agents) before using them.
• This project is licensed under GPL-3 (see LICENSE).

Project Summary

Gadget is a collection of practical tools accumulated during daily development, covering five categories: generating AI dialogue daily reports, paper discovery, performance testing, blog automation, and document translation.

Repository Structure

gadget/
├── tools/              # 五个独立工具产品
│   ├── summarize/      # AI 对话日/周/月报
│   ├── research/       # 论文发现 + 研究者分析 + 引用图谱
│   ├── benchmark/      # CPU/GPU 性能测评
│   ├── website/        # Hugo 博客（增量媒体压缩 + 自动构建发布）
│   └── translator/      # Gradio 文档翻译器
├── common/             # 共享基础设施包（LLM 调用/缓存/IO/翻译/Hugo 部署），被所有工具依赖
├── scripts/            # 运维维护脚本（sync.py、onboard.py、language.py、profile_translation.py 等）
├── docs/               # 设计文档、ECL 计划（docs/ecl/）、审计报告、历史归档（docs/archive/）
├── outputs/            # 所有生成产物（已 gitignore，可自动重建）
├── AGENTS.md           # 所有 AI agent 的工作流协议
├── CLAUDE.md           # Claude Code 仓库指南
└── pyproject.toml      # common/ + 各工具的打包配置（pip install -e .）

The development workflow engine and all Claude Code skills are located in the independent repository ai-companion (in the same directory as ../ai-companion/). They are accessed through hooks under .claude/ and .codex/.

What Each Tool Does

All LLM functions can switch the backend through --api: ollama (default, no key required locally), claude_cli (reuses Claude Code login state), anthropic, openai.

Usage Guide

1. Environment Preparation

• Python 3.10+ (it is recommended to use a conda environment AI)

conda activate AI

• Node.js 18+ (only when using the same repository ../ai-companion/)
• GPU testing automatically detects CUDA / Apple MPS / Intel XPU
• tokens/ directory stores API keys and onboarding configuration tables, already in gitignore, do not submit

2. Installation

Install the common package + dependencies of summarize/research/benchmark/website (all does not include translator):

pip install -e ".[all]"

If you need the translator, install it separately:

pip install -e ".[translator]"

If you need the website tools (including torch + transformers for translation):

pip install -e ".[website]"

3. Configuration

Tool configurations are stored in the root directory of the repository config.json (already in gitignore, copied from config.example.json), and can be overridden via environment variables GADGET_CONFIG.

One-time configuration for a new machine:

# 先填写 tokens/onboard.yaml，然后运行：
python scripts/onboard.py

This script automatically completes SSH configuration, Claude/Codex CLI installation and authentication, pip extras and ai-companion installation, tool configurations, and rclone bootstrap.

System-level installation on a server (Ubuntu 22.04/24.04), migrated to the ai-companion repository:

bash ../ai-companion/scripts/onboard-server.sh --help

4. Typical Usage of Each Tool

1. Summarize —— AI Dialogue Daily/Weekly/Monthly Reports

python -m summarize daily export                                   # 第一阶段：导出所有未导出的日期
python -m summarize daily merge --sync-all                         # 第二阶段：同步所有日期并逐日合并
python -m summarize weekly generate --week 2026-W12 --deploy       # 生成周报并部署
python -m summarize monthly generate --month 2026-02 --deploy      # 生成月报并部署
python -m summarize auto --deploy                                  # 一键全流程：导出→合并→周报→月报+部署

The old interfaces python tools/summarize/daily_summary.py ... / weekly_summary.py / monthly_summary.py are still available (backward compatible), but it is recommended to use the above python -m summarize format.

2. Research —— Paper Discovery and Researcher Analysis

python tools/research/research_scout.py report --project my-project        # 完整流水线：搜索→三阶段评估→生成报告
python tools/research/research_scout.py ask "找 Pieter Abbeel 最近的机器人操作论文"  # 自然语言搜索（自动路由数据源）
python tools/research/research_scout.py profile "Sergey Levine"             # 研究者画像
python tools/research/research_scout.py citations 2301.12597               # 引用图谱（按 arXiv ID / DOI）
python tools/research/research_scout.py deploy                              # 部署报告到 Hugo

3. Benchmark —— Performance Testing

Note: All commands need to be cd tools/benchmark/ first.

cd tools/benchmark
python -m benchmark.cli                # 运行所有测评（结果追加到 CSV）
python -m benchmark.cli --cpu-only     # 仅 CPU
python -m benchmark.cli --gpu-only     # 仅 GPU
python -m benchmark.cli --report       # 运行测评 + 生成 HTML 报告
python -m benchmark.cli --report --deploy  # 生成报告 + 发布到 Hugo /benchmark/

4. Website —— Hugo Blog

pip install -e ".[website]"                          # 安装依赖（含 torch + transformers 用于翻译）
cd tools/website && bash update.sh                   # macOS/Linux：增量压缩 + Hugo 构建 + 推送 Pages
powershell -ExecutionPolicy Bypass -File tools/website/update.ps1  # Windows（脚本会自行 cd 到自身目录）
cd tools/website && hugo server -D                   # 本地预览（开发服务器，含草稿）

5. Translator —— Document Translator

pip install -e ".[translator]"   # 安装依赖（gradio + GGUF 翻译栈）
python -m translator             # 启动 Gradio 图形界面（浏览器打开）

The default model is tencent/Hy-MT2-1.8B (automatically downloaded on first use), and can be overridden via environment variables GADGET_TRANSLATION_MODEL or model management in the GUI.

5. Common Operations Scripts

python scripts/sync.py          # 集中式 rclone 数据同步（push/pull/status/bootstrap/config）
bash scripts/smoke.sh           # 全工具只读冒烟测试（--help/--info/imports，无 LLM、网络、写入）
eval "$(bash scripts/serve_local_llm.sh env)"   # 创建 summarize 专用 Ollama 变体并打印环境变量

6. Other Notes

• The output directory is unified in outputs/ (subdirectory logs/reports/cache/data/images/backups), already in gitignore, can be automatically rebuilt.
• The LLM backend can be switched via --api parameters or global environment variables GADGET_LLM_BACKEND: ollama (default), claude_cli, anthropic, openai
• The translation backend is selected by GADGET_TRANSLATION_BACKEND: default ollama (when the model is downloaded) → llamacpp/vllm/transformers
• Cross-device data synchronization: python scripts/sync.py push/pull (requires rclone configuration)
• Never git add automatically generate content, data synchronized by rclone, build artifacts (build/, gadget.egg-info/) and deployment/theme repositories under tools/website/
• License is GPL-3 (see LICENSE)
• For more detailed step-by-step instructions, configuration items, data formats, and FAQs, refer to TUTORIAL.md and the documentation in each tool’s directory

Gadget Project Summary and Usage Guide

The following content is based solely on the README you provided, without adding any commands or parameters not mentioned in the README.

1. Project Summary

1. What is this?

Gadgets is a collection of practical tools accumulated during daily development, covering:

• Generating AI dialogue daily/weekly/monthly reports
• Paper discovery and research analysis
• Performance benchmarking
• Blog automation
• Document translation

Features:

• Each directory under tools/ is a standalone tool
• Shared capabilities are concentrated in the repository root directory’s common/ package (installed via pip install -e .), including LLM calls, JSON parsing, disk caching, atomic writing, translation, Hugo deployment, etc.
• The development workflow (planning/execution/changelog tracking) and all Claude Code skills are provided by the independent repository ai-companion (located in the same directory ../ai-companion/), accessed through hooks

More detailed step-by-step instructions, configurations, data formats, and FAQs can be found in TUTORIAL.md and each tool’s own documentation.

2. Repository Structure

gadget/
├── tools/              # 五个独立工具（改工具逻辑来这里）
│   ├── summarize/      # AI 对话日报/周报/月报
│   ├── research/       # 论文发现 + 研究者分析 + 引用图
│   ├── benchmark/      # CPU/GPU 性能基准测试
│   ├── website/        # Hugo 博客（增量媒体压缩 + 自动构建发布）
│   └── translator/     # Gradio 文档翻译器
├── common/             # 共享基础设施包（所有工具依赖）
├── scripts/            # 运维与维护脚本
├── docs/               # 设计文档、ECL 计划、审计报告、历史归档
├── outputs/            # 所有生成物（gitignored，可自动重建）
├── AGENTS.md           # AI agent 工作流协议
├── CLAUDE.md           # Claude Code 仓库指南
└── pyproject.toml      # common/ 与各工具的打包配置

Where to go for changes:

Other Notes:- build/, gadget.egg-info/ generated by pip install -e . and gitignoreed

• Data synchronization: python scripts/sync.py
• Chinese version documentation: docs/README.zh.md

3. Overview of Tools

4. What Each Tool Does

Summarize — AI conversation daily/weekly/monthly reports

• Automatically reads daily AI conversation records (Claude Code / Codex / Cursor Agent / ChatGPT / general JSON)
• Calls LLM to generate structured daily, weekly, and monthly reports
• Multi-device workflow: Each device exports conversations → synchronized to cloud disk or manual copy → merged to generate reports → accumulated to produce weekly/monthly reports
• Automatically discovers and counts token usage and costs of each agent CLI via ccusage 20.x
• LLM backend: --api, ollama (default), claude_cli, anthropic, openai

Research — Paper discovery and analysis

Unified entry tools/research/research_scout.py: Four major capabilities:

1. Paper discovery: Search in arXiv / bioRxiv / PubMed, three-stage LLM pipeline (fast screening → deep evaluation → citation impact), generate research weekly reports and deploy them on Hugo; supports search for conference papers and author searches
2. Deep paper insights --insight: Download full texts, analyze writing structure/ publication strategy/reusable knowledge; match with OpenReview to get review comments; synthesize cross-paper writing guides
3. Researcher profiles: ArXiv + Semantic Scholar data, LLM generates research trajectory analysis and hierarchical scoring; extract relationships and co-authorship patterns via homepage; supports name disambiguation and reverse lookup
4. Citation graph analysis: Based on Semantic Scholar forward/backward reference analysis + LLM impact interpretation

Benchmark — Performance benchmark suite

• Cross-platform CPU/GPU FLOPS benchmarks
• Supports NVIDIA (CUDA), Apple Silicon (MPS), Intel (XPU)
• Precision: FP64 / FP32 / FP16 / BF16 / FP8 (experimental)
• Warm-up + formal measurement + statistical analysis (median, IQR outlier removal)
• Results appended to CSV; can generate interactive HTML reports and leaderboards; can be deployed on Hugo or submitted to public leaderboards

Website — Hugo blog

• Static blog “TzJ’s Net” (PaperMod theme), deployed on GitHub Pages: https://tzj2006.github.io/
• Incremental image/video compression (pngquant, HandBrakeCLI)
• Local model bilingual translation (default Ollama; fallback to vLLM on Linux, transformers on Windows; model tencent/Hy-MT2-1.8B automatically downloaded on first run)
• Automatically generates content written into tools/website/content|static (marked with gadget_generated, handwritten files not overwritten)
• tools/website/public/ is an independent deployment repository tzj2006/tzj2006.github.io, automatically committed + pushed by build script, do not commit directly

Translator — Gradio document translator

• Local inference web interface based on common translation engine
• Supports text and files .md / .txt / .pdf / .docx / images, preserves Markdown format
• Backend automatic selection:优先 ollama, fallback transformers on Windows, priority vllm on Linux, low memory option llamacpp
• Default model tencent/Hy-MT2-1.8B; can be overridden by GADGET_TRANSLATION_MODEL or GUI
• Source language auto detected by CJK ratio; target auto switches between zh↔en

5. common / scripts / ai-companion

common/: LLM invocation, JSON parsing and fixing, SHA-256 disk cache, atomic writing, local translation engine, Hugo cross-platform deployment.

scripts/ Main scripts:

ai-companion: Independent Node.js repository, provides code change tracking, planning/execution pipeline, all Claude Code skills; installation see ../ai-companion/scripts/install.ts.

2. Detailed Chinese Usage Guide (First Time)