升级 NUPACK 4.1 并支持混合材料设计

2026-08-20 16:33:16 +08:00 · 2026-08-20 16:33:16 +08:00 · c6189d857d
commit c6189d857d
parent 5daa60a464
33 changed files with 5830 additions and 466 deletions
--- a/README.md
+++ b/README.md
@ -1,6 +1,28 @@
 # NP Replica

-This project provides a local Chinese web replica of the core NUPACK `analysis/input` workflow.
+This project provides a local Chinese web interface for the official NUPACK Analysis, Design, and Utilities workflows.
+
+## v5.4 · NUPACK 4.1 mixed-material Design
+
+The `v5.4-nupack4.1.0.1` image extends mixed-material support through the complete Design workflow:
+
+- RNA/DNA and RNA/2′OMe-RNA Domain constraints using explicit `r`, `d`, `m`, and wildcard `w` prefixes;
+- material-aware IUPAC parsing, fixed-domain reverse complements, and true nucleotide lengths;
+- mixed-material Pattern, Similarity, Window, and Library sequence inputs;
+- a mixed-segment preset, dynamic syntax guidance, and client-side validation in the Design editor;
+- material prefixes preserved in designed Domains, strands, reproducibility exports, and Analysis handoff;
+- Docker version metadata moved after dependency layers so version-only rebuilds reuse the expensive NUPACK/ViennaRNA layers.
+
+## v5.3 · NUPACK 4.1 mixed-material analysis
+
+The `v5.3-nupack4.1.0.1` image adds:
+
+- material-aware salt defaults and server-side physical-range validation;
+- RNA/DNA (`rna-dna06`) and RNA/2′OMe-RNA (`rna-merna06`) Analysis and Utilities inputs using `r`, `d`, and `m` prefixes;
+- NUPACK version and resolved parameter-set provenance in results;
+- `indistinguishable_search` and `max_subopt_count` controls for MFE/Subopt;
+- per-complex analysis bonus support.
+

 ## Run

@ -10,6 +32,16 @@ docker compose up --build -d

 Open `http://127.0.0.1:18765`.

+Application pages:
+
+- `/`: operational overview
+- `/workspace`: calculation workspace
+- `/cloud`: account jobs and shares
+- `/account`: identity and usage
+
+The system design and deployment boundaries are documented in
+[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).
+
 ## Reverse Proxy

 The service listens on port `18765`. The current local deployment is reverse proxied as `npt.lihato.icu`.
@ -34,9 +66,11 @@ environment:
 ```

 The `NP_OIDC_*` variables belong on the web service (`np-replica`); the
-worker does not perform browser login. Preserve the existing calculation
-settings, including `NP_WORKER_CONCURRENCY=16`,
-`NP_PER_JOB_THREAD_LIMIT=4`, and `NP_NUPACK_CACHE_GB=8.0`.
+worker does not perform browser login. Preserve the calculation settings on
+both application containers. The recommended portable defaults are
+`NP_WORKER_CONCURRENCY=auto`, `NP_PER_JOB_THREAD_LIMIT=4`,
+`NP_WORKER_MEMORY_RESERVE_GB=4.0`, `NP_ESTIMATED_JOB_MEMORY_GB=9.0`,
+and `NP_NUPACK_CACHE_GB=8.0`.

 After editing the Compose file on `.11`, recreate the web container:

@ -70,7 +104,7 @@ is still running with the wrong environment and must be recreated.

 ## 容器资源配置

-当前 `docker-compose.yml` 面向 64 核 / 64 GB 级别的服务器配置，服务拆成三个容器：
+当前 `docker-compose.yml` 可随 CPU 和容器内存限制自动调整 worker 并发，服务拆成三个容器：

 - `redis`：只保存实时队列、运行状态和会话缓存，限制为 `mem_limit: 3g`。Redis 开启 AOF，数据写入 `./runtime/redis`。
 - `np-replica`：Web/API 容器，限制为 `mem_limit: 4g`，负责页面、登录、历史记录、任务提交和状态查询。它不应该承担大规模计算。
@ -78,12 +112,26 @@ is still running with the wrong environment and must be recreated.

 计算相关环境变量需要在 `np-replica` 和 `np-worker` 中保持一致：

- `NP_WORKER_CONCURRENCY=16`：最多同时执行 16 个后台任务，超过后进入 Redis 队列等待。
+- `NP_WORKER_CONCURRENCY=auto`：根据容器可用 CPU、每任务线程数、内存限制、预留内存和估算的单任务内存，自动选择同时运行的任务数。也可填正整数进行手动覆盖。
 - `NP_PER_JOB_THREAD_LIMIT=4`：单个任务最多使用 4 个 native 计算线程，同时写入 `OMP_NUM_THREADS`、`OPENBLAS_NUM_THREADS`、`MKL_NUM_THREADS`、`NUMEXPR_NUM_THREADS`、`VECLIB_MAXIMUM_THREADS`、`GOTO_NUM_THREADS`，并设置 `nupack.config.threads`。
 - `NP_NUPACK_CACHE_GB=8.0`：单个 NUPACK 进程可使用的缓存上限。
+- `NP_WORKER_MEMORY_GB=0`：为 0 时自动读取 cgroup 内存限制或系统内存；在特殊运行环境中可填实际可用 GB 数覆盖检测值。
+- `NP_WORKER_MEMORY_RESERVE_GB=4.0`：不给计算任务使用的内存，留给系统、Redis、Web 服务和运行时开销。
+- `NP_ESTIMATED_JOB_MEMORY_GB=9.0`：调度器用于计算并发的单任务内存预算。根据其他设备上观测到的峰值灵活调整；它不限制单任务实际内存。
 - `NP_JOB_TTL_SECONDS=3600`：Redis 中实时任务状态的保留时间；长期历史记录写入 SQLite。

-理论上当前峰值为 `16 * 4 = 64` 个 native 计算线程。若机器被压满，优先把 `NP_WORKER_CONCURRENCY` 从 `16` 降到 `8` 或 `4`；如果单任务仍过重，再把 `NP_PER_JOB_THREAD_LIMIT` 从 `4` 降到 `2` 或 `1`。`np-replica` 的内存不建议调高来跑计算，应该把计算压力留给 `np-worker`。
+`auto` 取 CPU 容量 `有效 CPU 数 / 单任务线程数` 与内存容量
+`(worker 内存 - 预留内存) / 单任务内存估算` 中较小者。CPU 检测会考虑
+进程 affinity 和 cgroup quota，内存检测会考虑 cgroup limit，因此同一份配置可迁移到
+不同设备。若出现 OOM 或交换分区抖动，提高 `NP_ESTIMATED_JOB_MEMORY_GB` 或直接降低
+`NP_WORKER_CONCURRENCY`；空闲资源很多时则可谨慎反向调整。`/health` 会返回 worker
+最终采用的资源计划。
+
+增加线程不会保证单个 design 等比例提速：NUPACK 的部分设计阶段实际上主要使用一个
+核心，线程更适合底层可并行计算；同时运行多个独立任务通常更容易利用多核。增加内存也
+不会直接缩短搜索，只能减少 OOM、换页和缓存压力。当前 NUPACK 4.1.0.1 wheel 只暴露 CPU
+线程配置，不能通过环境变量直接调用 GPU；CUDA 加速需要重新构建或替换 NUPACK 核心
+wheel，并重新做数值一致性验证。

 设计任务默认使用官方行为：固定序列 target 也参与 `tube_design` / `complex_design`。如果确认某些 target 完全固定、只需要展示结果、不需要参与优化，可在页面中把 “固定目标策略” 改为 “剥离固定目标以提速”，这样会减少 off-target 集合和优化搜索量。

@ -96,6 +144,35 @@ is still running with the wrong environment and must be recreated.
 - `POST /api/jobs`
 - `GET /api/jobs/<job_id>`

+Submitting an identical active payload for the same account returns
+`409 duplicate`. The Workspace asks the user whether to cancel or explicitly
+submit another calculation through `POST /api/jobs?force=1`. Fingerprints are
+account-scoped, so different users are never deduplicated against one another.
+
+## Admin Login
+
+Admin uses a dedicated password and does not require OIDC. Generate a strong
+password and store it in the ignored `.env`:
+
+```bash
+openssl rand -hex 24
+```
+
+```dotenv
+NP_ADMIN_TOKEN=replace-with-the-generated-password
+```
+
+After deployment, open `https://<host>/admin` from the Account page or directly.
+The password is submitted in a JSON `POST /api/admin/login` request and never
+appears in the URL. A successful login sets a random, short-lived HttpOnly
+session cookie and redirects to `/admin/panel`.
+
+Admin can query all task metadata, stop active jobs, move terminal jobs to the
+trash, restore them, permanently delete trashed jobs, and configure trash
+retention (two days by default). Normal account deletion is a soft delete and
+also disables that job's public shares. Set `NP_ADMIN_SESSION_TTL_SECONDS` to
+control session duration and keep `NP_ADMIN_COOKIE_SECURE=1` on HTTPS deployments.
+

 ## Persistent Shares

@ -103,16 +180,23 @@ Live jobs and sessions are stored in Redis when `NP_REDIS_URL` is enabled. Accou

 ## Notes

- `tube` mode supports strand concentrations and `SetSpec(max_size=...)`.
+- `tube` mode supports strand concentrations and the official `SetSpec(max_size=..., include=..., exclude=...)` semantics.
 - `complex` mode supports explicit complex definitions such as `A+B`.
 - Pair matrices are returned as a preview block to keep responses manageable.
+- Tube `pairs` now runs inside the same official `tube_analysis` call as the other selected calculations, including `ensemble_pair_fractions` and `fraction_bases_unpaired`.
+- `result_limit` limits display and structure plotting only; it does not silently reduce the Tube calculation ensemble.
+- Analysis and design results include phase timings and workload diagnostics to explain long-running jobs without changing thermodynamic results.
+- Design target tubes support per-tube `SetSpec(max_size/include/exclude)`, target-complex bonuses, weighted-objective trial selection, and multi-tube result tables.
+- Utilities directly exposes the official `pfunc`, `structure_energy`, `structure_probability`, `sample`, `pairs`, `mfe`, `subopt`, `ensemble_size`, `des`, `defect`, `seq_distance`, and `struc_distance` calls.
+- Active calculations write only small stage-transition records. The result panel performs one status request per manual refresh and does not poll or serialize NUPACK intermediate results.
+- Identical active submissions from the same account reuse the existing job instead of consuming another worker slot.
 - The Docker image compiles ViennaRNA `RNAplot` from `rna/ViennaRNA-2.7.2.tar.gz` and embeds SVG structure plots into MFE results.
 - Multistrand MFE plots now use a split-strand layout by default when a structure contains `+`, with `RNAplot` retained as the fallback path.
- The web UI supports Chinese and English switching, account-scoped cloud history, result export, share controls, and async polling for submitted jobs.
+- The web UI supports Chinese and English switching, account-scoped cloud history, result export, share controls, and manually refreshed async job status.
 - Docker now starts three services: `np-replica`, `np-worker`, and `redis`.
 - Async jobs use Redis-backed storage and queueing when `NP_REDIS_URL` is configured; otherwise the app falls back to the older in-memory mode.
 - The browser-facing API and page behavior stay the same after enabling Redis-backed jobs.
- Worker concurrency is controlled by `NP_WORKER_CONCURRENCY` and defaults to `2`.
+- Worker concurrency is controlled by `NP_WORKER_CONCURRENCY` and defaults to `auto`; a positive integer remains available as an explicit override.
 - Per-job native thread usage is capped by `NP_PER_JOB_THREAD_LIMIT` and defaults to `1` to prevent a single analysis from monopolizing the machine.
 - `NP_STRUCTURE_PLOT_MODE` supports `auto` (default), `split`, and `rnaplot`.
 - Static pages are served with ETag-based browser caching; JSON API responses remain uncached.