lihato/np_app
1
0
Fork 0
Code Issues Pull requests Projects Releases 2 Packages 1 Wiki Activity Actions
np_app/README.md

12 KiB
Raw Blame History

NP Replica

This project provides a local Chinese web interface for the official NUPACK Analysis, Design, and Utilities workflows.

v5.4 · NUPACK 4.1 mixed-material Design

The v5.4-nupack4.1.0.1 image extends mixed-material support through the complete Design workflow:

  • RNA/DNA and RNA/2′OMe-RNA Domain constraints using explicit r, d, m, and wildcard w prefixes;
  • material-aware IUPAC parsing, fixed-domain reverse complements, and true nucleotide lengths;
  • mixed-material Pattern, Similarity, Window, and Library sequence inputs;
  • a mixed-segment preset, dynamic syntax guidance, and client-side validation in the Design editor;
  • material prefixes preserved in designed Domains, strands, reproducibility exports, and Analysis handoff;
  • Docker version metadata moved after dependency layers so version-only rebuilds reuse the expensive NUPACK/ViennaRNA layers.

v5.3 · NUPACK 4.1 mixed-material analysis

The v5.3-nupack4.1.0.1 image adds:

  • material-aware salt defaults and server-side physical-range validation;
  • RNA/DNA (rna-dna06) and RNA/2′OMe-RNA (rna-merna06) Analysis and Utilities inputs using r, d, and m prefixes;
  • NUPACK version and resolved parameter-set provenance in results;
  • indistinguishable_search and max_subopt_count controls for MFE/Subopt;
  • per-complex analysis bonus support.

Run

docker compose up --build -d

Open http://127.0.0.1:18765.

Application pages:

  • /: operational overview
  • /workspace: calculation workspace
  • /cloud: account jobs and shares
  • /account: identity and usage

The system design and deployment boundaries are documented in docs/ARCHITECTURE.md.

Reverse Proxy

The service listens on port 18765. The current local deployment is reverse proxied as npt.lihato.icu.

Traefik only performs TLS termination, compression, and reverse proxying for npt; authentication is enforced by this application.

Deploying the independent np instance

np.lihato.icu runs independently on 100.64.0.11. Do not copy the npt OIDC values unchanged. In the np-replica service of the docker-compose.yml deployed on .11, use:

ports:
  - "100.64.0.11:18765:18765"
environment:
  NP_AUTH_REQUIRED: 1
  NP_OIDC_ISSUER: https://auth.lihato.icu/application/o/nupack-account/
  NP_OIDC_CLIENT_ID: np-replica-web
  NP_OIDC_REDIRECT_URI: https://np.lihato.icu/auth/callback
  NP_OIDC_POST_LOGOUT_URI: https://np.lihato.icu/

The NP_OIDC_* variables belong on the web service (np-replica); the worker does not perform browser login. Preserve the calculation settings on both application containers. The recommended portable defaults are NP_WORKER_CONCURRENCY=auto, NP_PER_JOB_THREAD_LIMIT=4, NP_WORKER_MEMORY_RESERVE_GB=4.0, NP_ESTIMATED_JOB_MEMORY_GB=9.0, and NP_NUPACK_CACHE_GB=8.0.

After editing the Compose file on .11, recreate the web container:

docker compose up -d --force-recreate np-replica

If the deployed image predates the account/OIDC changes, rebuild both application containers instead:

docker compose up -d --build --force-recreate np-replica np-worker

Verify the result:

curl -I https://np.lihato.icu/
curl -I 'https://np.lihato.icu/auth/login?next=%2F'

The first response should redirect to /auth/login?next=%2F. The second Location header must contain all of the following:

  • client_id=np-replica-web
  • redirect_uri=https%3A%2F%2Fnp.lihato.icu%2Fauth%2Fcallback
  • code_challenge_method=S256

If it contains npt-replica-web or npt.lihato.icu, the .11 container is still running with the wrong environment and must be recreated.

容器资源配置

当前 docker-compose.yml 可随 CPU 和容器内存限制自动调整 worker 并发，服务拆成三个容器：

  • redis：只保存实时队列、运行状态和会话缓存，限制为 mem_limit: 3g。Redis 开启 AOF，数据写入 ./runtime/redis。
  • np-replica：Web/API 容器，限制为 mem_limit: 4g，负责页面、登录、历史记录、任务提交和状态查询。它不应该承担大规模计算。
  • np-worker：后台计算容器，限制为 mem_limit: 56g，负责实际 NUPACK 计算。大任务应该由这个容器消耗 CPU 和内存。

计算相关环境变量需要在 np-replica 和 np-worker 中保持一致：

  • NP_WORKER_CONCURRENCY=auto：根据容器可用 CPU、每任务线程数、内存限制、预留内存和估算的单任务内存，自动选择同时运行的任务数。也可填正整数进行手动覆盖。
  • NP_PER_JOB_THREAD_LIMIT=4：单个任务最多使用 4 个 native 计算线程，同时写入 OMP_NUM_THREADS、OPENBLAS_NUM_THREADS、MKL_NUM_THREADS、NUMEXPR_NUM_THREADS、VECLIB_MAXIMUM_THREADS、GOTO_NUM_THREADS，并设置 nupack.config.threads。
  • NP_NUPACK_CACHE_GB=8.0：单个 NUPACK 进程可使用的缓存上限。
  • NP_WORKER_MEMORY_GB=0：为 0 时自动读取 cgroup 内存限制或系统内存；在特殊运行环境中可填实际可用 GB 数覆盖检测值。
  • NP_WORKER_MEMORY_RESERVE_GB=4.0：不给计算任务使用的内存，留给系统、Redis、Web 服务和运行时开销。
  • NP_ESTIMATED_JOB_MEMORY_GB=9.0：调度器用于计算并发的单任务内存预算。根据其他设备上观测到的峰值灵活调整；它不限制单任务实际内存。
  • NP_JOB_TTL_SECONDS=3600：Redis 中实时任务状态的保留时间；长期历史记录写入 SQLite。

auto 取 CPU 容量 有效 CPU 数 / 单任务线程数 与内存容量 (worker 内存 - 预留内存) / 单任务内存估算 中较小者。CPU 检测会考虑 进程 affinity 和 cgroup quota，内存检测会考虑 cgroup limit，因此同一份配置可迁移到 不同设备。若出现 OOM 或交换分区抖动，提高 NP_ESTIMATED_JOB_MEMORY_GB 或直接降低 NP_WORKER_CONCURRENCY；空闲资源很多时则可谨慎反向调整。/health 会返回 worker 最终采用的资源计划。

增加线程不会保证单个 design 等比例提速：NUPACK 的部分设计阶段实际上主要使用一个 核心，线程更适合底层可并行计算；同时运行多个独立任务通常更容易利用多核。增加内存也 不会直接缩短搜索，只能减少 OOM、换页和缓存压力。当前 NUPACK 4.1.0.1 wheel 只暴露 CPU 线程配置，不能通过环境变量直接调用 GPU；CUDA 加速需要重新构建或替换 NUPACK 核心 wheel，并重新做数值一致性验证。

设计任务默认使用官方行为：固定序列 target 也参与 tube_design / complex_design。如果确认某些 target 完全固定、只需要展示结果、不需要参与优化，可在页面中把 “固定目标策略” 改为 “剥离固定目标以提速”，这样会减少 off-target 集合和优化搜索量。

Endpoints

  • GET /
  • GET /health
  • GET /api/example
  • POST /api/analyze
  • POST /api/jobs
  • GET /api/jobs/<job_id>

Submitting an identical active payload for the same account returns 409 duplicate. The Workspace asks the user whether to cancel or explicitly submit another calculation through POST /api/jobs?force=1. Fingerprints are account-scoped, so different users are never deduplicated against one another.

Admin Login

Admin uses a dedicated password and does not require OIDC. Generate a strong password and store it in the ignored .env:

openssl rand -hex 24

NP_ADMIN_TOKEN=replace-with-the-generated-password

After deployment, open https://<host>/admin from the Account page or directly. The password is submitted in a JSON POST /api/admin/login request and never appears in the URL. A successful login sets a random, short-lived HttpOnly session cookie and redirects to /admin/panel.

Admin can query all task metadata, stop active jobs, move terminal jobs to the trash, restore them, permanently delete trashed jobs, and configure trash retention (two days by default). Normal account deletion is a soft delete and also disables that job's public shares. Set NP_ADMIN_SESSION_TTL_SECONDS to control session duration and keep NP_ADMIN_COOKIE_SECURE=1 on HTTPS deployments.

Persistent Shares

Live jobs and sessions are stored in Redis when NP_REDIS_URL is enabled. Account history and share links are stored in ./runtime/account/np-replica.sqlite3; back up both runtime directories when moving the deployment.

Notes

  • tube mode supports strand concentrations and the official SetSpec(max_size=..., include=..., exclude=...) semantics.
  • complex mode supports explicit complex definitions such as A+B.
  • Pair matrices are returned as a preview block to keep responses manageable.
  • Tube pairs now runs inside the same official tube_analysis call as the other selected calculations, including ensemble_pair_fractions and fraction_bases_unpaired.
  • result_limit limits display and structure plotting only; it does not silently reduce the Tube calculation ensemble.
  • Analysis and design results include phase timings and workload diagnostics to explain long-running jobs without changing thermodynamic results.
  • Design target tubes support per-tube SetSpec(max_size/include/exclude), target-complex bonuses, weighted-objective trial selection, and multi-tube result tables.
  • Utilities directly exposes the official pfunc, structure_energy, structure_probability, sample, pairs, mfe, subopt, ensemble_size, des, defect, seq_distance, and struc_distance calls.
  • Active calculations write only small stage-transition records. The result panel performs one status request per manual refresh and does not poll or serialize NUPACK intermediate results.
  • Identical active submissions from the same account reuse the existing job instead of consuming another worker slot.
  • The Docker image compiles ViennaRNA RNAplot from rna/ViennaRNA-2.7.2.tar.gz and embeds SVG structure plots into MFE results.
  • Multistrand MFE plots now use a split-strand layout by default when a structure contains +, with RNAplot retained as the fallback path.
  • The web UI supports Chinese and English switching, account-scoped cloud history, result export, share controls, and manually refreshed async job status.
  • Docker now starts three services: np-replica, np-worker, and redis.
  • Async jobs use Redis-backed storage and queueing when NP_REDIS_URL is configured; otherwise the app falls back to the older in-memory mode.
  • The browser-facing API and page behavior stay the same after enabling Redis-backed jobs.
  • Worker concurrency is controlled by NP_WORKER_CONCURRENCY and defaults to auto; a positive integer remains available as an explicit override.
  • Per-job native thread usage is capped by NP_PER_JOB_THREAD_LIMIT and defaults to 1 to prevent a single analysis from monopolizing the machine.
  • NP_STRUCTURE_PLOT_MODE supports auto (default), split, and rnaplot.
  • Static pages are served with ETag-based browser caching; JSON API responses remain uncached.
  • The force-directed structure viewer lazy-loads D3 only when that view is opened, so the initial page load is not blocked by the external CDN.
  • /health uses Redis queue length plus a running-job set instead of scanning every historical job on each poll.
  • Application login uses the public Authentik issuer at https://auth.lihato.icu/ with Authorization Code + PKCE.
  • The current npt instance uses issuer /application/o/nupack-account-npt/, client npt-replica-web, callback https://npt.lihato.icu/auth/callback, and logout return https://npt.lihato.icu/.
  • For the independent np deployment on 100.64.0.11, set NP_OIDC_ISSUER=https://auth.lihato.icu/application/o/nupack-account/, NP_OIDC_CLIENT_ID=np-replica-web, NP_OIDC_REDIRECT_URI=https://np.lihato.icu/auth/callback, and NP_OIDC_POST_LOGOUT_URI=https://np.lihato.icu/.
  • Redis stores the live queue and login sessions. SQLite WAL at /data/np-replica.sqlite3 stores users, owned jobs, compressed inputs/results/errors, usage totals, and durable share links.
  • History and job APIs enforce ownership by the Authentik OIDC subject. Public share links expose only the selected record and can be disabled or given an expiry by its owner.