升级 NUPACK 4.1 并支持混合材料设计
This commit is contained in:
Lihatoo
parent
5daa60a464
commit
c6189d857d
33 changed files with 5830 additions and 466 deletions
159
docs/ARCHITECTURE.md
Normal file
159
docs/ARCHITECTURE.md
Normal file
|
|
@ -0,0 +1,159 @@
|
|||
# NP Replica Architecture
|
||||
|
||||
## Goals
|
||||
|
||||
NP Replica separates user navigation, durable account data, live queue state,
|
||||
and NUPACK compute processes. Scientific calculation code remains isolated from
|
||||
page rendering and account administration so UI changes do not alter numerical
|
||||
semantics.
|
||||
|
||||
## Runtime Topology
|
||||
|
||||
```text
|
||||
Browser
|
||||
|
|
||||
| HTTPS / OIDC session
|
||||
v
|
||||
np-replica (Web/API, 4 GB)
|
||||
| |
|
||||
| live state | durable metadata/results
|
||||
v v
|
||||
Redis SQLite
|
||||
|
|
||||
| job queue
|
||||
v
|
||||
np-worker (NUPACK compute, 56 GB)
|
||||
```
|
||||
|
||||
- **Web/API** serves the multi-page UI, validates ownership, accepts jobs, and
|
||||
serializes API responses. It does not run Redis-backed NUPACK jobs.
|
||||
- **Redis** owns queue order, active job state, heartbeats, duplicate claims,
|
||||
sessions, and temporary result state.
|
||||
- **SQLite** owns users, durable job inputs/results, usage totals, and shares.
|
||||
- **Worker** consumes Redis jobs and runs NUPACK in isolated child processes.
|
||||
|
||||
## Frontend Structure
|
||||
|
||||
The UI uses a small multi-page application structure. Each page has one primary
|
||||
operational purpose.
|
||||
|
||||
| Route | File | Responsibility |
|
||||
| --- | --- | --- |
|
||||
| `/` | `service/home.html` | Queue overview, usage summary, recent jobs |
|
||||
| `/workspace` | `service/index.html` | Analysis, Design, Utilities, result inspection |
|
||||
| `/cloud` | `service/cloud.html` | Job search, JSON import/export, result links, cancellation, deletion, shares |
|
||||
| `/account` | `service/account.html` | Identity, account usage, worker resource information |
|
||||
| `/admin` | `service/admin-login.html` | Dedicated Admin password login |
|
||||
| `/admin/panel` | `service/admin.html` | Cross-account task and trash management |
|
||||
|
||||
`service/static/app-shell.css` provides the shared application shell and
|
||||
responsive operational layout. `service/static/portal.js` contains shared
|
||||
request, formatting, navigation, table, Cloud, Account, and Admin controllers.
|
||||
The calculation workspace retains its self-contained renderer because it
|
||||
contains specialized structure, heatmap, force-layout, and result-export code.
|
||||
|
||||
## Backend Boundaries
|
||||
|
||||
- `service/server.py`
|
||||
- HTTP routes and authentication gates
|
||||
- Redis live-job repository and queue
|
||||
- resource planning and worker orchestration
|
||||
- NUPACK request validation and calculation adapters
|
||||
- `service/account.py`
|
||||
- SQLite schema and durable repository
|
||||
- account-scoped job/share queries
|
||||
- cross-account Admin queries and trash lifecycle
|
||||
- `service/split_strand_svg.py`
|
||||
- structure rendering adapter
|
||||
|
||||
The next safe backend extraction point is moving Redis job coordination into a
|
||||
`job_repository.py` module. NUPACK adapters can then move into separate
|
||||
`analysis_service.py`, `design_service.py`, and `utilities_service.py`
|
||||
modules without changing HTTP contracts.
|
||||
|
||||
## Job Submission And Duplicate Control
|
||||
|
||||
Duplicate detection is scoped by `user_id` and a canonical SHA-256 hash of the
|
||||
JSON payload. Different users never share duplicate state.
|
||||
|
||||
```text
|
||||
POST /api/jobs
|
||||
|
|
||||
+-- no active match --> 202 accepted
|
||||
|
|
||||
+-- active match ----> 409 duplicate + existing job metadata
|
||||
|
|
||||
+-- user cancels: no write
|
||||
|
|
||||
+-- user confirms:
|
||||
POST /api/jobs?force=1 --> 202 accepted
|
||||
```
|
||||
|
||||
Redis uses an atomic claim key. Terminal updates delete a claim only when it
|
||||
still belongs to that job, preventing an older job from deleting a newer forced
|
||||
submission's claim. The in-memory development backend mirrors this behavior
|
||||
under `JOB_LOCK`.
|
||||
|
||||
## Resource Planning
|
||||
|
||||
`NP_WORKER_CONCURRENCY=auto` selects the smaller of:
|
||||
|
||||
- effective CPU capacity divided by `NP_PER_JOB_THREAD_LIMIT`;
|
||||
- usable worker memory divided by `NP_ESTIMATED_JOB_MEMORY_GB`.
|
||||
|
||||
CPU detection considers process affinity and cgroup quota. Memory detection
|
||||
uses an explicit override, cgroup memory limit, or host memory in that order.
|
||||
The published plan is visible through `/health`.
|
||||
|
||||
These controls affect scheduling only. They do not change models, sequence
|
||||
constraints, complexes, thermodynamic parameters, or result serialization.
|
||||
|
||||
## Authentication And Authorization
|
||||
|
||||
- Normal pages and account APIs require an OIDC-backed session.
|
||||
- Job, history, and share APIs enforce ownership by stable OIDC `user_id`.
|
||||
- Public share routes expose only the selected shared record.
|
||||
- Admin uses a dedicated password configured by `NP_ADMIN_TOKEN` and bypasses
|
||||
OIDC. The password is accepted only in the JSON body of
|
||||
`POST /api/admin/login`; it is never embedded in a route, redirect, cookie, or
|
||||
local storage value.
|
||||
- Successful login creates a random session in Redis (or process memory in
|
||||
development) and sets a short-lived HttpOnly, SameSite=Strict cookie.
|
||||
- Admin may list all durable job metadata, stop active jobs, move terminal jobs
|
||||
to trash, restore jobs, permanently delete trashed jobs, and configure trash
|
||||
retention. APIs do not return stored payload or result blobs.
|
||||
|
||||
The Admin password must be random, kept in the ignored `.env`, and rotated after
|
||||
exposure. Keep `NP_ADMIN_COOKIE_SECURE=1` in HTTPS deployments.
|
||||
|
||||
## Trash Lifecycle
|
||||
|
||||
Normal user deletion sets `deleted_at` and `purge_after` rather than deleting a
|
||||
job row. Deleted jobs disappear from account history, usage, and result APIs;
|
||||
all share records for that job are deleted. Admin can inspect the trash, restore a
|
||||
job, or permanently delete it. Expired rows are purged lazily during job-list
|
||||
reads. Retention is stored in SQLite `app_settings`, defaults to two days, and
|
||||
can be changed from one hour to 365 days.
|
||||
|
||||
## Persistence And Recovery
|
||||
|
||||
- Redis AOF persists live queue state under `runtime/redis`.
|
||||
- SQLite WAL persists account data under `runtime/account`.
|
||||
- Worker recovery requeues jobs marked running after an interrupted worker.
|
||||
- Active Redis job records do not expire while heartbeat updates continue;
|
||||
terminal records use `NP_JOB_TTL_SECONDS`.
|
||||
|
||||
Back up both runtime directories. SQLite is the durable source for the Cloud and
|
||||
Admin pages; Redis is the source for current queue and heartbeat state.
|
||||
|
||||
## Deployment
|
||||
|
||||
1. Set `NP_ADMIN_TOKEN` in `.env`.
|
||||
2. Validate with `docker compose config --quiet`.
|
||||
3. Wait for active calculations to finish before recreating `np-worker`.
|
||||
4. Build and recreate Web and Worker from the same source revision.
|
||||
5. Verify `/health`, normal OIDC pages, duplicate confirmation, Admin password
|
||||
login, task controls, and the trash lifecycle.
|
||||
|
||||
The Web container may be recreated independently for page-only changes, but
|
||||
backend contract changes should deploy Web and Worker from the same image.
|
||||
Loading…
Add table
Add a link
Reference in a new issue