lihato/np_app
1
0
Fork 0
Code Issues Pull requests Projects Releases 2 Packages 1 Wiki Activity Actions
np_app/docs/ARCHITECTURE.md

6.5 KiB
Raw Permalink Blame History

NP Replica Architecture

Goals

NP Replica separates user navigation, durable account data, live queue state, and NUPACK compute processes. Scientific calculation code remains isolated from page rendering and account administration so UI changes do not alter numerical semantics.

Runtime Topology

Browser
  |
  | HTTPS / OIDC session
  v
np-replica (Web/API, 4 GB)
  |                 |
  | live state      | durable metadata/results
  v                 v
Redis             SQLite
  |
  | job queue
  v
np-worker (NUPACK compute, 56 GB)
  • Web/API serves the multi-page UI, validates ownership, accepts jobs, and serializes API responses. It does not run Redis-backed NUPACK jobs.
  • Redis owns queue order, active job state, heartbeats, duplicate claims, sessions, and temporary result state.
  • SQLite owns users, durable job inputs/results, usage totals, and shares.
  • Worker consumes Redis jobs and runs NUPACK in isolated child processes.

Frontend Structure

The UI uses a small multi-page application structure. Each page has one primary operational purpose.

Route File Responsibility
/ service/home.html Queue overview, usage summary, recent jobs
/workspace service/index.html Analysis, Design, Utilities, result inspection
/cloud service/cloud.html Job search, JSON import/export, result links, cancellation, deletion, shares
/account service/account.html Identity, account usage, worker resource information
/admin service/admin-login.html Dedicated Admin password login
/admin/panel service/admin.html Cross-account task and trash management

service/static/app-shell.css provides the shared application shell and responsive operational layout. service/static/portal.js contains shared request, formatting, navigation, table, Cloud, Account, and Admin controllers. The calculation workspace retains its self-contained renderer because it contains specialized structure, heatmap, force-layout, and result-export code.

Backend Boundaries

  • service/server.py
    • HTTP routes and authentication gates
    • Redis live-job repository and queue
    • resource planning and worker orchestration
    • NUPACK request validation and calculation adapters
  • service/account.py
    • SQLite schema and durable repository
    • account-scoped job/share queries
    • cross-account Admin queries and trash lifecycle
  • service/split_strand_svg.py
    • structure rendering adapter

The next safe backend extraction point is moving Redis job coordination into a job_repository.py module. NUPACK adapters can then move into separate analysis_service.py, design_service.py, and utilities_service.py modules without changing HTTP contracts.

Job Submission And Duplicate Control

Duplicate detection is scoped by user_id and a canonical SHA-256 hash of the JSON payload. Different users never share duplicate state.

POST /api/jobs
  |
  +-- no active match --> 202 accepted
  |
  +-- active match ----> 409 duplicate + existing job metadata
                           |
                           +-- user cancels: no write
                           |
                           +-- user confirms:
                               POST /api/jobs?force=1 --> 202 accepted

Redis uses an atomic claim key. Terminal updates delete a claim only when it still belongs to that job, preventing an older job from deleting a newer forced submission's claim. The in-memory development backend mirrors this behavior under JOB_LOCK.

Resource Planning

NP_WORKER_CONCURRENCY=auto selects the smaller of:

  • effective CPU capacity divided by NP_PER_JOB_THREAD_LIMIT;
  • usable worker memory divided by NP_ESTIMATED_JOB_MEMORY_GB.

CPU detection considers process affinity and cgroup quota. Memory detection uses an explicit override, cgroup memory limit, or host memory in that order. The published plan is visible through /health.

These controls affect scheduling only. They do not change models, sequence constraints, complexes, thermodynamic parameters, or result serialization.

Authentication And Authorization

  • Normal pages and account APIs require an OIDC-backed session.
  • Job, history, and share APIs enforce ownership by stable OIDC user_id.
  • Public share routes expose only the selected shared record.
  • Admin uses a dedicated password configured by NP_ADMIN_TOKEN and bypasses OIDC. The password is accepted only in the JSON body of POST /api/admin/login; it is never embedded in a route, redirect, cookie, or local storage value.
  • Successful login creates a random session in Redis (or process memory in development) and sets a short-lived HttpOnly, SameSite=Strict cookie.
  • Admin may list all durable job metadata, stop active jobs, move terminal jobs to trash, restore jobs, permanently delete trashed jobs, and configure trash retention. APIs do not return stored payload or result blobs.

The Admin password must be random, kept in the ignored .env, and rotated after exposure. Keep NP_ADMIN_COOKIE_SECURE=1 in HTTPS deployments.

Trash Lifecycle

Normal user deletion sets deleted_at and purge_after rather than deleting a job row. Deleted jobs disappear from account history, usage, and result APIs; all share records for that job are deleted. Admin can inspect the trash, restore a job, or permanently delete it. Expired rows are purged lazily during job-list reads. Retention is stored in SQLite app_settings, defaults to two days, and can be changed from one hour to 365 days.

Persistence And Recovery

  • Redis AOF persists live queue state under runtime/redis.
  • SQLite WAL persists account data under runtime/account.
  • Worker recovery requeues jobs marked running after an interrupted worker.
  • Active Redis job records do not expire while heartbeat updates continue; terminal records use NP_JOB_TTL_SECONDS.

Back up both runtime directories. SQLite is the durable source for the Cloud and Admin pages; Redis is the source for current queue and heartbeat state.

Deployment

  1. Set NP_ADMIN_TOKEN in .env.
  2. Validate with docker compose config --quiet.
  3. Wait for active calculations to finish before recreating np-worker.
  4. Build and recreate Web and Worker from the same source revision.
  5. Verify /health, normal OIDC pages, duplicate confirmation, Admin password login, task controls, and the trash lifecycle.

The Web container may be recreated independently for page-only changes, but backend contract changes should deploy Web and Worker from the same image.