np_app/docs/ARCHITECTURE.md

# NP Replica Architecture

## Goals

NP Replica separates user navigation, durable account data, live queue state,
and NUPACK compute processes. Scientific calculation code remains isolated from
page rendering and account administration so UI changes do not alter numerical
semantics.

## Runtime Topology

```text
Browser
  |
  | HTTPS / OIDC session
  v
np-replica (Web/API, 4 GB)
  |                 |
  | live state      | durable metadata/results
  v                 v
Redis             SQLite
  |
  | job queue
  v
np-worker (NUPACK compute, 56 GB)
```

- **Web/API** serves the multi-page UI, validates ownership, accepts jobs, and
  serializes API responses. It does not run Redis-backed NUPACK jobs.
- **Redis** owns queue order, active job state, heartbeats, duplicate claims,
  sessions, and temporary result state.
- **SQLite** owns users, durable job inputs/results, usage totals, and shares.
- **Worker** consumes Redis jobs and runs NUPACK in isolated child processes.

## Frontend Structure

The UI uses a small multi-page application structure. Each page has one primary
operational purpose.

| Route | File | Responsibility |
| --- | --- | --- |
| `/` | `service/home.html` | Queue overview, usage summary, recent jobs |
| `/workspace` | `service/index.html` | Analysis, Design, Utilities, result inspection |
| `/cloud` | `service/cloud.html` | Job search, JSON import/export, result links, cancellation, deletion, shares |
| `/account` | `service/account.html` | Identity, account usage, worker resource information |
| `/admin` | `service/admin-login.html` | Dedicated Admin password login |
| `/admin/panel` | `service/admin.html` | Cross-account task and trash management |

`service/static/app-shell.css` provides the shared application shell and
responsive operational layout. `service/static/portal.js` contains shared
request, formatting, navigation, table, Cloud, Account, and Admin controllers.
The calculation workspace retains its self-contained renderer because it
contains specialized structure, heatmap, force-layout, and result-export code.

## Backend Boundaries

- `service/server.py`
  - HTTP routes and authentication gates
  - Redis live-job repository and queue
  - resource planning and worker orchestration
  - NUPACK request validation and calculation adapters
- `service/account.py`
  - SQLite schema and durable repository
  - account-scoped job/share queries
  - cross-account Admin queries and trash lifecycle
- `service/split_strand_svg.py`
  - structure rendering adapter

The next safe backend extraction point is moving Redis job coordination into a
`job_repository.py` module. NUPACK adapters can then move into separate
`analysis_service.py`, `design_service.py`, and `utilities_service.py`
modules without changing HTTP contracts.

## Job Submission And Duplicate Control

Duplicate detection is scoped by `user_id` and a canonical SHA-256 hash of the
JSON payload. Different users never share duplicate state.

```text
POST /api/jobs
  |
  +-- no active match --> 202 accepted
  |
  +-- active match ----> 409 duplicate + existing job metadata
                           |
                           +-- user cancels: no write
                           |
                           +-- user confirms:
                               POST /api/jobs?force=1 --> 202 accepted
```

Redis uses an atomic claim key. Terminal updates delete a claim only when it
still belongs to that job, preventing an older job from deleting a newer forced
submission's claim. The in-memory development backend mirrors this behavior
under `JOB_LOCK`.

## Resource Planning

`NP_WORKER_CONCURRENCY=auto` selects the smaller of:

- effective CPU capacity divided by `NP_PER_JOB_THREAD_LIMIT`;
- usable worker memory divided by `NP_ESTIMATED_JOB_MEMORY_GB`.

CPU detection considers process affinity and cgroup quota. Memory detection
uses an explicit override, cgroup memory limit, or host memory in that order.
The published plan is visible through `/health`.

These controls affect scheduling only. They do not change models, sequence
constraints, complexes, thermodynamic parameters, or result serialization.

## Authentication And Authorization

- Normal pages and account APIs require an OIDC-backed session.
- Job, history, and share APIs enforce ownership by stable OIDC `user_id`.
- Public share routes expose only the selected shared record.
- Admin uses a dedicated password configured by `NP_ADMIN_TOKEN` and bypasses
  OIDC. The password is accepted only in the JSON body of
  `POST /api/admin/login`; it is never embedded in a route, redirect, cookie, or
  local storage value.
- Successful login creates a random session in Redis (or process memory in
  development) and sets a short-lived HttpOnly, SameSite=Strict cookie.
- Admin may list all durable job metadata, stop active jobs, move terminal jobs
  to trash, restore jobs, permanently delete trashed jobs, and configure trash
  retention. APIs do not return stored payload or result blobs.

The Admin password must be random, kept in the ignored `.env`, and rotated after
exposure. Keep `NP_ADMIN_COOKIE_SECURE=1` in HTTPS deployments.

## Trash Lifecycle

Normal user deletion sets `deleted_at` and `purge_after` rather than deleting a
job row. Deleted jobs disappear from account history, usage, and result APIs;
all share records for that job are deleted. Admin can inspect the trash, restore a
job, or permanently delete it. Expired rows are purged lazily during job-list
reads. Retention is stored in SQLite `app_settings`, defaults to two days, and
can be changed from one hour to 365 days.

## Persistence And Recovery

- Redis AOF persists live queue state under `runtime/redis`.
- SQLite WAL persists account data under `runtime/account`.
- Worker recovery requeues jobs marked running after an interrupted worker.
- Active Redis job records do not expire while heartbeat updates continue;
  terminal records use `NP_JOB_TTL_SECONDS`.

Back up both runtime directories. SQLite is the durable source for the Cloud and
Admin pages; Redis is the source for current queue and heartbeat state.

## Deployment

1. Set `NP_ADMIN_TOKEN` in `.env`.
2. Validate with `docker compose config --quiet`.
3. Wait for active calculations to finish before recreating `np-worker`.
4. Build and recreate Web and Worker from the same source revision.
5. Verify `/health`, normal OIDC pages, duplicate confirmation, Admin password
   login, task controls, and the trash lifecycle.

The Web container may be recreated independently for page-only changes, but
backend contract changes should deploy Web and Worker from the same image.