ui-encryption #10

Merged

mhahl-a merged 20 commits from ui-encryption into main 2026-08-15 07:01:39 +00:00

Conversation 0 Commits 20 Files changed 177 +9176 -4952

mhahl-a commented 2026-08-14 03:23:46 +00:00

Owner

Copy link

No description provided.

mhahl-a added 3 commits 2026-08-14 03:23:46 +00:00

ui: modernise project encryption settings and create wizard ... 39590e63d9

Tighten Settings → Encryption into status badges plus a compact
action button, and simplify the new-project encryption step to
selectable cards without long explanatory copy.

feat: external HSM (SoftHSM2) support for per-project BYOK ... 88b3a3c1f5

Add an HSM-backed BYOK tier where the project data-encryption key is wrapped
by an HSM key-encryption key instead of MASTER_KEY:

- app/hsm.py: PKCS#11 wrapper (available, ensure_kek, wrap_dek, unwrap_dek)
  using python-pkcs11; lazily generates an AES-256 KEK in the token.
- crypto.py: provider-aware DEK resolution (local -> MASTER_KEY, hsm -> HSM).
- project_keys.py: ensure_project_key(provider='hsm') stores key_provider='hsm'
  + kms_key_ref; rewrap_project_keys skips HSM rows.
- Onboarding wizard + settings: 'External HSM' option (gated on availability
  and owner/admin); settings Adopt gains a provider select.
- Containerised SoftHSM2 for dev: softhsm2/ service (Dockerfile + init.sh)
  initialising a token into a shared hsmdata volume; app Dockerfile installs
  softhsm2 and mounts the same volume; compose wires HSM_* env.
- config.py HSM_* env vars, python-pkcs11 dependency, docs/dev/hsm.md guide,
  and byok/architecture/database doc updates; tests for the HSM path.

fix: harden HSM BYOK (Fernet DEK wrap, SoftHSM mount, UX) ... 400ce24ece

Accept Fernet.generate_key() material in wrap/unwrap (32 raw bytes under
HSM), prefer AES key-wrap when available, open RW PKCS#11 sessions, and
guard project create when HSM is unavailable. Mount SoftHSM at /hsm/tokens
with shared conf + healthcheck so appuser can access the token; tighten
encryption UI copy and provider cards. Add DEK-contract tests and install
build tools for python-pkcs11 in the image.

mhahl-a added 1 commit 2026-08-14 03:33:06 +00:00

feat(hsm): onboarding polish and local->HSM migration ... 21073761d2

- Cache hsm.available() with a 30s TTL and log session-open failures.
- Surface real HSM errors in create/adopt/migrate flash messages; pass the
  key provider through adopt_project_key.
- Add local-BYOK -> HSM migration (migrate_project_key + 'Migrate to HSM'
  action, audited as project_key_migrated).
- Wizard: HSM health indicator + KEK label, trust-boundary explanation,
  HSM confirm dialog, sr-only legend; settings: 'all synced' note + legend.
- HSM/BYOK badges in project lists via api.project_key_provider()
  (migration 0025).
- Admin HSM deployment guide + tests (wizard card, adopt/migrate routes,
  migrate_project_key).

mhahl-a added 1 commit 2026-08-14 03:43:42 +00:00

feat(admin): Encryption settings tab and HSM key management ... a5e11bea87

Add a Server Settings -> Encryption tab for global admins:

- MASTER_KEY status (default vs custom) + rotation guidance.
- External HSM status panel (connected, module/token/KEK, error) with a
  wrap/unwrap round-trip health check and 'how to enable' guide.
- Rotate HSM KEK (re-wrap all HSM-backed DEKs) and bulk 'migrate all local
  BYOK to HSM' actions, plus rekey-hsm-kek CLI.
- Project encryption posture table with summary counts.

Supporting: hsm.status()/test_roundtrip()/generate_kek(label) + label-aware
wrap/unwrap, crypto passes kms_key_ref, project_keys.rotate_hsm_kek/
encryption_summary/migrate_all_local_to_hsm, config.master_key_is_default,
sidebar Encryption link, team-page encryption summary, and per-project key id.
Documents HSM KEK rotation, bulk migration, and the intentionally-unoffered
revert-to-managed / require-HSM policy.

mhahl-a added 1 commit 2026-08-14 03:48:07 +00:00

fix(ui): stringify UUID before slicing project key id ... 966bec5146

Postgres returns key_id as a UUID object; Jinja [:8] raised TypeError
on the project Settings encryption panel.

mhahl-a added 1 commit 2026-08-14 03:56:46 +00:00

ui: declutter encryption onboarding, HSM status, and project lists ... bc32a378e3

Simplify the new-project wizard cards, render External HSM status as a
table, move posture search under Server settings only (drop sidebar
Encryption link), and show Managed/Local/HSM as a list column instead of
BYOK/HSM badges.

mhahl-a added 1 commit 2026-08-14 04:20:14 +00:00

fix(hsm): address review bugs and polish ... d05a97bf56

- decrypt_for_project: raise user-facing ValueError when the HSM is offline
  (RuntimeError) instead of a 500.
- hsm.status(): read-only session; reuse cached available() and short-circuit.
- rotate_hsm_kek: single admin connection/transaction (was N per-project).
- encryption_summary: fold pending count into the query (no per-project loop).
- Project lists: batch key-provider lookup via api.project_key_providers()
  (migration 0026) instead of N+1 calls.
- Add hsm.test_connection() (non-destructive) and use it for the admin HSM
  health check; add hsm.delete_kek() for post-rotation cleanup.
- Audit hsm_kek_rotate / hsm_migrate_all as org events; add encryption-key
  constants + admin audit filter; key-ID column in project posture table.
- Docs: fix BYOK/MASTER_KEY trust-path wording and expand code maps.

mhahl-a added 1 commit 2026-08-14 04:58:44 +00:00

feat(hsm): multi-slot support and PKCS#11 URL configuration ... d54f6babbd

Add named HSM slots so projects can be linked to specific HSMs via RFC 7512
PKCS#11 URLs instead of the single global env-var config.

- Migration 0027: private.hsm_slots + project_crypto_keys.hsm_slot_id and
  SECURITY DEFINER list/upsert/delete/url helpers (admin-gated).
- hsm.py: parse_pkcs11_url/redact_pkcs11_url (+ redact_pin Jinja filter),
  slot-aware _for_slot wrappers, and explicit-label wrap for KEK rotation.
- crypto.py: slot-URL resolution (_slot_url cache) and slot-aware DEK unwrap
  with legacy fallback.
- project_keys: ensure/adopt/migrate with slot/target-slot, between-slot
  re-wrap without re-encryption, per-slot KEK rotation, link_legacy_to_slot.
- Admin: slot CRUD/test/link actions + HSM slots section; project wizard and
  Migrate-to-HSM target-slot selectors; slot badge/column in UI.
- Docs: PKCS#11 URL format, multi-slot architecture, Kubernetes PIN setup.
- Tests: URL parser/redact, slot-aware project-key paths.

mhahl-a added 1 commit 2026-08-14 11:12:22 +00:00

refactor(hsm): remove legacy env-var HSM config (named slots only) ... 0510b81777

HSM-backed BYOK now requires a named slot (PKCS#11 URL in private.hsm_slots);
the global HSM_PKCS11_MODULE/HSM_TOKEN_LABEL/HSM_PIN/HSM_KEK_LABEL env vars
are removed.

- hsm.py: drop available/status/test_connection/test_roundtrip/ensure_kek/
  kek_label/wrap_dek/unwrap_dek; generate_kek/delete_kek take a pkcs11_url.
- crypto: _dek_for raises when an HSM project lacks a slot/URL (surfaced as
  'HSM is unavailable' by decrypt_for_project).
- project_keys: ensure/migrate/migrate_all require a slot; rotate_hsm_kek is
  per-slot; link_legacy_to_slot remains for pre-slot projects.
- Routes/UI: HSM gated on slot existence; adopt/migrate require slot fields;
  per-slot Test/Rotate/Link/Delete kebab menu; bulk migrate takes a slot.
- CLI: rekey-hsm-kek --slot-id. Compose drops HSM_* env (keeps SOFTHSM2_CONF).
- Docs + tests updated to the slot-only model.

mhahl-a added 1 commit 2026-08-14 11:28:46 +00:00

ui: remove bulk migrate, add kebab + pager to posture table, modernise add-slot ... ea9667ca4d

- Remove the 'Migrate N local → HSM' bulk form from the HSM slots section.
- Add a kebab menu column to the Project posture table: 'Migrate to HSM' for
  local projects, 'Re-encrypt' for HSM projects with pending rows.
- Paginate the posture table (25/projects per page) using paging.page_window.
- Wrap the add-slot form in fieldset.field-group with a legend, matching the
  project settings form pattern.

mhahl-a added 1 commit 2026-08-14 11:40:57 +00:00

ui: HSM slot wizard, posture table pagination, remove summary ... 147776dab6

Replace the inline HSM slot add form with a dedicated wizard page
(/settings/encryption/hsm-slots/new) that validates the PKCS#11 URL
via a Test-connection button before the Create button saves it.

Also add pagination to the Project posture table (25/projects per page)
and remove the summary counts line from the bottom.

mhahl-a added 1 commit 2026-08-14 11:45:04 +00:00

scripts: seed dev HSM slot in seed_mock.py ... 29b4d6766c

Insert a dev-hsm slot into private.hsm_slots with the SoftHSM2 PKCS#11 URL,
idempotent via ON CONFLICT DO NOTHING. Clears the slot URL cache after.
Called from the existing main() seed flow.

mhahl-a added 1 commit 2026-08-15 02:17:52 +00:00

HSM security & usability: edit slots, save-without-testing, status badges, legacy warnings ... f047f6218a

Route (settings.py):
- Wizard supports edit mode (?slot_id= pre-fills form)
- Save-without-testing (force_create action) for offline HSM setup
- PKCS#11 URL validation before connection test
- Slot availability (Connected/Offline) computed on page load
- Audit events for slot CRUD (hsm_slot_added/edited/deleted/linked)
- Inline PIN warning in flash message
- hsm_slot_test uses db.as_user() instead of db.connect_admin()
- Migrate-all slot selector reads target_slot from form

Templates:
- settings.html: legacy warning banner, slot status badge column,
  Edit menu item, rotate confirm with slot name, migrate-all form,
  posture table '⚠ No slot' instead of 'Legacy'
- hsm_slot_new.html: dynamic New/Edit title, slot_id hidden field,
  Save-without-testing button, PIN source guidance hint
- team_new_project.html + project.html: slot description in option title
- project.html: '⚠ No slot' label in crypto status

HSM (hsm.py):
- parse_pkcs11_url catches OSError on PIN file read
- test_connection_for_slot returns True when KEK missing (lazy creation)
- has_inline_pin() helper for inline PIN detection

Project keys (project_keys.py):
- link_legacy_to_slot verifies slot reachability before linking

Tests:
- test_hsm_slots: has_inline_pin, test_connection_for_slot, pin-source error
- test_hsm: wizard edit pre-fill, create/force_create/edit audit events
- test_project_keys: link_legacy_to_slot reachability check, unreachable raises

Docs:
- docs/dev/hsm.md: has_inline_pin, test_connection_for_slot behavior,
  link_legacy_to_slot reachability
- docs/admin/byok.md: edit slot, status badge, save-without-testing,
  migrate-all selector, legacy warning banner, '⚠ No slot' label

mhahl-a added 1 commit 2026-08-15 02:29:19 +00:00

security: harden HSM slots and RLS authorization 14f5712769

mhahl-a added 1 commit 2026-08-15 02:46:51 +00:00

quality: document Python code and add live API smoke tests d5d53cd3b0

mhahl-a added 1 commit 2026-08-15 03:32:07 +00:00

Restructure: group flat app modules into packages ... 25446bd137

Move 24 flat app/ modules into 6 logical packages:
- core/ (config, db, schema, migrations, settings_svc)
- crypto/ (__init__ from crypto.py, project_keys, hsm)
- auth/ (authz, lockout, passwords, totp_svc, user_sessions, pats, rbac_sync)
- integrations/ (ldap_auth, oidc_auth, mailer, dir_sync)
- secret_svc/ (secret_ops, secret_kinds)
- ui/ (nav, pins, paging)

Update all imports across ~80 files (app, routes, lib, tests, scripts).
Update tox.ini pylint paths, pyproject.toml isort first-party list.
Update docs (architecture, hsm, database, testing, configuration, deploy).
Update AGENTS.md and Dockerfile comments.

mhahl-a added 1 commit 2026-08-15 03:49:14 +00:00

security: harden RLS privilege boundaries 0a8b64c23e

mhahl-a added 1 commit 2026-08-15 04:18:12 +00:00

db: squash migrations into fresh-install baseline 4d9d2998ba

mhahl-a added 1 commit 2026-08-15 07:01:30 +00:00

infra: move application caches to Redis 2fbb8f0073

mhahl-a merged commit b2d8448ade into main 2026-08-15 07:01:39 +00:00

mhahl-a referenced this pull request from a commit 2026-08-15 07:01:39 +00:00

Merge pull request 'ui-encryption' (#10) from ui-encryption into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Sigaint/corvus!10

No description provided.