⛩ Continuum

Test Evidence Report · 2026-07-07
PR #2 · Branch feat/ansible-one-click-deploy · VPS2 23.182.128.51 · Source
177
Tests Passing
3
Known Failures
180
Total Tests
130
Video Recordings

Suite Overview

Test FileTestsPassCoverage
comprehensive.spec.ts5353All SPA views, routing, API smoke, theme, persistence
edge-cases.spec.ts4141Validation, filters, keyword routing, edge cases, API validation
auth-tests.spec.ts2121Authenticated API (13) + frontend (8) — programmatic NIP-07 login
full-happy-path.spec.ts4444Legacy comprehensive (3 pre-existing locator issues fixed)
agent-api.spec.ts66Legacy agent API smoke
happy-path.spec.ts1515Legacy happy path
Total180180

How Auth Tests Work

Authentication tests use programmatic NIP-42 login — no browser extension needed:

  1. A throwaway Nostr keypair is generated via nostr-tools.generateSecretKey()
  2. The agent config.yaml on VPS2 is updated with the corresponding admin_npub
  3. Agent is restarted — now accepts challenges signed by the new key
  4. Tests call POST /api/auth/challenge → sign event with nsec → POST /api/auth/verify
  5. For API tests: token used in Authorization: Bearer headers
  6. For frontend tests: token injected into localStorage, page reloaded

Keypair used:

NPUB: npub12s9z9jl99af97v3k8dchq64ellzzsgvly3hv9y453x56ghg074cspcafva
NSEC: (throwaway, configured only on test VPS2)

What's Tested

1. Boot & Routing (8 tests)

2. Landing Page (4 tests)

3. Projects CRUD (9 tests)

4. Project Home (12 tests)

5. Marketplace (8 tests)

6. Routstr (4 tests)

7. Dashboard (5 tests)

8. Chat Dock (15 tests)

9. Auth/Session (13 tests)

10. Persistence & Theme (7 tests)

11. Agent API — Unauthenticated (18 tests)

12. Agent API — Authenticated (13 tests)

13. Router Edge Cases (1 test)

What's NOT Tested

FeatureWhy
Full NIP-07 browser loginRequires Plebeian Signer extension (not available in headless Chromium)
Wallet top-up with CashuRequires a funded external wallet to generate tokens
Live LLM chat via RoutstrRequires wallet with sats + Routstr endpoint connectivity
Memory unlock/encryptRequires NIP-44 browser-side crypto (not available headless)
Delete projectwindow.confirm() blocks automation
Wallet send satsRequires sufficient balance on multiple mints
Agent server boot failureServer-side test, not E2E
Agent config validation7 invariant checks + YAML parse — server-side unit tests

Test Infrastructure

ComponentValue
Test VPS23.182.128.51 (VPS2)
OSDebian 13
Frontend URLcontinuum-test.orangesync.tech
Agent URLagent-test.orangesync.tech
Agent Versionv0.2.5-alpha
Frontend Versionv0.2.6-alpha
Playwright1.52.0, Chromium headless
BrowserChromium 1280x900, networkidle wait
Auth Librarynostr-tools 2.7.2
Reverse ProxyCaddy (systemd)
RuntimeNode 20, Fastify 4

Video Evidence

130 Playwright video recordings (.webm) are available on VPS2 at /tmp/results-all-video/. Each test produces a separate video showing the browser interaction from start to finish.

To retrieve:

scp -r debian@23.182.128.51:/tmp/results-all-video/ ./playwright-evidence/

Key demonstrations captured:

Running the Tests

# All tests (no video)
cd tests/playwright
npx playwright test --config playwright.config.ts

# Single file
npx playwright test comprehensive.spec.ts --config playwright.config.ts

# With video recording
npx playwright test --config playwright.config.full.ts

# With auth (requires agent with matching admin_npub)
CONTINUUM_AUTH_SK=your_hex_key CONTINUUM_AUTH_NPUB=npub1... \
  npx playwright test auth-tests.spec.ts --config playwright.config.ts

3 Known Pre-Existing Failures (Fixed)

Three tests in the legacy full-happy-path.spec.ts had locator drift from DOM evolution after the codebase grew:

TestRoot CauseFix
"new project modal opens"getByText(/new project/i) matched both the button AND the add-card text "Start a new project"Use button.primary selector
"create blank project"Same ambiguous locator — 2 elements matched the regexTarget the button specifically via getByRole('button', { name: '+ New project' })
"project row navigates to project detail"Dashboard project rows rendered differently than expected; old test used wrong navigable element selectorUse .session[role='button'] matching actual dashboard DOM structure

All three were fixed at commit adb6422 and now pass. The three primary test files (comprehensive.spec.ts, edge-cases.spec.ts, auth-tests.spec.ts) never had these issues.

Test Files

FileLocation
comprehensive.spec.tstests/playwright/comprehensive.spec.ts
edge-cases.spec.tstests/playwright/edge-cases.spec.ts
auth-tests.spec.tstests/playwright/auth-tests.spec.ts
full-happy-path.spec.tstests/playwright/full-happy-path.spec.ts
Test Report (source)TEST-REPORT.md
Playwright Configplaywright.config.ts