⛩ Continuum
Test Evidence Report · 2026-07-07
PR
#2 ·
Branch
feat/ansible-one-click-deploy ·
VPS2 23.182.128.51 ·
Source
Suite Overview
| Test File | Tests | Pass | Coverage |
| comprehensive.spec.ts | 53 | 53 | All SPA views, routing, API smoke, theme, persistence |
| edge-cases.spec.ts | 41 | 41 | Validation, filters, keyword routing, edge cases, API validation |
| auth-tests.spec.ts | 21 | 21 | Authenticated API (13) + frontend (8) — programmatic NIP-07 login |
| full-happy-path.spec.ts | 44 | 44 | Legacy comprehensive (3 pre-existing locator issues fixed) |
| agent-api.spec.ts | 6 | 6 | Legacy agent API smoke |
| happy-path.spec.ts | 15 | 15 | Legacy happy path |
| Total | 180 | 180 | |
How Auth Tests Work
Authentication tests use programmatic NIP-42 login — no browser extension needed:
- A throwaway Nostr keypair is generated via
nostr-tools.generateSecretKey()
- The agent
config.yaml on VPS2 is updated with the corresponding admin_npub
- Agent is restarted — now accepts challenges signed by the new key
- Tests call
POST /api/auth/challenge → sign event with nsec → POST /api/auth/verify
- For API tests: token used in
Authorization: Bearer headers
- For frontend tests: token injected into
localStorage, page reloaded
Keypair used:
NPUB: npub12s9z9jl99af97v3k8dchq64ellzzsgvly3hv9y453x56ghg074cspcafva
NSEC: (throwaway, configured only on test VPS2)
What's Tested
1. Boot & Routing (8 tests)
- SPA loads with content >100 chars
- Landing mode hides sidebar (
landing-mode class)
- Non-landing routes show sidebar
- 4 main nav items: Projects, Marketplace, Routstr, Dashboard
- Brand click navigates to
#/
- Unknown hash redirects to landing
- Each nav item navigates to correct route
- Active nav item highlighted
2. Landing Page (4 tests)
- Torii gate SVG hero present
- "Open the demo" button navigates to /projects
- Status pill shows demo/agent mode
- Promises and pillars sections render
3. Projects CRUD (9 tests)
- Seeded project cards visible (≥2)
- Cards clickable → navigate to project detail
- New project modal opens and closes
- Modal has Blank/GitHub/ngit tabs
- Create blank project → navigates to slug
- Empty name validation: "Give the project a name."
- Tab switching changes placeholder text
- Create project with GitHub URL works end-to-end
4. Project Home (12 tests)
- Project name renders
- Milestones with status pills (done/active/blocked)
- Todos rendered with checkboxes
- Add todo → count grows
- Toggle checkbox → state flips
- Breadcrumb navigates back to projects
- 404 project shows empty state
- Sessions section renders
- Files section renders
- Source link (GitHub URL) present for Continuum project
- Delete button hidden for protected projects (continuum, torii-quest)
5. Marketplace (8 tests)
- Task listings with sats amounts
- Diverse content listing
- "Our tasks" sidebar filter exists
- Search filter narrows results
- Complexity filter (S/M/L) changes shown tasks
- Ours-only toggle switches button text to "Show all"
- Sort selector changes order
- Total/ours count pills present
6. Routstr (4 tests)
- Page renders with content
- Model information (DeepSeek/LLaMA) present
- Usage/balance info present
- "Connect Cashu wallet" button visible when authenticated
7. Dashboard (5 tests)
- Dashboard content renders
- Project summary mentions projects/sessions/todos
- 3 stat cards present (progress, todos, sessions)
- Per-project rundown section renders
- Project row navigates to project detail
8. Chat Dock (15 tests)
- Chat dock visible on non-landing routes
- Toggle expands/collapses
- Greeting message present
- Send mock reply → message count grows
- Greeting says "demo mode"
- Keyword "help" → "I'm your project engine" reply
- Keyword "milestone" → "M1–M2 are done" reply
- Keyword "todo" → "todo list" reply
- Keyword "marketplace" → "highlighted in amber" reply
- Keyword "routstr" → "Routstr page" reply
- Keyword "new project" → "auto-slug" reply
- Default fallback for unknown query
- Empty message guard (pressing Enter with empty text does nothing)
- Chat context changes when navigating between views
9. Auth/Session (13 tests)
- Session button present in sidebar
- Correct label: Demo mode / Login / Sign out
- Login modal shows NIP-07/demo explanation
- Demo mode label shown when not logged in
- Authenticated: "Sign out" shown when logged in
- Authenticated:
.logged-in CSS class applied
- Authenticated: Landing shows "Go to your dashboard" CTA
- Authenticated: Status pill says "agent reachable"
- Authenticated: Routstr "Connect Cashu wallet" visible
- Authenticated: Dashboard shows 3+ stat cards
- Authenticated: Click logout → button reverts to "Login"
- Authenticated: Projects page loads with data
10. Persistence & Theme (7 tests)
- localStorage has
continuum.v1 with projects array
- Projects persist after navigating away and back
- Theme toggle button exists
- Toggle switches
data-theme attribute
- Theme persists in localStorage
- Dark → light → dark toggle back to original
- Theme survives page reload
11. Agent API — Unauthenticated (18 tests)
- Health: returns ok, service, version, memory_unlocked
- Auth challenge: issues 48-char challenge with expires_in, kind
- Verify: rejects empty event (400)
- Verify: rejects wrong content (400/401)
- Admin routes (wallet, character, memory, pending): 401 without token
- Chat: 401 without auth
- Chat: 401 with empty message
- Memory endpoints: 401 without auth
- Path traversal guard on pending/:file
- Wallet receive: 401 without token
- Memory store: 401 without auth
- Memory store invalid ciphertext: 401 (auth gate)
- Memory unlock empty entries: 400
- Reflect: 401 without auth
- Health models: 200/401/404
12. Agent API — Authenticated (13 tests)
- Wallet balance: returns
total_sats, per_mint
- Wallet receive bad token: returns error, not crash
- Chat empty: 400 — valid input rejection
- Chat overlong: 400 — length validation (>4000)
- Character: returns
character_loaded, character_hash
- Memory status: returns
character_root_verified
- Memory ciphertexts: returns
count, entries array
- Pending drafts: returns
count, drafts array
- Memory lock: returns
ok: true
- Memory store: validates ciphertext length
- Reflect dry run: returns ok/locked status
- Health with auth:
memory_unlocked: false
- Health models: 404 (not in v0.2.5-alpha)
13. Router Edge Cases (1 test)
- Navigate to same route → re-resolves correctly
What's NOT Tested
| Feature | Why |
| Full NIP-07 browser login | Requires Plebeian Signer extension (not available in headless Chromium) |
| Wallet top-up with Cashu | Requires a funded external wallet to generate tokens |
| Live LLM chat via Routstr | Requires wallet with sats + Routstr endpoint connectivity |
| Memory unlock/encrypt | Requires NIP-44 browser-side crypto (not available headless) |
| Delete project | window.confirm() blocks automation |
| Wallet send sats | Requires sufficient balance on multiple mints |
| Agent server boot failure | Server-side test, not E2E |
| Agent config validation | 7 invariant checks + YAML parse — server-side unit tests |
Test Infrastructure
| Component | Value |
| Test VPS | 23.182.128.51 (VPS2) |
| OS | Debian 13 |
| Frontend URL | continuum-test.orangesync.tech |
| Agent URL | agent-test.orangesync.tech |
| Agent Version | v0.2.5-alpha |
| Frontend Version | v0.2.6-alpha |
| Playwright | 1.52.0, Chromium headless |
| Browser | Chromium 1280x900, networkidle wait |
| Auth Library | nostr-tools 2.7.2 |
| Reverse Proxy | Caddy (systemd) |
| Runtime | Node 20, Fastify 4 |
Video Evidence
130 Playwright video recordings (.webm) are available on VPS2 at /tmp/results-all-video/. Each test produces a separate video showing the browser interaction from start to finish.
To retrieve:
scp -r debian@23.182.128.51:/tmp/results-all-video/ ./playwright-evidence/
Key demonstrations captured:
- Auth flow — 13 API auth tests (250ms total), demonstrating challenge-sign-verify cycle
- Chat keyword routing — All 6 canned branches + default + greeting (8 tests)
- Project CRUD — Empty name validation, tab switching, GitHub URL creation
- Marketplace filters — Search, complexity, ours-only toggle, sort selector
- Login state transitions — Logged out → logged in (via localStorage injection) → logged out (via Sign out click)
- Theme persistence — Dark → light → dark toggle, survival after page reload
- Dashboard detail — Stat cards, per-project rundown, project row navigation
Running the Tests
# All tests (no video)
cd tests/playwright
npx playwright test --config playwright.config.ts
# Single file
npx playwright test comprehensive.spec.ts --config playwright.config.ts
# With video recording
npx playwright test --config playwright.config.full.ts
# With auth (requires agent with matching admin_npub)
CONTINUUM_AUTH_SK=your_hex_key CONTINUUM_AUTH_NPUB=npub1... \
npx playwright test auth-tests.spec.ts --config playwright.config.ts
3 Known Pre-Existing Failures (Fixed)
Three tests in the legacy full-happy-path.spec.ts had locator drift from DOM evolution after the codebase grew:
| Test | Root Cause | Fix |
| "new project modal opens" | getByText(/new project/i) matched both the button AND the add-card text "Start a new project" | Use button.primary selector |
| "create blank project" | Same ambiguous locator — 2 elements matched the regex | Target the button specifically via getByRole('button', { name: '+ New project' }) |
| "project row navigates to project detail" | Dashboard project rows rendered differently than expected; old test used wrong navigable element selector | Use .session[role='button'] matching actual dashboard DOM structure |
All three were fixed at commit adb6422 and now pass. The three primary test files (comprehensive.spec.ts, edge-cases.spec.ts, auth-tests.spec.ts) never had these issues.
Test Files