notes

15.2 KB

1. # notes.worldapi.org
3. A notes app in Hybriel (hl:web). Log in with ident; the **sidebar** lists your notes by subject, last edited first;
4. a click opens the note on the right, edited with the Markdown editor `<md-editor>`; **New note** and **Delete note**.
5. The notes are also kept **in the browser (IndexedDB)**: they open and can be edited offline, and sync with the server
6. (and other devices) when the network is back. Concept: `CONCEPT.md`.
8. ## How it works
9. - **Subject** = the first non-empty line of the note without leading `#` (at most 80 characters); an empty note is
10. "Untitled note". No separate title field.
11. - **Order** = `edited` (ms), last first. Only a *changed* text moves a note to the top.
12. - **Pages**: `/` (the list; on a phone the list alone) and `/note/:id` (the note; on a phone it replaces the list,
13. "All notes" leads back; from 50rem wide both show side by side). Signed out: only "Log in with ident".
14. - **Offline + sync** (ticket notes#2): `notes-offline.js` fills the notes area from the browser's own copy (IndexedDB
15. `notes-<user>`), every change is written there first and sent to `/api/notes` about 0.6 s later (and every 10 s a
16. pull looks for other devices' changes; also on "online", on returning to the tab; other tabs follow through
17. BroadcastChannel). `sw.js` (service worker, network first) keeps the page and assets, so a reload works with no
18. network; the page carries no note data. A note made offline gets a local id (`l…`) and its server id at the first
19. push (the address follows). **Conflict** (changed here and elsewhere): the later edit wins, the other version is kept
20. as a copy note ("kept as a copy …" line at its end). A delete of a note changed elsewhere meanwhile is refused (the
21. changed note stays). Deleted notes stay on the server as tombstones (`gone`) so other devices learn of it. Log out
22. first sends what is unsent, then this browser deletes its copy of the notes.
23. - **Private**: every function in `lib/notes.hl` takes the owner's user id and only reaches that user's notes; another
24. user's note id answers "does not exist"; saving/deleting it is refused. A note is at most 200 000 characters
25. (refused, never cut).
26. - **Login**: ident's button flow (`/login/callback` → `?ident_code=` → server-side exchange) plus the identity
27. selector, as in gitoria; the session cookie is `notessid`, idle time 14 days. The session carries the users `@id`
28. only; the ident identity id never leaves the server.
29. - **Short ids (ident#23, antcolony mission 039)**: `users.identity` holds what ident's exchange answers — since ident#23 the identity's
30. public 5-character short id (`a68sz`), before that the per-app id (32 hex); `lib/util.hl isIdentId` accepts both (the old
31. `isHex` check refused short ids). The switch: one-off `tools/migrate-short-ids.hl` (old → short id, idempotent, never
32. `finish`), gate `node tests/short-id-switch.mjs` (ports 8720/8721, no browser), runbook
33. `antcolony-docs/docs/short-id-switch.md` (Byrodin: `/CONTAINERS/projects/antcolony/docs/short-id-switch.md` once synced).
35. ## Files
36. Code order (antcolony `docs/code-order.md`, notes mission 002): `project.hl` is the map (config, routes, audience, an
37. index of which file handles what); one `lib/` file per topic holds its central logic, `lib/<topic>-helpers.hl` what it
38. looks up; the function routes are thin wrappers. `let` only for a variable that is reassigned (or re-bound in a loop
39. body). Imports go one way: util ← users, notes-helpers ← notes ← api-helpers ← api ← project.hl.
41. | file | what |
42. |---|---|
43. | `project.hl` | the map: index comment, PWA config, routes, audience, server |
44. | `lib/notes.hl` | `notes` table (`@id`, index `owner`; `{ owner, text, created, edited, gone, local }`), every write to it: sync push (`pushNote`) / delete (`removeNote`) / conflict rules, `metaRows`, `noteView` |
45. | `lib/notes-helpers.hl` | size limit (`textError`), `isAlive` (tombstones), `stampOf` (version stamps), `copyMark` |
46. | `lib/users.hl` | `users` table (`@id`, `!identity`), ident config + exchange, `userOfLoginCode` (both logins), `userIdOfSession`, `tagOf` |
47. | `lib/api.hl` | the function routes: `/login/callback` (+ `loginFailed`; take `&req`, `&sessions`), `/api/me`, `/api/notes` |
48. | `lib/api-helpers.hl` | JSON `reply`, `isText` / `isTime`, `safePath` (the login's `?next=`) |
49. | `lib/jsoncheck.hl` | JSON syntax pre-check (copy from tickets; workaround for hybriel#6/#12 — delete when fixed) |
50. | `lib/util.hl` | `envOr`, `storageDir`, `countOfList`, `firstOf`, `isHex`, `isIdentId` |
51. | `components/main.hl` | the shell: header, login/logout, faces `notesLogin` / `notesLogOut` |
52. | `components/notes.hl` | the page (`/`, `/note/:id`): only `<note-book id="notesapp">` |
53. | `components/loginfailed.hl` | `/login/failed`: the reason parked in the session |
54. | `components/styles.hl` | all CSS (mobile first; accent = yellow); `shared/tokens.hl` = copy of worldapi-tokens |
55. | `notes-offline.js` | the notes area: sidebar, editor, New / Delete, IndexedDB, sync (served at `/notes-offline.js`) |
56. | `sw.js` | service worker: keeps the page + assets for offline use |
57. | `icons/` | app icons: `icon.svg` (source), `icon-192/512.png`, `apple-touch-icon.png`, `favicon.svg`, `favicon.ico` |
58. | `shared/md-editor.js` | vendored copy of worldapi-components `md-editor.js` (served at `/md-editor.js`) |
59. | `login.js` | ident selector ↔ shell bridge |
60. | `tools/migrate-short-ids.hl` | one-off ident#23 migration (README "Short ids") |
61. | `tests/browser.mjs` | the gate (own server, own ident copy, real Chrome); `deploy.sh` runs it |
62. | `tests/short-id-switch.mjs` | gate of the short-id switch (fixed ports 8720/8721) |
63. | `tests/realdata-baseline.mjs`, `tests/realdata-compare.py` | same-output check of two code trees on a live-data copy (below) |
64. | `tests/letcount.py` | code-order counts: root `.hl` files, project.hl lines, `let`s (`-v` lists any that should be plain) |
66. Data: `storage/mpackdb/{users,notes}.db` (never sent by deploy.sh). Only ever open COPIES of live data.
68. ## PWA (installable app, antcolony mission 046)
69. - `project.hl` `appIcons` / `appTouchIcon` / `appFavicon` / `appThemeColor` (token `darker`, the header) / `appBackgroundColor`
70. (`dark`) → hl:web serves `/__hl/manifest.webmanifest` (name `notes`, start_url/scope `/`, standalone) and links it,
71. the apple-touch-icon and theme-color from every head. Each icon has its own `file` route in `project.hl`.
72. - **Offline is notes' own**, not hl:web's: no `offline = [...]` in `project.hl`, because hl:web would then register
73. `/__hl/sw.js` for scope `/` and replace `/sw.js` (or be replaced by it). `/sw.js` is registered by `notes-offline.js`
74. once signed in; it is the only worker (the gate checks that, and that `/__hl/sw.js` is 404). Moving to hl:web's
75. offline = ticket notes#4.
76. - **Icons** (`icons/`): `icon.svg` = source (512; a note page with a folded corner and a pencil, yellow on
77. rgb(25,30,35); everything inside the maskable safe circle r=204, so one image serves `any` and `maskable`);
78. `favicon.svg` = the same drawing cropped tight, thicker strokes. Render:
79. ```
80. rsvg-convert -w 192 -h 192 icons/icon.svg -o icons/icon-192.png
81. rsvg-convert -w 512 -h 512 icons/icon.svg -o icons/icon-512.png
82. rsvg-convert -w 180 -h 180 icons/icon.svg -o icons/apple-touch-icon.png
83. for s in 16 32 48; do rsvg-convert -w $s -h $s icons/favicon.svg -o /tmp/fav-$s.png; done
84. magick /tmp/fav-16.png /tmp/fav-32.png /tmp/fav-48.png icons/favicon.ico
85. ```
87. ## The sync API (`lib/api.hl`; JSON, the session cookie is the login; 401 signed out, 400 for bad JSON/fields)
88. - `GET /api/me` → `{ user }` · `GET /api/notes` → `{ notes: [{ id, edited, gone }] }` · `GET /api/notes?id=` → `{ id, text, edited, created }`
89. - `POST /api/notes` `{ op:'save', id, local, text, base, at }` → `{ id, edited, text, copy? }` ·
90. `{ op:'delete', id, base }` → `{ ok }` | `{ kept, id, edited, text }`. `base` = the `edited` the device last saw, `at` = when it edited.
92. ## Run and test
93. ```
94. NOTES_PORT=8710 NOTES_WATCH=0 ./bin/hybriel project.hl # dev; needs IDENT_API_KEY/SECRET for a login
95. node tests/browser.mjs # the gate: ports 8701 (ident), 8702 (app), Chrome 8703–8709
96. # (other ports: NOTES_GATE_PORT / _IDENT_PORT / _CHROME=8722-8726)
97. # includes the PWA checks; screenshots in .scratch/gate-*.png
98. ./deploy.sh [--dry-run] # gate → backup → rsync to Byrodin → restart → URL 200 (architect)
99. python3 tests/letcount.py . [-v] # code order: must say `never reassigned 0`, root .hl files 1
100. ```
101. **Same output** (for a cleanup that must not change behaviour; notes mission 002): a copy of the live tables, then the
102. old tree and the new tree each answer every page (signed in as az5b2 = users `0muh6o5a3kkd`, and signed out), every
103. API read, the POST refusals, the login routes/faces against a fake ident, and a fixed write sequence (save, retry,
104. conflict both ways, delete, tombstone, logout); then compare:
105. ```
106. tar -C /CONTAINERS/projects/notes.worldapi.org -cf - storage/mpackdb | ssh loreana tar -C <repo>/.scratch/realdata -xf - # on Byrodin
107. git archive <old commit> | tar -C .scratch/old -xf - && cp -a bin plugins .scratch/old/
108. node tests/realdata-baseline.mjs .scratch/old 8762 .scratch/base-old # app on 8762, fake ident on 8767 (port + 5)
109. node tests/realdata-baseline.mjs . 8762 .scratch/base-new
110. python3 tests/realdata-compare.py .scratch/base-old .scratch/base-new -v # exit 0 = the same
111. ```
112. It masks only source positions, `?v=` hashes, the seed time and (writes) new ids, `edited` stamps and login tags; two
113. runs of the same tree compare clean. Delete the live copy and the outputs afterwards (they hold live notes).
114. Environment: `NOTES_PORT` (dev 8710, container 45007), `NOTES_STORAGE`, `NOTES_SESSIONS`, `NOTES_WATCH=0`,
115. `NOTES_PUBLIC_URL` (default https://notes.worldapi.org), `IDENT_URL`, `IDENT_EXCHANGE_URL`, `IDENT_API_KEY`,
116. `IDENT_API_SECRET` (`.env` beside `project.hl`, never read by workers).
118. ## First deploy (architect)
119. Container `notes.worldapi.org` on `127.0.0.1:45007` replaces the demo-notes at the same address: back the demo up,
120. stop it, point the vhost to 45007 (WebSocket Upgrade headers needed), register the app in ident with the origin
121. `https://notes.worldapi.org`, put the key and secret into `.env`.
123. ## Hybriel notes
124. - `bin/hybriel` + `plugins/` (core crypto data fetch fs http http1 mpackdb proc time web) are vendored = hybriel **master
125. 06617221** (2026-10-03, antcolony mission 074: plugin allocators 3a781359 + 413f60e4 (#126, plugins allocate with malloc via
126. plugin_api.zig), mpackdb 2cb7ae5e, http1 773de63e, f0ac2d2d (plugin ABI field — bin and .so must match); no lambda semantics
127. change), sha256 `21059cc7…d77bdb`, built the same way from `git archive 06617221` (`~/scratch-074/src`, removed); previous copy
128. (190aa11d) in `.scratch/pre-074/`. Gates: browser 50/0 (`NOTES_GATE_PORT=8760 NOTES_GATE_IDENT_PORT=8761
129. NOTES_GATE_CHROME=8762-8769`), short-id-switch 18/0 (temp copy with ports 8760/8761). Memory: `LOADS=N ROUNDS=N [MODE=pull|push]
130. node .scratch/w074/memtest.mjs <app dir> <mpackdb copy> <label> <users @id>` (port 8760, Chrome 8761–8769; loadMarks/apiMarks =
131. RSS every 250). Pages and pulls are flat now; **saves still grow ~0.09 MB per save on old and new alike** (mpackdb update path).
132. Before: master 190aa11d (2026-10-02, antcolony mission 072: fc838894 GC correctness (string index / plugin error read freed memory),
133. 038d84b3 #126 returned closure scopes, #127 d98926c6/04df4428; no lambda semantics change since 8efba065), sha256
134. `860f5e61…0a23626`, built read-only from `git archive 190aa11d` (`~/scratch-072/src`, removed); previous copy (8efba065) in
135. `.scratch/pre-072/`. Gates: browser 50/0 (`NOTES_GATE_PORT=8720 NOTES_GATE_IDENT_PORT=8721 NOTES_GATE_CHROME=8722-8729`),
136. short-id-switch 18/0. Memory: `node .scratch/w072/memtest.mjs <app dir> <mpackdb copy> <label> <users @id>` (port 8720,
137. Chrome 8721-8728; LOADS / ROUNDS env): 200 loads + 200 sync rounds new 114 → 186 → 230 MB, old (8efba065) 124 → 222 → 277 MB;
138. 1000 + 1000: 213 MB after the loads (pages plateau), 486 MB after the sync rounds (~0.27 MB per round, still grows).
139. Before: master 8efba065 (2026-10-02, antcolony mission 069: #126 = f685f240 the interpreter GC also collects by bytes, #48 = 4371b7aa a
140. lambda parameter COPIES its argument (`&p` = reference), #112 #117 #119 #120 opt-in, not used), sha256 `ebf31876…5c0ad2a2`,
141. same build command as below; previous copy (ff51cf46) in `.scratch/pre-069/`.
142. **#48 audit**: no lambda in notes writes a parameter the caller relies on (`project.hl` failed/loginCallback alias
143. `req.session` and save it themselves; faces `on server …(session)` are not lambdas) → no `&` needed. WRONG for
144. `failed(req, why)` with a cookie: its write was lost ("the login did not finish" instead of the reason) — fixed in
145. notes mission 002 by `&req`/`&sessions` (lib/api.hl). Audit:
146. `python3 .scratch/w069/lambdas.py` (writes to params) + `lambdas2.py` (params handed on/aliased). **Memory (#126)**:
147. `node .scratch/w069/memtest.mjs <app copy without .env> <mpackdb copy> <label> <users @id>` (port 8730, Chrome
148. 8722-8729; LOADS / ROUNDS env) — live-data copy, 200 Chrome loads + 200 sync rounds: new 124 → 287 MB, old 97 → 1282 MB.
149. Before that: master ff51cf46 (2026-10-01, antcolony mission 048: #113 mpackdb update() duplicate ids, #115/#116 lambda members on faces, #118 `for`
150. over a literal list, client-built SVG namespace, hl:markdown #111 — notes uses none of these directly; before: e6720b1f,
151. antcolony mission 037: #103/#104 client fixes, #105 `headers`, #106 `let` per loop pass, #94 files in emit;
152. earlier #83 hashed `/__hl/…?v=` URLs + immutable cache, #82 socket reconnect, #95–#97 offline/PWA from project.hl — the
153. manifest (`appIcons`) is used since antcolony mission 046, `offline` is NOT: notes keeps its own sw.js / notes-offline.js). Built from a read-only `git archive master` (`zig build
154. -Doptimize=ReleaseFast -Dtarget=x86_64-linux-gnu.2.39` in `native/`), sha256 `e70631f0…eff3471b`. No local patch:
155. re-vendor = copy binary + plugins, run the gate. Copy before 048 (e6720b1f): `.scratch/pre-048/`.
156. - The notes area is plain JS on purpose: IndexedDB, a service worker and offline edits cannot go through hl:web's
157. faces (they need the socket; it never reopens, hybriel#82). The framework's link click and `popstate` would navigate
158. through the socket, so the area claims its own link clicks and uses `history.replaceState` (Back leaves the app).
159. - Gate needs a secure origin for the service worker: Chrome runs with `--unsafely-treat-insecure-origin-as-secure`.
160. - Server restart closed open tabs' sockets for good (hybriel#82) — fixed in the vendored master since 035 (reconnect); not
161. checked by the notes gate. The notes area still uses its own JS (candidate for hl:web offline, #95–#97).
163. ## History and worker briefs
164. - `LOG.md` — append-only history, one dated line per step (moved here from the antcolony LOG on 2026-10-01).
165. - `missions/NNN-*.md` — worker briefs for this app; `reports/NNN-*.md` — their reports (same name). Numbered per
166. project since 2026-10-01 (antcolony#40); older text, code comments and commits use the old antcolony numbers →
167. map: `/media/STORAGE/projects/antcolony-docs/docs/mission-map.md` (Byrodin: `/CONTAINERS/projects/antcolony/docs/mission-map.md`).