notes

2.7 KB

1. # `hl:mpackdb` — the MessagePack file database
3. A document store in plain files. Server realm. The class surface and its
4. generated API docs live in `MPackDB.hl` and `cursor.hl`
5. (`projects/homepage/docs/mpackdb.md`).
7. ```hybriel
8. import { MPackDB } from 'hl:mpackdb'
10. db = new MPackDB(file = 'users.db', primaryKey = '*id', indexes = ['!email', 'age'])
11. id = db.put({ name = "Alice"; email = "[email protected]"; age = 30; }) \\ 1
12. ```
14. ## Primary keys
16. | declaration | key | generated by `put()` |
17. |---|---|---|
18. | `*name` | a number | the table's counter: **1, 2, 3, …** |
19. | `@name` | a string | a random 12-character id |
20. | `name` | a string | nothing — the record must carry it |
22. **A fresh `*id` table starts at 1** (ticket #4, the creator's ruling "start at
23. 1"). The counter is stored as `nextId` in `<base>.meta.json`, and an existing
24. table keeps counting from its stored value. A table written before this rule
25. therefore goes on from wherever it was, and its record 0 remains record 0.
26. A record put with an explicit key takes that key if it is free, 0 included.
28. ## One open table per process (ticket #110)
30. Within one process, a table path has **one** open table, shared by every
31. `open()` — another module instance, another realm, it doesn't matter, they
32. all get the same handle. The file is never compacted or rewritten while a
33. handle on it is live; compaction happens only when asked for explicitly
34. (`compactNow()` / `#handle.compact()`) or when the **last** handle on that
35. path closes. A second `open()` of an already-open table ignores its own
36. `primaryKey`/`indexes`/`compact` arguments — the schema was fixed by whoever
37. opened it first.
39. This matters because compaction rewrites `<base>.mpack` and shifts every
40. record after the first tombstone to a new offset. Before this rule, a second
41. `open()` from anywhere in the process ran its own compact-on-open (ticket
42. #21) against the same file, and the first handle's already-looked-up offsets
43. went stale under it — reads through the first handle then missed or
44. misread records until something compacted again. Related: ticket #21
45. ("opening a table rewrote its files").
47. ## Compatibility with the JS mpackdb
49. The on-disk format is the JS mpackdb 1.0.7 format: `<base>.mpack`,
50. `<base>.meta.json`, one `<base>.<field>.txt` per index, `<base>.idxstate.json`
51. and the `<base>.lock` protocol. Either side reads and changes the other's
52. files. The first id is not part of the format; `nextId` is. Therefore:
54. - a table the JS mpackdb created carries `nextId` and keeps its sequence here,
55. even when that sequence started at 0;
56. - a table Hybriel created carries `nextId` too, and the JS mpackdb continues it;
57. - the one divergence is a table with **no** `.meta.json` yet: Hybriel's first
58. id is 1, the JS mpackdb's is 0.
60. The other documented divergences are listed at the top of `engine.zig`.