Spec: Per-package KCL config fragments — service-owned defaults (db, embeddednats, embeddedfga) #14

New issue

Closed

opened 2026-08-06 21:23:44 +00:00 by ginjiruu · 2 comments

ginjiruu commented 2026-08-06 21:23:44 +00:00

Owner

Copy link

Problem Statement

Each toolbelt service package (db, embeddednats, embeddedfga) owns default values for its data locations (NATS data/nats, SQLite data/sqlite/<app>.db, FGA file://data/fga/openfga.db), but those defaults are hand-written in two places today: as Go consts/values in the service package and as schema DB/schema NATS/schema FGA blocks repeated in every app's schema.k. Every app re-declares the service schemas it uses, and the platform's default for a service can drift from what apps ship. The composition root (toolbelt/app) reads platform fields by name, but today each app hand-wires that wiring in internal/app/app.go.

Apps should be able to compose a service's config with one line (nats: NATS = NATS {}) and inherit the service's schema and defaults without re-declaring them — and the service package should own its own defaults in exactly one place.

Solution

Service packages ship their own KCL fragment: db/schema.k, embeddednats/schema.k, embeddedfga/schema.k, plus a toolbelt-owned core fragment app/schema.k (app identity, logging, session, rebuild, addr). Apps compose them via native KCL imports (import toolbelt.embeddednats) in their schema.k, so the schema the app authors is the schema that is validated and the defaults the service ships are the defaults the app inherits. The universal Log schema is no longer embedded in the tool — it moves into the app fragment. kcl-generate resolves these imports at build time instead of string-merging fragment sources; the runtime .k loader resolves the same imports via kcl.mod. Service packages drop their hand-written Go const defaults; the composition root always passes the resolved field value.

This ticket reshapes kcl-generate + example/; consumer migration (go-template, ttrpg) is a separate later project.

User Stories

1. As an app author, I want each service package to ship its own KCL fragment declaring its schema and defaults, so that I never re-declare a service's config in my schema.k.
2. As an app author, I want to compose a service's config with a one-line KCL import (nats: embeddednats.NATS = embeddednats.NATS {}), so that the service's fields and defaults appear in my generated config without hand-copying.
3. As an app author, I want to override a service-neutral default for my app (e.g. the SQLite filename) with an explicit composition value, so that app identity lives in my schema where I can see it.
4. As an app author, I want the schema I write to be the schema that is validated at runtime, so that codegen and runtime never disagree about what fields exist.
5. As an app author, I want kcl-generate to fail loudly when a required field (e.g. db.path) is not set, so that I cannot ship an app with a missing data location.
6. As a platform user, I want the universal Log schema to come from the platform's own fragment rather than the tool's embedded copy, so that platform-owned fields live with the platform.
7. As a platform user, I want service packages to drop their hand-written Go const defaults, so that each default exists in exactly one place (the fragment).
8. As a composition-root author, I want SetupNATS to take the store directory as a required parameter, so that the composition root always passes the resolved field value and no hidden Go default survives.
9. As a composition-root author, I want to read platform fields by typed path readers against the app's registry, so that toolbelt/app does not depend on app-generated field symbols.
10. As a platform maintainer, I want apps to compose platform fragments under canonical keys (nats:, db:, fga:, log:, session:, rebuild:), so that the composition root's path-based reads cannot be silently broken by a renamed key.
11. As a platform maintainer, I want the canonical-key contract enforced at codegen, so that a mis-named composition fails while the app author is still in the editor, not at runtime.
12. As a composition-root author, I want a hard error at New() when a required platform field is absent, so that misconfiguration surfaces at startup instead of as an empty string.
13. As an app author, I want fragments to be able to import other toolbelt fragments, but never the app's own schema, so that toolbelt never depends on a downstream app.
14. As a platform maintainer, I want kcl.mod to reference the toolbelt fragments at the same revision as the Go module, so that published pseudo-version consumers resolve fragments correctly.
15. As an app author, I want the FGA File/Tuple schemas to ride the embeddedfga fragment, so that tuple-sync config is service-owned.
16. As an app author, I want the existing schema names (NATS, DB, FGA, Log) preserved, so that my schema composes naturally via qualified imports.
17. As a platform maintainer, I want fragment-vs-fragment schema name collisions to be KCL module-level errors, so that no merge-precedence logic is needed.
18. As a platform maintainer, I want option() available for genuine per-fragment knobs but not used as the app-identity mechanism, so that app identity stays visible in the schema composition.

Implementation Decisions

• KCL-native fragment imports (Q1). Each service package ships a fragment file in its own directory (db/schema.k, embeddednats/schema.k, embeddedfga/schema.k), plus a toolbelt-owned core fragment app/schema.k. The app's schema.k imports them via import toolbelt.<pkg> and composes via nats: embeddednats.NATS = embeddednats.NATS {}. kcl-generate resolves these imports (build-time) instead of string-merging fragment sources; the runtime .k loader resolves the same imports via kcl.mod. No generated/merged schema file is emitted — the authored schema.k is the runtime schema.

• Universal Log moves to app/schema.k (Q5). The tool's embedded universal_schema.k is removed. The app fragment owns app identity (name), Log, Session, Rebuild, and addr as service-neutral or identity defaults. Resolves the coordination from ticket #4 (toolbelt/app owns the composition-root bootstrap).

• Service-neutral vs app-owned defaults (Q3, Q8, Q12). Fragments ship service-neutral defaults (NATS data/nats, FGA file://data/fga/openfga.db, mode=embedded, datastore=sqlite, the check: block). App identity values (SQLite filename, FGA store_id) live in the app's composition overrides: db: DB = DB { path = "data/sqlite/kanban.db" }. A prototype showed KCL option() does not observe the app's composition value, so option() is not the identity mechanism — explicit composition overrides are. db.path has no default at all: the app must compose it, and kcl-generate fails loudly if it is absent.

• Schema names preserved (Q14). Fragment schema names stay NATS, DB, FGA, Log. The KCL-qualified import (embeddednats.NATS) disambiguates; no renaming churn for consumers.

• Collision semantics (Q6). App wins by simply not importing a fragment and defining its own schema. Fragment-vs-fragment schema name collisions are KCL module-level errors. No merge-precedence code in kcl-generate.

• app/schema.k ownership and scope (Q4, Q11, Q15). This ticket ships fragments for db, embeddednats, embeddedfga, and the app core fragment. File/Tuple (FGA tuples) ride embeddedfga/schema.k. The OIDC schema stays app-authored in this ticket; authn/schema.k (declaring the standard OIDC schema) is ticket #4's responsibility — a coordination comment was posted on #4.

• Field-path contract (Q10, Q13). toolbelt/app reads platform fields via typed path readers against the app-local registry (the config.New object from ticket #3). A prototype showed the alternatives are unbuildable: sharing toolbelt-owned *Field pointers across apps mutates shared state at init; path-string lookup alone silently degrades on unknown paths. Therefore: canonical composition keys (nats:, db:, fga:, log:, session:, rebuild:) are enforced at codegen, and toolbelt/app fails loudly at New() when a required platform field is absent. Recorded as the #4↔#12 coordination in app-bootstrap.md.

• embeddednats Go API (Q7). SetupNATS(ctx, logger, storeDir string) — the store directory becomes a required positional parameter. DefaultStoreDir const and the WithStoreDir option are removed. app/init.go drops the empty-string guard. ttrpg's two WithStoreDir(t.TempDir()) test calls become positional.

• kcl.mod wiring (Q2, Q9). Fragment discovery needs no flag list: the app declares its fragments via import + kcl.mod. A Taskfile stamp step derives the toolbelt fragment reference from go list -m and rewrites the toolbelt entry in kcl.mod, so kcl.mod cannot drift from go.mod — including for published pseudo-version consumers.

• Toolbelt does not import downstream (Q3). Fragments may import other toolbelt fragments but never the app's schema; toolbelt never depends on a downstream app.

• No runtime merging (decision 3, revised). The runtime .k loader already evaluates the config program and now resolves schema imports via kcl.mod — same resolution path as codegen. What dies is string-merging at codegen and any runtime notion of merging. The generated var DefaultX ldflags-overridable pattern survives: defaults are still evaluated at codegen into Go vars.

• The #@kcl:validate = "ValidateFGAURI" directive rides the FGA fragment and references config.ValidateFGAURI (resolvable because the app imports toolbelt/config). No new mechanism.

Testing Decisions

Good tests here exercise external behavior: a schema.k plus fragment files plus a kcl.mod in a temp dir, in → generated fields/config out (or a loud error out). They must not reach into kcl-generate's internals beyond what the existing tests already exercise (parse → collect → evaluate → render pipeline).

• Primary seam: tools/kcl-generate/main_test.go (existing prior art: TestFlattenSchema, TestEvaluateDefaults, TestRenderConfigStruct, TestFieldCollision, TestUnknownDirective). Add tests that drive the full pipeline against a schema.k that imports fragment files:

  • fragment import resolution produces fields with the fragment's defaults (nats.store_dir → data/nats);
  • a canonical-key violation (jetstream: NATS = NATS {}) fails codegen;
  • a missing required db.path fails codegen;
  • app-wins-by-not-importing works (app defines its own NATS and kcl-generate accepts it);
  • fragment-vs-fragment schema collision surfaces as an error.

• Secondary seam: config/kcl/kcl_test.go (existing prior art: TestLoadKConfigWithOptions, TestLoadKConfigRelativePath, TestValidateConfig). Add one test proving a config file → schema.k → fragment import chain resolves at runtime load (the authored schema.k is the validated schema).

• Integration: the example's check-kcl drift guard (existing Taskfile task). After the example/ reshape, task check-kcl regenerates and fails on stale output, proving a real app composes fragments end-to-end.

Out of Scope

• Consumer migration (go-template, ttrpg) — separate later project, per the config-bootstrap plan.
• authn/schema.k (the OIDC fragment) — owned by ticket #4; coordination comment posted on #4.
• toolbelt/app's typed path readers and New() fail-loud behavior — the mechanism is specified here as a contract, but the implementation lands with ticket #4 (composition-root bootstrap) which depends on ticket #3's config.New object.
• config.New object bootstrap — ticket #3's decision; this ticket's fragments feed the same field registry shape it already expects.

Further Notes

• Prototypes were run during the grilling session: (a) KCL option() in schema defaults does not observe the app's composition value — confirming explicit composition overrides as the identity mechanism; (b) field-path read mechanisms — confirming that shared toolbelt-owned *Field pointers are unbuildable (shared mutable state) and that typed path readers with fail-loud are the correct contract.
• CONTEXT.md terminology was updated (Config fragment, Service-owned default, App-owned default); the fuller post-resolution glossary update is tracked in ticket #13.
• The canonical-composition-key contract is a naming convention KCL cannot enforce; codegen enforcement + runtime fail-loud are the two enforcement points.

## Problem Statement Each toolbelt service package (`db`, `embeddednats`, `embeddedfga`) owns default values for its data locations (NATS `data/nats`, SQLite `data/sqlite/<app>.db`, FGA `file://data/fga/openfga.db`), but those defaults are hand-written in two places today: as Go consts/values in the service package and as `schema DB`/`schema NATS`/`schema FGA` blocks repeated in every app's `schema.k`. Every app re-declares the service schemas it uses, and the platform's default for a service can drift from what apps ship. The composition root (`toolbelt/app`) reads platform fields by name, but today each app hand-wires that wiring in `internal/app/app.go`. Apps should be able to compose a service's config with one line (`nats: NATS = NATS {}`) and inherit the service's schema and defaults without re-declaring them — and the service package should own its own defaults in exactly one place. ## Solution Service packages ship their own KCL fragment: `db/schema.k`, `embeddednats/schema.k`, `embeddedfga/schema.k`, plus a toolbelt-owned core fragment `app/schema.k` (app identity, logging, session, rebuild, `addr`). Apps compose them via native KCL imports (`import toolbelt.embeddednats`) in their `schema.k`, so the schema the app authors is the schema that is validated and the defaults the service ships are the defaults the app inherits. The universal `Log` schema is no longer embedded in the tool — it moves into the `app` fragment. `kcl-generate` resolves these imports at build time instead of string-merging fragment sources; the runtime `.k` loader resolves the same imports via `kcl.mod`. Service packages drop their hand-written Go const defaults; the composition root always passes the resolved field value. This ticket reshapes `kcl-generate` + `example/`; consumer migration (go-template, ttrpg) is a separate later project. ## User Stories 1. As an app author, I want each service package to ship its own KCL fragment declaring its schema and defaults, so that I never re-declare a service's config in my `schema.k`. 2. As an app author, I want to compose a service's config with a one-line KCL import (`nats: embeddednats.NATS = embeddednats.NATS {}`), so that the service's fields and defaults appear in my generated config without hand-copying. 3. As an app author, I want to override a service-neutral default for my app (e.g. the SQLite filename) with an explicit composition value, so that app identity lives in my schema where I can see it. 4. As an app author, I want the schema I write to be the schema that is validated at runtime, so that codegen and runtime never disagree about what fields exist. 5. As an app author, I want `kcl-generate` to fail loudly when a required field (e.g. `db.path`) is not set, so that I cannot ship an app with a missing data location. 6. As a platform user, I want the universal `Log` schema to come from the platform's own fragment rather than the tool's embedded copy, so that platform-owned fields live with the platform. 7. As a platform user, I want service packages to drop their hand-written Go const defaults, so that each default exists in exactly one place (the fragment). 8. As a composition-root author, I want `SetupNATS` to take the store directory as a required parameter, so that the composition root always passes the resolved field value and no hidden Go default survives. 9. As a composition-root author, I want to read platform fields by typed path readers against the app's registry, so that `toolbelt/app` does not depend on app-generated field symbols. 10. As a platform maintainer, I want apps to compose platform fragments under canonical keys (`nats:`, `db:`, `fga:`, `log:`, `session:`, `rebuild:`), so that the composition root's path-based reads cannot be silently broken by a renamed key. 11. As a platform maintainer, I want the canonical-key contract enforced at codegen, so that a mis-named composition fails while the app author is still in the editor, not at runtime. 12. As a composition-root author, I want a hard error at `New()` when a required platform field is absent, so that misconfiguration surfaces at startup instead of as an empty string. 13. As an app author, I want fragments to be able to import other toolbelt fragments, but never the app's own schema, so that toolbelt never depends on a downstream app. 14. As a platform maintainer, I want `kcl.mod` to reference the toolbelt fragments at the same revision as the Go module, so that published pseudo-version consumers resolve fragments correctly. 15. As an app author, I want the FGA `File`/`Tuple` schemas to ride the `embeddedfga` fragment, so that tuple-sync config is service-owned. 16. As an app author, I want the existing schema names (`NATS`, `DB`, `FGA`, `Log`) preserved, so that my schema composes naturally via qualified imports. 17. As a platform maintainer, I want fragment-vs-fragment schema name collisions to be KCL module-level errors, so that no merge-precedence logic is needed. 18. As a platform maintainer, I want `option()` available for genuine per-fragment knobs but not used as the app-identity mechanism, so that app identity stays visible in the schema composition. ## Implementation Decisions - **KCL-native fragment imports (Q1).** Each service package ships a fragment file in its own directory (`db/schema.k`, `embeddednats/schema.k`, `embeddedfga/schema.k`), plus a toolbelt-owned core fragment `app/schema.k`. The app's `schema.k` imports them via `import toolbelt.<pkg>` and composes via `nats: embeddednats.NATS = embeddednats.NATS {}`. `kcl-generate` resolves these imports (build-time) instead of string-merging fragment sources; the runtime `.k` loader resolves the same imports via `kcl.mod`. No generated/merged schema file is emitted — the authored schema.k is the runtime schema. - **Universal `Log` moves to `app/schema.k` (Q5).** The tool's embedded `universal_schema.k` is removed. The `app` fragment owns app identity (`name`), `Log`, `Session`, `Rebuild`, and `addr` as service-neutral or identity defaults. Resolves the coordination from ticket #4 (toolbelt/app owns the composition-root bootstrap). - **Service-neutral vs app-owned defaults (Q3, Q8, Q12).** Fragments ship service-neutral defaults (NATS `data/nats`, FGA `file://data/fga/openfga.db`, `mode=embedded`, `datastore=sqlite`, the `check:` block). App identity values (SQLite filename, FGA `store_id`) live in the app's composition overrides: `db: DB = DB { path = "data/sqlite/kanban.db" }`. A prototype showed KCL `option()` does not observe the app's composition value, so `option()` is **not** the identity mechanism — explicit composition overrides are. `db.path` has **no default at all**: the app must compose it, and `kcl-generate` fails loudly if it is absent. - **Schema names preserved (Q14).** Fragment schema names stay `NATS`, `DB`, `FGA`, `Log`. The KCL-qualified import (`embeddednats.NATS`) disambiguates; no renaming churn for consumers. - **Collision semantics (Q6).** App wins by simply not importing a fragment and defining its own schema. Fragment-vs-fragment schema name collisions are KCL module-level errors. No merge-precedence code in `kcl-generate`. - **`app/schema.k` ownership and scope (Q4, Q11, Q15).** This ticket ships fragments for `db`, `embeddednats`, `embeddedfga`, and the `app` core fragment. `File`/`Tuple` (FGA tuples) ride `embeddedfga/schema.k`. The `OIDC` schema stays app-authored in this ticket; `authn/schema.k` (declaring the standard `OIDC` schema) is ticket #4's responsibility — a coordination comment was posted on #4. - **Field-path contract (Q10, Q13).** `toolbelt/app` reads platform fields via typed path readers against the app-local registry (the `config.New` object from ticket #3). A prototype showed the alternatives are unbuildable: sharing toolbelt-owned `*Field` pointers across apps mutates shared state at init; path-string lookup alone silently degrades on unknown paths. Therefore: **canonical composition keys** (`nats:`, `db:`, `fga:`, `log:`, `session:`, `rebuild:`) are enforced at codegen, and `toolbelt/app` fails loudly at `New()` when a required platform field is absent. Recorded as the #4↔#12 coordination in `app-bootstrap.md`. - **`embeddednats` Go API (Q7).** `SetupNATS(ctx, logger, storeDir string)` — the store directory becomes a required positional parameter. `DefaultStoreDir` const and the `WithStoreDir` option are removed. `app/init.go` drops the empty-string guard. ttrpg's two `WithStoreDir(t.TempDir())` test calls become positional. - **`kcl.mod` wiring (Q2, Q9).** Fragment discovery needs no flag list: the app declares its fragments via `import` + `kcl.mod`. A Taskfile stamp step derives the toolbelt fragment reference from `go list -m` and rewrites the toolbelt entry in `kcl.mod`, so `kcl.mod` cannot drift from `go.mod` — including for published pseudo-version consumers. - **Toolbelt does not import downstream (Q3).** Fragments may import other toolbelt fragments but never the app's schema; toolbelt never depends on a downstream app. - **No runtime merging (decision 3, revised).** The runtime `.k` loader already evaluates the config program and now resolves schema imports via `kcl.mod` — same resolution path as codegen. What dies is string-merging at codegen and any runtime notion of merging. The generated `var DefaultX` ldflags-overridable pattern survives: defaults are still evaluated at codegen into Go vars. - **The `#@kcl:validate = "ValidateFGAURI"` directive rides the FGA fragment** and references `config.ValidateFGAURI` (resolvable because the app imports `toolbelt/config`). No new mechanism. ## Testing Decisions Good tests here exercise **external behavior**: a schema.k plus fragment files plus a `kcl.mod` in a temp dir, in → generated fields/config out (or a loud error out). They must not reach into `kcl-generate`'s internals beyond what the existing tests already exercise (parse → collect → evaluate → render pipeline). - **Primary seam: `tools/kcl-generate/main_test.go`** (existing prior art: `TestFlattenSchema`, `TestEvaluateDefaults`, `TestRenderConfigStruct`, `TestFieldCollision`, `TestUnknownDirective`). Add tests that drive the full pipeline against a schema.k that imports fragment files: - fragment import resolution produces fields with the fragment's defaults (`nats.store_dir` → `data/nats`); - a canonical-key violation (`jetstream: NATS = NATS {}`) fails codegen; - a missing required `db.path` fails codegen; - app-wins-by-not-importing works (app defines its own `NATS` and `kcl-generate` accepts it); - fragment-vs-fragment schema collision surfaces as an error. - **Secondary seam: `config/kcl/kcl_test.go`** (existing prior art: `TestLoadKConfigWithOptions`, `TestLoadKConfigRelativePath`, `TestValidateConfig`). Add one test proving a config file → schema.k → fragment import chain resolves at runtime load (the authored schema.k is the validated schema). - **Integration: the example's `check-kcl` drift guard** (existing Taskfile task). After the `example/` reshape, `task check-kcl` regenerates and fails on stale output, proving a real app composes fragments end-to-end. ## Out of Scope - **Consumer migration** (go-template, ttrpg) — separate later project, per the config-bootstrap plan. - **`authn/schema.k`** (the `OIDC` fragment) — owned by ticket #4; coordination comment posted on #4. - **`toolbelt/app`'s typed path readers and `New()` fail-loud behavior** — the mechanism is specified here as a contract, but the implementation lands with ticket #4 (composition-root bootstrap) which depends on ticket #3's `config.New` object. - **`config.New` object bootstrap** — ticket #3's decision; this ticket's fragments feed the same field registry shape it already expects. ## Further Notes - Prototypes were run during the grilling session: (a) KCL `option()` in schema defaults does not observe the app's composition value — confirming explicit composition overrides as the identity mechanism; (b) field-path read mechanisms — confirming that shared toolbelt-owned `*Field` pointers are unbuildable (shared mutable state) and that typed path readers with fail-loud are the correct contract. - CONTEXT.md terminology was updated (Config fragment, Service-owned default, App-owned default); the fuller post-resolution glossary update is tracked in ticket #13. - The canonical-composition-key contract is a naming convention KCL cannot enforce; codegen enforcement + runtime fail-loud are the two enforcement points.

ginjiruu added the

ready-for-agent

label 2026-08-06 21:23:47 +00:00

ginjiruu referenced this issue 2026-08-06 21:33:59 +00:00

embeddednats: store directory becomes a required parameter #15

ginjiruu referenced this issue 2026-08-06 21:34:00 +00:00

kcl-generate: resolve fragment imports and enforce the composition contract #16

ginjiruu referenced this issue 2026-08-06 21:34:12 +00:00

Ship toolbelt KCL fragments; reshape the example to compose them; validate at runtime #17

ginjiruu referenced this issue 2026-08-06 21:34:15 +00:00

kcl.mod stays in sync with go.mod via a Taskfile stamp step #18

ginjiruu referenced this issue from a commit 2026-08-06 22:18:14 +00:00

feat(config): per-package KCL config fragments with service-owned defaults

ginjiruu referenced this issue from a commit 2026-08-06 22:18:14 +00:00

docs: record config-bootstrap plans and config-fragment terminology

ginjiruu commented 2026-08-06 22:43:47 +00:00

Author

Owner

Copy link

All sub-tickets landed and closed: #15 (embeddednats positional store dir), #16 (kcl-generate fragment import resolution + composition contract enforcement), #17 (fragments shipped, example composes them, runtime validation), #18 (kcl.mod sync stamp step). Out-of-scope follow-ups tracked separately: #4 (composition-root typed readers / New fail-loud), authn/schema.k, consumer migration.

All sub-tickets landed and closed: #15 (embeddednats positional store dir), #16 (kcl-generate fragment import resolution + composition contract enforcement), #17 (fragments shipped, example composes them, runtime validation), #18 (kcl.mod sync stamp step). Out-of-scope follow-ups tracked separately: #4 (composition-root typed readers / New fail-loud), authn/schema.k, consumer migration.

ginjiruu closed this issue 2026-08-06 22:43:47 +00:00

ginjiruu commented 2026-08-07 01:26:12 +00:00

Author

Owner

Copy link

Lessons learned (post-implementation)

All four sub-tickets (#15–#18) landed and closed; commit f3cbcd0 on master.
Beyond the disposition note above, here is what the implementation surfaced that
the spec did not fully anticipate — worth reading before #4 (composition-root
bootstrap) and the consumer migration.

Deviations from the spec's mechanism

1. kcl-go does not read kcl.mod [dependencies] itself. The spec said
   "the runtime .k loader resolves the same imports via kcl.mod". In
   practice kcl.Run ignores the dependencies table: both the generator's
   evaluateDefaults and the runtime loader must explicitly pass the deps as
   kcl.WithExternalPkgAndPath(name, path) (and validation via
   ValidateCodeArgs.ExternalPkgs). Also, the external module root — the
   toolbelt checkout — must ship its own kcl.mod declaring name = "toolbelt"
   or the resolution fails with "CannotFindModule".

2. addr ownership forced schema inheritance. A root-level scalar field
   cannot be shipped by a fragment unless the app's root schema inherits it.
   The app fragment owns the root Config schema and apps write
   schema Config(app.Config). kcl-generate gained parent-schema flattening
   (module-qualified parent resolution so the child's Config resolves to the
   fragment's, not itself) — this was not called out in the decision list.

3. KCL key/package shadowing. db: db.DB = db.DB {} is a compile error
   (the attribute key db shadows the import db). The db fragment must be
   aliased: import toolbelt.db as tdb. The other fragments compose unaliased
   because their canonical keys (nats, fga, log) differ from their
   package names (embeddednats, embeddedfga, app). The spec's example
   nats: embeddednats.NATS = embeddednats.NATS {} works for exactly this
   reason.

4. Required-field enforcement came for free. path: str and
   store_id: str (no default) are type-checked by KCL itself: an instance
   that omits them errors with "attribute is required and can't be None or
   Undefined". The "fail loudly" user stories ride evaluation rather than extra
   generator logic. Both db.path and fga.store_id are required by design;
   the example composes both explicitly.

Smaller findings

5. The example's kcl.mod uses path = "../" (not "../../") because the
   example lives inside the toolbelt repo. The #18 stamp step derives the path
   from go list -m -f '{{.Dir}}' so it cannot drift; for published consumers
   the same step stamps the module-cache path.

6. Merging fragment files broke #@kcl: directive correlation — line numbers
   collide across files. Fields and directives are now correlated per source
   file.

7. kcl.mod parsing was first duplicated across tools/kcl-generate and
   config/kcl, and the two copies already drifted (one swallowed errors, the
   other didn't). The "separate modules, can't share" justification was wrong —
   tools/go.mod already requires the root module — so it was consolidated
   into the shared kclmod package. Lesson: check module dependency direction
   before duplicating a helper.

8. Group labels were preserved verbatim from the pre-fragment example (e.g. the
   db fragment keeps the odd "Server" group) purely to keep the generated
   fields byte-identical and task check-kcl green. Revisit if a rename is
   ever wanted.

9. Consumer break recorded: ttrpg's two
   SetupNATS(..., WithStoreDir(t.TempDir())) test calls become positional —
   noted in the SetupNATS doc comment. Migration is the separate later
   project.

Forward pointers

• kclmod (git.animeteamspeak.moe/ginjiruu/toolbelt/kclmod) is the single
  home for path-dep resolution; reuse it in any new consumer tooling rather
  than re-parsing kcl.mod.
• Canonical keys live in kcl-generate (canonicalKeys). Adding a new
  canonical fragment schema means touching that map — consider a
  fragment-declared #@kcl:key directive if the set grows.
• Runtime validation now validates against the authored schema.k (File
  variant + ExternalPkgs), so "the schema you author is the schema that is
  validated" holds end to end.

## Lessons learned (post-implementation) All four sub-tickets (#15–#18) landed and closed; commit `f3cbcd0` on master. Beyond the disposition note above, here is what the implementation surfaced that the spec did not fully anticipate — worth reading before #4 (composition-root bootstrap) and the consumer migration. ### Deviations from the spec's mechanism 1. **kcl-go does not read `kcl.mod` `[dependencies]` itself.** The spec said "the runtime `.k` loader resolves the same imports via `kcl.mod`". In practice `kcl.Run` ignores the dependencies table: both the generator's `evaluateDefaults` and the runtime loader must explicitly pass the deps as `kcl.WithExternalPkgAndPath(name, path)` (and validation via `ValidateCodeArgs.ExternalPkgs`). Also, the external module root — the toolbelt checkout — must ship its own `kcl.mod` declaring `name = "toolbelt"` or the resolution fails with "CannotFindModule". 2. **`addr` ownership forced schema inheritance.** A root-level scalar field cannot be shipped by a fragment unless the app's root schema inherits it. The app fragment owns the root `Config` schema and apps write `schema Config(app.Config)`. `kcl-generate` gained parent-schema flattening (module-qualified parent resolution so the child's `Config` resolves to the fragment's, not itself) — this was not called out in the decision list. 3. **KCL key/package shadowing.** `db: db.DB = db.DB {}` is a compile error (the attribute key `db` shadows the import `db`). The db fragment must be aliased: `import toolbelt.db as tdb`. The other fragments compose unaliased because their canonical keys (`nats`, `fga`, `log`) differ from their package names (`embeddednats`, `embeddedfga`, `app`). The spec's example `nats: embeddednats.NATS = embeddednats.NATS {}` works for exactly this reason. 4. **Required-field enforcement came for free.** `path: str` and `store_id: str` (no default) are type-checked by KCL itself: an instance that omits them errors with "attribute is required and can't be None or Undefined". The "fail loudly" user stories ride evaluation rather than extra generator logic. Both `db.path` and `fga.store_id` are required by design; the example composes both explicitly. ### Smaller findings 5. The example's `kcl.mod` uses `path = "../"` (not `"../../"`) because the example lives inside the toolbelt repo. The #18 stamp step derives the path from `go list -m -f '{{.Dir}}'` so it cannot drift; for published consumers the same step stamps the module-cache path. 6. Merging fragment files broke `#@kcl:` directive correlation — line numbers collide across files. Fields and directives are now correlated per source file. 7. kcl.mod parsing was first duplicated across `tools/kcl-generate` and `config/kcl`, and the two copies already drifted (one swallowed errors, the other didn't). The "separate modules, can't share" justification was wrong — `tools/go.mod` already requires the root module — so it was consolidated into the shared `kclmod` package. Lesson: check module dependency direction before duplicating a helper. 8. Group labels were preserved verbatim from the pre-fragment example (e.g. the `db` fragment keeps the odd `"Server"` group) purely to keep the generated fields byte-identical and `task check-kcl` green. Revisit if a rename is ever wanted. 9. Consumer break recorded: ttrpg's two `SetupNATS(..., WithStoreDir(t.TempDir()))` test calls become positional — noted in the `SetupNATS` doc comment. Migration is the separate later project. ### Forward pointers - `kclmod` (`git.animeteamspeak.moe/ginjiruu/toolbelt/kclmod`) is the single home for path-dep resolution; reuse it in any new consumer tooling rather than re-parsing `kcl.mod`. - Canonical keys live in `kcl-generate` (`canonicalKeys`). Adding a new canonical fragment schema means touching that map — consider a fragment-declared `#@kcl:key` directive if the set grows. - Runtime validation now validates against the authored `schema.k` (File variant + `ExternalPkgs`), so "the schema you author is the schema that is validated" holds end to end.

ginjiruu removed the

ready-for-agent

label 2026-08-07 01:26:13 +00:00

ginjiruu referenced this issue 2026-08-07 01:29:52 +00:00

Per-package KCL config fragments — service-owned defaults (db, embeddednats, embeddedfga) #12

ginjiruu referenced this issue 2026-08-07 01:34:59 +00:00

Toolbelt glue absorption — map: what the toolbelt absorbs from duplicated consumer glue #1

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

master

No results found.

Labels

Clear labels   

needs-info

  

needs-triage

  

ready-for-agent

  

ready-for-human

  

wayfinder:grilling

  

wayfinder:map

  

wayfinder:prototype

  

wayfinder:research

  

wayfinder:task

  

wontfix

No labels

needs-info

needs-triage

ready-for-agent

ready-for-human

wayfinder:grilling

wayfinder:map

wayfinder:prototype

wayfinder:research

wayfinder:task

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

ginjiruu/toolbelt#14

No description provided.