# Changelog
## Unreleased
### Added
- Union underlying types: `union U : int32 { ... }` (any integral
scalar or a previously declared enum). Discriminators are read and
written at the underlying type's full width, matching what `flatc`'s
generated C++/TS code does on the wire. Note that `flatc`'s own JSON
tooling does not implement the feature (it refuses such schemas for
`--json` and mis-writes 1-byte discriminators for `--binary`), so
text-level interop with `flatc` is not possible for these schemas.
- Property-based round-trip tests (StreamData): generators derived
from the schema IR produce arbitrary valid values and round-trip
them through encode/decode/verify and through `flatc`.
- Generated root tables expose `file_extension/0` when the schema
declares `file_extension`, parallel to `file_identifier/0`.
- `verify/2` and `verify_size_prefixed/2` accept `max_depth:`
(default 64) to bound recursion into nested tables.
- Encode-direction differential suite: buffers produced by the
generated encoders are now verified against `flatc` across the
feature matrix (scalars, strings, vectors, structs, fixed arrays,
enums/bit_flags, unions incl. vectors of unions, nesting,
file identifiers, size prefixes, key sorting, shared strings,
force_align). Previously only the decode direction was covered.
- Schema validation in the resolver, mirroring `flatc` semantics:
duplicate field/variant names, explicit `(id: N)` rules
(all-or-none, consecutive from 0, unions take two ids), default
value typing and range checks, enum value ranges (incl. `bit_flags`
shifts), `required` rejected on scalar fields, `force_align`
power-of-two/bounds checks, fixed arrays restricted to structs,
union variant cap (255), empty structs and struct defaults
rejected, nested vectors rejected. Errors carry the offending name
and source line.
- Lexer: `\b`, `\f`, `\/`, `\uXXXX` (with surrogate pairs) and
`\xHH` string escapes; `.5`/`1.` float literals; malformed or
non-UTF-8 input returns tagged errors instead of raising.
- `root_type` now resolves with the same lookup rules `flatc` uses
(name-as-written first, then current-namespace-qualified).
- GitHub Actions CI: lint, offline test, full-corpus test (fixtures
and `flatc` cached), and dialyzer jobs.
- `Mix.Tasks.Compile.Flatbuf` — a Mix compiler that regenerates `.ex`
files from configured `.fbs` schemas on every `mix compile`. Wire
it in with `compilers: [:flatbuf | Mix.compilers()]` and configure
schemas under `config :my_app, :flatbuf`. A per-build manifest
gates writes (unchanged outputs aren't rewritten) and drops stale
outputs when schemas are removed from config.
- `mix flatbuf.gen.check` — CI gate that exits non-zero if running
the gen pipeline would change any committed file.
- `Flatbuf.Table` behaviour and `--niceties` opt-in for generated
root tables. `--niceties behaviour` attaches the behaviour;
`--niceties jason` derives `Jason.Encoder` on the struct (the
consumer must depend on `:jason`). Default remains dependency-free.
- File identifier emission. When a schema declares
`file_identifier "XXXX";`, the root encode helpers write the
4-byte marker into the buffer header, and `file_identifier/0`
surfaces the value on the generated module.
- Size-prefixed buffer support. Every root table now exposes
`encode_size_prefixed/1`, `decode_size_prefixed/1`, and
`verify_size_prefixed/1`.
- Required-field enforcement. Fields marked `(required)` in the
schema cause `encode/1` to fail with
`{:error, {:flatbuf_required, :name}}` when missing, and `verify/1`
to fail with `{:error, {:missing_required, :name}}` when the
buffer's vtable doesn't list the slot.
### Changed
- **Breaking (generated code):** verifier errors are now three-element
tuples `{:error, reason, path}` where `path` is a root-first list of
field atoms, vector indices, and union variant atoms locating the
failure (e.g. `[:inventory, 3, :name]`); buffer-level failures carry
`[]`. Reasons keep their tagged shapes; field identity that the path
now carries was dropped from `:union_vector_*` reasons. Regenerate
committed output and update any `{:error, _}` matches on `verify`
results.
- Deprecated fields are skipped on encode (their vtable slots stay
reserved), matching `flatc`'s generated builders — previously they
were written. Decode still surfaces a deprecated field's value when
a buffer contains it; `to_json/1` omits deprecated fields.
- Codegen threads the namespace override explicitly instead of via
the process dictionary, and the wire template uses a validating
multi-hole fill (internal; emitted sources are byte-identical).
- `Flatbuf.Codegen.generate/2` accepts a `:niceties` option.
- The generated `encode/1` now catches the required-field throw and
returns it as a tagged error tuple instead of crashing the caller.
- Generated sources are piped through `Code.format_string!/1` before
being returned/written, so emitted files pass
`mix format --check-formatted` as-is. Regenerating previously
committed output will produce a one-time formatting diff.
- The generated `decode/1` no longer catches every exception. It
rescues only buffer-shaped failures (`MatchError`, `ArgumentError`,
`FunctionClauseError`) and returns
`{:error, {:malformed_buffer, exception}}`; other exceptions
propagate. Use `verify/1` first on untrusted input.
### Fixed
- The `alignment_test` upstream fixture round-trips: the harness had
been pairing the binary with a stale orphaned JSON from an older
schema revision. The two remaining pinned fixtures are annotated
with one-line reasons (`evolution_v1`: `flatc` segfaults generating
text for its own buffer; `test_64bit`: 64-bit offsets unsupported
and `flatc` has no JSON oracle for them), and recorded errors no
longer embed machine-specific paths.
- Scalar values are validated on the encode path. Out-of-range or
wrong-typed values used to be silently truncated into the wire
(`encode(%{a: 70_000})` on a `ushort` produced `4464`); they now
return `{:error, {:scalar_out_of_range | :invalid_scalar, field,
kind, value}}`. Covers table fields, vectors, struct members,
fixed arrays, enum underlying values, and hash fields.
- Optional scalar/enum fields (`= null`) no longer crash `encode/1`
when absent, and explicitly-passed type-default values (`0`,
`false`, first enum variant) are now written instead of dropped —
absence and explicit defaults are distinguishable on the wire.
- The verifier bounds-checks inline fields (scalars, enums, inline
structs, union discriminators) against the table's inline area. A
crafted vtable could previously direct reads past the buffer while
`verify/1` returned `:ok`.
- Vector-of-union verification checks both parallel vectors,
requires equal element counts
(`{:error, {:union_vector_count_mismatch, ...}}`), and no longer
raises on count-inflated buffers.
- `nil` (NONE) elements in vectors of unions encode as
discriminator 0 / offset 0 (the layout `flatc`'s binary layer
accepts) instead of raising `ArithmeticError`; decode yields `nil`.
- A table whose only depth-recursing field is a vector of unions
generated a `__verify_at__/3` that failed to compile.
- `:nstandard` was an unconditional runtime dependency; it is now
dev/test-only, so consumers pull no transitive deps from
`:flatbuf`.
- Unknown `--niceties` values (e.g. `behavior`) raised silently no
effect; they are now rejected with the valid values listed.
- `Mix.Tasks.Compile.Flatbuf` tolerates a corrupted manifest,
reports a missing `:schemas` config key with a friendly message,
and no longer rewrites byte-identical artifacts (and their mtimes)
when an unrelated schema changes.
- A fresh clone's `mix test` is green: corpus-gated smoke tests skip
(instead of failing) with a notice pointing at
`mix flatbuf.fetch_fixtures` / `mix flatbuf.fetch_flatc`.
- Codegen crashed with a `CaseClauseError` on enum types declared
with no variants (e.g. `enum Foo : int {}`).
## v0.1.0
- Initial extraction from `sprawl/arrow`: parser, resolver, codegen for
tables/structs/enums/unions, JSON converter, verifier, and the
`mix flatbuf.gen` task with namespace/include overrides.