4. Format families and their lossless readings
Contents of FBL 0.6
For each family, this section defines a lossless reading: a tree of nodes over the body's bytes in which every byte belongs to exactly one node or to the trivia (whitespace, comments, line endings) that one node owns. Rules (section 5) select nodes; splices (section 6) replace the spans of nodes. A host MAY parse a body however it likes, but the spans it uses MUST be the ones defined here.
An entry is a node that a rule can match: a mapping member or sequence item (yaml, json), an element (xml), a statement (lines, blocks). Every entry has:
- its own span: the bytes of the entry itself, from its first to its last significant byte;
- its line span, when the entry starts its line and ends its line (nothing but trivia before it on its first line and after it on its last): from the first byte of its first line to the end of the line ending of its last line, extended upwards over its leading comments, the comment lines directly above it at the same indentation with no blank line in between. A trailing comment on the entry's last line is part of its line span.
A value is a node that a slot binds: a scalar, a flow collection, an attribute value, the text of an element, a named group. Its span excludes quotes and delimiters for the purpose of reading, and includes them for the purpose of replace-value (section 6.1).
Blank lines belong to no entry: they are trivia of the enclosing container and are never removed or added by a splice unless this document says so.
Everything no rule binds is unbound: unknown keys, attributes, elements and statements, comments, directives. Unbound content is read past, kept byte for byte, and moves only when a splice removes an entry that contains it (an unknown key inside a removed entry goes with the entry).
Entries have document order, the order of their first bytes. Reading order is document order (DISL relies on it for positionIn); for a folder subject, files are read in the ordinal order of their relative paths, and document order within each.
An entry is presented to CEL (section 2.4) as follows. A yaml or json mapping is a map from key to value; a sequence is a list; a scalar is a string, int, double, bool or null by the YAML 1.2 core schema (JSON by its own types). An xml element is a map from attribute name to string, plus the key text holding its text content. A lines or blocks statement is presented as its groups.
A key that is present with no value is present in the map with the value null, so has(entry.end) is true for end: and false where the key is absent.
Every scalar of a yaml or json body has, beside the typed value of section 4.1.4, a written text: what its author wrote.
| Scalar | Written text |
|---|---|
| yaml plain scalar | Its characters, without leading and trailing spaces and without a comment; a plain scalar over several lines is folded as YAML folds it. |
| yaml quoted scalar (single or double) | Its content, without the quotes, with its escapes resolved. |
yaml block scalar (|, >) |
Its content as YAML defines it for its indicators. |
yaml key with no value (end:) |
The empty text. |
| json string | Its content, with its escapes resolved. |
json number, true, false, null |
Its literal characters. |
A tag does not change the written text. A scalar reached through an alias or a merge key has the written text of the scalar it stands for. The members of a flow collection have a written text each, by the same rules. The xml, lines and blocks families read text and nothing else, so there the value a slot reads is already what was written.
So yes, "yes" and 'yes' have the written text yes; 007 has 007; 1.0 has 1.0; 0x10 has 0x10. The typed value of each is what the YAML 1.2 core schema gives: a bool, a string, a string, the int 7, the double 1.0, the int 16.
Tree rules (yaml, json) and xml rules select entries with a selector: segments separated by /.
| Segment | Matches |
|---|---|
| a key or name | the member with that key (yaml, json) or the child element with that name (xml) |
* |
any member of a mapping or any item of a sequence (yaml, json), any child element (xml) |
** |
any number of levels, including none |
{name} |
any member of a mapping, whose key is bound to path.name |
name[@A='v'] |
xml: a child element with that name whose attribute A has the value v |
A selector that starts with / is absolute, from the document's root (the root mapping or sequence; the document element's parent in xml, so /map is the root element map). A selector without it is relative to the entry the rule is written for (in insert.container, child), or to the root elsewhere. / alone selects the root. A key containing /, *, { or } cannot be selected in FBL 0.1.
The body is a YAML 1.2 stream. Only the first document is bound; any further document is unbound. The lossless reading follows the YAML 1.2 syntax, with these definitions:
- A block mapping entry is a key and its value; its own span runs from the first byte of the key to the last byte of the value (for a block collection, its last entry's last byte). A block sequence entry is
-and its value; its own span runs from the-. - The indentation of an entry is the column of its first byte: the key, or the
-. For a mapping that is a sequence item (- id: a), the mapping's entries are indented at the column of their keys (idat 4 in- id: a). - A scalar value's span is the scalar as written: with its quotes for quoted scalars; for a plain scalar, without trailing spaces or a comment; for a block scalar (
|,>with indicators), from the indicator to the end of its last content line, without that line's ending. - A flow collection (
[…],{…}) is one value. Its members are readable (CEL sees them), but its only writable span is the whole collection, written byreplace-valuein the style the attribute binding asks for (flowwrites a one-line flow sequence). - Anchors, aliases, tags and merge keys are read (an alias reads as the anchored value; a tag is ignored for typing when it is a core-schema tag and kept otherwise;
<<merges as YAML 1.1 defines it, for reading only). A slot reached through an alias or a merge is read-only. - A YAML syntax error makes the body unreadable (section 7.5). A document whose structure is well-formed YAML but not what a rule expects (a sequence item that is not a mapping, a scalar where a mapping is needed) yields unreadable entries (section 7.4), never an unreadable body.
The body is a JSON text (RFC 8259). Comments are not allowed: a body with a comment is unreadable. The nodes are objects, arrays, members, items and values, with these spans:
- A member's own span runs from the opening quote of its name to the last byte of its value; an item's from the first to the last byte of its value.
- A member or item starts its line when only whitespace precedes it on its line. The separator of an entry is the comma between it and the next entry of the same container.
- A string value's span includes its quotes.
A duplicate member name is read as a finding on the second and later occurrences (std.duplicateId is DISL's for ids; a duplicate key is fbl.duplicate-key), and only the first is bound.
The body is an XML 1.0 document. The lossless reading keeps the prolog, comments, processing instructions and a document type declaration as unbound content; entity declarations are not processed, and a reference to an entity other than the five predefined ones and character references makes the containing element an unreadable entry. Nodes are elements, attributes and text:
- An element's own span runs from
<of its start tag to>of its end tag, or of its tag when it is self-closed (<x/>). - An attribute's value span is the text between its quotes; its own span runs from its name to its closing quote, and attribute order is significant (kept, and used for new attributes, section 6.3).
- An element's text is its character data with references decoded; a
textslot's span is the character data between the start tag and the first child element or the end tag. - Carriage returns are part of the content, not normalised: a reader that uses an XML parser MUST make it keep CR, since XML 1.0 normalises line endings and the body's bytes are the model.
- Escaping when writing: in text,
&,<and>are escaped as&,<and>; in attribute values also"as", LF as
and CR as
. Other characters are written literally in UTF-8. - A
content: "html-paragraphs"text slot reads the element's HTML body as plain text, one line per<p>element, whitespace inside a paragraph collapsed to single spaces; writing it emits one<p>line</p>per line with the escaping above.
The body is a sequence of lines. Each line is exactly one of:
- blank: only whitespace;
- a comment: it matches the binding's
comment; it is owned by the statement directly below it, if no blank line intervenes (a leading comment, section 4.1.1); - a statement: any other line. A statement matched by a rule's
lineis an entry of that rule; the first rule in the binding's order whoselinematches (and whosewhenholds) wins. A statement no rule matches is unbound (and reported whenunmatchedisreport).
A statement's own span is the line without its ending; its line span adds its leading comments and its line ending. Its values are its named groups. A group used with word is split into words: runs of non-whitespace, where a run starting with " extends to the next " and includes both quotes. A word slot's value is the first word of the group that matches the slot's word expression; if that expression has a named group value, the value is that group, otherwise the whole word (without quotes when the word is quoted). A flag word slot's value is true when such a word is present.
The body is lines as in the lines family, with nesting: a statement whose last non-whitespace character, outside double-quoted strings, is { opens a block, which extends to the line whose only non-whitespace content is the matching }. A statement's entry then spans from its line to the end of the closing line. Blocks nest; braces inside double-quoted strings do not count. A body whose braces do not balance is unreadable.
blocks (BlockRule) name the statements that open a block and produce no element (model {, views {), so that rules can state where they apply:
| Property | Type | Description |
|---|---|---|
name |
Name | Required. |
line |
Regex | Required. The statement. |
within |
Name[] | The block rules or element rules whose block the statement must be directly inside; ^ is the top level. Absent: anywhere. |
caseInsensitive |
bool | Match ignoring the case of ASCII letters. |
view |
group name | The block defines a view of the body, named by this group's value. A registration's view header selects it (section 9.3). |
An element rule with opens: true may open a block (it may also appear on one line without one); within on element and relation rules works as for block rules. A statement is matched only by rules whose within admits its enclosing block.