Skip to content
Source on GitHubVisitors

4. Format families and their lossless readings

Contents of FBL 0.15

4.1 The common model

For each family, this section defines a lossless reading: a tree of nodes over the body's bytes in which every byte belongs to exactly one node or to the trivia (whitespace, comments, line endings) that one node owns. Rules (section 5) select nodes; splices (section 6) replace the spans of nodes. A host MAY parse a body however it likes, but the spans it uses MUST be the ones defined here.

4.1.1 Entries and spans

An entry is a node that a rule can match: a mapping member or sequence item (yaml, json), an element (xml), a statement (lines, blocks, bullets), a list item (bullets), (FBL 0.9) a node of the parse tree (grammar). Every entry has:

  • its own span: the bytes of the entry itself, from its first to its last significant byte;
  • its line span, when the entry starts its line and ends its line (nothing but trivia before it on its first line and after it on its last): from the first byte of its first line to the end of the line ending of its last line, extended upwards over its leading comments, the comment lines directly above it at the same indentation with no blank line in between. A trailing comment on the entry's last line is part of its line span.

A value is a node that a slot binds: a scalar, a flow collection, an attribute value, the text of an element, a named group, (FBL 0.9) a node or token of the parse tree, or the part of its text that match takes (grammar). Its span excludes quotes and delimiters for the purpose of reading, and includes them for the purpose of replace-value (section 6.1).

Blank lines belong to no entry: they are trivia of the enclosing container and are never removed or added by a splice unless this document says so.

4.1.2 Unbound content

Everything no rule binds is unbound: unknown keys, attributes, elements and statements, comments, directives. Unbound content is read past, kept byte for byte, and moves only when a splice removes an entry that contains it (an unknown key inside a removed entry goes with the entry).

4.1.3 Document order

Entries have document order, the order of their first bytes. Reading order is document order (DISL relies on it for positionIn); for a folder subject, files are read in the ordinal order of their relative paths, and document order within each.

4.1.4 Entries as CEL values

An entry is presented to CEL (section 2.4) as follows. A yaml or json mapping is a map from key to value; a sequence is a list; a scalar is a string, int, double, bool or null by the YAML 1.2 core schema (JSON by its own types). An xml element is a map from attribute name to string, plus the key text holding its text content. A lines, blocks or bullets statement, and a bullets item, is presented as its groups. (FBL 0.9) A node of a grammar body is a map with type, its name, text, its text, and children, the list of its children as such maps, an optional part that matched nothing being null; a token is such a map whose children is empty.

A key that is present with no value is present in the map with the value null, so has(entry.end) is true for end: and false where the key is absent.

4.1.5 Written text (FBL 0.5)

Every scalar of a yaml or json body has, beside the typed value of section 4.1.4, a written text: what its author wrote.

Scalar Written text
yaml plain scalar Its characters, without leading and trailing spaces and without a comment; a plain scalar over several lines is folded as YAML folds it.
yaml quoted scalar (single or double) Its content, without the quotes, with its escapes resolved.
yaml block scalar (|, >) Its content as YAML defines it for its indicators.
yaml key with no value (end:) The empty text.
json string Its content, with its escapes resolved.
json number, true, false, null Its literal characters.

A tag does not change the written text. A scalar reached through an alias or a merge key has the written text of the scalar it stands for. The members of a flow collection have a written text each, by the same rules. The xml, lines, blocks, bullets and (FBL 0.9) grammar families read text and nothing else, so there the value a slot reads is already what was written.

So yes, "yes" and 'yes' have the written text yes; 007 has 007; 1.0 has 1.0; 0x10 has 0x10. The typed value of each is what the YAML 1.2 core schema gives: a bool, a string, a string, the int 7, the double 1.0, the int 16.

4.2 Selectors

Tree rules (yaml, json), xml rules and (FBL 0.9) grammar rules select entries with a selector: segments separated by /. Section 4.9 says what a segment names in the grammar family.

Segment Matches
a key or name the member with that key (yaml, json) or the child element with that name (xml)
* any member of a mapping or any item of a sequence (yaml, json), any child element (xml)
** any number of levels, including none
{name} any member of a mapping, whose key is bound to path.name
name[@A='v'] xml: a child element with that name whose attribute A has the value v

A selector that starts with / is absolute, from the document's root (the root mapping or sequence; the document element's parent in xml, so /map is the root element map). A selector without it is relative to the entry the rule is written for (in insert.container, child), or to the root elsewhere. / alone selects the root. A key containing /, *, { or } cannot be selected in FBL 0.1.

4.3 The yaml family

The body is a YAML 1.2 stream. Only the first document is bound; any further document is unbound. The lossless reading follows the YAML 1.2 syntax, with these definitions:

  • A block mapping entry is a key and its value; its own span runs from the first byte of the key to the last byte of the value (for a block collection, its last entry's last byte). A block sequence entry is - and its value; its own span runs from the -.
  • The indentation of an entry is the column of its first byte: the key, or the -. For a mapping that is a sequence item (- id: a), the mapping's entries are indented at the column of their keys (id at 4 in - id: a).
  • A scalar value's span is the scalar as written: with its quotes for quoted scalars; for a plain scalar, without trailing spaces or a comment; for a block scalar (|, > with indicators), from the indicator to the end of its last content line, without that line's ending.
  • A flow collection ([…], {…}) is one value. Its members are readable (CEL sees them), but its only writable span is the whole collection, written by replace-value in the style the attribute binding asks for (flow writes a one-line flow sequence). (FBL 0.10) A flow sequence is one-line when its [ and its ] stand on one line. The items of a flow sequence are entries for the rules' selectors (section 4.2), each with the span of a json item (section 4.4): its own span runs from its first to its last byte, and its separator is the , between it and the next item. An insert and a removal of such an entry are planned as section 6.2 says for flow sequences: an empty flow sequence is opened into a block sequence, a one-line one takes a new item inside it, and a flow sequence written over several lines, like a flow mapping, takes no insert.
  • Anchors, aliases, tags and merge keys are read (an alias reads as the anchored value; a tag is ignored for typing when it is a core-schema tag and kept otherwise; << merges as YAML 1.1 defines it, for reading only). A slot reached through an alias or a merge is read-only.
  • A YAML syntax error makes the body unreadable (section 7.5). A document whose structure is well-formed YAML but not what a rule expects (a sequence item that is not a mapping, a scalar where a mapping is needed) yields unreadable entries (section 7.4), never an unreadable body.

4.4 The json family

The body is a JSON text (RFC 8259). Comments are not allowed: a body with a comment is unreadable. The nodes are objects, arrays, members, items and values, with these spans:

  • A member's own span runs from the opening quote of its name to the last byte of its value; an item's from the first to the last byte of its value.
  • A member or item starts its line when only whitespace precedes it on its line. The separator of an entry is the comma between it and the next entry of the same container.
  • A string value's span includes its quotes.

A duplicate member name is read as a finding on the second and later occurrences (std.duplicateId is DISL's for ids; a duplicate key is fbl.duplicate-key), and only the first is bound.

4.5 The xml family

The body is an XML 1.0 document. The lossless reading keeps the prolog, comments, processing instructions and a document type declaration as unbound content; entity declarations are not processed, and a reference to an entity other than the five predefined ones and character references makes the containing element an unreadable entry. Nodes are elements, attributes and text:

  • An element's own span runs from < of its start tag to > of its end tag, or of its tag when it is self-closed (<x/>).
  • An attribute's value span is the text between its quotes; its own span runs from its name to its closing quote, and attribute order is significant (kept, and used for new attributes, section 6.3).
  • An element's text is its character data with references decoded; a text slot's span is the character data between the start tag and the first child element or the end tag.
  • Carriage returns are part of the content, not normalised: a reader that uses an XML parser MUST make it keep CR, since XML 1.0 normalises line endings and the body's bytes are the model.
  • Escaping when writing: in text, &, < and > are escaped as &amp;, &lt; and &gt;; in attribute values also " as &quot;, LF as &#xa; and CR as &#xd;. Other characters are written literally in UTF-8.
  • A content: "html-paragraphs" text slot reads the element's HTML body as plain text, one line per <p> element, whitespace inside a paragraph collapsed to single spaces; writing it emits one <p>line</p> per line with the escaping above.

4.6 The lines family

The body is a sequence of lines. Each line is exactly one of:

  • blank: only whitespace;
  • a comment: it matches the binding's comment; it is owned by the statement directly below it, if no blank line intervenes (a leading comment, section 4.1.1);
  • a statement: any other line. A statement matched by a rule's line is an entry of that rule; the first rule in the binding's order whose line matches (and whose when holds) wins. A statement no rule matches is unbound (and reported when unmatched is report).

A statement's own span is the line without its ending; its line span adds its leading comments and its line ending. Its values are its named groups. A group used with word is split into words: runs of non-whitespace, where a run starting with " extends to the next " and includes both quotes. A word slot's value is the first word of the group that matches the slot's word expression; if that expression has a named group value, the value is that group, otherwise the whole word (without quotes when the word is quoted). A flag word slot's value is true when such a word is present. (FBL 0.7) A word slot of an attribute with many holds every word of the group that matches its word expression, in order, each an item: the word's group value, or the whole word.

4.7 The blocks family

The body is lines as in the lines family, with nesting: a statement whose last non-whitespace character, outside double-quoted strings, is { opens a block, which extends to the line whose only non-whitespace content is the matching }. A statement's entry then spans from its line to the end of the closing line. Blocks nest; braces inside double-quoted strings do not count. A body whose braces do not balance is unreadable.

blocks (BlockRule) name the statements that open a block and produce no element (model {, views {), so that rules can state where they apply:

Property Type Description
name Name Required.
line Regex Required. The statement.
within Name[] The block rules or element rules whose block the statement must be directly inside; ^ is the top level. Absent: anywhere.
caseInsensitive bool Match ignoring the case of ASCII letters.
view group name The block defines a view of the body, named by this group's value. A registration's view header selects it (section 9.3). (FBL 0.11) In the grammar family view is a slot, and a block rule has at, origins and insert (section 4.9).
origins origin[] (FBL 0.11) The readings the block is a view for (section 5.1).

An element rule with opens: true may open a block (it may also appear on one line without one); within on element and relation rules works as for block rules. A statement is matched only by rules whose within admits its enclosing block.

4.8 The bullets family (FBL 0.7)

The body is lines, as in the lines family, holding list items that nest by their indentation: the part of a Markdown file that a tool reads as a tree. FBL reads the list items under the headings a binding names, and nothing else of Markdown (section 1.5).

Columns. A line's indentation is the spaces and tabs it starts with, counted in columns from 0, a tab counting four columns wherever it stands (not up to the next multiple of four).

Line kinds. Each line is one of the following, decided in document order. An item is open from its item line until a line closes it.

  1. The lines after the one that opens a fence, up to and including the one that closes it, are fence lines; the opening line itself is decided by the rules below. A fence opens on a line whose text after its indentation starts with three or more backticks or three or more tildes, and closes on the next line whose text after its indentation is a run of at least as many of the same character and nothing else but whitespace, or at the end of the body. Nothing inside a fence is any other kind of line.
  2. A blank line holds only whitespace. It closes nothing.
  3. Any other line first closes every open item whose marker stands at the line's indentation or deeper.
  4. An item line is indentation, a marker -, * or +, then whitespace or the end of the line, and is not a thematic break (below). Its item is nested in the deepest item still open, if any, and is then open itself. The item's text is what follows the marker and the whitespace after it, to the end of the line, trailing whitespace included; its content column is the column where the text begins, or the marker's column plus two when the text is empty.
  5. While an item is still open, any other line is a continuation line of the deepest open item. A fence that opens on a continuation line belongs to that item: its lines are continuation lines of it, whatever their indentation.
  6. With no item open, a line is a heading when it has at most three columns of indentation, then one to six #, then whitespace or the end of the line. Its level is the number of #, and its text the rest of the line without leading and trailing whitespace and without a closing run of # that whitespace precedes. A line underlined with = or - is not a heading in this family.
  7. With no item open, a line is a thematic break when it holds three or more of one of -, * and _, and nothing else but spaces and tabs. It is never an item line, open or not.
  8. With no item open, a line that opens a fence makes the fence unbound, with every line in it.
  9. Any other line is a statement, as in the lines family.

Sections. A binding's sections name headings, as blocks name the statements that open a block:

Property Type Description
name Name Required. Referenced by a rule's within and by insert.container.
heading Regex Required. Matched against a heading's text.
caseInsensitive bool Match ignoring the case of ASCII letters.

A section is the lines after the first heading whose text matches heading, up to the next heading with as many # or fewer, or to the end of the body. A heading with more # inside it does not end it. A later heading that matches too is an ordinary heading. An item stands in a section when its item line is one of the section's lines.

Entries and spans. An item and a statement are entries. An item's own span runs from its marker to the end of its last continuation line or of its last nested item, whichever comes later, without that line's ending; its line span runs from the first byte of its item line to the end of that last line's ending. Blank lines inside an item's own span are part of it; those after it belong to no entry (section 4.1.1). A statement's spans are those of section 4.6. This family has no comments.

Rules. A rule with within matches items of the sections it names: its line is matched against the item's text. A rule without within matches statements, as in the lines family. Of the rules that may match an entry, the first in the binding's order whose line matches and whose when holds takes it (section 5.1). The named groups of line are the entry's values, split into words as section 4.6 says.

Continuation. An item's continuation is its continuation lines from the first to the last before its first nested item, the blank lines between them included; the continuation lines after a nested item are part of the item's span and are unbound. The slot {continuation: true} reads the continuation as text: its lines joined by LF, each with the whitespace before the item's content column taken off (a line indented less loses all of its indentation, and a tab the content column cuts into is replaced by the spaces that remain of it), a blank line read as an empty line. An item without continuation lines reads as the empty text.

Unbound. Headings, thematic breaks, blank lines and unbound fences are unbound. An item that stands in no section a rule's within names is unbound. An item that no rule matches, and an unreadable item (section 7.4), which includes an item whose rule's type the tool type's specification maps as unreadable (DESL section 5.2), are not looked into: they are unbound with everything in their span, and no rule matches an item nested in them. A statement that no rule matches is unbound, and reported only when unmatched is report.

The body is never unreadable for its family: Markdown has no malformed list (section 7.5).

4.9 The grammar family (FBL 0.9)

The body is text that a context-free grammar the binding carries reads: a format with a syntax of its own that no other family reads, such as Turtle, bound declaratively instead of read by a persistence plugin (section 11). The family's tree is the one the grammar gives, so its rules address the format's own constructs by their names.

The grammar. body.grammar is a grammar in ATD grammar notation, the subset of Lark's grammar language that ATD section 4 states (its rules are specifications/atd/grammar/notation.lark): either the grammar's text as a string, or {"file": "<name>.lark"}, a file beside the FBL document. A grammar MAY %import rules and terminals from other files beside the document (%import .turtle_terms (IRIREF) reads turtle_terms.lark) and from nowhere else. body.start names the start rule, start when it is absent. The grammar is built as ATD section 4.4 says, its terminals are written in the regular expression subset of ATD section 3.5, and its text is cut into tokens by the contextual lexing rule of ATD section 3.6: at each position the candidates are the terminals the parser's state can shift and the ignored ones; the longest match wins, then the higher terminal priority, then a string terminal over a pattern terminal. The grammar MUST build as an LALR(1) automaton with no shift/reduce and no reduce/reduce conflict, in which no two terminals a state accepts match the same text; a binding whose grammar does not is refused when it is loaded, with a problem that names the conflict (section 14.1). ATD section 4.7 normalizes the automaton, so that two implementations can show that they build the same one.

The tree. Reading a body gives a tree shaped as ATD section 4.5 says: a node per rule, named by the rule or by the alias of the alternative that matched; a rule written ?rule replaced by its child when it has one, and _rule by its children; a token of a named terminal a leaf named by its terminal, except a _TERMINAL; anonymous strings left out; and an optional part that matched nothing kept as an empty child. The root is the node of the start rule.

  • Every node and token has an own span. A token's is its bytes. A node's runs from the first byte of the first token its rule matched to the last byte of the last token its rule matched, the tokens the tree leaves out included, so that the . that ends a Turtle statement is in the statement's span though not in its tree. A node that matched no token has no span. The root's span is the whole body.

  • A node's or token's text is the bytes of its own span.

  • The tokens the grammar ignores (%ignore) are trivia. An ignored token that holds a character other than whitespace is a comment. (FBL 0.11) A comment is owned as follows, so that every host keeps the same comment with the same entry:

    1. A comment that is the first token on its line, with every comment on the lines directly above it at no blank line's distance, is a leading comment of the entry whose first token is the next token after them (section 4.1.1).
    2. A comment that follows a token on its line is a trailing comment of the innermost entry that holds that token: model { // the model is the comment of the model entry, because { is a token of that entry and of none of its children, and container "Web" // c is the container's. A trailing comment is part of that entry's line span and of no value's span.
    3. A comment that opens on one line and closes on a later one is owned by rule 1 when it starts its line and by rule 2 when it follows a token, and its owner's line span then runs to the end of the comment's last line. Nothing inside it is a token, so it holds no statement, element or brace; a statement's content before or after it on its line is read as if it were not there; removing or moving its owner takes it with the owner's line span. A comment that is never closed is trivia to the end of the body, reported under recover.unclosed (Reading in part) rather than making the body unreadable.
    4. A comment inside a value's span ("a" /* x */ "b" read by one slot) is part of the value's text, and a set of the value replaces it with the rest of the span; a binding that reads such a value SHOULD take one token with match.
    5. Every splice keeps a comment it does not remove: a replace-value changes the value's span only, a part written where it was absent is written before a trailing comment (Writing), and a removed entry takes its leading and trailing comments with its line span.

    A grammar states where a comment can start by its terminals alone, under the lexing rule of ATD section 3.6: where a longer token of another terminal starts before a comment's opening, such as a URL token that holds //, the comment terminal is never a candidate there, and the comment's characters are the token's.

  • (FBL 0.11) Blocks are what the grammar reads as blocks, and nothing else: a statement whose text ends in { and that the grammar reads as one token (a title, a note's text) opens nothing and keeps the { in its text, and a { or } inside a token is never the open or close of recover.nested, so recovery never treats it as a block.

  • (FBL 0.12) A grammar in which a block's opening brace may stand on the line after its statement reads both forms in one body. wardley-map.lark is the example: the token that opens a pipeline's block is a { that line endings may precede, and where the parser's state can shift it, it is longer than the line ending alone (ATD section 3.6), so pipeline Kettle followed by a line holding only { owns that block as pipeline Kettle { would. The block's open token, for recover.nested and for a finding, is that token, line endings included. An add writes a new block in the form the binding declares, by its insert's emit and by the rule's opens (Writing): {"open": "\n{", "close": "}"} for the Wardley map's brace on its own line, {"open": " {", "close": "}"} for Structurizr's on the statement's line.

Entries, values and selectors. An entry is a node a rule's at selects; it has its own span and, when it starts and ends its lines, its line span (section 4.1.1). A value is the text of a node or token a {node} slot reaches, or the group value of its match (section 5.2). A selector is section 4.2's, in which a segment names a child node by its rule or alias, or a token by its terminal:

Segment Matches
a name the child nodes of that rule or alias, and the child tokens of that terminal
* any child node or token
** any number of levels, including none
name[path='v'] a child named name that has, at the relative selector path (segments without tests), a node or token whose text is v
. in a {node} slot only, as the whole selector: the entry itself
.. (FBL 0.15) in a {node} slot and in the slots of body.prefixes only: the node that holds this one; nothing from the root
+ (FBL 0.15) in a {node} slot only: the next child of the node that holds this one, an empty optional part being no child; nothing from the last

An absolute selector is matched from the root, so /statement selects the root's children named statement, and / alone the root; a slot's selector is relative to its entry. {capture} segments are not admitted in this family. An entry's children are its nodes and tokens in document order, an empty optional part being no child.

(FBL 0.11) Escapes. A {node} slot may take escape, {"with": "\\", "quote": "\""}, one character each, for a value written inside quotes that escapes them. Reading replaces with followed by quote by quote and with followed by with by with, and keeps with before any other character as it is, so "Jo \"the user\"" reads Jo "the user" and C:\temp reads C:\temp; the escape applies to the value after match, whose group value stays the value's span. Writing escapes every quote and every with character of the value (section 6.3). Without escape, a value is read and written as it is.

(FBL 0.13) Several items in one node. A {node} slot of a many attribute may take split, a text of one character or more, for a format that writes several items in one token, as Structurizr writes the tags "Extra, More". The slot's value, the group value of its match or else the node's text, is cut at every occurrence of split, from the left; each piece, without the white space at its start and at its end (the characters of Unicode's White_Space property), is an item, whose span is those bytes, and a piece that is then empty is no item: " Extra ,More,, " holds the two items Extra and More, and "" none. With escape, each item's escapes are resolved after the cut; split MUST NOT hold the escape's with or quote character (section 14.1), so that cutting before the escapes are resolved gives the items cutting after would. The items of every node the slot reaches follow each other in document order, a node's items in the order they are written, and a slot of gather (section 5.2) may split its own nodes. A set of the attribute is written as section 6.3 says. Without split, a node is one item, as before 0.13.

(FBL 0.12) Segments in any order. A grammar may read a part of a statement as segments that stand in any order, each a node with its own span, as wardley-map.lark reads what follows a component's coordinates: label [-30, 12], ( market ), url(name) and inertia, and any other word. A slot bound to a segment selects it by its node, wherever it stands among the others. A set replaces that segment's value alone (replace-value), and every other byte of the statement stays; a segment that is absent is written by the attribute binding's create at the place it declares (Writing); and a segment set to the empty value with empty: "remove" is removed with one separator, the whitespace before it, and nothing else. A many attribute reads every segment its selector reaches, in written order, each an item, and a set writes only the difference (section 5.2): a new segment goes right after the slot's last one, or at create.after when there is none, and a removed one goes with the whitespace before it. So the Wardley map's decorators read and write as one set, a decorator the tool type does not know being an item as written. Where a segment stands twice for a single-valued slot, the first is read and written, and the second is kept as it is.

(FBL 0.15) A value above the entry. A format can write once what several entries share: Turtle writes a subject once for every triple of its statement. A {node} slot's selector reaches it with .. and +: from the object of ex:a ex:p ex:b ., ../../../../subject/iri reaches the statement's subject and ../../verb the predicate. Such a value is the value of each entry that reaches it, and a set of it is written in one place: the splices of one edit that replace one span with one text are one splice (section 6.4); two that replace it with different texts refuse the edit.

(FBL 0.15) Further slots. An attribute binding's or lists further slots of the entry for a value that a format writes in several forms. The first slot present, the attribute's own and then or's in order, is the value; a set writes it as that slot writes (section 6.3), a slot of or that is read-only refusing it. Reading a {value} slot in or always finds it present, so it stands last. map, default and the binding's other options apply to whichever slot was read.

(FBL 0.15) A node's name. A {node} slot with name: true reads the name of the node it reaches, its rule, alias or terminal, rather than its text, so that map can state what each form of a term is: {"node": "*", "name": true, "map": {"iri": "iri", "blank_node": "blank"}}. Such a slot is read-only.

(FBL 0.15) Terms, read through the body's directives. body.prefixes declares what a slot with expand reads a term through:

Property Description
declarations Required. The entries that declare a prefix: each {at, name, iri}, at a selector from the root, name and iri {node} slots relative to the entry.
bases The entries that declare a base: each {at, iri}.
absolute Required. A regular expression whose group value is the IRI of a term written whole: <(?<value>[^>]*)>.
prefixed Required. A regular expression whose groups prefix and local are the parts of a term written under a prefix.
local Required. What a local name must match, whole, for an IRI to be written under a prefix.
terminals Required. The terminals whose tokens are terms under a prefix.

A declaration takes force at the end of its entry, and a later declaration of a prefix replaces an earlier one from there on. A slot with expand, true or {"keywords": {text: iri}}, reads the text of the node it reaches, in this order: a text keywords holds is that IRI; a text prefixed matches is the IRI of the last declaration in force of that prefix followed by the local part, each \ in it dropped with the character after it kept; any other text is written whole, and its IRI is the group value of absolute, each \uXXXX and \UXXXXXXXX read as its code point, and resolved against the last base in force by RFC 3986, section 5, when there is one and it is not absolute already, the empty IRI being the base itself. The IRI of a declaration or a base is read as such a term, at its own place. A token of terminals outside a declaration whose prefix no declaration in force names makes the body unreadable (section 7.5), with the reason "The prefix 'x:' is not declared."; a body so has one meaning or none. Section 6.3 writes a term.

(FBL 0.15) A node's number. body.ordinals names numberings of nodes, each {at, same}: at lists selectors from the root, and same names a terminal. A slot with ordinal, the name of a numbering, reads the number of the node it reaches as decimal text, and is absent when that numbering does not hold the node. The nodes the selectors reach are numbered from 0 in document order, by the first byte of each, a node before the nodes it holds, of two with one span too; a token of the terminal same takes the number of the first token of that terminal with its text. So the numbers are a function of the body's bytes: reading the same bytes twice gives each node the same number, an edit before a node may change it, and nothing writes one. A slot with ordinal is read-only, and a host MUST NOT store such a number: DISL's ephemeral ids say the same of an element's id built from one (DISL §11.5.3).

(FBL 0.15) The sequences of a string. A slot's escape may take more than with and quote, for a format whose strings have more escapes: sequences maps a character after with to the character it stands for, unicode maps a letter after with to the number of hexadecimal digits after it that give a code point, writes lists the characters a value writes by their sequence, beside quote and with, and requote is the quote a set writes the whole node in. Reading replaces each as it stands; with before any other character, or before a letter of unicode without that many digits or with no code point, is kept as it is, as before 0.15. A grammar that states its strings' escapes in its terminals never gives it such a value. A match with one group value across several quoting forms reads each: (?:"\""|"|')(?<value>(?s:.*?))(?:"\""|"|').

(FBL 0.15) The order of a reading. With body.order closed, a reading's elements are in the order their entries end, by the last byte of each, an entry held by another before it, of two with one span too, rather than in document order: so the triples of a blank node's property list come before the triple that names the blank node, as a reader that states a triple when its construct closes gives them. Every rule of the binding reads so; the order is section 15.3's reading order, and what after-last follows stays document order.

(FBL 0.11) Views. A binding of this family may have blocks (section 4.7), which name nodes that produce no element:

  • A BlockRule of the family has at and no line. With view, a {node} or {value} slot, its entries are views of the body, each named by the slot's value, which a registration's view header selects (section 9.3); a {value} slot gives the name of a view written without one. With origins (section 5.1), it is a view for readings of those origins only.
  • An element or relation rule with selectedView: true matches only the entry of the view the reading shows and the entries inside it (section 9.3); in every other view the same entries are not read, and are unbound for that reading. A rule so reads what the view's own entry says, such as the kind of view it is, as well as its statements.
  • Every view is still an entry of the body, whichever view a reading shows, so a rule about the model (DISL's View) sees all of them, and an element rule may select the same node.
  • A BlockRule's insert adds one of its entries from its emit, at the place it states, planned and undone as an element's insert is (section 6.2); the new entry produces no element, and adding a view creates no registration (section 8.4).

Reading in part. Without recover, a body that the grammar does not read, or in which text is matched by no terminal the parser's state can shift, is unreadable (section 7.5). With recover: {"unit": u, "sync": [...]}, the start rule's one expansion MUST be u*, and the body is read unit by unit, so that a part that does not parse leaves the rest readable:

  1. From the start of the body, and after each unit or skipped text, the reader passes trivia and parses one unit, with u as the start symbol: it reads tokens while the parser can shift them. When the text at the position after the last token gives no token the parser's state can shift, or the body ends, and the parser can accept u there, the unit ends after its last token.
  2. When the parser can neither shift nor accept, reading stopped: at the first byte of the token the parser could not shift, or of the text no terminal matched. The reader then skips from the unit's first byte to the end of the first token of a synchronising terminal that ends after the place reading stopped, lexing from that place with every terminal of the grammar as a candidate and ATD section 3.6's rule among them, and passing a character no terminal matches; to the end of the body when no such token follows.
  3. Each sync entry names a terminal of the grammar, or is the text of one of its anonymous string terminals (".").

The units read are the root's children, in order, as they would be in a body that parses. The skipped text is unbound: it is kept byte for byte, no rule matches anything in it, and it is reported once, as fbl.unparsed-text at the line and column where reading stopped (section 7.4). A body that parses as a whole is read as the units it is made of, so recover changes nothing for it.

(FBL 0.11) With recover.nested, {"open": o, "close": c}, each a terminal of the grammar or the text of one of its anonymous string terminals, a unit is read in part inside its blocks too, continuing the rules above:

  1. The reader keeps, for each block it is inside, the parser's state at the block's last boundary: right after the block's open token, and after each sync token shifted at that depth.
  2. When reading stops inside a block, the text from that block's last boundary is skipped to the end of the first sync token that ends after the place reading stopped, lexing with every terminal as rule 2 does; when the skipped text reaches a close token at the block's own depth first, the skip ends before it. The parser is restored to the boundary's state, and reading continues after the skipped text. The skipped text is unbound and reported once, as fbl.unparsed-text at the line and column where reading stopped.
  3. When the body ends inside blocks, each block still open ends at the end of the body: the reader closes it without text (writing nothing, and a sync token first where the grammar needs one), its node's span ends at its last token, and it is reported as fbl.unclosed-block at its open token.
  4. A close token that the parser cannot shift and that closes no block is skipped alone, unbound, and reported as fbl.unmatched-close; it closes nothing.
  5. With recover.unclosed, a map from an ignored terminal to a text, a token of that terminal that does not end with its text is reported as fbl.unclosed-comment at its first line; it is still trivia.
  6. An add whose container is a block reported fbl.unclosed-block is refused with the reason "The block is not closed, so nothing can be added to it." Every other edit is planned as usual.

Without nested, FBL 0.9's recovery stands. A body that parses as a whole is read as without it, and a unit at the top level that does not parse is skipped as rule 2 says.

Writing. The family writes with replace-value, insert-entry, remove-entry and rewrite-reference (section 6.1), and, (FBL 0.11), insert-key, remove-key and open-block. Section 6.3 gives the new text.

  • (FBL 0.11) An absent part. An attribute whose node is absent from the entry is set by its binding's create, {"emit": " \"{value}\"", "after": ["description", "name"], "pad": true}, which declares the part: emit is its text, {value} the value written as section 6.3 writes it, with the slot's escape; after lists, nearest first, the attributes of the same rule whose parts precede this one. The part is written right after the last token of the first attribute of after that is present, so before a block, a trailing comment and anything else after it, by one insert-key; when none of them is present, the set is refused. With pad: true the part is positional: each attribute of after before the first present one is written too, before it, with its own create.emit and the empty value, nearest last, in the same splice. A part that is present, empty or not, is replaced and never padded again. Setting an attribute to its empty value with empty: "remove" is a remove-key of the part's span and the whitespace between it and the token before it. Clearing the last positional part removes that part alone and leaves the empty parts before it. absent and empty: "remove" then apply as in the other families; without create, an attribute whose node is absent cannot be set.
  • (FBL 0.11) A set, or a part written where it was absent, on an entry that opens a block changes neither the block's opening nor any byte of the block.
  • (FBL 0.11) A block written around a first child. An element rule's opens is, in this family, {"open": " {", "close": "}"}. An add with place: "last-child" into an entry that has no block writes open right after the entry's last token and before a trailing comment, then the child, then the closing line, close indented like the entry, as open-block, insert-entry and open-block splices; its undo removes all of it. A line feed in open writes a line ending and the entry's indentation, for a format whose opening brace stands on the next line.

Because the text a slot or an emit writes is not quoted by FBL, and escaped only by a slot's escape (a value is written as it is, inside what its match keeps), a host MUST read the changed entry again before it applies an edit: a set after which the entry does not read with the value set, and an add after which the body does not read with one more entry of the rule, that entry having the add's values, are refused (section 6.4). So a value that is not a token of its terminal, such as a Turtle label holding a " that match does not admit, or a value holding a line break in a one-line string that escape cannot write, never reaches the body.

The family's guarantees are FBL's: reading never fails on content (section 7.4), unbound text is kept byte for byte (section 4.1.2), every edit is splices with an exact undo (sections 6, 7), and two hosts plan the same splices (section 6.5). The family's round-trip fixtures are fixtures/w3c-turtle-grammar/ and fixtures/w3c-turtle-grammar-partial/; (FBL 0.15) fixtures/grammar-reach/, fixtures/grammar-expand/, fixtures/grammar-ordinals/, fixtures/grammar-string-sequences/ and fixtures/w3c-turtle-triples/ state 0.15's constructs.

Referenced from