On this page
  1. Files
  2. A chapter file
  3. A case
  4. Value encoding (inputs and expected)
  5. Comparing an implementation's output
  6. The kinds
  7. reference cases
  8. The counts
  9. The widget
  10. How the values are proven

ZooBC conformance test vectors — the catalogue format

Edition 0.6.1. The test vectors are © ZooBC Foundation, licensed CC BY 4.0. The test library js/zbc-spec.js is MIT.

Files

file content
tests/NN.json the cases of specification chapter NN (01 … 36), one file per chapter
tests/all.json every case of every chapter in one list (same case format)
js/zbc-spec.js the library that computes every case kind, dependency-free, browser or Node
examples/widget.html the reference test widget: one case, Compute, Check my output

A chapter file

{
  "format": "zoobc-conformance-vectors/1",
  "edition": "0.6.1",
  "chapter": "02",
  "license": "CC BY 4.0, ZooBC Foundation",
  "cases": [ { …case… }, … ]
}

A case

field type meaning
id string stable, unique name: <chapter>-<section>-<label>, e.g. 02-1-sign-v3 (a section 3.1 is written 3_1)
chapter string the chapter number, two digits
section string the section of that chapter whose Vector the case comes from, as numbered in its heading: 1, 3.1, 6b, 12
title string what the case checks, one plain sentence (fit for a page heading)
kind string the computation, a key of ZBCSpec.KINDS; it fixes the names of the inputs and outputs. reference = a stated value with no computation (see below)
validity "valid" or "invalid" valid: the input is well-formed and has the result in expected. invalid: the input must be refused. A refusal is stated in one of two ways: expected.error (the exact refusal text), or, for a kind whose outputs are a verdict (verdict, accepted, given and the like), in those outputs. A valid case can also expect an error when the specified answer to a well-formed question is a refusal (for example "no block at that height")
source "computed" derived from the specification or from the shipped genesis files, reproducible offline (every case of this edition; no value is read from a running chain)
inputs object of strings the inputs, by name: the kind's inputs, and sometimes further ones the kind does not declare (pass every input the case carries). An input the case does not carry is absent: leave it out, never send "" (absent and empty are different cases)
expected object of strings the outputs an implementation must produce, by name (a subset of the kind's outputs is allowed). For an invalid case: {"error": "<refusal text>"}; "*" means any refusal (the chapter gives no text)
status string, optional absent in this edition: every case states what the specification and the reference node both give
note string, optional one or two sentences for the reader

Value encoding (inputs and expected)

All values are strings.

  • bytes: hex, lower or upper case, no 0x (spaces, | and ‖ inside are ignored by the library);
  • integers: decimal, may be negative; thousands separators , or _ are ignored;
  • lists: one item per line (\n); the library also accepts commas where an item cannot contain one;
  • raw lines (type lines-raw, inputs only): one record per line, its fields separated by spaces as the label says; passed to the kind as written;
  • booleans: "true" / "false";
  • text: as is (UTF-8).

Comparing an implementation's output

ZBCSpec.compare(expected, got, type) → { match, message, offset }, with type the output's type from the kind's outputs table:

  • hex: byte by byte; on a difference, offset is the first differing byte and message names both bytes; different lengths say so with the common prefix length;
  • int: by value; bool: case-insensitive; text: exact, the first differing character;
  • lines: line by line, each line compared as hex when it looks like hex.

The kinds

ZBCSpec.KINDS[kind] is { title, inputs, outputs, run }. inputs and outputs are lists of [name, label, type] with type one of hex, int, text, lines, lines-raw, bool: the label is a ready-made form label. ZBCSpec.run(kind, inputs) returns the outputs object, or { error: "<refusal text>" }.

reference cases

A vector that states a value without anything to compute (a stored field, a constant, a "none") is a case of kind reference with inputs: {} and the stated values in expected. Show them as text; there is nothing to run. validity says whether the stated behaviour is an acceptance (valid) or a refusal (invalid).

The counts

all.json carries counts: the number of cases, of computed cases (every kind except reference) and of reference cases in this edition. A site or a test runner checks that it ran every computed case against that number.

The widget

examples/widget.html is the behaviour to reproduce, in the site's own design:

  1. the inputs of the kind and any further input the case carries, prefilled from the case (a label and a text box each); an input the case does not carry stays absent unless the reader types something;
  2. Compute runs ZBCSpec.run(case.kind, inputs) and shows every output; with the case's own inputs, each output that the case expects is marked as equal to the specification;
  3. Check my output: the reader picks an output and pastes their implementation's value; ZBCSpec.compare answers MATCH or the first differing byte (shown in the expected hex);
  4. an invalid case computes to its refusal text; the check then compares the pasted text with it.

A page can read a chapter's cases from tests/NN.json over HTTP, or embed them in a <script type="application/json"> element (needed under file://, where browsers refuse to fetch local files).

How the values are proven

Before every release each computed case runs through zbc-spec.js under Node and through the ZooBC reference node's own code; the release goes out only when both give the same value for every output and equal the expected values.

This page is tests/README.md of the test-vector set, unchanged. Download the vectors from the Tests page.