Saving and loading a .kino

The document and its encodings

kinogaki_project::Project (studio/core/project/src/lib.rs) holds five things:

• the scene;
• its SceneTime;
• the View it saves;
• Kept: what the file held outside objects and nodes that no installed package reads;
• file_base: the folder that relative paths resolve from, kept beside the document and left out of the file.

It is written as a lib-prism document, through the crate kinogaki-prism. The document is a set of elements addressed by path. Each element has a type token, typed properties (scalars and arrays, each of which may hold time samples), string metadata and child elements. Connections link one slot to another. Everything numeric is a typed value; names a user types are metadata strings.

Prism has three encodings, told apart by their first bytes (detect in libs/lib-prism/crates/prism/src/encoding.rs):

• text, starting with #prisma 4.0;
• a binary crate, starting with PRSMC\0;
• a package, starting with PRSMZ\0.

A save to a path that ends in .prisma writes text. A save to any other path, .kino included, writes the uncompressed binary crate (Format::Binary). A load ignores the extension: Document::decode_with decodes whatever encoding the magic names. A .kino that holds text therefore opens as text, as some of the projects in studio/examples/scenes do. A package that carries assets is refused.

Loading uses Prism's default limits with the input size raised to MAX_PROJECT_BYTES, 2 GiB (Limits::with_max_input, libs/lib-prism/crates/prism/src/limits.rs). The other limits are nesting 128, paths of 4 KiB and 1,048,576 records per collection. Before the schema reads anything, fields::plain (studio/core/project/src/schema/fields.rs) refuses a document that holds a layer's edits, or a number that isn't finite (Prism writes such a number as 0).

The schema

studio/core/project/src/schema/ maps a Project onto the document. The header of each file shows its part of the layout, and studio/docs/KINO-FORMAT.md shows the whole layout with its naming rules.

A stored value (studio/core/project/src/schema/stored.rs) is an element typed with the value's id and named after it, with - for each .: def "kinogaki.core.tolerances" "kinogaki-core-tolerances". Its package's Stored codec writes the fields.

Each object's element is core's. It holds the name as metadata, translate, rotate, scale, pivot, the flags lock, visible, template and render, quality and position. A keyed row holds its keys as the field's time samples. Below the element (studio/core/project/src/schema/objects.rs) come:

• the kind's body, a child named body typed with the kind's id, which the kind's ObjectCodec writes. A Part has none.
• the network the object holds, a child named network typed with its context's element, which core's one node codec writes (studio/core/project/src/schema/node.rs, studio/core/project/src/schema/levels.rs).

Every node, of any type, is spelt the same way: type, name, position, bypass, template, then its parameters under their declared names. Each value kind has one field encoding (studio/core/project/src/schema/values.rs): a number is a float64, a choice its key, a ramp a float64[]. Wires are Prism connections such as in_<port> from out_<port>. Parameters named <group>/<name> are written in group elements g<k> whose name metadata is the group. A File node's format options are written this way, grouped by the format's id.

A data parameter (an Image node's embedded file) is a child typed with its data type's id, written by that type's DataCodec. Of a node's results, the schema saves only what no cook makes again, each value through its type's DataCodec:

• a held edit's delta, base and result (studio/core/project/src/schema/held.rs);
• a frozen node's kept outputs (studio/core/project/src/schema/outputs.rs).

Every other result is cooked again after a load. The document holds /project, /assets and /scene at its top; any other top-level element refuses the file.

Codecs and shared values

Packages read and write elements only. Saving and Loading (studio/core/project/src/element.rs) implement the SDK's ElementWriter and ElementReader directly over Prism:

• A body goes through its kind's ObjectCodec. ObjectCodec::read also gets a ReadCx of the installed engines.
• A value goes through its data type's DataCodec, whose write and read take an optional base. A held node's result is written against its captured base, so a codec may write only what changed; the shipped codecs write whole values.
• A stored value goes through its Stored read and write.

A codec's element is its own. Core keeps an object's placement on the object's element, and the body in its body child, so neither can overwrite the other.

A value held in several places is written once per save. The codec calls ElementWriter::share, and later places refer to the first with shared:<name> metadata. On load, read_shared reads it once and every place gets the same Arc. An embedded file (sdk/crates/api/src/standard/embedded.rs) shares its bytes this way, keyed by the first eight bytes of its blake3 hash, so equal bytes are stored once, whatever their names.

Saving

1. Editor::save (studio/core/editor/src/frame/files.rs) builds a Project from the document. The scene clone shares its objects, so it costs a map of pointers (Undo shares them the same way). The project also takes the time, the view (the open object, the canvas pin, each view's orbit) and the Kept from the last open. With no path yet, the Save dialog asks for one.
2. Files::save (studio/core/editor/src/shell/files.rs) starts ProjectTask::save (studio/core/project/src/task.rs), which spawns a thread named kinogaki-project, and records the scene revision at which the save was taken.
3. On that thread, Project::save validates the project, builds the document with schema::write, and estimates its size; a file past 2 GiB is refused. file::atomic_replace (studio/core/project/src/file.rs) creates a new temporary file, .<name>.<pid>-<n>.tmp, in the target's folder. It streams Document::write_to into that file through a 1 MiB buffer and a writer that counts the bytes for progress and the cap. It then syncs the file, renames it over the target, and on Unix syncs the folder. A failure at any step removes the temporary file and leaves the previous file as it was.
4. The editor adopts Finished::Saved at the start of a frame. If the scene revision is unchanged, the document is marked saved. If the document changed while the file was written, mark_saved_as records what was saved, and the document stays modified.

Opening

1. Files::open starts ProjectTask::open with the installed engines.
2. Project::load reads the file with file::read: a regular file only, at most 2 GiB, in pieces of 4 MiB. It decodes the bytes with Document::decode_with and builds the project with schema::read, one object at a time, then validates it. The embedded files are hashed on the same thread.
3. The worker publishes Loaded { project, report } and wakes the UI.
4. At the start of the next frame, Editor::poll takes the result (Files::take) and calls adopt_file. It writes the report to the Console, makes a new Document from the project's scene and time, and calls replace_document. That resets the cook, the image pipeline, the panes, the views and every package's state, and gives the editor's Files the new path, Kept and view. If the file saved an open object, the editor opens it again.

The current document stays in place until that frame. A panic on the worker becomes "the project worker failed; nothing was replaced". The task has no cancel.

While a task runs:

• Progress (studio/core/project/src/progress.rs) splits it into weighted stages. An open is reading 0.3, decoding 0.3 and building 0.4. A save is building 0.25, writing 0.35 and syncing 0.4. The worker wakes the UI each time another hundredth is done.
• Files::hold_input drops presses, typing and dropped files.
• From the second frame, a modal shows the operation, the file and the progress.

Strictness

A file holds what its readers read, and the load refuses anything else. Every reader marks what it takes (Taken, studio/core/project/src/taken.rs): fields, fields read with their keys, metadata, and the children it visits. At the end of schema::read, Taken::leftover names the first thing nothing took, with its element's path:

• unknown field "meshCheck";
• a field that holds keys although its reader reads none: "holds keys, and can't be animated";
• unknown metadata;
• unknown element.

Each codec read runs inside Loading::read with its own Taken, so a codec is held to the same rule. A field a reader requires and doesn't find is refused, and so is one of the wrong type. An element is complete: a parameter the node's type declares, or a field a codec reads, has to be there, because a missing one takes no default. The children of the settings, of a pane and of the assets must be in name order. One refusal fails the whole load, with one error that names the element.

Content of packages that aren't installed

What the file holds of a package that isn't installed loads in a form that keeps it, and is written back unchanged:

• A node of a type no installed package provides, or holding a data parameter, delta or output of a type none reads, or holding a network of a context none declares. It loads as a kinogaki_network::Placeholder (studio/core/network/src/node.rs). Its element is kept as read (Verbatim, studio/core/network/src/verbatim.rs), with its wires. It draws grey, and its cook fails with "Needs" and the missing package.
• An object of a kind no package declares, or one whose codec reports an ErrorKind::Missing body. It loads as Body::Placeholder (studio/core/document/src/object/body.rs) with its element kept as read, and its child objects load as their own. It can be renamed, moved in the tree and deleted.
• A stored value of a package that isn't installed, and an asset definition of a context none declares. These are kept in Kept (studio/core/project/src/kept.rs) and written back among core's children in name order.

Kept elements carry their shared: references. A reference into a kept element moves with it. A reference from a kept element to an element elsewhere keeps a copy of that element. The reference stands while the written file holds the same element at that path; otherwise the copy is written in its place.

A package installed since the file was saved has nothing in the file. Its stored values start at their defaults, and the load says so. A missing stored value of core's own is refused.

The load report

LoadReport (studio/core/project/src/report.rs) collects what a load kept and what it found wrong, in the order it was read. A reader never fails the load for these.

unread() gives one line per missing package, counting what was kept by kind: "Couldn't load 3 nodes and 1 object: acme.trees isn't installed". The editor writes those lines to the Console as information. problems() gives the misfits and defaults, which the Console shows as errors. While a node is being read, a mark and rewind take back what was recorded for it if it turns out to be a placeholder.

Versions

The schema carries no version number; the version in Prism's header (#prisma 4.0, version 4 in a crate) is the container's. The format is the layout the code reads. A change to it converts the repository's example projects in the same change, and the readers keep no code for earlier layouts. The kino_* tests in studio/core/project/tests check the format:

• kino_examples opens every project in studio/examples/scenes with no plugin installed, saves each in every encoding, and checks that each saves back byte for byte.
• kino_unknown opens generated fixtures with packages missing and checks that they save back unchanged.
• kino_determinism checks that the same project gives the same bytes every time.
• kino_damaged feeds the loader damaged files and checks that it never panics.

How other file formats are read and written is on Import and export.