Import and export

What a format declares

The contract is the SDK's format module (sdk/crates/api/src/format.rs, sdk/crates/api/src/format/) in kinogaki-studio-api. A package adds a format with Package::with_format. Installing a package (registry::install in kinogaki-network) rebuilds the registry's Formats from the formats of all installed packages, in install order. Core's own package registers no format, so nothing can be imported until a format plugin is installed.

Purpose::Objects is File ▸ Import, which makes objects from a file's items. Purpose::Value asks for one value of a type the host names: a File node's output, or a picture for core's image store. A format's Value schema can leave rows out; OBJ's leaves out Embed Textures and Bump Maps as Normal Maps. format::options (sdk/crates/api/src/format/options.rs) holds what formats share:

• option reads a typed option (a choice by its key);
• scale, toggle and embed make common rows (options::EMBED, Embed Textures, off by default);
• row and from_rows name the rows a node keeps a format's options in, <format id>/<option>.

A read gets an Input, the options and a Reading:

• **Input** (sdk/crates/api/src/format/input.rs) is a file on disk (Input::file), a file kept in the project or packed in another (Input::embedded), or bytes lent from memory (Input::memory). It gives the file's name, head, bytes, its path when it has one, and held: all of its bytes as one shared EmbeddedFile. A format keeps ranges of that file as the files packed inside it, without a copy.
• **Reading** carries the Purpose, the project's Units, the host's ColorService (colors), a settings slice of project values a host may lend by type, the ReadBudget, and wants, the data type a Value read must give.

The read returns Imported: FileItems, parents before children, and warnings for what it read with loss. Each item has a name, its parent's index, a placement relative to that parent, a visible flag and its Data. An item with no data is a group.

Finding the format for a file

kinogaki_network::Formats (studio/core/network/src/formats.rs) is the one place that looks up a reader or a writer:

1. readers(types) keeps the formats whose produces holds a type the caller can use.
2. by_extension picks the one whose extension the file name ends in, ignoring case. The longest matching extension wins (print.gcode.3mf is a gcode.3mf before it is a 3mf). On a tie, the format registered first wins.
3. Otherwise pick takes the first format whose detects accepts the file's first bytes.

A writer is found by type with writers(types), which keeps the formats whose FormatWriter::accepts holds one of the items' types, then by extension alone (writer_for). When nothing fits, readable names what the installed formats read ("not an OBJ or FBX file (.obj, .fbx)").

The Import and Export file dialogs (studio/core/editor/src/shell/dialogs.rs) list one filter for each installed format that reads, or that has a writer.

File ▸ Import

Editor::import (studio/core/editor/src/frame/files.rs) decides by where the user is:

• At an object level, a picture (a file that a format reads as standard::image()) becomes an Image node in the root Material Library named Images. That library is made if there is none (Document::import_image).
• Inside a network, a picture becomes a displayed Image node if the network offers one. Any other file becomes a displayed File node that reads it. Import refuses where no installed format reads into the network (studio/core/document/src/document/edits/import.rs).
• At an object level, any other file is read for objects.

That read runs on the project worker (ProjectTask::import, studio/core/project/src/task.rs), a thread with Rust's default 2 MiB stack, while the UI keeps drawing. exchange::read_for_import (studio/core/project/src/exchange.rs) finds the format and reads with Reading::new(Purpose::Objects, units, colors) and the options the user set for that format on Settings ▸ Import (studio/core/editor/src/settings/import_page.rs, which draws each format's schema(options, Purpose::Objects)), keyed by the last part of the format's id (obj); a format with none set reads with its defaults. Items without a name take the file's stem.

At the start of the next frame, the editor takes the result and calls Document::import_items. That is one undo step, named Import (Undo), and it selects the top-level objects it added. Scene::import_items (studio/core/document/src/scene/import.rs) walks the items in order:

• A group becomes a Part.
• A value goes to the first installed object kind whose ObjectKindSpec::adopts returns an Adoption. Kinds from the package of the project's Default Engine are tried first, so a file's lights become that engine's lights. Geometry becomes a Mesh object (Mesh), materials a Material Library, instances one of core's Instances objects (studio/core/package/src/instances/kind.rs), and a camera a Camera. If the object holds a network, the network starts from the adoption's NetworkSeed, or else shows the value from a displayed Stash.
• The placement becomes the object's transform, and the visible flag becomes the object's. A shear or a singular matrix is simplified, with a warning.
• A value that no kind adopts is left out, with a warning.

The format's warnings and the adoption's go to the Console.

The File node

The File node, kinogaki.core.file (studio/core/package/src/file_node.rs), belongs to core's package, kinogaki-core-package, and holds no format of its own. Tab offers it in any network that carries a type an installed format reads into, apart from images, which the Image node reads. The node holds two kinds of row:

• the file's path, stored relative to the project's folder when the file is inside it;
• for each installed format that reads into its context, that format's options as rows <format id>/<option>. These are seeded from the format's Purpose::Value schema when the node is made, and each format's rows show as their own section. Rows of a format that isn't installed are kept as read and saved back unchanged.

The output port's type is the first type the extension's format produces that the network carries. The cook resolves the path and picks the reader as above. It reads with Purpose::Value, with wants set to the port's type, the project's units and colors, and a budget that the cook's own cancellation cancels. It takes exactly one value of that type; a file that reads as several values is an error. The node's external stamp hashes the path, size and modification time with blake3, so the node cooks again when the file changes on disk.

Units, up axis and colors

A project's settings hold its units: the up axis (Y or Z) and a length unit (millimetre, centimetre, metre, inch or foot), as lib-math's Units. A read receives them as the SDK's Units (the up axis and metres per unit, converted by units_to_sdk) in Reading::units, and converts the file's lengths into them. format::Conversion (sdk/crates/api/src/format/conversion.rs) is the change from a file's space into the project's, C = s·Q. Here s is a uniform scale, the project's units per file unit, and Q is a turn between up axes, or a mirroring for a left-handed file. It moves:

• points by C (point);
• directions by Q (direction);
• lengths by s (length);
• a group's or a mesh's placement M to Q·M·Q⁻¹, with its translation scaled (placement). No placement carries the change of units, so a hierarchy keeps its shape;
• a camera's or a light's placement to Q·M (frame), since its value keeps the frame it looks down.

inverse is the way back, for a writer. Turns are signed permutations, applied exactly.

Lengths inside materials, such as a transmission depth or glTF's thickness, convert as points do (NetworkSeed::scale_lengths). Colors convert from the file's space into the project's working space through Reading::colors. Spaces are named by their interop names (color::LIN_REC709, color::SRGB_TEXTURE, color::SRGB_DISPLAY).

Materials

A 3D format brings its materials as one item of standard::materials(), first among its items: Materials { materials, images }.

• Each ImportedMaterial holds the key that polygons name in their primitive material attribute (instances name it at the Instance domain), a name and double_sided.
• Its inside is the material's whole Shading network as the file records it. MaterialX nodes are made by NewNode::materialx and typed kinogaki.mtlx.<node>; each texture is an Image node made by image_files::image_node::reading, wired into the texture node's file input.
• Its standard is an OpenPBR Surface form, for a host that lacks the node types inside uses.
• ImportedImages are the images that lights name. A dome names one as op: followed by the image's key.

Import adopts the item as a Material Library (studio/core/package/src/material.rs):

• Each material becomes a Material node. It holds inside when every node type in it is installed, else standard when that one's types are installed, else nothing; each missing package gets one warning.
• Each image becomes an Image node.
• The adoption's renames turn keys into paths in the items after it. Polygons' and instances' material becomes /Materials/Red, and a dome's op:sky becomes op:/Materials/Sky. Each value is renamed once, however many items hold it.

Export passes the writer the same kind of item, built by exchange::materials: every Material the written polygons and instances name, with its network as authored, its standard form from the installed engines and its Double Sided, and the images the domes name, every path absolute against the project's folder. A writer of one shading model reads a material with MaterialNetwork::try_from and translates it with MaterialNetwork::translated (to standard_surface, gltf_pbr or UsdPreviewSurface) from kinogaki-render-api. MaterialX documents go through format::materialx::{read, write} (sdk/crates/api/src/format/materialx/mod.rs), over the SDK's kinogaki-mtlx, which reads and writes the document itself. A document's colors are in lin_rec709 (format::materialx::SPACE) unless it names another space, and image_color and colorspace map a texture's colorspace to an ImageColor and back.

Images a file names

An Image node reads an ImageSource, which pairs two things:

• an ImageFile: an absolute Path, or an Embedded EmbeddedFile with its name;
• an ImageColor: Auto, Data, or a space made by ImageColor::space.

The files a file names (an OBJ's MTL libraries and maps, a glTF's buffers and images, USD layers and textures, MaterialX includes) are reached through format::files::Named (sdk/crates/api/src/format/files/named.rs). It resolves a name relative to the read file's folder, or wherever an absolute name points, reads regular files only, identifies each by its canonical path, and charges what it reads to the budget. What it couldn't resolve is listed in one warning.

Named::image gives a texture on disk as a path, or as its bytes when Embed Textures is on. Files packed inside the read file are always embedded: a usdz's entries as held.slice ranges of the file's bytes, a GLB's images copied out once. A file held in memory has no folder, so its pictures keep the names it gives, and those names resolve from the project's folder.

Scene formats pass picture bytes through without decoding them. Core's image store decodes them: it reads an Image node's file through Formats::reader_for with Purpose::Value (studio/core/scene/src/render/image_files/read.rs). PNG, JPEG and OpenEXR read a file as one item holding one standard::image(), converting nothing: samples as stored, each channel group a layer (an OpenEXR's parts are layers of one Image), colors tagged as the file says (ColorTag) and every attribute as metadata. A writer whose display_referred is true (PNG, JPEG) receives a display's code values, tagged with the display; OpenEXR receives linear light in the working space.

Read limits

A file may be hostile, so every reader checks what the file claims against the read's ReadBudget (sdk/crates/api/src/format/budget.rs) before acting on it:

The 256 levels of depth fit in the 2 MiB stack a read runs on. Every check also polls the read's cancellation, and long loops call Reading::check. Clones of a budget share what has been spent, so a reader's workers and the files it names all draw on one budget. An Input reads regular files only, up to the size each had when it was opened (files::regular). A refusal is an Error that names the file. What a reader left out is a warning.

kinogaki_studio_api_testing::read_hostile (sdk/crates/api-testing/src/hostile.rs) reads a test file the way Import does: on a 2 MiB thread, within small limits. A File node's read is cancelled by its cook and an image store read by the store; File ▸ Import reads with the default budget, which nothing cancels, since the project task has no cancel.

Parallel reading

Packages start no threads. Their parallel steps go through the SDK's geometry::parallel (map, map_blocks, chunks_mut; sdk/crates/geometry/src/parallel.rs). These run on the workers the host installs as it starts, or on the calling thread when none are installed.

OBJ streams a file in blocks of whole lines (plugins/obj/src/obj/read/stream.rs). One parallel::map of two jobs parses one block while the next is read, and each block's chunks are parsed in parallel. JPEG, OpenEXR, USD and FBX decode on the same workers.

Export

File ▸ Export… writes the scene's placed roots (exchange::placed_roots), Export Selected… the selected top-level objects (Editor::export). On the UI thread, exchange::items turns them into FileItems at the current time. A Part is a group; an object whose kind holds a network gives its shown result from the cook; one whose kind holds none gives its ObjectKindSpec::value; one of no installed kind is a group, said in the Console. Roots are placed in the world and children under their parents, and objects with no place in the world (a Material Library) are left out below the roots. The materials item goes first.

The write runs on the project worker (ProjectTask::export). exchange::export finds the writer for the path. An item of a type that writer doesn't accept becomes a group, with a warning for each. Export then calls FormatWriter::write with the writer's default options (FormatWriter::options). The Writing carries the project's units and colors, a CancelToken, and meshed for a writer of polygons: it meshes B-reps with the installed mesher at the project's display quality and tolerances, and realizes instances.

A writer writes each file whole or not at all, through format::files::replace. It writes a new file beside the target, <name>.<pid>-<n>.part, syncs it and renames it over the target; on Unix the folder is synced too. A written document names its images through format::files::Beside. A file on disk is named by its path relative to the document. An embedded file is written once, into a <stem>_textures folder beside the document. format::Unique gives each name out once. The writer's warnings ("3 curves left out") go to the Console.

Which plugins read and write what

OBJ's reading options are up, unit, scale, pieces (by o or g), normals, uvs, materials, Embed Textures and Bump Maps as Normal Maps; USD, glTF, FBX and MaterialX have Embed Textures. The writers' options are PNG's depth and compression, JPEG's quality and subsampling, and OpenEXR's compression and parts, each written at its default. The guide for writing a new format is docs/plugins/FORMAT.md; how plugins load is Plugins.