The image store

Pieces

ImagePipeline::new makes one store and holds it in an Arc; the same store is lent to every target, to plan runners through RunCx::files, to cooks and final renders as Waiting, and to the canvas's pictures written (studio/core/image-pipeline/src/lib.rs).

What is asked

A source (ImageSource) is a file and how its colors read. The file is ImageFile::Path, a path as typed, relative to the project's folder unless absolute, or ImageFile::Embedded, an EmbeddedFile kept in the project. Its color is an ImageColor. A relative path in a project that has no folder yet fails, saying so.

The store answers three kinds of read (Read):

• Read::Image: colors as light in linear ACEScg, premultiplied. What engines, Comp and the shaded views take.
• Read::Picture: what a page places. A display-referred color layer keeps its code values in its display's space; a scene-referred one is as Read::Image has it. Every layer the file holds, the main one first; a reduced picture holds the main layer alone.
• Read::Data: the values as stored, integers as 0 to 1. A source whose Color is Data is always read so.

A request is an ImageKey: the file (FileKey: a resolved path, or an embedded file known by its bytes' BLAKE3 hash and name), the read, the space when the source names one, the layer an image or data read is of (a MaterialX image node's layer; none is the file's main layer), and most, the most pixels a side the asker holds. Every answer is an SDK Image in 64-pixel tiles; its color layers, and the layer of an image or data read, are RGBA half floats, and a whole picture keeps a data layer (ids, depths) as the file stores it.

An ask returns at once or waits. ask and get return at once: they answer FileImage::Read, FileImage::Failed or FileImage::Reading, and queue the file for the store's one reader thread, which wakes the window once a file. wait and wait_reduced read on the asker's own thread, waiting for a read already under way. Waiting wraps the store as the SDK's ImageFiles trait for askers that wait, cooks and final renders; the store itself implements the trait for askers that never wait, such as runners.

Reading through the format plugins

Decoding is the format plugins' work. studio/core/scene/src/render/image_files/read.rs asks the installed formats for a reader of the standard Image type, chosen by the file's extension, then by its first bytes, and reads with Purpose::Value and a default ReadBudget; a read on the reader thread is cancelled when the store's epoch moves. A file on disk reads through Input::file, which takes regular files alone, so a pipe or a device fails at once; an embedded file reads from its bytes in memory through Input::embedded. A format reads the file as one Image as the file holds it: samples as stored, each channel group a layer (an OpenEXR's parts are layers), color layers tagged as the file says (ColorTag), data layers raw, the file's attributes as metadata. With no installed format reading the file, the answer names its extension and asks for a format plugin that reads it. A format that panics fails that file and says so.

Each file is decoded once for the answers asked of it together (studio/core/scene/src/render/image_files/decode.rs): the decoded Image is held while the answers are made and for two frames after, then weakly. Each answer is made tile by tile on the process's workers (studio/core/scene/src/render/image_files/tiles.rs), a tile's pixels converted as floats in runs and reduced by whole-pixel boxes when the asker gave most; no whole-image plane is made.

Three things in the project lead to a read:

• The Image node (studio/core/package/src/image_node.rs) is the one node that reads a picture. Its rows are Source (File, Embedded or Result), File, Embedded, Result and Color. It is offered wherever a network carries the Image type, and in Materials libraries. Where networks cook (Layout, Mesh, libraries) it asks the cook's Waiting store for a picture on the cook's worker. Where networks compile (Comp, Shading) the compiler reads its rows with image_source and the host's store answers as the plan runs. A Result source is resolved by core and reads no file while its result shows live.
• Embedded files. An EmbeddedFile is a name and a range of a shared byte buffer, so a file packed in a usdz or a GLB is a slice of that file's bytes. Its BLAKE3 hash is made the first time it is asked for, in 1 MiB pieces on the workers. Embed reads the node's file whole on the project's worker; the scene holds equal bytes once (studio/core/document/src/scene/images.rs), and EmbeddedFileCodec writes them once a save for every node holding them (project files). Extract writes the bytes out and points every Image node holding them at the file.
• The File node (studio/core/package/src/file_node.rs) reads other values through the same formats, straight into its output. The types it offers leave out the Image, which the Image node reads (registry::file_types), so every picture reaches the store through an Image node.

Kept and shared

The store keeps an entry per key: Reading, Read or Failed. A read image is held weakly, so it goes with its last holder, and while anything holds it every asker gets the same tiles: an upload keyed by a tile's identity uploads nothing new. An image read for an asker that didn't wait is held strongly until that asker asks again, or for two frames (ImageFiles::follow, once a frame); a failure is said until a look outside finds the file changed, and goes once nobody asked for two frames. The store has three counters askers compare: revision moves with every changed answer, epoch when the project's folder or the colors change (every entry goes and the read under way is told to stop), and changes when a look outside finds a file changed.

How images reach their readers

• Engines. A target's Images (studio/core/scene/src/render/images.rs) sends one Texture prim per key its materials' Image inputs and its domes read: Read::Image for colors, Read::Data where the node reads data, of the layer the node names. The prim is shared by every reader and removed with the last. While a file is read its node is left out of the material, and the material goes again when the file arrives. An engine that declares no Texture prims reads no file. A final render waits until every file it reads has arrived (Target::waiting). The engine keeps the picture as it chooses; Pixel (kinogaki-pixel) keeps each at most 2048 texels a side in one atlas.
• Comp. Comp's runner (plugins/comp/src/runner.rs) asks image for each file its plan reads, uploads the file's tiles once, and lets go of files the plan no longer reads. A plan that was waiting on a file runs again when the store's revision moves.
• Layout and the Canvas. An Image node cooks a picture, Place Image places it as a CanvasPiece::Image, and the canvas's tile worker (studio/core/image-pipeline/src/canvas/tiles.rs) converts the tiles on screen: display-referred tiles to the page display's code values, scene-referred ones to ACEScg, with coarser levels made as zooms ask for them. A picture written to a file reads the files its live pieces' results last wrote through Waiting.
• Shaded views. ImageFiles::shown, asked through the scene's Displays, gives a SurfaceImage: straight colors in the working space, reduced to at most 2048 pixels a side, made by the reader in the file's one wake. The views' looks draw it (kinogaki-geometry-view).
• Tiles and the inspector. An Image node's tile asks wait_reduced for a picture of at most 512 pixels a side on a worker (studio/core/image-pipeline/src/thumbnails.rs); the Image node the inspector shows asks for its picture to show on the Canvas.

What makes an image read again

A window's ask for a file the store holds reads nothing from the disk. A look outside looks at the files: when the window is focused again, or when the system's file-system events report a change to a file the document reads. The lib-ui shell's Watch (libs/lib-ui/crates/shell/src/watch.rs) reports them through FSEvents on macOS; on other systems focusing the window is what looks outside. The editor watches the files its nodes' File rows name and every path the store holds or failed to read (ImagePipeline::image_paths). ImageFiles::look_outside compares the size and modification time of each file it holds or failed to read with those it was read at; a file written over, or put in place where it was missing, loses its entries and its decoded image, and changes moves. Then a target removes that Texture prim and sends the file again once it is read, plans run again, node tiles and the shaded views' images are made again, and the cook's own look outside recooks Image nodes whose file stamp (size and modification time) changed. An asker that waits compares the same stamp on each ask and reads a changed file again.

An embedded file never changes. A new project folder or new colors start a new epoch, and every file is read again as it is next asked for; new colors also make every target send everything colored again. At a new frame, prims that name an Image node by op: are built again, so a frame-numbered file is read for the frame.

Color

ImageColor is the Image node's Color row: Auto (the default), Data, or a space of the project's color configuration (ImageColor::Space). For a color layer, read.rs resolves the space:

1. a space the source names, by its name or an alias;
2. with Auto, the file's tag: a space the configuration knows; an ICC profile describing one of its displays, that display; another profile, converted to a wide display as it is read;
3. else the configuration's file rules on the file's name (ColorManagement::file_space);
4. else srgb_texture for integer samples and the working space for float samples.

An image read converts from that space to ACEScg; a picture read keeps a display-referred space's code values. Straight colors are premultiplied after conversion; premultiplied files stay so. Data is never converted. Format plugins hold no color maths: the conversions are the color service's (kinogaki-color-service, an OpenColorIO configuration whose default is lib-color's built-in ACES 2.0 configuration).

Limits

• One read spends within ReadLimits' defaults (sdk/crates/api/src/format/budget.rs): a picture of at most 2²⁸ pixels (16,384 × 16,384), one allocation of 4 GiB, 16 GiB of allocations in all, 16 GiB of files it names, nesting 256 deep, and 1032 bytes decoded for each byte read. A layer of 32-bit integer channels is refused as an image.
• The store has no byte budget of its own: it holds what some asker holds, fresh answers and decoded files for two frames, and failures while they are asked for.
• Reduced reads bound what small surfaces hold: 512 pixels a side for tiles, 2048 for the shaded views.
• A project file holds at most 2 GiB (MAX_PROJECT_BYTES), embedded bytes included; they count apart from the 16 MiB of authored bytes a network holds. A save past it fails before writing and names the largest embedded files (studio/core/project/src/images.rs).

The render path follows Texture prims and Comp plans from here to the screen.