The render path

Who does what

Core names none of the plugins. A package brings an engine with Package::with_engine (a display::Engine that makes the engine on the host's wgpu device), a plan runner for a context with Package::with_runner, and formats with Package::with_format. ImagePipeline::new (studio/core/image-pipeline/src/lib.rs) collects the makers and runners from the installed packages through the network registry, and the engines are listed in install order, the first being the default.

From cooked values to prims

A data type renders through the SDK's Renderable facet. The host asks it with a RenderCx (the time, a Detail, the project's tolerances, a cancel token, the host's Held values) and it answers RenderItems: a Shape holding a ShapeSource (triangles or curves with shared columns) or Instances placing other items. The standard values' facets live in kinogaki-render-geometry (studio/core/render-geometry/src/lib.rs): GeometryRenderable gives a polygons value's display triangulation, the one the viewport draws, and its curves as GeometrySources; BrepRenderable gives one item a distinct body mesh made by the installed mesher. Items keeps each value's items per value, detail and tolerances, and makes each part's source after the one before it, so a drag's source knows which points moved. changes turns two sources of one part into dirty bits: TOPOLOGY when the topology differs, else POINTS narrowed to the moved ranges and MATERIALS when the subsets changed. Items are asked at Detail::Viewport of the object's display quality, so a render shows the tessellation the viewport shows.

RenderGeometry (studio/core/scene/src/render/geometry.rs) holds the document's prims, engine-neutral, one RenderObject per object: its parts (each a shape or a set of copies, its world placement, whether its Visible and Render flags let it render, its path and the Materials its looks name), the prims its kind declares (lights and fog) and, for a Camera, its camera. It rebuilds only the objects named changed; an entry rebuilt the same keeps its revision, and an index by revision answers what changed since any revision in time proportional to the change. A prototype many objects place renders as one source (HeldValues), and a value several objects show renders once too, as a prototype each object places where it stands (walk::placed_once). It also holds SceneMaterials, the scene's Materials compiled once each, and gives a keyed object's placements across an open shutter for motion blur (RenderGeometry::motion).

RenderScenes (studio/core/scene/src/render/scenes.rs) holds one RenderScene per Render Output object: its RenderOutput setup (engine, its values, the outputs beside the beauty, its Processing and its Color) and a revision that moves when the setup does. The active scene is the first Render Output at the root with its Render flag on.

The editor calls ImagePipeline::sync once a frame, after the frame's edits (studio/core/editor/src/frame/render.rs). It reads what changed in the scene and which shown values on the board are new, by allocation. While no run reads a film and no final render runs, the geometry stays as it was and only the scenes follow the document; the first film brings every object up to date once.

One target per engine instance

A Target (studio/core/scene/src/render/target.rs) is one engine instance's side of the shared geometry: what it sent, and on each update the prims of the objects changed since the revision it last saw. When nothing moved the update is empty and costs O(1). For each changed object it sends the shapes and Instancers by their dirty bits, the declared prims of the types the engine has a schema for (filled out with the schema's defaults) and Camera prims. Each distinct look is one Material prim, its network adopted for the engine (Engines::adopt); a changed Material goes out with PARAMS when only constants differ, else with NETWORK. Each image file a material or dome reads is one Texture prim from the image store. During a gesture moved points go as POINTS and shading normals wait for the update with settled. Colors are authored in the project's working space; the target converts each one to linear ACEScg (RENDER_SPACE) as it sends it, and sends everything colored again when the working space or the color configuration changes. Only prim kinds the engine's Capabilities list are sent, and a default dome lights a scene with no light when the engine declares the dome type.

A batch goes out in the order engines need: textures, then materials, then shapes and Instancers, then declared prims, with SceneSettings (units, frame rate and the Camera whose shutter placements blur across) when they change. A view's camera is view state: an orbit reaches the engine through RenderEngine::view and sends no scene update.

The engine contract

kinogaki-render-api is what an engine builds on (sdk/crates/render-api/src/engine.rs, sdk/crates/render-api/src/description.rs, sdk/crates/render-api/src/display.rs):

• EngineDescription: the engine without a device. Its id and label, its Capabilities (prim kinds, declared types, outputs as AovSpecs, realtime, Motion, film_bytes, the MaterialX nodes it implements), its settings as parameters, and its shading nodes with their MaterialRules.
• RenderEngine: update(SceneUpdate) queues a batch and returns; view(ViewId, ViewDesc) sets a film's camera, size, ViewMode (Interactive, Navigating, Final) and settings; drop_view, pause, focus; latest gives the newest Frame; problem and said report what failed and what the engine leaves out; finish gives a Pending<Image> of a view's outputs; preview renders a material swatch.
• SceneUpdate: added, removed, changed (each Change a prim with its Dirty bits and moved point Ranges), settings and settled. Batches the engine has yet to apply fold together with SceneUpdate::merge.
• thread::EngineThread: the render thread every engine runs on. Each whole pass is published as a Frame and the host's Wake is called once.
• display: an engine renders on the host's wgpu device. Display::acquire pins the texture holding a frame; Display::acquire_film gives the frame's film as one Image at film size, the beauty its main layer and the outputs the host asks for beside it as layers named by their ids, each layer a display::Texture held with no copy. Colors are linear ACEScg, premultiplied; a pixel-art film says so in its metadata (metadata::PIXELATED).

The engines in the tree are plugin libraries in the app's renders folder: Spectre (kinogaki-spectre), whose renderer (a tracer, a Metal backend and a denoiser) knows no SDK, and Pixel (kinogaki-pixel), which draws with WGSL passes on the host's wgpu device. Spectre states 600 bytes a film pixel, Pixel 320.

Live films

The live pipeline (studio/core/image-pipeline/src/live/mod.rs) is a graph of demand rebuilt each frame from what the panes ask for:

• A run is a Render view (up to four, VIEWS) or a LiveResult the Canvas asked for (an object, and optionally a node of its network): its plan, or none for the raw render, and the film it reads.
• A film (LiveFilm, studio/core/image-pipeline/src/live/films.rs) is one view of an engine. A Render view's film uses the active Render Output's engine and values, through the view's camera at the view's backing pixels. A Render Output's film has its resolution, through its Camera, else the active view's camera. Two results reading one Render Output share its film.
• A live engine is one engine instance per engine id, with its Target and Display. The engine is the Render Output's when installed, else the project's Default Engine, else the first installed (resolve_engine).

At the end of the frame whatever no run asked for goes: the run, the film (drop_view stops its passes) and an engine with no film, its target with it. The film of the surface under the pointer, a Render view or the Canvas, is named with RenderEngine::focus so it takes the first turn after an edit. While the camera moves, a view renders a smaller film in quarter-octave steps sized toward 4 ms a pass (studio/core/image-pipeline/src/view.rs); a realtime engine always gets the whole film. Render Outputs' films are budgeted by bytes: each film's pixels times its engine's film_bytes, summed, against Machine::film_budget, a quarter of the machine's memory by default. Past it every Render Output's film renders smaller by one factor, the Canvas's bar shows the percentage and the Console says so once.

On a Render view

Once a frame, before the views draw, Screen::pin (studio/core/image-pipeline/src/display.rs) takes each Render view's newest frame and pins it: its image through Display::acquire, or its film through Display::acquire_film when a plan reads it. The image is drawn through the color service's display and view with lib-color's ImageDisplay (lib-color), inside the paint callback the editor adds (studio/core/editor/src/paint.rs). A replaced pin is let go once the GPU has finished the last frame that sampled it. Until the first pass arrives the view shows the shaded view.

Comp on a Render Output's image

Core declares the Comp context (kinogaki.core.comp), and a Render Output object holds a Comp network. The Comp plugin (kinogaki-comp) fills it with node types, a compiler from the flattened network to lib-composit's Plan (ordered steps over slots: a film output, an image file, an earlier step) and a PlanRunner whose runner wraps lib-composit's GPU executor on the host's device (plugins/comp/src/runner.rs).

Which plan runs is core's choice (studio/core/image-pipeline/src/plan.rs). A Render view shows the open compiled network at its displayed node, else what the active Render Output's Processing names (Raw, a network, or one node). A Canvas result that is a Render Output shows its own Processing; a node of a network reads the render of the Render Output holding the network, or the active one's when another object holds it. Core flattens and compiles through kinogaki_network::plans and finds the runner by context. A plan whose Compiled::reads is empty asks for no film and no engine.

Screen::run calls Runner::run with a RunCx (the film, the color service, the image files) in an encoder it submits at once (sdk/crates/api/src/network/run.rs). The result is an Image of one texture layer, scene-referred ACEScg or a display's code values: a Render view draws it in the film's place, and the Canvas takes it for its result. A plan runs again on a new frame, a new plan, a new image-file epoch, a look outside that found a file changed, or a file it waited on arriving; lib-composit's executor encodes only the steps whose inputs changed.

On the Canvas, and Layout pages

The Canvas pane paints any value whose data type has the SDK's Canvas facet: a Picture of a frame and pieces, CanvasPiece::Image (an image layer placed by an Xform, with an opacity) and CanvasPiece::Text. One compositor (studio/core/image-pipeline/src/canvas/compose.rs) serves the pane and every picture written: pieces drawn back to front into an Rgba16Float target of one display's code values, premultiplied, blended in that display's encoding, then shown on the viewer's display.

Layout (kinogaki-layout) declares the context kinogaki.layout.network with on_canvas, and its value, a page and its pieces, paints through that facet (plugins/layout/src/value.rs). Place Image (plugins/layout/src/nodes.rs) places the Image wired into it from core's Image node: a file, an embedded file, or a Result. For a Result source core resolves the row as Written (the file the result last wrote, and the LiveResult showing it), and the cook gives an Image of the result's size whose layer is facets::Live, holding no pixels. Each frame the Canvas asks the pipeline for each live piece's result, which adds a run; the compositor draws that run's current image, developed by its Render Output's Color (target display and view, studio/core/image-pipeline/src/outputs.rs) and converted to the page's display when they differ. A new pass recooks nothing.

Unpinned, the Canvas shows a value that paints (a Layout page), else the open compiled network's displayed node, else the Image node the inspector shows, else the active Render Output's Processing (ImagePipeline::followed). Pin holds the pane on one LiveResult while the user works elsewhere; the pane asks for its run each frame it draws, so a hidden pin runs nothing.

Final renders

Render Image, Frame Range, Render Selected and Render All make a list of (render scene, frame) jobs (Finals::jobs, studio/core/scene/src/render/finals.rs). Each job (studio/core/image-pipeline/src/finals.rs) waits for the cook's answer at its frame, makes its engine anew with a fresh Target, sends everything, waits for the image files its materials and domes read, renders one ViewMode::Final view at the scene's resolution through its Camera (else the active view's camera) and calls RenderEngine::finish for the beauty and the chosen outputs. Live films pause meanwhile.

On the pipeline's one writer thread the Processing network, else the Render Output's own, is compiled for what it shows and for every File Out, run once on the finished outputs, and read back (studio/core/image-pipeline/src/final_plan.rs). studio/core/image-pipeline/src/files.rs picks the writer by the file's extension among the formats that write Images. A display-referred writer (PNG, JPEG) gets the beauty's code values through the Render Output's Color, tagged with the display; another (OpenEXR) gets every layer as linear light in the working space, tagged with its name and chromaticities. Each file is written beside its path and renamed into place. After the renders, Layout's File Outs composite and write their pictures (studio/core/image-pipeline/src/canvas/out.rs), a live piece showing the file its result last wrote.