# SimplyBeautyKit API reference (0.2.0)

The headers are the source of truth; this page summarises them by feature.

| Binding | Where | Entry point |
|---|---|---|
| C++17 | `core/include/simplybeauty/*.h` (umbrella `simplybeauty.h`) | `simplybeauty::SBEngine` |
| Kotlin | `android/lib/src/main/java/com/simplybeauty/*.kt` | `com.simplybeauty.SimplyBeautyEngine` |
| Kotlin (models) | `android/models/src/main/java/com/simplybeauty/models/` | `com.simplybeauty.models.SimplyBeautyModels` |
| ObjC / Swift | `ios/Sources/SimplyBeauty/SimplyBeauty.h`, `SimplyBeauty+Metal.h` | `SBBeautyEngine` |

Contents: [conventions](#1-conventions) · [status codes](#2-status-codes) ·
[versioning](#3-versioning) · [lifecycle and config](#4-lifecycle-and-config) ·
[frames](#5-frames-formats-rotation-mirror) · [effects](#6-effects) ·
[faces](#7-faces-tracking-and-injection) · [person mask](#8-person-mask) ·
[quality](#9-quality-tiers-and-governor) · [events, logging, stats](#10-events-logging-stats-introspection) ·
[GPU: GLES](#11-gpu-path-gles-android) · [GPU: Metal](#12-gpu-path-metal-apple) ·
[tracker and segmenter](#13-standalone-face-tracker-and-segmenter) · [studio](#14-studio) ·
[Kotlin](#15-kotlin--android) · [Swift](#16-objective-c--swift) · [experimental](#17-placeholders-and-experimental-apis)

---

## 1. Conventions

- **Ranges.** Intensities are `[0, 1]`; reshape values are `[-1, 1]` (0 = off).
  Out-of-range values (±inf included) are clamped to the range; NaN maps to 0.
  Out-of-range enum ordinals are ignored (and logged).
  `ApplyBeautyProfile` is a checked transaction: nonfinite strength or an
  invalid profile returns `InvalidArgument` without changing settings;
  finite strength is clamped to `[0, 1]`.
- **Coordinates.** Landmarks and person masks are normalised `[0, 1]`, origin
  top-left, in the **upright** image (the buffer turned clockwise by its
  `rotation`), before mirroring. With rotation 0 that is the buffer itself.
- **Colours.** Background colour is **ABGR** (`0xAABBGGRR`, low byte = R).
- **Frozen layouts** (see [API_STABILITY.md](API_STABILITY.md)):
  single-face inject = 936 floats (468 × x, y) + separate score;
  multi-face rows = `[score, x0, y0, …, x467, y467]` (937 floats per face);
  pose = `[rx, ry, rz, x, y, s, valid]`;
  expressions = `[mouthOpen, smile, browRaise, browFrown, eyeOpenL, eyeOpenR, valid]`;
  background fill modes `0` off, `1` blur, `2` colour, `3` image;
  chroma key `0` green, `1` blue, `2` red, `-1` clears.
- **Enum ordinals** are identical across C++, Kotlin (`ordinal`) and ObjC
  (raw value). The JNI and ObjC bridges check this (`bindingTableMismatches`,
  `static_assert`s in `SimplyBeauty.mm`).
- **Ownership.** The engine copies every input it keeps (config strings, LUT
  and image bytes, masks, landmarks). Frame buffers are borrowed for the call
  only. GL textures, Metal objects and CVPixelBuffers stay the caller's.
- **Threading** (C++ contract, `sb_engine.h`):
  - One frame at a time per engine: `Process*`, `ProcessTexture*` and the
    Metal calls return `Busy` when they overlap another frame. `Busy` is a
    reentrancy fence, not a queue. Frames may come from different threads over
    time.
  - Setters (`Set*`, `Clear*`, `Inject*`, `SetFilter`, `SetVirtualBackground`,
    `SetCompareMode`, `SetCallbacks`, `SetEventCallback`, `ApplyBeautyProfile`) and getters may be
    called from any thread, also during a frame. A frame snapshots the
    parameters when it starts; changes apply from the next frame. A LUT or
    background replaced mid-frame stays alive until that frame ends.
  - Getters return the last completed frame.
  - `Pause()` may be called from any thread (deferred to the end of a running
    frame). `Release()` waits for a running CPU frame. `Create`, `Init`,
    `Release` and the destructor must not overlap each other.
  - Kotlin and ObjC serialise every call on one engine lock, so a setter from
    another thread waits for the frame in flight; keep per-frame calls on the
    frame thread.
- **Allocation.** A steady-state frame performs no heap allocation on the
  CPU path, the tracker, the segmenter and Metal (all measured). GLES uses the
  same frame code, but its allocations are not measured on a device; see
  [PERFORMANCE.md](PERFORMANCE.md#allocation-policy).

## 2. Status codes

`SBStatus` (C++), `SimplyBeautyEngine.Status` (Kotlin, `code == ordinal`;
the Int-returning calls return the code), `SBStatus` (ObjC, Swift `.ok` …).
Values are frozen; new codes are appended.

| Code | Name | Typical cause | What to do |
|---|---|---|---|
| 0 | `Ok` | success | – |
| 1 | `InvalidConfig` | `Create` with width/height ≤ 0, an unknown format, or an odd size for a YUV format | fix `SBConfig` |
| 2 | `NotInitialized` | call before `Init()` / after `Release()`; GPU call before `GlInit`/`MtlInit` or after `GlTrim`/`MtlTrim` | init first; after a trim call `GlResize`/`MtlResize` (or `*Init`) |
| 3 | `UnsupportedFormat` | frame size ≠ configured size; CPU process call on a `gpu_only` engine; unsupported CVPixelBuffer format | `Resize` / `resize()` (Kotlin) / `resize(width:height:)` (Swift) / `GlResize`/`MtlResize`; use the path that matches the engine |
| 4 | `GpuError` | GL error, `GL_VERSION` below ES 3, shader compile/link failure, Metal device/compile failure, GPU call in a build without that renderer | read `GlInfo()` / `MtlInfo()`; after context loss `GlRelease()` + `GlInit()` on the new context |
| 5 | `ModelMissing` | `model_dir` set but the model of an enabled feature is missing or invalid | fix `model_dir` (Android: `SimplyBeautyModels.install`) or disable the feature |
| 6 | `Busy` | a frame (or GPU call) is already running on this engine | drop or retry the frame; never call `Process*` from an engine callback |
| 7 | `InvalidArgument` | null/undersized buffer, bad stride, pixel stride, rotation, mirror or enum; mismatched in/out sizes; zero or equal texture ids; LUT bytes that do not parse, including an image LUT that cannot be decoded, is over 8192 px per side or 36 MP, or has an unsupported size or layout; unknown fill mode | fix the argument; nothing was changed (the previous LUT stays) |
| 8 | `ResourceMissing` | a filter or background file cannot be read, or a background image cannot be decoded (including images over 8192 px per side or 36 MP) | check the path or bytes; the previous resource stays |
| 9 | `NotImplemented` | placeholder API (stickers, resource packs) | see [§17](#17-placeholders-and-experimental-apis) |

Loaders that fail also raise `SBEvent::ResourceLoadFailed` with the status as
`code`. Missing or rejected mask PNGs return no status: masks load at `Init`,
and the engine falls back to procedural masks (`ModelFallback` code 3).
Failed engine creation: Kotlin `init()` returns false and raises
`INITIALIZATION_FAILED` with the status as `code`; Swift/ObjC
`SBBeautyEngine(config:)` throws (`-initWithConfig:error:` returns nil) an
`NSError` with domain `SimplyBeauty` and `code` = the `SBStatus`, and raises
no event (no handler can be registered before the object exists).

## 3. Versioning

| C++ | Kotlin | Swift |
|---|---|---|
| `SB_VERSION_MAJOR/MINOR/PATCH`, `SB_VERSION_STRING`, `SB_VERSION_NUMBER` (0.2.0 → 200), `SB_VERSION_AT_LEAST(M,m,p)`, `SB_DEPRECATED(msg)` (C, ObjC and C++) | `SimplyBeautyEngine.getVersion()` | `SBBeautyEngine.sdkVersion` |
| `kSBVersion`, `kSBVersionString`, `kSBVersionNumber`, `SBEncodeVersion`, `SBVersionAtLeast`, `SBVersionCompatible(built, runtime)` | `SimplyBeautyModels.VERSION` (models artifact) | – |
| `SBEngine::GetVersion()` = version of the **linked** library | | |

`SBVersionCompatible`: same major, runtime ≥ headers, and for 0.x the same
minor. Policy: [API_STABILITY.md](API_STABILITY.md), [RELEASING.md](RELEASING.md).

## 4. Lifecycle and config

```cpp
#include <simplybeauty/simplybeauty.h>
using namespace simplybeauty;

SBConfig cfg;
cfg.width = 1280; cfg.height = 720;
cfg.format = SBPixFormat::RGBA;          // ProcessRGBA byte order; Process() takes any format
cfg.enableFaceTracking = true;           // built-in tracker (needs model_dir)
cfg.enableSegmentation = true;           // built-in segmenter (needs model_dir)
cfg.model_dir = "/path/models";          // face_landmarker.task, image_segmenter.tflite
cfg.resource_path = "/path/res";         // <resource_path>/masks/*.png (optional)

SBEngine::Result r = SBEngine::Create(cfg);   // validates config, loads models
if (!r.ok()) { /* r.status: InvalidConfig or ModelMissing */ }
std::unique_ptr<SBEngine> engine = std::move(r.engine);
engine->SetEventCallback(onEvent, user);      // before Init to see init events
engine->Init();                               // buffers, worker threads, masks, events
bool ready = engine->IsInitialized();         // true after Init, false after Release
// ... Process* ...
engine->Resize(640, 480);                     // keep models, effects, workers and governor
engine->Pause();                              // free scratch, reset temporal state
engine->Release();                            // idempotent; the destructor calls it
```

| `SBConfig` field | Default | Meaning |
|---|---|---|
| `width`, `height` | 0 | frame size in **buffer** orientation; must be > 0 (even for YUV formats) |
| `format` | `RGBA` | `I420`, `NV21`, `RGBA`, `BGRA`, `NV12`; sets the `ProcessRGBA` byte order (BGRA when `BGRA`) |
| `quality` | `Auto` | initial tier, see [§9](#9-quality-tiers-and-governor) |
| `enableFaceTracking` | true | built-in tracker; with `model_dir == nullptr` the engine is injection-only (`ModelFallback` code 1) |
| `enableSegmentation` | false | built-in segmenter for the virtual background (`ModelFallback` code 2 without `model_dir`) |
| `model_dir` | null | copied by `Create`. Set but missing a model → `Create` returns `ModelMissing` |
| `resource_path` | null | copied by `Create`; mask PNGs from `<resource_path>/masks/`, else procedural masks (`ModelFallback` code 3) |
| `external_gl_context` | null | reserved (unused) |
| `threads` | 0 | CPU lanes for effects and NN inference; 0 = hardware threads clamped to 4, 1 = single-threaded. Output is bit-identical for any count |
| `gpu_only` | false | GPU-only engine: no CPU buffers or worker threads, no tracker or segmenter; `Process`/`ProcessRGBA` return `UnsupportedFormat` |
| `max_faces` | 1 | faces the built-in tracker reports per frame (primary first) |

Model loading happens in `Create` (face_landmarker.task is 3.7 MB). Change
buffer dimensions with `SBEngine::Resize(width, height)`, Kotlin
`resize(width, height): Boolean`, or Swift `resize(width:height:) -> SBStatus`
(ObjC `resizeWithWidth:height:`). These retain the same loaded models, worker
pool, effect settings, decoded resources, statistics and quality governor.
Size-dependent scratch grows as needed; the first frame at a larger size
may allocate, followed by allocation-free steady-state frames.

An actual size change clears pending/held face and person-mask injections
and resets tracking/landmark history. Re-inject fresh data before the next
frame if using external detections. Same-size calls keep that history.
Face/pose getters continue to report the last completed frame until the
next frame completes.
Nonpositive, overflowing or odd YUV dimensions return `InvalidArgument`;
an uninitialized/released engine returns `NotInitialized`, and overlapping
frame work returns `Busy`. The Kotlin wrapper maps failure to `false`.

If GLES has been initialized, resize on its GL thread with the same current
context. Existing GLES/Metal targets are resized too. A GPU allocation
failure keeps the previous CPU dimensions but may discard GPU scratch;
retry `Resize`, `GlResize` or `MtlResize` before further GPU processing.

`Pause()` frees per-frame scratch and resets temporal state (tracker
smoothing, held injections, landmark smoothing, governor averaging); a
pending, not yet taken injection is kept. The next frame reallocates.
`Pause()` and `Release()` make no GL calls (see [§11](#11-gpu-path-gles-android)).

## 5. Frames: formats, rotation, mirror

### Packed RGBA / BGRA

```cpp
SBStatus ProcessRGBA(const uint8_t* src, int src_stride, uint8_t* dst, int dst_stride,
                     int width, int height,
                     SBFrameType type = SBFrameType::Video,
                     SBMirrorMode mirror = SBMirrorMode::None);
SBStatus ProcessRGBA(..., SBFrameType type, SBMirrorMode mirror, int rotation);
```

Stride ≤ 0 means `width*4`; a positive stride must be ≥ `width*4`. `src == dst`
is allowed (same stride). Byte order is BGRA when `SBConfig.format == BGRA`,
else RGBA. `width × height` must equal the config size (`UnsupportedFormat`).

### Any format: `SBFrame`

```cpp
struct SBFrame {
  SBPixFormat format = SBPixFormat::RGBA;
  int width = 0, height = 0;
  void* planes[3];
  int strides[3];            // 0 = tightly packed
  int64_t timestamp_ns = 0;
  int rotation = 0;          // clockwise degrees that turn the buffer upright
  int pixel_strides[3];      // 0 = format default
  static SBFrame Allocate(SBPixFormat fmt, int w, int h);  // owns planes: FreePlanes()
  void FreePlanes(); bool Valid() const; int PlaneCount() const; int BytesPerPixel() const;
};
SBStatus Process(const SBFrame& in, SBFrame* out,
                 SBFrameType type = SBFrameType::Video,
                 SBMirrorMode mirror = SBMirrorMode::None);
```

| Format | Planes | Strides |
|---|---|---|
| `RGBA`, `BGRA` | 1 | ≥ `w*4` |
| `I420` | 3 (Y, U, V) | Y ≥ `w`, U/V ≥ `w/2` |
| `NV12` (Y + UV, iOS 420v/420f) / `NV21` (Y + VU, Android camera1) | 2 (`planes[2]` unused) | Y ≥ `w`, chroma ≥ `w` |

- YUV sizes must be even. `in` and `out` must have the same size; their
  formats may differ (the engine converts, BT.601 video range).
- **Android `YUV_420_888`** is taken as delivered: format `I420`, planes =
  the Y/U/V buffers, `strides` = row strides, `pixel_strides[1] =
  pixel_strides[2] = uvPixelStride` (1 planar or 2 interleaved; U and V may
  be two views into one buffer). Only I420 chroma may use a non-default pixel
  stride. Chroma rows need `(w/2 - 1) * pixelStride + 1` bytes, so the
  camera's short last row is accepted.
- **Rotation** (`in.rotation`, 0/90/180/270; other multiples of 90 are taken
  modulo 360, anything else is `InvalidArgument`): effects, tracking and
  landmarks use the upright image; the output is written back in the
  buffer's orientation and size, so the frame's rotation metadata stays valid.
  `out.rotation` is ignored. Sizes are checked against `SBConfig` in buffer
  orientation.
- **Mirror** (`None`, `Horizontal`, `Vertical`, `Both`) flips the upright
  result after effects (Horizontal = left/right as viewed). Landmarks stay
  unmirrored.
- **Frame type**: `Video` tracks across frames and holds/smooths injections;
  `Image` treats the frame as an unrelated still (tracker reset before and
  after, a pending injection is used once without hold or smoothing).
- Invalid frames, strides, rotations or sizes are rejected before any work;
  a rejected call consumes no injection.

## 6. Effects

All setters are thread-safe and apply from the next frame. Kotlin/Swift names
are the same in lowerCamelCase (`setSmoothing`, …) unless noted.

### One-tap beauty profiles

`SBBeautyProfile` / Kotlin `BeautyProfile` defines `Off` (0), `Natural` (1),
`Fresh` (2), `SoftGlam` (3), `Glam` (4), `RoseGlow` (5), `PeachGlow` (6),
`GoldenHour` (7), `Dewy` (8), `RosyCheeks` (9), `VelvetNight` (10),
`MulberryNoir` (11), `CopperSunset` (12), `RoseQuartz` (13) and `CherryPop` (14).
These fourteen looks author smoothing/style, restrained whitening/style, rosiness,
sharpening, redden and pinking with skin-only processing. Natural is makeup-free;
the other looks coordinate lip and cheek colors, with eye makeup/highlight
where appropriate. All keep iris recoloring and face reshaping off. Existing
IDs are stable; recipes were deliberately retuned. See [BEAUTY_PROFILES.md](BEAUTY_PROFILES.md)
for the authored skin values and palettes.

| C++ | Kotlin | Swift |
|---|---|---|
| `ApplyBeautyProfile(SBBeautyProfile, float strength = 1.f)` → `SBStatus` | `applyBeautyProfile(BeautyProfile, strength: Float = 1f)` → status `Int` | `applyBeautyProfile(_:strength:)` → `SBStatus` |

After initialization, one call atomically replaces skin and makeup settings,
clears all reshaping, and sets LUT intensity to zero while retaining its
loaded resource. It leaves background/chroma, compare, quality, tracking,
frame dimensions and statistics intact. Strength scales effect intensities;
zero is equivalent to `Off`, finite values clamp to `[0,1]`, and invalid
profiles or nonfinite strength return `InvalidArgument` without mutation.
Before initialization or after release it returns `NotInitialized`.

Use `GetParams()` / `getParams()` / `params` to synchronize the app's sliders
after selection. Individual setters remain available for custom adjustments.
No profile loads a LUT or adds a renderer pass. Existing face requirements,
quality tiers and CPU/GLES/Metal effect support still apply. See
[BEAUTY_PROFILES.md](BEAUTY_PROFILES.md) for the looks and app integration.

### Skin

| C++ | Range / values |
|---|---|
| `SetSmoothing(v)`, `SetSmoothingStyle(SBSmoothingStyle)` | `[0,1]`; `Natural`, `Texture`, `Smooth` |
| `SetWhitening(v)`, `SetWhiteningStyle(SBWhiteningStyle)` | `[0,1]`; `ColdWhite`, `PinkWhite`, `WarmWhite`, `Wheat`, `Tan` |
| `SetSharpening(v)`, `SetRosiness(v)` | `[0,1]` |
| `SetRedden(v)`, `SetPinking(v)` | `[0,1]` (CainCamera whiten shader terms) |
| `SetBeautySkinOnly(bool)` | default true: skin effects inside the face mask (face oval minus eyes, brows, lips; union over faces). With no face (tracker off or not found) a soft skin-colour mask (BT.601 YCbCr) is used instead. false = whole frame |
| `SetCompareMode(bool)`, `IsCompareMode()` | press-and-hold before/after: output = input; face tracking still runs |

### Face reshape

`SetReshape(SBReshape, float)` with 26 params, `[-1, 1]`: `FaceThin`,
`FaceVShape`, `FaceNarrow`, `FaceShort`, `Cheekbone`, `Jawbone`, `Chin`,
`NoseSlim`, `EyeSize`, `EyeDistance`, `FaceSmall`, `Forehead`, `NoseLong`,
`Philtrum`, `MouthSize`, `MouthPosition`, `MouthSmile`, `LipThickness`,
`EyeRound`, `EyePosition`, `EyeAngle`, `EyeCornerOpen`, `LowerEyelid`,
`BrowPosition`, `BrowDistance`, `BrowThickness`. Most are two-sided (the sign
picks the direction), including `EyeSize`, `FaceSmall` and `EyeRound`. Warps
use face-local geometry and bounded displacements. Needs a current face.

`SetExtendedReshape(SBExtendedReshape, v)` supplies 21 additional controls
without changing the original 26-value layout. These include actual-iris
`PupilSize`, upper/lower lip fullness and the appended `NoseSize` (17),
`MouthWidth` (18), `MouthScale` (19) and `HeadShrink` (20). All are `[-1,1]`
except `HeadShrink`, which is `[0,1]`. Positive nose/mouth values enlarge the
named region. Head shrink acts on a broader envelope than face shrink.
Pupil size needs observed iris geometry; the legacy 468-point injection
alone does not supply it.

`SetBodyReshapeChecked(SBBodyReshape, v)` applies the original nine `[0,1]`
body controls. `SetExtendedBodyParam(SBExtendedBodyParam, v)` adds signed
`HipEnhance` and `NeckEnhance` separately; positive values widen the hip band
or narrow the observed neck band. Enable the body provider on the control
thread or inject current body landmarks. Missing/currently unreliable body
observations do not justify holding an old pose over a new frame.

### Makeup, teeth, eyes

| Item | Intensity | Style | Colour |
|---|---|---|---|
| Lipstick | `SetLipstick` | – | `SetLipstickColor(SBLipstickColor)`: Rouge, RetroRed, Peach, CoralOrange, GentlePink, VitalityOrange, Mulberry |
| Blush | `SetBlush` | `SetBlushStyle`: SunKissed, Igari, Soft, Apple, Classic, Doll, Rose | `SetBlushColor`: CoralPink, DustyRose, VividRed, Berry, SunsetOrange |
| Contour | `SetContour` | `SetContourStyle`: Natural, Sculpt, Glow, Slim, Nose, Glam | – |
| Eye shadow | `SetEyeShadow` | `SetEyeShadowStyle`: Soft, Crease, Smoky, Halo, Glow, Drama, Warm | `SetEyeShadowColor`: Plum, Brown, Gold, Pink |
| Eyeliner | `SetEyeLiner` | `SetEyeLinerStyle`: Classic, Flick, CatEye, Natural, Bold, Soft | `SetEyeLinerColor`: Burgundy, Plum, Chocolate, Coffee, Mauve |
| Eyebrow | `SetEyebrow` | `SetEyebrowStyle`: Natural, Soft, Feathered, Mist, Arched, Powder, Wild, Full, Straight | `SetEyebrowColor`: DarkBrown, Black, SoftBrown |
| Eyelash | `SetEyelash` | `SetEyelashStyle`: Classic, Manga, Winged, Wispy, Clustered, Doll | `SetEyelashColor`: Black, Brown, SoftBlack |
| Pupil | `SetPupil` | – | `SetPupilColor`: Hazel, Ice, Mocha, Olive, Gloss, Moss, Sand, Glow, Slate |
| Teeth | `SetTeethWhitening` | – | – |
| Eyes | `SetEyeBright` | – | – |

Intensities are `[0,1]`. Until a `Set*Color` call the item uses its default
colour (reported as `-1` by `GetParams`). Face-local effects use current
landmark geometry; actual semantic labels, when available, further clip
coverage. `SetMakeupColor` accepts custom ABGR pigments, `SetLipFinish`
selects Natural/Matte/Gloss, and `SetMakeupDetail` controls lower eyelid,
eye light and double eyelid. These controls do not imply learned makeup
reconstruction or guaranteed alignment during every motion.

### Detailed masks, observations and subjects

`EnableFaceParsing(true)` loads the optional, separately licensed
`face_parser_easyportrait.tflite` on the calling control thread. It produces
actual face skin, left/right brows, left/right whole eyes, lips and teeth;
it does not produce sclera, inner-mouth, pixel iris or hand occlusion.
See the [model license and packaging contract](BYTEPLUS_CONTINUATION_2026-09-28.md#optional-parser-model-and-license).
`EnableSemanticSegmentation` independently enables the six-class hair/body
provider. `InjectSemanticMask`/`CopySemanticMask` use typed owned masks with
capture timestamp, upright source geometry, alpha stride and confidence.
Only matching observations may influence a frame; geometric regions are
not relabeled as semantic tissue masks.

`GetIrisLandmarks`, `GetFaceBlendshapes` and `GetBodyLandmarkList` expose
separate results without widening the frozen 468-point face layout. Optional
body tracking returns up to four current 33-joint poses when configured for
multiple faces. `GetBodyLandmarks` retains the single-result accessor.

`GetTrackedFaces` returns session-local geometric subject IDs. Per-subject
`SetFaceEffect`/`GetFaceEffects`/`ClearFaceEffects` override selected global
controls; unset fields inherit the global value. `SetResourceSubject` selects
a current subject for resource slot 0 (2D) or 1 (3D). Lost or ambiguous IDs
do not transfer effects to another face. These IDs are stream continuity
handles, not biometric identity or account IDs.

### Filter (3D LUT)

```cpp
SBStatus SetFilter(const char* path);                          // .cube or PNG/JPEG image LUT
SBStatus SetFilter(const uint8_t* bytes, size_t len);
SBStatus SetFilter(const char* path, SBLutLayout layout);      // Auto, Square, Strip, Hald
SBStatus SetFilter(const uint8_t* bytes, size_t len, SBLutLayout layout);
void ClearFilter();                                            // keeps the intensity
void SetFilterIntensity(float);                                // [0,1], default 1
```

- `.cube`: `LUT_3D_SIZE` 2..128, optional `DOMAIN_MIN/MAX` or
  `LUT_3D_INPUT_RANGE`. 1D LUTs are rejected (`InvalidArgument`).
- Image LUTs (recognised by the PNG/JPEG signature): `Square` GPUImage tiles
  (512×512 = N 64, 64×64 = N 16), `Strip` (N tiles side by side or stacked),
  `Hald` (level L, N = L²). `Auto` picks Square for W == H, else Strip, so a
  HALD image needs `Hald` explicitly. N must be 2..128.
- A failed load keeps the current filter. An unreadable file is
  `ResourceMissing`, unparsable bytes `InvalidArgument`.
- Runs in every quality tier.

### Chroma key

`SetChromaKey(SBChromaKeyColor)` (Green, Blue, Red), `ClearChromaKey()`,
`SetChromaKeySimilarity(v)`, `SetChromaKeySmoothness(v)`,
`SetChromaKeyDesaturation(v)` (spill suppression), all `[0,1]`. Keying uses
key-channel dominance, so shading does not matter. Keyed pixels are replaced
by the background fill below (colour, blur or image); with the fill off only
the spill suppression is applied. Runs in every tier.

### Virtual background

| C++ | Effect |
|---|---|
| `SetVirtualBackgroundBlur(level)` | blur mode, `(0,1]`; 0 clears (Gaussian σ = 16·level px at 720p) |
| `SetBackgroundColor(abgr)` | colour mode |
| `SetVirtualBackground(path)` / `(bytes, len)` | image mode (PNG/JPEG, ≤ 8192 px per side and ≤ 36 MP); a failed load keeps the previous background and returns `ResourceMissing` |
| `ClearVirtualBackground()` | fill off |
| `SetBackgroundBlur(amount)` | sets only the blur level; it does not turn blur mode on |

The fills are mutually exclusive; chroma key is independent. The person mask
comes from, in order: an injected mask ([§8](#8-person-mask)), the built-in
segmenter, else a head-and-torso heuristic from the face landmarks. Runs in
every tier.

## 7. Faces: tracking and injection

### Built-in tracker

With `enableFaceTracking` and a `model_dir`, the engine runs MediaPipe
FaceLandmarker (BlazeFace short-range detector + 468-point mesh; the 10 iris
points are dropped) on its built-in interpreter, on every `Video` frame that
has no held injection. `max_faces` faces, primary first; tracked faces keep
their slot. At `Medium` quality the tracker runs every other frame. Tracked
faces use One-Euro smoothing with a per-axis lag limit of 0.002 in normalized
image coordinates. Failed current detections are cleared immediately; the
engine does not replay a missing face for five frames. The public standalone
`CreateFaceTracker` keeps its documented hold. Injected smoothing and hold
remain caller-controlled and unchanged. Hosts that already smooth their
landmarks should call `SetLandmarkSmoothing(0, 0)` to avoid filtering twice.

### Injection (platform detectors: MediaPipe, ML Kit, Vision, ARKit)

```cpp
void InjectFace(const FaceLandmarks& face);                    // score <= 0 / invalid = empty set
void InjectFace(const FaceLandmarks& face, int64_t timestamp_ns);
void InjectFaces(const FaceLandmarks* faces, int count);       // null/0 = empty set
void InjectFaces(const FaceLandmarks* faces, int count, int64_t timestamp_ns);
void ClearInjectedFace();                                      // empty set
void SetInjectedFaceHold(int frames, int max_age_ms);          // default 3 frames / 150 ms
void SetLandmarkSmoothing(float min_cutoff, float beta);       // One-Euro; default 1 Hz / 5
```

- An injection overrides the built-in tracker. The next frame takes it and it
  replaces the held set.
- **Hold**: the taken set is reused by up to `frames` more `Video` frames that
  start within `max_age_ms` of the inject call, so a detector slower than the
  camera does not flicker. `frames = 0` = one-shot; `max_age_ms <= 0` = no age
  limit. When your detector ran and found no face, inject an empty set;
  otherwise the last face stays for the hold. An empty set also lets the
  built-in tracker run again.
- **One-Euro smoothing** per face (faces matched by position): `min_cutoff`
  Hz at rest (`<= 0` disables), `beta` cutoff gain per face size per second
  (higher = less lag, more jitter). `timestamp_ns` (capture time of the frame
  the landmarks came from, one monotonic clock) is used as the smoothing
  interval when two consecutive injects carry one 1 ms..1 s apart; 0 = call
  time.
- `FaceLandmarks { float xy[936]; float z[468]; float score; }`; `valid()` =
  `score > 0`. On the GPU paths the score in `(0, 1]` is the face-mask weight.

### Reading faces

| C++ | Kotlin | Swift |
|---|---|---|
| `GetFaces()` → `std::vector<FaceDetectionResult>` (bounds, score, 936 points) | `getFaceList()` → `List<Face>`; `getFaces()` → `[count, (score, l, t, r, b)…]` | `faces` → `[SBFace]`; `faceCount()`; `getFirstFaceScore:left:top:right:bottom:` |
| `CopyFirstFaceLandmarks(float* destination, size_t count)` → `bool` | `copyFirstFaceLandmarks(destination: FloatArray)` → `Boolean` | – |
| `GetFacePose()` → `FacePose {rx, ry, rz, x, y, s, valid}` | `getFacePose()` → 7 floats | `facePose` (`SBFacePose`); `getFacePose(_:)` |
| `GetFaceExpressions()` → `FaceExpressions` | `getFaceExpressions()` → 7 floats | `faceExpressions`; `getFaceExpressions(_:)` |
| `SetCallbacks(EngineCallbacks{on_face_landmarks, on_engine_event})` | – | – |

The direct-copy getter writes the first face from the last completed frame into
a reusable buffer, avoiding face-list and temporary landmark-array creation.
Success writes exactly 936 floats (468 upright normalized `x, y` pairs); any
remaining elements stay unchanged. No valid face returns `false` without
changing the buffer, so check success before using its coordinates. C++ also
returns `false` for a null destination or count below 936; Kotlin requires at
least 936 elements and returns `false` before initialization or after release.
The existing face getters remain unchanged.

```kotlin
val landmarks = FloatArray(936) // Allocate once and reuse after each processed frame.
if (engine.copyFirstFaceLandmarks(landmarks)) {
    // Use this frame's available face coordinates.
} else {
    // Skip face effects; the buffer may still contain a previous face.
}
```

Pose: Euler radians, centre `x, y` in `[-1, 1]` (x right, y up), `s` = face
width / frame width. Expressions are `[0, 1]`. Both are geometric (from the
mesh) and valid after a frame with a face. Free functions:
`ComputeFacePose(face[, aspect])`, `ComputeFaceExpressions(face[, aspect])`.
`on_face_landmarks` runs on the frame's thread at the end of `Process*`, with
no engine lock held; it must not call `Init`, `Release` or destroy the engine.

## 8. Person mask

```cpp
SBStatus InjectPersonMask(const uint8_t* mask, int w, int h, int stride,
                          int64_t timestamp_ns = 0);   // null clears
void ClearPersonMask();
```

8-bit, 0 = background … 255 = person, any size (resized bilinearly to the
upright frame), stride ≥ w (0 = w), same space as landmarks, same hold as
injected faces. Consecutive masks blend lightly (motion-adaptive EMA) unless
over 0.5 s apart. It is preferred over the built-in segmenter and the face
heuristic. `Image` frames use a pending mask once. The mask is copied.

## 9. Quality tiers and governor

`SetQuality(SBQuality)` (initially `SBConfig.quality`), applies from the next frame:

| Tier | Effects |
|---|---|
| `High` | all |
| `Medium` | makeup, teeth whitening, eye brightening off; built-in tracker every other frame |
| `Low` | cheap smoother and whitening only (reshape, makeup, teeth, eyes, sharpen, rosiness, redden, pinking off; smoothing style ignored) |
| `Off` | no beauty effects |
| `Auto` | `High`, or the governor's pick |

Filter, chroma key and background run in every tier.

`SetFrameBudgetMs(ms)` (governor, active while the quality is `Auto`; `<= 0`
= off, the default): when the average `Process*` time stays above `ms` it
steps High → Medium → Low (at most one step per 15 frames), and back up after
90 frames under 60 % of the budget (the wait doubles, up to 32×, when a step
up has to be undone quickly). Effects a governor change turns off fade over
10 frames. The active tier is `SBStats.active_quality`; changes raise
`SBEvent::QualityChanged` (code = new tier).

## 10. Events, logging, stats, introspection

### Events

| `SBEvent` | Value | `code` |
|---|---|---|
| `LicenseValidationSuccess` / `LicenseValidationFailed` | 0 / 1 | never raised (Facebetter shape only) |
| `InitializationComplete` | 100 | 0 |
| `InitializationFailed` | 101 | `SBStatus` |
| `ModelFallback` | 102 | 1 tracker injection-only, 2 no segmentation, 3 mask PNGs missing (procedural masks) |
| `ResourceLoadFailed` | 103 | `SBStatus` of the failed load |
| `QualityChanged` | 104 | new `SBQuality` |

| C++ | Kotlin | Swift |
|---|---|---|
| `SetEventCallback(void(*)(SBEvent, int code, const char* msg, void* user), user)` and/or `EngineCallbacks::on_engine_event`; both receive every event, on the thread whose call raised it, no lock held | `setEventListener(EventListener?)`: delivered on the raising thread right after the engine lock is released; may call any method incl. `release()`; exceptions are logged and dropped (except `VirtualMachineError`) | `setEventHandler(_:queue:)`: `dispatch_async` to `queue` (nil = main), never on the calling thread; retained, capture the engine weakly |

Register before `Init()` (Kotlin: before `init()`) to see init and fallback events.

### Logging (process-wide, all engines, off by default)

| C++ | Kotlin | Swift |
|---|---|---|
| `SBEngine::SetLogConfig(SBLogConfig{console_enabled, file_enabled, level 0 Error..3 Debug, file_name})`; console = stderr (logcat on Android) | `SimplyBeautyEngine.setLogConfig(LogConfig(consoleEnabled, fileEnabled, LogLevel, fileName))`, logcat tag `SimplyBeauty` | `SBBeautyEngine.setLogLevel(.warn)`, `setLogLevel(_:console:filePath:)`; `.off` disables |

C++ also exposes `SetLogCallback`, `DrainLogMessages` and
`DispatchLogMessages`. Log messages are buffered; drain or dispatch them on
an application control thread, never from a frame callback. Dispatch does
not hold the engine lock while invoking the callback.

### Stats, capabilities, params

- `GetStats()` → `SBStats {frames_processed, avg_process_ms, last_process_ms,
  fps (smoothed frame-start rate), face_detected, active_quality}`.
- `GetCapabilities()` → `SBCaps {gles3, metal, faceMesh, segmentation,
  bodyModel, tier, device}`. `faceMesh`/`segmentation` = a built-in model is
  loaded; `gles3` = GLES renderer compiled in; `metal` = true from a
  successful `MtlInit` until `MtlRelease`/`Release`; `bodyModel` reports a
  loaded built-in body provider; `tier` = tier of the last frame (before the first frame: the
  requested one, `Auto` as `High`).
- `GetParams()` → `SBEngineParams`: every setter value as stored after
  clamping (makeup `*_color` = `-1` until set; `chroma_key` = `-1` when off),
  plus `quality`, `frame_budget_ms`, `face_hold_frames/ms`,
  `landmark_min_cutoff/beta`. Kotlin `getParams(): Params?`, Swift `params`.

## 11. GPU path: GLES (Android)

A per-engine GLES renderer processes GL textures without any CPU copy
(`SB_RENDERER=GLES` builds, i.e. the Android AAR). Other builds return
`SBStatus::GpuError` from every call below.

```cpp
SBConfig cfg; cfg.width = 1280; cfg.height = 720;
cfg.format = SBPixFormat::RGBA;
cfg.gpu_only = true;          // no CPU buffers / worker threads, no face tracker
                              // or segmenter; Process/ProcessRGBA -> UnsupportedFormat
auto engine = std::move(SBEngine::Create(cfg).engine);
engine->Init();               // creates NO GL context, makes no GL calls

// ---- all of the following on ONE thread with the SAME current ES 3.0+ context
SBStatus st = engine->GlInit();          // compile/link once + scratch (GpuError if < ES 3)
engine->GlInfo();                        // "GL_VERSION | GL_RENDERER | compiles=N | first log"
                                         // [+ " | GPU reshape/makeup off: <log>", see below]
engine->IsGlReady();                     // programs + scratch present
engine->GlResize(960, 540);              // scratch only, never recompiles; updates cfg size
                                         // (stale caller GL errors: drained + logged, not counted)
engine->InjectFace(face);                // per frame; score in (0,1] = face-mask weight
st = engine->ProcessTextureTo(src_tex, dst_tex, 960, 540, timestamp_ns);
engine->GlTrim();                        // camera session end: free scratch + LUT, keep programs;
                                         // ProcessTextureTo -> NotInitialized until GlResize/GlInit
engine->GlRelease();                     // delete every GL object; idempotent (GlInit again later)
engine->SetGpuDebugView(1);              // 0 normal, 1 skin weight m as grey, 2 regions tinted
engine->SetGpuQuality(1);                // 0 full, 1 low (smoother one pyramid level coarser)
engine->SetGpuSharpenMode(1);            // 0 sharpen x m (CPU parity), 1 sharpen x (1 - 0.5 m)
```

**`ProcessTextureTo` contract**

- `src` and `dst` are `GL_TEXTURE_2D` RGBA8, `width x height` equal to the
  current GL size (config size or the last `GlResize`), `src != dst`.
- Storage is **top-down**: texture row 0 is the top of the image (draw camera
  frames with WebRTC's `FLIP` matrix, `T(.5,.5)·S(1,-1)·T(-.5,-.5)`). Landmarks
  are normalised `(u, v)` with the origin at the top left of the same buffer
  (buffer space: sensor orientation, not mirrored; `frame.rotation` is
  metadata only).
- The kit never writes `src` (its texture parameters are not touched either:
  the kit samples through its own sampler objects). It attaches `dst` to its
  own FBO and detaches it before returning.
- On entry stale `glGetError` values are drained (logged, not failed);
  blend, depth, scissor, cull, rasterizer discard and dither are disabled and
  the colour mask is enabled (left that way).
- On exit: FBO 0, VAO 0, `ARRAY_BUFFER` 0, `PIXEL_PACK_BUFFER` 0,
  `PIXEL_UNPACK_BUFFER` 0, program 0, sampler bindings 0, texture units 0-4
  unbound, active texture `GL_TEXTURE0`, `UNPACK_ALIGNMENT` 4 and the caller's
  viewport are restored (WebRTC drawers use client-side vertex arrays, which
  need VAO 0 and `ARRAY_BUFFER` 0). Units 5 and up are never touched. Other
  unpack parameters (`ROW_LENGTH`, `SKIP_*`, `IMAGE_HEIGHT`) are read and
  zeroed once per frame, at the frame's first upload (LUT, makeup masks,
  reshape op and makeup tables), and given back before returning; a frame
  with no upload reads none of them. The kit enables the scissor test for its
  face-box draws and leaves it disabled. Any GL error then returns
  `GpuError`.
- Output alpha is 1 (every pass that writes `dst` writes alpha 1). Compare
  mode copies `src` to `dst`.
- Nothing is allocated on the frame path: after `GlTrim()` the call returns
  `NotInitialized` until `GlResize()` or `GlInit()` has allocated scratch
  again.
- `timestamp_ns` is accepted for future temporal filtering and is currently
  unused.
- Faces come only from `InjectFace`/`InjectFaces` (the GPU path never runs
  the tracker). Since 0.2.0 an injection is held like on the CPU path
  (`SetInjectedFaceHold`, default 3 frames / 150 ms); `SetInjectedFaceHold(0, 0)`
  restores the consumed-by-one-call behaviour. Without an inject the frame
  has no face and the skin mask follows skin colour.
- The quality tier applies as on the CPU path (`Medium` turns makeup, teeth
  and eye-bright off; `Low` also turns reshape off).

| Status | When |
|---|---|
| `Ok` (0) | success |
| `NotInitialized` (2) | `GlInit` has not succeeded on the current EGL context, or scratch was freed by `GlTrim` |
| `UnsupportedFormat` (3) | size mismatch |
| `GpuError` (4) | any GL error, incomplete `dst` attachment |
| `Busy` (6) | a call is already running |
| `InvalidArgument` (7) | a zero texture id, or `src == dst` |

**What the GPU renders** (order matches the CPU graph): skin weight
`m` = face-region mask (oval lifted over the forehead, minus inflated eyes,
brows and lips) x the YCbCr skin-colour gate (colour gate alone without a
face); edge-preserving smoothing on a 1/2-1/8 pyramid (mean + R8 mean absolute
deviation; the level has hysteresis and its sigma ramps with the face
weight); whitening; rosiness; redden/pinking; sharpen (weight `sharpen x m`,
CPU parity, or `sharpen x (1 - 0.5 m)` with `SetGpuSharpenMode(1)`); 3D LUT
(global, uploaded with `UNPACK_ALIGNMENT` 1; a failed upload returns
`GpuError` and is retried on the next frame). Exclusions grow by
`(k - 1) x` each ring's minor half-axis (eyes 1.4, brows 1.3, lips 1.15), so
thin brows and lips grow by a fraction of their thickness.

Face reshape, makeup (all 8 items with their styles, colours and the lips and
teeth masks), teeth whitening and eye-bright are also drawn, in the CPU
graph's order:

- **Reshape** runs first. The GPU uses the CPU's own warp ops and node
  lattice, so it never folds. For one face, landmarks are moved exactly as on
  the CPU; with several faces they differ by at most 0.2 px. The skin mask,
  smoothing and makeup follow the warped face.
- **Makeup, teeth and eye-bright** run after sharpen and before the LUT,
  inside the face boxes only. Without reshape they match the CPU to 1–2
  levels. Eye-bright matches while at most two faces' eye regions touch.
  Only the first 16 usable faces get reshape and makeup (one log line when
  more are dropped). Compare mode and debug views skip makeup.
- **Optional programs.** If the driver rejects the reshape or makeup
  programs, `GlInit` still succeeds: those effects are skipped, the skin and
  LUT path runs, and `GlInfo()` ends with `" | GPU reshape off: <log>"`,
  `" | GPU makeup off: <log>"` or `" | GPU reshape + makeup off: <log>"`.
- **Memory** at 720p: about 6 MB more than the skin path (one full-size
  RGBA8 scratch texture, 3.7 MB, and a node atlas of up to 512 x 512 RG32UI,
  2 MB), plus three rotating table textures (about 420 KB). `GlTrim` frees
  them and the mask atlas.

Chroma key and virtual background are **CPU only** and ignored by
`ProcessTextureTo`.

`Pause()` and `Release()` make no GL calls. If `Release()` finds live GL
objects (no `GlRelease()`), it logs one warning and leaks them rather than
deleting them on the wrong thread or context. `Release()` must not overlap a
`ProcessTextureTo` (or any process call) running on another thread: call it
on the GL thread after the last frame.

**Contexts.** `GlRelease()` must run while the context `GlInit()` ran on is
current. Calling `GlInit()` on a different context without `GlRelease()`
drops the old objects without GL calls (leaked, logged) and builds new ones;
`GlResize` and `ProcessTextureTo` on a different context return
`NotInitialized`.

Legacy `ProcessTexture(tex, …)` (deprecated) = `ProcessTextureTo(tex ->
internal scratch)` + `glBlitFramebuffer` back into `tex`; it calls `GlInit()`
lazily (also after `GlTrim()`), and a failed lazy `GlInit()` is not retried
until `GlRelease()` or an explicit `GlInit()`. It returns the texture handle
(unchanged on failure) and processes the texture as upright.

LiveKit/WebRTC recipe for OES camera textures: [LIVEKIT.md](LIVEKIT.md#gpu-texture-path-android).

## 12. GPU path: Metal (Apple)

`SB_RENDERER_METAL` builds: SwiftPM, CocoaPods, the XCFramework script and
CMake `-DSB_RENDERER=METAL`. Other builds return `GpuError`. The Metal frame
is the GLES skin + LUT frame (same passes, shaders and parameters). Reshape,
makeup, teeth, eye-bright and background are not drawn on Metal yet: they
stay CPU-only there, although GLES now draws reshape, makeup, teeth and
eye-bright. Metal has no thread-bound context: the calls may
come from any thread, one at a time (`Busy` otherwise). Objects are passed as
borrowed opaque pointers (`(__bridge void*)obj`); the engine retains what it
keeps. `SetGpuDebugView`, `SetGpuQuality` and `SetGpuSharpenMode` apply here too.

```cpp
SBStatus MtlInit(void* mtl_device = nullptr);   // null = current/system default; compiles
                                                // shaders once per device per process
SBStatus MtlResize(int width, int height);      // scratch only; updates the configured size
void MtlTrim();                                 // free scratch/staging/LUT; keep device+pipelines
void MtlRelease();                              // idempotent; waits for a frame (not from callbacks)
bool IsMetalReady() const;                      // any thread
const char* MtlInfo() const;                    // "Metal | <device> | library compiled|cached | ..."
SBStatus ProcessMetalTexture(void* src_mtltexture, void* dst_mtltexture, int width, int height,
                             int64_t timestamp_ns = 0, SBFrameType type = SBFrameType::Video,
                             void* mtl_command_buffer = nullptr);
SBStatus ProcessPixelBufferMetal(void* cvpixelbuffer_in, void* cvpixelbuffer_out,
                                 int64_t timestamp_ns = 0, SBFrameType type = SBFrameType::Video);
```

- **Textures**: 2D `RGBA8Unorm`/`BGRA8Unorm`, configured size, on the engine's
  device, `src` sampleable (not `framebufferOnly`), `dst` a render target,
  `src != dst`. Top-down, upright (no rotation or mirror); alpha 1; compare
  mode copies. With `mtl_command_buffer` (not yet committed, same device) the
  passes are encoded and the caller commits (`Ok` = encoded); a command
  buffer with unretained references is fine (the engine keeps its own objects
  alive until it completes). Without one the engine commits and waits.
- **Pixel buffers**: `32BGRA` or NV12 (`420v`, `420f`), IOSurface-backed,
  configured size (NV12 even); formats may differ; `in == out` allowed.
  BT.709 is honoured from `kCVImageBufferYCbCrMatrixKey` (BT.601 otherwise);
  an untagged NV12 output is encoded with the input's matrix and colour tags
  it lacks are copied from the input.
- **Faces**: injected ones (held as on the CPU path); else, in
  `ProcessPixelBufferMetal` on an engine with the built-in tracker, the
  tracker runs on a CPU copy of the input (box-downscaled while the short side
  is ≥ 1080 px). `ProcessMetalTexture` uses injected faces only.
- A call rejected for its arguments changes no frame state (a pending
  injection stays, no stats or callbacks).
- Status: `Ok`, `NotInitialized` (no `MtlInit`, or trimmed), `InvalidArgument`
  (null, wrong object type, wrong texture size or usage, committed command
  buffer), `UnsupportedFormat` (pixel format, size, no IOSurface), `Busy`,
  `GpuError`.

ObjC/Swift category `SBBeautyEngine (Metal)` ([§16](#16-objective-c--swift)).

## 13. Standalone face tracker and segmenter

```cpp
std::unique_ptr<FaceTracker> CreateFaceTracker(const char* model_dir);  // null if missing/invalid
std::unique_ptr<Segmenter>   CreateSegmenter(const char* model_dir);
std::unique_ptr<FaceTracker> MakeTemporalFaceTracker(std::unique_ptr<FaceTracker> inner,
    float ema_alpha = 0.35f, int max_hold_frames = 5, float min_score = 0.5f);
```

- `CreateFaceTracker` loads `<model_dir>/face_landmarker.task` (or
  `face_detector.tflite` + `face_landmarks_detector.tflite`) and wraps it in
  `MakeTemporalFaceTracker` with `ema_alpha = 1` (score gate 0.5 and a
  5-frame hold, no EMA). The engine uses an internal variant without the
  missing-frame hold and applies bounded One-Euro smoothing.
- `FaceTracker`: `Detect(rgba, w, h, FaceLandmarks*)` (primary face),
  `DetectAll(rgba, w, h, std::vector<FaceLandmarks>*)` (up to `max_faces()`,
  primary first; use one of the two per frame), `Reset()` (before each
  unrelated still), `SetMinScore`, `SetMaxFaces`, `SetThreadPool`,
  `SetNumThreads`. Input is tightly packed RGBA (stride `w*4`). Results are
  bit-identical for any thread count.
- `CreateSegmenter` uses the first model that loads:
  `image_segmenter.tflite`, `selfie_segmenter.tflite`, `image_segmenter.task`,
  `selfie_segmenter.task` (a `.task` may be a raw `.tflite` or a ZIP with a
  STORED `.tflite` entry). `Segment(rgba, w, h, SegmentationMask*)` returns a
  0..255 mask at `w × h`; `SetTemporalSmoothing(0..1)` (default 0), `Reset()`,
  `SetThreadPool`, `SetNumThreads`.
- Neither is thread-safe: one instance per thread.

## 14. Studio

`SBStudio` (`sb_studio.h`) is an async preview shell ported from the
Facebetter demo Studio: a worker thread, latest-wins input, publish/acquire
snapshots, no GLFW/OpenCV.

```cpp
SBStudio studio;
studio.InitRGBA(1280, 720, /*enable_face=*/true);   // or Init(SBConfig)
studio.set_filter_dir("/path/to/luts"); studio.ScanAssets();
studio.params().smoothing = 0.5f;                   // SBStudioParams (sb_params.h)
studio.PushVideo(rgba, 1280, 720);                  // or LoadImage(...) for a still
SBStudio::Snapshot snap;
if (studio.WaitForFrame(&snap, 1000)) { /* snap.processed.rgba, snap.faces, snap.stats */ }
studio.Shutdown();
```

- Inputs, outputs, `SetParams` and the hook setters may be called from any
  thread; `params()` belongs to the thread that last called the non-const
  overload (`NotifyParamsChanged()` / `SetParams()` publish).
- `ApplyStudioParams(engine, p, prev, filter_dir, background_path, stickers)`
  applies an `SBStudioParams` diff-aware (filter, sticker, chroma and
  background reloads only on change).
- `SBStudioParams` encodings: `bg_fill` 0 off, 1 blur, 2 colour, 3 preset
  image (`SBBgFill`, not `SBBgMode`); `chroma` 0 off, 1 green, 2 blue, 3 red.

`SBStudioColorParams` and `SBStudioEffectParams` are separate extension bags;
the original `SBStudioParams` layout is unchanged. Use
`ApplyStudioEffectParams(engine, effects, previous_effects)` for the latter,
or the `ApplyStudioParams` overload accepting both bags. Calls that enable
models or change resources belong on the control thread; dependency failures
return a status and do not promise rollback of earlier successful setters.

The effect bag keeps its original `extended_reshape[17]` prefix fixed.
Appended `extended_reshape_tail[4]` maps to NoseSize, MouthWidth, MouthScale
and HeadShrink, in that order; the first three are `[-1,1]`, HeadShrink is
`[0,1]`. Appended `extended_body[2]` maps to signed HipEnhance/NeckEnhance,
and `face_parsing` defaults false. Nonzero body extensions automatically
request the body provider; enabling precise parsing loads its optional
model only when that setting changes. Pass the last successfully applied
bag as `previous`. The extended bag grows at the end, so C++ callers must
recompile against the headers shipped with the linked SDK.

## 15. Kotlin / Android

Artifacts: `com.simplyapphub:simplybeauty` (AAR, no assets) and the optional
`com.simplyapphub:simplybeauty-models` (models + masks). minSdk 24.

```kotlin
val paths = SimplyBeautyModels.install(context)          // off the main thread; idempotent
val engine = SimplyBeautyEngine()
engine.setEventListener { event, code, msg -> Log.i("beauty", "$event $code $msg") }
check(engine.init(SimplyBeautyEngine.Config(
    width = 1280, height = 720,
    format = SimplyBeautyEngine.Format.I420,             // I420 | RGBA | GPU
    enableFaceTracking = true, enableSegmentation = true,
    modelDir = paths.modelDir, resourcePath = paths.resourcePath,
    threads = 0, quality = Quality.AUTO, maxFaces = 1,
)))
engine.setSmoothing(0.6f); engine.setReshape(Reshape.FACE_THIN, 0.2f)
engine.setLipstickColor(LipstickColor.RETRO_RED); engine.setLipstick(0.5f)

// CameraX ImageProxy (YUV_420_888), no repacking
val p = image.planes
val input = SimplyBeautyEngine.Frame.yuv420888(image.width, image.height,
    p[0].buffer, p[0].rowStride, p[1].buffer, p[1].rowStride, p[2].buffer, p[2].rowStride,
    p[1].pixelStride)
val output = SimplyBeautyEngine.Frame.rgba(image.width, image.height, outBuffer)
val status = engine.process(input, output, rotation = image.imageInfo.rotationDegrees,
    mirror = MirrorMode.NONE, timestampNs = image.imageInfo.timestamp)
engine.release()                                          // or use { } (AutoCloseable)
```

| Area | API |
|---|---|
| Lifecycle | `init(Config)`, `init(width, height, format, enableFaceTracking, enableSegmentation, modelDir)`, `isInitialized`, `width`/`height`, `resize(w, h)` (in place; retains models, settings, decoded resources, statistics and governor; clears injections and tracking history on an actual size change), `pause()`, `release()` / `close()` (idempotent; later calls are no-ops, process calls return 2) |
| Frames | `process(Frame, Frame, rotation, mirror, timestampNs, frameType)`; `Frame.rgba/bgra/i420/nv12/nv21/yuv420888/fromImage(Image)`; legacy `processI420(6 planes + 6 strides[, w, h, ts, mirror, frameType])` (`inY, inYStride, inU, …, outV, outVStride`), `processRGBA(src, stride, dst, stride[, w, h, ts, mirror, frameType])` |
| Buffers | direct `ByteBuffer`s, data from index 0 (position ignored: pass `slice()`), strides in bytes (≤ 0 = packed); a plane needs `stride*(rows-1)+rowBytes` bytes; non-direct, undersized or read-only output → 7 without touching memory |
| Skin / reshape / makeup | `setSmoothing`, `setSmoothingStyle`, `setWhitening`, `setWhiteningStyle`, `setSharpening`, `setRosiness`, `setRedden`, `setPinking`, `setTeethWhitening`, `setEyeBright`, `setBeautySkinOnly`, `setCompareMode`/`isCompareMode`, `setReshape(Reshape\|Int, v)`, `setBodyReshape`, `setLipstick[Color]`, `setBlush[Style\|Color]`, `setContour[Style]`, `setEyeShadow[Style\|Color]`, `setEyeLiner[Style\|Color]`, `setEyebrow[Style\|Color]`, `setEyelash[Style\|Color]`, `setPupil[Color]` (typed enum and Int overloads; out-of-range Ints ignored) |
| Filter | `setFilter(path \| ByteArray \| InputStream): Int`, `setFilterIntensity`, `clearFilter` |
| Background / chroma | `setBackgroundFill(mode \| BackgroundFill, blur = 0.5f, colorAbgr, imagePath): Int` (blur ≤ 0 → 0.5), `setVirtualBackgroundBlur`, `setBackgroundColor(abgr)`, `setBackgroundImage(path \| ByteArray \| InputStream): Int`, `clearVirtualBackground`, `setChromaKey(ChromaKeyColor \| Int; negative clears)`, `clearChromaKey`, `setChromaKeySimilarity/Smoothness/Desaturation` |
| Faces | `injectFace(xy, score)` (first 936 floats; MediaPipe's 956-float 478-point output is fine; score ≤ 0 or shorter array clears), `injectFace(points, score, stride 2\|3[, timestampNs]): Boolean`, `injectFaces(List<FloatArray>[, timestampNs])` (each ≥ 937 floats, else `IllegalStateException`; empty list clears), `clearInjectedFace`, `setInjectedFaceHold(frames, maxAgeMs)`, `setLandmarkSmoothing(minCutoff, beta)`, `getFaces()`, `getFaceList()`, `getFacePose()`, `getFaceExpressions()` |
| Person mask | `injectPersonMask(ByteBuffer \| ByteArray, w, h, stride = 0, timestampNs = 0): Int`, `clearPersonMask` |
| Quality | `setQuality(Quality \| Int)`, `setFrameBudgetMs`, `activeQuality()` |
| Introspection | `stats(): Stats`, `avgProcessMs()`, `lastProcessMs()`, `fps()`, `faceDetected()`, `framesProcessed()`, `capabilities(): Capabilities?`, `getParams(): Params?` |
| Events / logging | `setEventListener(EventListener?)`, `SimplyBeautyEngine.setLogConfig(LogConfig)`, `SimplyBeautyEngine.getVersion()` |
| Stickers (placeholders) | `setSticker`, `set3DSticker` (path or bytes; return 9), `clearSticker`, `clear3DSticker` |
| LiveKit seam | `com.simplybeauty.livekit.BeautyFrameHook(engine).processI420(…, width, height, timestampNs, mirror, rotation)`: resizes the engine in place on a size change; forward the original frame on a non-zero status. Full processor: [LIVEKIT.md](LIVEKIT.md) |

Enums (`SimplyBeautyTypes.kt`, ordinals = C++ values): `Reshape`,
`BodyReshape`, `SmoothingStyle`, `WhiteningStyle`, every makeup style/colour,
`ChromaKeyColor`, `BackgroundFill` (fill-mode ints, not `SBBgMode`),
`MirrorMode`, `FrameType`, `Quality`, `FrameFormat`, `LogLevel`,
`EngineEvent(value)`.

**Threading.** Every native call runs under one engine lock: calls from
different threads are memory-safe, `release()` waits for an in-flight frame,
and a setter from another thread waits for the current frame.

### GPU path (Kotlin)

```kotlin
val gpu = SimplyBeautyEngine()
gpu.init(1280, 720, SimplyBeautyEngine.Format.GPU, enableFaceTracking = false)
// on the GL thread (ES 3 context current):
check(gpu.glInit() == 0) { gpu.glInfo() }
gpu.glResize(960, 540)                    // also updates gpu.width / gpu.height
gpu.injectFace(xy936, weight)             // before processTexture (held like the CPU path)
val st = gpu.processTexture(srcTexId, dstTexId, 960, 540, frame.timestampNs)
gpu.glTrim()                              // camera session end
gpu.setGpuDebugView(1); gpu.setGpuQuality(1); gpu.setGpuSharpenMode(0)
gpu.glRelease()                           // on the GL thread, before release()
gpu.release()
```

A `Format.GPU` engine has no CPU buffers: `processI420` / `processRGBA` /
`process` return 3. The C++ contract in [§11](#11-gpu-path-gles-android)
applies unchanged (statuses, GL state on return, top-down storage).
`isGlReady()` is true after a successful `glInit`/`glResize` and false after
`glTrim`.

### Models artifact

`SimplyBeautyModels.install(context): Paths(modelDir, resourcePath)` copies
`assets/simplybeauty/{models,masks}` once to
`noBackupFilesDir/simplybeauty/<VERSION>/` (about 4 MB). Every copy is
verified against the SHA-256 manifest and moved into place atomically;
later calls only check sizes and a stamp. Thread- and multi-process-safe;
deletes model sets of other SDK versions. Throws `IOException` when the
assets are missing or corrupt, or the copy fails. Call it off the main thread.

## 16. Objective-C / Swift

SwiftPM product `SimplyBeauty` (iOS 13+, macOS 10.15+ for host builds),
CocoaPods `SimplyBeauty`, or `SimplyBeauty.xcframework`. `import SimplyBeauty`.

```swift
let cfg = SBConfig()
cfg.width = 1280; cfg.height = 720
cfg.format = .BGRA                   // byte order of processRGBA*; CVPixelBuffer calls read each buffer
cfg.enableFaceTracking = true
cfg.useBundledResources = true       // models + masks from SimplyBeauty_SimplyBeauty.bundle
let engine = try SBBeautyEngine(config: cfg)
engine.setEventHandler({ event, code, message in print(event.rawValue, code, message) }, queue: nil)
guard engine.initEngine() == .ok else { return }
engine.setSmoothing(0.6)
engine.setReshape(.faceThin, value: 0.15)

// camera: 420v / 420f / BGRA, in place, rotation = clockwise degrees to upright
let st = engine.process(pixelBuffer, rotation: 90, mirror: .none)
// or into another buffer:
// engine.process(src, output: dst, rotation: 90, mirror: .none, frameType: .video)
engine.release()                     // ObjC -releaseEngine (Swift imports it as release())
```

| Area | ObjC selector (Swift name where set by `NS_SWIFT_NAME` or verified) |
|---|---|
| Config | `SBConfig`: `width`, `height`, `format`, `enableFaceTracking`, `enableSegmentation`, `modelDir`, `threads`, `quality`, `maxFaces`, `resourcePath`, `useBundledResources` (default NO; nil `modelDir`/`resourcePath` then use the bundle; explicit values, `@""` included, win; a missing bundle or model never fails, it raises `ModelFallback`) |
| Bundle lookup | `+defaultResourceDirectory`, `+defaultModelDirectory`, `+resourceDirectoryInBundle:` (`resourceDirectory(in:)`) |
| Lifecycle | `-initWithConfig:error:` (`try SBBeautyEngine(config:)`), `-initEngine`, `isInitialized`, `-resizeWithWidth:height:` (`resize(width:height:)`), `-pause`, `-releaseEngine` (Swift: `release()`), `+sdkVersion` |
| CVPixelBuffer | `-processPixelBuffer:output:rotation:mirror:frameType:` (`process(_:output:rotation:mirror:frameType:)`), `-processPixelBuffer:rotation:mirror:` (`process(_:rotation:mirror:)`, in place). 32BGRA, 420v, 420f; formats may differ; strides read from the buffer. 420f is converted to video range and back (untouched pixels keep full-range values when 420f → 420f without mirror); capture 420v for the least work |
| Raw buffers | `-processRGBA:srcLength:srcStride:dst:dstLength:dstStride:width:height:[rotation:]frameType:mirror:`, `-processRGBAData:…` (NSData), `-processI420Y:yLength:yStride:U:…:width:height:[rotation:]frameType:mirror:`, `-processNV12Y:…:rotation:frameType:mirror:` (length-checked); unchecked legacy `-processRGBA:srcStride:dst:dstStride:width:height:frameType:mirror:` and `-processI420Y:…:frameType:` |
| Effects | same names as C++ in lowerCamelCase (`setSmoothing:` …); typed Swift overloads `setLipstickColor(_:)`, `setBlushStyle(_:)`, `setChromaKey(_:)`; `-setFilterAtPath:`, `-setFilterData:` (`setFilterData(_:)`), `-setFilterIntensity:`, `-clearFilter`; `-setBackgroundFillMode:blur:colorAbgr:imagePath:`, typed `setBackgroundFill(_:blur:colorAbgr:imagePath:)`, `-setVirtualBackgroundBlur:`, `-setBackgroundColor:`, `-setBackgroundImageAtPath:`, `-setBackgroundImageData:` (`setBackgroundImageData(_:)`), `-clearVirtualBackground`, chroma setters |
| Faces | `-injectFaceWithXy:count:score:` (`injectFace(withXy:count:score:)`), `injectFace(data:stride:score:[timestampNs:])` → Bool, `-injectFaces:faceCount:`, `injectFaces(data:faceCount:[timestampNs:])`, `-clearInjectedFace`, `setInjectedFaceHold(frames:maxAgeMs:)`, `setLandmarkSmoothing(minCutoff:beta:)`, `faces` (`[SBFace]`, landmarks as 936-float `NSData`), `faceCount()`, `facePose`, `faceExpressions`, `-getFacePose:`, `-getFaceExpressions:` |
| Person mask | `injectPersonMask(_:length:width:height:stride:timestampNs:)`, `injectPersonMask(data:width:height:stride:timestampNs:)`, `injectPersonMask(_:timestampNs:)` (CVPixelBuffer OneComponent8 / 16Half / 32Float, e.g. Vision person segmentation), `-clearPersonMask` |
| Quality / introspection | `-setQuality:`, `-setFrameBudgetMs:`, `stats()` (`SBStats`: `avgProcessMs`, `lastProcessMs`, `fps`, `faceDetected`, `framesProcessed`, `activeQuality`), `capabilities` (`SBCapabilities`), `params` (`SBEngineParams`) |
| Events / logging | `setEventHandler(_:queue:)`, `SBBeautyEngine.setLogLevel(_:)`, `setLogLevel(_:console:filePath:)` |
| Metal category | `setupMetal(device:)`, `-releaseMetal`, `isMetalReady`, `metalInfo`, `processPixelBufferGPU(_:output:[timestampNs:]frameType:)` (sets up the default device on first use), `processTexture(_:to:commandBuffer:frameType:)` |

All calls are serialised by an internal lock: `-releaseEngine` waits for an
in-flight frame, later calls are no-ops (process calls return
`SBStatusNotInitialized`). `SBCapabilities.metal` follows the C++ meaning
(true after a successful `setupMetal`); the header comment that calls Apple
builds CPU-only predates the Metal renderer.

**Debug builds are slow.** SwiftPM and CocoaPods compile the C++ core with the
app's configuration, so a Debug app runs the CPU pipeline at -O0 (5-50×
slower). Measure in Release, or link the Release XCFramework
([BUILD.md](BUILD.md#5-ios)).

## 17. Resources and remaining experimental APIs

`AddResourcePack` validates a bounded version-1 SimplyBeauty JSON manifest.
`SetSticker`/`Set3DSticker` select original assets, and their clear methods
remove the selection. CPU, GLES and Metal implement supported sprites and
textured meshes with bounded animation, materials and skeletal influences.
These APIs do not load proprietary BytePlus/Snapchat formats. See the
[source-delivery limits](BYTEPLUS_CONTINUATION_2026-09-28.md#source-delivered)
for scene-depth, lighting and rigging boundaries.

`GetResourceCatalog` and `GetResourceRequirements` report resources and their
dependencies. Resource parsing/loading and `SetHostTexture` belong on the
control thread; host RGBA textures are copied into immutable owned storage
(maximum eight slots and 32 MiB total). `SendInteraction` consumes normalized
upright pointer coordinates and `GetInteractionRegions` returns current
hit regions. Control calls may return dependency/validation errors; query
`GetFeatureAvailability` for the intended render path before enabling a
feature. A supported renderer is not evidence that a particular frame has
the required face, body or semantic observation.

`StartRecording`/`StopRecording` provide bounded opt-in local recording.
Observations and configuration metadata are recorded by default; raw camera
pixels require `include_input_pixels=true`. The writer uses a preallocated
bounded queue and background disk I/O; stats report dropped records and disk
limits. `SBReplayReader` validates stream sizes/version and reads into
caller-owned results. Re-rendering still needs the original resources and
configuration; recording does not guarantee bit-identical reconstruction of
adaptive runtime state.

| API | Current limitation |
|---|---|
| `ProcessTexture` (legacy in-place GLES) | deprecated: use `ProcessTextureTo` |
| `SBConfig.external_gl_context` | unused |
| `SBEvent::LicenseValidation*` | never raised |

Explicitly experimental declarations follow [API_STABILITY.md](API_STABILITY.md).
Body controls and resource methods are implemented and are no longer
placeholder no-ops.

## 18. Demo settings

Both demos save catalog control values by stable ID and the active preset
in app-local preferences (Android SharedPreferences / iOS UserDefaults).
Restore happens before processing starts. Unknown IDs are ignored; invalid
values use defaults or are clamped to the control range. Reset restores
all catalog defaults, including non-look controls, and persists the result.

The preview offers one-tap compare, default beauty, performance HUD and
camera-resolution controls, alongside the original hold-to-compare action.
Android reports p50/p95 and the percentage of its last 60 processed frame
timings above the currently selected budget (zero when disabled). These
are demo metrics; `SBStats` has not acquired percentile/drop-count fields.

## Current XYZ/iris injection and facial lighting (September 28 continuation)

`InjectFaceObservations(faces, irises, count, identity)` preserves the original
`InjectFaces` format and adds measured XYZ plus optional real ten-point iris
observations. At most 16 faces are accepted; the iris order must match the face
order. Identity is upright/unmirrored, with positive timestamp and actual output
dimensions. Zero generation/capture IDs bind to the next frame with that timestamp
and size; nonzero IDs must also match. Each positive-confidence iris observation
must carry the same identity. Stale, malformed or mismatched inputs do not become
current observations. Enriched input bypasses legacy XY-only temporal smoothing,
which would otherwise detach raw iris/depth from the face mesh.

Mobile rows are `[score, x0,y0,z0, ..., x467,y467,z467]` (1,405 floats), with optional
`[confidence, x468,y468,z468, ..., x477,y477,z477]` iris rows (31 floats). Depth is in
image-width units. Rotating 90/270 degrees changes that width unit; mirroring does
not invert depth. BestBee retains actual MediaPipe observations where available.
Its existing effect opacity remains separate from accepted-observation validity;
the mobile task does not provide a calibrated per-landmark confidence score.

`SetFaceRelighting(SBFaceRelighting)` validates strength `[0,1]`, warmth `[-1,1]`
and a finite nonzero direction, then normalizes it. Positive light Z faces the
camera; X points right and Y down. `GetFaceRelighting` returns the normalized
settings. `SBFeature::FaceRelighting` is ordinal 32. Texture paths and CPU engines
without a face model require external depth observations. Flat/missing/stale depth
and strongly overlapping face boxes suppress illumination; zero strength bypasses
the added pass. The CPU/GLES/Metal effect is directional facial fill from measured
mesh normals, not reconstruction of the scene's lights, shadows or reflectance.

Studio appends `face_relighting` after `face_parsing`; rebuild with matching C++
headers/library. BestBee exposes Face light strength and Soft/Left/Right styles,
defaulting off. Detailed source/build evidence is in
[NEAR_PARITY_2026-09-28.md](NEAR_PARITY_2026-09-28.md).