Until this release, logging camera footage meant logging pixels. An Image archetype
took an array, and that array went into the recording uncompressed. For a six-camera rig at 30 fps
this is brutal arithmetic: a ten-minute run produced tens of gigabytes, most of it redundant
between adjacent frames.
0.28 adds EncodedImage, which stores the compressed bytes you already have and
decodes them lazily on the GPU.
# Before: decode on the CPU, store raw pixels
frame = decode(packet)
dl.log("world/cam", dl.Image(frame))
# After: store the packet, let the viewer decode when it needs to
dl.log("world/cam", dl.EncodedImage(bytes=packet, media_type="video/h264"))
Why this was not trivial
Video codecs are built for sequential playback. Scrubbing is the opposite: you want frame 4,183 right now, and frame 4,183 is meaningless without the keyframe before it and every inter-frame between them.
So the viewer keeps a keyframe index built at load, and decoding a seek target means finding the preceding keyframe and running forward. With a typical two-second GOP that is up to sixty frames of work for one seek — fast on a GPU decoder, unacceptable on a CPU one. Hence GPU-only.
Where hardware decode is unavailable we fall back to CPU and mark the stream degraded in the UI rather than silently dropping frame rate, because a viewer that is quietly lying about what it showed you is worse than one that admits it is slow.
Sizes
On our warehouse test recording: 41 GB of raw frames became 890 MB. The recording opens in about a second instead of a minute, and the difference is almost entirely that there is far less to read off disk.
Caveat
Pixel readout under the cursor now requires a decode of that frame, so it is marginally slower than reading from a raw array. In exchange you can keep a month of footage instead of a day.