ConsciousNode's Frame Grid project claims its client-side video-to-contact-sheet format with ElasticTOK fingerprints enables LLMs to ingest and reason about video content without external dependencies, and whether this format becomes a standard ingestion path for local and agent workflows resolves the episode.

Open article · Retrieved 2026-10-11T15:42:40.212529+00:00

# Frame Grid

**ConsciousNode SoftWorks** · Browser-native video and GIF contact-sheet generator with ElasticTOK frame fingerprinting.

[Live](https://consciousnode.github.io/frame-grid)
[Version](https://github.com/ConsciousNode/frame-grid)
[License](https://github.com/ConsciousNode/frame-grid/blob/main/LICENSE)
[Xinu](https://github.com/ConsciousNode)

```
camera feed, uploaded video, or GIF
  → temporal frame sampling / GIF frame extraction
  → ROSA ElasticTOK fingerprinting
  → labeled contact-sheet grids (PNG)
  → JSON frame manifests (FPSS-compatible)
```

Single HTML file. Zero dependencies. Zero uploads. Runs entirely in your browser.

---

## What it does

Frame Grid samples frames from a video or GIF at a configurable rate, assembles them into labeled contact-sheet grids, and exports them as downloadable PNGs. With ElasticTOK enabled, each frame also gets a ROSA suffix automaton fingerprint — a structural signature that captures how repetitive or complex the frame's content is, independent of pixel-level appearance.

Originally designed as a proof of concept for the ManosNowAware visual pipeline. The GIF ingestion pipeline (v2.2–2.3) extends this to animated GIFs, enabling AI models to "watch" GIFs as a contact-sheet grid.

---

## Features

### Core

- **Camera, video, or GIF source** — live camera capture with recording, uploaded video files, or animated GIFs
- **Configurable sampling** — FPS, frame dimensions, grid columns/rows, max grids
- **Label styles** — minimal (F##), standard (F## | time), full (F## | time | fps), ROSA (F## | time | ◈CR)
- **Download** — PNG per grid with header bar showing grid number, frame range, and timestamps

### GIF Ingestion (v2.2–2.3)

Native animated GIF support. Drop a GIF → get a contact-sheet grid of every frame, timestamped to the GIF's own timing. Designed so AI models can receive the full temporal content of an animated GIF as a single image.

- **Primary decoder**: browser-native `ImageDecoder` API (Chrome 94+, Safari 17+, Firefox 113+) — handles all GIF compositing, disposal methods, interlace, and edge cases natively
- **Fallback decoder**: pure-JS GIF89a parser with LZW decompression, interlace support, and per-frame compositing for older browsers
- Animated preview via `<img>` element while frames are processed
- Frame timing labels use the GIF's native cumulative timestamps
- Dedup and ElasticTOK work identically on GIF frames as on video frames

### ElasticTOK (v2.0)

ROSA suffix automaton fingerprint per frame. Toggle in sidebar; `◈ ELASTICTOK` chip appears in header when active.

- 16×16 patch grid over the frame → RGB mean per patch → 768 quantized tokens
- Suffix automaton built on token sequence → compression ratio (CR) + state count
- b1.58 ternary packing of Float32[128] transition-density fingerprint → Uint8[32]
- CR displayed in frame label bar (ROSA label style)
- **JSON manifest download** per grid when ElasticTOK is on — FPSS-compatible format:

```
{
  "tool": "FrameGrid v2.3",
  "grid": 1,
  "frames": [
    {
      "index": 0,
      "time": 0.000,
      "isDup": false,
      "cr": 2.34,
      "stateCount": 328,
      "packed": [0, 1, 2, ...]
    }
  ]
}
```

High CR = structurally repetitive frame (static shot, uniform background).
Low CR = structurally complex frame (busy scene, high detail).

### Frame Deduplication (v2.0)

Mean Absolute Difference comparison between consecutive kept frames. Configurable threshold (MAD 1–30, default 8).

- **Skip mode** — duplicate frames are dropped entirely; grid is denser with unique content
- **Mark mode** — duplicates kept but highlighted with red label bar and `[DUP]` tag
- Session panel shows kept / duped / dup% stats

---

## Usage

**[Open Frame Grid →](https://consciousnode.github.io/frame-grid)**

1. Select **Camera**, **Upload** (video), or **GIF**
2. Set FPS, frame size, and grid dimensions in Parameters
3. Enable **ElasticTOK** for ROSA fingerprinting (optional)
4. Enable **Dedup** to filter near-identical frames (optional)
5. Hit **Process** (or **Capture** → **Stop** for camera)
6. Download grids as PNG. Download JSON manifests if ElasticTOK is on.

Works offline. No data leaves your device.

---

## Stack integration

Frame Grid's JSON manifests are designed to flow into the ConsciousNode stack:

| Manifest field | Destination |
| --- | --- |
| `packed` (Uint8[32] fingerprint) | FPSS `.cns` image entry · SheafMemory ingestion |
| `cr` (ROSA compression ratio) | RAG Time corpus-driven embedding signal |
| `isDup`, `time` | ManosNowAware visual pipeline metadata |



---

## Architecture

```
Source (camera / video file / GIF)
  ├─ Video path
  │    └─ resolveDuration()          — WebM Infinity fix (mobile Chrome)
  │    └─ seekTo() × N               — frame-accurate temporal sampling
  │    └─ proc-canvas                — single reusable extraction canvas
  └─ GIF path
       └─ gifDecode()                — routes to native or pure-JS
            ├─ gifDecodeNative()     — ImageDecoder API (primary)
            └─ gifDecodePureJS()     — GIF89a parser + LZW (fallback)
                 └─ gifLZW()         — variable-width LZW decompressor
  └─ frameMad()                      — MAD dedup vs previous kept frame
  └─ elasticTok()                    — ROSA fingerprint (if enabled)
       └─ 16×16 patch → tokens
       └─ SuffixAutomaton
       └─ CR + state count
       └─ b1.58 pack → Uint8[32]
  └─ scratch-canvas                  — single reusable draw canvas
  └─ Grid assembly                   — label bars, header bar, border
  └─ PNG download                    — hdrCanvas.toDataURL()
  └─ JSON manifest                   — Blob → URL → <a download>
```

---

## Changelog

### v2.3 — 2026-10-11 · Komorebi Interim

- **LZW bug fixed.** `dict.length > codeMask + 1` → `dict.length > codeMask`. The code-size growth check was off by one: the decoder read 9-bit codes when the bitstream had already switched to 10-bit codes, producing garbage indices that all failed the color table bounds check and silently rendered every frame black. Root cause of all-black output in v2.2.x.
- **`ImageDecoder` API as primary GIF decoder.** Native browser GIF decoding (Chrome 94+, Safari 17+, Firefox 113+). Handles compositing, disposal methods, interlace, and all edge cases automatically. Pure-JS decoder retained as fallback for older browsers.
- `gifDecode()` is now async and routes to native or fallback automatically. Log reports which path was taken.
- `index.html` updated to v2.3.
- Version string updated throughout.

### v2.2 — 2026-10-11 · Komorebi Interim

- **GIF ingestion pipeline.** Animated GIFs can be loaded as a source alongside camera and video upload. Pure-JS GIF89a decoder with full LZW decompression, interlace support, and per-frame compositing respecting all three disposal methods (leave, restore-background, restore-previous).
- GIF source button (🌀) added to source panel.
- Animated preview via `<img>` element while frames are decoded and processed.
- Frame timing labels use GIF's native cumulative timestamps.
- `processSource()` router added — PROCESS button works for both video and GIF.
- Note: v2.2.x builds produced all-black GIF grids due to the LZW bug fixed in v2.3.

### v2.1 — 2026-06-07 · Kehai Interim

- **GIF export** — pure JS GIF89a encoder, zero dependencies, Xinu-compliant. Toggle in Analysis panel.
  - LZW compression with numeric-keyed Map
  - 8×8×4 uniform 256-color palette, O(1) quantization per pixel
  - Frames sub-sampled evenly up to configurable max (default 60)
  - Configurable GIF width (160–480px), frame delay (4–50 centiseconds)
  - NETSCAPE2.0 loop extension for infinite loop
- Mobile Chrome WebM blob duration=Infinity splice fix (Kehai Interim, 2026-10-11).

### v2.0 — 2026-06-07 · Kehai Interim

- **Xinu compliance** — Google Fonts CDN import removed. System monospace stack. Zero external calls.
- **Viewport** — `user-scalable=no` removed. Browser zoom restored.
- **ElasticTOK** — `SuffixAutomaton` class inline. ROSA CR + state count + b1.58 packed fingerprint per frame.
- **Frame deduplication** — `frameMad()` MAD comparison. Configurable threshold. Skip or mark mode.
- **Single scratch canvas** — eliminates N canvas allocations per grid assembly pass.

### v1.3 — Ed Interim

- Mobile Chrome WebM blob duration=Infinity fix. `resolveDuration()` force-seeks to 1e10 and waits for `durationchange` event.

### v1.0–1.2 — Kham / Ed Interim

- Initial implementation. Proof of concept for the ManosNowAware visual pipeline.

---

## License

MIT — ConsciousNode SoftWorks