diff options
| -rw-r--r-- | AGENTS.md | 267 | ||||
| -rw-r--r-- | PLAN-sixel-expansion.md | 298 | ||||
| -rw-r--r-- | README.org | 70 | ||||
| -rw-r--r-- | kitty-graphics.el | 580 |
4 files changed, 1090 insertions, 125 deletions
diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..62d8813 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,267 @@ +# AGENTS.md -- kitty-graphics + +## Project Overview + +Pure Emacs Lisp package (`kitty-graphics.el`) for displaying images in +terminal Emacs (`emacs -nw`) using either the Kitty graphics protocol or +Sixel, with automatic protocol detection. Also renders org-mode headings +at scaled font sizes using the Kitty text sizing protocol (OSC 66). +Integrates with org-mode, image-mode, shr/eww/mu4e/gnus, dired, and dirvish. + +Language: **Emacs Lisp** (`lexical-binding: t`). Single source file (~1876 lines). +Current version: **0.3.0** (see Versioning section). + +## Build / Validate + +### Byte-compile (primary lint check -- run after every change) + +```sh +emacs -Q -batch -f batch-byte-compile kitty-graphics.el +``` + +**This is the closest thing to CI** -- always run it before considering +a change complete. + +### Load-test (verify file evaluates without error) + +```sh +emacs -Q -batch -l kitty-graphics.el -f kill-emacs +``` + +## Testing + +No automated test framework. All testing is manual/interactive and +requires a supported terminal (Kitty, WezTerm, Ghostty for Kitty backend; +foot, Konsole, xterm, mlterm, mintty for Sixel backend). +The `tests/` directory contains `test-kitty-gfx.org`, `test-image.png`, +and `test-document.pdf` for manual testing -- use these rather than +creating new test files. + +**Important**: Always set `TERM=xterm-256color` when launching terminal +Emacs -- the `xterm-kitty` terminfo is often missing. + +### Test with org-mode + +```sh +TERM=xterm-256color emacs -nw -Q -l kitty-graphics.el \ + --eval "(kitty-graphics-mode 1)" tests/test-kitty-gfx.org +# Then: C-c C-x C-v (org-toggle-inline-images) +``` + +### Test text sizing (org heading scales) + +```sh +TERM=xterm-256color emacs -nw -Q -l kitty-graphics.el \ + --eval "(setq kitty-gfx-heading-sizes-auto t)" \ + --eval "(kitty-graphics-mode 1)" tests/test-kitty-gfx.org +# Headings should render at scaled sizes automatically. +# S-TAB to unfold all, then scroll to verify fold/unfold/scroll. +``` + +### Test image-mode + +```sh +TERM=xterm-256color emacs -nw -Q -l kitty-graphics.el \ + --eval "(kitty-graphics-mode 1)" tests/test-image.png +``` + +### Debug logging + +Set `kitty-gfx-debug` to `t` to log debug info to `/tmp/kitty-gfx.log`. + +### Dev environment + +The repo uses a Nix flake (`.envrc` → `use flake`). If you have direnv +and Nix, dependencies are provided automatically. This is optional -- +only `emacs` and optionally `imagemagick` are needed. + +### Agent workflow + +When you need the user to manually test a feature in a Kitty-compatible +terminal, or have any question that requires user action, notify them: + +```sh +notify-send "kitty-graphics" "Please test: <description of what to test>" +``` + +## Code Style + +### File structure (mandatory order) + +``` +;;; file.el --- Description -*- lexical-binding: t; -*- +;;; Commentary: +;;; Code: +(require ...) ; dependencies +(declare-function ...) ; forward declarations for optional deps +(defgroup ...) ; customization group +(defcustom ...) ; user options +(defconst ...) ; constants +(defvar / defvar-local) ; internal state +;; ... implementation sections with ;;;; headers ... +(provide 'feature) +;;; file.el ends here +``` + +### Formatting + +- **Spaces only**, no tabs (`indent-tabs-mode: nil`; enforced by `Local Variables` block) +- `lexical-binding: t` required in the first line +- `;;;;` section headers to separate logical sections + +### Naming conventions + +| Category | Pattern | Example | +|-----------------------|--------------------------------|----------------------------------| +| Public API | `kitty-gfx-` prefix | `kitty-gfx-display-image` | +| Internal functions | `kitty-gfx--` (double dash) | `kitty-gfx--transmit-image` | +| Buffer-local vars | `defvar-local` + double dash | `kitty-gfx--overlays` | +| Customization vars | `defcustom` + single dash | `kitty-gfx-max-width` | +| Global internal vars | `defvar` + double dash | `kitty-gfx--next-id` | +| Advice functions | `kitty-gfx--MODE-VERB-advice` | `kitty-gfx--org-display-advice` | +| Minor mode | `kitty-graphics-mode` | | + +All identifiers use `kebab-case`. + +### Requires / Dependencies + +- Only `cl-lib` is `require`d directly +- All other dependencies (org, image-mode, shr, dired, dirvish) use + `declare-function` for forward declarations and `with-eval-after-load` + for deferred integration -- **never `require` optional packages at + top level** + +### Error handling + +- `user-error` for user-facing errors (e.g., unsupported terminal) +- `condition-case` around I/O (terminal queries, files, image processing) +- `ignore-errors` around `send-string-to-terminal` -- terminal writes + must never crash Emacs +- `when`/`unless` guards rather than explicit error branches +- `unwind-protect` for cleanup (see `kitty-gfx--refresh`) + +### Advice convention + +All advice uses `:around` and guards on both mode and terminal: + +```elisp +(defun kitty-gfx--FOO-advice (orig-fn &rest args) + (if (and kitty-graphics-mode (not (display-graphic-p))) + ... ;; terminal path + (apply orig-fn args))) +``` + +New mode integrations **must** follow this pattern. Each mode needs +explicit advice -- there is no generic image display hook. + +## Architecture + +### Backend dispatch + +Protocol-specific code is isolated behind an alist-based backend system: + +- `kitty-gfx--backends` -- alist mapping `'kitty` / `'sixel` to operation alists +- `kitty-gfx--active-backend` -- symbol for the detected backend +- `kitty-gfx--backend-fn` -- dispatch helper: `(funcall (kitty-gfx--backend-fn 'place) ...)` +- `kitty-gfx--detect-protocol` -- tries Kitty (env check), then Sixel; sets active backend + +Each backend implements: `detect`, `prepare`, `place`, `delete`, `cleanup`, `cleanup-all`. + +### Image pipeline (shared) + +1. **Prepare**: backend-specific -- Kitty transmits via APC `a=t`; Sixel encodes via ImageMagick to temp-file +2. **Reserve space**: `kitty-gfx--make-overlay` -- overlay with blank `display` property +3. **Place**: backend-specific -- Kitty emits `a=p` with `p=PID`; Sixel reads temp-file and emits DCS +4. **Refresh**: `kitty-gfx--refresh` -- debounced via `run-at-time`; walks windows, + re-places moved overlays, deletes hidden ones +5. **Cache**: `kitty-gfx--image-cache` -- keyed by file path → image ID (integer). + Dimensions are computed fresh each time. LRU eviction via `kitty-gfx--cache-lru` + (max `kitty-gfx-cache-size` entries, default 64). + Sixel also caches encoded data in temp-files under `/tmp/kitty-gfx-sixel-*.six`. + +Overlay properties: `kitty-gfx` (bool marker), `kitty-gfx-id` (image ID), +`kitty-gfx-pid` (placement ID), `kitty-gfx-cols`/`kitty-gfx-rows` (cell size), +`kitty-gfx-last-row`/`kitty-gfx-last-col` (cached position, nil if hidden), +`kitty-gfx-file` (source file path, needed by Sixel for re-encoding). +All terminal output is wrapped in synchronized output (BSU/ESU, DEC mode +2026) to prevent flicker. + +### Text sizing (OSC 66) + +Unlike images, the text sizing protocol is **stateless** — there are no +image IDs or placement IDs. The terminal renders scaled text inline +but any redraw clears it. + +1. **Detect**: `kitty-gfx--query-text-sizing-support` — 3 CPR queries + to probe width-only vs scale vs no support. Cached in + `kitty-gfx--text-sizing-support` (`scale`, `width`, or `none`). +2. **Scan**: `kitty-gfx--org-apply-heading-sizes` — walks org headings, + looks up scale from `kitty-gfx-heading-scales` alist, creates overlays. +3. **Overlay**: `kitty-gfx--make-heading-overlay` — overlay **without** + `display` property (Emacs shows the heading line normally; OSC 66 is + painted on top by the refresh cycle). No vertical space reservation. +4. **Place**: `kitty-gfx--place-heading` — emits `\e]66;s=SCALE;TEXT\a` + at cursor position with SGR bold + 24-bit foreground color from + `org-level-N` faces. +5. **Refresh**: `kitty-gfx--refresh-heading-overlay` — **always re-emits** + OSC 66 when visible (stateless protocol). Erases old position on move + via `kitty-gfx--erase-heading` (writes spaces across all rows — + per spec, overwriting any cell in the topmost row of a multicell + character erases the entire block). + +Heading overlay properties (in addition to the common ones above): +`kitty-gfx-heading` (bool), `kitty-gfx-heading-text`, `kitty-gfx-heading-scale`, +`kitty-gfx-heading-level`. Heading overlays have `modification-hooks` so +editing the heading text removes the stale overlay for re-creation. +Re-scan after edits is debounced via `kitty-gfx--heading-rescan-timer` +(buffer-local, 0.2s) to prevent redundant scans from rapid typing. + +Key design differences from images: + +- Heading overlays **preserve their position cache** in + `on-window-change` (images clear it). This lets the refresh cycle + compare old→new position and erase properly. Images can afford to + clear cache because re-placing with the same PID is idempotent. +- In `on-buffer-change`, heading overlays are **defensively erased** + when their buffer becomes hidden (not visible in any window). This + prevents ghost multicell artifacts if the new buffer's content does + not fully overwrite the heading's terminal cells. +- Overlap detection (`kitty-gfx--heading-occupied-rows`) is reset + **per-window** inside `walk-windows`, not globally. This ensures + headings in one window don't block headings at different terminal + rows in another window showing the same buffer. + +## Known Issues & Planned Work + +- **Fixed**: Overlays remain visible when org headings are collapsed (#1) +- **Limitation**: Does not work inside tmux (neither Kitty passthrough nor Sixel) (#2) +- **Limitation**: Each mode needs explicit `:around` advice integration +- **Limitation**: GIF files are not supported (multi-frame conversion issues, + no animation support in terminal) +- **Limitation**: Sixel is 256 colors (vs truecolor on Kitty), stateless (re-emits on scroll) +- **Done**: LaTeX fragment preview in org-mode (#3) +- **Done**: doc-view / pdf-view-mode integration (#4) +- **Done**: Text sizing protocol for org heading sizes (OSC 66, Kitty >= 0.40.0) +- **Done**: Sixel backend with auto-detection (v0.3.0) +- **WIP**: Sixel image misplacement in org-mode (reported in foot terminal) +- **Planned**: comint-mime integration for shell buffer images (#5) +- **Planned**: Newsticker (RSS reader) integration (#6) + +## Dependencies + +- **Required**: Emacs >= 27.1, a supported terminal (Kitty/WezTerm/Ghostty + for Kitty backend; foot/Konsole/xterm/mlterm/mintty for Sixel) +- **Required for Sixel**: ImageMagick (`magick`/`convert`) for encoding +- **Optional for Kitty**: ImageMagick for non-PNG formats and pixel-accurate sizing +- **No external Elisp deps** beyond built-in `cl-lib` + +## Versioning + +Version is declared in the file header (`Version: X.Y.Z` in +`kitty-graphics.el` line 6). This is the **single source of truth**. + +- **Patch** (0.2.x): bug fixes, minor improvements +- **Minor** (0.x.0): new features, new mode integrations +- **Major** (x.0.0): breaking changes to public API (`kitty-gfx-` symbols) + +Bump the version in the header when preparing a release. diff --git a/PLAN-sixel-expansion.md b/PLAN-sixel-expansion.md new file mode 100644 index 0000000..ae32efc --- /dev/null +++ b/PLAN-sixel-expansion.md @@ -0,0 +1,298 @@ +# PLAN: Sixel Backend Expansion — terminal-graphics.el + +## Meta +- **Date**: 2026-03-20 +- **Mode**: SCOPE EXPANSION +- **Version**: 0.2.1 → 0.3.0 +- **Estimated Effort**: 28-41 hours (~3-5 days) +- **LOC Delta**: +300-400 lines (1948 → ~2300) +- **Risk**: MEDIUM (terminal quirks) +- **Confidence**: HIGH (clean abstraction perimeter) + +--- + +## Motivation + +kitty-graphics.el is the first-ever native Emacs Lisp package for terminal +image display. Currently supports Kitty/WezTerm/Ghostty (~3 terminals). +Sixel support unlocks ~20 additional terminals including foot, Konsole, +xterm, mlterm, mintty, VS Code Terminal, **and tmux**. + +tmux is the killer use case — it's the only way to get images in tmux +from Emacs, since Kitty protocol doesn't work through tmux. + +## Key Architecture Decisions + +### 1. Backend Dispatch: Alist (Decision A) + +A backend is an alist of functions: +```elisp +((prepare . kitty-gfx--kitty-prepare) + (place . kitty-gfx--kitty-place) + (delete . kitty-gfx--kitty-delete) + (cleanup . kitty-gfx--kitty-cleanup)) +``` + +`kitty-gfx--backends` is an alist of `(SYMBOL . BACKEND-ALIST)`. +`kitty-gfx--active-backend` holds the active key ('kitty or 'sixel). +Dispatch: `(funcall (alist-get 'place (alist-get backend kitty-gfx--backends)) ...)` + +**Why**: ~2h more than if/else, but every future protocol (iTerm2, etc.) +is a 5-line alist entry instead of branches everywhere. + +### 2. Sixel Cache: Temp-Files (Decision B) + +Sixel-encoded data is stored in temp-files under `/tmp/kitty-gfx-sixel-*.six`. +Cache lookup: file-exists-p → insert-file-contents → send-string-to-terminal. + +**Why**: Eliminates GC pressure from multi-100KB Emacs strings. Disk I/O +is negligible vs. the ImageMagick shell-out that already happens. +Cache-size can stay at 64 entries. Cleanup via kill-emacs-hook. + +### 3. Refresh Strategy: Dirty-Flag (Decision A) + +Sixel overlays are only re-emitted when `last-row`/`last-col` has changed +OR an explicit dirty flag is set (resize, buffer-switch). Standing images +on a static screen = zero cost. + +**Why**: Minimal-invasive. One bool per overlay. The existing position +tracking provides the dirty signal for free. + +### 4. Encoding: Synchronous with Debounce (Decision B) + +Window resize triggers synchronous re-encode with 500ms debounce for +Sixel backend. User sees 1-2s freeze for 5+ visible images on resize. +Accepted tradeoff — resize is infrequent. + +**Why**: Async encoding (make-process + sentinel) adds race conditions +and ~6h complexity for a rare interaction. Ship first, optimize if +anyone complains. + +--- + +## Protocol Comparison + +``` + KITTY PROTOCOL SIXEL PROTOCOL + Statefulness: STATEFUL (transmit once, STATELESS (emit all data + place many times) every time) + Image Storage: TERMINAL-SIDE CLIENT-SIDE (temp-files) + Placement: NAMED (PID) — atomic replace ANONYMOUS — erase + re-emit + Delete: EXPLICIT (APC a=d) IMPLICIT (overwrite spaces) + Encoding: BASE64 PNG (lossless) 6-bit palette + RLE + Wire Format: APC (\e_G...\e\\) DCS (ESC P q ... ESC \) + Max Colors: Full RGBA (truecolor + alpha) 256 colors (no alpha) +``` + +--- + +## System Architecture — Target State + +``` + ┌─────────────────────────────────────────────────────────────┐ + │ INTEGRATION LAYER │ + │ org-mode │ image-mode │ shr │ doc-view │ dired │ eshell │ + │ (20+ :around advice functions — ALL UNCHANGED) │ + │ ALL call: kitty-gfx-display-image │ + └──────────────────────────┬──────────────────────────────────┘ + │ + ┌──────────────────────────▼──────────────────────────────────┐ + │ ORCHESTRATION LAYER │ + │ kitty-gfx-display-image │ + │ 1. convert-to-png (ImageMagick — shared) │ + │ 2. compute-cell-dims (shared) │ + │ 3. DISPATCH to active backend: │ + │ (funcall (kitty-gfx--backend-fn 'prepare) ...) │ + │ 4. make-overlay (shared) │ + │ 5. schedule-refresh │ + └──────────────────────────┬──────────────────────────────────┘ + │ + ┌──────────────────────────▼──────────────────────────────────┐ + │ BACKEND DISPATCH │ + │ │ + │ kitty-gfx--active-backend ──▶ 'kitty / 'sixel │ + │ │ + │ ┌─────────────────┐ ┌──────────────────────────┐ │ + │ │ KITTY BACKEND │ │ SIXEL BACKEND │ │ + │ │ │ │ │ │ + │ │ prepare: │ │ prepare: │ │ + │ │ transmit APC │ │ png→sixel encode │ │ + │ │ cache: id only │ │ cache: temp-file path │ │ + │ │ │ │ │ │ + │ │ place: │ │ place: │ │ + │ │ APC a=p (50B) │ │ read tmpfile → DCS Pq │ │ + │ │ │ │ (50-500KB) │ │ + │ │ │ │ │ │ + │ │ delete: │ │ delete: │ │ + │ │ APC a=d (30B) │ │ overwrite with spaces │ │ + │ │ │ │ │ │ + │ │ cleanup: │ │ cleanup: │ │ + │ │ delete-by-id │ │ delete temp-files │ │ + │ └─────────────────┘ └──────────────────────────┘ │ + └─────────────────────────────────────────────────────────────┘ +``` + +--- + +## Protocol-Specific Functions — Refactor Map + +Only 6 functions emit terminal-specific escape sequences today. +These become backend dispatch points: + +| Current Function | Backend Method | Kitty | Sixel | +|-------------------------------|---------------|----------------------|---------------------------| +| `kitty-gfx--transmit-image` | `prepare` | APC a=t chunked | magick → sixel tmpfile | +| `kitty-gfx--place-image` | `place` | APC a=p (50 bytes) | read tmpfile → DCS emit | +| `kitty-gfx--delete-placement` | `delete` | APC a=d,d=i | overwrite with spaces | +| `kitty-gfx--delete-by-id` | `cleanup` | APC a=d,d=I | delete temp-file | +| `kitty-gfx--delete-all-images`| `cleanup-all` | APC a=d,d=A | delete all temp-files | +| `kitty-gfx--supported-p` | (detection) | KITTY_PID/TERM check | DA1 query + env check | + +Everything else (~70% of codebase) is protocol-agnostic. + +--- + +## Performance Model + +``` + OPERATION | KITTY | SIXEL | RATIO + ---------------------------|------------------|--------------------|-------- + First display (cold) | ~200ms | ~300ms | 1.5x + Placement (warm cache) | ~0.1ms (50B) | ~5-50ms (50-500KB) | 50-500x + Refresh (no change) | ~0ms (skip) | ~0ms (dirty-flag) | 1x + Refresh (position change) | ~0.1ms | ~10-100ms | 100-1000x + Window resize (5 images) | ~0.5ms | ~1500ms | 3000x + Delete | ~0.1ms | ~1-5ms | 10-50x +``` + +Dirty-flag eliminates steady-state cost. Resize is debounced (500ms). + +--- + +## Error & Rescue Registry + +| Method | Error | Rescued | Action | User Sees | +|---------------------------|-----------------------|---------|-----------------------|---------------------| +| detect-protocol (DA1) | timeout | Y | env-var fallback | Transparent | +| detect-protocol (DA1) | parse-error | Y | env-var fallback | Transparent | +| sixel-encode | missing ImageMagick | Y | user-error | "Install ImageMagick"| +| sixel-encode | process-error | Y | skip image, log | Image missing | +| sixel-encode | empty output | Y | skip image, log | Image missing | +| sixel-place (DCS emit) | io-error | Y | ignore-errors | Image missing | +| sixel-cache-put | disk full | Y | condition-case | Fallback in-memory | +| backend-dispatch | missing backend key | Y | error with message | Clear error msg | +| sixel-delete (erase) | position stale | PARTIAL | sync-begin/end | Brief flicker | + +CRITICAL GAPS: 0 + +--- + +## Failure Modes Registry + +| Codepath | Failure Mode | Rescued? | Test? | User Sees? | Logged? | +|------------------|---------------------|----------|-------|-------------|---------| +| sixel-encode | magick not found | Y | #2 | user-error | Y | +| sixel-encode | corrupt PNG input | Y | — | skip image | Y | +| sixel-encode | huge output (>1MB) | Y | #16 | downscaled | Y | +| sixel-place | terminal overflow | PARTIAL | #6 | slow/hang | Y | +| sixel-cache | disk full | Y | — | fallback | Y | +| detect-protocol | DA1 timeout | Y | #5 | fallback | Y | +| detect-protocol | terminal lies | N | — | wrong proto | N | +| refresh (sixel) | rapid scroll 5 imgs | Y (flag) | #6 | skip re-emit| Y | +| resize (sixel) | 5 imgs re-encode | Y (dbnc) | #7 | 1-2s freeze | Y | + +--- + +## Rollout Phases + +### Phase 1: Backend Refactor (in-place) +- Introduce alist dispatch +- Extract existing Kitty code into 'kitty backend +- NO new functionality, only restructuring +- Byte-compile + load-test +- Commit: "refactor: extract protocol backend dispatch" + +### Phase 2: Sixel Backend +- kitty-gfx--sixel-encode, -place, -delete, -cleanup +- Temp-file cache system +- Dirty-flag in refresh cycle +- Commit: "feat: add Sixel graphics protocol backend" + +### Phase 3: Auto-Detection +- DA1 query + env-var fallback chain +- kitty-gfx-preferred-protocol defcustom ('auto / 'kitty / 'sixel) +- Commit: "feat: auto-detect terminal graphics protocol" + +### Phase 4: Polish +- Repo rename: kitty-graphics → terminal-graphics.el +- README update +- Version bump: 0.3.0 +- MELPA recipe submission + +--- + +## Test Matrix + +| # | What | Type | Spec | +|---|-------------------------------|--------|-----------------------------------------| +| 1 | Sixel encode happy path | Manual | PNG in org-buffer, foot terminal | +| 2 | Sixel encode failure | Manual | Remove ImageMagick, attempt display | +| 3 | Backend auto-detect Kitty | Manual | In Kitty: mode on → backend = 'kitty | +| 4 | Backend auto-detect Sixel | Manual | In foot: mode on → backend = 'sixel | +| 5 | Fallback when DA1 fails | Manual | Set TERM_PROGRAM manually, block stdin | +| 6 | Scroll performance (5 images) | Manual | Org file with 5 images, rapid scroll | +| 7 | Window resize | Manual | Display image, C-x 3 split → re-encode | +| 8 | Cache hit | Manual | Toggle images off/on → faster 2nd time | +| 9 | Cache cleanup on kill-emacs | Manual | Exit Emacs, check /tmp → files gone | +| 10| tmux + Sixel | Manual | tmux (--enable-sixel) + foot | +| 11| Kitty regression | Manual | Kitty: +/-/0 zoom still works | +| 12| Sixel image-mode zoom | Manual | foot: +/-/0 zoom → re-encodes | +| 13| doc-view in Sixel | Manual | foot: open PDF → pages render | +| 14| shr/eww in Sixel | Manual | foot: eww with images | +| 15| defcustom override | Manual | Force 'sixel in Kitty → uses Sixel | +| 16| Large image DoS protection | Manual | 4K image → capped, no terminal hang | + +--- + +## NOT in Scope + +| Item | Rationale | +|------------------------------|--------------------------------------------------| +| iTerm2 Protocol Backend | Trivial after Sixel (~2h), separate PR | +| tmux Kitty Passthrough | Separate feature, separate PR | +| Async Sixel Encoding | Only if users complain about resize freeze | +| OSC 66 Text Sizing Ship | Already designed in AGENTS.md, separate feature | +| GIF/Animation Support | Out of scope for this iteration | +| In-Emacs Sixel Encoder | Would eliminate ImageMagick dep, ~40h, not now | +| comint-mime Integration | Issue #5, separate feature | + +--- + +## Delight Opportunities (post-ship) + +1. **tmux Kitty passthrough** — DCS wrapper, ~1h after backend abstraction +2. **Auto-detect with modeline indicator** — `[K]` or `[S]` in lighter +3. **kitty-gfx-preferred-protocol defcustom** — user override for power users +4. **Sixel color palette size defcustom** — 256 vs 1024, speed vs quality +5. **iTerm2 backend** — ~2h, covers macOS iTerm2 users + +--- + +## Naming Strategy + +- Emacs symbol prefix: keep `kitty-gfx-` (breaking change otherwise) +- Repository name: rename `kitty-graphics` → `terminal-graphics.el` +- GitHub rename preserves all stars, forks, issues, redirects old URLs +- MELPA package name: `terminal-graphics` (or keep `kitty-graphics`) +- Explain in Commentary section why prefix differs from repo name + +--- + +## GitHub Rename Notes + +- Stars, forks, issues ALL preserved on rename +- Old URLs auto-redirect to new name +- git clone/fetch/push from old URL continues to work +- CAVEAT: "do not reuse the original name" — if you create a NEW repo + called `kitty-graphics` after renaming, redirects break +- Recommendation: simple rename, no new-repo trick needed @@ -1,7 +1,7 @@ #+TITLE: kitty-graphics.el #+AUTHOR: cashmere -Display images in terminal Emacs (~emacs -nw~) using the [[https://sw.kovidgoyal.net/kitty/graphics-protocol/][Kitty graphics protocol]]. +Display images in terminal Emacs (~emacs -nw~) using the [[https://sw.kovidgoyal.net/kitty/graphics-protocol/][Kitty graphics protocol]] or [[https://en.wikipedia.org/wiki/Sixel][Sixel]]. * Demo @@ -17,11 +17,16 @@ Besides, I don't even have the guts to release it, even though my intention of s * Overview -~kitty-graphics.el~ renders images directly in terminal Emacs using Kitty's -graphics protocol with direct placements. Images are transmitted once to the -terminal, then positioned at overlay screen coordinates after each redisplay. -They scroll with text, survive buffer switches, and work in split windows --- -they're managed as overlays in Emacs's display engine. +~kitty-graphics.el~ renders images directly in terminal Emacs using either the +Kitty graphics protocol or Sixel, selected automatically at startup. The backend +dispatch tries Kitty first (fast ~KITTY_PID~ env check), then probes for Sixel +support, so the package works across ~20+ terminals without configuration. + +With the Kitty backend, images are transmitted once and positioned via direct +placements at overlay screen coordinates after each redisplay. The Sixel backend +encodes images via ImageMagick and re-emits them on scroll. Both backends +integrate with Emacs's overlay display engine --- images scroll with text, +survive buffer switches, and work in split windows. Integrations are provided for: - *org-mode* --- inline images via =C-c C-x C-v= (both =org-toggle-inline-images= and =org-link-preview= for org 10.0+) @@ -35,8 +40,10 @@ Integrations are provided for: * Requirements - Emacs >= 27.1 -- [[https://sw.kovidgoyal.net/kitty/][Kitty]] >= 0.20.0 (or WezTerm/Ghostty with direct placement support) -- ImageMagick (~magick~ / ~convert~ / ~identify~) for non-PNG formats and image sizing +- A supported terminal: + - *Kitty backend*: [[https://sw.kovidgoyal.net/kitty/][Kitty]] >= 0.20.0, WezTerm, or Ghostty + - *Sixel backend*: foot, Konsole, xterm (with ~+sixel~), mlterm, mintty, VS Code Terminal, and others with Sixel support +- ImageMagick (~magick~ / ~convert~ / ~identify~) --- required for Sixel, and for non-PNG formats on Kitty - For LaTeX preview: a TeX distribution with ~dvipng~ (e.g. ~texlive~) - For doc-view: ~ghostscript~ (for PDF), ~dvipng~ (for DVI) --- same tools as GUI Emacs - Launch Emacs with ~TERM=xterm-256color~ (Emacs often can't find the ~xterm-kitty~ terminfo) @@ -52,7 +59,7 @@ Integrations are provided for: #+begin_src emacs-lisp (use-package kitty-graphics :straight (:local-repo "~/projects/kitty-graphics") - :if (and (not (display-graphic-p)) (getenv "KITTY_PID")) + :if (not (display-graphic-p)) :config (kitty-graphics-mode 1)) #+end_src @@ -62,7 +69,7 @@ Integrations are provided for: #+begin_src emacs-lisp (add-to-list 'load-path "~/projects/kitty-graphics") (require 'kitty-graphics) -(when (and (not (display-graphic-p)) (getenv "KITTY_PID")) +(when (not (display-graphic-p)) (kitty-graphics-mode 1)) #+end_src @@ -95,28 +102,33 @@ Then: ** Customization -| Variable | Default | Description | -|------------------------+---------+------------------------------------------| -| ~kitty-gfx-max-width~ | 120 | Maximum inline image width in columns | -| ~kitty-gfx-max-height~ | 40 | Maximum inline image height in rows | -| ~kitty-gfx-render-delay~ | 0.016 | Debounce delay for re-rendering (s) | -| ~kitty-gfx-debug~ | nil | Log debug info to ~/tmp/kitty-gfx.log~ | +| Variable | Default | Description | +|--------------------------------+---------+------------------------------------------| +| ~kitty-gfx-preferred-protocol~ | auto | Graphics protocol: auto, kitty, or sixel | +| ~kitty-gfx-max-width~ | 120 | Maximum inline image width in columns | +| ~kitty-gfx-max-height~ | 40 | Maximum inline image height in rows | +| ~kitty-gfx-render-delay~ | 0.016 | Debounce delay for re-rendering (s) | +| ~kitty-gfx-debug~ | nil | Log debug info to ~/tmp/kitty-gfx.log~ | These defaults control inline image sizing (org-mode, eww, dired previews). Doc-view ignores these and fills the window, with =+= / =-= / =0= for zoom. * How It Works -1. On startup, cell pixel size is queried via ~CSI 16 t~ (XTWINOPS) for - accurate image scaling. Falls back to 8x16 if the query times out. -2. Image data is transmitted once via APC escape sequences (~a=t~, store only) -3. Overlays with a ~display~ property reserve blank space in Emacs buffers -4. After each redisplay, direct placements (~a=p~) are emitted via - ~send-string-to-terminal~ at the overlay's screen coordinates -5. All placement output is wrapped in synchronized output (~BSU/ESU~, - =DEC mode 2026=) to prevent partial rendering and flicker -6. Each placement uses a unique placement ID (~p=PID~) so re-placements - at new positions replace the old rendering without accumulation +1. On startup, backend dispatch selects a protocol: if ~kitty-gfx-preferred-protocol~ + is ~auto~, it checks for Kitty (via ~KITTY_PID~), then probes Sixel via a DA1 + query. The mode-line shows ~[K]~, ~[S]~, or ~[?]~ to indicate the active backend. +2. Cell pixel size is queried via ~CSI 16 t~ (XTWINOPS) for accurate image + scaling. Falls back to 8x16 if the query times out. +3. *Kitty path*: image data is transmitted once via APC escape sequences (~a=t~, + store only). Direct placements (~a=p~) position the image at overlay screen + coordinates after each redisplay, using unique placement IDs (~p=PID~). +4. *Sixel path*: images are encoded to Sixel via ImageMagick and emitted + directly at overlay screen coordinates. Since Sixel is stateless, images + are re-emitted whenever they scroll to a new position. +5. Overlays with a ~display~ property reserve blank space in Emacs buffers +6. All output is wrapped in synchronized output (~BSU/ESU~, =DEC mode 2026=) + to prevent partial rendering and flicker 7. Position caching skips redundant re-placements when nothing moved 8. Placements are deleted when overlays scroll out of view @@ -127,10 +139,10 @@ TIFF, BMP) are converted to PNG via ImageMagick before transmission. * Known Limitations -- No tmux support -- Primary focus on kitty terminal. While other terminals should work too, I wont give any guarantee +- No tmux support (neither Kitty passthrough nor Sixel) - Animated GIFs display only the first frame -- Without ImageMagick, only PNG files work and image sizing falls back to a fixed default +- Sixel is limited to 256 colors (vs truecolor on Kitty) and is stateless --- images are re-emitted on every scroll, which may be slower on large images +- Without ImageMagick, only PNG files work on the Kitty backend; the Sixel backend requires ImageMagick unconditionally - Doc-view page may briefly flash at wrong position before centering on initial load * License diff --git a/kitty-graphics.el b/kitty-graphics.el index 098de48..84f5cce 100644 --- a/kitty-graphics.el +++ b/kitty-graphics.el @@ -3,7 +3,7 @@ ;; Copyright (C) 2025-2026 ;; ;; Author: cashmere -;; Version: 0.2.1 +;; Version: 0.3.0 ;; URL: https://github.com/cashmeredev/kitty-graphics.el ;; Keywords: terminals, images, multimedia ;; Package-Requires: ((emacs "27.1")) @@ -104,11 +104,21 @@ This debounces rapid redisplay events. Default is ~1 frame at 60fps." "File path for debug log output.") (defun kitty-gfx--log (fmt &rest args) - "Log to `kitty-gfx--log-file' when `kitty-gfx-debug' is non-nil." + "Log to `kitty-gfx--log-file' and *kitty-gfx-debug* buffer when debug is on." (when kitty-gfx-debug (let ((msg (concat (format-time-string "%H:%M:%S.%3N ") (apply #'format fmt args) "\n"))) - (ignore-errors (append-to-file msg nil kitty-gfx--log-file))))) + (ignore-errors (append-to-file msg nil kitty-gfx--log-file)) + (ignore-errors + (let ((buf (get-buffer-create "*kitty-gfx-debug*"))) + (with-current-buffer buf + (goto-char (point-max)) + (insert msg) + ;; Trim buffer at 2500 lines + (when (> (line-number-at-pos (point-max)) 2500) + (goto-char (point-min)) + (forward-line 500) + (delete-region (point-min) (point))))))))) ;;;; Constants — kept for reference if switching back to Unicode placeholders ;; (defconst kitty-gfx--placeholder-char #x10EEEE) @@ -119,6 +129,20 @@ This debounces rapid redisplay events. Default is ~1 frame at 60fps." ;; Forward declaration — defined by `define-minor-mode' below. (defvar kitty-graphics-mode) +(defvar kitty-gfx--active-backend nil + "Symbol identifying the active graphics backend: `kitty' or `sixel'. +Set by `kitty-gfx--detect-protocol'.") + +(defvar kitty-gfx--backends nil + "Alist mapping backend symbols to operation alists. +Each backend alist maps operation symbols to functions: + `detect' — () -> bool: return non-nil if backend is supported + `prepare' — (file image-id) -> id-or-nil: prepare/transmit image + `place' — (ov id pid cols rows term-row term-col): place image + `delete' — (ov id pid): delete placement + `cleanup' — (file id): cleanup resources for file + `cleanup-all' — (): cleanup all resources.") + (defvar kitty-gfx--next-id 1 "Next image ID to assign (1-4294967295). With direct placements, any uint32 ID works — no 256-color constraint.") @@ -130,6 +154,15 @@ terminal data deleted via `a=d'." :type 'integer :group 'kitty-graphics) +(defcustom kitty-gfx-preferred-protocol 'auto + "Preferred graphics protocol to use. +Choices: `auto' (try Kitty first, then Sixel), `kitty', or `sixel'. +Default is `auto'." + :type '(choice (const :tag "Auto-detect (Kitty → Sixel)" auto) + (const :tag "Kitty graphics protocol" kitty) + (const :tag "Sixel protocol" sixel)) + :group 'kitty-graphics) + (defvar kitty-gfx--image-cache (make-hash-table :test 'equal) "Maps file paths to image IDs (integers). Only stores the terminal-side image ID — display dimensions are @@ -161,17 +194,52 @@ rather than accumulate.") ;;;; Terminal detection -(defun kitty-gfx--supported-p () - "Return non-nil if the terminal supports Kitty graphics." - (let ((supported (and (not (display-graphic-p)) - (or (getenv "KITTY_PID") - (equal (getenv "TERM_PROGRAM") "kitty") - (equal (getenv "TERM_PROGRAM") "WezTerm") - (equal (getenv "TERM_PROGRAM") "ghostty"))))) - (kitty-gfx--log "supported-p: %s (graphic=%s KITTY_PID=%s TERM_PROGRAM=%s)" - supported (display-graphic-p) - (getenv "KITTY_PID") (getenv "TERM_PROGRAM")) - supported)) +(defun kitty-gfx--backend-fn (op) + "Return the backend function for operation OP. +Looks up OP in the active backend's operation alist. +Signals an error if no backend is active or OP is missing." + (unless kitty-gfx--active-backend + (error "No active graphics backend")) + (let* ((backend-alist (alist-get kitty-gfx--active-backend kitty-gfx--backends)) + (fn (alist-get op backend-alist))) + (unless fn + (error "Backend %s does not implement operation %s" + kitty-gfx--active-backend op)) + fn)) + +(defun kitty-gfx--detect-protocol () + "Detect and activate a graphics protocol backend. +Returns non-nil if a supported backend is found. +Sets `kitty-gfx--active-backend' to the detected backend symbol." + (if (display-graphic-p) + (progn + (kitty-gfx--log "detect-protocol: GUI frame, no terminal graphics") + (setq kitty-gfx--active-backend nil) + nil) + (let ((pref kitty-gfx-preferred-protocol) + (detected nil)) + (kitty-gfx--log "detect-protocol: preference=%s" pref) + (cond + ;; Explicit backend preference + ((eq pref 'kitty) + (let ((fn (alist-get 'detect (alist-get 'kitty kitty-gfx--backends)))) + (when (and fn (funcall fn)) + (setq detected 'kitty)))) + ((eq pref 'sixel) + (let ((fn (alist-get 'detect (alist-get 'sixel kitty-gfx--backends)))) + (when (and fn (funcall fn)) + (setq detected 'sixel)))) + ;; Auto: try Kitty first (fast env check), then Sixel + (t + (let ((kitty-fn (alist-get 'detect (alist-get 'kitty kitty-gfx--backends)))) + (if (and kitty-fn (funcall kitty-fn)) + (setq detected 'kitty) + (let ((sixel-fn (alist-get 'detect (alist-get 'sixel kitty-gfx--backends)))) + (when (and sixel-fn (funcall sixel-fn)) + (setq detected 'sixel))))))) + (setq kitty-gfx--active-backend detected) + (kitty-gfx--log "detect-protocol: result=%s" detected) + detected))) (defun kitty-gfx--query-cell-size () "Query terminal for cell size in pixels using CSI 16 t (XTWINOPS). @@ -290,6 +358,218 @@ Uses direct placement: move cursor, then `a=p' with `c' and `r' params." (format "\e7\e[%d;%dH\e_Gq=2,a=p,i=%d,p=%d,c=%d,r=%d\e\\\e8" term-row term-col image-id placement-id cols rows)))) +;;;; Kitty backend + +(defun kitty-gfx--kitty-detect () + "Return non-nil if the terminal supports Kitty graphics protocol." + (let ((supported (or (getenv "KITTY_PID") + (equal (getenv "TERM_PROGRAM") "kitty") + (equal (getenv "TERM_PROGRAM") "WezTerm") + (equal (getenv "TERM_PROGRAM") "ghostty")))) + (kitty-gfx--log "kitty-detect: %s (KITTY_PID=%s TERM_PROGRAM=%s)" + supported (getenv "KITTY_PID") (getenv "TERM_PROGRAM")) + supported)) + +(defun kitty-gfx--kitty-prepare (file image-id) + "Prepare image FILE for Kitty display. +Converts to PNG if needed, encodes to base64, transmits to terminal. +Returns IMAGE-ID on success, nil on failure." + (let* ((png (kitty-gfx--convert-to-png file)) + (temp-p (and png (not (string= png file))))) + (unwind-protect + (let ((b64 (when png (kitty-gfx--read-file-base64 png)))) + (if (not b64) + (progn + (kitty-gfx--log "kitty-prepare: skipped %s (conversion failed)" file) + nil) + (kitty-gfx--log "kitty-prepare: transmit id=%d b64-len=%d png=%s" + image-id (length b64) png) + (kitty-gfx--transmit-image image-id b64) + image-id)) + (when temp-p + (ignore-errors (delete-file png)))))) + +(defun kitty-gfx--kitty-place (_ov image-id placement-id cols rows term-row term-col) + "Place Kitty image at terminal position. +OV is the overlay (unused by Kitty backend but part of the interface)." + (kitty-gfx--place-image image-id placement-id cols rows term-row term-col)) + +(defun kitty-gfx--kitty-delete (_ov image-id placement-id) + "Delete Kitty placement PID of image ID." + (kitty-gfx--delete-placement image-id placement-id)) + +(defun kitty-gfx--kitty-cleanup (_file image-id) + "Cleanup Kitty image data for FILE (identified by IMAGE-ID)." + (kitty-gfx--delete-by-id image-id)) + +(defun kitty-gfx--kitty-cleanup-all () + "Cleanup all Kitty images." + (kitty-gfx--delete-all-images)) + +;;;; Sixel backend + +(defvar kitty-gfx--sixel-temp-files nil + "List of temporary Sixel files created for caching.") + +(defvar kitty-gfx--sixel-cache (make-hash-table :test 'equal) + "Maps (file . dims-string) to temp sixel file paths.") + +(defun kitty-gfx--sixel-detect () + "Return non-nil if the terminal likely supports Sixel protocol." + (let* ((term (getenv "TERM")) + (term-prog (getenv "TERM_PROGRAM")) + (in-tmux (getenv "TMUX")) + (supported (and term + (not in-tmux) ; Sixel doesn't work in tmux + (or (string-match-p "xterm\\|vt[0-9]" term) + (member term-prog '("foot" "Konsole" "mintty" "mlterm"))) + t))) + (kitty-gfx--log "sixel-detect: %s (TERM=%s TERM_PROGRAM=%s TMUX=%s)" + supported term term-prog (if in-tmux "yes" "no")) + supported)) + +(defun kitty-gfx--sixel-cache-path (file cols rows) + "Return deterministic temp file path for FILE at COLS x ROWS." + (let* ((key (format "%s:%dx%d" file cols rows)) + (hash (md5 key))) + (expand-file-name (concat "kitty-gfx-sixel-" hash ".six") temporary-file-directory))) + +(defun kitty-gfx--sixel-encode (png-file cols rows) + "Encode PNG-FILE as Sixel data for COLS x ROWS cells. +Returns Sixel data string or nil on failure. +Computes pixel dimensions from cell size." + (let* ((cw (or kitty-gfx--cell-pixel-width 8)) + (ch (or kitty-gfx--cell-pixel-height 16)) + (pixel-w (* cols cw)) + (pixel-h (* rows ch)) + (convert (or (executable-find "magick") (executable-find "convert")))) + (if (not convert) + (progn + (kitty-gfx--log "sixel-encode: no ImageMagick, cannot encode") + (message "kitty-gfx: Sixel backend requires ImageMagick") + nil) + (when (string-suffix-p "convert" convert) + (kitty-gfx--log "sixel-encode: WARNING deprecated `convert' binary resolved: %s (use `magick' instead)" convert)) + (kitty-gfx--log "sixel-encode: %s -> %dx%d pixels via %s" + png-file pixel-w pixel-h convert) + (with-temp-buffer + (set-buffer-multibyte nil) + (let* ((args (list convert nil '(t nil) nil + png-file "-geometry" (format "%dx%d" pixel-w pixel-h) + "sixel:-")) + (exit-code (apply #'call-process args))) + (if (zerop exit-code) + (let ((data (buffer-string))) + (kitty-gfx--log "sixel-encode: success (%d bytes)" (length data)) + data) + (kitty-gfx--log "sixel-encode: FAILED exit=%d" exit-code) + nil)))))) + +(defun kitty-gfx--sixel-prepare (file _image-id) + "Prepare FILE for Sixel display. +For Sixel, preparation just validates the file exists and is convertible. +Actual encoding happens at place-time (needs dimensions). +Returns non-nil on success." + (let ((png (kitty-gfx--convert-to-png file))) + (when png + (kitty-gfx--log "sixel-prepare: %s -> %s" file png) + ;; Cache the PNG path for later encoding + (puthash file png kitty-gfx--sixel-cache) + t))) + +(defun kitty-gfx--sixel-place (ov _image-id _placement-id cols rows term-row term-col) + "Place Sixel image at terminal position. +Encodes on-demand if not cached, then emits DCS sequence." + (let* ((file (overlay-get ov 'kitty-gfx-file)) + (png (gethash file kitty-gfx--sixel-cache)) + (cache-path (kitty-gfx--sixel-cache-path file cols rows)) + (sixel-data nil)) + (if (not png) + (kitty-gfx--log "sixel-place: no PNG cached for %s" file) + ;; Check if sixel encoding is cached + (if (file-exists-p cache-path) + (progn + (kitty-gfx--log "sixel-place: using cached sixel %s" cache-path) + (with-temp-buffer + (set-buffer-multibyte nil) + (insert-file-contents-literally cache-path) + (setq sixel-data (buffer-string)))) + ;; Encode on-demand + (kitty-gfx--log "sixel-place: encoding %s at %dx%d" png cols rows) + (setq sixel-data (kitty-gfx--sixel-encode png cols rows)) + (when sixel-data + ;; Cache the encoding + (ignore-errors + (with-temp-file cache-path + (set-buffer-multibyte nil) + (insert sixel-data))) + (push cache-path kitty-gfx--sixel-temp-files))) + ;; Emit Sixel sequence if we have data + (when sixel-data + (let* ((cw (or kitty-gfx--cell-pixel-width 8)) + (ch (or kitty-gfx--cell-pixel-height 16))) + (kitty-gfx--log "sixel-place: emitting at row=%d col=%d data-len=%d pixel-target=%dx%d" + term-row term-col (length sixel-data) (* cols cw) (* rows ch))) + (ignore-errors + (send-string-to-terminal + (format "\e7\e[%d;%dH%s\e8" term-row term-col sixel-data))))))) + +(defun kitty-gfx--sixel-delete (ov _image-id _placement-id) + "Delete Sixel placement by overwriting with spaces. +Sixel has no placement IDs — erase by writing spaces over the region." + (let ((last-row (overlay-get ov 'kitty-gfx-last-row)) + (last-col (overlay-get ov 'kitty-gfx-last-col)) + (rows (overlay-get ov 'kitty-gfx-rows)) + (cols (overlay-get ov 'kitty-gfx-cols))) + (when (and last-row last-col rows cols) + (kitty-gfx--log "sixel-delete: erase row=%d col=%d %dx%d" + last-row last-col cols rows) + (ignore-errors + (send-string-to-terminal + (format "\e7%s\e8" + (mapconcat + (lambda (r) + (format "\e[%d;%dH%s" (+ last-row r) last-col (make-string cols ?\s))) + (number-sequence 0 (1- rows)) + ""))))))) + +(defun kitty-gfx--sixel-cleanup (file _image-id) + "Cleanup Sixel resources for FILE." + (remhash file kitty-gfx--sixel-cache) + ;; Remove cached sixel encodings for this file + (dolist (temp-file kitty-gfx--sixel-temp-files) + (when (string-match-p (regexp-quote (md5 file)) temp-file) + (kitty-gfx--log "sixel-cleanup: deleting %s" temp-file) + (ignore-errors (delete-file temp-file)) + (setq kitty-gfx--sixel-temp-files (delete temp-file kitty-gfx--sixel-temp-files))))) + +(defun kitty-gfx--sixel-cleanup-all () + "Cleanup all Sixel resources." + (kitty-gfx--log "sixel-cleanup-all: deleting %d temp files" + (length kitty-gfx--sixel-temp-files)) + (dolist (temp-file kitty-gfx--sixel-temp-files) + (ignore-errors (delete-file temp-file))) + (setq kitty-gfx--sixel-temp-files nil) + (clrhash kitty-gfx--sixel-cache)) + +;; Register backends +(setq kitty-gfx--backends + `((kitty . ((detect . ,#'kitty-gfx--kitty-detect) + (prepare . ,#'kitty-gfx--kitty-prepare) + (place . ,#'kitty-gfx--kitty-place) + (delete . ,#'kitty-gfx--kitty-delete) + (cleanup . ,#'kitty-gfx--kitty-cleanup) + (cleanup-all . ,#'kitty-gfx--kitty-cleanup-all))) + (sixel . ((detect . ,#'kitty-gfx--sixel-detect) + (prepare . ,#'kitty-gfx--sixel-prepare) + (place . ,#'kitty-gfx--sixel-place) + (delete . ,#'kitty-gfx--sixel-delete) + (cleanup . ,#'kitty-gfx--sixel-cleanup) + (cleanup-all . ,#'kitty-gfx--sixel-cleanup-all))))) + +;; Cleanup temp files on exit +(add-hook 'kill-emacs-hook #'kitty-gfx--sixel-cleanup-all) + ;;;; Position mapping (defun kitty-gfx--in-folded-region-p (pos) @@ -334,31 +614,23 @@ the window's scroll range." ;; (outline, org-fold, etc.) but allow cosmetic invisibility ;; like org-link bracket hiding. (not (kitty-gfx--in-folded-region-p pos))) - (let* ((edges (window-edges win)) - (win-top (nth 1 edges)) - (win-left (nth 0 edges)) + ;; posn-col-row returns coordinates relative to the window BODY + ;; (text area). Use window-body-edges to convert to frame coords. + ;; body-left accounts for margins/fringes; body-top accounts for + ;; header-line. +1 converts 0-based frame coords to 1-based terminal. + (let* ((body (window-body-edges win)) + (body-left (nth 0 body)) + (body-top (nth 1 body)) (win-pos (posn-at-point pos win))) (when win-pos (let* ((col-row (posn-col-row win-pos)) (row (cdr col-row)) (posn-col (car col-row)) - (body-left (nth 0 (window-body-edges win))) - (buf-col (with-current-buffer buf - (save-excursion - (goto-char pos) - (current-column)))) - ;; posn-col-row is correct for most overlays (includes - ;; line numbers, margins). But overlays with display - ;; properties wider than the underlying text inflate - ;; posn-col. Detect this: if posn-col minus buf-col - ;; exceeds a reasonable gutter width, it's inflated — - ;; fall back to body-left + buf-col. - (col (if (> (- posn-col buf-col) - (- body-left win-left)) - (+ body-left buf-col) - (+ win-left posn-col)))) + (posn-xy (posn-x-y win-pos))) + (kitty-gfx--log "screen-pos-detail: pid=%s posn-col=%d posn-row=%d posn-xy=%S body-left=%d body-top=%d" + (overlay-get ov 'kitty-gfx-pid) posn-col row posn-xy body-left body-top) (when col-row - (let ((result (cons (+ win-top row 1) (1+ col)))) + (let ((result (cons (+ body-top row 1) (+ body-left posn-col 1)))) (kitty-gfx--log "screen-pos: pid=%s pos=%d -> row=%d col=%d" (overlay-get ov 'kitty-gfx-pid) pos (car result) (cdr result)) @@ -375,6 +647,10 @@ Deletes placements for overlays that scrolled out of view. All terminal output is wrapped in synchronized output (BSU/ESU) to prevent flicker." (when (and kitty-graphics-mode (not (display-graphic-p))) + ;; Force redisplay so posn-at-point sees up-to-date pixel positions + ;; after display property changes (e.g., org-toggle-inline-images + ;; creating multi-line blank overlays). + (redisplay t) ;; Re-query cell size if invalidated (e.g., after terminal resize) (unless (and kitty-gfx--cell-pixel-width kitty-gfx--cell-pixel-height) (kitty-gfx--query-cell-size)) @@ -443,16 +719,15 @@ if the overlay scrolled out of view." new-row new-col) (overlay-put ov 'kitty-gfx-last-row new-row) (overlay-put ov 'kitty-gfx-last-col new-col) - (kitty-gfx--place-image id pid - (overlay-get ov 'kitty-gfx-cols) - rows new-row new-col))) + (funcall (kitty-gfx--backend-fn 'place) + ov id pid (overlay-get ov 'kitty-gfx-cols) rows new-row new-col))) ;; Not visible or overflows — delete if was placed (when last-row (kitty-gfx--log "refresh-ov: pid=%d hiding (was row=%d col=%d)" pid last-row last-col) (overlay-put ov 'kitty-gfx-last-row nil) (overlay-put ov 'kitty-gfx-last-col nil) - (kitty-gfx--delete-placement id pid)))))) + (funcall (kitty-gfx--backend-fn 'delete) ov id pid)))))) (defvar kitty-gfx--refresh-pending nil "Non-nil if a refresh was requested during the cooldown period.") @@ -494,6 +769,8 @@ then invalidates position caches and schedules a refresh." (let ((visible-bufs nil)) (walk-windows (lambda (w) (push (window-buffer w) visible-bufs)) nil 'visible) + (kitty-gfx--log "on-buffer-change: visible-bufs=(%s)" + (mapconcat #'buffer-name visible-bufs ", ")) ;; Delete placements for buffers that are no longer in any window (dolist (buf (buffer-list)) (with-current-buffer buf @@ -505,8 +782,10 @@ then invalidates position caches and schedules a refresh." (when (overlay-buffer ov) (let ((id (overlay-get ov 'kitty-gfx-id)) (pid (overlay-get ov 'kitty-gfx-pid))) - (when (and id pid (overlay-get ov 'kitty-gfx-last-row)) - (kitty-gfx--delete-placement id pid))) + (if (overlay-get ov 'kitty-gfx-last-row) + (when (and id pid) + (kitty-gfx--delete-placement id pid)) + (kitty-gfx--log "on-buffer-change: ov pid=%s has no last-row (never placed or already cleaned)" pid))) (overlay-put ov 'kitty-gfx-last-row nil) (overlay-put ov 'kitty-gfx-last-col nil))))))) ;; Reset cache for visible buffers so they re-place correctly. @@ -576,14 +855,16 @@ buffer before settling to one)." (defun kitty-gfx--image-pixel-size (file) "Return (WIDTH . HEIGHT) in pixels for image FILE, or nil." - (let ((identify (or (executable-find "identify") - (executable-find "magick")))) + (let ((identify (or (executable-find "magick") + (executable-find "identify")))) (when identify + (when (string-suffix-p "identify" identify) + (kitty-gfx--log "image-pixel-size: WARNING deprecated `identify' binary resolved: %s (use `magick' instead)" identify)) (with-temp-buffer (let ((args (if (string-suffix-p "magick" identify) - (list identify nil t nil "identify" "-format" "%w %h" + (list identify nil '(t nil) nil "identify" "-format" "%w %h" (concat file "[0]")) ; first frame only - (list identify nil t nil "-format" "%w %h" + (list identify nil '(t nil) nil "-format" "%w %h" (concat file "[0]"))))) (let ((exit-code (apply #'call-process args))) (kitty-gfx--log "identify: exit=%d output=%S" exit-code (buffer-string)) @@ -615,9 +896,7 @@ conversion fails — callers must handle nil gracefully." (let ((out (make-temp-file "kitty-gfx-" nil ".png"))) (kitty-gfx--log "convert-to-png: %s -> %s via %s" file out convert) (let ((exit-code - (if (string-suffix-p "magick" convert) - (call-process convert nil nil nil "convert" file out) - (call-process convert nil nil nil file out)))) + (call-process convert nil nil nil file out))) (kitty-gfx--log "convert-to-png: exit-code=%s" exit-code) ;; Check that conversion produced a non-empty file (if (and (file-exists-p out) @@ -677,8 +956,8 @@ With direct placements, COLS and ROWS map directly to terminal columns/rows." kitty-gfx--cache-lru) (let* ((victim (car (last kitty-gfx--cache-lru))) (victim-id (gethash victim kitty-gfx--image-cache))) - (when victim-id - (kitty-gfx--delete-by-id victim-id)) + (when (and victim-id kitty-gfx--active-backend) + (funcall (kitty-gfx--backend-fn 'cleanup) victim victim-id)) (remhash victim kitty-gfx--image-cache) (setq kitty-gfx--cache-lru (butlast kitty-gfx--cache-lru)) (kitty-gfx--log "cache-evict: %s id=%s (remaining=%d)" @@ -709,8 +988,9 @@ underline/color from bleeding through the overlay." (mapconcat (lambda (_) (propertize (make-string cols ?\s) 'face 'default)) (number-sequence 1 rows) "\n")) -(defun kitty-gfx--make-overlay (beg end image-id cols rows &optional reuse-pid) +(defun kitty-gfx--make-overlay (beg end image-id cols rows file &optional reuse-pid) "Create overlay from BEG to END for image IMAGE-ID (COLS x ROWS). +FILE is the source file path (needed by some backends for re-encoding). The overlay's display property shows blank space that the terminal fills with the image via direct placement. When REUSE-PID is non-nil, reuse that placement ID instead of @@ -727,6 +1007,7 @@ delete step, avoiding visual glitches in some terminals." (overlay-put ov 'kitty-gfx-pid pid) (overlay-put ov 'kitty-gfx-cols cols) (overlay-put ov 'kitty-gfx-rows rows) + (overlay-put ov 'kitty-gfx-file file) ;; Don't set evaporate — zero-width overlays (beg==end) would be ;; deleted immediately if evaporate is set. (push ov kitty-gfx--overlays) @@ -756,8 +1037,8 @@ visual glitches from delete+re-place sequences in some terminals)." (when (overlay-buffer ov) (unless keep-placement (condition-case err - (when (and id pid) - (kitty-gfx--delete-placement id pid)) + (when (and id pid kitty-gfx--active-backend) + (funcall (kitty-gfx--backend-fn 'delete) ov id pid)) (error (kitty-gfx--log "remove-overlay: error deleting placement: %s" (error-message-string err))))) @@ -772,8 +1053,8 @@ visual glitches from delete+re-place sequences in some terminals)." "Display image FILE in the current buffer. BEG/END span the overlay region. MAX-COLS/MAX-ROWS limit size." (interactive "fImage file: ") - (unless (kitty-gfx--supported-p) - (user-error "Terminal does not support Kitty graphics")) + (unless kitty-gfx--active-backend + (user-error "Terminal does not support graphics")) (let* ((max-c (or max-cols kitty-gfx-max-width)) (max-r (or max-rows kitty-gfx-max-height)) (abs-file (expand-file-name file)) @@ -792,22 +1073,13 @@ BEG/END span the overlay region. MAX-COLS/MAX-ROWS limit size." (stop (or end (point)))) (kitty-gfx--log "display-image: file=%s id=%d cols=%d rows=%d beg=%s end=%s cached=%s" abs-file image-id cols rows start stop (if cached-id "yes" "no")) - ;; Transmit image if not cached + ;; Prepare image if not cached (backend-specific: transmit or validate) (unless cached-id - (let* ((png (kitty-gfx--convert-to-png abs-file)) - (temp-p (and png (not (string= png abs-file))))) - (unwind-protect - (let ((b64 (when png (kitty-gfx--read-file-base64 png)))) - (if (not b64) - (kitty-gfx--log "display-image: skipped %s (conversion failed)" abs-file) - (kitty-gfx--log "transmit: id=%d b64-len=%d png=%s" image-id (length b64) png) - (kitty-gfx--transmit-image image-id b64) - (kitty-gfx--cache-put abs-file image-id))) - (when temp-p - (ignore-errors (delete-file png)))))) + (when (funcall (kitty-gfx--backend-fn 'prepare) abs-file image-id) + (kitty-gfx--cache-put abs-file image-id))) ;; Create overlay with blank space (even for cached images, dims are fresh) (when (or cached-id (gethash abs-file kitty-gfx--image-cache)) - (kitty-gfx--make-overlay start stop image-id cols rows) + (kitty-gfx--make-overlay start stop image-id cols rows abs-file) ;; Schedule initial render (kitty-gfx--schedule-refresh)))) @@ -850,19 +1122,11 @@ The buffer should be writable (caller handles `inhibit-read-only')." (cached-id (kitty-gfx--cache-get abs-file)) (image-id (or cached-id (kitty-gfx--alloc-id)))) (unless cached-id - (let* ((png (kitty-gfx--convert-to-png abs-file)) - (temp-p (and png (not (string= png abs-file))))) - (unwind-protect - (let ((b64 (when png (kitty-gfx--read-file-base64 png)))) - (if (not b64) - (kitty-gfx--log "centered: skipped %s (conversion failed)" abs-file) - (kitty-gfx--transmit-image image-id b64) - (kitty-gfx--cache-put abs-file image-id))) - (when temp-p - (ignore-errors (delete-file png)))))) + (when (funcall (kitty-gfx--backend-fn 'prepare) abs-file image-id) + (kitty-gfx--cache-put abs-file image-id))) ;; Create overlay at the scaled dimensions (when (or cached-id (gethash abs-file kitty-gfx--image-cache)) - (kitty-gfx--make-overlay img-start (point) image-id img-cols img-rows reuse-pid) + (kitty-gfx--make-overlay img-start (point) image-id img-cols img-rows abs-file reuse-pid) (kitty-gfx--schedule-refresh))))) (defun kitty-gfx-remove-images (&optional beg end) @@ -885,42 +1149,164 @@ The buffer should be writable (caller handles `inhibit-read-only')." (with-current-buffer buf (when kitty-gfx--overlays (kitty-gfx-remove-images)))) - (kitty-gfx--delete-all-images) + (when kitty-gfx--active-backend + (funcall (kitty-gfx--backend-fn 'cleanup-all))) (clrhash kitty-gfx--image-cache) (setq kitty-gfx--cache-lru nil) (setq kitty-gfx--next-id 1) (setq kitty-gfx--next-placement-id 1) (kitty-gfx--log "clear-all: done (reset IDs to 1)")) +;;;; Debug commands + +(defun kitty-gfx-debug-state () + "Dump all critical kitty-gfx state to *kitty-gfx-debug-state* buffer." + (interactive) + (let ((buf (get-buffer-create "*kitty-gfx-debug-state*"))) + (with-current-buffer buf + (erase-buffer) + (insert "=== kitty-gfx debug state ===\n") + (insert (format "Timestamp: %s\n" (format-time-string "%Y-%m-%d %H:%M:%S.%3N"))) + (insert (format "Backend: %s\n" kitty-gfx--active-backend)) + (insert (format "Cell pixel size: %sx%s\n" + kitty-gfx--cell-pixel-width kitty-gfx--cell-pixel-height)) + (insert (format "ImageMagick: magick=%s convert=%s identify=%s\n" + (executable-find "magick") + (executable-find "convert") + (executable-find "identify"))) + (insert (format "Next ID: %d Next PID: %d\n" + kitty-gfx--next-id kitty-gfx--next-placement-id)) + (insert "\n--- Windows ---\n") + (walk-windows + (lambda (win) + (let ((edges (window-edges win)) + (body (window-body-edges win))) + (insert (format " win=%s buf=%s edges=%S body=%S size=%dx%d\n" + win (buffer-name (window-buffer win)) + edges body + (window-body-width win) (window-body-height win))))) + nil 'visible) + (insert "\n--- Overlays ---\n") + (let ((count 0)) + (dolist (b (buffer-list)) + (let ((ovs (buffer-local-value 'kitty-gfx--overlays b))) + (when ovs + (dolist (ov ovs) + (cl-incf count) + (let ((alive (not (null (overlay-buffer ov))))) + (insert (format " [%d] buf=%s alive=%s id=%s pid=%s cols=%s rows=%s\n" + count (buffer-name b) alive + (overlay-get ov 'kitty-gfx-id) + (overlay-get ov 'kitty-gfx-pid) + (overlay-get ov 'kitty-gfx-cols) + (overlay-get ov 'kitty-gfx-rows))) + (insert (format " file=%s\n" + (overlay-get ov 'kitty-gfx-file))) + (insert (format " buf-pos=%s-%s last-row=%s last-col=%s\n" + (and alive (overlay-start ov)) + (and alive (overlay-end ov)) + (overlay-get ov 'kitty-gfx-last-row) + (overlay-get ov 'kitty-gfx-last-col))) + (when alive + (let ((screen-pos (kitty-gfx--overlay-screen-pos ov))) + (insert (format " computed-screen-pos=%S\n" screen-pos))))))))) + (insert (format "\nTotal overlays: %d\n" count))) + (insert "\n--- Image cache ---\n") + (insert (format " entries=%d lru-len=%d\n" + (hash-table-count kitty-gfx--image-cache) + (length kitty-gfx--cache-lru))) + (maphash (lambda (k v) (insert (format " %s -> %s\n" k v))) + kitty-gfx--image-cache) + (insert "\n--- Sixel cache ---\n") + (insert (format " entries=%d\n" (hash-table-count kitty-gfx--sixel-cache))) + (maphash (lambda (k v) (insert (format " %s -> %s\n" k v))) + kitty-gfx--sixel-cache)) + (display-buffer buf) + (message "kitty-gfx: debug state dumped to *kitty-gfx-debug-state*"))) + +(defun kitty-gfx-debug-overlay-at-point () + "Show deep debug info for the kitty-gfx overlay at point." + (interactive) + (let ((found nil)) + (dolist (ov (overlays-at (point))) + (when (overlay-get ov 'kitty-gfx-id) + (setq found ov))) + (if (not found) + (message "kitty-gfx: no overlay at point") + (let* ((id (overlay-get found 'kitty-gfx-id)) + (pid (overlay-get found 'kitty-gfx-pid)) + (cols (overlay-get found 'kitty-gfx-cols)) + (rows (overlay-get found 'kitty-gfx-rows)) + (file (overlay-get found 'kitty-gfx-file)) + (cw (or kitty-gfx--cell-pixel-width 8)) + (ch (or kitty-gfx--cell-pixel-height 16)) + (pixel-w (* (or cols 0) cw)) + (pixel-h (* (or rows 0) ch)) + (last-row (overlay-get found 'kitty-gfx-last-row)) + (last-col (overlay-get found 'kitty-gfx-last-col)) + (pos (overlay-start found)) + (win (selected-window)) + (win-pos (and pos (posn-at-point pos win))) + (col-row (and win-pos (posn-col-row win-pos))) + (edges (window-edges win)) + (body-edges (window-body-edges win)) + (buf-col (save-excursion + (goto-char pos) + (current-column))) + (screen-pos (kitty-gfx--overlay-screen-pos found)) + (disp-prop (overlay-get found 'display)) + (disp-len (if (stringp disp-prop) (length disp-prop) nil))) + (message (concat + "kitty-gfx overlay: id=%s pid=%s file=%s\n" + " cols=%s rows=%s cell=%dx%d pixel=%dx%d\n" + " posn-col-row=%S win-edges=%S body-edges=%S buf-col=%d\n" + " computed-screen-pos=%S last-row=%s last-col=%s\n" + " display-prop-len=%s") + id pid file + cols rows cw ch pixel-w pixel-h + col-row edges body-edges buf-col + screen-pos last-row last-col + disp-len))))) + ;;;; Minor mode ;;;###autoload (define-minor-mode kitty-graphics-mode - "Display images in terminal Emacs via Kitty graphics protocol." + "Display images in terminal Emacs via graphics protocol (Kitty or Sixel)." :global t - :lighter " KittyGfx" + :lighter (:eval (concat " KittyGfx[" + (pcase kitty-gfx--active-backend + ('kitty "K") + ('sixel "S") + (_ "?")) + "]")) (if kitty-graphics-mode - (if (kitty-gfx--supported-p) + (if (kitty-gfx--detect-protocol) (progn - (kitty-gfx--log "mode: enabling") - (kitty-gfx--delete-all-images) ; clear stale state + (kitty-gfx--log "mode: enabling (backend=%s)" kitty-gfx--active-backend) + (when kitty-gfx--active-backend + (funcall (kitty-gfx--backend-fn 'cleanup-all))) ; clear stale state (kitty-gfx--query-cell-size) (kitty-gfx--install-hooks) (kitty-gfx--install-integrations) - (kitty-gfx--log "mode: enabled (cell=%dx%d)" + (kitty-gfx--log "mode: enabled (backend=%s cell=%dx%d)" + kitty-gfx--active-backend kitty-gfx--cell-pixel-width kitty-gfx--cell-pixel-height) - (message "Kitty graphics mode enabled")) + (message "Kitty graphics mode enabled (%s backend)" + kitty-gfx--active-backend)) (kitty-gfx--log "mode: terminal not supported, aborting enable") (setq kitty-graphics-mode nil) (message "Kitty graphics: terminal not supported")) (kitty-gfx--log "mode: disabling") (kitty-gfx--uninstall-hooks) (kitty-gfx--uninstall-integrations) - (kitty-gfx--delete-all-images) + (when kitty-gfx--active-backend + (funcall (kitty-gfx--backend-fn 'cleanup-all))) (when kitty-gfx--render-timer (cancel-timer kitty-gfx--render-timer)) (setq kitty-gfx--render-timer nil - kitty-gfx--refresh-pending nil) + kitty-gfx--refresh-pending nil + kitty-gfx--active-backend nil) (kitty-gfx--log "mode: disabled"))) (defun kitty-gfx--install-hooks () @@ -947,7 +1333,7 @@ Deletes image placements from the terminal, clears position caches, then schedules a refresh that re-places only the overlays that are still visible (not inside a fold)." (kitty-gfx--log "on-org-cycle: overlays=%d" (length kitty-gfx--overlays)) - (when (and kitty-graphics-mode kitty-gfx--overlays) + (when (and kitty-graphics-mode kitty-gfx--overlays kitty-gfx--active-backend) (kitty-gfx--sync-begin) (unwind-protect (dolist (ov kitty-gfx--overlays) @@ -955,7 +1341,7 @@ still visible (not inside a fold)." (let ((id (overlay-get ov 'kitty-gfx-id)) (pid (overlay-get ov 'kitty-gfx-pid))) (when (and id pid) - (kitty-gfx--delete-placement id pid))) + (funcall (kitty-gfx--backend-fn 'delete) ov id pid))) (overlay-put ov 'kitty-gfx-last-row nil) (overlay-put ov 'kitty-gfx-last-col nil))) (kitty-gfx--sync-end)) @@ -994,7 +1380,7 @@ Scans for file:, attachment:, and relative path links." ((string= link-type "attachment") (ignore-errors (require 'org-attach) - (when-let ((dir (org-attach-dir))) + (when-let* ((dir (org-attach-dir))) (expand-file-name path dir)))) (t path)))) (when (and file @@ -1114,7 +1500,7 @@ image via Kitty graphics instead of an Emacs image spec." (kitty-gfx-display-image movefile beg end) ;; Tag the most recently created overlay with org properties ;; so org-clear-latex-preview can find and clean it up. - (when-let ((ov (car kitty-gfx--overlays))) + (when-let* ((ov (car kitty-gfx--overlays))) (overlay-put ov 'org-overlay-type 'org-latex-overlay) (overlay-put ov 'modification-hooks (list (lambda (o after &rest _) @@ -1140,7 +1526,7 @@ Values > 1.0 zoom in, < 1.0 zoom out.") (defun kitty-gfx--image-mode-render () "Render the current image file centered at current scale." - (when-let ((file (buffer-file-name))) + (when-let* ((file (buffer-file-name))) (when (kitty-gfx--image-file-p file) (let* ((inhibit-read-only t) (win-w (- (window-body-width) 2)) @@ -1392,7 +1778,7 @@ FILE is the file to preview, PREVIEW-WINDOW is the target window. Returns a buffer recipe, or nil if not in terminal or not an image." (when (and kitty-graphics-mode (not (display-graphic-p)) - (kitty-gfx--supported-p)) + kitty-gfx--active-backend) (kitty-gfx--log "dirvish-preview: %s" file) (let* ((buf-name (format " *kitty-dirvish: %s*" (file-name-nondirectory file))) (buf (get-buffer-create buf-name)) @@ -1584,19 +1970,21 @@ and image-mode simultaneously)." (condition-case nil (let ((id (overlay-get ov 'kitty-gfx-id)) (pid (overlay-get ov 'kitty-gfx-pid))) - (when (and id pid) + (when (and id pid kitty-gfx--active-backend) ;; Always delete the placement (it's buffer-specific) - (kitty-gfx--delete-placement id pid)) + (funcall (kitty-gfx--backend-fn 'delete) ov id pid)) ;; Only delete the image data if no other buffer uses it (when (and id (not (memq id deleted-ids))) (if (kitty-gfx--image-id-in-other-buffers-p id) (kitty-gfx--log "kill-buffer-hook: id=%d still used in other buffers, keeping" id) - (kitty-gfx--delete-by-id id) (push id deleted-ids)))) (error nil))) ;; Remove cache entries only for IDs we actually deleted (when deleted-ids (kitty-gfx--log "kill-buffer-hook: cleaning cache for ids=%S" deleted-ids) + (when kitty-gfx--active-backend + (dolist (id deleted-ids) + (funcall (kitty-gfx--backend-fn 'cleanup) nil id))) (maphash (lambda (file id) (when (memq id deleted-ids) (kitty-gfx--cache-remove file))) |