# Cmd-K search case-study reproduction kit

This kit accompanies [The Cmd-K Input Was Fine. The Search Architecture Wasn't](https://www.thedjpetersen.com/llm-thoughts/the-cmd-k-input-was-fine/).

It has three independent layers:

1. Read-only profiling against your local Codex state and app-server.
2. A standalone first-principles search prototype with deterministic tests.
3. An unsupported, hash-locked patch for macOS app build `6872` only.

The first two layers are enough to reproduce the diagnosis and proposed design. Do not modify the installed app unless you have the exact supported build, accept that an updater can replace the patch, and have a tested backup.

## Requirements

- macOS with the Codex desktop app installed at `/Applications/ChatGPT.app`
- Node.js 22.5 or newer; the reference run used Node.js 24.0.2
- A local Codex state database at `~/.codex/state_5.sqlite`
- No npm dependencies

Check the versions first:

```sh
node --version
plutil -extract CFBundleShortVersionString raw /Applications/ChatGPT.app/Contents/Info.plist
plutil -extract CFBundleVersion raw /Applications/ChatGPT.app/Contents/Info.plist
/Applications/ChatGPT.app/Contents/Resources/codex --version
```

The reference measurements came from app `26.818.22352`, build `6872`, with `codex-cli 0.148.0-alpha.21`. Results from another build are useful, but they are not a byte-for-byte reproduction.

## Files

- `thread-search-load-profile.mjs`: measures sequential or overlapping `thread/search` calls through the installed app-server.
- `profile.mjs`: benchmarks the shipped-shaped metadata work and the proposed metadata/FTS indexes against local data.
- `search-index.mjs`: dependency-free action, metadata, typo, fallback, and latest-only coordinator prototype.
- `search-index.test.mjs`: six behavioral regression tests.
- `inspect-asar.mjs`: finds and extracts renderer chunks using semantic marker strings.
- `patch-cmdk-action-filter-asar.mjs`: rebuilds the build-6872 ASAR with a narrow action-typo fallback.
- `install-build-6872.sh`: guarded installer that refuses every other build or source hash.
- `reference-results.json`: sanitized reference measurements and hashes.

No raw conversation content, rollout file, SQLite database, executable, or ASAR archive is included.

## 1. Run the deterministic prototype tests

```sh
npm test
```

Expected result:

```text
tests 6
pass 6
fail 0
```

The suite checks:

- 60,000-character titles are capped before immediate search.
- Actions remain mounted through asynchronous chat enrichment.
- `new caht` resolves to `New chat` through bounded Levenshtein distance.
- An unmatched phrase creates an immediate selected new-chat row.
- Content requests are serial and retain only the latest pending query.
- Chat row keys survive content enrichment.

## 2. Benchmark the installed `thread/search` path

Start with sequential, warmed, synthetic misses:

```sh
node thread-search-load-profile.mjs \
  --mode=sequential \
  --runs=12 \
  --interval=0 \
  --prefix=__cmdk_seq_
```

Then reproduce overlap pressure:

```sh
node thread-search-load-profile.mjs \
  --mode=overlap \
  --runs=20 \
  --interval=10 \
  --prefix=__cmdk_overlap_
```

The script starts the app-server over stdio, performs `initialize`, primes one unrelated query, and times `thread/search`. It does not print conversation content.

Use `--binary=/path/to/codex` if the bundled CLI is elsewhere. Add `--sample=/tmp/thread-search.sample.txt` to capture a macOS native sample during the run.

The build-6872 reference was 55.01 ms median sequentially and 333.06 ms median with twenty requests started at 10 ms intervals.

## 3. Benchmark the proposed indexes against local data

```sh
node profile.mjs \
  --state="$HOME/.codex/state_5.sqlite" \
  --runs=200
```

The state database opens read-only. Rollout JSONL is read to create an in-memory FTS5 database. Nothing is written back to Codex state.

The reference dataset had 37 active threads, 601,876 title characters, 445.9 MiB of rollout source, and 7.8 MiB of searchable text across 2,501 messages.

Reference p95 metadata time fell from 1.3355 ms to 0.0612 ms. The one-time FTS build took 2.893 seconds; FTS queries measured 5.3466 ms median and 27.5443 ms p95.

## 4. Inspect the packaged renderer

Work from a copy:

```sh
mkdir -p ./asar-audit
cp -p /Applications/ChatGPT.app/Contents/Resources/app.asar ./asar-audit/app.asar
node inspect-asar.mjs ./asar-audit/app.asar --out=./asar-audit/extracted
```

The inspector searches JavaScript entries for:

- `command-menu-thread-search`
- `search-threads-for-host`
- `Loading chats`
- `No matches`

Search the extracted candidate with `rg` and format a copy before reading it. Minified symbol names and asset filenames are build artifacts, not stable APIs.

```sh
rg -n "command-menu-thread-search|search-threads-for-host|Loading chats|No matches" \
  ./asar-audit/extracted
```

On build `6872`, the relevant asset was `webview/assets/app-initial-2HRzhJVF.js`.

## 5. Generate the build-6872 patch without installing it

First verify the source hash:

```sh
shasum -a 256 ./asar-audit/app.asar
```

The patcher accepts only the exact minified filter body found once in the archive. The reference source archive hash is in `reference-results.json`.

```sh
node patch-cmdk-action-filter-asar.mjs \
  ./asar-audit/app.asar \
  ./asar-audit/app.patched.asar \
  ./asar-audit/app-initial.patched.js \
  ./asar-audit/patch-manifest.json
```

The patcher validates direct and typo behavior before touching the archive. It then rebuilds every packed offset and verifies every entry hash and block hash in the resulting ASAR.

## 6. Install only on the exact reference build

This step modifies `/Applications/ChatGPT.app`. Quit the app first. The installer refuses any build other than `6872`, refuses an unknown source hash, and refuses to overwrite a backup.

```sh
sh install-build-6872.sh \
  /Applications/ChatGPT.app \
  "$PWD/chatgpt-cmdk-backup-6872"
```

The installer backs up `app.asar`, `Info.plist`, the executable, and `CodeResources`. It changes only `app.asar` and the ASAR header hash in `Info.plist`.

After restarting the app, open Cmd-K and type `new caht`. It should match `New chat`. Also verify that `new chat` still ranks as a direct match and that unrelated short inputs do not become fuzzy.

## 7. Roll back

Quit the app, then restore the two changed files from the backup directory:

```sh
cp -p ./chatgpt-cmdk-backup-6872/app.asar.original \
  /Applications/ChatGPT.app/Contents/Resources/app.asar
cp -p ./chatgpt-cmdk-backup-6872/Info.plist.original \
  /Applications/ChatGPT.app/Contents/Info.plist
```

Restart the app. A normal app update may also replace the local patch. In the reference case, the updater replaced build `6872` with build `6962` the next day.

## Why the ASAR rebuild is necessary

Electron can validate a SHA-256 hash for each packed entry, block hashes for entry chunks, and a separate hash for the serialized ASAR header. On macOS, the header hash is stored under `ElectronAsarIntegrity` in `Info.plist`.

Changing one renderer function grew the target entry by 400 bytes. Every following packed offset therefore moved. Replacing only the JavaScript bytes would corrupt offsets and fail integrity checks.

See Electron's [ASAR integrity documentation](https://github.com/electron/electron/blob/main/docs/tutorial/asar-integrity.md).

## What this kit does not claim

The local ASAR patch installs only typo recovery for command actions. It does not install the packed metadata index, FTS5 content index, latest-only coordinator, or automatic new-chat fallback.

Those broader changes are a tested reference design for upstream integration. The renderer is not published as source, while the backend scanner is visible in OpenAI's public [`search_threads.rs`](https://github.com/openai/codex/blob/main/codex-rs/thread-store/src/local/search_threads.rs).

The workflow is unsupported and version-specific. Keep the backup, expect updates to overwrite it, and never run the installer against an unrecognized build.