order recent chats first

[arkml]

Summary

This change updates chat history ordering to match the behavior users expect from apps like ChatGPT and Claude:
conversations are now ordered by most recent message activity, not by when the chat was originally created.

Before this change, the sidebar list was effectively sorted by run creation order because the backend listed run
files by filename/run ID. That meant an older conversation with a new reply could remain buried below newer-but-
inactive chats. With this change, any chat with a new user or assistant message moves to the top of the history list.

What Changed

• Added lastMessageAt to the shared run/list schema so the backend can return both chat creation time and latest
  message activity time.
• Added a small compatibility helper to derive an ISO timestamp from existing monotonic message IDs. This lets older
  runs participate in activity-based ordering even if they were created before non-start events consistently carried
  a ts field.
• Updated the runs repo metadata reader to scan each run for:
  • the first user message, which is still used as the chat title
  • the most recent message event, which is now used as the activity sort key
• Changed runs:list to sort by latest message activity instead of file/run creation order. Pagination now happens
  after this activity sort is computed.
• Updated event persistence so appended run events get a ts value if they do not already have one. That gives new
  runs an explicit timestamp source going forward instead of relying only on ID parsing.
• Updated renderer-side run state to track lastMessageAt and optimistically reorder chats in the sidebar as messages
  are sent or received, so the UI updates immediately without waiting for a full reload.
• Updated the chat history timestamp badge in the sidebar to display relative time from lastMessageAt when available,
  falling back to createdAt only when needed.

Behavioral Impact

• Active conversations now rise to the top as soon as a user sends a message.
• Conversations also rise to the top when an assistant response arrives.
• Existing/legacy runs continue to sort correctly via fallback timestamp extraction from message IDs.
• The visible “time ago” label in chat history now reflects last activity, which better matches the ordering.

Why This Approach

The key constraint was backward compatibility. Existing run logs do not reliably store ts on every non-start event,
so sorting purely by explicit timestamps would have produced inconsistent results for older chats. The fallback to
parsing the monotonic message ID preserves correct ordering for legacy runs, while the new ts backfill ensures future
runs have explicit event timestamps.

[ramnique]

A few concerns from a review pass — flagging before this lands.

1. runs:list is now O(total bytes of all run files)

This is the biggest one. Before, the list path sorted filenames at the FS level, loaded only PAGE_SIZE files, and each scan stopped at the first user/assistant message (typically a few lines). After this PR, list loads every run file in the directory and scans the full file end-to-end to find the latest message before sorting and paginating (repo.ts list + readRunMetadata).

For a user with a few hundred chats — especially long agentic sessions that are multi-MB — every call to runs:list now reads hundreds of MB and parses thousands of JSON lines. Cold sidebar load scales with total history size rather than page size.

Mitigations roughly in order of effort:

• Reverse-read each file to find the last message event (the typical case is a few lines from the end).
• Sidecar metadata file ({runId}.meta.json) updated on appendEvents with lastMessageAt + lastMessageSortKey. list becomes O(N file stats + N small JSON reads).
• In-memory cache keyed by file mtime; only re-scan files whose mtime changed since last list.

2. Unbounded concurrent file reads

Promise.all(files.map(...)) in list opens a read stream per run with no concurrency cap. With a few hundred runs you'll hit macOS fd limits intermittently (EMFILE). Batch with a concurrency limit (e.g. p-limit(16)).

3. Cursor pagination on a now-mutable sort order

The old cursor was stable because the sort key was the filename. The new sort key is the latest-message ID, which mutates whenever a message arrives. If a run moves up past the page-1 cursor between page 1 and page 2 fetches, page 2 will skip it; if one moves down past the cursor, page 2 will duplicate it. Probably rare in practice but worth knowing. A more robust cursor would be (sortKey, fileName) sliced strictly by sortKey rather than by indexOf(fileName).

4. monotonicIdToIsoTimestamp is silently coupled to the ID generator format

The regex in shared/runs.ts matches the current generator (application/lib/id-gen.ts: no ms, always Z, exact YYYY-MM-DDTHH-MM-SSZ- shape) — but there's no test, no comment pointing back to the generator, and a silent undefined on mismatch. Either add a unit test that round-trips a real generated ID, or add a comment in id-gen.ts warning that the format is parsed elsewhere and must not change.

Smaller notes

• appendEvents shares one appendedAt across all events in a batch — semantically fine, just worth noting if anyone later uses ts for intra-run ordering.
• Renderer post-send setRuns and the message event handler both derive lastMessageAt via the same ts ?? monotonicIdToIsoTimestamp(id) ?? now() ladder — a tiny helper would dedupe.

Functionally correct, schema migration is sound, optimistic UI is well-done. #1 is the one I'd want addressed before this lands, especially given the trend toward longer agentic chats.