Skip to main content
An integration can type-check, build, answer curl on every editor route, and still break the moment a person opens it in the Studio. On the two sites this command was designed from, every problem found after the coding agent reported “done” lived somewhere those checks never go:
  • a CMS live-preview SDK that threw a full-page error as soon as anything but its own app framed the page;
  • template security headers that refused the editor frame;
  • a blog post that answered 404 in the preview once its draft held a block type the template no longer rendered;
  • bullets a CSS reset had removed, invisible until real content arrived with a list;
  • a whole blog post mapped to one block, so clicking anywhere selected everything;
  • an image component that crashed on any URL not from its own CDN.
avocado qa exercises exactly those paths. It is the last step of the avocado-integrate skill, and an agent should not report an integration finished until it exits 0.
avocado is a bin of @avocadostudio-ai/site-sdk, so npx avocado qa finds it wherever the SDK is installed. The unscoped name avocado on npm belongs to an unrelated package, so where the SDK is only a transitive dependency (an Astro site under pnpm, for example) use the scoped form, which always resolves to this command:

​
What it does

It runs against the running site and the orchestrator the site reads drafts from. Everything it renders comes from a throwaway session named qa-<timestamp>, seeded with the site’s own pages. Your session is never read from or written to, and the throwaway sessions are removed at the end (library mode has no delete route; there they stay in the database, harmlessly). Every check reports pass, warn, fail, skip or info. Any fail makes the command exit 1. Warnings do not, but each carries a fix, and an agent should read them.

​
0 · Preflight

​
1 · Static

​
2 · Content contract

​
3 · Editor render

Each page is framed from the editor’s own origin — the host page is served by intercepting a path on that origin, so the site’s real frame-ancestors and postMessage target are what gets tested, and it works while the editor itself is running. The fixtures — rich text, the draft probe, image sources — each go into a throwaway session of their own (qa-<ts>-fx<n>), so the plain pass measures your real content. Without Playwright the stage still runs on the server HTML: load, block ids, coverage, the draft probe, image sources and drift are all in the markup. Errors, granularity and rich-text styles need a browser and are reported as skipped. Playwright is looked for in the project, next to the SDK and in a pnpm workspace; it is not a dependency of the SDK.

​
Invasiveness (informational)

A git diff against --base, or the merge-base with the default branch, including uncommitted and untracked files — an agent’s integration usually is not committed when this runs. Installed skills (.claude, .agents, AGENTS.md, CLAUDE.md) and lockfiles are excluded. It reports new and changed files and lines, files outside the integration’s own folders, public route files that changed, and — as a warning — library modules that something imported at the base and nothing imports now, the data layer an integration left behind.

​
5 · Manual pass

Printed at the end: ten minutes in the Studio once the gate is green.
  • Clicking each visible section selects a block with a sensible name and only its fields
  • Fields that cannot be written (asset alt text, slugs, dates) are not offered as editable
  • A chat edit and a panel edit both update the preview within a few seconds
  • Rich text in the panel looks like rich text on the page (lists, headings, links)
  • Asking for a language the site does not have produces a question, not an overwrite
  • Publishing one page changes only that page in the CMS, and the public site shows it after reload

​
Not yet automated

Printed on every run, so a green summary is not read as covering them:
  • the CMS space holds every content type the site queries (needs CMS credentials);
  • lint;
  • a round trip that is a no-op for every page × locale, a synthetic edit per field kind, and a publish dry run — all three need the site’s lens, which the SDK has no endpoint to run;
  • rich text in the Studio’s property panel (the panel is the editor’s, not the site’s);
  • session compatibility: seeding a session from the previous manifest and loading it against the new one. The snapshot half is done — see manifest.lock below.

​
Files

​
Options

The env files are read the way the dev server reads them — .env, .env.development, .env.local, .env.development.local — so the command checks the orchestrator, secret and editor origin the site actually uses.