Indexing automation
Indexing automation keeps a local queue of sitemap URLs, submits what is allowed within fresh provider quotas and records what happened. It is a CLI and scheduler feature; the desktop 0.9.8 card renders the same data read-only.
What it does
- Queue.
indexing runreads the project sitemap (<project.base_url>/sitemap.xml, root plus five child documents, 10,000 URLs) and merges explicit--urlvalues. Off-host URLs are skipped and every cap is reported. - Priority. Never-submitted URLs first, then URLs whose sitemap
lastmodis newer thanlast_submitted_at, then URLs whoselast_checked_atis older thancheck_interval_days. A URL submitted after itslastmodis skipped unless--forceis set. - Checks and writes. Checks are read-only: GSC URL Inspection gives a per-URL verdict and Bing
GetUrlInfogivescrawledwhen a crawl date exists. Without--allow-writethe run only checks and plans. With it, the engine re-reads quotas, re-validates the plan hash, mints and consumes an internal confirm token through the write gate and records oneindexing_runsrow per provider. - Daily job.
schedule install --kind indexingrenders a dailylv24-seo-indexingjob at 04:00 runningindexing run --allow-write --config <abs> --jsonfor cron, systemd and Windows Task Scheduler. The generated command never contains the secret.
lv24-seo indexing status --live --json
lv24-seo indexing status --provider bing --limit 50 --json
lv24-seo indexing run --from-sitemap --max-submit 20 --max-inspect 5 --json
lv24-seo indexing run --from-sitemap --allow-write --json
lv24-seo indexing history --limit 20 --json
lv24-seo schedule install --kind indexing --allow-write --jsonQuotas and caps
Caps are enforced before the provider call. An exhausted quota is quota_exhausted with submitted: 0 and not_attempted; zero never means "all indexed".
| Engine | Remaining quota source | Per-run bound |
|---|---|---|
| Bing | GetUrlSubmissionQuota daily and monthly remaining values | 500 URLs per SubmitUrlbatch; the plan is capped to the smaller remaining window |
| Yandex | recrawl/quota quota_remainder | One POST per URL; no batch endpoint |
| GSC | No remaining-count API; local UTC-day quota_usage counter | 2,000 inspections/day/site by default, max_inspect_per_run 20 |
| IndexNow | No status or quota API; local UTC-day counter | 1,000 submissions/day by default, 10,000 URLs per POST |
indexing status reports per provider urls, submitted, crawled, queued, indexed (sample), indexed_total (GSC aggregate), pending, quota and limitations. --live reads Bing and Yandex quotas now; without it the last recorded quota is shown with quota_not_live.
Write secret and gates
indexing run --allow-write needs [indexing] enabled = true and LV24_SEO_WRITE_SECRET in the same environment. A scheduled apply needs the secret in the scheduler environment (systemd drop-in, user environment or cron export), never in the command line. indexing status reports only whether the secret is present as write_secret_present; the value is never read into an envelope. A replayed token is skipped/duplicate; a provider failure releases the claim.
Honest index counts
Only GSC exposes a per-URL verdict (PASS/Submitted and indexed → indexed; NEUTRAL/FAIL → not_indexed; anything else unknown). The GSC sitemap contents[].indexed aggregate is reported separately as indexed_total and always tagged gsc_indexed_count_deprecated. Bing GetUrlInfo yields http_status and last_crawled_at, so a crawled URL is crawled, not indexed (bing_no_per_url_index_status). Yandex and IndexNow have no per-URL status; their indexed_total is null with no_per_url_index_count.
Bing Site Scan has no public API. Every Bing entry carries site_scan_no_api; the desktop only opens the official page and never claims to start, monitor or import a scan.
Desktop Indexing automation card
Desktop 0.9.2 added a read-only card to Integrations; 0.9.8 keeps it. It renders indexing status, indexing history and schedule status; there are no write, confirm or --allow-write controls.
- Schedule.
Scheduled/host state and the documented Daily 04:00 cadence forlv24-seo-indexing. - Per provider. Bing, Yandex, GSC and IndexNow rows with
submitted,indexed(GSC only),pendingand quota: Bing daily/monthly remaining, Yandex remaining, GSC/IndexNow used/limit. A provider with no state shows "No data yet", never a zero. - Last run and history. The newest
indexing historyrow with status, submitted/checked counts and up to fiveerror_items(URL, code, message), plus a compact 7/30-day day/provider table fromindexing history --limit 30. - Honesty. Every render includes
data_age_daysandlimitations; the deprecated GSC aggregate is labelled and kept separate. Bing/Yandex/IndexNow show no per-URL count instead of inventing one. - Compatibility. If the engine answers
INVALID_COMMAND, the card shows "Indexing status needs engine >= 0.13.1" and hides the tables. An engine older than0.15.2is reported as incompatible by the shell. - Bridge.
indexing_status(provider,limit,live),indexing_history(limit) andschedule_status(kind,target) are allow-listed; the Rust layer validates every value, rejects write flags and cross-command arguments and uses a 120 s timeout.--liveis the only provider call; a missing credential becomes a limitation, not a failure.
Configuration
[indexing] is disabled by default. enabled = true is required before an apply; without it the run is a read-only plan. providers selects the engines, and max_submit_per_run (500), max_inspect_per_run (20), stale_days (7), check_interval_days (7), gsc_inspect_daily_limit (2,000) and indexnow_daily_limit (1,000) bound the work.
Limits
- The desktop card is read-only: it cannot install, edit or run the job.
--livereads Bing and Yandex quotas; a missing credential is a limitation.- A large site is sampled over several runs;
meta.truncatedsays so. - Yandex recrawl can return a provider-side 5xx; the daily job retries on its next run.
- IndexNow has no status API and no published quota; the local daily cap is a conservative guard.
- GSC sitemap submit and Bing sitemap delete remain
WRITE_DISABLED.
Related
Source: _source/engine-releases-0.12.0.md, _source/engine-releases-0.13.1.md, _source/engine-releases-0.15.2.md, _source/engine-releases-0.15.1.md, _source/engine-releases-0.15.0.md, _source/engine-releases-0.14.1.md, _source/engine-releases-0.14.0.md, _source/desktop-releases-0.9.8.md, _source/desktop-releases-0.9.7.md, _source/desktop-releases-0.9.6.md, _source/desktop-releases-0.9.5.md, _source/desktop-releases-0.9.4.md, _source/desktop-releases-0.9.2.md, _source/desktop-releases-0.9.2-update.md, _source/engine-cli.md, _source/engine-config.md.