MMSReadinessIndex reads three files, once a quarter.

The crawler collects data for the Small Business AI Readiness Index. Each quarter it measures about 2,000 small business websites, sampled from public business listings in all 50 states and Washington, DC. It runs the same checks as our Site Check, AI Search Kit, Domain Health and Own It connectors.

The Index is published as totals and percentages only. We never publish a list that names a business, a domain or a single site's score, and any group smaller than 10 sites is left out of the published figures.

Once a quarter. A run reads each sampled site once and does not come back during that quarter. The sample is drawn fresh each quarter, so a site measured once may not be measured again. The most recent run was October 12, 2026 (UTC).

For each sampled site, one request each, in this order:

1. /robots.txt, before anything else.
2. The homepage, as any browser receives it, if robots.txt allows it.
3. /llms.txt, if robots.txt allows the homepage.
4. The robots.txt of the final host, only when the homepage redirects to a different host.

When the homepage was reachable, it also makes one TLS handshake to read the site's certificate. It reads public DNS records (MX, SPF, DMARC, DKIM at common selectors, NS, A) and the domain registry's public RDAP record, neither of which reaches your server. If your DNS publishes an MTA-STS policy or a BIMI logo, it reads those two files the way a mail server would, under the user agent of our Domain Health connector.

It never fetches any other page, image or script, never submits a form, and never tries to sign in. Every request has a hard timeout: 15 seconds for the homepage, 8 seconds for small files, and 100 seconds for a whole site before it gives up and moves on.

The crawler follows the group for MMSReadinessIndex in your robots.txt, or the * group if there is none. If those rules disallow your homepage, it fetches neither the homepage nor llms.txt, and your site is counted only as one that excluded our crawler.

A robots.txt that answers with a server error is read as “disallow everything”, as RFC 9309 says. A robots.txt that is missing (404) means no rules apply.

Add these two lines to your robots.txt. Every future run will then skip your homepage and llms.txt:

User-agent: MMSReadinessIndex
Disallow: /

To also have us delete what we already hold about your site, write to sarah@modernmustardseed.com with the domain. A person reads every message and writes back.

The result for each site is stored privately. It is never published, never sold, and nobody is contacted because of it. The Index is not a sales list: being measured does not put you on one.

Questions, a request that came in too fast, or anything that looks wrong: write to sarah@modernmustardseed.com. Include the domain and, if you can, the time from your log.

Modern Mustard Seed LLC, Kalispell, Montana. See also our privacy policy and support page.