Your site has a
second reader now.

Technical SEO for the agent web. People get your full design. AI crawlers and agents get clean Markdown, structured data and tools they can call, from the same URL. We build it, or brief your team, and test it with real agents.

Trusted by B2B teams at

  • UnderDefense
  • HROne
  • EasyWebinar
  • Oliv AI
  • B2Brain
  • Explorium
  • EdgeLabs
  • Luca AI
  • Nidra
  • Teamvoy

In short

What technical SEO with MaximusLabs is, in two sentences.

Technical SEO for the agent web makes one site work for four readers: search engines, AI crawlers, AI agents and people.

MaximusLabs makes your pages fast, indexable, readable without JavaScript and usable through Markdown, MCP, WebMCP and A2A, then measures every visit.

Your site now has four readers

One URL, served right to each
  • Search enginesIndex it

    Googlebot and Bingbot render your pages and rank them. Still the base layer.

  • AI crawlersQuote it

    GPTBot, ClaudeBot and PerplexityBot read your HTML or Markdown, then cite you.

  • AI agentsAct on it

    Browser and API agents call your tools, fill your forms and book the demo.

  • PeopleUse it

    Your full design, unchanged. Same facts for every reader, never a different story.

Where agents
get stuck today.

We test your site the way each reader meets it: Googlebot, AI crawlers that fetch raw HTML, browser agents that click, and API agents that call tools. You get a score per layer and the fixes that move it.

  • Every reader tested

    The same pages fetched as Googlebot, an AI crawler, a browser agent and an API agent.

  • Each layer checked

    Access, rendering, Markdown, schema, protocols and actions, one by one.

  • Fixes in order

    What to build first, ranked by what it unlocks for buyers and agents.

Sample agent audit of Acme's pricing page across four readers, and the three fixes to make first
/pricing, read four ways
Same URL, four readers · acme.com · sample
5 blockers
Sample agent audit of Acme's pricing page: what Googlebot, an AI crawler, a browser agent and an API agent can each do
CheckGooglebotAI crawlerBrowser agentAPI agent
Can reach the pagePass2 of 4 botsPass—
Sees the prices and plansPassJS onlyPass—
Gets a Markdown version—None—None
Knows who Acme isPassPassPassPass
Can book a demo——Unlabelled formNo tool
Fix first
1Server-render pricing and product templates
2Allow Claude-User and PerplexityBot at the CDN
3Ship Markdown copies and a book_demo tool

Rendered for
every reader.

The major AI crawlers fetch your raw HTML without running JavaScript. If pricing, features or docs only appear after scripts load, they see an empty shell. We move key templates to server or static rendering and keep the interactive UI for people.

  • Raw HTML vs rendered

    What each crawler sees on every key template, side by side.

  • The right mode per template

    Server, static or hybrid rendering, built in the stack you already run.

  • Text first, interactivity after

    Interactive parts load on top of the text, never instead of it.

Sample rendering check: what a person sees on Acme's pricing page next to the empty HTML an AI crawler fetched, and the result after server rendering
Rendering check · /pricing
Before the fix · client-side rendered · sample
12% of text in HTML
What a person seesPlans that grow with your team
Starter
Growth
Scale
Book a demo
What an AI crawler fetched
<html>
  <head>
    <title>Pricing | Acme</title>
    <script src="/app.js"></script>
  </head>
  <body>
    <div id="root"></div>
  </body>
</html>
No plans, no prices, no features. Nothing to quote.
After server renderingVerified
96%
of page text now in the first HTML response. The UI people use is unchanged.

One URL,
two formats.

When an AI agent asks for a page, it gets clean Markdown: the content without layout or scripts, so it reads and quotes you accurately. A person on the same URL still gets your full design.

  • Content negotiation

    Agents that ask for Markdown get Markdown. Browsers get HTML. Same URL.

  • llms.txt and Markdown copies

    A map of your key pages for AI tools, plus a .md copy of each.

  • Parity check

    Both formats carry the same facts on every release. Never a separate version for bots.

Sample Markdown copies of Acme pages with their size against the HTML and a parity check that caught one price mismatch, and the llms.txt file
Markdown copies · parity check
212 of 230 key pages · checked on every release · sample
Same facts
Sample Markdown copies of Acme pages: HTML size against Markdown size, and whether both carry the same facts
PageHTML → MarkdownParity
/pricing214 KB → 4 KBMatch
/product/forecasting256 KB → 7 KBMatch
/vs/kestrel188 KB → 5 KBPrice differs
/docs/integrations142 KB → 9 KBMatch
/security96 KB → 3 KBMatch
/llms.txtCheap to ship
# Acme
> Revenue forecasting for B2B teams.

## Product
- [Pricing](/pricing.md)
- [Integrations](/integrations.md)
- [Security](/security.md)

Speak the
agent protocols.

Agents don't only read pages. They call tools. We build these endpoints for you, or spec them for your team, each scoped to exactly what you choose to expose.

  • MCP server

    Lets assistants like ChatGPT and Claude search your docs, pricing and integrations.

  • WebMCP tools

    Page actions like book_demo that browser agents can call. A proposed standard, in a Chrome trial.

  • A2A and NLWeb

    An agent card other agents can read, and a plain-language endpoint over your content.

Sample agent protocol map for Acme (MCP, WebMCP, A2A, NLWeb) and an illustrative WebMCP tool registration for booking a demo
Agent protocols · acme.com
What agents can call · sample
3 of 4 live
Sample agent protocol map for Acme: MCP, WebMCP, A2A and NLWeb, what each gives agents, where it lives and its status
ProtocolWhat agents getWhere it livesStatus
MCPSearch docs, pricing and integrations from any assistantmcp.acme.comLive
WebMCPPage tools a browser agent can call: book_demo, compare_planson /pricing, /demoBuilding
A2AAn agent card other agents can read/.well-known/agent-card.jsonLive
NLWebPlain-language questions over your site/askLive
Selling online? Checkout protocols (ACP, UCP, AP2) live on Agentic commerce.See it →
WebMCP · /demoChrome origin trial
document.modelContext.registerTool({
  name: "book_demo",
  description: "Book an Acme demo",
  inputSchema: { email, teamSize },
  execute: bookDemo
})

Pages agents
can operate.

Browser agents read your page much like a screen reader does: through labels, roles and structure. Unlabelled buttons and div-only forms stop them. We make every key action clear to agents and to people.

  • Accessibility tree audit

    Every button, link and field checked for a name, role and state.

  • Forms agents can finish

    Labelled fields, clear errors, and no hover-only menus or hidden steps.

  • Tested with real agents

    We run browser agents through your demo and trial flows, before and after.

Sample accessibility tree of Acme's demo page with two blockers, and an agent test run before and after the fix
What an agent sees · /demo
Accessibility tree · acme.com · sample
2 blockers
Sample accessibility tree of Acme's demo page: each element's role, accessible name and whether a browser agent can use it
RoleAccessible nameAgent can use it?
heading"Book a demo"Yes
textbox"Work email"Yes
textbox(no name)No · team size has no label
combobox"Region"Yes
button(no name)No · icon-only submit
link"See pricing"Yes
Agent test run · book a demo
Before the fixStuck at step 3
After labels + named buttonBooked in 4 steps

Tell machines
what you are.

Structured data tells agents and AI engines exactly what your company and product are. We write one connected JSON-LD graph into your server HTML, tie it to your profiles, and tell engines when it changes.

  • One graph, server-rendered

    Company, product, articles and FAQs in the first HTML response, linked by @id.

  • One connected entity

    sameAs links tie your site to the profiles you already own.

  • Freshness signals

    Accurate lastmod dates and IndexNow pings on every change.

Sample JSON-LD for Acme, the structured data status of each page template, and the linked profiles behind one entity
Schema by template
acme.com · JSON-LD · sample
86% of pages valid
{
  "@context": "https://schema.org",
  "@type": "SoftwareApplication",
  "name": "Acme",
  "applicationCategory":
    "BusinessApplication",
  "publisher": {
    "@type": "Organization",
    "@id": "#acme",
    "sameAs": [
      "linkedin.com/company/…",
      "wikidata.org/wiki/…"
    ]
  }
}
Sample structured data status for each Acme page template
TemplateStatus
HomepageOrganization · WebSiteValid
Product pagesSoftwareApplicationValid
Blog articlesArticle · FAQPageValid
Comparison pagesWebPage · ItemListWarning
PricingProduct · OfferMissing
One entity · sameAs4 linked
LinkedInWikidataCrunchbaseG2

Decide who gets
in, and why.

Not every bot wants the same thing. Some train models, some answer searches, some fetch a page because a person asked. We set a policy per purpose and keep the crawlers behind AI answers open.

  • A policy per purpose

    Training, search and user-triggered fetches, each allowed or blocked on purpose.

  • Verified agents

    Signed agent requests are recognised, not challenged.

  • CDN and firewall rules

    Stops scrapers without blocking AI answers.

Sample bot access for Acme by crawler and purpose, with two answer crawlers challenged by mistake at the CDN, and an illustrative robots.txt policy
Bot access by purpose
robots.txt + CDN rules · acme.com · sample
2 challenged by mistake
Sample bot access for Acme: each crawler, what it is used for, and what robots.txt and the CDN firewall let through
CrawlerUsed forrobots.txtCDN / WAF
OAI-SearchBotChatGPT searchAllowedAllowed
GPTBotModel trainingBy choice: noAllowed
Claude-UserUser-asked fetchAllowedChallenged
PerplexityBotPerplexity searchAllowedChallenged
Google-ExtendedGemini trainingAllowedn/a
GooglebotGoogle SearchAllowedAllowed
/robots.txt · policyillustrative
User-agent: *
Content-Signal: search=yes,
  ai-input=yes, ai-train=no
Allow: /

User-agent: Claude-User
Allow: /

The basics,
still first.

None of this works on a site Google can't index. We still run the full classic audit, ranked by the revenue pages each issue touches, and verify every fix.

  • Indexation and canonicals

    Every page you want found is indexed, under one clear URL.

  • Core Web Vitals

    LCP, INP and CLS per template, fixed where buyers land.

  • Site structure

    Topic clusters, orphan pages and click depth to pricing.

Sample site health for Acme since the audit: pages indexed, templates passing Core Web Vitals, classic checks before and after, and fix progress
Site health · since the audit
Search Console + crawl · acme.com · sample
18 of 31 fixes verified
Pages indexed
412of 530 · ↑ 61
Templates passing Core Web Vitals
3 / 5pricing, blog next
Sample classic technical checks for Acme, at the audit and now
Classic checkAt auditNow
Canonical conflicts220
Redirect chains6412
Orphan pages234
Clicks from home to /pricing52
Every fix, verified done
18 verified · 4 in progress · 2 with your dev · 7 planned

See every
agent visit.

Server logs show what page analytics miss: which crawlers and agents fetched which pages, in which format, and what happened next. We set up the reporting and read it with you every month.

  • Visits by purpose

    Training crawls, search crawls and user-asked fetches, counted apart.

  • Formats and tool calls

    Markdown served, llms.txt reads and MCP tool calls, per day.

  • AI referrals

    People arriving from AI answers, tied to demos in GA4.

Sample agent visits to Acme by purpose over eight weeks, the pages agents fetch most and in which format, and MCP tool calls
Agent visits by purpose
Server logs · last 8 weeks · sample
SearchUser-askedTraining
Sample pages Acme's agent visitors fetch most, and in which format
Most fetched by agentsFormatFetches
/pricingmarkdown3.1K
/docs/integrationsmarkdown2.4K
/vs/kestrelhtml1.2K
MCP tool calls · 7 days
search_docs1,204
get_pricing388
book_demo41

From audit to
agent-ready.

Every engagement runs the same five steps. Each one leaves something you can open and check.

  1. 01

    We test every reader

    Googlebot, AI crawlers, browser agents and API agents on your key pages.

    Audit31 findings · 5 blockers
  2. 02

    You get one roadmap

    Fixes and builds ranked by what they unlock, each with an owner.

    Roadmap31 items · 4 critical
  3. 03

    We build, or we brief

    We ship it ourselves, or spec it for your developers and review the work.

    Agent tools6 live via MCP + WebMCP
  4. 04

    We prove it works

    Re-crawls, agent test runs and parity checks after every release.

    Fix tracker18 of 31 verified
  5. 05

    We watch every visit

    Agent visits, tool calls and AI referrals, from your logs and GA4.

    Agent visits18.4K in 30 days

Most stop at Google.
We build for agents.

SEO agencies stop at Google. Dev shops build without a search plan. We do both, for every engine and agent.

Table 1. How MaximusLabs technical SEO compares with a traditional SEO agency and a development agency.
What you compareSEO agencyDev agencyMaximusLabs
Who it tests forGooglebotPeople in a browserGooglebot, AI crawlers, browser agents and API agents
Markdown for agentsNot offeredIf asked, with no search planSame URL, same facts, parity checked every release
MCP, WebMCP and A2ANot offeredCan buildBuilt for you, or specced for your team
RenderingFlags the problemFixes it without search in mindPlanned around what crawlers and agents read
Tied to AI visibilityPartlyNoLinked to citations in every AI model

Questions

What technical teams ask us first. Something else on your mind? Ask us on the strategy call.

What is technical SEO for AI agents?

It is the work that lets search engines, AI crawlers and AI agents reach, read and use your site: server rendering, Markdown for agents, structured data, bot policy and protocols like MCP and WebMCP. MaximusLabs audits all of it, builds the fixes or briefs your team, and measures every agent visit.

Do AI crawlers run JavaScript?

The major AI crawlers fetch your raw HTML and do not run JavaScript; a 2024 Vercel and MERJ study found none of them rendered it, while Google’s Gemini uses Googlebot, which does. MaximusLabs server-renders the pages that matter so every crawler sees the same content.

Is serving Markdown to AI agents cloaking?

Not when both formats carry the same content. Cloaking means showing bots different content to manipulate results. MaximusLabs serves the same facts as Markdown on the same URL, with Vary: Accept and a canonical, and checks parity on every release.

What is llms.txt, and does it help?

llms.txt is a proposed file, from September 2024, that maps your key pages for AI tools. No major AI engine has confirmed using it, and Google says Search ignores it. MaximusLabs ships it because it is cheap and helps agents, then measures whether anyone fetches it.

What are MCP, WebMCP and A2A, and do we need them?

MCP lets AI assistants call your tools and data, WebMCP (a proposed standard now in a Chrome origin trial) lets your pages expose actions to browser agents, and A2A lets agents discover and work with each other. MaximusLabs recommends them only where an agent has a real job to do on your site.

Should we block AI training crawlers?

That is a business choice. Blocking training crawlers such as GPTBot and ClaudeBot does not block the search and answer crawlers such as OAI-SearchBot, Claude-SearchBot and PerplexityBot. MaximusLabs sets a policy per purpose, so you can opt out of training and still be cited.

How is this different from MaximusLabs’ B2B SEO service?

Technical SEO makes the site readable and usable by search engines, AI crawlers and agents. B2B SEO decides what to rank for and writes the pages that win buyers. Most MaximusLabs clients fix the foundation first.

Who builds the fixes: MaximusLabs or our developers?

Both, depending on the format you choose. MaximusLabs either builds and ships the fixes itself, or specs each one for your developers and reviews the work, then re-checks every fix either way.

Ready for the agent web.

Book a call. We'll test your site the way Google, AI crawlers and agents see it, and show you what to fix first.

This page is coming soon.