Compare

Screaming Frog vs Semrush

Semrush scores higher on the AgentReady, 46/100 against 37/100. They differ on 6 of the 41 signals. Which ones decides whether an agent can adopt them without a person watching.

What each one is

Screaming Frog. UK based SEO agency and creators of website crawler and log file analyser tools

Semrush. The leading platform to grow and measure brand visibility across every digital channel.

Where Screaming Frog is ahead

Screaming Frog passes copyable quickstart and official python sdk, and Semrush does not. That is adopt, whether an agent can get a key and make its first successful call without a human in the loop.

Where Semrush is ahead

Semrush passes llms.txt published and mcp discoverable, and Screaming Frog does not. That is discover, whether an agent can find the product at all without being told it exists.

It also holds adopt: self-service signup and mcp integration available. Screaming Frog misses those.

What neither does

Both fail public docs discoverable, llms-full.txt / full agent docs, structured api reference, openapi / spec quality, authentication documented, request examples provided, response examples provided, errors and status codes documented, limits / constraints documented, no mandatory sales call, programmatic credential creation, fast time to first request, cli available, structured, predictable output, machine-readable errors, retry behavior documented, idempotency support, rate-limit behavior predictable, observable execution, agent compatibility verified. If your agent needs any of those, you will be building it yourself either way.

Score, pillar by pillar

The AgentReady splits into four pillars, scored separately, because a product can be easy to find and still impossible to adopt.

Discover is whether an agent can find the product at all without being told it exists. Semrush leads 80 to 60. Screaming Frog misses public docs discoverable, llms.txt published, llms-full.txt / full agent docs, mcp discoverable; Semrush misses public docs discoverable, llms-full.txt / full agent docs.

Understand. Both sit at 23/100 here. Screaming Frog misses structured api reference, openapi / spec quality, authentication documented, request examples provided, response examples provided, errors and status codes documented, limits / constraints documented; Semrush misses structured api reference, openapi / spec quality, authentication documented, request examples provided, response examples provided, errors and status codes documented, limits / constraints documented.

Adopt. Semrush leads 55 to 40. Screaming Frog misses self-service signup, no mandatory sales call, programmatic credential creation, fast time to first request, cli available, mcp integration available; Semrush misses no mandatory sales call, programmatic credential creation, fast time to first request, copyable quickstart, official python sdk, cli available.

Operate. Both sit at 24/100 here. Screaming Frog misses structured, predictable output, machine-readable errors, retry behavior documented, idempotency support, rate-limit behavior predictable, observable execution, agent compatibility verified; Semrush misses structured, predictable output, machine-readable errors, retry behavior documented, idempotency support, rate-limit behavior predictable, observable execution, agent compatibility verified.

Pricing

Screaming Frog does not publish a machine-readable starting price and has a free tier. Semrush does not publish one with no free tier.

Signal by signal

Which to pick

Semrush clears more of the signals an agent needs, so it is the safer default for unattended use. Full profiles: Screaming Frog and Semrush. Alternatives to each: Screaming Frog, Semrush.

An agent can fetch this as data: POST /v1/compare {"slugs": ["screamingfrog", "semrush"]}