About the GPUFind crawler

If you found this page from your server logs, this is the request that sent you here:

User-Agent: GPUFind/1.0 (+https://gpufind.cloud/crawler)

What we collect

gpufind.cloud compares published cloud GPU rental prices. The crawler fetches one thing from each provider: the public pricing page, or the public pricing API where one is documented. It stores the raw response byte for byte, records its hash, and extracts the listed prices from it. Every number on this site links back to that archived copy, which is the point of the project — we publish evidence, not just a figure.

We do not log in, submit forms, run a headless browser, or fetch anything beyond the pricing surface. We never authenticate: if a provider's prices are behind a login, we drop the provider and say so on its page rather than route around it.

How often

Every source is re-read on the same schedule: once every six hours, from a single machine, with one request in flight per domain at a time. Requests carry If-None-Match and If-Modified-Since where a source supports them, so an unchanged page costs a 304 and nothing more.

Rules we follow

• robots.txt is honoured. If a pricing path is disallowed, the provider is dropped and the reason is recorded publicly on its page.
• One request at a time per domain. Never concurrent fetches against the same host.
• Back off on 429 and 5xx, exponentially, honouring Retry-After when you send one.
• Five consecutive failures trips a circuit breaker for that provider, and it stays off until a human looks.
• We identify ourselves with the User-Agent above on every request, so you can allow or block us by name.

To have your site removed

Email crawler@gpufind.cloud from an address at your domain, or add a Disallow rule for the pricing path to your robots.txt — the next run will pick it up and stop. Either way the provider is listed on the providers page as excluded, with the reason, rather than quietly disappearing. Blocking the User-Agent string works too.

The collector code, the ethics rules it enforces, and the per-provider exclusion reasons are part of the same codebase that renders this page.