Getting blocked isn't bad luck — it's a signal. Anti-bot systems score every request on IP reputation, request rate, headers and behavior. Fix the signals that give you away and block rates fall dramatically. Here's the practical checklist, roughly in order of impact.
The single biggest signal is your IP's ASN. Datacenter ranges are on every anti-bot blocklist; residential IPs from real ISPs carry ordinary-user reputation. If a site blocks you instantly regardless of how carefully you throttle, your IP type is the problem — switch to residential.
Thousands of requests from one IP is the clearest bot tell. Rotate to a fresh IP per request for wide crawls, or hold a sticky IP for flows that need continuity (login, cart, pagination). With Roam you control both from the username: a random -session-<id> rotates, a fixed one sticks.
Humans don't fire 20 requests a second on a metronome. Add delays, randomize them, and cap concurrency per target. Respect robots.txt and any published rate limits — both to stay polite and to stay unblocked.
A default library User-Agent screams 'bot'. Send a real browser User-Agent, a consistent set of Accept/Accept-Language headers, and keep them consistent with the IP's country. Don't mismatch a Japanese IP with an English-only header set.
Some targets need a real browser (Playwright, Puppeteer) to render JS and pass fingerprint checks. Pair a headless or antidetect browser with residential proxies, and give each browser profile its own sticky IP so the IP and fingerprint stay consistent.
加了请求间隔为什么还被封？
采集需要多少代理？
用代理采集违法吗？
Sign up in seconds, test your targets on real residential IPs, and only pay for what you use. $2/GB, forever-valid balance.