Domain-level politeness throttle #1
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
DomainThrottle with configurable req/sec, exponential backoff on failures, per-domain tracking. Thread-safe. Files: web_miner/politeness.py, tests/test_miner.py (8 tests). Status: implemented/tested.