Pipeline orchestrator #4
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
MiningPipeline chains: URL list -> DomainThrottle -> Firecrawl -> sibyl-extractor -> quality gate -> SibylStore. CLI via web_miner/cli.py with --file, --rate, --delay, --min-words, --max options. Tests: 10/10 passing.