JobAlfredBot
About our crawler
JobAlfredBot reads public job postings so that people can find them. It sends applicants straight to your own job pages.
What it reads
Public job boards, career pages and the job data employers publish for search engines. It keeps the title, company, location, country, whether the role is remote, the posting link, the date, and the salary when it is published. It does not store job descriptions.
How it behaves
- It always identifies itself as
JobAlfredBot/1.0 (+https://jobalfred.com/bot). - It reads your robots.txt and obeys it, including Crawl-delay.
- It limits how many requests it sends to each host.
- If a site answers 403, 429 or a bot challenge, it leaves that host alone for the rest of the run.
- It never logs in, never solves a CAPTCHA, and uses no proxies or disguised browsers.
- Links to your jobs are your own addresses, unchanged, with nothing added.
How to opt out
Add this to your robots.txt and it stops at its next run:
User-agent: JobAlfredBot
Disallow: /To ask us to remove your roles, or to report a problem, write to support@jobpilot.pt.
The crawler's code is public on GitHub.