Issue in 2014
scrapy/scrapyd/issues/43
Pull request in 2019
scrapy/scrapyd/pull/326
The trial
- Installation:
pip install -U git+https://github.com/my8100/scrapyd.git@add_basic_auth
- Updating the Configuration Filescrapyd.confFor details about other configuration items, seeThe official documentation
[scrapyd]
username = yourusername
password = yourpassword
Copy the code
- Activation:
scrapyd
In [1]: import requests
In [2]: requests.get('http://127.0.0.1:6800/').status_code
Out[2]: 401
In [3]: requests.get('http://127.0.0.1:6800/', auth=('admin'.'admin')).status_code
Out[3]: 401
In [4]: requests.get('http://127.0.0.1:6800/', auth=('yourusername'.'yourpassword')).status_code
Out[4]: 200
Copy the code
- Since the latest GitHub submission for Scrapyd has reconfigured the Jobs page, if you are using ScrapydWeb to manage Scrapyd, you need to update your ScrapydWeb synchronously:
pip install -U git+https://github.com/my8100/scrapydweb.git
GitHub
my8100/scrapyd