A scalable admin ui for spider service
Features
- Manage your spiders from a dashboard. Schedule them to run automatically
- With a single click deploy the scrapy project
- Show spider running stats
- Provide api
Current Support spider service
Screenshot
Getting Started
pip install spiderkeeper
Copy the code
Deployment
Usage: spiderkeeper [options] Options: -h, --help show this help message and exit --type=SERVER_TYPE access spider server type, Default :scrapyd --host= host host, default:0.0.0.0 --port= port port, default:5000 --server=SERVERS, default:http://localhost:6800 Visit: - web ui : http://localhost:5000 - api swagger: http://localhost:5000/api.html
Copy the code
- User Authentication
- Collect & Show scrapy crawl stats
- Optimize load balancing
Versioning
We use SemVer for versioning. For the versions available, see the tags on this repository.
Authors
- Initial work – DormyMo
See also the list of contributors who participated in this project.
License
This project is licensed under the MIT License – see the LICENSE.md file for details
Contributing
Contributions are welcomed!