暗网采集(新版) ——推送到采集平台的数据字段都和新闻的一样
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

24 lines
531 B

6 months ago
  1. #!/usr/bin/env python
  2. # -*- coding:utf-8 -*-
  3. import schedule
  4. import time
  5. from scrapy.cmdline import execute
  6. def crawl_1_news():
  7. # news_denkbares新闻网站采集启动
  8. execute(['scrapy', 'crawl', 'news_denkbares'])
  9. def crawl_2_pot():
  10. # shop_pot 毒品网站采集启动
  11. execute(['scrapy', 'crawl', 'shop_pot'])
  12. # 每周五凌晨五点运行程序
  13. schedule.every().friday.at('5:00').do(crawl_1_news)
  14. schedule.every().friday.at('7:00').do(crawl_2_pot)
  15. while True:
  16. schedule.run_pending()
  17. time.sleep(1)