#python#crawler#crawling#framework#hacktoberfest#python#scraping#web_scraping#web_scraping_python
Scrapy is a powerful tool for extracting data from websites. It works on many platforms and requires Python 3.9 or higher. Scrapy is free, stable, and can handle complex tasks efficiently. It allows you to manage multiple requests at once, making it fast and efficient for large-scale data extraction. Scrapy also supports various formats for storing data and has features like auto-throttling to prevent overwhelming websites. This makes it a great choice for users who need to collect data from many websites quickly and reliably.
https://github.com/scrapy/scrapy
RsS iS dEaD LOL: discover RSS Feeds of your follows on Mastodon
频道曾经提及过一个叫 FeedsMage 的服务,用于从你 fo 的推友的 Bio 里找链接,再从链接里找 Feed ,最后可生成一个 #OPML 文件。RsS iS dEaD LOL 则是长毛象版的 FeedsMage,从你 fo 的 Fediverse 用户的 Bio 里找链接,发现 RSS,然后可生成 #OPML:
https://rss-is-dead.lol/
例如我的:
https://rss-is-dead.lol/user?profileUrl=https%3A%2F%2Fmastodon.social%2Fusers%2FAboutRSS
发现于作者嘟文:
https://mastodon.social/@paulcuth/112178886374464145