{"full_name":"scrapy/scrapy","owner":"scrapy","name":"scrapy","shard":"1","url":"https://github.com/scrapy/scrapy","description":"Scrapy, a fast high-level web crawling & scraping framework for Python.","homepage":"https://scrapy.org","topics":["python","scraping","crawling","framework","crawler","hacktoberfest","web-scraping","web-scraping-python"],"tag_pages":["crawler","crawling","framework","hacktoberfest","python","scraping","web-scraping"],"language":"Python","license":"BSD-3-Clause","avatar":"https://avatars.githubusercontent.com/u/733635?v=4","created_at":"2010-02-22T02:01:14Z","pushed_at":"2026-11-13T14:45:54Z","metrics":{"stars":65415,"forks":12146,"watchers":1744,"issues":3325,"prs":5052,"releases":55,"issues_open":99,"issues_closed":3226,"prs_open":73,"prs_closed":1789,"prs_merged":3190,"discussions":25},"growth":{"day":{"stars":26,"forks":6,"base_day":"2026-11-15"},"week":{"stars":128,"forks":35,"base_day":"2026-11-09"}},"history":{"dates":["2026-11-09","2026-11-10","2026-11-11","2026-11-12","2026-11-13","2026-11-14","2026-11-15","2026-11-16"],"stars":[65287,65305,65326,65343,65350,65365,65389,65415],"forks":[12111,12113,12120,12123,12124,12132,12140,12146],"watchers":[1743,1743,1743,1743,1743,1744,1744,1744],"issues_open":[null,null,103,103,99,99,99,99],"prs_open":[null,null,77,76,74,72,73,73]},"boards":{},"same_owner":[],"issues":[{"number":1306,"title":"Speedup & fix URL parsing","comments":39,"created_at":"2015-06-15T14:23:48Z"},{"number":6597,"title":"Scrapy needs to have sane defaults that do no harm","comments":32,"created_at":"2025-01-01T08:45:45Z"},{"number":4240,"title":"Wrong type(response) for binary responses","comments":28,"created_at":"2019-12-17T14:51:59Z"},{"number":1144,"title":"Awaiting on Requests in callbacks directly","comments":25,"created_at":"2015-04-10T11:50:01Z"},{"number":356,"title":"Support list type arguments from command line.","comments":24,"created_at":"2013-07-24T07:12:58Z"},{"number":1988,"title":"\"Content-Encoding\" header gets stripped from response headers","comments":20,"created_at":"2016-05-12T15:26:02Z"},{"number":1225,"title":"dupefilter skips a request when a page is redirected to itself ","comments":20,"created_at":"2015-05-12T18:10:15Z"},{"number":713,"title":"Bring docstrings back","comments":17,"created_at":"2014-05-07T09:34:54Z"}],"discussions":[{"number":8368,"title":"PR automatically flagged as spam, looking for maintainer help","comments":1,"category":"Q&A","created_at":"2026-10-17T14:17:18Z"},{"number":7582,"title":"Choosing Issue","comments":1,"category":"Q&A","created_at":"2026-06-08T03:45:55Z"},{"number":7362,"title":"Newbie question about scraping","comments":3,"category":"Q&A","created_at":"2026-03-25T14:00:02Z"},{"number":7343,"title":"How to Automatically Retrieve Web Resources That Require On-Site Search to Locate, Instead of Just Recursively Crawling URLs?","comments":2,"category":"Q&A","created_at":"2026-03-21T13:53:07Z"},{"number":7326,"title":"Scrapy doesn't seem to understand what 403 and 444 mean.","comments":3,"category":"Q&A","created_at":"2026-03-11T17:43:55Z"},{"number":7325,"title":"Providing Spider specific settings to custom DownloadHandlers","comments":1,"category":"Q&A","created_at":"2026-03-11T14:45:09Z"},{"number":7322,"title":"License and Attribution of the Scrapy Logo?","comments":2,"category":"Q&A","created_at":"2026-03-09T16:57:15Z"},{"number":7320,"title":"Localization of The Scrapy Documentation","comments":0,"category":"Q&A","created_at":"2026-03-09T08:05:07Z"}],"latest_release":{"name":null,"tag":"2.20.0","published_at":"2026-11-06T18:23:09Z","body":"- Web-browser-like detection of response encodings, base URLs and meta refresh redirects, powered by [frostwork](https://github.com/scrapy/frostwork)\n- Requests can be awaited to get their response\n- Non-asyncio Twisted reactors are deprecated\n\n[Full changelog](https://docs.scrapy.org/en/latest/news.html#release-2-20-0)\n","is_prerelease":false},"readme":false,"readme_zh":false,"news":[],"stale":false,"snapshot_at":"2026-11-16T09:23:14Z","history_start":"2026-09-06"}