CVE-2024-3572: XML External Entity (XXE) Vulnerability in scrapy/scrapy
Duplicate Advisory This advisory has been withdrawn because it is a duplicate of GHSA-7j7m-v7m3-jqm7. This link is maintained to preserve external references.
Original Description The scrapy/scrapy project is vulnerable to XML External Entity (XXE) attacks due to the use of lxml.etree.fromstring for parsing untrusted XML data without proper validation. This vulnerability allows attackers to perform denial of service attacks, access local files, generate network connections, or circumvent firewalls by submitting specially crafted XML data.
Other sources
Impact
Scrapy limits allowed response sizes by default through the DOWNLOADMAXSIZE and DOWNLOADWARNSIZE settings.
However, those limits were only being enforced during the download of the raw, usually-compressed response bodies, and not during decompression, making Scrapy vulnerable to decompression bombs.
A malicious website being scraped could send a small response that, on decompression, could exhaust the memory available to the Scrapy process, potentially affecting any other process sharing that memory, and affecting disk usage in case of uncompressed response caching.
Patches
Upgrade to Scrapy 2.11.1.
If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.11.1 is not an option, you may upgrade to Scrapy 1.8.4 instead.
Workarounds
There is no easy workaround.
Disabling HTTP decompression altogether is impractical, as HTTP compression is a rather common practice.
However, it is technically possible to manually backport the 2.11.1 or 1.8.4 fix, replacing the corresponding components of an unpatched version of Scrapy with patched versions copied into your own code.
Acknowledgements
This security issue was reported by @dmandefy through huntr.com.
— GitHub
Affected Software
Remediation
Event History
Frequently Asked Questions
What is the severity of CVE-2024-3572?
CVE-2024-3572 is classified as a moderate severity vulnerability due to its potential for XML External Entity (XXE) attacks.
How do I fix CVE-2024-3572?
To mitigate CVE-2024-3572, upgrade to Scrapy version 2.11.1 or 1.8.4 or later.
Which versions of Scrapy are affected by CVE-2024-3572?
CVE-2024-3572 affects Scrapy versions below 2.11.1 and 1.8.4.
What type of vulnerability is CVE-2024-3572?
CVE-2024-3572 is an XML External Entity (XXE) vulnerability.
Is CVE-2024-3572 related to any other advisories?
CVE-2024-3572 has been withdrawn as it is a duplicate of GHSA-7j7m-v7m3-jqm7.