The rss-fetcher tries to generate a clearer error Exception for HTML by catching ExpatError exceptions and sniffing the top of the document:
p = sitemap_parser.XMLSitemapParser(url, text)
try:
sitemap = p.sitemap()
except ExpatError: # try translateing ExpatError
top = response.content[:1024].lower()
if top.find(b'<!doctype') or top.find(b'<html'):
raise Exception("html?")
raise
Maybe pull this up into XMLSitemapParser.sitemap so that mc_sitemap_tools.discover (etc) can also benefit?
The rss-fetcher tries to generate a clearer error Exception for HTML by catching
ExpatErrorexceptions and sniffing the top of the document:Maybe pull this up into
XMLSitemapParser.sitemapso thatmc_sitemap_tools.discover(etc) can also benefit?