nojhan/weboob-devel
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

boobot: Force shitty Windows encoding

Browse source 
Because web browsers actually do that!

Test URL: http://www.universfreebox.com/article/20715/Deutche-Telekom-met-fin-a-l-internet-illimite-bientot-le-tour-de-la-France
This commit is contained in:
Laurent Bachelier 2013-05-05 20:43:58 +02:00
commit 1b2031796b
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
contrib/boobot.py
View file

@ -101,7 +101,7 @@ class BoobotBrowser(StandardBrowser):
title = None
if is_html:
r = self.openurl(url)
encoding = EncodingFinder('windows-1252').encoding(r)
encoding = EncodingFinder('windows-1252').encoding(r).lower().replace('iso-8859-1', 'windows-1252')
h = self.get_document(r, parser='lxml', encoding=encoding)
for title in h.xpath('//head/title'):
title = to_unicode(title.text_content())