nojhan/weboob-devel
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

browser2: Better redirection fix

Browse source 
We have to redo request building for all redirection codes even if we
don't fix them, for consistency.
And only 302 seems to be problematic.
This commit is contained in:
Laurent Bachelier 2012-03-27 06:08:29 +02:00 • committed by Romain Bignon
commit 240abaf412
1 changed files with 16 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

27
weboob/tools/browser2/browser.py
View file

@ -140,36 +140,41 @@ class BaseBrowser(object):
""" """
TL;DR: Web browsers and web developers suck. TL;DR: Web browsers and web developers suck.
Most browsers do not follow the RFC for HTTP 301 and 302 Most browsers do not follow the RFC for HTTP 302
but python-requests does. but python-requests does.
And web developers assume we don't follow it either. And web developers assume we don't follow it either:
https://en.wikipedia.org/wiki/Post/Redirect/Get https://en.wikipedia.org/wiki/Post/Redirect/Get
Gets a Response, and returns a new Response. Gets a Response, and returns a new Response.
Used as a 'response' hook for python-requests. Used as a 'response' hook for python-requests.
This is a hack, it would be better as an option in python-requests. This is a hack, it would be better as an option in python-requests.
What we do is run again the response building, but this time with allow_redirects,
and with a fake method and data if we have a HTTP 302.
""" """
request = response.request request = response.request
# If the request wasn't redirected, and is a redirection, # If the request wasn't redirected, and is a redirection,
# and we allowed it to be fixed, # and we allowed it to be fixed,
# restart the request building, but with a changed action. # restart the request building, but with a changed action.
if request.allow_redirects is False \ if request.allow_redirects is False \
and request.response.status_code in (codes.moved, codes.found) \ and request.response.status_code in requests.models.REDIRECT_STATI \
and request.config.get('fix-redirect'): and request.config.get('fix-redirect'):
# force the next request to be GET if request.response.status_code is codes.found:
real_method = request.method # force the next request to be GET
request.method = 'GET' real_method = request.method
real_data = request.data request.method = 'GET'
request.data = None real_data = request.data
request.data = None
# build the response again # build the response again
request.allow_redirects = True request.allow_redirects = True
request._build_response(response.raw) request._build_response(response.raw)
# restore info if request.response.status_code is codes.found:
request.method = real_method # restore info
request.data = real_data request.method = real_method
request.data = real_data
return request.response return request.response