nojhan/weboob-devel
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
5,452 commits 2 branches 0 tags 13 MiB
Commit graph

48 commits

Author SHA1 Message Date
Romain Bignon
c409675e6c fix compatibility with python2.6 2014-05-19 07:23:12 +02:00
Laurent Bachelier
14b1b56914 browser2: Add an option to convert POST data to the proper encoding 
And autodetect it on forms. There is no other way to know what is the
expected encoding.
 2014-05-19 01:01:25 +02:00
Romain Bignon
6fcac89dd5 first step in python3 support 2014-05-17 14:27:55 +02:00
Romain Bignon
55c80097fe add BaseBrowser.prepare_request virtual method 2014-04-14 16:42:39 +02:00
Romain Bignon
07f6507096 improve documentation of browser2 2014-04-12 15:02:34 +02:00
Laurent Bachelier
c4b40e85e9 Limit debug filename to a sane number of characters 2014-03-27 23:41:31 +01:00
Romain Bignon
9770b2af1c fix compatibility with requests < 2.2 2014-03-26 09:58:27 +01:00
Romain Bignon
42028c390f add a 'settings' attribute on loggers to store user values like 'save_responses', 'responses_dirname', 'ssl_insecure', etc. 2014-03-25 22:58:35 +01:00
Romain Bignon
3bf7ef62c8 fix wrong behavior with cookies handle 2014-03-22 16:06:51 +01:00
Romain Bignon
16615dd337 correctly save file content with -a 2014-03-19 20:27:09 +01:00
Johann Broudin
7621da74e6 change max_retries to 2 2014-03-14 12:30:55 +01:00
Johann Broudin
7f82f45e70 correct the usage of proxy, timeout and ssl verification in browser2 2014-03-14 12:30:55 +01:00
Laurent Bachelier
9146f2f312 Save request/response metadata with as much info as possible 2014-03-14 01:53:49 +01:00
Laurent Bachelier
3a9bc80f8a Saving cookies for debug isn't very interesting 
We have them in headers in a clearer way, at least for the purpose of
debugging.
Showing new cookies as they appear could be done, though.
 2014-03-14 01:53:49 +01:00
Laurent Bachelier
058cafe69e Save headers of response and request 2014-03-14 01:53:49 +01:00
Laurent Bachelier
6acfd75780 Save cookies on save_responses and pave the way for more good stuff 
We apparently can't override the class in the Response, even if it is
changed in the Session. Still, it will be useful to have our own class.
 2014-03-14 01:53:49 +01:00
Laurent Bachelier
95e71e329b Better filenames for save responses 2014-03-14 01:53:49 +01:00
Laurent Bachelier
fca0ab8733 Quick and dirty -a and -I support for Browser2 2014-03-13 22:04:20 +01:00
Romain Bignon
4cffb3038b raise errors on responses 2014-03-13 21:34:18 +01:00
Romain Bignon
bf8675c88b PagesBrowser.open and PagesBrowser.location always return a Response object 2014-03-13 21:27:59 +01:00
Johann Broudin
0f1726d01c add ssl proxy support to browser and browser2 and add the option _proxy_ssl 2014-03-13 20:20:02 +01:00
Johann Broudin
cafd6c8f05 Add proxy support to browser2 2014-03-13 20:20:02 +01:00
Laurent Bachelier
1e5f5940c5 Split _open() to allow request customization 2014-03-13 19:01:56 +01:00
Laurent Bachelier
b07da18081 More updates and cleanups for new requests 2014-03-13 18:07:17 +01:00
Laurent Bachelier
5711158f80 Require python-requests version 2014-03-13 17:53:01 +01:00
Laurent Bachelier
eb54528b95 Update Firefox to latest ESR 2014-03-13 17:36:44 +01:00
Laurent Bachelier
62e5d23f23 Another python-requests bug fixed in newer versions! 2014-03-13 17:32:45 +01:00
Laurent Bachelier
82c61f3668 Cleanups due to the usage of python-requests>=2.0 2014-03-13 17:18:26 +01:00
Romain Bignon
ecd3a9c5ab handle Refresh HTTP header 2014-03-12 23:41:08 +01:00
Romain Bignon
a4b7f5cdf1 user-friendly error message when python-requests is not installed 2014-03-12 10:17:20 +01:00
Romain Bignon
5e199bdfa9 fix browser2 to be compliant with python-requests >= 2.0 2014-03-10 13:11:07 +01:00
Laurent Bachelier
61e7b93779 Some PEP8 fixes 2014-03-10 13:11:07 +01:00
Laurent Bachelier
1ff07273b3 Add base URL restriction 
For security reasons (SSL only, no leakage, etc.)
 2014-03-10 13:11:07 +01:00
Laurent Bachelier
b701d9519a browser2: Updates for latest python-requests developments 2014-03-10 13:11:07 +01:00
Laurent Bachelier
d50784faae browser2: Minor rewrites 2014-03-10 13:11:07 +01:00
Laurent Bachelier
1c404639c1 browser2: Separate CookieJar from CookiePolicy 
And avoid methods starting by _!
 2014-03-10 13:11:07 +01:00
Laurent Bachelier
ef379f830e browser2: Fix cookie security in redirections 
With tests (another reason why it's good to write them).
 2014-03-10 13:11:07 +01:00
Laurent Bachelier
ec64a52643 browser2: Misc doc/refactor 2014-03-10 13:11:06 +01:00
Laurent Bachelier
2d70d11822 browser2: Handle cookies in the request/response cycle 
Unlike the one by python-requests, it is secure, and generally behaves
like a real browser.
Basic test added.
 2014-03-10 13:11:06 +01:00
Laurent Bachelier
57e16e9fe4 browser2: Handle redirects internally 
python-requests isn't secure enough, and some behavior depends on the
latest version.
Tested on 0.10.6.

So instead of the previous hack, we have some copy-paste.
But we gain secure cookies handling (not there yet),
referrer handling, "proper" redirect on POST behavior.
 2014-03-10 13:11:06 +01:00
Laurent Bachelier
4b802f32dd browser2: Evolve _fix_redirect for latest python-requests 
See commits:
e831947be43bc97b39d6e6e573cb0842b67fea80
dacc1d8aafedf80f8736d794005b6c8e8b1d4ad1
b20d171c1c11ecee83b7e4a0beead1e54902d37b

The entire redirect thingy will have to be overriden anyway, as it is
still broken. Cookies are given domain to domain without any checks.
 2014-03-10 13:11:06 +01:00
Laurent Bachelier
538c0ee92b browser2: Handle Referrer, more documentation 2014-03-10 13:11:06 +01:00
Laurent Bachelier
42b5eeeb77 browser2: Try to have extensive documentation 2014-03-10 13:11:06 +01:00
Laurent Bachelier
240abaf412 browser2: Better redirection fix 
We have to redo request building for all redirection codes even if we
don't fix them, for consistency.
And only 302 seems to be problematic.
 2014-03-10 13:11:06 +01:00
Laurent Bachelier
b70d722c03 browser2: Add support for relative URLs / BASEURL 2014-03-10 13:11:06 +01:00
Laurent Bachelier
60d1d03c1c browser2: Handle server issues of postbin.org 
And move tests outside.
 2014-03-10 13:11:06 +01:00
Laurent Bachelier
72cbc5bd8b browser2: Add a default timeout 
Because no timeout is pretty dangerous
 2014-03-10 13:11:06 +01:00
Laurent Bachelier
a1b7232521 browser2: Browser load urls, post, track state 
With working tests!
 2014-03-10 13:11:06 +01:00