Commit graph

5,024 commits

Author SHA1 Message Date
Laurent Bachelier
e793fd14da browser2: Use cookie testing services
Both are aimed at real browsers.
This also adds a "real-life" cookie deletion/expiration test.
2014-03-10 13:11:07 +01:00
Laurent Bachelier
26025d725d browser2: Document why all this for handling cookies 2014-03-10 13:11:07 +01:00
Laurent Bachelier
ef379f830e browser2: Fix cookie security in redirections
With tests (another reason why it's good to write them).
2014-03-10 13:11:07 +01:00
Laurent Bachelier
83e08c103d browser2: Add easy cookie building 2014-03-10 13:11:07 +01:00
Laurent Bachelier
ec64a52643 browser2: Misc doc/refactor 2014-03-10 13:11:06 +01:00
Laurent Bachelier
baef03c198 browser2: Test cookie override 2014-03-10 13:11:06 +01:00
Laurent Bachelier
2d70d11822 browser2: Handle cookies in the request/response cycle
Unlike the one by python-requests, it is secure, and generally behaves
like a real browser.
Basic test added.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
57e16e9fe4 browser2: Handle redirects internally
python-requests isn't secure enough, and some behavior depends on the
latest version.
Tested on 0.10.6.

So instead of the previous hack, we have some copy-paste.
But we gain secure cookies handling (not there yet),
referrer handling, "proper" redirect on POST behavior.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
4b802f32dd browser2: Evolve _fix_redirect for latest python-requests
See commits:
e831947be43bc97b39d6e6e573cb0842b67fea80
dacc1d8aafedf80f8736d794005b6c8e8b1d4ad1
b20d171c1c11ecee83b7e4a0beead1e54902d37b

The entire redirect thingy will have to be overriden anyway, as it is
still broken. Cookies are given domain to domain without any checks.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
db304b955c browser2: Handle cookie expiration, session cookies
Every related method accepts a "now" parameter. If provided, it will be
used instead of the system time.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
f17e61d73f browser2: PostBin is now disabled
But RequestBin can't help test the bug.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
295e07b3ed browser2: Add CookieJar and related cookie handling
Not used by Browser for now, but with very detailed tests.
Does not handle expirations yet.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
cd9b9300bd browser2: Import package "cookies"
I certainly don't like doing this, but:
* It is the only cookie lib that does not completely suck.
* It handles multiple cookies in a line, parses flags correctly.
* Clean API
* "One file, so you can just drop cookies.py into your project if you like"
* MIT license

Despite being the best cookie lib available, it isn't well-packaged.

And since for now some light monkeypatching is done, it is even more
interesting to import it.

http://pypi.python.org/pypi/cookies
2014-03-10 13:11:06 +01:00
Laurent Bachelier
538c0ee92b browser2: Handle Referrer, more documentation 2014-03-10 13:11:06 +01:00
Laurent Bachelier
d2c4085508 browser2: More tests 2014-03-10 13:11:06 +01:00
Laurent Bachelier
42b5eeeb77 browser2: Try to have extensive documentation 2014-03-10 13:11:06 +01:00
Laurent Bachelier
240abaf412 browser2: Better redirection fix
We have to redo request building for all redirection codes even if we
don't fix them, for consistency.
And only 302 seems to be problematic.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
92073a2cf1 browser2: Better tests
Allow using local services, add RequestBin support.
Still using PostBin for the empty POST issue.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
b70d722c03 browser2: Add support for relative URLs / BASEURL 2014-03-10 13:11:06 +01:00
Laurent Bachelier
60d1d03c1c browser2: Handle server issues of postbin.org
And move tests outside.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
72cbc5bd8b browser2: Add a default timeout
Because no timeout is pretty dangerous
2014-03-10 13:11:06 +01:00
Laurent Bachelier
a1b7232521 browser2: Browser load urls, post, track state
With working tests!
2014-03-10 13:11:06 +01:00
Laurent Bachelier
10be4fb695 youtube-dl: Fix returning the best format 2014-03-08 20:20:07 +01:00
Romain Bignon
77ee2a53a2 fix navigation 2014-03-07 10:51:54 +01:00
Romain Bignon
14de4cbeb8 fix login error detection 2014-03-07 10:51:43 +01:00
Romain Bignon
35cdd3b69a support LCR 2014-03-07 10:42:43 +01:00
Romain Bignon
d82cc615a3 steal youtube-dl code to fix youtube module 2014-03-04 21:02:05 +01:00
Romain Bignon
c85176f9b1 forge a random IP source address to avoid redirection to other website 2014-03-04 11:56:01 +01:00
Vincent A
43ec8a8a00 boilerplate: fix variable misuse in 'cap' recipe 2014-03-02 21:32:43 +01:00
Romain Bignon
ec10ec3382 add module alloresto 2014-03-02 20:58:16 +01:00
Vincent A
7b6c5e8544 boilerplate: add 'cap' recipe to prepare methods 2014-03-02 20:57:28 +01:00
Romain Bignon
7502310d7e fix crash when the card history is empty 2014-03-01 19:23:06 +01:00
Raphaël Rigo
9d211201c7 card history parser
Signed-off-by: Raphaël Rigo <devel-git@syscall.eu>
Signed-off-by: Romain Bignon <romain@symlink.me>
2014-03-01 19:19:27 +01:00
Raphaël Rigo
31af964696 Handle card history
Signed-off-by: Raphaël Rigo <devel-git@syscall.eu>
Signed-off-by: Romain Bignon <romain@symlink.me>
2014-03-01 19:19:27 +01:00
Vincent Paredes
5cc41a78b7 increasing range for ChaoticDateGuesser 2014-03-01 18:15:45 +01:00
Romain Bignon
7c64ad162c fix parsing accounts on specific regions 2014-03-01 18:14:03 +01:00
Bezleputh
7bd059b99b [senscritique] fix : site changed 2014-02-28 13:43:21 +01:00
Florent
695b98ee79 Always success in untrack if the id is in the storage 2014-02-27 16:12:33 +01:00
Florent
2a0d4a5e7f Do not try to get status of not loaded backends 2014-02-27 16:03:57 +01:00
Romain Bignon
5345645b0f fix accounts parsing 2014-02-27 08:28:25 +01:00
Bezleputh
95ca604472 convert string to unicode to avoid warning message 2014-02-26 20:12:00 +01:00
Bezleputh
7492c0bdc4 [lefigaro] fix : site changed 2014-02-26 20:11:59 +01:00
Bezleputh
32e265a943 [ina] remove useless commented line 2014-02-26 20:11:59 +01:00
Romain Bignon
2af2a3ee08 add Banque Rhône-Alpes 2014-02-26 18:39:58 +01:00
Vincent Paredes
fee1fe96b6 increase the mode parameter to get more history 2014-02-26 18:37:52 +01:00
Vincent Paredes
b9c8108d04 adding support for leroy merlin 2014-02-26 18:37:52 +01:00
Vincent Paredes
158efcded0 adding ChaoticDateGuesser when you know an interval 2014-02-26 18:37:52 +01:00
Vincent Paredes
64ef0de61d fixing date not in the good interval 2014-02-26 18:35:31 +01:00
Vincent Paredes
9855ae8fa9 fixing wrong date guessing 2014-02-26 18:35:31 +01:00
Romain Bignon
0ce45770fb certificate changed 2014-02-26 07:43:54 +01:00