Florent
36cc82d34a
Traling whitespace and pep8 cleaning
2014-03-10 20:18:22 +01:00
Yann Rouillard
9010ffb025
add support for videos available at kids.dailymotion.com
2014-03-10 20:18:22 +01:00
Romain Bignon
158e8c5fdc
add dependencie for requests
2014-03-10 14:28:53 +01:00
Romain Bignon
af9197fba7
upgrade to browser2
2014-03-10 13:11:08 +01:00
Romain Bignon
1b2d3cfe48
upgrade to browser2
2014-03-10 13:11:07 +01:00
Romain Bignon
8b9073de12
add specific filters for browser2
2014-03-10 13:11:07 +01:00
Romain Bignon
0a0865542a
temp hack to support new browser
2014-03-10 13:11:07 +01:00
Romain Bignon
8d7f613f59
do not require an id, because of ItemElement which constructs objects without arguments
2014-03-10 13:11:07 +01:00
Romain Bignon
dd3b8277d0
import regex_helper from django
2014-03-10 13:11:07 +01:00
Romain Bignon
8ea92461dc
add PagesBrowser, ListElement, ItemElement, TableElement, and filters
2014-03-10 13:11:07 +01:00
Romain Bignon
5e199bdfa9
fix browser2 to be compliant with python-requests >= 2.0
2014-03-10 13:11:07 +01:00
Laurent Bachelier
61e7b93779
Some PEP8 fixes
2014-03-10 13:11:07 +01:00
Laurent Bachelier
1ff07273b3
Add base URL restriction
...
For security reasons (SSL only, no leakage, etc.)
2014-03-10 13:11:07 +01:00
Laurent Bachelier
b701d9519a
browser2: Updates for latest python-requests developments
2014-03-10 13:11:07 +01:00
Laurent Bachelier
53c298b069
browser2: Update cookies.py
2014-03-10 13:11:07 +01:00
Laurent Bachelier
d50784faae
browser2: Minor rewrites
2014-03-10 13:11:07 +01:00
Laurent Bachelier
f7fabfeb7d
browser2: More tests for absolute URLs
2014-03-10 13:11:07 +01:00
Laurent Bachelier
1c404639c1
browser2: Separate CookieJar from CookiePolicy
...
And avoid methods starting by _!
2014-03-10 13:11:07 +01:00
Laurent Bachelier
083caf15e5
browser2: Fix call to wrong method name
2014-03-10 13:11:07 +01:00
Laurent Bachelier
e793fd14da
browser2: Use cookie testing services
...
Both are aimed at real browsers.
This also adds a "real-life" cookie deletion/expiration test.
2014-03-10 13:11:07 +01:00
Laurent Bachelier
26025d725d
browser2: Document why all this for handling cookies
2014-03-10 13:11:07 +01:00
Laurent Bachelier
ef379f830e
browser2: Fix cookie security in redirections
...
With tests (another reason why it's good to write them).
2014-03-10 13:11:07 +01:00
Laurent Bachelier
83e08c103d
browser2: Add easy cookie building
2014-03-10 13:11:07 +01:00
Laurent Bachelier
ec64a52643
browser2: Misc doc/refactor
2014-03-10 13:11:06 +01:00
Laurent Bachelier
baef03c198
browser2: Test cookie override
2014-03-10 13:11:06 +01:00
Laurent Bachelier
2d70d11822
browser2: Handle cookies in the request/response cycle
...
Unlike the one by python-requests, it is secure, and generally behaves
like a real browser.
Basic test added.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
57e16e9fe4
browser2: Handle redirects internally
...
python-requests isn't secure enough, and some behavior depends on the
latest version.
Tested on 0.10.6.
So instead of the previous hack, we have some copy-paste.
But we gain secure cookies handling (not there yet),
referrer handling, "proper" redirect on POST behavior.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
4b802f32dd
browser2: Evolve _fix_redirect for latest python-requests
...
See commits:
e831947be43bc97b39d6e6e573cb0842b67fea80
dacc1d8aafedf80f8736d794005b6c8e8b1d4ad1
b20d171c1c11ecee83b7e4a0beead1e54902d37b
The entire redirect thingy will have to be overriden anyway, as it is
still broken. Cookies are given domain to domain without any checks.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
db304b955c
browser2: Handle cookie expiration, session cookies
...
Every related method accepts a "now" parameter. If provided, it will be
used instead of the system time.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
f17e61d73f
browser2: PostBin is now disabled
...
But RequestBin can't help test the bug.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
295e07b3ed
browser2: Add CookieJar and related cookie handling
...
Not used by Browser for now, but with very detailed tests.
Does not handle expirations yet.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
cd9b9300bd
browser2: Import package "cookies"
...
I certainly don't like doing this, but:
* It is the only cookie lib that does not completely suck.
* It handles multiple cookies in a line, parses flags correctly.
* Clean API
* "One file, so you can just drop cookies.py into your project if you like"
* MIT license
Despite being the best cookie lib available, it isn't well-packaged.
And since for now some light monkeypatching is done, it is even more
interesting to import it.
http://pypi.python.org/pypi/cookies
2014-03-10 13:11:06 +01:00
Laurent Bachelier
538c0ee92b
browser2: Handle Referrer, more documentation
2014-03-10 13:11:06 +01:00
Laurent Bachelier
d2c4085508
browser2: More tests
2014-03-10 13:11:06 +01:00
Laurent Bachelier
42b5eeeb77
browser2: Try to have extensive documentation
2014-03-10 13:11:06 +01:00
Laurent Bachelier
240abaf412
browser2: Better redirection fix
...
We have to redo request building for all redirection codes even if we
don't fix them, for consistency.
And only 302 seems to be problematic.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
92073a2cf1
browser2: Better tests
...
Allow using local services, add RequestBin support.
Still using PostBin for the empty POST issue.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
b70d722c03
browser2: Add support for relative URLs / BASEURL
2014-03-10 13:11:06 +01:00
Laurent Bachelier
60d1d03c1c
browser2: Handle server issues of postbin.org
...
And move tests outside.
2014-03-10 13:11:06 +01:00
Laurent Bachelier
72cbc5bd8b
browser2: Add a default timeout
...
Because no timeout is pretty dangerous
2014-03-10 13:11:06 +01:00
Laurent Bachelier
a1b7232521
browser2: Browser load urls, post, track state
...
With working tests!
2014-03-10 13:11:06 +01:00
Laurent Bachelier
10be4fb695
youtube-dl: Fix returning the best format
2014-03-08 20:20:07 +01:00
Romain Bignon
77ee2a53a2
fix navigation
2014-03-07 10:51:54 +01:00
Romain Bignon
14de4cbeb8
fix login error detection
2014-03-07 10:51:43 +01:00
Romain Bignon
35cdd3b69a
support LCR
2014-03-07 10:42:43 +01:00
Romain Bignon
d82cc615a3
steal youtube-dl code to fix youtube module
2014-03-04 21:02:05 +01:00
Romain Bignon
c85176f9b1
forge a random IP source address to avoid redirection to other website
2014-03-04 11:56:01 +01:00
Vincent A
43ec8a8a00
boilerplate: fix variable misuse in 'cap' recipe
2014-03-02 21:32:43 +01:00
Romain Bignon
ec10ec3382
add module alloresto
2014-03-02 20:58:16 +01:00
Vincent A
7b6c5e8544
boilerplate: add 'cap' recipe to prepare methods
2014-03-02 20:57:28 +01:00
