nojhan/weboob-devel
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
2,919 commits 2 branches 0 tags 13 MiB
Commit graph

81 commits

Author SHA1 Message Date
Romain Bignon
5a046f00a8 create weboob.tools.browser.BrowserPasswordExpired 2012-04-21 18:22:07 +02:00
Romain Bignon
7a619a78d0 StandardBrowser.openurl uses mechanize.Browser.openurl instead of openurl_novisit 2012-04-13 15:00:12 +02:00
Romain Bignon
06599f48bc add a default timeout to 10s on connections 2012-04-12 21:50:56 +02:00
Romain Bignon
7082571acf update code documentation 2012-04-01 17:41:20 +02:00
Romain Bignon
c6a141595c change way to describe fields of CapBaseObject, and lot of documentation 2012-03-26 14:35:54 +02:00
Romain Bignon
98968d50d1 use urlparse to parse proxy url 2012-03-14 19:44:46 +01:00
Laurent Bachelier
391783a8b6 Support gzipped responses 2012-03-09 01:35:51 +01:00
Romain Bignon
5b9b1bfa16 better browser exceptions handling 
Are considered to BrowserHTTPNotFound 403 errors and BrowserStateError.
Also, during StandardBrowser.openurl(), do not retry if error is
BrowserStateError.
 2012-02-15 23:56:14 +01:00
Romain Bignon
05026b2f18 check that url isn't an unicode string (closes #784) 2012-02-07 14:49:25 +01:00
Laurent Bachelier
1621f3c3c0 Allow more freedom in page regexps 
If the user starts with ^ or ends with $, do not add them. This allows
using only $ or ^.
If it's not a string, don't do anything to it (it has to be a regexp, or
mimic one).
Use search() instead of match(). They are the same, except match implies ^.

This does not break any module, it only adds more possibilities.
 2012-02-01 23:19:01 +01:00
Laurent Bachelier
03f65bb121 Try to separate saved responses by backend 
closes #661
 2012-02-01 23:04:04 +01:00
Romain Bignon
4edeba3d62 fix crash when there isn't any photos on a profile 2011-12-12 14:37:24 +01:00
Romain Bignon
3110a1cd5a this is the end of the 'oh my fucking god' error message 2011-12-03 13:30:27 +01:00
Florent
7a2b21f1b6 Add the nologin option to "submit" 
Signed-off-by: Romain Bignon <romain@peerfuse.org>
 2011-11-29 17:14:41 +01:00
Laurent Bachelier
478696379c Try to add an extension when saving responses 2011-09-26 12:55:57 +02:00
Romain Bignon
5337e53b49 split BaseBrowser into new class StandardBrowser which is usable without inheriting it 2011-09-23 09:59:12 +02:00
Romain Bignon
0767ee40aa call 'on_loaded' only if logged 2011-07-24 17:46:56 +02:00
Romain Bignon
abb9b3dfc3 enable http and mechanize debugs before calling home() 2011-07-16 11:49:14 +02:00
Romain Bignon
ba6f31dac8 move remove_html_tags function into weboob.tools.misc 2011-07-03 10:52:33 +02:00
Romain Bignon
cb90621100 retry location after login 2011-07-02 13:22:25 +02:00
Romain Bignon
cb1478f455 check exception attribute (closes #581) 2011-05-08 13:56:00 +02:00
Laurent Bachelier
05a6bb79e4 Also log redirects when saving responses and debugging (refs #398) 
set_debug_responses is not needed as we already save the responses in a
better way.
 2011-05-08 13:55:41 +02:00
Laurent Bachelier
8075d538f0 Better messages and return codes in applications 
Use stderr and return codes >0 when it makes sense.
Loose return code conventions:
* 1 for generic errors
* 2 for user input errors
* 3 for remote errors (not found, etc.)
* 4 for not implemented
 2011-05-04 00:44:43 +02:00
Laurent Bachelier
740e7f12d2 pastebin backend: Support login 
Also add an option to Browser to avoid making useless requests.
 2011-04-30 22:28:03 +02:00
Romain Bignon
fa37ef38e3 fix to remove fucking hack from feedparser >= 5.0 (closes #649) 2011-04-24 16:58:20 +02:00
Romain Bignon
923c2ffc84 remove dead code 2011-04-23 09:04:41 +02:00
Laurent Bachelier
2c6d5d4e70 Fix previous commit on domain checking of id2url 
* the condition on empty self.DOMAIN didn't work
* it introduced a regression as subdomains were accepted by the old code
 2011-04-22 10:37:30 +02:00
Laurent Bachelier
4d8030b8dd Add check_url decorator, remove check_domain 
check_domain was not used anywhere and was bugged (the domain argument
was ignored in favor of self.DOMAIN).
The check_url decorator checks if the URL can be handled, if not it will
return None. The idea is to avoid making unnecessary requests.
 2011-04-21 22:18:17 +02:00
Laurent Bachelier
619b97669d Enhance domain checking of url2id 
* Do not check for the domain if it is empty in the Browser class
* Better check of the domain (actually parse the URL domain)
* Add docstring
 2011-04-21 20:15:18 +02:00
Laurent Bachelier
31b4892c48 Add support for https in the id2url decorator 2011-04-21 11:03:01 +02:00
Christophe Benz
a329bd3d1e Remove fragment from URLs (closes #636). 2011-04-14 16:19:51 +02:00
Laurent Bachelier
e5597424fc Add an option to change the default factory 
Changing the factory is the recommended method in the FAQ for handling
invalid HTML.
 2011-04-14 16:19:50 +02:00
Romain Bignon
9afb301ebe move select() in parser 2011-04-09 11:25:13 +02:00
Romain Bignon
7e2bb91b3b change license to AGPLv3+ 2011-04-08 12:48:07 +02:00
Juke
18281b2013 change user agent (closes #580) 2011-03-22 21:48:39 +01:00
Romain Bignon
7b61871f51 Do not use retry() with 404 errors 2011-03-19 11:24:52 +01:00
Romain Bignon
ea1dd01520 upgrade firefox from 3.0.3 to 3.6.13 2011-03-07 17:50:54 +01:00
Romain Bignon
4a7a90cb46 remove the 'refresh' feature from Browser 2011-03-06 09:58:38 +01:00
Romain Bignon
a33c15e2e3 fix when uri is empty (closes #488) 2011-03-01 08:20:41 +01:00
Romain Bignon
09bb78258a absurl() method to build an url in form protocol://domain/path with constant values of Browser 2011-02-24 21:35:32 +01:00
Johann Broudin
21a7ab33cb Correct an error in the description of the function buildurl 2011-02-07 14:15:32 +01:00
Romain Bignon
6e51b37ef1 rename class constant debug_http to DEBUG_HTTP 2010-11-29 21:07:28 +01:00
Jocelyn Jaubert
d03e4b456d Add optional debug message on http transfers 2010-11-29 10:46:15 +01:00
Christophe Benz
ff28c1bd51 create responses dir for errors 2010-11-22 15:28:32 +01:00
Christophe Benz
8724acd19d save response basename in url match file 2010-11-22 11:13:51 +01:00
Christophe Benz
0a801006dc enhance save_response option 2010-11-22 01:07:28 +01:00
Christophe Benz
27e337d520 implement developer bundle 2010-11-20 19:56:16 +01:00
Christophe Benz
afae06efc1 factorize wget user agent 2010-11-16 12:16:14 +01:00
Romain Bignon
d2f9bbb94c when a page isn't recognized, print 'Response saved to' as a warning 2010-11-11 17:24:22 +01:00
Romain Bignon
150b07b306 parameter 'no_login' to Browser.location() to don't check in we're logged 2010-10-31 00:15:25 +02:00