nojhan/weboob-devel
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
2,022 commits 2 branches 0 tags 13 MiB
Commit graph

61 commits

Author SHA1 Message Date
Romain Bignon
cb1478f455 check exception attribute (closes #581) 2011-05-08 13:56:00 +02:00
Laurent Bachelier
05a6bb79e4 Also log redirects when saving responses and debugging (refs #398) 
set_debug_responses is not needed as we already save the responses in a
better way.
 2011-05-08 13:55:41 +02:00
Laurent Bachelier
8075d538f0 Better messages and return codes in applications 
Use stderr and return codes >0 when it makes sense.
Loose return code conventions:
* 1 for generic errors
* 2 for user input errors
* 3 for remote errors (not found, etc.)
* 4 for not implemented
 2011-05-04 00:44:43 +02:00
Laurent Bachelier
740e7f12d2 pastebin backend: Support login 
Also add an option to Browser to avoid making useless requests.
 2011-04-30 22:28:03 +02:00
Romain Bignon
fa37ef38e3 fix to remove fucking hack from feedparser >= 5.0 (closes #649) 2011-04-24 16:58:20 +02:00
Romain Bignon
923c2ffc84 remove dead code 2011-04-23 09:04:41 +02:00
Laurent Bachelier
2c6d5d4e70 Fix previous commit on domain checking of id2url 
* the condition on empty self.DOMAIN didn't work
* it introduced a regression as subdomains were accepted by the old code
 2011-04-22 10:37:30 +02:00
Laurent Bachelier
4d8030b8dd Add check_url decorator, remove check_domain 
check_domain was not used anywhere and was bugged (the domain argument
was ignored in favor of self.DOMAIN).
The check_url decorator checks if the URL can be handled, if not it will
return None. The idea is to avoid making unnecessary requests.
 2011-04-21 22:18:17 +02:00
Laurent Bachelier
619b97669d Enhance domain checking of url2id 
* Do not check for the domain if it is empty in the Browser class
* Better check of the domain (actually parse the URL domain)
* Add docstring
 2011-04-21 20:15:18 +02:00
Laurent Bachelier
31b4892c48 Add support for https in the id2url decorator 2011-04-21 11:03:01 +02:00
Christophe Benz
a329bd3d1e Remove fragment from URLs (closes #636). 2011-04-14 16:19:51 +02:00
Laurent Bachelier
e5597424fc Add an option to change the default factory 
Changing the factory is the recommended method in the FAQ for handling
invalid HTML.
 2011-04-14 16:19:50 +02:00
Romain Bignon
9afb301ebe move select() in parser 2011-04-09 11:25:13 +02:00
Romain Bignon
7e2bb91b3b change license to AGPLv3+ 2011-04-08 12:48:07 +02:00
Juke
18281b2013 change user agent (closes #580) 2011-03-22 21:48:39 +01:00
Romain Bignon
7b61871f51 Do not use retry() with 404 errors 2011-03-19 11:24:52 +01:00
Romain Bignon
ea1dd01520 upgrade firefox from 3.0.3 to 3.6.13 2011-03-07 17:50:54 +01:00
Romain Bignon
4a7a90cb46 remove the 'refresh' feature from Browser 2011-03-06 09:58:38 +01:00
Romain Bignon
a33c15e2e3 fix when uri is empty (closes #488) 2011-03-01 08:20:41 +01:00
Romain Bignon
09bb78258a absurl() method to build an url in form protocol://domain/path with constant values of Browser 2011-02-24 21:35:32 +01:00
Johann Broudin
21a7ab33cb Correct an error in the description of the function buildurl 2011-02-07 14:15:32 +01:00
Romain Bignon
6e51b37ef1 rename class constant debug_http to DEBUG_HTTP 2010-11-29 21:07:28 +01:00
Jocelyn Jaubert
d03e4b456d Add optional debug message on http transfers 2010-11-29 10:46:15 +01:00
Christophe Benz
ff28c1bd51 create responses dir for errors 2010-11-22 15:28:32 +01:00
Christophe Benz
8724acd19d save response basename in url match file 2010-11-22 11:13:51 +01:00
Christophe Benz
0a801006dc enhance save_response option 2010-11-22 01:07:28 +01:00
Christophe Benz
27e337d520 implement developer bundle 2010-11-20 19:56:16 +01:00
Christophe Benz
afae06efc1 factorize wget user agent 2010-11-16 12:16:14 +01:00
Romain Bignon
d2f9bbb94c when a page isn't recognized, print 'Response saved to' as a warning 2010-11-11 17:24:22 +01:00
Romain Bignon
150b07b306 parameter 'no_login' to Browser.location() to don't check in we're logged 2010-10-31 00:15:25 +02:00
Romain Bignon
84b4003bf4 use several loggers for parts of weboob 2010-10-30 16:09:33 +02:00
Romain Bignon
73d33c0597 fix when HTTP server returns shit (closes #406) 2010-10-16 17:32:29 +02:00
Romain Bignon
570459f4e2 if page is not recognized, save response even if -a is not supplied 2010-10-16 10:05:23 +02:00
Romain Bignon
5794973932 new backend 'redmine' (implements ICapContent) 2010-10-15 16:32:06 +02:00
Romain Bignon
0b7ef16103 readurl() can take same arguments than openurl() 2010-10-14 14:08:21 +02:00
Laurent Bachelier
0cdcd10b52 Fix syntax error with Python<2.6 2010-09-30 23:49:15 +02:00
Laurent Bachelier
a301177175 Also hook in the readurl method 2010-09-30 23:28:33 +02:00
Laurent Bachelier
da55da0d9d Optional global argument to save responses 
By adding -S or --save-responses to an application's arguments, every
response will be saved in /tmp/weboob. This is best used with the
-d/--debug option.
It might be interesting to add the headers to the saved response.
 2010-09-30 23:28:33 +02:00
Christophe Benz
8afff42465 compare instance to basestring instead of (str, unicode) 2010-08-17 19:33:06 +02:00
Romain Bignon
404d7290ba a lot of pyflakes and pylint fixes 2010-08-13 22:23:10 +02:00
Romain Bignon
30cc736fbc new function get_document 2010-08-13 10:43:22 +02:00
Romain Bignon
9b02bad7ed fix when the DOMAIN is None 2010-08-11 16:00:39 +02:00
Laurent Bachelier
05b9bbdb2e Fix for ClientForm in mechanize >=0.2 
refs #362
 2010-08-10 18:15:22 +02:00
Christophe Benz
46f4de696e handle read URL data failures 2010-08-09 20:29:07 +02:00
Christophe Benz
70d618862a add android user agent 2010-07-20 16:47:41 +02:00
Romain Bignon
08700f4a31 new exception BrowserHTTPError, retry only when this one is raised 
For example, on aum, the exception AdopteWait is derived from
BrowserUnavailable, which is normal. But when AdopteWait is raised, we
don't care about retry.

So only HTTP errors attempt to retry.
 2010-07-15 23:47:01 +02:00
Romain Bignon
59169cd281 BaseBrowser.openurl uses the open_novisit() method 2010-07-15 22:02:23 +02:00
Christophe Benz
3175883351 implement fillobj() browser method 2010-07-15 01:21:49 +02:00
Christophe Benz
b4c672fa46 new select() helper 2010-07-14 17:14:53 +02:00
Romain Bignon
cd5085f564 moved firefox_cookies into weboob.tools.browser 2010-07-12 23:53:37 +02:00