nojhan/weboob-devel
Watch
1
0
Fork
You've already forked weboob-devel
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Rename generic-munin to weboob-generic. Move documentation

Browse source 
Fixes #1355
This commit is contained in:
Florent 2014-09-01 12:29:21 +02:00
commit 58b0977ad9
2 changed files with 140 additions and 163 deletions
Download patch file Download diff file Expand all files Collapse all files

163
contrib/munin/generic/README
View file

@ -1,163 +0,0 @@
How to use it
1)
Create a symlink from /etc/munin/plugins/nameyouwant to the script
the "nameyouwant" is used to create cache directory is ~/.config/weboob/munin and is usefull to configure the plugin.
2)
Configure the plugin in /etc/munin/plugin-conf.d/ (you can go at the end of this files for some examples).
=== Mandatory options are: ===
==== env.capa ====
Example: env.capa CapBank
The Weboob capability to load.
==== env.do ====
Example: env.do get_balance
The Weboob command to call the capability. It can take more than one argument. With two argument, the second is used as parameter for the command. The third is used to restrict backends.
If you use get_object_list option, you can use only one argument in env.do.
==== env.import ====
Example: from weboob.capabilities.bank import CapBank
The import line to import the capability
==== env.attribvalue ====
Example: env.attribvalue balance
The attribut name of objects that we get with the "do" command. In the example, we get some Account objects and the relevant value is "balance".
If the attribut is itself one object, it is possible the use a hierarchical call (with / as separator). To get the temperature of a Current object use "temp/value".
=== Optionnals options are ===
==== env.id_monitored ====
Example: env.id_monitored account1@backend1 account2@backend2
A list of object (space as separator). Only there objects are diplayed.
==== env.exclude ====
Example: env.exclude 550810@sachsen
Same as env.id_monitored, but to exclude an object of results
==== env.cache_expire ====
Example: env.cache_expire 7200 (default 3600)
To avoid site flooding, we use a cache in ~/.config/weboob/munin/ The default value is to get new value every 3600 seconds, you can change this delay with this option.
==== env.cumulate ====
Example: env.cumulate 0 (default 1)
Display data in Area mode (default) or in line mode.
==== env.get_object_list ====
Exemple: env.get_object_list="iter_subscriptions"
Run a first time a cap's method to get a list of objects. env.do is then applied on all results.
==== env.attribid ====
Example: env.attribid id
Munin need an id for each value. The default is to use the id of results objects, but you can configure an other attribut. Like for env.attribvalue, you can configure a hierarchical attribut with / as separator, for example "temp/id".
==== env.title ====
Example: env.title a wonderful graph
A title for the graph (default: nothing)
==== env.vlabel ====
Example: env.vlabel Balance
A vlabel for the graph (default: env.attribvalue)
==== env.label ====
Example: env.label id
Each data in munin as a label. Per default, we take the "label" attribute of object, but it is not always possible (label attribute does not exist, etc). Configure the attribut to use as label here.
3) restart munin-node
Examples:
# Like boobank-munin does
# Only for the example, you should use boobank-munin instead
[bank]
user florent
group florent
env.cache_expire 7200
env.HOME /home/flo
env.capa CapBank
env.do iter_accounts
env.import from weboob.capabilities.bank import CapBank
env.attribvalue balance
env.title Solde des comptes
# Balance of your leclercmobile subscription
[leclercmobile]
user florent
group florent
env.cache_expire 16800
env.HOME /home/flo
env.capa CapBill
env.do get_balance,06XXXXXXXX,leclercmobile
env.import from weboob.capabilities.bill import CapBill
env.attribvalue price
env.title Forfait leclercmobile
env.vlabel Solde
Result: http://fourcot.fr/weboob/leclercmobile-day.png
# Monitor water level in Dresden
[leveldresden]
user florent
group florent
env.cache_expire 7200
env.HOME /home/flo
env.capa CapGauge
env.do get_last_measure,501060-level
env.import from weboob.capabilities.gauge import CapGauge
env.attribvalue level
env.title Niveau de l'elbe
env.label id
# The level of the elbe in all Sachsen's cities
[levelelbesachsen]
user florent
env.cache_expire 800
env.HOME /home/flo
env.cumulate 0
env.capa CapGauge
env.do iter_gauges,Elbe,sachsen
env.import from weboob.capabilities.gauge import CapGauge
env.attribvalue sensors/lastvalue/level
env.title Niveau de l'elbe en Saxe
env.label name
env.vlabel Hauteur du fleuve (cm)
env.exclude 550810@sachsen
Result: http://fourcot.fr/weboob/elbesachsen-day.png
# Temperature in Rennes
[temprennes]
user florent
env.HOME /home/flo
env.cumulate 0
env.capa CapWeather
env.do get_current,619163,yahoo
env.import from weboob.capabilities.weather import CapWeather
env.attribvalue temp/value
env.attribid temp/id
env.title Température à Rennes
env.vlabel Température
env.label id
Result: http://fourcot.fr/weboob/temprennes-day.png

434
contrib/munin/weboob-generic Executable file
View file

@ -0,0 +1,434 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
# Copyright(C) 2013 Romain Bignon, Florent Fourcot
#
# This file is part of weboob.
#
# weboob is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# weboob is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
### Installation ###
# 1) Create a symlink from /etc/munin/plugins/yourchoice to the script
# 2) Configure the plugin in /etc/munin/plugin-conf.d/ See below for the options
# 3) Restart/reload munin-node
# 4) Note that cached values are stored in folder ~/.config/weboob/munin/
### Configuration ###
## Mandatory options ##
# env.capa: The Weboob capability to load
# Example: env.capa CapBank
#
# env.do: The Weboob command to call. It can take more than one argument.
# With two argument, the second is used as parameter for the command.
# The third is used to restrict backends.
# Example: env.do get_balance
#
# env.import: The import line to import the capabilities
# Example: from weboob.capabilities.bank import CapBank
#
# env.attribvalue: The attribut name of objects returned by the do command.
# For example, the "balance" member of Account objects
# If the attribut is itself one object, a hierarchical call can be done
# with the "/" operators.
# Example: env.attribvalue balance
# Example: env.attribvalue temp/value
## Optionals -- more configuration ##
# env.id_monitored: Restrict the results to a list of ids (space is used as separator)
# Example: env.id_monitored account1@backend1 account2@backend2
#
# env.exclude: Exclude some results (space is used as separator)
# Example: env.exclude 550810@sachsen
#
# env.cache_expire: To avoid site flooding, results are cached in folder
# /.config/weboob/munin/. The default lifetime of a cache value is 3600s
# Example: env.cache_expire 7200
#
# env.cumulate: Display data in Area mode (default) or in line mode.
# Example: env.cumulate 0
#
# env.get_object_list: optional pre-call to get a list of objects, to applied
# the do function on it.
# Exemple: env.get_object_list="iter_subscriptions"
#
# env.attribid: Munin needs an id for each value. The default is to use the id of results,
# but another attribute can be used. "/" can be used as separator for
# hierarchical calls
# Example: env.attribid id
#
# env.title: A title for the graph (default: nothing)
# Example: env.title a wonderful graph
#
# env.vlabel: A vertical label for the graph
# Example: env.vlabel Balance
#
# env.label: Each data in munin as a label. Per default, the script takes the
# "label" attribute of objects. However, it does not always exist,
# and a better choice can be possible
# Example: env.label id
# For some running examples, see at the end of the script
import os
import sys
import locale
import time
import logging
from weboob.capabilities.base import NotAvailable
from weboob.core import Weboob, CallErrors
from weboob.tools.browser import BrowserIncorrectPassword
class GenericMuninPlugin(object):
def __init__(self):
if 'weboob_path' in os.environ:
self.weboob = Weboob(os.environ['weboob_path'])
else:
self.weboob = Weboob()
self.cache_expire = long(os.environ.get('cache_expire', 3600))
self.cumulate = int(os.environ.get('cumulate', 1))
self.cache = None
self.name = sys.argv[0]
if "/" in self.name:
self.name = self.name.split('/')[-1]
# Capability to load
self.capa = os.environ['capa']
# Command to pass to Weboob
self.do = os.environ['do'].split(',')
# Not easy to load modules automatically...
self.mimport = os.environ["import"]
exec(self.mimport)
# We can monitore only some objects
self.object_list = None
if 'get_object_list' in os.environ:
self.object_list = os.environ["get_object_list"]
self.tomonitore = None
if 'id_monitored' in os.environ:
self.tomonitore = os.environ['id_monitored'].decode('utf-8').split(' ')
self.exclude = None
if 'exclude' in os.environ:
self.exclude = os.environ['exclude'].split(' ')
# Attribut of object to use as ID (default: id)
self.attribid = "id"
if 'attribid' in os.environ:
self.attribid = os.environ['attribid']
self.attribvalue = os.environ['attribvalue']
self.title = ''
if 'title' in os.environ:
self.title = os.environ['title'].decode('utf-8')
self.attriblabel = "label"
if 'label' in os.environ:
self.attriblabel = os.environ['label']
self.vlabel = self.attribvalue
if 'vlabel' in os.environ:
self.vlabel = os.environ['vlabel'].decode('utf-8')
def display_help(self):
print 'generic-munin is a plugin for munin'
print ''
print 'Copyright(C) 2013 Romain Bignon, Florent Fourcot'
print ''
print 'To use it, create a symlink /etc/munin/plugins/nameyouwant to this script'
print 'and add this section in /etc/munin/plugin-conf.d/munin-node:'
print ''
print '[nameyouwant]'
print 'user romain'
print 'group romain'
print 'env.HOME /home/romain'
print '# The weboob directory path.'
print 'env.weboob_path /home/romain/.config/weboob/'
print '# Monitored objects. If this parameter is missing, all objects'
print '# will be displayed.'
print 'env.id_monitored myid@backend1 otherid@backend2'
print '# To prevent mass connections to websites, results are cached.'
print '# You can set here the expiration delay (in seconds).'
print 'env.cache_expire 7200'
print '# Cumulate values'
print 'env.cumulate 1'
print ''
def cachepath(self, name):
tmpdir = os.path.join(self.weboob.workdir, "munin")
if not os.path.isdir(tmpdir):
os.makedirs(tmpdir)
return os.path.join(tmpdir, name)
def check_cache(self, name):
return self.print_cache(name, check=True)
def print_cache(self, name, check=False):
try:
f = open(self.cachepath(name), 'r')
except IOError:
return False
try:
last = int(f.readline().strip())
except ValueError:
return False
if check and (last + self.cache_expire) < time.time():
return False
for line in f.xreadlines():
sys.stdout.write(line)
return True
def new_cache(self, name):
os.umask(0077)
new_name = '%s.new' % name
filename = self.cachepath(new_name)
try:
f = open(filename, 'w')
except IOError, e:
print >>sys.stderr, 'Unable to create the cache file %s: %s' % (filename, e)
return
self.cache = f
self.cache.write('%d\n' % time.time())
def flush_cache(self):
old_name = self.cache.name
new_name = self.cache.name[:-4]
self.cache.close()
os.rename(old_name, new_name)
def write_output(self, line):
sys.stdout.write('%s\n' % line)
if self.cache:
self.cache.write('%s\n' % line)
def build_do(self):
if self.object_list:
results = []
for result in self.weboob.do(self.object_list):
results.append(result)
for backend, result in results:
try:
for i in self.weboob.do(self.do[0], result.id, backends=backend):
yield i
# Do not crash if one module does not implement the feature
except CallErrors:
pass
elif len(self.do) == 1:
for i in self.weboob.do(self.do[0]):
yield i
elif len(self.do) == 2:
for i in self.weboob.do(self.do[0], self.do[1]):
yield i
elif len(self.do) == 3:
for i in self.weboob.do(self.do[0], self.do[1], backends=self.do[2]):
yield i
def get_value(self, result):
attribs = self.attribvalue.split('/')
for attrib in attribs:
result = getattr(result, attrib)
if type(result) is list:
result = result[0]
return result
def monitored(self, result):
id = self.result2weboobid(result)
if self.exclude and id in self.exclude:
return False
return not self.tomonitore or id in self.tomonitore
def result2weboobid(self, result):
attribs = self.attribid.split('/')
id = '%s@%s' % (getattr(result, attribs[0]), result.backend)
return id
def result2id(self, result):
attribs = self.attribid.split('/')
id = result
for attrib in attribs:
id = getattr(id, attrib)
return '%s_%s' % (result.backend, id)
def config(self):
if self.check_cache('%s-config' % self.name):
return
self.new_cache('%s-config' % self.name)
self.weboob.load_backends(self.capa)
self.write_output('graph_title %s' % self.title.encode('iso-8859-15'))
self.write_output('graph_vlabel %s' % self.vlabel.encode('iso-8859-15'))
self.write_output('graph_category weboob')
self.write_output('graph_args --rigid')
try:
objects = []
if self.tomonitore or self.exclude:
d = {}
for backend, result in self.build_do():
if self.monitored(result):
d[self.result2weboobid(result)] = result
if self.tomonitore:
for id in self.tomonitore:
try:
objects.append(d[id])
except KeyError:
pass
else:
for id in d:
objects.append(d[id])
else:
objects = reversed([a for b, a in self.build_do()])
first = True
for result in objects:
id = self.result2id(result)
type = 'STACK'
if first:
type = 'AREA'
first = False
self.write_output('%s.label %s' % (id.encode('iso-8859-15'), getattr(result, self.attriblabel).encode('iso-8859-15')))
if self.cumulate:
self.write_output('%s.draw %s' % (id, type))
except CallErrors, errors:
self.print_errors(errors)
self.print_cache('%s-config' % self.name)
else:
self.flush_cache()
def print_errors(self, errors):
for backend, err, backtrace in errors:
print >>sys.stderr, (u'%s(%s): %s' % (type(err).__name__, backend.name, err)).encode(sys.stdout.encoding or locale.getpreferredencoding(), 'replace')
if isinstance(err, BrowserIncorrectPassword):
self.weboob.backends_config.edit_backend(backend.name, backend.NAME, {'_enabled': False})
def execute(self):
if self.check_cache(self.name):
return
self.new_cache(self.name)
self.weboob.load_backends(self.capa)
try:
for backend, result in self.build_do():
if self.monitored(result):
value = self.get_value(result)
if value is not NotAvailable:
self.write_output('%s.value %f' % (self.result2id(result).encode('iso-8859-15'), value))
except CallErrors, errors:
self.print_errors(errors)
self.print_cache(self.name)
else:
self.flush_cache()
def run(self):
cmd = (len(sys.argv) > 1 and sys.argv[1]) or "execute"
if cmd == 'execute':
self.execute()
elif cmd == 'config':
self.config()
elif cmd == 'autoconf':
print 'no'
sys.exit(1)
elif cmd == 'suggest':
sys.exit(1)
elif cmd == 'help' or cmd == '-h' or cmd == '--help':
self.display_help()
if self.cache:
self.cache.close()
sys.exit(0)
if __name__ == '__main__':
logging.basicConfig()
GenericMuninPlugin().run()
### Examples ###
## Like boobank-munin does
## Only for the example, you should use boobank-munin instead
#[bank]
#user florent
#group florent
#env.cache_expire 7200
#env.HOME /home/flo
#env.capa CapBank
#env.do iter_accounts
#env.import from weboob.capabilities.bank import CapBank
#env.attribvalue balance
#env.title Solde des comptes
#
#
## Balance of your leclercmobile subscription
#[leclercmobile]
#user florent
#group florent
#env.cache_expire 16800
#env.HOME /home/flo
#env.capa CapBill
#env.do get_balance,06XXXXXXXX,leclercmobile
#env.import from weboob.capabilities.bill import CapBill
#env.attribvalue price
#env.title Forfait leclercmobile
#env.vlabel Solde
#
#Result: http://fourcot.fr/weboob/leclercmobile-day.png
#
## Monitor water level in Dresden
#[leveldresden]
#user florent
#group florent
#env.cache_expire 7200
#env.HOME /home/flo
#env.capa CapGauge
#env.do get_last_measure,501060-level
#env.import from weboob.capabilities.gauge import CapGauge
#env.attribvalue level
#env.title Niveau de l'elbe
#env.label id
#
#
## The level of the elbe in all Sachsen's cities
#[levelelbesachsen]
#user florent
#env.cache_expire 800
#env.HOME /home/flo
#env.cumulate 0
#env.capa CapGauge
#env.do iter_gauges,Elbe,sachsen
#env.import from weboob.capabilities.gauge import CapGauge
#env.attribvalue sensors/lastvalue/level
#env.title Niveau de l'elbe en Saxe
#env.label name
#env.vlabel Hauteur du fleuve (cm)
#env.exclude 550810@sachsen
#
#Result: http://fourcot.fr/weboob/elbesachsen-day.png
#
## Temperature in Rennes
#[temprennes]
#user florent
#env.HOME /home/flo
#env.cumulate 0
#env.capa CapWeather
#env.do get_current,619163,yahoo
#env.import from weboob.capabilities.weather import CapWeather
#env.attribvalue temp/value
#env.attribid temp/id
#env.title Température à Rennes
#env.vlabel Température
#env.label id
#
#Result: http://fourcot.fr/weboob/temprennes-day.png