nojhan/paradiseo
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

* New tree configuration of the project:

Browse source 
.../
   ...           + -- EO
   |             |
   |             |
   +-- src ----- + -- EDO
   |             |
   |             |
   +-- test      + -- MO
   |             |
   |             |
   +-- tutorial  + -- MOEO
   |             |
   |             |
   +-- doc       + -- SMP
   |             |
   |             |
   ...           + -- EOMPI
                 |
                 |
                 + -- EOSERIAL

Question for current maintainers: ./README: new release?

Also:

* Moving out eompi & eoserial modules (issue #2).

* Correction of the errors when executing "make doc" command.

* Adding a solution for the conflicting headers problem (see the two CMake Cache
 Values: PROJECT_TAG & PROJECT_HRS_INSTALL_SUBPATH) (issue #1)

* Header inclusions:
        ** src: changing absolute paths into relative paths ('#include <...>' -> '#include "..."')
        ** test, tutorial: changing relative paths into absolute paths ('#include "..."' -> '#include <...>')

* Moving out some scripts from EDO -> to the root

* Add a new script for compilation and installation (see build_gcc_linux_install)

* Compilation with uBLAS library or EDO module: now ok

* Minor modifications on README & INSTALL files

* Comment eompi failed tests with no end

*** TODO: CPack (debian (DEB) & RedHat (RPM) packages) (issues #6 & #7) ***
This commit is contained in:
Adèle Harrissart 2014-08-04 13:40:28 +02:00
commit 490e837f7a
2359 changed files with 7688 additions and 16329 deletions
Download patch file Download diff file Expand all files Collapse all files

53
test/eompi/CMakeLists.txt Executable file
View file

@ -0,0 +1,53 @@
###############################################################################
##
## CMakeLists file for eo/test/mpi
##
###############################################################################
######################################################################################
### 1) Include the sources
######################################################################################
#include_directories(${EOMPI_SRC_DIR}/src)
#include_directories(${EOSERIAL_SRC_DIR}/src)
#include_directories(${EO_SRC_DIR}/src)
#include_directories(${CMAKE_CURRENT_SOURCE_DIR})
######################################################################################
### 2) Specify where CMake can find the libraries
######################################################################################
link_directories(${EO_BIN_DIR}/lib)
link_directories(${EOMPI_BIN_DIR}/lib)
link_directories(${EOSERIAL_BIN_DIR}/lib)
######################################################################################
### 3) Define your targets and link the librairies
######################################################################################
set (TEST_LIST
t-mpi-parallelApply
t-mpi-wrapper
t-mpi-multipleRoles
#t-mpi-eval
#t-mpi-multistart
t-mpi-distrib-exp
)
foreach (test ${TEST_LIST})
set ("T_${test}_SOURCES" "${test}.cpp")
endforeach (test)
set(CMAKE_CXX_COMPILER "mpicxx")
add_definitions(-DWITH_MPI)
if(ENABLE_CMAKE_TESTING)
foreach (test ${TEST_LIST})
add_executable(${test} ${T_${test}_SOURCES})
add_test(${test} ${test})
target_link_libraries(${test} eoutils eompi eoserial eo)
install(TARGETS ${test} RUNTIME DESTINATION share/eo/test COMPONENT test)
endforeach (test)
endif()
######################################################################################

25
test/eompi/DISTRIB_XP_README.md Executable file
View file

@ -0,0 +1,25 @@
README
------
To launch a set of experiments with t-mpi-distrib-exp:
0) Compile it:
mpic++ -o distrib-exp t-mpi-distrib-exp.cpp -I../../src/ -I../../src/mpi/ -DWITH_MPI -L ../../../build/eo/lib/ -leoutils -leo -leompi -leoserial
1) Generate the experiments, thanks to the script gen-xp.py
This script will guide you and ask you for all experiments. The prefix is used in the results filenames.
You may want to modify the name of the experiments file (default value: "experiments.json") or
the pattern of the results files. However, you have to ensure that the pattern is an one-to-one
function of the parameters, otherwise some results could be lost.
2) Launch the t-mpi-distrib-exp program with mpirun:
For 4 cores (= 1 master + 3 workers)
mpirun -np 4 ./t-mpi-distrib-exp --use-experiment-file=1 --experiment-file=/home/eodev/eo/test/mpi/experiments.json
For 16 cores (= 1 master + 15 workers)
mpirun -np 5 ./t-mpi-distrib-exp --use-experiment-file=1 --experiment-file=/home/eodev/eo/test/mpi/experiments.json
3) The program will generate the results of the experiments, as txt files. There is one result file for each run of each
experiment.

6
test/eompi/experiments.json Executable file
View file

@ -0,0 +1,6 @@
{
"experiments":[
{"size":"10", "packet_size":"1", "seed":"1337", "distribution":{"name":"normal", "mean":"500", "stddev":"100"}, "worker_print_waiting_time":"1", "filename":"exp1.result.txt"},
{"size":"10", "packet_size":"1", "seed":"1337", "distribution":{"name":"normal", "mean":"100", "stddev":"20"}, "worker_print_waiting_time":"1", "filename":"exp2.result.txt"}
]
}

152
test/eompi/gen-xp.py Executable file
View file

@ -0,0 +1,152 @@
# -*- coding:utf-8 -*-
"""
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Authors:
Benjamin Bouvier <benjamin.bouvier@gmail.com>
"""
import json
# Where will be saved the experiments?
EXPERIMENTS_FILENAME = "experiments.json"
# What will be the pattern for experiments filenames?
FILENAME_PATTERN = "%(prefix)s_%(distrib_name)s_%(size)s_%(packet_size)s_%(run)s.txt"
def input_number_at_least( min ):
n = min - 1
while n < min:
try:
n = int(raw_input("Enter a number greater or equal to %s: "% min))
except Exception:
print "Please enter an integer."
return n
def input_number_between( min, max ):
n = min - 1
while n < min or n > max:
try:
n = int(raw_input("Enter a number between %s and %s: " % (min,max)))
except Exception:
print "Please enter a number."
return n
def choose_continue():
print """Do you want to continue?
0. No
1. Yes"""
return bool( input_number_between(0,1) )
def choose_distribution_uniform():
distribution = {}
distribution["name"] = "uniform"
print "Enter the minimum value (in milliseconds): "
min = input_number_at_least( 0 )
distribution["min"] = str(min)
print "Enter the maximum value (in milliseconds): "
distribution["max"] = str(input_number_at_least( min ))
return distribution
def choose_distribution_normal():
distribution = {}
distribution["name"] = "normal"
print "Enter the mean (in milliseconds): "
distribution["mean"] = str(input_number_at_least( 0 ))
print "Enter the standard deviation (in milliseconds): "
distribution["stddev"] = str(input_number_at_least( 0 ))
return distribution
def choose_distribution_power():
distribution = {}
distribution["name"] = "exponential"
print "Enter the mean (in milliseconds): "
distribution["mean"] = str(input_number_at_least( 0 ))
return distribution
def choose_distribution():
print """Choose your distribution:
0. Uniform
1. Normal
2. Exponential"""
choice = input_number_between( 0, 2 )
choose_distrib_functions = [ choose_distribution_uniform, choose_distribution_normal, choose_distribution_power ]
return choose_distrib_functions[ choice ]()
def choose_packet_size():
print "Enter the size of a packet (group of elements):"
return str(input_number_at_least( 0 ))
def choose_size():
print "Enter the total size (size of population):"
return str(input_number_at_least( 0 ))
def choose_worker_print():
print """Should the workers print the time they sleep on stdout?
0. No
1. Yes"""
return str(input_number_between( 0, 1 ))
def choose_nb_runs():
print """How many runs should be launched for this configuration? Seeds will be automatically affected to the number
of run+1 (for instance, the first run has a seed of 1, the second has a seed of 2, etc.)."""
return input_number_at_least( 1 )
def choose_prefix():
print """What is the name of the experiment? It will be used as the prefix of file names."""
return raw_input("Enter the prefix name: ")
def main():
prefix = choose_prefix()
exps = []
while True:
exp = {}
exp["distribution"] = choose_distribution()
exp["size"] = choose_size()
exp["packet_size"] = choose_packet_size()
exp["worker_print_waiting_time"] = choose_worker_print()
runs = choose_nb_runs()
for i in range( runs ):
exp["seed"] = str(i+1)
filename_map = exp.copy()
filename_map["run"] = exp["seed"]
filename_map["distrib_name"] = exp["distribution"]["name"]
filename_map["prefix"] = prefix
filename = FILENAME_PATTERN % filename_map
exp["filename"] = filename
copy = exp.copy()
exps.append( copy )
if not choose_continue():
break
# Write the experiments in a file
f = file( EXPERIMENTS_FILENAME , 'wb')
f.write("""{"experiments":[""")
i = 0
for exp in exps:
if i > 0:
f.write(",\n")
i += 1
f.write( json.dumps(exp) )
f.write("]}")
f.close()
if __name__ == "__main__":
main()

51
test/eompi/t-mpi-common.h Executable file
View file

@ -0,0 +1,51 @@
# ifndef __T_MPI_COMMON_H__
# define __T_MPI_COMMON_H__
#include <paradiseo/eoserial.h>
/**
* @file t-mpi-common.h
*
* This file shows an example of serialization of a primitive type, so as to be used in a parallel algorithm.
* It is fully compatible with the basic type, by implementing conversion operator and constructor based on type.
* It can contain any simple type which can be written in a std::ostream.
*/
template< class T >
struct SerializableBase : public eoserial::Persistent
{
public:
operator T&()
{
return _value;
}
SerializableBase() : _value()
{
// empty
}
SerializableBase( T base ) : _value( base )
{
// empty
}
void unpack( const eoserial::Object* obj )
{
eoserial::unpack( *obj, "value", _value );
}
eoserial::Object* pack(void) const
{
eoserial::Object* obj = new eoserial::Object;
obj->add("value", eoserial::make( _value ) );
return obj;
}
private:
T _value;
};
# endif // __T_MPI_COMMON_H__

556
test/eompi/t-mpi-distrib-exp.cpp Executable file
View file

@ -0,0 +1,556 @@
/*
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
* Authors:
* Benjamin Bouvier <benjamin.bouvier@gmail.com>
*/
/**
* @file t-mpi-distrib-exp.cpp
* @brief File for parallel experimentations.
*
* When using parallel evaluation, the individuals to evaluate are sent by packets (group),
* so as to avoid that communication time be more important than worker's execution time.
* However, the ideal size of packet depends on the problem and the time needed to carry out
* the atomic operation on each individual. This experiment tries to find a relation between
* the total number of elements to process (size), the execution time and the size of packet.
* This could lead to an heuristic allowing to optimize the size of packet according to the
* processing times.
*/
# include <unistd.h> // usleep
# include <iostream>
# include <iomanip>
# include <string>
# include <sstream>
# include <vector>
# include <paradiseo/eo.h>
# include <paradiseo/eompi.h>
# include "t-mpi-common.h"
using namespace eo::mpi;
// Serializable int
typedef SerializableBase<int> type;
/*
* The task is the following: the worker receives a number of milliseconds to wait, which
* simulates the process of one individual. This way, the sequences of processing times are
* generated only by the master and are more easily reproductible.
*/
struct Wait : public eoUF< type &, void >
{
Wait( bool print ) : _print( print )
{
// empty
}
void operator()( type & milliseconds )
{
if( _print )
std::cout << "Sleeping for " << milliseconds << "ms..." << std::endl;
// usleep takes an input in microseconds
usleep( milliseconds * 1000 );
}
private:
bool _print;
};
/**
* @brief Represents a distribution of processing times.
*/
class Distribution : public std::vector< type >, public eoserial::Persistent
{
public:
/**
* @brief Really fills the vector with the distribution values.
*/
void fill( unsigned size )
{
for( unsigned i = 0; i < size; ++i )
{
int next = next_element();
if( next < 0 ) next = 0;
push_back( next );
}
}
/**
* @brief Returns the next element of the distribution to put in the
* vector.
*
* @returns Number of milliseconds to wait. Can be negative ; in this case,
* the number will be truncated to 0ms.
*/
virtual int next_element() = 0;
/**
* @brief Creates params and retrieves values from parser
*
* Parser's params should take milliseconds as inputs.
*/
virtual void make_parser( eoParser & parser ) = 0;
/**
* @brief Returns true if this distribution has been activated by the
* command line.
*
* Used by the main program so as to check if at least one distribution has been
* activated.
*/
bool isActive() { return _active; }
/**
* @brief Prints the name and the parameters of the distribution
*/
virtual std::string toString() const = 0;
protected:
bool _active;
};
/**
* @brief Uniform distribution.
*
* This is an uniform distribution, defined by a minimum value and a maximum value.
* In the uniform distribution, every number from min to max has the same probability
* to appear.
*
* The 3 parameters activable from a parser are the following:
* - uniform=1 : if we want to use the uniform distribution
* - uniform-min=x : use x milliseconds as the minimum value of waiting time.
* - uniform-max=y : use y milliseconds as the maximum value of waiting time.
* Ensure that x < y, or the results are unpredictable.
*/
class UniformDistribution : public Distribution
{
public:
UniformDistribution()
{
// empty
}
void make_parser( eoParser & parser )
{
_active = parser.createParam( false, "uniform", "Uniform distribution", '\0', "Uniform").value();
_min = parser.createParam( 0.0, "uniform-min", "Minimum for uniform distribution, in ms.", '\0', "Uniform").value();
_max = parser.createParam( 1.0, "uniform-max", "Maximum for uniform distribution, in ms.", '\0', "Uniform").value();
}
int next_element()
{
return std::floor( eo::rng.uniform( _min, _max ) );
}
eoserial::Object* pack( void ) const
{
eoserial::Object* obj = new eoserial::Object;
obj->add( "name", eoserial::make( "uniform" ) );
obj->add( "min", eoserial::make( _min ) );
obj->add( "max", eoserial::make( _max ) );
return obj;
}
void unpack( const eoserial::Object* obj )
{
eoserial::unpack( *obj, "min", _min );
eoserial::unpack( *obj, "max", _max );
}
std::string toString() const
{
std::stringstream ss;
ss << "uniform" << '\n'
<< "min: " << _min << '\n'
<< "max: " << _max << '\n';
return ss.str();
}
protected:
double _min;
double _max;
} uniformDistribution;
/**
* @brief Normal (gaussian) distribution of times.
*
* A normal distribution is defined by a mean and a standard deviation.
* The 3 parameters activable from the parser are the following:
* - normal=1: activates the gaussian distribution.
* - normal-mean=50: use 50ms as the mean of the distribution.
* - normal-stddev=10: use 10ms as the standard deviation of the distribution.
*/
class NormalDistribution : public Distribution
{
public:
NormalDistribution()
{
// empty
}
void make_parser( eoParser & parser )
{
_active = parser.createParam( false, "normal", "Normal distribution", '\0', "Normal").value();
_mean = parser.createParam( 0.0, "normal-mean", "Mean for the normal distribution (0 by default), in ms.", '\0', "Normal").value();
_stddev = parser.createParam( 1.0, "normal-stddev", "Standard deviation for the normal distribution (1ms by default), 0 isn't acceptable.", '\0', "Normal").value();
}
int next_element()
{
return std::floor( eo::rng.normal( _mean, _stddev ) );
}
eoserial::Object* pack( void ) const
{
eoserial::Object* obj = new eoserial::Object;
obj->add( "name", eoserial::make( "normal" ) );
obj->add( "mean", eoserial::make( _mean ) );
obj->add( "stddev", eoserial::make( _stddev ) );
return obj;
}
void unpack( const eoserial::Object* obj )
{
eoserial::unpack( *obj, "mean", _mean );
eoserial::unpack( *obj, "stddev", _stddev );
}
std::string toString() const
{
std::stringstream ss;
ss << "normal" << '\n'
<< "mean: " << _mean << '\n'
<< "stddev: " << _stddev << '\n';
return ss.str();
}
protected:
double _mean;
double _stddev;
} normalDistribution;
/**
* @brief Exponential distribution.
*
* This distribution belongs to the category of the decreasing power laws and are affected by long trails
* phenomenons.
* An exponential distribution is only defined by its mean.
*
* The 2 parameters activable from the parser are the following:
* - exponential=1: to activate the exponential distribution.
* - exponential-mean=50: indicates that the mean must be 50ms.
*/
class ExponentialDistribution : public Distribution
{
public:
ExponentialDistribution()
{
// empty
}
void make_parser( eoParser & parser )
{
_active = parser.createParam( false, "exponential", "Exponential distribution", '\0', "Exponential").value();
_mean = parser.createParam( 0.0, "exponential-mean", "Mean for the exponential distribution (0 by default), in ms.", '\0', "Exponential").value();
}
int next_element()
{
return std::floor( eo::rng.negexp( _mean ) );
}
eoserial::Object* pack( void ) const
{
eoserial::Object* obj = new eoserial::Object;
obj->add( "name", eoserial::make( "exponential" ) );
obj->add( "mean", eoserial::make( _mean ) );
return obj;
}
void unpack( const eoserial::Object* obj )
{
eoserial::unpack( *obj, "mean", _mean );
}
std::string toString() const
{
std::stringstream ss;
ss << "exponential" << '\n'
<< "mean: " << _mean << '\n';
return ss.str();
}
protected:
double _mean;
} exponentialDistribution;
/**
* @brief Serializable experiment.
*
* Allows an experiment to be saved and loaded via a file, using eoserial.
*
* Construct the experiment with the good parameters from the command line or load experiments from a file. Then call run() to launch the parallel job.
*
* If a filename is given to the constructor (or during the loading), the results of the experiments (time series) will
* be redirected to the file with the given file name. Otherwise (filename == ""), the output will just be shown on the
* standard output.
*/
class Experiment : public eoserial::Persistent
{
public:
Experiment() : _distribution(0), _worker_print_waiting_time( false ), _fileName("")
{
// empty
}
Experiment( Distribution* distrib, unsigned size, unsigned packet_size, bool print_waiting_time, unsigned seed, const std::string& fileName = "" ) :
_distribution( distrib ),
_size( size ),
_packet_size( packet_size ),
_worker_print_waiting_time( print_waiting_time ),
_seed( seed ),
_fileName( fileName )
{
// empty
}
eoserial::Object* pack( void ) const
{
eoserial::Object* obj = new eoserial::Object;
obj->add( "size", eoserial::make( _size ) );
obj->add( "packet_size", eoserial::make( _packet_size ) );
obj->add( "worker_print_waiting_time", eoserial::make( _worker_print_waiting_time ) );
obj->add( "seed", eoserial::make( _seed ) );
if( _distribution )
{
obj->add( "distribution", _distribution );
}
obj->add( "filename", eoserial::make( _fileName ) );
return obj;
}
void unpack( const eoserial::Object* obj )
{
eoserial::unpack( *obj, "size", _size );
eoserial::unpack( *obj, "packet_size", _packet_size );
eoserial::unpack( *obj, "worker_print_waiting_time", _worker_print_waiting_time );
eoserial::unpack( *obj, "seed", _seed );
eoserial::unpack( *obj, "filename", _fileName );
eoserial::Object* distribObject = static_cast<eoserial::Object*>( obj->find("distribution")->second );
std::string distribName = *static_cast<eoserial::String*>( distribObject->find("name")->second );
// TODO find a better design...
if( distribName == "normal" ) {
_distribution = & normalDistribution;
} else if( distribName == "uniform" ) {
_distribution = & uniformDistribution;
} else if( distribName == "exponential" ) {
_distribution = & exponentialDistribution;
} else {
throw std::runtime_error("When unpacking experience, no distribution found.");
}
eoserial::unpackObject( *obj, "distribution", *_distribution );
}
void run()
{
mpi::communicator& comm = eo::mpi::Node::comm();
// reinits every objects
eo::rng.reseed( _seed );
eo::rng.clearCache(); // trick for repeatable sequences of normal numbers, cf eo::rng
_distribution->clear();
_distribution->fill( _size );
eo::mpi::timerStat.start("run");
Wait wait( _worker_print_waiting_time );
ParallelApplyStore< type > store( wait, DEFAULT_MASTER, _packet_size );
store.data( *_distribution );
DynamicAssignmentAlgorithm scheduling;
ParallelApply< type > job( scheduling, DEFAULT_MASTER, store );
job.run();
eo::mpi::timerStat.stop("run");
if( job.isMaster() )
{
EmptyJob( scheduling, DEFAULT_MASTER ); // to terminate parallel apply
// Receive statistics
typedef std::map< std::string, eoTimerStat::Stat > typeStats;
std::ostream* pout;
std::ofstream file;
bool fileSaveActivated = false;
if( _fileName == "" ) {
pout = & std::cout;
} else {
pout = & file;
file.open( _fileName.c_str() );
fileSaveActivated = true;
}
std::ostream& out = *pout;
// Reminder of the parameters
out << "size: " << _size << '\n'
<< "packet_size: " << _packet_size << '\n'
<< "distribution: " << _distribution->toString()
<< "seed: " << _seed << '\n' << std::endl;
// Results
out << std::fixed << std::setprecision( 5 );
for( int i = 1, s = comm.size(); i < s; ++i )
{
eoTimerStat timerStat;
comm.recv( i, eo::mpi::Channel::Commands, timerStat );
typeStats stats = timerStat.stats();
for( typeStats::iterator it = stats.begin(),
end = stats.end();
it != end;
++it )
{
out << i << " " << it->first << std::endl;
for( int j = 0, t = it->second.wtime.size(); j < t; ++j )
{
out << it->second.wtime[j] << " ";
}
out << std::endl;
}
out << std::endl;
}
if( fileSaveActivated ) {
file.close();
}
} else
{
// Send statistics
comm.send( DEFAULT_MASTER, eo::mpi::Channel::Commands, eo::mpi::timerStat );
}
timerStat.clear();
}
private:
Distribution* _distribution;
unsigned _size;
unsigned _packet_size;
bool _worker_print_waiting_time;
unsigned _seed;
std::string _fileName;
};
int main( int argc, char** argv )
{
Node::init( argc, argv );
eoParser parser( argc, argv );
// forces the statistics to be retrieved
eo::mpi::timerStat.forceDoMeasure();
// General parameters for the experimentation
unsigned size = parser.createParam( 10U, "size", "Number of elements to distribute.", 's', "Distribution").value();
unsigned packet_size = parser.createParam( 1U, "packet-size", "Number of elements to distribute at each time for a single worker.", 'p', "Parallelization").value();
bool worker_print_waiting_time = parser.createParam( false, "print-waiting-time", "Do the workers need to print the time they wait?", '\0', "Parallelization").value();
unsigned seed = parser.createParam( 0U, "seed", "Seed of random generator", '\0', "General").value();
std::string fileName = parser.createParam( std::string(""), "filename", "File name to which redirect the results (for a single experiment)", '\0', "General").value();
bool useExperimentFile = parser.createParam( false, "use-experiment-file", "Put to true if you want to launch experiments from a file formatted in JSON (see experiment-file).", '\0', "General").value();
std::string experimentFile = parser.createParam( std::string("experiments.json"), "experiment-file", "File name of experiments to provide, in format JSON.", '\0', "General").value();
if( !useExperimentFile )
{
std::vector<Distribution*> distribs;
distribs.push_back( &uniformDistribution );
distribs.push_back( &normalDistribution );
distribs.push_back( &exponentialDistribution );
// for each available distribution, check if activated.
// If no distribution is activated, show an error message
// If two distributions or more are activated, show an error message
// Otherwise, use the activated distribution as distrib
bool isChosenDistrib = false;
Distribution* pdistrib = 0;
for( int i = 0, s = distribs.size(); i < s; ++i )
{
distribs[i]->make_parser( parser );
if( distribs[i]->isActive() )
{
if( isChosenDistrib )
{
throw std::runtime_error("Only one distribution can be chosen during a launch!");
} else
{
isChosenDistrib = true;
pdistrib = distribs[i];
}
}
}
make_parallel( parser );
make_help( parser );
if( !isChosenDistrib )
{
throw std::runtime_error("No distribution chosen. One distribution should be chosen.");
}
Experiment e( pdistrib, size, packet_size, worker_print_waiting_time, seed, fileName );
e.run();
}
else // use experiments file
{
// read content of file
std::ifstream file( experimentFile.c_str() );
std::string fileContent;
while( file )
{
char temp[4096];
file.getline( temp, 4096, '\n' );
fileContent += temp;
fileContent += '\n';
}
file.close();
// transform content into array of experiments
eoserial::Object* wrapper = eoserial::Parser::parse( fileContent );
eoserial::Array& experiments = *static_cast< eoserial::Array* >( wrapper->find("experiments")->second );
for( unsigned i = 0, s = experiments.size(); i < s; ++i )
{
std::cout << "Launching experiment " << (i+1) << "..." << std::endl;
eoserial::Object* expObj = static_cast< eoserial::Object* >( experiments[i] );
Experiment exp;
exp.unpack( expObj );
exp.run();
}
delete wrapper;
}
return 0;
}

206
test/eompi/t-mpi-eval.cpp Executable file
View file

@ -0,0 +1,206 @@
/*
(c) Thales group, 2012
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation;
version 2 of the License.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Contact: http://eodev.sourceforge.net
Authors:
Benjamin Bouvier <benjamin.bouvier@gmail.com>
*/
/*
* This file shows an example of parallel evaluation of a population, when using an eoEasyEA algorithm.
* Moreover, we add a basic wrapper on the parallel evaluation, so as to show how to retrieve the best solutions.
*/
//-----------------------------------------------------------------------------
#include <paradiseo/eo.h>
#include <paradiseo/eo/eoPopEvalFunc.h>
#include <paradiseo/eo/es/make_real.h>
#include "../eo/real_value.h"
#include <paradiseo/eompi.h>
#include <vector>
using namespace std;
//-----------------------------------------------------------------------------
class eoRealSerializable : public eoReal< eoMinimizingFitness >, public eoserial::Persistent
{
public:
eoRealSerializable(unsigned size = 0, double value = 0.0):
eoReal<eoMinimizingFitness>(size, value) {}
eoserial::Object* pack() const
{
eoserial::Object* obj = new eoserial::Object;
obj->add( "array",
eoserial::makeArray< vector<double>, eoserial::MakeAlgorithm >
( *this )
);
bool invalidFitness = invalid();
obj->add("invalid", eoserial::make( invalidFitness ) );
if( !invalidFitness )
{
double fitnessVal = fitness();
obj->add("fitness", eoserial::make( fitnessVal ) );
}
return obj;
}
void unpack( const eoserial::Object* obj )
{
this->clear();
eoserial::unpackArray< vector<double>, eoserial::Array::UnpackAlgorithm >
( *obj, "array", *this );
bool invalidFitness;
eoserial::unpack( *obj, "invalid", invalidFitness );
if( invalidFitness ) {
invalidate();
} else {
double fitnessVal;
eoserial::unpack<double>( *obj, "fitness", fitnessVal );
fitness( fitnessVal );
}
}
};
typedef eoRealSerializable EOT;
/*
* Wrapper for HandleResponse: shows the best answer, as it is found.
*
* Finding the best solution is an associative operation (as it is based on a "min" function, which is associative too)
* and that's why we can perform it here. Indeed, the min element of 5 elements is the min element of the 3 first
* elements and the min element of the 2 last elements:
* min(1, 2, 3, 4, 5) = min( min(1, 2, 3), min(4, 5) )
*
* This is a reduction. See MapReduce example to have another examples of reduction.
*/
struct CatBestAnswers : public eo::mpi::HandleResponseParallelApply<EOT>
{
CatBestAnswers()
{
best.fitness( 1000000000. );
}
/*
our structure inherits the member _wrapped from HandleResponseFunction,
which is a HandleResponseFunction pointer;
it inherits also the member _d (like Data), which is a pointer to the
ParallelApplyData used in the HandleResponseParallelApply&lt;EOT&gt;. Details
of this data are contained in the file eoParallelApply. We need just to know that
it contains a member assignedTasks which maps a worker rank and the sent slice
to be processed by the worker, and a reference to the processed table via the
call of the data() function.
*/
// if EOT were a template, we would have to do: (thank you C++ :)
// using eo::mpi::HandleResponseParallelApply<EOT>::_wrapped;
// using eo::mpi::HandleResponseParallelApply<EOT>::d;
void operator()(int wrkRank)
{
eo::mpi::ParallelApplyData<EOT> * d = _data;
// Retrieve informations about the slice processed by the worker
int index = d->assignedTasks[wrkRank].index;
int size = d->assignedTasks[wrkRank].size;
// call to the wrapped function HERE
(*_wrapped)( wrkRank );
// Compare fitnesses of evaluated individuals with the best saved
for(int i = index; i < index+size; ++i)
{
if( best.fitness() < d->table()[ i ].fitness() )
{
eo::log << eo::quiet << "Better solution found:" << d->table()[i].fitness() << std::endl;
best = d->table()[ i ];
}
}
}
protected:
EOT best;
};
int main(int ac, char** av)
{
eo::mpi::Node::init( ac, av );
// eo::log << eo::setlevel( eo::debug );
eo::log << eo::setlevel( eo::quiet );
eoParser parser(ac, av);
unsigned int popSize = parser.getORcreateParam((unsigned int)100, "popSize", "Population Size", 'P', "Evolution Engine").value();
unsigned int dimSize = parser.getORcreateParam((unsigned int)10, "dimSize", "Dimension Size", 'd', "Evolution Engine").value();
uint32_t seedParam = parser.getORcreateParam((uint32_t)0, "seed", "Random number seed", 0).value();
if (seedParam == 0) { seedParam = time(0); }
make_parallel(parser);
make_help(parser);
rng.reseed( seedParam );
eoUniformGenerator< double > gen(-5, 5);
eoInitFixedLength< EOT > init( dimSize, gen );
eoEvalFuncPtr< EOT, double, const std::vector< double >& > mainEval( real_value );
eoEvalFuncCounter< EOT > eval( mainEval );
// until this point, everything (but eo::mpi::Node::init) is exactly as in an sequential version.
// We then instanciate the parallel algorithm. The store is directly used by the eoParallelPopLoopEval, which
// internally uses parallel apply.
int rank = eo::mpi::Node::comm().rank();
eo::mpi::DynamicAssignmentAlgorithm assign;
if( rank == eo::mpi::DEFAULT_MASTER )
{
eoPop< EOT > pop( popSize, init );
eo::log << "Size of population : " << popSize << std::endl;
eo::mpi::ParallelApplyStore< EOT > store( eval, eo::mpi::DEFAULT_MASTER );
store.wrapHandleResponse( new CatBestAnswers );
eoParallelPopLoopEval< EOT > popEval( assign, eo::mpi::DEFAULT_MASTER, &store );
//eoParallelPopLoopEval< EOT > popEval( assign, eo::mpi::DEFAULT_MASTER, eval, 5 );
eo::log << eo::quiet << "Before first evaluation." << std::endl;
popEval( pop, pop );
eo::log << eo::quiet << "After first evaluation." << std::endl;
pop = eoPop< EOT >( popSize, init );
popEval( pop, pop );
eo::log << eo::quiet << "After second evaluation." << std::endl;
eo::log << eo::quiet << "DONE!" << std::endl;
} else
{
eoPop< EOT > pop; // the population doesn't have to be initialized, as it is not used by workers.
eoParallelPopLoopEval< EOT > popEval( assign, eo::mpi::DEFAULT_MASTER, eval );
popEval( pop, pop );
}
return 0;
}
//-----------------------------------------------------------------------------

196
test/eompi/t-mpi-multipleRoles.cpp Executable file
View file

@ -0,0 +1,196 @@
/*
(c) Thales group, 2012
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation;
version 2 of the License.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Contact: http://eodev.sourceforge.net
Authors:
Benjamin Bouvier <benjamin.bouvier@gmail.com>
*/
/*
* This file shows an example of how to make a hierarchy between nodes, when using a parallel apply. In this basic
* test, the master delegates the charge of finding workers to 2 "sub" masters, which then send part of the table to
* their workers.
*
* It's convenient to establish a role map, so as to clearly identify every role:
* - The node 0 is the general master, that delegates the job. It sends the table to the 2 submasters, and waits for the
* results.
* - Nodes 1 and 2 are the worker of the first job: the delegates. They receive the elements of the table and
* retransmit them to the subworkers. They play the roles of worker in the delegating job, and master in the plus one
* job.
* - Following nodes (3 to 6) are workers of the plus one job. They do the real job. Nodes 3 and 5 are attached to
* submaster 1, 4 and 6 to submaster 2.
*
* This test requires exactly 7 hosts. If the size is bigger, an exception will be thrown at the beginning.
**/
# include <paradiseo/eompi.h>
# include <paradiseo/eompi/eoParallelApply.h>
# include <paradiseo/eompi/eoTerminateJob.h>
# include "t-mpi-common.h"
# include <iostream>
# include <vector>
using namespace std;
using namespace eo::mpi;
/*
* This class allows the user to easily serialize a vector of elements which implement eoserial::Persistent too.
*
* T is the type of the contained element, which must implement eoserial::Persistent too.
*
* Here, it contains SerializableBase<int>, which is a serializable integer that can be used as an integer.
*/
template< class T >
struct SerializableVector : public std::vector<T>, public eoserial::Persistent
{
public:
void unpack( const eoserial::Object* obj )
{
this->clear();
eoserial::Array* vector = static_cast<eoserial::Array*>( obj->find("vector")->second );
vector->deserialize< std::vector<T>, eoserial::Array::UnpackObjectAlgorithm >( *this );
}
eoserial::Object* pack( void ) const
{
eoserial::Object* obj = new eoserial::Object;
obj->add("vector", eoserial::makeArray< std::vector<T>, eoserial::SerializablePushAlgorithm >( *this ) );
return obj;
}
};
// The real job to execute, for the subworkers: add one to each element of a table.
struct SubWork: public eoUF< SerializableBase<int>&, void >
{
void operator() ( SerializableBase<int> & x )
{
cout << "Subwork phase." << endl;
++x;
}
};
// Function called by both subworkers and delegates.
// v is the vector to process, rank is the MPI rank of the sub master
void subtask( vector< SerializableBase<int> >& v, int rank )
{
// Attach workers according to nodes.
// Submaster with rank 1 will have ranks 3 and 5 as subworkers.
// Submaster with rank 2 will have ranks 4 and 6 as subworkers.
vector<int> workers;
workers.push_back( rank + 2 );
workers.push_back( rank + 4 );
DynamicAssignmentAlgorithm algo( workers );
SubWork sw;
// Launch the job!
ParallelApplyStore< SerializableBase<int> > store( sw, rank );
store.data( v );
ParallelApply< SerializableBase<int> > job( algo, rank, store );
job.run();
EmptyJob stop( algo, rank );
}
// Functor applied by submasters. Wait for the subworkers responses and then add some random processing (here, multiply
// each result by two).
// Note that this work receives a vector of integers as an entry, while subworkers task's operator receives a simple
// integer.
struct Work: public eoUF< SerializableVector< SerializableBase<int> >&, void >
{
void operator() ( SerializableVector< SerializableBase<int> >& v )
{
cout << "Work phase..." << endl;
subtask( v, Node::comm().rank() );
for( unsigned i = 0; i < v.size(); ++i )
{
v[i] *= 2;
}
}
};
int main(int argc, char** argv)
{
// eo::log << eo::setlevel( eo::debug );
Node::init( argc, argv );
if( Node::comm().size() != 7 ) {
throw std::runtime_error("World size should be 7.");
}
SerializableVector< SerializableBase<int> > v;
v.push_back(1);
v.push_back(3);
v.push_back(3);
v.push_back(7);
v.push_back(42);
// As submasters' operator receives a vector<int> as an input, and ParallelApply takes a vector of
// operator's input as an input, we have to deal with a vector of vector of integers for the master task.
vector< SerializableVector< SerializableBase<int> > > metaV;
// Here, we send twice the same vector. We could also have splitted the first vector into two vectors, one
// containing the beginning and another one containing the end.
metaV.push_back( v );
metaV.push_back( v );
// Assigning roles is done by comparing MPI ranks.
switch( Node::comm().rank() )
{
// Nodes from 0 to 2 are implicated into the delegating task.
case 0:
case 1:
case 2:
{
Work w;
DynamicAssignmentAlgorithm algo( 1, 2 );
ParallelApplyStore< SerializableVector< SerializableBase<int> > > store( w, 0 );
store.data( metaV );
ParallelApply< SerializableVector< SerializableBase<int> > > job( algo, 0, store );
job.run();
if( job.isMaster() )
{
EmptyJob stop( algo, 0 );
v = metaV[0];
cout << "Results : " << endl;
for(unsigned i = 0; i < v.size(); ++i)
{
cout << v[i] << ' ';
}
cout << endl;
}
}
break;
// Other nodes are implicated into the subwork task.
default:
{
// all the other nodes are sub workers
int rank = Node::comm().rank();
if ( rank == 3 or rank == 5 )
{
subtask( v, 1 );
} else {
subtask( v, 2 );
}
}
break;
}
return 0;
}

169
test/eompi/t-mpi-multistart.cpp Executable file
View file

@ -0,0 +1,169 @@
# include <paradiseo/eompi/eoMultiStart.h>
using namespace eo::mpi;
#include <stdexcept>
#include <iostream>
#include <sstream>
#include <paradiseo/eo.h>
#include <paradiseo/eo/es.h>
/*
* This file is based on the tutorial lesson 1. We'll consider that you know all the EO
* related parts of the algorithm and we'll focus our attention on parallelization.
*
* This file shows an example of multistart applied to a eoSGA (simple genetic
* algorithm). As individuals need to be serialized, we implement a class inheriting
* from eoReal (which is the base individual), so as to manipulate individuals as they
* were eoReal AND serialize them.
*
* The main function shows how to launch a multistart job, with default functors. If you
* don't know which functors to use, these ones should fit the most of your purposes.
*/
using namespace std;
/*
* eoReal is a vector of double: we just have to serializes the value and the fitness.
*/
class SerializableEOReal: public eoReal<double>, public eoserial::Persistent
{
public:
SerializableEOReal(unsigned size = 0, double value = 0.0) :
eoReal<double>(size, value)
{
// empty
}
void unpack( const eoserial::Object* obj )
{
this->clear();
eoserial::unpackArray
< std::vector<double>, eoserial::Array::UnpackAlgorithm >
( *obj, "vector", *this );
bool invalidFitness;
eoserial::unpack( *obj, "invalid_fitness", invalidFitness );
if( invalidFitness )
{
this->invalidate();
} else
{
double f;
eoserial::unpack( *obj, "fitness", f );
this->fitness( f );
}
}
eoserial::Object* pack( void ) const
{
eoserial::Object* obj = new eoserial::Object;
obj->add( "vector", eoserial::makeArray< std::vector<double>, eoserial::MakeAlgorithm >( *this ) );
bool invalidFitness = this->invalid();
obj->add( "invalid_fitness", eoserial::make( invalidFitness ) );
if( !invalidFitness )
{
obj->add( "fitness", eoserial::make( this->fitness() ) );
}
return obj;
}
};
// REPRESENTATION
//-----------------------------------------------------------------------------
// define your individuals
typedef SerializableEOReal Indi;
// EVAL
//-----------------------------------------------------------------------------
// a simple fitness function that computes the euclidian norm of a real vector
// @param _indi A real-valued individual
double real_value(const Indi & _indi)
{
double sum = 0;
for (unsigned i = 0; i < _indi.size(); i++)
sum += _indi[i]*_indi[i];
return (-sum); // maximizing only
}
/************************** PARALLELIZATION JOB *******************************/
int main(int argc, char **argv)
{
Node::init( argc, argv );
// PARAMETRES
// all parameters are hard-coded!
const unsigned int SEED = 133742; // seed for random number generator
const unsigned int VEC_SIZE = 8; // Number of object variables in genotypes
const unsigned int POP_SIZE = 100; // Size of population
const unsigned int T_SIZE = 3; // size for tournament selection
const unsigned int MAX_GEN = 100; // Maximum number of generation before STOP
const float CROSS_RATE = 0.8; // Crossover rate
const double EPSILON = 0.01; // range for real uniform mutation
const float MUT_RATE = 0.5; // mutation rate
eoEvalFuncPtr<Indi> eval( real_value );
eoPop<Indi> pop;
eoUniformGenerator< double > generator;
eoInitFixedLength< Indi > init( VEC_SIZE, generator );
pop = eoPop<Indi>( POP_SIZE, init );
eoDetTournamentSelect<Indi> select(T_SIZE);
eoSegmentCrossover<Indi> xover;
eoUniformMutation<Indi> mutation(EPSILON);
eoGenContinue<Indi> continuator(MAX_GEN);
/* Does work too with a steady fit continuator. */
// eoSteadyFitContinue< Indi > continuator( 10, 50 );
eoSGA<Indi> gga(select, xover, CROSS_RATE, mutation, MUT_RATE,
eval, continuator);
/* How to assign tasks, which are starts? */
DynamicAssignmentAlgorithm assignmentAlgo;
/* Before a worker starts its algorithm, how does it reinits the population?
* There are a few default usable functors, defined in eoMultiStart.h.
*
* This one (ReuseSamePopEA) doesn't modify the population after a start, so
* the same population is reevaluated on each multistart: the solution tend
* to get better and better.
*/
ReuseSamePopEA< Indi > resetAlgo( continuator, pop, eval );
/**
* How to send seeds to the workers, at the beginning of the parallel job?
* This functors indicates that seeds should be random values.
*/
GetRandomSeeds< Indi > getSeeds( SEED );
// Builds the store
MultiStartStore< Indi > store(
gga,
DEFAULT_MASTER,
resetAlgo,
getSeeds);
// Creates the multistart job and runs it.
// The last argument indicates that we want to launch 5 runs.
MultiStart< Indi > msjob( assignmentAlgo, DEFAULT_MASTER, store, 5 );
msjob.run();
if( msjob.isMaster() )
{
msjob.best_individuals().sort();
std::cout << "Global best individual has fitness " << msjob.best_individuals().best_element().fitness() << std::endl;
}
MultiStart< Indi > msjob10( assignmentAlgo, DEFAULT_MASTER, store, 10 );
msjob10.run();
if( msjob10.isMaster() )
{
msjob10.best_individuals().sort();
std::cout << "Global best individual has fitness " << msjob10.best_individuals().best_element().fitness() << std::endl;
}
return 0;
}

220
test/eompi/t-mpi-parallelApply.cpp Executable file
View file

@ -0,0 +1,220 @@
/*
(c) Thales group, 2012
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation;
version 2 of the License.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Contact: http://eodev.sourceforge.net
Authors:
Benjamin Bouvier <benjamin.bouvier@gmail.com>
*/
/*
* This file shows an example of use of parallel apply, in the following context: each element of a table is
* incremented... in a parallel fashion. While this operation is very easy to perform even on a single host, it's just
* an example for parallel apply use.
*
* The table of integers has to be serialized before it's sent. The wrapper object SerializableBase allows to serialize
* any type and manipulate it like this type: SerializableBase<int> can be exactly be used as an integer.
*
* Besides, this is also a test for assignment (scheduling) algorithms, in different cases. The test succeeds if and
* only if the program terminates without any segfault ; otherwise, there could be a deadlock which prevents the end or
* a segfault at any time.
*
* One important thing is to instanciate an EmptyJob after having launched a ParallelApplyJob, so as the workers to be
* aware that the job is done (as it's a MultiJob).
*
* This test needs at least 3 processes to be launched. Under this size, it will directly throw an exception, at the
* beginning;
*/
# include <paradiseo/eompi.h>
# include <paradiseo/eompi/eoParallelApply.h>
# include <paradiseo/eompi/eoTerminateJob.h>
# include "t-mpi-common.h"
# include <iostream>
# include <cstdlib>
# include <vector>
using namespace std;
using namespace eo::mpi;
/*
* The function to be called on each element of the table: just increment the value.
*/
struct plusOne : public eoUF< SerializableBase<int>&, void >
{
void operator() ( SerializableBase<int> & x )
{
++x; // implicit conversion of SerializableBase<int> in the integer it contains
}
};
/*
* Internal structure representating a test.
*/
struct Test
{
AssignmentAlgorithm * assign; // used assignment algorithm for this test.
string description; // textual description of the test
int requiredNodesNumber; // number of required nodes. NB : chosen nodes ranks must be sequential
};
int main(int argc, char** argv)
{
// eo::log << eo::setlevel( eo::debug ); // if you like tty full of rainbows, decomment this line and comment the following one.
eo::log << eo::setlevel( eo::quiet );
bool launchOnlyOne = false ; // Set this to true if you wanna launch only the first test.
Node::init( argc, argv );
// Initializes a vector with random values.
srand( time(0) );
vector< SerializableBase<int> > v;
for( int i = 0; i < 1000; ++i )
{
v.push_back( rand() );
}
// We need to be sure the values are correctly incremented between each test. So as to check this, we save the
// original vector into a variable originalV, and put an offset variable to 0. After each test, the offset is
// incremented and we can compare the returned value of each element to the value of each element in originalV +
// offset. If the two values are different, there has been a problem.
int offset = 0;
vector< SerializableBase<int> > originalV = v;
// Instanciates the functor to apply on each element
plusOne plusOneInstance;
vector< Test > tests;
const int ALL = Node::comm().size();
if( ALL < 3 ) {
throw std::runtime_error("Needs at least 3 processes to be launched!");
}
// Tests are auto described thanks to member "description"
Test tIntervalStatic;
tIntervalStatic.assign = new StaticAssignmentAlgorithm( 1, REST_OF_THE_WORLD, v.size() );
tIntervalStatic.description = "Correct static assignment with interval."; // workers have ranks from 1 to size - 1
tIntervalStatic.requiredNodesNumber = ALL;
tests.push_back( tIntervalStatic );
if( !launchOnlyOne )
{
Test tWorldStatic;
tWorldStatic.assign = new StaticAssignmentAlgorithm( v.size() );
tWorldStatic.description = "Correct static assignment with whole world as workers.";
tWorldStatic.requiredNodesNumber = ALL;
tests.push_back( tWorldStatic );
Test tStaticOverload;
tStaticOverload.assign = new StaticAssignmentAlgorithm( v.size()+100 );
tStaticOverload.description = "Static assignment with too many runs.";
tStaticOverload.requiredNodesNumber = ALL;
tests.push_back( tStaticOverload );
Test tUniqueStatic;
tUniqueStatic.assign = new StaticAssignmentAlgorithm( 1, v.size() );
tUniqueStatic.description = "Correct static assignment with unique worker.";
tUniqueStatic.requiredNodesNumber = 2;
tests.push_back( tUniqueStatic );
Test tVectorStatic;
vector<int> workers;
workers.push_back( 1 );
workers.push_back( 2 );
tVectorStatic.assign = new StaticAssignmentAlgorithm( workers, v.size() );
tVectorStatic.description = "Correct static assignment with precise workers specified.";
tVectorStatic.requiredNodesNumber = 3;
tests.push_back( tVectorStatic );
Test tIntervalDynamic;
tIntervalDynamic.assign = new DynamicAssignmentAlgorithm( 1, REST_OF_THE_WORLD );
tIntervalDynamic.description = "Dynamic assignment with interval.";
tIntervalDynamic.requiredNodesNumber = ALL;
tests.push_back( tIntervalDynamic );
Test tUniqueDynamic;
tUniqueDynamic.assign = new DynamicAssignmentAlgorithm( 1 );
tUniqueDynamic.description = "Dynamic assignment with unique worker.";
tUniqueDynamic.requiredNodesNumber = 2;
tests.push_back( tUniqueDynamic );
Test tVectorDynamic;
tVectorDynamic.assign = new DynamicAssignmentAlgorithm( workers );
tVectorDynamic.description = "Dynamic assignment with precise workers specified.";
tVectorDynamic.requiredNodesNumber = tVectorStatic.requiredNodesNumber;
tests.push_back( tVectorDynamic );
Test tWorldDynamic;
tWorldDynamic.assign = new DynamicAssignmentAlgorithm;
tWorldDynamic.description = "Dynamic assignment with whole world as workers.";
tWorldDynamic.requiredNodesNumber = ALL;
tests.push_back( tWorldDynamic );
}
for( unsigned int i = 0; i < tests.size(); ++i )
{
// Instanciates a store with the functor, the master rank and size of packet (see ParallelApplyStore doc).
ParallelApplyStore< SerializableBase<int> > store( plusOneInstance, eo::mpi::DEFAULT_MASTER, 3 );
// Updates the contained data
store.data( v );
// Creates the job with the assignment algorithm, the master rank and the store
ParallelApply< SerializableBase<int> > job( *(tests[i].assign), eo::mpi::DEFAULT_MASTER, store );
// Only master writes information
if( job.isMaster() )
{
cout << "Test : " << tests[i].description << endl;
}
// Workers whose rank is inferior to required nodes number have to run the test, the other haven't anything to
// do.
if( Node::comm().rank() < tests[i].requiredNodesNumber )
{
job.run();
}
// After the job run, the master checks the result with offset and originalV
if( job.isMaster() )
{
// This job has to be instanciated, not launched, so as to tell the workers they're done with the parallel
// job.
EmptyJob stop( *(tests[i].assign), eo::mpi::DEFAULT_MASTER );
++offset;
for(unsigned i = 0; i < v.size(); ++i)
{
cout << v[i] << ' ';
if( originalV[i] + offset != v[i] )
{
cout << " <-- ERROR at this point." << endl;
exit( EXIT_FAILURE );
}
}
cout << endl;
}
// MPI synchronization (all the processes wait to be here).
Node::comm().barrier();
delete tests[i].assign;
}
return 0;
}

133
test/eompi/t-mpi-wrapper.cpp Executable file
View file

@ -0,0 +1,133 @@
/*
(c) Thales group, 2012
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation;
version 2 of the License.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Contact: http://eodev.sourceforge.net
Authors:
Benjamin Bouvier <benjamin.bouvier@gmail.com>
*/
/*
* This file shows an example of how to wrap a handler of a job store. Here, the wrapped handler is the "IsFinished"
* one. The only function that has been added is that the wrapper prints a message on standard output, indicating what
* the wrapped function returns as a result.
*
* This test is performed on a parallel apply job, the same as in parallelApply. The main difference is when
* instanciating the store.
*/
# include <paradiseo/eompi.h>
# include <paradiseo/eompi/eoParallelApply.h>
# include <paradiseo/eompi/eoTerminateJob.h>
# include "t-mpi-common.h"
# include <iostream>
# include <cstdlib>
# include <vector>
using namespace std;
using namespace eo::mpi;
// Job functor.
struct plusOne : public eoUF< SerializableBase<int>&, void >
{
void operator() ( SerializableBase<int>& x )
{
++x;
}
};
/*
* Shows the wrapped result of IsFinished, prints a message and returns the wrapped value.
* times is an integer counting how many time the wrapper (hence the wrapped too) has been called.
*/
template< class EOT >
struct ShowWrappedResult : public IsFinishedParallelApply<EOT>
{
using IsFinishedParallelApply<EOT>::_wrapped;
ShowWrappedResult ( IsFinishedParallelApply<EOT> * w = 0 ) : IsFinishedParallelApply<EOT>( w ), times( 0 )
{
// empty
}
bool operator()()
{
bool wrappedValue = _wrapped->operator()(); // (*_wrapped)();
cout << times << ") Wrapped function would say that it is " << ( wrappedValue ? "":"not ") << "finished" << std::endl;
++times;
return wrappedValue;
}
private:
int times;
};
int main(int argc, char** argv)
{
// eo::log << eo::setlevel( eo::debug );
eo::log << eo::setlevel( eo::quiet );
Node::init( argc, argv );
srand( time(0) );
vector< SerializableBase<int> > v;
for( int i = 0; i < 1000; ++i )
{
v.push_back( rand() );
}
int offset = 0;
vector< SerializableBase<int> > originalV = v;
plusOne plusOneInstance;
StaticAssignmentAlgorithm assign( v.size() );
ParallelApplyStore< SerializableBase<int> > store( plusOneInstance, eo::mpi::DEFAULT_MASTER, 1 );
store.data( v );
// This is the only thing which changes: we wrap the IsFinished function.
// According to RAII, we'll delete the invokated wrapper at the end of the main ; the store won't delete it
// automatically.
ShowWrappedResult< SerializableBase<int> > wrapper;
store.wrapIsFinished( &wrapper );
ParallelApply< SerializableBase<int> > job( assign, eo::mpi::DEFAULT_MASTER, store );
// Equivalent to:
// Job< ParallelApplyData<int> > job( assign, 0, store );
job.run();
EmptyJob stop( assign, eo::mpi::DEFAULT_MASTER );
if( job.isMaster() )
{
++offset;
for(unsigned i = 0; i < v.size(); ++i)
{
cout << v[i] << ' ';
if( originalV[i] + offset != v[i] )
{
cout << " <-- ERROR at this point." << endl;
exit( EXIT_FAILURE );
}
}
cout << endl;
}
return 0;
}

104
test/eompi/template-job.cpp Executable file
View file

@ -0,0 +1,104 @@
# include <paradiseo/eompi.h>
using namespace eo::mpi;
/*
* This file is a template for a new eo::mpi::Job. You have everything that should be necessary to implement a new
* parallelized algorithm.
*
* Replace __TEMPLATE__ by the name of your algorithm (for instance: MultiStart, ParallelApply, etc.).
*/
template< class EOT >
struct __TEMPLATE__Data
{
};
template< class EOT >
class SendTask__TEMPLATE__ : public SendTaskFunction< __TEMPLATE__Data< EOT > >
{
public:
using SendTaskFunction< __TEMPLATE__Data< EOT > >::_data;
void operator()( int wrkRank )
{
// TODO implement me
}
};
template< class EOT >
class HandleResponse__TEMPLATE__ : public HandleResponseFunction< __TEMPLATE__Data< EOT > >
{
public:
using HandleResponseFunction< __TEMPLATE__Data< EOT > >::_data;
void operator()( int wrkRank )
{
// TODO implement me
}
};
template< class EOT >
class ProcessTask__TEMPLATE__ : public ProcessTaskFunction< __TEMPLATE__Data< EOT > >
{
public:
using ProcessTaskFunction< __TEMPLATE__Data<EOT> >::_data;
void operator()()
{
// TODO implement me
}
};
template< class EOT >
class IsFinished__TEMPLATE__ : public IsFinishedFunction< __TEMPLATE__Data< EOT > >
{
public:
using IsFinishedFunction< __TEMPLATE__Data< EOT > >::_data;
bool operator()()
{
// TODO implement me
}
};
template< class EOT >
class __TEMPLATE__Store : public JobStore< __TEMPLATE__Data< EOT > >
{
public:
__TEMPLATE__Data<EOT>* data()
{
// TODO implement me
return 0;
}
};
template< class EOT >
class __TEMPLATE__ : public MultiJob< __TEMPLATE__Data< EOT > >
{
public:
__TEMPLATE__( AssignmentAlgorithm & algo,
int masterRank,
__TEMPLATE__Store< EOT > & store ) :
MultiJob< __TEMPLATE__Data< EOT > >( algo, masterRank, store )
{
// TODO implement me
}
};
/*
int main(int argc, char **argv)
{
Node::init( argc, argv );
DynamicAssignmentAlgorithm assignmentAlgo;
__TEMPLATE__Store<int> store;
__TEMPLATE__<int> job( assignmentAlgo, DEFAULT_MASTER, store );
}
*/