grid_map(f,
args_list,
cleanup=True,
mem_free=u'1G',
name=u'gridmap_job',
num_slots=1,
temp_dir=u'/scratch/',
white_list=None,
queue='nlp.q',
quiet=True)
| source code
|
Maps a function onto the cluster.
- Parameters:
f (function) - The function to map on args_list
args_list (list) - List of arguments to pass to f
cleanup (bool) - Should we remove the stdout and stderr temporary files for each
job when we're done? (They are left in place if there's an
error.)
mem_free (basestring) - Estimate of how much memory each job will need (for scheduling).
(Not currently used, because our cluster does not have that
setting enabled.)
name (basestring) - Base name to give each job (will have a number add to end)
num_slots (int) - Number of slots each job should use.
temp_dir (basestring) - Local temporary directory for storing output for an individual
job.
white_list (list of basestring) - If specified, limit nodes used to only those in list.
queue (basestring) - The SGE queue to use for scheduling.
quiet (bool) - When true, do not output information about the jobs that have
been submitted.
Note:
This can only be used with picklable functions (i.e., those that
are defined at the module or class level).
|