Jobard

Overview

The Jobard framework (Job Array Daemon) allows submitting and monitoring the execution of embarassingly parallel jobs, in an harmonized, efficient and distributed way, on different target multiprocessor machines or infrastructures :

• standalone computer
• HPC infrastructures with batch schedulers such as PBS, HT Condor, Slurm, ...
• cloud computing infrastructure : Swarm, Kubernetes

Jobard includes many features to balance the processing load, manage priorities, report on the overall processing progress, report on processing errors, restart failed jobs, etc...

Homepage: https://jobard.gitlab-pages.ifremer.fr/documentation/

Usage

Jobard is used to process thousands of similar commands in a distributed infrastructure, while preventing from any processing bottleneck. It is mainly used at Ifremer/CERSAT for reprocessing activities and to process multiple inputs with the same command. This kind of task can be executed on multiple platforms in the same way, depending on where the input data are located.

Credits

This software is developed and maintained by Ifremer and supported by Copernicus though EUMETSAT.