Skip to content

salvus_flow.api

Central SalvusFlow remote job execution API.

User interactions should almost always happen through the functions defined in this module.

Documentation of the individual functions

salvus_flow.api.add_job_group()

def add_job_group(name: str, description: str, info: Optional[Dict] = None):
    ...

Add a new job group to the database.

  • Parameters

    • name – Name of the job group.

    • description – Description of the job group.

    • info – Optional info dictionary.


salvus_flow.api.get_all_job_groups()

def get_all_job_groups(include_job_count: bool = False):
    ...

Get all job groups from the database.

  • Parameters

    include_job_count – Count the jobs for each group. Potentially somewhat slow.


salvus_flow.api.get_job()

def get_job(site_name: str, job_name: str):
    ...

Get a SalvusJob object by querying the database.

  • Parameters

    • site_name – The site this job is on.

    • job_name – The name of the job.


salvus_flow.api.get_job_array()

def get_job_array(site_name: str, job_array_name: str):
    ...

Get a SalvusJobArray object.

  • Parameters

    • site_name – The site this job array is on.

    • job_array_name – The name of the job array.


salvus_flow.api.get_job_arrays()

def get_job_arrays(
    limit: int = 100,
    site_name: Optional[str] = None,
    job_array_status: Optional[
        List[Union[salvus_flow.sites.types.JobStatus, str]]
    ] = None,
    order_by: Iterable[str] = ("last_updated", "desc"),
    update_job_arrays: bool = False,
):
    ...

Query for a list of SalvusJobArray objects.

  • Parameters

    • limit – Maximum number of returned job arrays.

    • site_name – Only return job arrays from this site.

    • job_array_status – Limit returned jobs to job arrays with these statuses. Will return any job if not given.

    • order_by – Choose how to order by (database column, direction). Available database columns are "last_updated", "job_array_start_time", "job_array_end_time", and "status".

    • update_job_arrays – Update pending/running job arrays.


salvus_flow.api.get_jobs()

def get_jobs(
    limit: int = 100,
    site_name: Optional[str] = None,
    job_status: Optional[
        List[Union[salvus_flow.sites.types.JobStatus, str]]
    ] = None,
    order_by: Iterable[str] = ("last_updated", "desc"),
    update_jobs: bool = False,
    skip_jobs_from_job_arrays: bool = True,
    hide_archived_jobs: bool = True,
):
    ...

Query for a list of SalvusJob objects.

  • Parameters

    • limit – Maximum number of returned jobs.

    • site_name – Only return jobs from this site.

    • job_status – Limit returned jobs to jobs with these statuses. Will return jobs job if not given.

    • order_by – Choose how to order by (database column, direction). Available database columns are "last_updated", "job_start_time", "job_end_time", and "status".

    • update_jobs – Update pending/running jobs.

    • skip_jobs_from_job_arrays – Don’t return jobs that are part of a job array.

    • hide_archived_jobs – Hide archived jobs.


salvus_flow.api.get_site()

def get_site(site_name: str, verbosity: int = 1):
    ...

Get an initialized site object from the chose site.

  • Parameters

    • site_name – The name of the site.

    • verbosity – Verbosity level.


salvus_flow.api.initialize_site()

def initialize_site(site_name: str, verbosity: int = 1):
    ...

Initialize a site using information from the config TOML file.

  • Parameters

    • site_name – Name of the site.

    • verbosity – Verbosity level.


salvus_flow.api.run()

def run(
    *,
    site_name: str,
    input_file: Union[
        str, pathlib.Path, Dict, salvus_flow.utils.deep_setters._DeepSetter
    ],
    output_folder: Union[str, pathlib.Path],
    ranks: Optional[int] = None,
    wall_time_in_seconds: Optional[int] = None,
    ping_interval_in_seconds: Optional[int] = None,
    job_groups: List[str] = None,
    job_name: str = None,
    get_all: bool = False,
    overwrite: bool = False,
    delete_remote_files: bool = True,
    verbosity: int = 1
):
    ...

Run one Salvus simulation and block until it is done.

Will run the simulation, wait for it to be done, download all output files, and finally delete all remote files and move the job to the archive group.

  • Parameters

    • site_name – The site to run on.

    • input_file – The input file or simulation object for the run.

    • output_folder – Output files will be copied here.

    • ranks – The number of ranks to run with. Defaults to the site’s configuration value.

    • wall_time_in_seconds – The wall time in seconds if the site requires one.

    • ping_interval_in_seconds – How often to query the remote site about the job status. Defaults to the site’s configuration value.

    • job_groups – Job groups to attach to the job in the database.

    • job_name – Custom job name. Be careful with this.

    • get_all – Get all output files. Wavefield outputs are omitted by default.

    • overwrite – Delete the output folder before running if it already exists.

    • delete_remote_files – Delete remote files after the run has finished.

    • verbosity – The verbosity level.


salvus_flow.api.run_async()

def run_async(
    *,
    site_name: str,
    input_file: Union[
        str, pathlib.Path, Dict, salvus_flow.utils.deep_setters._DeepSetter
    ],
    ranks: Optional[int] = None,
    wall_time_in_seconds: Optional[int] = None,
    job_groups: List[str] = None,
    job_name: Optional[str] = None,
    verbosity: int = 1
):
    ...

Launch one Salvus simulation asynchronously.

This function will return immediately upon successful submission of the job.

  • Parameters

    • site_name – The site to run on.

    • input_file – The input file or simulation object for the run.

    • ranks – The number of ranks to run with. Defaults to the site’s configuration value.

    • wall_time_in_seconds – The wall time in seconds if the site requires one.

    • job_groups – Job groups to attach to the job in the database.

    • job_name – Custom job name. Be careful with this.

    • verbosity – The verbosity level.


salvus_flow.api.run_many()

def run_many(
    *,
    site_name: str,
    input_files: List[
        Union[
            str, pathlib.Path, Dict, salvus_flow.utils.deep_setters._DeepSetter
        ]
    ],
    output_folder: Union[str, pathlib.Path],
    ranks_per_job: Optional[int] = None,
    wall_time_in_seconds_per_job: Optional[int] = None,
    ping_interval_in_seconds: Optional[int] = None,
    get_all: bool = False,
    overwrite: bool = False,
    delete_remote_files: bool = True,
    verbosity: int = 1
):
    ...

Run many Salvus simulations and block until they are done.

Will run the simulations, wait them to be done, download all output files, and finally delete all remote files and move the jobs to the archive group.

  • Parameters

    • site_name – The site to run on.

    • input_files – A list of input files or simulation objects to be run.

    • output_folder – Output files will be copied here.

    • ranks_per_job – The number of ranks to run each job with. Defaults to the site’s configuration value.

    • wall_time_in_seconds_per_job – The wall time in seconds per job if the site requires one.

    • ping_interval_in_seconds – How often to query the remote site about the jobs’ status. Defaults to the site’s configuration value.

    • get_all – Get all output files. Wavefield outputs are omitted by default.

    • overwrite – Delete the output folder before running if it already exists.

    • delete_remote_files – Delete remote files after the run has finished.

    • verbosity – The verbosity level.


salvus_flow.api.run_many_async()

def run_many_async(
    *,
    site_name: str,
    input_files: List[
        Union[
            str, pathlib.Path, Dict, salvus_flow.utils.deep_setters._DeepSetter
        ]
    ],
    ranks_per_job: Optional[int] = None,
    wall_time_in_seconds_per_job: Optional[int] = None,
    job_groups: List[str] = None,
    verbosity: int = 1
):
    ...

Launch many Salvus simulations asynchronously.

This function will return immediately upon successful submission of the jobs.

  • Parameters

    • site_name – The site to run on.

    • input_files – A list of input files or simulation objects to be run.

    • ranks_per_job – The number of ranks to run each job with. Defaults to the site’s configuration value.

    • wall_time_in_seconds_per_job – The wall time in seconds per job if the site requires one.

    • job_groups – Job groups to attach to the job in the database.

    • verbosity – The verbosity level.


salvus_flow.api.set_config()

def set_config(site_config: Union[pathlib.Path, Dict], db_path: str):
    ...

Set the configuration for all API calls in this Python session.

  • Parameters

    • site_config – Filename or existing dictionary with the site configuration.

    • db_path – Path to the sqlite database used to internally keep track of things.


salvus_flow.api.set_default_config()

def set_default_config():
    ...

Reset the configuration for all API calls to the defaults.