AWS S3 Sync Helper

Usage

s3sync(src, dest, pats = NULL, workflow = NULL, dryrun = FALSE)

Arguments

src: (character(1))
S3 source path.
dest: (character(1))
Local destination path.
pats: (tibble())
Patterns tibble with inex ("in" or "ex") and pat (pattern) columns. If NULL and workflow is given, the workflow's schema-derived patterns are used (see wf_sync_patterns()); if both are NULL, everything is excluded.
workflow: (character(1))
Workflow name (e.g. "wigits", "dragen") whose schema-derived sync patterns to use. Ignored when pats is supplied.
dryrun: (logical(1))
If TRUE, passes --dryrun to aws s3 sync so operations are displayed without being executed.

Examples

if (FALSE) { # \dontrun{
src <- "s3://my-awesome-bucket/path/to/run1"
dest <- sub("s3:/", "~/s3", src)
pats <- tibble::tribble(
  ~inex, ~pat,
  "ex", "*",
  "in", "*foo.csv"
)
s3sync(src, dest, pats)

# let the workflow's schemas decide what to pull down
s3sync(src, dest, workflow = "wigits")
} # }