getSparkCluster({ name })
Look up Spark cluster details before submitting a job. Use oleander for the built-in managed cluster or the name of a registered external cluster.
Return shape
uploadSparkJob(options)
Upload a Spark job artifact (mirrors oleander spark jobs upload). Creates a pending artifact version, uploads the file contents to presigned URLs, then commits the artifact so it’s ready to submit with submitSparkJob().
entrypoint must be a basename ending in .py or .jar. Pass file contents as a UTF-8 string (Python sources only) or as bytes (Uint8Array / ArrayBuffer).
Parameters
Entrypoint file contents. Strings are only allowed for
.py entrypoints.Artifact language. Inferred from the entrypoint extension when omitted (
.py → python; .jar requires java or scala).Basename of a Python dependencies archive, ending in
.zip or .egg. Requires pyFilesContent. Python artifacts only.Basename of a virtualenv archive, ending in
.tar.gz. Requires virtualenvContent. Python artifacts only.Return type: UploadSparkJobResult
Every upload creates a new artifact version, so re-uploading the same entrypoint never overwrites a previous version in place.
listSparkJobs(options?)
List your uploaded Spark job artifacts with pagination.
Parameters
Return type: ListSparkJobsResult
Each
SparkArtifactSummary has:
submitSparkJob(options)
Submit a Spark job for execution. cluster defaults to oleander.
For oleander-managed Spark, entrypoint is the uploaded script name. For external clusters, entrypoint is cluster-specific, such as an S3 URI for EMR Serverless or a Glue job name for Glue.
Common options
Oleander-managed options
EMR Serverless options
Glue options
For Glue jobs,
args are converted into key-value pairs. Pass them as alternating entries such as ["--source", "s3://bucket/input", "--target", "s3://bucket/output"].External cluster example
Machine types
TheSparkMachineType enum covers compute-optimized (c), balanced (b), and memory-optimized (m) options:
submitSparkJobAndWait(options)
Submit a Spark job and poll until it reaches a terminal state (COMPLETE, FAIL, or ABORT). Throws an error if the timeout is exceeded.
submitSparkJob options plus:
getRun(runId)
Get the current status of a run. Use this to poll a job submitted with submitSparkJob().