|Parameter||Description||Applicable CLI Commands|
(Required) The server and port number of the
Replace this value with the host and port of the running
. If it is not provided,
NOTE: In some environments, the
https:// protocol identifier may be required as part of the
Internal name of the connection. This name is referenced in your CLI scripts. It should be a single value without spaces.
NOTE: This value must be unique among your connection names.
The internal identifier for the connection. When a connection is created, it is assigned an internal numeric identifier. This ID or the
connection_name can be used to reference the connection in future commands.
Tip: This value is available when you hover over a connection in the application. See Flows Page.
The execution environment in which to run the job:
photon = Run on Photon running environment on
NOTE: If the
job_type parameter is not specified, CLI jobs are run on the Photon running environment.
hadoop = Run in the default running environment for your Hadoop cluster.
NOTE: When this job type is applied, your CLI scripts automatically transition to running jobs in Spark.
spark = Run on the Spark running environment in Hadoop.
databricksSpark = Run the job on the Azure Databricks running environment in Azure.
For more information on these running environments, see Running Environment Options.
|The internal identifier for the job. This value can be retrieved from the output of a completed |
on, profiling of your job is enabled. Default is
|Full UNIX path to the source TSV file. This file contains a URL pointing to the actual Hive or HDFS source: one TSV file for each job run. Executing user must have access to this file.|
Full UNIX path from the
|D s item|
to the CLI script file. Executing user must have access.
(Optional) Defines the action taken on second and subsequent publish operations:
(Optional), The output for a CSV job with
create publishing action includes the column headers as the first row. Default is
NOTE: If you use the
header option, you must also include the
single_file option, or this setting is ignored.
|(Optional) When |
true, CSV or JSON outputs are written to a single file. Default is
(Required) Defines the fully qualified URI to where the job results are written, as in the following examples:
output_path must include the protocol identifier or host and port number (if applicable).
This parameter specifies the base filename. If you are publishing files, the
publish_action parameter value may change the exact filename that is written.
Protocol is set in
pqt (Parquet), and
job_type=spark is required.
job_type=photon, you may generate
Name of Redshift or Hive database to which you are publishing or loading.
The table of the database to which you are publishing or loading.
The format of the output file from which to publish to Hive or Redshift tables. Accepted values:
pqt (Parquet), or
|Path to file containing definitions for multiple file or table targets to which to write the job's results. For more information, see CLI Publishing Options File.|
By default, the CLI automatically checks for schema validation when generating results to a pre-existing source.
If this flag is set, schema validation is skipped on results output.