In the Run Job page, you can specify transformation and profiling jobs for the currently loaded recipe. Available options include output formats and output destinations.
NOTE: When you run a job in , the job is queued and executed on . observes the job in progress and reports progress as needed back into the application. does not control the execution of the job.
Tip: Jobs can be scheduled for periodic execution through Flow View page. For more information, see Add Schedule Dialog.
Tip: Columns that have been hidden in the Transformer page still appear in the generated output. Before you run a job, you should verify that all currently hidden columns are ok to include in the output.
Run Job Page
Profile Results: Optionally, you can disable profiling of your output, which can improve the speed of overall job execution. When the profiling job finishes, details are available through the Job Details page, including links to download results.
NOTE: Percentages for valid, missing, or mismatched column values may not add up to 100% due to rounding.
See Job Details Page.
You can add, remove, or edit the outputs generated from this job. By default, a CSV output for your home directory on the selected datastore is included in the list of destinations, which can be removed if needed. You must include at least one output destination.
From the available datastores in the left column, select the target for your publication.
Add Publishing Action
NOTE: Do not create separate publishing actions that apply to the same file or database table.
New/Edit: You can create new or modify existing connections. By default, the displayed connections support publishing. See Create Connection Window.
BigQuery: You can published your results to the current project or to a different one to which you have access.
NOTE: You must have read and write access to any BigQuery database to which you are publishing. For more information, see Using BigQuery.
To publish to a different project, click the BigQuery link at the front of the breadcrumb trail. Then, enter the identifier for the project where you wish to publish your job results.
Tip: Your projects and their identifiers are available for review through the menu bar. See UI Reference.
Click Go. Navigate to the database where you wish to write your BigQuery results. For more information, see BigQuery Browser.
Locate a publishing destination: Do one of the following.
NOTE: The publishing location must already exist before you can publish to it. The publishing user must have write permissions to the location.
Create a new file: Enter the filename under which to save the dataset.
As needed, you can parameterize the outputs that you are creating. Click Parameterize destination in the right panel. See Parameterize destination settings below.
To save the publishing destination, click Add.
To update a publishing action, hover over its entry. Then, click Edit.
To delete a publishing action, select Delete from its context menu.
If any variable parameters have been specified for the datasets or outputs of the flow, you can apply overrides to their default values. Click the listed default value and insert a new value. A variable can have an empty value.
NOTE: Unless this output is a scheduled destination, variable overrides apply only to this job. Subsequent jobs use the default variable values, unless specified again. No data validation is performed on entries for override values.
Tip: At the flow level, you can specify overrides at the flow level. Override values are applied to parameters of all types that are a case-sensitive match. However, values that are specified at runtime override flow-level overrides. For more information, see Manage Parameters Dialog.
For more information on variables, see Overview of Parameterization.
Depending on the type of output that you are generating, you must specify additional settings to define location, format, and other settings.
To execute the job as configured, click Run. The job is queued for execution.
imposes a limit on the size of the job as represented by the JSON passed in.
Tip: If this limit is exceeded, the job may fail with a
You can use the available REST APIs to execute jobs for known datasets. For more information, see API Reference.