Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r097

...

  1. To run a job, open the flow containing the recipe whose output you wish to generate.
  2. Locate the recipe. Click the recipe's output object icon. 
  3. On the right side of the screen, information about the output object is displayed. The output object defines:
    1. The type and location of the outputs, including filenames and method of updating.
    2. Profiling options
    3. Execution options 
  4. For now, you can ignore the options for the output object. Click Run Job.
  5. In the Run Job page, you can review the job as it is currently specified. 
  6. To run the job on 
    D s dataflow
    , select Dataflow.
  7. Click Run Job.
  8. The job is queued with default settings for execution on 
    D s dataflow
    .

...

  1. .

Tracking progress

You can track progress of your job through the following areas:

  • Flow View: select the output object. On the right side of the screen, click the Jobs tab. Your job in progress is listed. See Flow View Page. 
  • Job Details Page: Click the link in the Jobs tab. You can review progress and individual details related to your job. See Job Details Page. 

Download results

When your job has finished successfully, a Completed message is displayed in the Job Details page. 

...

Info

NOTE: You must have a connection configured to publish to an external datastore . For more information, see Connections Page.

...

available through the Connections page.

Output Options

When specifying the job you wish to run, you can define the following types of output options.

...

Tip

Tip: You can download PDF and JSON versions of the visual profile for offline analysis .

...

in the Job Details

...

page.

For more information, see Overview of Visual Profiling.

...

  • Parameter values can be defined at the flow level through Flow View. For more information, see Manage Parameters Dialog. 
  • These parameters values can be passed into the running environment and inserted into the output filename or table name. 
  • For more information, see Overview of Parameterization.

...

  • Execution settings: Within your preferences, you can define your execution options for jobs. By default, all of your jobs executed from flows within the project use these settings. For more information, see User Execution Settings Page.
  • Output object settings: The execution settings in the Execution Settings page can be overridden at the output object level. When you define an individual output object for a recipe, the execution settings that you specify in the Run Job page apply whenever the outputs are generated for this flow. See Runtime Dataflow Execution Settings.

Some examples of how these settings can be used are provided below.

...

For more information, see API Workflow Task - Run Job.

Configure Machine Resources

...

By default, 

D s product
 uses the service account that is configured for use with your project. In the Dataflow Execution Settings, enter the name for the Service Account to use. 

Info

NOTE: Under the Permissions tab, please verify that Include Google-provided role grants is selected.

To see the current service account for your project:

...

  • Each label must have a unique key within your project. 
  • You can create up to 64 labels per project.

D s also
inCQLtrue
labellabel = "job"