uses service accounts to execute all its jobs on . Access to is governed by Google service accounts. A service account is used by the to access services and resources in the .
The Compute Engine service account is the default account for all project users to run jobs on . A Compute Engine service account enables access to platform services for the compute engine instance on which the is hosted. When the product is enabled for your project, the appropriate compute engine service account is assigned at the project level. Automatically, all users of the project are assigned this account by default.
This service account has the following name:
<project-number>- identifier for the the project using the compute service account.
Tip: For all project users to use the default compute service engine account, no additional configuration is required.
NOTE: A user may be able cancel a job from the , even though the user is not permitted to cancel the job in the running environment. The service account associated with the user's may have the appropriate permissions, but the user's personal account does not.
A service account must have the following permissions.
For the list of minimum required permissions for access , see https://cloud.google.com/dataflow/docs/concepts/access-control#roles.
To enable users to cancel jobs, the service account must have the following permission:
To run jobs on , the actAs permission must be provisioned based on the following applicable scenario:
iam.serviceAccounts.actAspermission specified at the project level or in the default compute engine service account.
iam.serviceAccounts.actAspermission on companion service account or granted explicitly to the user.
Project owners require no additional permissions on the projects that they own.
For more information, see https://cloud.google.com/dataflow/docs/concepts/security-and-permissions#security_and_permissions_for_pipelines_on_google_cloud_platform.
NOTE: Any service account that is used to run jobs must have at least the same permissions that are available through the IAM role to connect to data through the . For example, to run a job sourced from datasets, the service account must have the ability to read those datasets accessed through a user's IAM role. The same applies to publishing datasets.
For more information on and BigQuery permissions, see Required Dataprep User Permissions.
If users are permitted to access data in or BigQuery that is owned by another project, additional permissions are required.
only supports running jobs on . Every job must be executed with a service account.
The service account that is used for a job is determined based on the following priority level, highest to lowest:
Job-level overrides: Individual users can override the default service account or their companion service account when executing individual jobs. Scheduled jobs use job-level overrides.
For more information, see Dataflow Execution Settings.
User preferences: If defined here, these service accounts are applied to individual users.
Compute Engine service account: If no other service account is specified, then the project default service account is used.
At the project level, all users are assigned the Compute Engine service account by default. See above.
Companion Service Account:
Optionally, project owners can require that service accounts be assigned to individual users. When enabled, companion service accounts can be assigned by the project owner or by the individual user.
NOTE: When companion service accounts are enabled, the Compute Engine service account is no longer available.
See "Companion Service Accounts" below.
Users can specify the service account to use for all of their jobs. For example, if a user is invited into multiple projects, that user may be required to submit jobs in all projects using the same service account.
NOTE: A service account assigned to a user's preferences takes precedence over the project-level service account.
For more information, see Execution Settings Page.
A user's service account can be assigned by the user or, if companion service accounts is enabled, by the project owner, or both. See "Companion Service Accounts" below.
For individual jobs, a user can select the service account to use. This value overrides user preferences and project owner selections. For more information, see Run Job Page.
Any custom service accounts must be created through the Google IAM console.
Any custom service account or companion service account must meet the following requirements:
Tip: After custom service accounts are specified in the IAM console and assigned to the project, they can be used in the product. Custom service accounts can be applied at the project level, user level, or job execution level.
A companion service account is a replacement for the single Compute Engine service account for submitting jobs to on behalf of the user. For example, separate companion service accounts can be specified to enable access to different BigQuery tables between users. In this manner, a project owner can provide finer-grained access controls to individual users.
This service account must be specified in the Google IAM console and contain all of the permissions required to access a user's data and run jobs on . For more information on permissions, see "Service Account Permissions" above.
When companion service accounts are enabled:
Tip: Before enabling the feature, you should create and specify the companion service accounts. Then, when the feature is enabled, there is no service disruption.
NOTE: Changes to a user's permissions must be reflected in and in the related companion service account.
Like any custom service account, companion service accounts must be created in the IAM console and applied to the project. See "Custom Service Accounts" above.
After these service accounts have been created, you can assign a companion service account to each user of the project. For more information, see Service Accounts Page.
Tip: Individual users can also specify the companion service account through their user preferences. User preferences selections override any selections made by the project owner. See Execution Settings Page.
A project owner must enable the use of companion service accounts.
NOTE: If the use of companion service accounts is later disabled, all project users revert to using the Compute Engine service account.
For more information, see Dataprep Project Settings Page.