- A service account can be used by one or more users, who are accessing the platform. For more information on service accounts, see https://cloud.google.com/compute/docs/access/service-accounts.
- For more information on the service accounts used by Dataflow to manage security and permissions while running Dataprep by Trifacta jobs, see https://cloud.google.com/dataflow/docs/concepts/security-and-permissions#security_and_permissions_for_pipelines_on_google_cloud_platform.
- Dataflow is a cloud-based data processing service for both batch and real-time data streaming applications. You can use it to set up processing pipelines for integrating, preparing and analyzing large data sets, such as those found in Web analytics or big data analytics applications. For more information, see https://cloud.google.com/dataflow.
Project Service Accounts
In the Google Cloud Console, select IAM > Service Accounts. The following service accounts are used by the product: Dataprep by Trifacta
Service Account Name
Service Account Name
<project-number> is the numeric project identifier.
In the Google Cloud Console, select IAM > Service Accounts. The following service accounts are used by the product:
Dataprep by Trifacta
Compute Engine Service Account
The Compute Engine service account is the default account for all project users to run jobs on Dataflow. A Compute Engine service account enables access to platform services for the compute engine instance on which the Dataprep by Trifacta application is hosted. When the product is enabled for your project, the appropriate compute engine service account is assigned at the project level. Automatically, all users of the project are assigned this account by default.
This service account has the following name:
<project-number>- identifier for the the project using the compute service account.
Tip: For all project users to use the default compute service engine account, no additional configuration is required.
Service Account Permissions
NOTE: A user may be able cancel a job from the Dataprep by Trifacta application , even though the user is not permitted to cancel the job in the running environment. The service account associated with the user's Dataprep by Trifacta account may have the appropriate permissions, but the user's personal account does not.
A service account must have the following permissions.
Required Dataflow permissions
For the list of minimum required permissions for access Dataflow, see https://cloud.google.com/dataflow/docs/concepts/access-control#roles.
NOTE: The ability to cancel a job from within the Dataprep by Trifacta application is temporarily disabled. When it is re-enabled, this permission will be required. You should leave this permission enabled, if possible.
To enable users to cancel Dataflow jobs, the service account must have the following permission:
To run Dataprep by Trifacta jobs on Dataflow, the actAs permission must be provisioned based on the following applicable scenario:
- When not using companion service account: User must have
iam.serviceAccounts.actAspermission specified at the project level or in the default compute engine service account.
- When using companion service account: User must have
iam.serviceAccounts.actAspermission on companion service account or granted explicitly to the user.
- IAM disabled: If you are not using IAM roles and have enabled companion service accounts, the Dataprep Service account role, which is assigned to the default service account has the actAs permission for the project.
Project owners require no additional permissions on the projects that they own.
If you have deployed a Virtual Private Cloud Security Controls perimeter, additional configuration is required to ensure that Dataprep by Trifacta can operate within the perimeter. For more information, see Configure VPC-SC Perimeter.
Data connectivity permissions
NOTE: Any service account that is used to run jobs must have at least the same permissions that are available through the IAM role to connect to data through the Dataprep by Trifacta application . For example, to run a job sourced from Cloud Storage datasets, the service account must have the ability to read those datasets accessed through a user's IAM role. The same applies to publishing datasets.
For more information on Cloud Storage and BigQuery permissions, see Required Dataprep User Permissions.
Cross-project access to data
If users are permitted to access data in Cloud Storage or BigQuery that is owned by another project, additional permissions are required.
- For more information, see Access Cross-Project Cloud Storage Buckets.
- For more information, see Access Cross-Project BigQuery Datasets.
Using Service Accounts
The service account that is used for a job is determined based on the following priority level, highest to lowest:
Job-level overrides: Individual users can override the default service account or their companion service account when executing individual jobs. Scheduled jobs use job-level overrides.
For more information, see Runtime Dataflow Execution Settings.
User preferences: If defined here, these service accounts are applied to individual users.
NOTE: If preferred, project owners can require the use of individual service accounts for each user of the project. Companion service accounts are described below.
Compute Engine service account: If no other service account is specified, then the project default service account is used.
NOTE: When the product is enabled, the default Compute Engine service account is provisioned for each user of the project.
Project service accounts
At the project level, all users are assigned the Compute Engine service account by default. See above.
Companion Service Account:
Optionally, project owners can require that service accounts be assigned to individual users. When enabled, companion service accounts can be assigned by the project owner or by the individual user.
NOTE: When companion service accounts are enabled, the Compute Engine service account is no longer available.
See "Companion Service Accounts" below.
Users can specify the service account to use for all of their jobs. For example, if a user is invited into multiple projects, that user may be required to submit jobs in all projects using the same service account.
NOTE: A service account assigned to a user's preferences takes precedence over the project-level service account.
For more information, see User Execution Settings Page.
A user's service account can be assigned by the user or, if companion service accounts is enabled, by the project owner, or both. See "Companion Service Accounts" below.
For individual jobs, a user can select the service account to use. This value overrides user preferences and project owner selections. For more information, see Run Job Page.
Using service accounts in your VPC
If you are running Dataprep by Trifacta in your enterprise Virtual Private Cloud (VPC), use of service accounts is different from above.
To execute sampling or other jobs that are initiated from inside the Transformer Page, the project compute-engine service account or the user's credentials are used.
For any job that is batched and delivered to an external running environment for execution within your VPC, service accounts are used as follows:
If a Companion Service Account is available, it is used.
- If a Companion Service Account is not available:
- If a user-specified service account is available, it is used.
- Otherwise, an auto-provisioned service account for the project is used for batch job execution.
NOTE: To run jobs in your VPC using a service account, you must enable the use of Workload Identity in the Google Cloud Platform and enable it for use in Dataprep by Trifacta. See Dataprep In-VPC Execution.
Custom Service Accounts
Any custom service accounts must be created through the Google IAM console.
- These accounts must have the required permissions for the user or users that will use them.
- For more information on creating IAM roles, see https://console.cloud.google.com/iam-admin/iam.
- For more information on service accounts, see https://cloud.google.com/compute/docs/access/service-accounts.
- For more information on creating service accounts, see https://cloud.google.com/iam/docs/creating-managing-service-accounts#iam-service-accounts-create-console.
Custom service account requirements
Any custom service account or companion service account must meet the following requirements:
- Service account must be defined in IAM console.
- The minimum set of permissions to access Dataprep by Trifacta and any related datastores must be included in the custom service account. See "Service Account Permissions" above.
- Permissions in each user's IAM role must be reflected in any custom service account applied to the user's account. Changes in one must be reflected in the other.
- Service account must be applied to the project in IAM console.
Tip: After custom service accounts are specified in the IAM console and assigned to the project, they can be used in the product. Custom service accounts can be applied at the project level, user level, or job execution level.
Companion Service Accounts
NOTE: A companion service account is applied only to the execution of Dataflow transformation jobs. Other job types, such as ingestion, publishing, Trifacta Photon transformation, or pushdown, use the service agent account or the submitting user's permissions.
This service account must be specified in the Google IAM console and contain all of the permissions required to access a user's data and run jobs on Dataflow. For more information on permissions, see "Service Account Permissions" above.
When companion service accounts are enabled:
- Companion service accounts must be specified for individual users of the project, instead of all users relying on the default Compute Engine service account in the project.
- Project owners can apply them for each user.
- Individual users can apply their own.
- The default Compute Engine service account is no longer available for use.
- Companion service accounts can be overridden for individual jobs when defining the job to execute.
- Previously created scheduled jobs automatically inherit and use the companion service account specified for the user.
Tip: Before enabling the feature, you should create and specify the companion service accounts. Then, when the feature is enabled, there is no service disruption.
NOTE: Changes to a user's permissions must be reflected in Dataprep by Trifacta and in the related companion service account.
Create companion service accounts
Like any custom service account, companion service accounts must be created in the IAM console and applied to the project. See "Custom Service Accounts" above.
Manage companion service accounts
After these service accounts have been created, you can assign a companion service account to each user of the project. For more information, see Service Accounts Page.
Tip: Individual users can also specify the companion service account through their user preferences. User preferences selections override any selections made by the project owner. See User Execution Settings Page.
Enable companion service accounts
A project owner must enable the use of companion service accounts.
NOTE: If the use of companion service accounts is later disabled, all project users revert to using the Compute Engine service account.
For more information, see Dataprep Project Settings Page .
This page has no comments.