This section covers changes between release on the following topics:

Release 8.10

Introducing project users and roles management

Beginning in this release, administrators can manage the access and roles of users of , which includes basic access to the product and role-based access controls to types of objects created in the project.

A Dataprep admin can create and assign roles to project users.

For more information on these distinctions, see Overview of Authorization.

For more information, see Roles Page.

For more information, see Privileges and Roles Reference.

Dataprep admin is a super user

The Dataprep admin role is a super-user of the product. It is automatically assigned to anyone who is designated as the project owner in .

NOTE: This role has owner access to user-created objects, such as flows and connections, within the project.

NOTE: The project owner is automatically granted the Dataprep admin role. This role can be assigned to non-project owners. It grants a project user all of the privileges of the project owner within . If the Dataprep admin role is un-assigned to a project owner, it is automatically granted back to the project owner on next login.

NOTE: When roles are modified, some menu items may not be displayed to specific users because of their role assignments. Dataprep admins may receive inquiries about menu option availability. A user's assigned roles could be a likely source for why a menu option is not available to the user.

Dataprep admin can edit any global connection

After an administrator has made a connection global (available to all users):

Fine-grained sharing permissions on individual objects

At a more granular level, role-based access controls (RBAC) determine access to individual  at finer-grained levels for flows, connections, and plans. This capability has been present in the product for a few releases. With this release, RBAC is elevated to project-level privileges within a role definition.

Tip: You can still change the permissions to a shared object for individual users. These fine-grained permissions can be assigned at the time of sharing by the object's owner or a Dataprep admin. They can also be changed at a later time.


NOTE: Project-level permissions that are defined through a user's assigned roles define the maximum and default level of permissions that can be assigned when an object is shared. When an object is shared, you can determine if you wish to set the access level on the object to a lower level if desired.

For more information, see Overview of Authorization.

For more information, see Overview of Sharing.


Release 8.1

Fine-grained sharing permissions on individual objects

Beginning in this release, you can change the permissions to a shared object for individual users. These fine-grained permissions can be assigned at the time of sharing by the object's Owner or a workspace admin. They can also be changed at a later time.


NOTE: In this release, fine-grained sharing permissions apply to flows and connections only.

For more information, see Overview of Sharing.


Release 8.0

Scheduled outputs now inherit service account settings

Previously, when companion service accounts were enabled for use, outputs for scheduled jobs that had already been created did not inherit the use of the companion service accounts. Without a companion service account, these scheduled outputs cannot be executed, and their jobs would fail with a No Dataflow service account provided error message. The issue and prior workaround are described below.

Beginning in Release 8.0, scheduled outputs that do not have a companion service account inherit the companion service account defined in the user's preferences.

NOTE: The workaround described below to fix scheduled outputs is no longer required.

See User Profile Page.

Release 7.10

Changes to IAM roles for service accounts

Recently, Google announced changes to the required permissions for IAM roles used by service accounts to access platform resources.

NOTE: The following changes will be deployed by Google on January 27, 2021. These changes have been enacted by Google and are outside of the platform. All administrators of should review the following changes to requirements and verify the impacts on their deployments.

customers

Steps:

  1. Login as an administrator.
  2. From the left nav bar, select User menu > Admin console > Project settings.
  3. Locate the following setting: Manage access to data using user IAM permissions. Check the current setting:

    SettingDescription
    EnabledUsers in your deployment are using IAM permissions. Please review and complete the following steps.
    DisabledUsers in your deployment are not affected. No further action is required. See "Other product editions" below for details.
    DefaultDefault setting is Disabled. See previous.

    For more information, see Dataprep Project Settings Page.

New requirements:

NOTE: With this change, users who connect to platform resources using IAM roles must meet one of the following requirements to run jobs on .

Recommendations:

For uninterrupted service, please do one of the following:

  1. Project administrators can grant their users the iam.serviceAccounts.actAs permission on the default compute service account:

    <project-number>-compute@developer.gserviceaccount.com


  2. (preferred) Project administrators should provision compute service accounts of narrower scope for their users. Users should be educated on how to use them.

    If users are now using companion service accounts, the outputs of any scheduled jobs must be updated. See "Fixing schedules" below.

Fixing schedules:

If individual users are now using companion service accounts to run jobs, all affected schedules must be updated.

Steps:

NOTE: Each user should complete the following steps for each of flow that they own.

  1. Identify if the flow contains a schedule:
    1. Open the flow in Flow View.
    2. At the top of the page, you may see one of the following icons.
    3. This icon indicates that there is an enabled schedule. Its outputs must be updated. Please see Step 2.

    4. This icon indicates that there is a schedule, but it is disabled. Its outputs must be updated. Please see Step 2.

    5. If neither icon is present, then the flow does not have a schedule. Please go to the next flow.
  2. For flows that do contain enabled or disabled schedules, please do the following:
    1. Locate the scheduled outputs. In the Flow View canvas, these outputs are labeled Scheduled Output.
    2. Select a scheduled output.
    3. In the right panel, under Scheduled Destinations click Edit.
    4. In the Scheduled publishing settings, click the Advanced Settings caret to open it.
    5. In the Service Account textbox, insert the new companion service account that you should use for the scheduled job.
  3. Repeat Step 2 for any other scheduled outputs in the flow.
  4. Repeat these steps for other scheduled flows that you own.

References:

Other product editions

No action is required. Actions on are already executed with the proper permissions.

For more information on permissions, see Required Dataprep User Permissions.



Release 7.9

Manage access to data using IAM permissions

You can optionally configure the workspace to manage access to BigQuery and data based on a fine-grained set of permissions in the user's IAM role.

Reduced scope of minimum required permissions

Prior to Release 7.9, the required IAM permissions for were the following list. These permissions had to be included in any IAM role assigned to a product user.

However, many of these previously required permissions do not directly apply to use of the product. Beginning in Release 7.9, the list of required permissions has been reduced to only those required to access the and other elements of the product.

NOTE: Permissions that are needed for access to other services are now considered optional for use of the product itself. In the list below, these optional permissions are primarily tied to use of BigQuery.

NOTE: The storage.buckets.list permission must be enabled at the project level. All other storage.* permissions only need to be enabled on the staging bucket.

NOTE: The bigquery.jobs.create permission is required if you wish to use to BigQuery at all. The other permissions are optional and can be applied at the project or dataset level.

AreaPermissionRelease 7.8 and earlierRelease 7.9 and later
Generalresourcemanager.projects.getrequiredrequired

dataprep.projects.userequiredrequired
BigQuerybigquery.datasets.getrequiredoptional

bigquery.jobs.createrequired

optional

NOTE: This permission is required if you wish to use to BigQuery at all.


bigquery.tables.createrequiredoptional

bigquery.tables.getrequiredoptional

bigquery.tables.getDatarequiredoptional

bigquery.tables.listrequiredoptional

compute.machineTypes.getrequiredrequired

dataflow.jobs.createrequiredrequired

dataflow.jobs.getrequiredrequired

dataflow.messages.listrequiredrequired

dataflow.metrics.getrequiredrequired
base storagestorage.buckets.getrequiredrequired

storage.buckets.listrequired

required

NOTE: This permission must be enabled at the project level.



storage.objects.createrequiredrequired

storage.objects.deleterequiredrequired

storage.objects.getrequiredrequired

storage.objects.listrequiredrequired

storage.objects.updaterequiredoptional

New permissions

The following permissions are newly tracked for use with :

NOTE: Please verify that any newly required permissions are added to user roles.


AreaPermissionRelease 7.8 and earlierRelease 7.9 and later
BigQuerybigquery.tables.deleten/a

optional

NOTE: This permission is required on a table if you wish to publish to it. Otherwise, the table is reported as being read-only. For more information, see Required Dataprep User Permissions.


bigquery.datasets.createn/aoptional

bigquery.datasets.updaten/aoptional

dataflow.jobs.canceln/aoptional

For more information, see Required Dataprep User Permissions.