Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r0822

Warning

EXPERIMENTAL FEATURE: This feature is intended for demonstration purposes only. This feature may be modified or removed from the

D s platform
without warning in a future release. It should not be deployed in a production environment.

D toc

You can enable an API endpoint that converts the your recipe steps to generate a specific output into Python Pandas code. When executed, this generated Python Pandas code applies transformations to your specified dataset and generates the specified output. 

Alpha Release

This endpoint is the beginning of enabling the 

D s platform
rtrue
 to integrate with pre-existing Python data pipelines.

This documentation provides information on how to explore the capabilities of the 

D s platform
 to generate Python code for your external data pipelines. 

Enable

To enable generation of Python Pandas code, please complete the following:

Steps:

  1. D s config
    methodws
  2. Locate the following setting and set it to Enabled:

    Code Block
    Wrangle to Python Conversion
  3. The feature is now enabled.

Limitations

  • This endpoint does not currently support multi-dataset operations.
  • The generated Python code does not yield readable columns in the output code.
  • Conversion of Wrangle script to Python code is supported for CSV files only.

v4 OutputObjects WrangleToPython Create

This section contains reference documentation on the API endpoint. This endpoint method is applied to a specified outputObject. This outputObject is the result of execution of a specific recipe (wrangledDataset). That recipe has references to its source importedDatasets and connections.

D s api version

This API enables generation of Python Pandas code for the Wrangle recipe associated with an output object.

Version:  v4

D s terms
areaoutputobjects

Required Permissions

D s api auth

Request

Request Type: POST

Endpoint:

Code Block
/v4/outputObjects/<id>/wrangleToPython

Request URI - Example:

Code Block
/v4/outputObjects/3/wrangleToPython

Request Body:

The following defines the running environment used for the outputObject. 

Code Block
{
    "execution": "spark"
}

Response

Response Status Code - Success:  200 - OK

Response Body Example: 

The generated response is the Python Pandas code:

Code Block
{
from trifacta.transform_functions.function_definitions import Replace
import pandas as pd
import numpy as np

def run_transforms(df0=None):
  if (df0 is None):
    df0 = pd.read_csv('input.csv', skip_blank_lines=False, lineterminator='\n', dtype=str, encoding='UTF-8')
  replace1 = Replace('x', 'u', False, False)
  replace2 = Replace('x', 'u', True, False)
  replace3 = Replace('x', 'u', True, True)
  replace4 = Replace('x', '\\\\', True, False)
  df3 = pd.DataFrame({'new_column1': df0['col1'].apply((lambda x: replace1.exec(x))), 'new_column2': df0['col1'].apply((lambda x: replace2.exec(x))), 'new_column3': df0['col1'].apply((lambda x: replace3.exec(x))), 'new_column4': df0['col1'].apply((lambda x: replace4.exec(x)))})
  return df3
}

Reference

For more information , see 

D s api refdoclink
operation/getPythonScriptForOutputObjectInput