To improve performance of your Hive or relational connections, custom SQL queries can be enabled to push the initial filtration of table rows and columns back the database, which is more efficient at performing this task. Instead of loading the entire table into the and then performing the filtration through the Transformer page, you can insert basic SQL commands as part of your relational queries to collect only the rows and columns of interest from the source.
When enabled, custom SQL query is available for Hive, Redshift, and all relational sources.
See Create Dataset with SQL.
Locate the following settings:
"webapp.connectivity.customSQLQuery.enabled": true, "webapp.connectivity.customSQLQuery.enableMultiStatement": false,
|Set to |
When set to
When custom SQL query is enabled, you can enter customized SQL statements in the imported dataset page as part of the import process. See Import Data Page.
For examples, see Create Dataset with SQL.
After a dataset has been imported using custom SQL, you can edit the SQL as needed. See Dataset Details Page.