Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version next

...

Info

NOTE: If you are installing custom UDFs and the

d-s-item
item
node
does not have an Internet connection, you should download the Java UDF SDK in an Internet-accessible location, build your customer UDF JAR there, and then upload the JAR to the
d-s-item
item
node
.

 

Overview

Each UDF can take one or more inputs and produces a single output value (map only).

...

  • In the
    D s webapp
    , previews are not available for user-defined functions.
  • Retaining state information across the exec method is unstable. More information is provided below.

    Info

    NOTE: When a recipe containing a user-defined function is applied to text data, any null characters cause records to be truncated by the running environment during

    d-s-serverphoton
    job execution. In these cases, please execute the job on Hadoopin the Spark running environment.

Enable Service

You must enable the Java UDF service in the 

D s platform
.

...

The above issue is likely to be caused by the Photon running

D s photon
 running environment sending too much data through the buffer of the UDF's Websocket service. By default, this buffer size is set to 1048576 bytes (1 MB).

The Photon running

D s photon
 running environment processes data through the Websocket service in 1024 (1 K) rows at a time for the input and output columns of the UDF. If the data in the input columns to the UDF or output columns from the UDF exceeds 1 KB (1024 characters) in total size for each row, the default size of the buffer is too small, since Photon the 
D s photon
 running environment processed 1K records at a time (1 K characters * 1 K rows > 1048576). The query then fails.

...