Protobuf Functions¶

This page lists all protobuf functions available in Spark SQL.

---

from_protobuf¶

from_protobuf(data, messageName, descFilePath, options) - Converts a binary Protobuf value into a Catalyst value.

Examples:

> SELECT from_protobuf(s, 'Person', '/path/to/descriptor.desc', map()) IS NULL AS result FROM (SELECT NAMED_STRUCT('name', name, 'id', id) AS s FROM VALUES ('John Doe', 1), (NULL,  2) tab(name, id));
 [false]

Note:

The specified Protobuf schema must match actual schema of the read data, otherwise the behavior is undefined: it may fail or return arbitrary result. To deserialize the data with a compatible and evolved schema, the expected Protobuf schema can be set via the corresponding option.

Since: 4.0.0

to_protobuf¶

to_protobuf(child, messageName, descFilePath, options) - Converts a Catalyst binary input value into its corresponding Protobuf format result.

Examples:

> SELECT to_protobuf(s, 'Person', '/path/to/descriptor.desc', map('emitDefaultValues', 'true')) IS NULL FROM (SELECT NULL AS s);
 [true]

Since: 4.0.0