pyspark.pandas.Series.str.split#

str.split(pat=None, n=-1, expand=False)#

Split strings around given separator/delimiter.

Splits the string in the Series from the beginning, at the specified delimiter string. Equivalent to str.split().

Parameters:
patstr, optional

String or regular expression to split on. If not specified, split on whitespace.

nint, default -1 (all)

Limit number of splits in output. None, 0 and -1 will be interpreted as return all splits.

expandbool, default False

Expand the split strings into separate columns.

  • If True, n must be a positive integer, and return DataFrame expanding dimensionality.

  • If False, return Series, containing lists of strings.

Returns:
Series, DataFrame

Type matches caller unless expand=True (see Notes).

See also

str.rsplit

Splits string around given separator/delimiter, starting from the right.

str.join

Join lists contained as elements in the Series/Index with passed delimiter.

Notes

The handling of the n keyword depends on the number of found splits:

  • If found splits > n, make first n splits only

  • If found splits <= n, make all splits

  • If for a certain row the number of found splits < n, append None for padding up to n if expand=True

If using expand=True, Series callers return DataFrame objects with n + 1 columns.

Note

Even if n is much larger than found splits, the number of columns does NOT shrink unlike pandas.

Examples

>>> s = ps.Series(["this is a regular sentence",
...                "https://docs.python.org/3/tutorial/index.html",
...                np.nan])

In the default setting, the string is split by whitespace.

>>> s.str.split()
0                   [this, is, a, regular, sentence]
1    [https://docs.python.org/3/tutorial/index.html]
2                                               None
dtype: object

Without the n parameter, the outputs of rsplit and split are identical.

>>> s.str.rsplit()
0                   [this, is, a, regular, sentence]
1    [https://docs.python.org/3/tutorial/index.html]
2                                               None
dtype: object

The n parameter can be used to limit the number of splits on the delimiter. The outputs of split and rsplit are different.

>>> s.str.split(n=2)
0                     [this, is, a regular sentence]
1    [https://docs.python.org/3/tutorial/index.html]
2                                               None
dtype: object

>>> s.str.rsplit(n=2)
0                     [this is a, regular, sentence]
1    [https://docs.python.org/3/tutorial/index.html]
2                                               None
dtype: object

The pat parameter can be used to split by other characters.

>>> s.str.split(pat = "/")
0                         [this is a regular sentence]
1    [https:, , docs.python.org, 3, tutorial, index...
2                                                 None
dtype: object

When using expand=True, the split elements will expand out into separate columns. If NaN is present, it is propagated throughout the columns during the split.

>>> s.str.split(n=4, expand=True)
                                               0     1     2        3         4
0                                           this    is     a  regular  sentence
1  https://docs.python.org/3/tutorial/index.html  None  None     None      None
2                                           None  None  None     None      None

For slightly more complex use cases like splitting the html document name from a url, a combination of parameter settings can be used.

>>> s.str.rsplit("/", n=1, expand=True)
                                    0           1
0          this is a regular sentence        None
1  https://docs.python.org/3/tutorial  index.html
2                                None        None

Remember to escape special characters when explicitly using regular expressions.

>>> s = ps.Series(["1+1=2"])
>>> s.str.split(r"\+|=", n=2, expand=True)
   0  1  2
0  1  1  2