# EXSLT - str:tokenize - Implementer Page

Version: 1 Status: new User Page: index.html XML Definition: str.tokenize.xml Function Package: str.tokenize.zip #### Function Syntax
    node-set str:tokenize(string, string?)
#### Template Syntax
    <xsl:call-template name="str:tokenize">
       <xsl:with-param name="string" select="string" />
       <xsl:with-param name="delimiters" select="string" />?
    </xsl:call-template>
The `str:tokenize` function splits up a string and returns a node set of `token` elements, each containing one token from the string. The first argument is the string to be tokenized. The second argument is a string consisting of a number of characters. Each character in this string is taken as a delimiting character. The string given by the first argument is split at any occurrence of any of these characters. For example:
    str:tokenize('2001-06-03T11:40:23', '-T:')
Gives the node set consisting of:
    <token>2001</token>
    <token>06</token>
    <token>03</token>
    <token>11</token>
    <token>40</token>
    <token>23</token>
If the second argument is omitted, the default is the string `'&#x9;&#xA;&#xD;&#x20;'` (i.e. whitespace characters). Thus:
    str:tokenize('date math str')
Gives the node set consisting of:
    <token>date</token>
    <token>math</token>
    <token>str</token>
If the second argument is an empty string, the function returns a set of `token` elements, each of which holds a single character. Thus:
    str:tokenize('foo', '')
Gives the node set consisting of:
    <token>f</token>
    <token>o</token>
    <token>o</token>
The template version of this function returns a result tree fragment consisting of a number of `token` elements. An implementation of this extension function in the EXSLT str namespace must conform to the behaviour described in this document. ## Implementations Built-in support for `str:tokenize` is available in the following XSLT processors:
Processor | Processor Version | Implemented Version
--- | --- | ---
4XSLT, from 4Suite. | 0.12.0a3 | 1
libxslt from Daniel Veillard et al. | 1.0.19 | 1
The following implementations of `str:tokenize` are available:
Language | Implemented Version | Creator | Date | Download
--- | --- | --- | --- | ---
EXSLT Function | 1 | Jeni Tennison | 2001-06-03 | str.tokenize.function.xsl
XSLT Template | 1 | Jeni Tennison | 2001-06-03 | str.tokenize.template.xsl
Javascript | 1 | Chris Bayes | 2001-06-21 | str.tokenize.js
Msxsl | 1 | Chris Bayes | 2001-06-21 | str.tokenize.msxsl.xsl
## Use Cases Use Case Package: str.tokenize.use-cases.zip The following use cases illustrate the functionality of `str:tokenize`.
Details | Calling Stylesheet | XML Data | Expected Result | Category | Template
--- | --- | --- | --- | --- | ---
1 | str.tokenize.1.xsl | str.tokenize.data.1.xml | - | example | N
## Change History Submitted: 2001-06-03 Creator: Jeni Tennison(http://www.jenitennison.com/) The str:tokenize function splits a string at the occurrences of given characters.
Version | Modified | By | Details
--- | --- | --- | ---
1.1 | 2001-06-21 | Jeni Tennison | Added Javascript and MSXML implementations.
1.2 | 2001-11-18 | Jeni Tennison | Indicated behaviour when second argument is an empty string; str:tokenize() should return a list of tokens, each containing a single character.
1.3 | 2002-08-21 | Craig Stewart | Added 4XSLT and libxslt implementation to the list.
1.4 | 2002-11-12 | Craig Stewart | Updated 4XSLT version to 0.12.0a3.

http://www.exslt.org/str/functions/tokenize/str.tokenize.html last modified 2002-11-12