tokenized

| Home | Libraries | People | FAQ | More

##### tokenized

Syntax | Code
--- | ---

Pipe

    rng | boost::adaptors::tokenized(regex)
    rng | boost::adaptors::tokenized(regex, i)
    rng | boost::adaptors::tokenized(regex, rndRng)
    rng | boost::adaptors::tokenized(regex, i, flags)
    rng | boost::adaptors::tokenized(regex, rndRng, flags)

Function

    boost::adaptors::tokenize(rng, regex)
    boost::adaptors::tokenize(rng, regex, i)
    boost::adaptors::tokenize(rng, regex, rndRng)
    boost::adaptors::tokenize(rng, regex, i, flags)
    boost::adaptors::tokenize(rng, regex, rndRng, flags)

- Precondition:
  - Let `T` denote `typename range_value<decltype(rng)>::type`, then `regex` has the type `basic_regex<T>` or is implicitly convertible to one of these types.
  - `i` has the type `int`.
  - the `value_type` of `rndRng` is `int`.
  - `flags` has the type `regex_constants::syntax_option_type`.
- Returns: A range whose iterators behave as if they were the original iterators wrapped in `regex_token_iterator`. The first iterator in the range would be constructed by forwarding all the arguments of `tokenized()` to the `regex_token_iterator` constructor.
- Throws: Whatever constructing and copying equivalent `regex_token_iterator`s might throw.
- Range Category: Random Access Range
- Range Return Type: `boost::tokenized_range<decltype(rng)>`
- Returned Range Category: Random Access Range

###### tokenized_example

    #include <boost/range/adaptor/tokenized.hpp>
    #include <boost/range/algorithm/copy.hpp>
    #include <boost/assign.hpp>
    #include <iterator>
    #include <iostream>
    #include <vector>

    int main(int argc, const char* argv[])
    {
        using namespace boost::adaptors;

        typedef boost::sub_match< std::string::iterator > match_type;

        std::string input = " a b c d e f g hijklmnopqrstuvwxyz";
        boost::copy(
            input | tokenized(boost::regex("\\w+")),
            std::ostream_iterator<match_type>(std::cout, "\n"));

        return 0;
    }

This would produce the output:

    a
    b
    c
    d
    e
    f
    g
    hijklmnopqrstuvwxyz