7.1 Overview

The Internationalization Module contains two classes for finding delimiters in Unicode strings:

• RWUBreakSearch finds the locations of breaks in text. This class correctly interprets whitespace and punctuation based on a specific locale.

• RWUTokenizer finds delimiters, and sequentially returns the tokens between those delimiters. By default, RWUTokenizer uses a predefined set of whitespace characters as delimiters. Optionally, it uses a specified set of arbitrary characters or a regular expression. Using a regular expression as a token delimiter permits complex, multicharacter delimiters.

This chapter describes how to use these classes.

© Copyright Rogue Wave Software, Inc. All Rights Reserved.
Rogue Wave and SourcePro are registered trademarks of Rogue Wave Software, Inc. in the United States and other countries. All other trademarks are the property of their respective owners.
Contact Rogue Wave about documentation or support issues.