Character preprocessor allows to translate a sequence
of characters to a single character so it may be used
for example for Unicode sequences translation.
Browsable list of tokens.
org.netbeans.api.lexer.TokenSequence delegates
all its operation to this class so any service provider
delivering this class will be able to produce token sequences.
There are various implementations of the token list:
BatchTokenList
- predecessor of batch token lists
TextTokenList
- token list over immutable char sequence
CopyTextTokenList
- token list over text input
that needs to be copied.
List of token lists that collects all token lists for a given language path.
There can be both lists with/without joining of the embedded sections.
Initial implementation attempted to initialize the list of token lists lazily
upon asking for it by client.
The character translator allows to translate a sequence
of characters to a single character so it may be used
for example for generic Unicode sequences translation.