com.cburch.editor.tokens
Class Tokenizer<T extends BasicToken>

java.lang.Object
  com.cburch.editor.tokens.Tokenizer<T>

public class Tokenizer<T extends BasicToken>
extends java.lang.Object

The class that implements the basic functionality behind tracking the tokens within a document and notifying other classes about the changes.

Version:

0.1 2005-05-31

Author:

Carl Burch

Constructor Summary

Tokenizer()
Constructs a Tokenizer for a null scanner and document.

Tokenizer(javax.swing.text.Document document, Scanner<? extends T> scanner)
Constructs a Tokenizer for the given document and scanner.

Method Summary

void	addTokenizerListener(TokenizerListener listener, boolean synchronous) Adds a listener to be notified whenever the token list has been changed.
javax.swing.text.Document	getDocument() Returns the document currently tracked by this tokenizer.
Scanner<? extends T>	getScanner() Returns the scanner used for finding tokens.
TokenList<T>	getTokenList() Returns the list of tokens that always holds the current list as maintained by this document.
boolean	isEnabled() Indicates whether the tokenizer is currently enabled.
void	removeTokenizerListener(TokenizerListener listener) Removes the designated listener from being notified, whether it is synchronous or asynchronous.
void	setDocument(javax.swing.text.Document value) Changes which document whose tokens should be tracked.
void	setEnabled(boolean value) Enables or disables the tokenizer.
void	setScanner(Scanner<? extends T> value) Changes the scanner used for finding tokens.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Tokenizer

public Tokenizer()

Constructs a Tokenizer for a null scanner and document. This leads to an empty list of tokens.

Tokenizer

public Tokenizer(javax.swing.text.Document document,
                 Scanner<? extends T> scanner)

Constructs a Tokenizer for the given document and scanner.

Parameters:

document - the initial document to be scanned.

scanner - the initial scanner to use on the document.

Method Detail

addTokenizerListener

public void addTokenizerListener(TokenizerListener listener,
                                 boolean synchronous)

Adds a listener to be notified whenever the token list has been changed. The listener can be either synchronous (notified immediately after the change takes place) or asynchronous (notified shortly after the change, by another thread).

A listener must be asynchronous if it will alter the document in any way: The Java API prevents these alterations from taking place when it is in the process of sending out modifications to the document. A syntax highlighter, in particular, must be asynchronous, because the changes to character styles in the document constitute changes to the document.

An example where a synchronous listener is useful is in parenthesis matching. Here, when the user inserts a parenthesis, we would want a listener to compute the parenthesis matches immediately, so that the subsequent change to caret position can be based on the new matching information. Generally, a listener should be synchronous unless it will change the document.

Parameters:

listener - the listener to be added.

synchronous - true if the listener should be notified immediately after the change.

removeTokenizerListener

public void removeTokenizerListener(TokenizerListener listener)

Removes the designated listener from being notified, whether it is synchronous or asynchronous.

Parameters:

listener - the listener to be removed.

getTokenList

public TokenList<T> getTokenList()

Returns the list of tokens that always holds the current list as maintained by this document.

Returns:

the list of tokens in the document.

getScanner

public Scanner<? extends T> getScanner()

Returns the scanner used for finding tokens.

Returns:

the scanner currently in use.

setScanner

public void setScanner(Scanner<? extends T> value)

Changes the scanner used for finding tokens. This will necessitate scanning the entire document. A null value will result in an empty token list.

Parameters:

value - the new scanner to use.

getDocument

public javax.swing.text.Document getDocument()

Returns the document currently tracked by this tokenizer.

Returns:

the document tracked.

setDocument

public void setDocument(javax.swing.text.Document value)

Changes which document whose tokens should be tracked.

Parameters:

value - the document whose tokens should be tracked, or null if we should use the empty document.

isEnabled

public boolean isEnabled()

Indicates whether the tokenizer is currently enabled.

Returns:

true if the tokenizer is enabled.

setEnabled

public void setEnabled(boolean value)

Enables or disables the tokenizer. Disabling the tokenizer will freeze the token list in its current state, until the tokenizer is enabled again. Enabling the tokenizer will force a rescan of the entire document.

Parameters:

value - true to enable, false to disable.