tokenizer
Class Tokenizer

java.lang.Object
  tokenizer.Tokenizer

public class Tokenizer
extends java.lang.Object

Tokenizer for the CIT 594 Logo project, Spring 2010.

Version:

February 14, 2010

Author:

David Matuszek

Constructor Summary

Tokenizer(java.io.Reader reader)
Creates a Tokenizer with an input source.

Tokenizer(java.lang.String input)
Creates a Tokenizer for a specific string.

Method Summary

Token	next() Returns the next Token.
void	pushBack() Requests that the Token most recently returned be returned again on the next call to next().

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Tokenizer

public Tokenizer(java.io.Reader reader)

Creates a Tokenizer with an input source.

Parameters:

reader - The input source to be tokenized.

Tokenizer

public Tokenizer(java.lang.String input)

Creates a Tokenizer for a specific string.

Parameters:

input - The string to be tokenized.

Method Detail

next

public Token next()

Returns the next Token.

Returns:

The next Token.

pushBack

public void pushBack()

Requests that the Token most recently returned be returned again on the next call to next().