1 StringTokenization Overview l StringTokenizer class l Some StringTokenizer methods l StringTokenizer examples

2 StringTokenizer class l A token is a portion of a string that is separated from another portion of that string by one or more chosen characters (called delimiters). Example: Assuming that a while space character (i.e., blank, ‘\n’ (new line ), ‘\t’ (tab), or ‘\r’ (carriage return)) is a delimiter, then the string: “I like KFUPM very much” has the tokens: “I”, “like”, “KFUPM”, “very”, and “much” l The StringTokenizer class contained in the java.util package can be used to break a string into separate tokens. This is particularly useful in those situations in which we want to read and process one token at a time; the BufferedReader class does not have a method to read one token at a time. l The StringTokenizer constructors are: StringTokenizer(String str)Uses white space characters as a delimiters. The delimiters are not returned. StringTokenizer(String str, String delimiters) delimiters is a string that specifies the delimiters. The delimiters are not returned. StringTokenizer(String str, String delimiters, boolean delimAsToken) If delimAsToken is true, then each delimiter is also returned as a token; otherwise delimiters are not returned.

3 Some StringTokenizer methods l Some StringTokenizer methods are: l To break a string into tokens, a loop having one of the following forms may be used: StringTokenizer tokenizer = new StringTokenizer(stringName); while(tokenizer.hasMoreTokens( )) { String token = tokenizer.nextToken( ); // process the token... } int countTokens( )Using the current set of delimiters, the method returns the number of tokens left. boolean hasMoreTokens( )Returns true if one or more tokens remain in the string; otherwise it returns false. String nextToken( ) throws NoSuchElementException Returns the next token as a string. Throws an exception if there are no more tokens String nextToken(String newDelimiters) throws NoSuchElementException Returns the next token as a string and sets the delimiters to newDelimiters. Throws an exception if there are no more tokens.

4 StringTokenizer examples StringTokenizer tokenizer = new StringTokenizer (stringName); int tokenCount = tokenizer.countTokens( ); for(int k = 1; k <= tokenCount; k++) { String token = tokenizer.nextToken( ); // process token... } l Example1: import java.util.StringTokenizer; public class Tokenizer1 { public static void main(String[ ] args) { StringTokenizer wordFinder = new StringTokenizer ("We like KFUPM very much"); while( wordFinder.hasMoreTokens( ) ) System.out.println( wordFinder.nextToken( ) ); }

5 StringTokenizer examples (Cont’d) l Example2: The following program reads grades from the keyboard and finds their average. The grades are read in one line. import java.io.*; import java.util.StringTokenizer; public class Tokenizer5 { public static void main(String[ ] args) throws IOException { BufferedReader stdin = new BufferedReader(new InputStreamReader(System.in)); System.out.println("Enter grades in one line:"); String inputLine = stdin.readLine( ); StringTokenizer tokenizer = new StringTokenizer(inputLine); int count = 0; float grade, sum = 0.0F; try { while( tokenizer.hasMoreTokens( ) ) { grade = Float.parseFloat( tokenizer.nextToken( ) ); if(grade >= 0 && grade <= 100) { sum += grade; count++; } if(count > 0) System.out.println("\nThe average = "+ sum / count); else System.out.println("No valid grades entered"); }

6 StringTokenizer examples (Cont’d) catch(NumberFormatException e) { System.err.println("Error - an invalid float value read"); } l Example3: Given that a text file grades.txt contains ids and quiz grades of students: the program on the next slide will display the id, number of quizzes taken, and average of each student:

7 StringTokenizer examples (Cont’d) import java.io.*; import java.util.StringTokenizer; public class Tokenizer6 { public static void main(String[ ] args) throws IOException { BufferedReader inputStream = new BufferedReader(new FileReader("grades.txt")); StringTokenizer tokenizer; String inputLine, id; int count; float sum; System.out.println("ID# Number of QuizzesAverage\n"); while((inputLine = inputStream.readLine( )) != null) { tokenizer = new StringTokenizer(inputLine); id = tokenizer.nextToken( ); count = tokenizer.countTokens( ); sum = 0.0F; while( tokenizer.hasMoreTokens( ) ) sum += Float.parseFloat( tokenizer.nextToken( ) ); System.out.println(id + " " + count + " ” + sum / count); }