Word Count (Map Reduce) 499

Question

Using map reduce to count word frequency.

Example

chunk1: "Google Bye GoodBye Hadoop code"

chunk2: "lintcode code Bye"

Get MapReduce result:

Bye: 2

GoodBye: 1

Google: 1

Hadoop: 1

code: 2

lintcode: 1

Solution

MapReduce的map和reduce基本操作。

代码如下：

/**
 * Definition of OutputCollector:
 * class OutputCollector<K, V> {
 *     public void collect(K key, V value);
 *         // Adds a key/value pair to the output buffer
 * }
 */
public class WordCount {

    public static class Map {
        public void map(String key, String value, OutputCollector<String, Integer> output) {
            // Write your code here
            // Output the results into output buffer.
            // Ps. output.collect(String key, int value);
            StringTokenizer iter = new StringTokenizer(value);
            while(iter.hasMoreTokens()){
                String s = iter.nextToken();
                output.collect(s, 1);
            }
        }
    }

    public static class Reduce {
        public void reduce(String key, Iterator<Integer> values,
                           OutputCollector<String, Integer> output) {
            // Write your code here
            // Output the results into output buffer.
            // Ps. output.collect(String key, int value);
            int sum = 0;
            while(values.hasNext()){
                sum += values.next();
            }
            output.collect(key, sum);
        }
    }
}