Package org.apache.storm.starter

Class SkewedRollingTopWords

java.lang.Object
org.apache.storm.topology.ConfigurableTopology
org.apache.storm.starter.SkewedRollingTopWords
public class SkewedRollingTopWords extends ConfigurableTopology
This topology does a continuous computation of the top N words that the topology has seen in terms of cardinality. The top N computation is done in a completely scalable way, and a similar approach could be used to compute things like trending topics or trending images on Twitter. It takes an approach that assumes that some works will be much more common then other words, and uses partialKeyGrouping to better balance the skewed load.

  • Method Details

    • main

      public static void main(String[] args) throws Exception
      Throws:
      Exception

    • run

      protected int run(String[] args)
      Submits (runs) the topology.

      Usage: "SkewedRollingTopWords [topology-name] [-local]"

      By default, the topology is run locally under the name "slidingWindowCounts".

      Examples:

      ``` # Runs in remote/cluster mode, with topology name "production-topology" $ storm jar storm-starter-jar-with-dependencies.jar org.apache.storm.starter.SkewedRollingTopWords production-topology ```

      Specified by:
      run in class ConfigurableTopology
      Parameters:
      args - First positional argument (optional) is topology name, second positional argument (optional) defines whether to run the topology locally ("-local") or remotely, i.e. on a real cluster