org.apache.storm.topology

Class PersistentWindowedBoltExecutor<T extends State>

java.lang.Object

org.apache.storm.topology.WindowedBoltExecutor

org.apache.storm.topology.PersistentWindowedBoltExecutor<T>

All Implemented Interfaces:

Serializable, IBolt, IComponent, IRichBolt, IStatefulBolt<T>, IStatefulComponent<T>

public class PersistentWindowedBoltExecutor<T extends State>
extends WindowedBoltExecutor
implements IStatefulBolt<T>

Wraps a IStatefulWindowedBolt and handles the execution. Uses state and the underlying checkpointing mechanisms to save the tuples in window to state. The tuples are also kept in-memory by transparently caching the window partitions and checkpointing them as needed.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.storm.topology.WindowedBoltExecutor

LATE_TUPLE_FIELD

Constructor Summary

Constructors

Constructor and Description
PersistentWindowedBoltExecutor(IStatefulWindowedBolt<T> bolt)

Method Summary

Modifier and Type	Method and Description
void	execute(Tuple input) Process a single tuple of input.
void	initState(T state) This method is invoked by the framework with the previously saved state of the component.
protected WindowLifecycleListener<Tuple>	newWindowLifecycleListener()
void	preCommit(long txid) This is a hook for the component to perform some actions just before the framework commits its state.
void	prepare(Map<String,Object> topoConf, TopologyContext context, OutputCollector collector) Called when a task for this component is initialized within a worker on the cluster.
void	prePrepare(long txid) This is a hook for the component to perform some actions just before the framework prepares its state.
void	preRollback() This is a hook for the component to perform some actions just before the framework rolls back the prepared state.
protected void	start() Start the trigger policy and waterMarkEventGenerator if set.
protected void	validate(Map<String,Object> topoConf, BaseWindowedBolt.Count windowLengthCount, BaseWindowedBolt.Duration windowLengthDuration, BaseWindowedBolt.Count slidingIntervalCount, BaseWindowedBolt.Duration slidingIntervalDuration)

Methods inherited from class org.apache.storm.topology.WindowedBoltExecutor

boltExecute, boltExecute, cleanup, declareOutputFields, doPrepare, getComponentConfiguration, getState, getTopologyTimeoutMillis, restoreState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.storm.topology.IStatefulBolt

cleanup

Methods inherited from interface org.apache.storm.topology.IComponent

declareOutputFields, getComponentConfiguration

Constructor Detail

PersistentWindowedBoltExecutor

public PersistentWindowedBoltExecutor(IStatefulWindowedBolt<T> bolt)

Method Detail

prepare

public void prepare(Map<String,Object> topoConf,
                    TopologyContext context,
                    OutputCollector collector)

Description copied from interface: IBolt

Called when a task for this component is initialized within a worker on the cluster. It provides the bolt with the environment in which the bolt executes.

This includes the:

Specified by:

prepare in interface IBolt

Specified by:

prepare in interface IStatefulBolt<T extends State>

Overrides:

prepare in class WindowedBoltExecutor

Parameters:

topoConf - The Storm configuration for this bolt. This is the configuration provided to the topology merged in with cluster configuration on this machine.

context - This object can be used to get information about this task's place within the topology, including the task id and component id of this task, input and output information, etc.

collector - The collector is used to emit tuples from this bolt. Tuples can be emitted at any time, including the prepare and cleanup methods. The collector is thread-safe and should be saved as an instance variable of this bolt object.

See Also:

IBolt.prepare(Map, TopologyContext, OutputCollector)

validate

protected void validate(Map<String,Object> topoConf,
                        BaseWindowedBolt.Count windowLengthCount,
                        BaseWindowedBolt.Duration windowLengthDuration,
                        BaseWindowedBolt.Count slidingIntervalCount,
                        BaseWindowedBolt.Duration slidingIntervalDuration)

Overrides:

validate in class WindowedBoltExecutor

start

protected void start()

Description copied from class: WindowedBoltExecutor

Start the trigger policy and waterMarkEventGenerator if set.

Overrides:

start in class WindowedBoltExecutor

execute

public void execute(Tuple input)

Description copied from interface: IBolt

Process a single tuple of input. The Tuple object contains metadata on it about which component/stream/task it came from. The values of the Tuple can be accessed using Tuple#getValue. The IBolt does not have to process the Tuple immediately. It is perfectly fine to hang onto a tuple and process it later (for instance, to do an aggregation or join).

Tuples should be emitted using the OutputCollector provided through the prepare method. It is required that all input tuples are acked or failed at some point using the OutputCollector. Otherwise, Storm will be unable to determine when tuples coming off the spouts have been completed.

For the common case of acking an input tuple at the end of the execute method, see IBasicBolt which automates this.

Specified by:

execute in interface IBolt

Specified by:

execute in interface IStatefulBolt<T extends State>

Overrides:

execute in class WindowedBoltExecutor

Parameters:

input - The input tuple to be processed.

See Also:

IBolt.execute(Tuple)

initState

public void initState(T state)

Description copied from interface: IStatefulComponent

This method is invoked by the framework with the previously saved state of the component. This is invoked after prepare but before the component starts processing tuples.

Specified by:

initState in interface IStatefulComponent<T extends State>

Parameters:

state - the previously saved state of the component.

prePrepare

public void prePrepare(long txid)

Description copied from interface: IStatefulComponent

This is a hook for the component to perform some actions just before the framework prepares its state.

Specified by:

prePrepare in interface IStatefulComponent<T extends State>

preCommit

public void preCommit(long txid)

Description copied from interface: IStatefulComponent

This is a hook for the component to perform some actions just before the framework commits its state.

Specified by:

preCommit in interface IStatefulComponent<T extends State>

preRollback

public void preRollback()

Description copied from interface: IStatefulComponent

This is a hook for the component to perform some actions just before the framework rolls back the prepared state.

Specified by:

preRollback in interface IStatefulComponent<T extends State>

newWindowLifecycleListener

protected WindowLifecycleListener<Tuple> newWindowLifecycleListener()

Overrides:

newWindowLifecycleListener in class WindowedBoltExecutor