Package org.apache.storm.hdfs.bolt
Class AvroGenericRecordBolt
java.lang.Object
org.apache.storm.topology.base.BaseComponent
org.apache.storm.topology.base.BaseRichBolt
org.apache.storm.hdfs.bolt.AbstractHdfsBolt
org.apache.storm.hdfs.bolt.AvroGenericRecordBolt
- All Implemented Interfaces:
Serializable
,IBolt
,IComponent
,IRichBolt
- See Also:
-
Field Summary
Fields inherited from class org.apache.storm.hdfs.bolt.AbstractHdfsBolt
collector, configKey, fileNameFormat, fileRetryCount, fs, fsUrl, hdfsConfig, maxOpenFiles, offset, partitioner, rotationActions, rotationCounterMap, rotationPolicy, rotationTimer, syncPolicy, tickTupleInterval, writeLock, writers
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionaddRotationAction
(RotationAction action) protected void
doPrepare
(Map<String, Object> conf, TopologyContext topologyContext, OutputCollector collector) protected String
getWriterKey
(Tuple tuple) AvroGenericRecordBolt must override this method because messages with different schemas cannot be written to the same file.protected AbstractHDFSWriter
makeNewWriter
(org.apache.hadoop.fs.Path path, Tuple tuple) withConfigKey
(String configKey) withFileNameFormat
(FileNameFormat fileNameFormat) withMaxOpenFiles
(int maxOpenFiles) withPartitioner
(Partitioner partitioner) withRotationPolicy
(FileRotationPolicy rotationPolicy) withSyncPolicy
(SyncPolicy syncPolicy) withTickTupleIntervalSeconds
(int interval) Methods inherited from class org.apache.storm.hdfs.bolt.AbstractHdfsBolt
cleanup, declareOutputFields, execute, getBasePathForNextFile, getComponentConfiguration, prepare, rotateOutputFile
-
Constructor Details
-
AvroGenericRecordBolt
public AvroGenericRecordBolt()
-
-
Method Details
-
withFsUrl
-
withConfigKey
-
withFileNameFormat
-
withSyncPolicy
-
withRotationPolicy
-
addRotationAction
-
withTickTupleIntervalSeconds
-
withMaxOpenFiles
-
withPartitioner
-
doPrepare
protected void doPrepare(Map<String, Object> conf, TopologyContext topologyContext, OutputCollector collector) throws IOException- Specified by:
doPrepare
in classAbstractHdfsBolt
- Throws:
IOException
-
getWriterKey
AvroGenericRecordBolt must override this method because messages with different schemas cannot be written to the same file. By treating the complete schema as the "key" AbstractHdfsBolt will associate a different writer for every distinct schema.- Specified by:
getWriterKey
in classAbstractHdfsBolt
-
makeNewWriter
protected AbstractHDFSWriter makeNewWriter(org.apache.hadoop.fs.Path path, Tuple tuple) throws IOException - Specified by:
makeNewWriter
in classAbstractHdfsBolt
- Throws:
IOException
-