Package org.apache.storm.kafka.spout
Class ByTopicRecordTranslator<K,V>
- java.lang.Object
-
- org.apache.storm.kafka.spout.ByTopicRecordTranslator<K,V>
-
- Type Parameters:
K
- the key of the incoming RecordsV
- the value of the incoming Records
- All Implemented Interfaces:
Serializable
,Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>>
,RecordTranslator<K,V>
public class ByTopicRecordTranslator<K,V> extends Object implements RecordTranslator<K,V>
Based off of a given Kafka topic a ConsumerRecord came from it will be translated to a Storm tuple and emitted to a given stream.- See Also:
- Serialized Form
-
-
Field Summary
-
Fields inherited from interface org.apache.storm.kafka.spout.RecordTranslator
DEFAULT_STREAM
-
-
Constructor Summary
Constructors Constructor Description ByTopicRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields)
Create a simple record translator that will use func to extract the fields of the tuple, named by fields, and emit them to the default stream.ByTopicRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields, String stream)
Create a simple record translator that will use func to extract the fields of the tuple, named by fields, and emit them to stream.ByTopicRecordTranslator(RecordTranslator<K,V> defaultTranslator)
Create a record translator with the given default translator.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description List<Object>
apply(org.apache.kafka.clients.consumer.ConsumerRecord<K,V> record)
Translate the ConsumerRecord into a list of objects that can be emitted.ByTopicRecordTranslator<K,V>
forTopic(String topic, Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields)
Configure a translator for a given topic with tuples to be emitted to the default stream.ByTopicRecordTranslator<K,V>
forTopic(String topic, Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields, String stream)
Configure a translator for a given topic.ByTopicRecordTranslator<K,V>
forTopic(String topic, RecordTranslator<K,V> translator)
Configure a translator for a given kafka topic.Fields
getFieldsFor(String stream)
Get the fields associated with a stream.List<String>
streams()
Get the list of streams this translator will handle.
-
-
-
Constructor Detail
-
ByTopicRecordTranslator
public ByTopicRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields, String stream)
Create a simple record translator that will use func to extract the fields of the tuple, named by fields, and emit them to stream. This will handle all topics not explicitly set elsewhere.- Parameters:
func
- extracts and turns them into a list of objects to be emittedfields
- the names of the fields extractedstream
- the stream to emit these fields on.
-
ByTopicRecordTranslator
public ByTopicRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields)
Create a simple record translator that will use func to extract the fields of the tuple, named by fields, and emit them to the default stream. This will handle all topics not explicitly set elsewhere.- Parameters:
func
- extracts and turns them into a list of objects to be emittedfields
- the names of the fields extracted
-
ByTopicRecordTranslator
public ByTopicRecordTranslator(RecordTranslator<K,V> defaultTranslator)
Create a record translator with the given default translator.- Parameters:
defaultTranslator
- a translator that will be used for all topics not explicitly set with one of the variants offorTopic(java.lang.String, org.apache.storm.kafka.spout.RecordTranslator)
.
-
-
Method Detail
-
forTopic
public ByTopicRecordTranslator<K,V> forTopic(String topic, Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields)
Configure a translator for a given topic with tuples to be emitted to the default stream.- Parameters:
topic
- the topic this should be used forfunc
- extracts and turns them into a list of objects to be emittedfields
- the names of the fields extracted- Returns:
- this to be able to chain configuration
- Throws:
IllegalStateException
- if the topic is already registered to another translatorIllegalArgumentException
- if the Fields for the stream this emits to do not match any already configured Fields for the same stream
-
forTopic
public ByTopicRecordTranslator<K,V> forTopic(String topic, Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields, String stream)
Configure a translator for a given topic.- Parameters:
topic
- the topic this should be used forfunc
- extracts and turns them into a list of objects to be emittedfields
- the names of the fields extractedstream
- the stream to emit the tuples to.- Returns:
- this to be able to chain configuration
- Throws:
IllegalStateException
- if the topic is already registered to another translatorIllegalArgumentException
- if the Fields for the stream this emits to do not match any already configured Fields for the same stream
-
forTopic
public ByTopicRecordTranslator<K,V> forTopic(String topic, RecordTranslator<K,V> translator)
Configure a translator for a given kafka topic.- Parameters:
topic
- the topic this translator should handletranslator
- the translator itself- Returns:
- this to be able to chain configuration
- Throws:
IllegalStateException
- if the topic is already registered to another translatorIllegalArgumentException
- if the Fields for the stream this emits to do not match any already configured Fields for the same stream
-
apply
public List<Object> apply(org.apache.kafka.clients.consumer.ConsumerRecord<K,V> record)
Description copied from interface:RecordTranslator
Translate the ConsumerRecord into a list of objects that can be emitted.- Specified by:
apply
in interfaceFunc<K,V>
- Specified by:
apply
in interfaceRecordTranslator<K,V>
- Parameters:
record
- the record to translate- Returns:
- the objects in the tuple. Return a
KafkaTuple
if you want to route the tuple to a non-default stream. Returnnull
to discard an invalidConsumerRecord
ifKafkaSpoutConfig.Builder.setEmitNullTuples(boolean)
is set tofalse
.
-
getFieldsFor
public Fields getFieldsFor(String stream)
Description copied from interface:RecordTranslator
Get the fields associated with a stream. The streams passed in are returned by theRecordTranslator.streams()
method.- Specified by:
getFieldsFor
in interfaceRecordTranslator<K,V>
- Parameters:
stream
- the stream the fields are for- Returns:
- the fields for that stream.
-
streams
public List<String> streams()
Description copied from interface:RecordTranslator
Get the list of streams this translator will handle.- Specified by:
streams
in interfaceRecordTranslator<K,V>
- Returns:
- the list of streams that this will handle.
-
-