org.apache.storm.blobstore

Class ClientBlobStore

java.lang.Object

org.apache.storm.blobstore.ClientBlobStore

All Implemented Interfaces:

AutoCloseable, Shutdownable

Direct Known Subclasses:

HdfsClientBlobStore, LocalModeClientBlobStore, NimbusBlobStore

public abstract class ClientBlobStore
extends Object
implements Shutdownable, AutoCloseable

The ClientBlobStore has two concrete implementations 1. NimbusBlobStore 2. HdfsClientBlobStore.

Create, update, read and delete are some of the basic operations defined by this interface. Each operation is validated for permissions against an user. We currently have NIMBUS_ADMINS and SUPERVISOR_ADMINS configuration. NIMBUS_ADMINS are given READ, WRITE and ADMIN access whereas the SUPERVISOR_ADMINS are given READ access in order to read and download the blobs form the nimbus.

The ACLs for the blob store are validated against whether the subject is a NIMBUS_ADMIN, SUPERVISOR_ADMIN or USER who has read, write or admin privileges in order to perform respective operations on the blob.

For more detailed implementation

See Also:

NimbusBlobStore

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	ClientBlobStore.WithBlobstore

Constructor Summary

Constructors

Constructor and Description
ClientBlobStore()

Method Summary

Modifier and Type	Method and Description
abstract void	close()
AtomicOutputStream	createBlob(String key, SettableBlobMeta meta) Client facing API to create a blob.
protected abstract AtomicOutputStream	createBlobToExtend(String key, SettableBlobMeta meta) Client facing API to create a blob.
abstract void	createStateInZookeeper(String key) Creates state inside a zookeeper.
abstract void	deleteBlob(String key) Client facing API to delete a blob.
abstract InputStreamWithMeta	getBlob(String key) Client facing API to read a blob.
abstract ReadableBlobMeta	getBlobMeta(String key) Client facing API to read the metadata information.
abstract int	getBlobReplication(String key) Client facing API to read the replication of a blob.
abstract long	getRemoteBlobstoreUpdateTime() Client facing API to get the last update time of existing blobs in a blobstore.
abstract boolean	isRemoteBlobExists(String blobKey) Decide if the blob is deleted from cluster.
abstract Iterator<String>	listKeys() List keys.
abstract void	prepare(Map<String,Object> conf) Sets up the client API by parsing the configs.
void	setBlobMeta(String key, SettableBlobMeta meta) Client facing API to set the metadata for a blob.
protected abstract void	setBlobMetaToExtend(String key, SettableBlobMeta meta) Client facing API to set the metadata for a blob.
abstract boolean	setClient(Map<String,Object> conf, NimbusClient client) Client facing API to set a nimbus client.
abstract AtomicOutputStream	updateBlob(String key) Client facing API to update a blob.
abstract int	updateBlobReplication(String key, int replication) Client facing API to update the replication of a blob.
static void	withConfiguredClient(ClientBlobStore.WithBlobstore withBlobstore)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.storm.daemon.Shutdownable

shutdown

Constructor Detail

ClientBlobStore

public ClientBlobStore()

Method Detail

withConfiguredClient

public static void withConfiguredClient(ClientBlobStore.WithBlobstore withBlobstore)
                                 throws Exception

Throws:

Exception

prepare

public abstract void prepare(Map<String,Object> conf)

Sets up the client API by parsing the configs.

Parameters:

conf - The storm conf containing the config details

createBlobToExtend

protected abstract AtomicOutputStream createBlobToExtend(String key,
                                                         SettableBlobMeta meta)
                                                  throws AuthorizationException,
                                                         KeyAlreadyExistsException

Client facing API to create a blob.

Parameters:

key - blob key name

meta - contains ACL information

Returns:

AtomicOutputStream returns an output stream into which data can be written

Throws:

AuthorizationException

KeyAlreadyExistsException

updateBlob

public abstract AtomicOutputStream updateBlob(String key)
                                       throws AuthorizationException,
                                              KeyNotFoundException

Client facing API to update a blob.

Parameters:

key - blob key name

Returns:

AtomicOutputStream returns an output stream into which data can be written

Throws:

AuthorizationException

KeyNotFoundException

getBlobMeta

public abstract ReadableBlobMeta getBlobMeta(String key)
                                      throws AuthorizationException,
                                             KeyNotFoundException

Client facing API to read the metadata information.

Parameters:

key - blob key name

Returns:

AtomicOutputStream returns an output stream into which data can be written

Throws:

AuthorizationException

KeyNotFoundException

isRemoteBlobExists

public abstract boolean isRemoteBlobExists(String blobKey)
                                    throws AuthorizationException

Decide if the blob is deleted from cluster.

Parameters:

blobKey - blob key

Throws:

AuthorizationException

setBlobMetaToExtend

protected abstract void setBlobMetaToExtend(String key,
                                            SettableBlobMeta meta)
                                     throws AuthorizationException,
                                            KeyNotFoundException

Client facing API to set the metadata for a blob.

Parameters:

key - blob key name

meta - contains ACL information

Throws:

AuthorizationException

KeyNotFoundException

deleteBlob

public abstract void deleteBlob(String key)
                         throws AuthorizationException,
                                KeyNotFoundException

Client facing API to delete a blob.

Parameters:

key - blob key name

Throws:

AuthorizationException

KeyNotFoundException

getBlob

public abstract InputStreamWithMeta getBlob(String key)
                                     throws AuthorizationException,
                                            KeyNotFoundException

Client facing API to read a blob.

Parameters:

key - blob key name

Returns:

an InputStream to read the metadata for a blob

Throws:

AuthorizationException

KeyNotFoundException

listKeys

public abstract Iterator<String> listKeys()

List keys.

Returns:

Iterator for a list of keys currently present in the blob store.

getBlobReplication

public abstract int getBlobReplication(String key)
                                throws AuthorizationException,
                                       KeyNotFoundException

Client facing API to read the replication of a blob.

Parameters:

key - blob key name

Returns:

int indicates the replication factor of a blob

Throws:

AuthorizationException

KeyNotFoundException

updateBlobReplication

public abstract int updateBlobReplication(String key,
                                          int replication)
                                   throws AuthorizationException,
                                          KeyNotFoundException

Client facing API to update the replication of a blob.

Parameters:

key - blob key name

replication - int indicates the replication factor a blob has to be set

Returns:

int indicates the replication factor of a blob

Throws:

AuthorizationException

KeyNotFoundException

setClient

public abstract boolean setClient(Map<String,Object> conf,
                                  NimbusClient client)

Client facing API to set a nimbus client.

Parameters:

conf - storm conf

client - NimbusClient

Returns:

indicates where the client connection has been setup.

createStateInZookeeper

public abstract void createStateInZookeeper(String key)

Creates state inside a zookeeper. Required for blobstore to write to zookeeper when Nimbus HA is turned on in order to maintain state consistency.

close

public abstract void close()

Specified by:

close in interface AutoCloseable

createBlob

public final AtomicOutputStream createBlob(String key,
                                           SettableBlobMeta meta)
                                    throws AuthorizationException,
                                           KeyAlreadyExistsException

Client facing API to create a blob.

Parameters:

key - blob key name

meta - contains ACL information

Returns:

AtomicOutputStream returns an output stream into which data can be written

Throws:

AuthorizationException

KeyAlreadyExistsException

setBlobMeta

public final void setBlobMeta(String key,
                              SettableBlobMeta meta)
                       throws AuthorizationException,
                              KeyNotFoundException

Client facing API to set the metadata for a blob.

Parameters:

key - blob key name

meta - contains ACL information

Throws:

AuthorizationException

KeyNotFoundException

getRemoteBlobstoreUpdateTime

public abstract long getRemoteBlobstoreUpdateTime()
                                           throws IOException

Client facing API to get the last update time of existing blobs in a blobstore. This is only required for use on supervisors.

Returns:

the timestamp of when the blobstore was last updated. -1L if the blobstore does not support this.

Throws:

IOException