Package org.apache.storm.hdfs.blobstore

Class HdfsBlobStoreImpl

java.lang.Object
org.apache.storm.hdfs.blobstore.HdfsBlobStoreImpl
public class HdfsBlobStoreImpl extends Object
HDFS blob store impl.

  • Field Details

    • BLOBSTORE_DIR_PERMISSION

      public static final org.apache.hadoop.fs.permission.FsPermission BLOBSTORE_DIR_PERMISSION

  • Constructor Details

  • Method Details

    • listKeys

      public Iterator<String> listKeys() throws IOException
      List relevant keys.
      Returns:
      all keys that are available for reading
      Throws:
      IOException - on any error

    • listKeys

      protected Iterator<String> listKeys(org.apache.hadoop.fs.Path path) throws IOException
      Throws:
      IOException

    • read

      public BlobStoreFile read(String key) throws IOException
      Get an input stream for reading a part.
      Parameters:
      key - the key of the part to read
      Returns:
      the where to read the data from
      Throws:
      IOException - on any error

    • write

      public BlobStoreFile write(String key, boolean create) throws IOException
      Get an object tied to writing the data.
      Parameters:
      key - the key of the part to write to.
      create - whether the file needs to be new or not.
      Returns:
      an object that can be used to both write to, but also commit/cancel the operation.
      Throws:
      IOException - on any error

    • exists

      public boolean exists(String key)
      Check if the key exists in the blob store.
      Parameters:
      key - the key to check for
      Returns:
      true if it exists else false.

    • deleteKey

      public void deleteKey(String key) throws IOException
      Delete a key from the blob store.
      Parameters:
      key - the key to delete
      Throws:
      IOException - on any error

    • getKeyDir

      protected org.apache.hadoop.fs.Path getKeyDir(String key)

    • fullCleanup

      public void fullCleanup(long age) throws IOException
      Throws:
      IOException

    • listBlobStoreFiles

      protected Iterator<BlobStoreFile> listBlobStoreFiles(org.apache.hadoop.fs.Path path) throws IOException
      Throws:
      IOException

    • getBlobReplication

      protected int getBlobReplication(String key) throws IOException
      Throws:
      IOException

    • updateBlobReplication

      protected int updateBlobReplication(String key, int replication) throws IOException
      Throws:
      IOException

    • delete

      protected void delete(org.apache.hadoop.fs.Path path) throws IOException
      Throws:
      IOException

    • shutdown

      public void shutdown()

    • getLastBlobUpdateTime

      public long getLastBlobUpdateTime() throws IOException
      Get the last update time of any blob.
      Returns:
      the last updated time of blobs within the blobstore.
      Throws:
      IOException - on any error

    • updateLastBlobUpdateTime

      public void updateLastBlobUpdateTime() throws IOException
      Updates the last updated time of existing blobstores to the current time.
      Throws:
      IOException - on any error

    • validateBlobUpdateTime

      public void validateBlobUpdateTime() throws IOException
      Validates that the last updated blob time of the blobstore is up to date with the current existing blobs.
      Throws:
      IOException - on any error