Package org.apache.storm.container

Interface ResourceIsolationInterface

All Known Implementing Classes:
CgroupManager, DefaultResourceIsolationManager, DockerManager, OciContainerManager, RuncLibContainerManager
public interface ResourceIsolationInterface
A plugin to support resource isolation and limitation within Storm.

  • Method Details

    • prepare

      void prepare(Map<String,Object> conf) throws IOException
      Called when starting up.
      Parameters:
      conf - the cluster config
      Throws:
      IOException - on any error.

    • reserveResourcesForWorker

      void reserveResourcesForWorker(String workerId, Integer workerMemory, Integer workerCpu, String numaId)
      This function should be used prior to starting the worker to reserve resources for the worker.
      Parameters:
      workerId - worker id of the worker to start
      workerMemory - the amount of memory for the worker or null if not enforced
      workerCpu - the amount of cpu for the worker or null if not enforced
      numaId - NUMA zone if applicable the worker should be bound to

    • cleanup

      void cleanup(String user, String workerId, int port) throws IOException
      This function will be called when the worker needs to shutdown. This function should include logic to clean up after a worker is shutdown.
      Parameters:
      user - the user of the worker
      workerId - worker id to shutdown and clean up after
      port - the port of the worker
      Throws:
      IOException

    • launchWorkerProcess

      void launchWorkerProcess(String user, String topologyId, Map<String,Object> topoConf, int port, String workerId, List<String> command, Map<String,String> env, String logPrefix, ExitCodeCallback processExitCallback, File targetDir) throws IOException
      After reserving resources for the worker (i.e. calling reserveResourcesForWorker), this function can be used to launch worker process.
      Parameters:
      user - the user who runs the command as
      topologyId - the Id of the topology
      topoConf - the topology configuration
      port - the port where the worker is on
      workerId - the Id of the worker
      command - the command to run
      env - the environment to run the command
      logPrefix - the prefix to include in the logs
      processExitCallback - a callback for when the process exits
      targetDir - the working directory to run the command in
      Throws:
      IOException - on I/O exception

    • getMemoryUsage

      long getMemoryUsage(String user, String workerId, int port) throws IOException
      Get the current memory usage of the a given worker.
      Parameters:
      user - the user that the worker is running as
      workerId - the id of the worker
      port - the port of the worker
      Returns:
      the amount of memory the worker is using in bytes or -1 if not supported
      Throws:
      IOException - on I/O exception

    • getSystemFreeMemoryMb

      long getSystemFreeMemoryMb() throws IOException
      Get the amount of free memory in MB. This might not be the entire box, it might be within a parent resource group.
      Returns:
      The amount of memory in MB that are free on the system.
      Throws:
      IOException - on I/O exception

    • kill

      void kill(String user, String workerId) throws IOException
      Kill the given worker.
      Parameters:
      user - the user that the worker is running as
      workerId - the id of the worker to kill
      Throws:
      IOException - on I/O exception

    • forceKill

      void forceKill(String user, String workerId) throws IOException
      Kill the given worker forcefully.
      Parameters:
      user - the user that the worker is running as
      workerId - the id of the worker to kill
      Throws:
      IOException - on I/O exception

    • areAllProcessesDead

      boolean areAllProcessesDead(String user, String workerId) throws IOException
      Check if all the processes are dead.
      Parameters:
      user - the user that the processes are running as
      workerId - the id of the worker to kill
      Returns:
      true if all the processed are dead; false otherwise
      Throws:
      IOException - on I/O exception

    • runProfilingCommand

      boolean runProfilingCommand(String user, String workerId, List<String> command, Map<String,String> env, String logPrefix, File targetDir) throws IOException, InterruptedException
      Run profiling command.
      Parameters:
      user - the user that the worker is running as
      workerId - the id of the worker
      command - the command to run
      env - the environment to run the command
      logPrefix - the prefix to include in the logs
      targetDir - the working directory to run the command in
      Returns:
      true if succeeds; false otherwise
      Throws:
      IOException - on I/O exception
      InterruptedException - if interrupted

    • isResourceManaged

      boolean isResourceManaged()
      Return true if resources are being managed. The DefaultResourceIsolationManager will have it return false since it doesn't really manage resources.
      Returns:
      true if resources are being managed.