Constructor and Description |
---|
TopologyResources(TopologyDetails td,
Assignment assignment) |
TopologyResources(TopologyDetails td,
SchedulerAssignment assignment) |
Modifier and Type | Method and Description |
---|---|
TopologyDetails |
Topologies.getById(String topologyId)
Get a topology given an ID
|
TopologyDetails |
Topologies.getByName(String topologyName)
Get a topology given a topology name.
|
Modifier and Type | Method and Description |
---|---|
Collection<TopologyDetails> |
Topologies.getTopologies() |
Collection<TopologyDetails> |
Topologies.getTopologiesOwnedBy(String user)
Get all topologies submitted/owned by a given user.
|
Iterator<TopologyDetails> |
Topologies.iterator() |
List<TopologyDetails> |
Cluster.needsSchedulingTopologies() |
List<TopologyDetails> |
ISchedulingState.needsSchedulingTopologies()
Get all of the topologies that need scheduling.
|
Modifier and Type | Method and Description |
---|---|
int |
Cluster.getAssignedNumWorkers(TopologyDetails topology) |
int |
ISchedulingState.getAssignedNumWorkers(TopologyDetails topology)
Get the number of workers assigned to a topology.
|
Map<String,List<ExecutorDetails>> |
Cluster.getNeedsSchedulingComponentToExecutors(TopologyDetails topology) |
Map<String,List<ExecutorDetails>> |
ISchedulingState.getNeedsSchedulingComponentToExecutors(TopologyDetails topology)
Get the component name to executor list for executors that need to be scheduled.
|
Map<ExecutorDetails,String> |
Cluster.getNeedsSchedulingExecutorToComponents(TopologyDetails topology) |
Map<ExecutorDetails,String> |
ISchedulingState.getNeedsSchedulingExecutorToComponents(TopologyDetails topology)
Get the executor to component name map for executors that need to be scheduled.
|
Collection<ExecutorDetails> |
Cluster.getUnassignedExecutors(TopologyDetails topology) |
Collection<ExecutorDetails> |
ISchedulingState.getUnassignedExecutors(TopologyDetails topology)
get the unassigned executors of the topology.
|
boolean |
Cluster.needsScheduling(TopologyDetails topology) |
boolean |
ISchedulingState.needsScheduling(TopologyDetails topology)
Does the topology need scheduling.
|
boolean |
Cluster.needsSchedulingRas(TopologyDetails topology) |
boolean |
ISchedulingState.needsSchedulingRas(TopologyDetails topology)
Like
ISchedulingState.needsScheduling(TopologyDetails) but does not take into account the number of workers requested. |
void |
Cluster.setStatus(TopologyDetails td,
String statusMessage)
set scheduler status for a topology.
|
boolean |
Cluster.wouldFit(WorkerSlot ws,
ExecutorDetails exec,
TopologyDetails td,
NormalizedResourceOffer resourcesAvailable,
double maxHeap) |
boolean |
ISchedulingState.wouldFit(WorkerSlot ws,
ExecutorDetails exec,
TopologyDetails td,
NormalizedResourceOffer resourcesAvailable,
double maxHeap)
Would scheduling exec on ws fit? With a heap <= maxHeap total memory added <= memoryAvailable and cpu added <= cpuAvailable.
|
Constructor and Description |
---|
Topologies(TopologyDetails... details)
Create a new Topologies from a list of TopologyDetails.
|
Constructor and Description |
---|
Topologies(Map<String,TopologyDetails> topologies)
Create a new Topologies from a map of id to topology
|
Modifier and Type | Method and Description |
---|---|
void |
DefaultPool.addTopology(TopologyDetails td) |
void |
FreePool.addTopology(TopologyDetails td) |
void |
IsolatedPool.addTopology(TopologyDetails td) |
abstract void |
NodePool.addTopology(TopologyDetails td)
Add a topology to the pool
|
boolean |
DefaultPool.canAdd(TopologyDetails td) |
boolean |
FreePool.canAdd(TopologyDetails td) |
boolean |
IsolatedPool.canAdd(TopologyDetails td) |
abstract boolean |
NodePool.canAdd(TopologyDetails td)
Check if this topology can be added to this pool
|
Constructor and Description |
---|
RoundRobinSlotScheduler(TopologyDetails td,
int slotsToUse,
Cluster cluster)
Create a new scheduler for a given topology
|
Modifier and Type | Method and Description |
---|---|
TopologyDetails |
User.getNextTopologyToSchedule(ISchedulingState cluster) |
TopologyDetails |
User.getRunningTopologyWithLowestPriority(ISchedulingState cluster) |
Modifier and Type | Method and Description |
---|---|
TreeSet<TopologyDetails> |
User.getPendingTopologies(ISchedulingState cluster) |
TreeSet<TopologyDetails> |
User.getRunningTopologies(ISchedulingState cluster) |
Modifier and Type | Method and Description |
---|---|
void |
RAS_Node.assign(WorkerSlot target,
TopologyDetails td,
Collection<ExecutorDetails> executors)
Assigns a worker to a node.
|
void |
RAS_Node.assignSingleExecutor(WorkerSlot ws,
ExecutorDetails exec,
TopologyDetails td)
Assign a single executor to a slot, even if other things are in the slot.
|
boolean |
RAS_Node.couldEverFit(ExecutorDetails exec,
TopologyDetails td)
Is there any possibility that exec could ever fit on this node.
|
void |
RAS_Node.freeSingleExecutor(ExecutorDetails exec,
TopologyDetails topo)
frees a single executor.
|
void |
User.markTopoUnsuccess(TopologyDetails topo) |
void |
User.markTopoUnsuccess(TopologyDetails topo,
Cluster cluster) |
boolean |
RAS_Node.wouldFit(WorkerSlot ws,
ExecutorDetails exec,
TopologyDetails td)
Would scheduling exec in ws fit with the current resource constraints.
|
Modifier and Type | Field and Description |
---|---|
protected LinkedList<TopologyDetails> |
DefaultSchedulingPriorityStrategy.SimulatedUser.tds |
Modifier and Type | Method and Description |
---|---|
TopologyDetails |
DefaultSchedulingPriorityStrategy.SimulatedUser.getNextHighest() |
TopologyDetails |
DefaultSchedulingPriorityStrategy.SimulatedUser.simScheduleNextHighest() |
Modifier and Type | Method and Description |
---|---|
List<TopologyDetails> |
ISchedulingPriorityStrategy.getOrderedTopologies(ISchedulingState schedulingState,
Map<String,User> userMap)
Prioritize the list of all topologies in the cluster.
|
List<TopologyDetails> |
DefaultSchedulingPriorityStrategy.getOrderedTopologies(ISchedulingState cluster,
Map<String,User> userMap) |
Modifier and Type | Method and Description |
---|---|
protected double |
DefaultSchedulingPriorityStrategy.SimulatedUser.getScore(double availableCpu,
double availableMemory,
TopologyDetails td)
Get a score for the simulated user.
|
Modifier and Type | Method and Description |
---|---|
protected WorkerSlot |
BaseResourceAwareStrategy.findWorkerForExec(ExecutorDetails exec,
TopologyDetails td,
Iterable<String> sortedNodes)
Find a worker to schedule executor exec on.
|
protected List<Component> |
BaseResourceAwareStrategy.getSpouts(TopologyDetails td)
Get a list of all the spouts in the topology.
|
protected SchedulingResult |
BaseResourceAwareStrategy.mkNotEnoughResources(TopologyDetails td) |
protected List<ExecutorDetails> |
BaseResourceAwareStrategy.orderExecutors(TopologyDetails td,
Collection<ExecutorDetails> unassignedExecutors)
Order executors based on how many in and out connections it will potentially need to make, in descending order.
|
SchedulingResult |
DefaultResourceAwareStrategy.schedule(Cluster cluster,
TopologyDetails td) |
SchedulingResult |
ConstraintSolverStrategy.schedule(Cluster cluster,
TopologyDetails td) |
SchedulingResult |
IStrategy.schedule(Cluster schedulingState,
TopologyDetails td)
This method is invoked to calculate a scheduling for topology td.
|
SchedulingResult |
GenericResourceAwareStrategy.schedule(Cluster cluster,
TopologyDetails td) |
protected boolean |
BaseResourceAwareStrategy.scheduleExecutor(ExecutorDetails exec,
TopologyDetails td,
Collection<ExecutorDetails> scheduledTasks,
Iterable<String> sortedNodes)
Schedule executor exec from topology td.
|
protected Iterable<String> |
BaseResourceAwareStrategy.sortAllNodes(TopologyDetails td,
ExecutorDetails exec,
List<String> favoredNodeIds,
List<String> unFavoredNodeIds) |
protected TreeSet<BaseResourceAwareStrategy.ObjectResources> |
BaseResourceAwareStrategy.sortNodes(List<RAS_Node> availNodes,
ExecutorDetails exec,
TopologyDetails topologyDetails,
String rackId,
Map<String,AtomicInteger> scheduledCount)
Nodes are sorted by two criteria.
|
protected TreeSet<BaseResourceAwareStrategy.ObjectResources> |
DefaultResourceAwareStrategy.sortObjectResources(BaseResourceAwareStrategy.AllResources allResources,
ExecutorDetails exec,
TopologyDetails topologyDetails,
BaseResourceAwareStrategy.ExistingScheduleFunc existingScheduleFunc)
Sort objects by the following two criteria.
|
protected TreeSet<BaseResourceAwareStrategy.ObjectResources> |
ConstraintSolverStrategy.sortObjectResources(BaseResourceAwareStrategy.AllResources allResources,
ExecutorDetails exec,
TopologyDetails topologyDetails,
BaseResourceAwareStrategy.ExistingScheduleFunc existingScheduleFunc) |
protected TreeSet<BaseResourceAwareStrategy.ObjectResources> |
GenericResourceAwareStrategy.sortObjectResources(BaseResourceAwareStrategy.AllResources allResources,
ExecutorDetails exec,
TopologyDetails topologyDetails,
BaseResourceAwareStrategy.ExistingScheduleFunc existingScheduleFunc)
Sort objects by the following two criteria.
|
protected abstract TreeSet<BaseResourceAwareStrategy.ObjectResources> |
BaseResourceAwareStrategy.sortObjectResources(BaseResourceAwareStrategy.AllResources allResources,
ExecutorDetails exec,
TopologyDetails topologyDetails,
BaseResourceAwareStrategy.ExistingScheduleFunc existingScheduleFunc) |
static TreeSet<BaseResourceAwareStrategy.ObjectResources> |
GenericResourceAwareStrategy.sortObjectResourcesImpl(BaseResourceAwareStrategy.AllResources allResources,
ExecutorDetails exec,
TopologyDetails topologyDetails,
BaseResourceAwareStrategy.ExistingScheduleFunc existingScheduleFunc)
Implementation of the sortObjectResources method so other strategies can reuse it.
|
static boolean |
ConstraintSolverStrategy.validateSolution(Cluster cluster,
TopologyDetails td)
Determines if a scheduling is valid and all constraints are satisfied.
|
Copyright © 2022 The Apache Software Foundation. All rights reserved.