Package org.apache.storm.scheduler.resource.strategies.scheduling.sorter

Class NodeSorter

java.lang.Object

org.apache.storm.scheduler.resource.strategies.scheduling.sorter.NodeSorter

All Implemented Interfaces:

INodeSorter

public class NodeSorter
extends Object
implements INodeSorter

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	NodeSorter.ExistingScheduleFunc	interface for calculating the number of existing executors scheduled on a object (rack or node).

Field Summary

Fields

Modifier and Type	Field	Description
protected Cluster	cluster
protected List<String>	favoredNodeIds
protected List<String>	greyListedSupervisorIds
protected final BaseResourceAwareStrategy.NodeSortType	nodeSortType
protected TopologyDetails	topologyDetails
protected List<String>	unFavoredNodeIds

Constructor Summary

Constructors

Constructor	Description
NodeSorter(Cluster cluster, TopologyDetails topologyDetails, BaseResourceAwareStrategy.NodeSortType nodeSortType)	Initialize for the default implementation node sorting.

Method Summary

Modifier and Type	Method	Description
List<ObjectResourcesItem>	getSortedRacks()	Racks are sorted by two criteria.
List<RasNode>	hostnameToNodes(String hostname)	hostname to Ids.
protected List<String>	makeHostToNodeIds(List<String> hosts)
void	prepare(ExecutorDetails exec)	Prepare for node sorting.
Iterable<String>	sortAllNodes()
protected List<ObjectResourcesItem>	sortObjectResources(ObjectResourcesSummary resourcesSummary, ExecutorDetails exec, NodeSorter.ExistingScheduleFunc existingScheduleFunc)	Scheduling uses sortAllNodes() which eventually calls this method whose behavior can be altered by setting nodeSortType.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

nodeSortType

protected final BaseResourceAwareStrategy.NodeSortType nodeSortType

cluster

protected Cluster cluster

topologyDetails

protected TopologyDetails topologyDetails

greyListedSupervisorIds

protected List<String> greyListedSupervisorIds

favoredNodeIds

protected List<String> favoredNodeIds

unFavoredNodeIds

protected List<String> unFavoredNodeIds

Constructor Details

NodeSorter

public NodeSorter(Cluster cluster,
 TopologyDetails topologyDetails,
 BaseResourceAwareStrategy.NodeSortType nodeSortType)

Initialize for the default implementation node sorting.

Parameters:

cluster - for which nodes will be sorted.

topologyDetails - the topology to sort for.

nodeSortType - type of sorting to be applied to object resource collection BaseResourceAwareStrategy.NodeSortType.

Method Details

prepare

public void prepare(ExecutorDetails exec)

Description copied from interface: INodeSorter

Prepare for node sorting. This method must be called before INodeSorter.getSortedRacks() and INodeSorter.sortAllNodes().

Specified by:

prepare in interface INodeSorter

Parameters:

exec - optional, may be null.

sortObjectResources

protected List<ObjectResourcesItem> sortObjectResources(ObjectResourcesSummary resourcesSummary,
 ExecutorDetails exec,
 NodeSorter.ExistingScheduleFunc existingScheduleFunc)

Scheduling uses sortAllNodes() which eventually calls this method whose behavior can be altered by setting nodeSortType.

Parameters:

resourcesSummary - contains all individual ObjectResourcesItem as well as cumulative stats

exec - executor for which the sorting is done

existingScheduleFunc - a function to get existing executors already scheduled on this object

Returns:

a sorted list of ObjectResourcesItem

makeHostToNodeIds

protected List<String> makeHostToNodeIds(List<String> hosts)

sortAllNodes

public Iterable<String> sortAllNodes()

Specified by:

sortAllNodes in interface INodeSorter

getSortedRacks

public List<ObjectResourcesItem> getSortedRacks()

Racks are sorted by two criteria.

1) the number executors of the topology that needs to be scheduled is already on the rack in descending order. The reasoning to sort based on criterion 1 is so we schedule the rest of a topology on the same rack as the existing executors of the topology.

2) the subordinate/subservient resource availability percentage of a rack in descending order We calculate the resource availability percentage by dividing the resource availability on the rack by the resource availability of the entire cluster By doing this calculation, racks that have exhausted or little of one of the resources mentioned above will be ranked after racks that have more balanced resource availability. So we will be less likely to pick a rack that have a lot of one resource but a low amount of another.

Specified by:

getSortedRacks in interface INodeSorter

Returns:

a sorted list of racks

hostnameToNodes

public List<RasNode> hostnameToNodes(String hostname)

hostname to Ids.

Parameters:

hostname - the hostname.

Returns:

the ids n that node.