Skip to main content

Multi-Dimensional Resource Scheduling for Parallel Queries

Scheduling query execution plans is an important component of query optimization in parallel database systems. The problem is particularly complex in a shared-nothing execution environment, where each system node represents a collection of time-shareable resources (e.g., CPU(s), disk(s), etc.) and communicates with other nodes only by message-passing. Significant research effort has concentrated on only a subset of the various forms of intra-query parallelism so that scheduling and synchronization is simplified. In addition, most previous work has focused its attention on one-dimensional models of parallel query scheduling, effectively ignoring the potential benefits of resource sharing. In this paper, we develop an approach that is more general in both directions, capturing all forms of intra-query parallelism and exploiting sharing of multi-dimensional resource nodesamong concurrent plan operators. This allows scheduling a set of independent query tasks (i.e., operator pipelines) to be seen as an instance of the multi-dimensional bin-design problem. Using a novel quantification of coarse grain parallelism, we present a list scheduling heuristic algorithm that is provably near-optimal in the class of coarse gram parallel executions (with a worst-case performance ratio that depends on the number of resources per nocde and the granularity parameter). We then extend this algorithm to handle the operator precedence constrains in a bushy query plan by splitting the execution of the plan into synchronized phases. Preliminary performance results confirm the effectiveness of our scheduling algorithm compared both to previous approaches and the optimal solution. Finally, we present a technique that allows us to relax the coarse granularity restriction and obtain a list scheduling method that is provably near-optimal in the space of all possible parallel schedules.

Citation
Minos N. Garofalakis, Yannis Ioannidis, "Multi-Dimensional Resource Scheduling for Parallel Queries ", Int’l ACM SIGMOD Conference, Montreal, Canada, May 1996, pp. 365-376, 1996
TAGS
Access
Unknown
Published at
Int’l ACM SIGMOD Conference, Montreal, Canada, May 1996, pp. 365-376
Related research area
No related research area
Related Organizations
No related organizations