Dryad: distributed data-parallel programs from sequential building blocks

From AcaWiki
Jump to: navigation, search

Citation: Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrell, Dennis Fetterly (2007/03/21) Dryad: distributed data-parallel programs from sequential building blocks. EuroSys (RSS)
DOI (original publisher): 10.1145/1272996.1273005
Semantic Scholar (metadata): 10.1145/1272996.1273005
Sci-Hub (fulltext): 10.1145/1272996.1273005
Internet Archive Scholar (search for fulltext): Dryad: distributed data-parallel programs from sequential building blocks
Download: https://doi.org/10.1145/1272996.1273005
Tagged: Computer Science (RSS) parallel systems (RSS)

Summary

Placeholder


Introduction

  • Single-core frequency slowing down, big clusters exist, need to write parallel programs.
  • Automatic parallelization is too hard.
  • GPU shaders, MapReduce, and parallel databases fix the communication topology, and allow the programmer to supply the nodes.
  • Contributions: built a high-performance distributed processing engine (Dryad), evaluation of that engine, and programming interface (low- and high-levels).

Dryad System

  • Channels can be implemented with TCP, files, or shared memory, and the choice is transparent to user-code.
  • SQL can be mapped to Dryad.

Dryad Graphs

  • Screenshot from 2021-07-22 21-45-38.png

High-level programming

Evaluation