- Title Information
- Title
- C-MR: Continuous Execution of MapReduce Workflows for Stream Processing
- Name:
Personal
- Name Part
- Backman, Nathan John
- Role
- Role Term:
Text
- creator
- Origin Information
- Copyright Date
- 2013
- Physical Description
- Extent
- x, 98 p.
- digitalOrigin
- born digital
- Note
- Thesis (Ph.D. -- Brown University (2013)
- Name:
Personal
- Name Part
- Cetintemel, Ugur
- Role
- Role Term:
Text
- Director
- Name:
Personal
- Name Part
- Fonseca, Rodrigo
- Role
- Role Term:
Text
- Reader
- Name:
Personal
- Name Part
- Zdonik, Stanley
- Role
- Role Term:
Text
- Reader
- Name:
Corporate
- Name Part
- Brown University. Computer Science
- Role
- Role Term:
Text
- sponsor
- Genre (aat)
- theses
- Subject
- Topic
- Stream processing
- Subject
- Topic
- distributed computing
- Subject (FAST)
(authorityURI="http://id.worldcat.org/fast", valueURI="http://id.worldcat.org/fast/1134622")
- Topic
- Streamflow--Data processing
- Subject (FAST)
(authorityURI="http://id.worldcat.org/fast", valueURI="http://id.worldcat.org/fast/906987")
- Topic
- Electronic data processing--Distributed processing
- Record Information
- Record Content Source (marcorg)
- RPB
- Record Creation Date
(encoding="iso8601")
- 20131217
- Language
- Language Term:
Code (ISO639-2B)
- eng
- Language Term:
Text
- English
- Abstract
- Data processing frameworks provide application programmers an interface to manipulate and analyze data. This thesis studies a novel parallel stream processing model, designed for workflow-based data processing frameworks, that leverages application performance requirements to motivate the flexible scheduling and fine-grained allocation of data to computing nodes.<br/>
<br/>
We feature this processing model through the design and implementation of the Continuous-MapReduce (C-MR) data processing framework. C-MR abstracts away the complexities of parallel stream processing and workflow scheduling while providing the simple and familiar MapReduce programming interface with the addition of stream window semantics. Its novel processing model enables: 1) fine-grained, workflow-wide load balancing across computing nodes; 2) the evolving application of data and task parallelism models as guided by application performance requirements; and 3) a novel scheduling framework which supports gradual transitions between scheduling policies relative to application performance and/or resource availability.<br/>
<br/>
This work explores the potential of the C-MR processing model by studying our single-host implementation of C-MR that supports parallel execution on non-dedicated and heterogeneous computing nodes (both multi-core CPUs and GPUs). We then study this processing model through the implementation of a distributed version of C-MR that supports execution on multiple hosts. This endeavor involved the generalizable strategy of employing hierarchical instances of the C-MR processing model while requiring modifications to the data acquisition and load balancing strategies. Experimental results from these studies show that the C-MR processing model can effectively support the continuous execution of workflows of MapReduce jobs for stream processing while being resilient to stream and resource fluctuations due to the processing model's flexibility and diversification of processing responsibilities.<br/>
- Identifier:
DOI
- 10.7301/Z0JD4V3C
- Access Condition:
rights statement
(href="http://rightsstatements.org/vocab/InC/1.0/")
- In Copyright
- Access Condition:
restriction on access
- Collection is open for research.
- Type of Resource (primo)
- dissertations