Online Safety Community

In order to understand the goals of MapReduce, it is important to realize for which scenarios MapReduce is optimized. The MapReduce programming model is created for processing data which requires “DATA PARALLELISM”, the ability to compute multiple independent operations in any order (King). In parallel processing, commutative operations are operations where the order of execution does not matter to the results of the equation. Commutativity can apply to complex operations and even processes, as long as they don’t manipulate the same memory. For example, in the figure below, as long as foo(a) and bar(b) don’t manipulate the same variable, they can occur in parallel in different threads. However, the write operation must wait for both foo() and bar() to complete. The figure below illustrates a dependency graph between foo(a), bar(a) and the write command.

Figure 1 – Parallelism Dependency Graph

One of the goals of parallelism is identifying the logical “tasks” or units which can be run in parallel as threads. Parallel programming techniques require developers to implement dependency graphs, which can become much more as the amount of shared information and sequence of operations increases. Techniques such as locks and barriers, critical sections, semaphores, monitors, RPC and rendezvous have been proposed to aid in the design of multi threaded and distributed. In Parallel and Distributed processing, intelligent task design attempts to eliminate as many synchronization points as possible, but some will still be required. Patterns such as “Master/Worker” and “Producer/Consumer” are different patterns that developers can use to implement parallel thread processing.

MapReduce provides a programming model which abstracts many of the aforementioned complexities of parallel processing from the software engineer. The MapReduce implementation performs much of the “wiring” associated with parallel processing, leaving the developer to implement relatively simple methods. The use of MapReduce does come with some constraints, making it less appropriate for some tasks. MapReduce models are optimized for tasks where a large number of key*value input lists must be processed somewhat independently. MapReduce map() method must be commutative, in order for the MapReduce implementation to make use of parallelization. MapReduce enables the parallelization across hundreds and even thousands of CPU’s.

Views: 45

Reply to This

Take our poll!

Take our poll!

Latest Activity

HrishikeshRam posted blog posts
2 hours ago
Russel Stuart posted a blog post

The Core Challenges of Leadership

It is a given that leadership is indispensable for organizations. Why? Because it is leadership that takes the organization to its intended, charted out destination. Since it is leadership that is the organization's anchor; it is but natural that leadership faces challenges. The challenge associated with leadership is so inherent within it that it is right to say that if the challenge of leadership did not exist; there would be no need for leadership at all.The major challenges of…See More
2 hours ago
Mark Nilson posted an event

Online Webinar on Manage a Multi-Generation in Today’s Workplace – Training Doyens at Training Doyens 26468 E Walker Dr,Aurora, Colorado

October 23, 2018 from 1pm to 2pm
OVERVIEW A multi-generational workforce requires many moving parts and also leads to generational differences in the workplace. Join our upcoming webinar to learn how to manage generational differences in the workplace and make the organization productive and profitable.Key points to be covered in the webinar:Overview of the different generationsWork expectations of different generationsThe challenges faced due to generational…See More
3 hours ago
Jen McDade joined Fahd Siddiqui's group
Thumbnail

LMXOnline.com - Safety and Health online training

LMXonline.com features online safety & health training courses (OSHA standardized) including spanish courses. It comes with a web-based learning management system for corporations to manage assignments to employees,track progress & view certificatesSee More
4 hours ago

Forum

Road Safety

What can be the best measures for Road Safety?Continue

Started by Jen McDade Sep 27.

safety equipments in breweries 4 Replies

safety equipments in breweries -- kindly provide detailsContinue

Started by ajit. Last reply by Jen McDade Sep 24.

Unusual Safety Tips 10 Replies

Please share some unusual but genuine safety precautions.My unusual safety tips - Don't drink sweet fruity juice while driving in the night as it contains sugar and sugar slows down your blood…Continue

Started by Arnold Brame. Last reply by Jen McDade Sep 24.

Safety products 9 Replies

While working or at some work stations, employees are exposed to threats or at times suffer severe injuries. The safety of the employee should be the foremost priority. Safety equipment aids help in…Continue

Tags: equipments, and, products

Started by Enna Henry. Last reply by Jen McDade Sep 24.

Talend Application Integration

APPLICATION INTEGRATION:Talend ESB and Application Integration delivers agile real-time integration of applications and data.Heterogeneous Application and Data IntegrationTalend’s open source ESB…Continue

Tags: york, new, certification, training, talend

Started by nicolewells Sep 21.

Badge

Loading…

© 2018   Created by Safety Community.   Powered by

Badges  |  Report an Issue  |  Terms of Service