Online Safety Community

In order to understand the goals of MapReduce, it is important to realize for which scenarios MapReduce is optimized. The MapReduce programming model is created for processing data which requires “DATA PARALLELISM”, the ability to compute multiple independent operations in any order (King). In parallel processing, commutative operations are operations where the order of execution does not matter to the results of the equation. Commutativity can apply to complex operations and even processes, as long as they don’t manipulate the same memory. For example, in the figure below, as long as foo(a) and bar(b) don’t manipulate the same variable, they can occur in parallel in different threads. However, the write operation must wait for both foo() and bar() to complete. The figure below illustrates a dependency graph between foo(a), bar(a) and the write command.

Figure 1 – Parallelism Dependency Graph

One of the goals of parallelism is identifying the logical “tasks” or units which can be run in parallel as threads. Parallel programming techniques require developers to implement dependency graphs, which can become much more as the amount of shared information and sequence of operations increases. Techniques such as locks and barriers, critical sections, semaphores, monitors, RPC and rendezvous have been proposed to aid in the design of multi threaded and distributed. In Parallel and Distributed processing, intelligent task design attempts to eliminate as many synchronization points as possible, but some will still be required. Patterns such as “Master/Worker” and “Producer/Consumer” are different patterns that developers can use to implement parallel thread processing.

MapReduce provides a programming model which abstracts many of the aforementioned complexities of parallel processing from the software engineer. The MapReduce implementation performs much of the “wiring” associated with parallel processing, leaving the developer to implement relatively simple methods. The use of MapReduce does come with some constraints, making it less appropriate for some tasks. MapReduce models are optimized for tasks where a large number of key*value input lists must be processed somewhat independently. MapReduce map() method must be commutative, in order for the MapReduce implementation to make use of parallelization. MapReduce enables the parallelization across hundreds and even thousands of CPU’s.

Views: 63

Reply to This

Take our poll!

Take our poll!

Latest Activity

Mark Nilson posted events
Friday
John Robinson posted an event
Thumbnail

A guide to practical Risk Management - Applying ISO14971 and IEC62304 at Philadelphia, PA

October 3, 2019 at 9am to October 4, 2019 at 6pm
Description:Gaps, incorrect or incomplete implementation of safety functionality can delay or make the certification/approval of medical products impossible. Most activities cannot be retroactively performed since they are closely linked into the development lifecycle. Diligent, complete and correct implementation of risk management from the start of product development is therefore imperative. This course will introduce all necessary steps to design, implement and test critical medical devices…See More
Friday
Adam Fleming posted a blog post

FOOD SUPPLY CHAIN – An Incredibly Easy Method That Works For All

A few sciences are changing our lives in more ways than we could have imagined a few decades ago. Artificial Intelligence (AI) is at the forefront of these. Although a few decades old, it has received a huge boost of late with the emergence of the cloud, which is set to help it overcome one of its biggest stumbling blocks, namely space, the added power of computing, and wider methods of algorithms.While one would like to associate AI with robotics, this is taking a very limited view of a topic…See More
Thursday
Mark Nilson posted events
Thursday

Forum

Green Smart Technical 800+ Turnkey Projects in UAE – Rope Access Company in Dubai

Green smart is an ISO 9001: 2015 QMS, OHSAS 18001: 2017 OHSMS, ISO 14001: …Continue

Tags: window, cleaning, services, facade, companies

Started by Manikandan Apr 23.

Workplace safety for workers other than your own 2 Replies

Do your workplace/traffic safety plans include safety measures for workers who are not your own employees? Do they take into account the safety of those who will be - or could be - at your workplace,…Continue

Tags: emt, paramedics, emergency, occupational, health

Started by John Petropoulos Memorial Fund. Last reply by Tara safe Mar 1.

What % of Dollars should be bugeted for All Safety Training for a "Heavy Equipment Road Construction" company with 100 employees? 1 Reply

I have been at a Highway and Road Construction company for six months. I am developing a (first) complete training program. I am the 1st full time safety hire for this company. I need to develop a…Continue

Started by JTurpening. Last reply by Jen McDade Feb 19.

Technology and Safety 1 Reply

What are the most useful tools to have at your fingertips? -Toolbox Talks-JSA-JHA-Daily Reports, etc. What is falling through the cracks that could be an easy fix? Safety Managers, Coordinators and…Continue

Tags: safety

Started by Drew stone. Last reply by Jen McDade Feb 15.

Important of Warning sign 1 Reply

Warning sign is a type of traffic sign that guide a hazard ahead on the road. Having proper warning sign on the road provide a healthy environment.Continue

Tags: Signs, Workplace, Safety, Sign, Warning

Started by healthandsafetysigns. Last reply by Jen McDade Jan 14.

Badge

Loading…

© 2019   Created by Safety Community.   Powered by

Badges  |  Report an Issue  |  Terms of Service