Online Safety Community

In order to understand the goals of MapReduce, it is important to realize for which scenarios MapReduce is optimized. The MapReduce programming model is created for processing data which requires “DATA PARALLELISM”, the ability to compute multiple independent operations in any order (King). In parallel processing, commutative operations are operations where the order of execution does not matter to the results of the equation. Commutativity can apply to complex operations and even processes, as long as they don’t manipulate the same memory. For example, in the figure below, as long as foo(a) and bar(b) don’t manipulate the same variable, they can occur in parallel in different threads. However, the write operation must wait for both foo() and bar() to complete. The figure below illustrates a dependency graph between foo(a), bar(a) and the write command.

Figure 1 – Parallelism Dependency Graph

One of the goals of parallelism is identifying the logical “tasks” or units which can be run in parallel as threads. Parallel programming techniques require developers to implement dependency graphs, which can become much more as the amount of shared information and sequence of operations increases. Techniques such as locks and barriers, critical sections, semaphores, monitors, RPC and rendezvous have been proposed to aid in the design of multi threaded and distributed. In Parallel and Distributed processing, intelligent task design attempts to eliminate as many synchronization points as possible, but some will still be required. Patterns such as “Master/Worker” and “Producer/Consumer” are different patterns that developers can use to implement parallel thread processing.

MapReduce provides a programming model which abstracts many of the aforementioned complexities of parallel processing from the software engineer. The MapReduce implementation performs much of the “wiring” associated with parallel processing, leaving the developer to implement relatively simple methods. The use of MapReduce does come with some constraints, making it less appropriate for some tasks. MapReduce models are optimized for tasks where a large number of key*value input lists must be processed somewhat independently. MapReduce map() method must be commutative, in order for the MapReduce implementation to make use of parallelization. MapReduce enables the parallelization across hundreds and even thousands of CPU’s.

Views: 45

Reply to This

Take our poll!

Take our poll!

Latest Activity

HrishikeshRam posted blog posts
4 hours ago
Russel Stuart posted blog posts
5 hours ago
Nakul Pratap posted a blog post

Which is the best NEBOSH course for you?

Learn From the Best The NEBOSH IGC course will be delivered by highly skilled and experienced professionals, who are already with NEBOSH qualification. The tutors will not just teach this as subject, but also makes you understand and gives the skill to implement the safety measures at workplace.Do complete the NEBOSH IGC with the leading…See More
5 hours ago
Benny Smith posted a blog post

Sciatica Pain Treatment and Sources Of the Illness

Sciatic nerve pain is identified by discomfort that spreads out along the sciatic nerve, radiating from the buttocks and back of the leg to the foot. The two sciatic nerves (one nerve on each side of the reduced body) are the longest in the human nerves. They are born in the lower back; expand in the thigh as well as calf to the foot.Sciatic nerve pain is caused by various kinds of stress on the sciatic nerve, lower back, or buttocks. It can also be caused by nerve origin sores that belong to…See More
7 hours ago

Forum

Road Safety

What can be the best measures for Road Safety?Continue

Started by Jen McDade Sep 27.

safety equipments in breweries 4 Replies

safety equipments in breweries -- kindly provide detailsContinue

Started by ajit. Last reply by Jen McDade Sep 24.

Unusual Safety Tips 10 Replies

Please share some unusual but genuine safety precautions.My unusual safety tips - Don't drink sweet fruity juice while driving in the night as it contains sugar and sugar slows down your blood…Continue

Started by Arnold Brame. Last reply by Jen McDade Sep 24.

Safety products 9 Replies

While working or at some work stations, employees are exposed to threats or at times suffer severe injuries. The safety of the employee should be the foremost priority. Safety equipment aids help in…Continue

Tags: equipments, and, products

Started by Enna Henry. Last reply by Jen McDade Sep 24.

Talend Application Integration

APPLICATION INTEGRATION:Talend ESB and Application Integration delivers agile real-time integration of applications and data.Heterogeneous Application and Data IntegrationTalend’s open source ESB…Continue

Tags: york, new, certification, training, talend

Started by nicolewells Sep 21.

Badge

Loading…

© 2018   Created by Safety Community.   Powered by

Badges  |  Report an Issue  |  Terms of Service