Online Safety Community

In order to understand the goals of MapReduce, it is important to realize for which scenarios MapReduce is optimized. The MapReduce programming model is created for processing data which requires “DATA PARALLELISM”, the ability to compute multiple independent operations in any order (King). In parallel processing, commutative operations are operations where the order of execution does not matter to the results of the equation. Commutativity can apply to complex operations and even processes, as long as they don’t manipulate the same memory. For example, in the figure below, as long as foo(a) and bar(b) don’t manipulate the same variable, they can occur in parallel in different threads. However, the write operation must wait for both foo() and bar() to complete. The figure below illustrates a dependency graph between foo(a), bar(a) and the write command.

Figure 1 – Parallelism Dependency Graph

One of the goals of parallelism is identifying the logical “tasks” or units which can be run in parallel as threads. Parallel programming techniques require developers to implement dependency graphs, which can become much more as the amount of shared information and sequence of operations increases. Techniques such as locks and barriers, critical sections, semaphores, monitors, RPC and rendezvous have been proposed to aid in the design of multi threaded and distributed. In Parallel and Distributed processing, intelligent task design attempts to eliminate as many synchronization points as possible, but some will still be required. Patterns such as “Master/Worker” and “Producer/Consumer” are different patterns that developers can use to implement parallel thread processing.

MapReduce provides a programming model which abstracts many of the aforementioned complexities of parallel processing from the software engineer. The MapReduce implementation performs much of the “wiring” associated with parallel processing, leaving the developer to implement relatively simple methods. The use of MapReduce does come with some constraints, making it less appropriate for some tasks. MapReduce models are optimized for tasks where a large number of key*value input lists must be processed somewhat independently. MapReduce map() method must be commutative, in order for the MapReduce implementation to make use of parallelization. MapReduce enables the parallelization across hundreds and even thousands of CPU’s.

Views: 53

Reply to This

Take our poll!

Take our poll!

Latest Activity

Nakul Pratap shared their blog post on Twitter
yesterday
HrishikeshRam posted blog posts
yesterday
Kyle C. Johnson posted a discussion

[General Industry] What is your workplace's policy on headphones? Working on one currently.

I have been tasked to create a headphones (and cell phone) policy for my employer. I am relatively new to this company, but so far they've let everyone listen to headphones and mess around with their phones at any time, anywhere. Even the PIT drivers. It's insane. My initial reaction is to say NO HEADPHONES WHATSOEVER, but my ops manager is concerned it will drastically reduce morale. I suggested that each department should have their own radio and they can play music for everyone to hear, at a…See More
yesterday
John Robinson posted a blog post

Nutrition Facts labels of pure maple syrup and honey

FDA Commissioner, Scott Gottlieb, has announced a major overhaul of the FDA’s update to the Nutrition Facts label on products. This overhaul, the first in nearly twenty years, is aimed at empowering consumers by furnishing information on Nutrition Facts labels pure maple syrup and honey that ensures greater accuracy and is more science-based. With this, the FDA wants to help consumers make more informed and healthier choices.This update to the Nutrition Facts label has incorporated the latest…See More
Friday

Forum

[General Industry] What is your workplace's policy on headphones? Working on one currently.

I have been tasked to create a headphones (and cell phone) policy for my employer. I am relatively new to this company, but so far they've let everyone listen to headphones and mess around with their…Continue

Tags: general, industry, distraction, music, phone

Started by Kyle C. Johnson yesterday.

AI Machine Learning In Businesses How wоuld уоu define a small buѕinеѕѕ, thе number оf реrѕоnеl, thе size оf the budget, оr ѕаlеѕ?

How wоuld уоu define a small buѕinеѕѕ, thе number оf реrѕоnеl, thе size оf the budget, оr ѕаlеѕ?However you define уоur оwn ѕmаll business, it tоо can tаkе advantage оf thе AI/ …Continue

Tags: elite, crest, technologies, intelligence, Artificial

Started by emmablisa Dec 10.

Distracted Driving 1 Reply

****WARNING**** Despite the fact that this is a life event portrayed by actors, it's graphic. This video should be part of…Continue

Tags: texting, fatality, safety, car, driving

Started by Wendy. Last reply by Jen McDade Dec 5.

Safety update: OSHA announces fit-test procedures 1 Reply

Fabricators as well as other workers may be required to use respirators to protect themselves from inhaling fumes, particles, or dust when performing cutting, grinding, welding, coating, or painting,…Continue

Tags: online, training, Safety, workplace, PPE

Started by Safety Training International. Last reply by Jen McDade Dec 4.

Badge

Loading…

© 2018   Created by Safety Community.   Powered by

Badges  |  Report an Issue  |  Terms of Service