Online Safety Community

In order to understand the goals of MapReduce, it is important to realize for which scenarios MapReduce is optimized. The MapReduce programming model is created for processing data which requires “DATA PARALLELISM”, the ability to compute multiple independent operations in any order (King). In parallel processing, commutative operations are operations where the order of execution does not matter to the results of the equation. Commutativity can apply to complex operations and even processes, as long as they don’t manipulate the same memory. For example, in the figure below, as long as foo(a) and bar(b) don’t manipulate the same variable, they can occur in parallel in different threads. However, the write operation must wait for both foo() and bar() to complete. The figure below illustrates a dependency graph between foo(a), bar(a) and the write command.

Figure 1 – Parallelism Dependency Graph

One of the goals of parallelism is identifying the logical “tasks” or units which can be run in parallel as threads. Parallel programming techniques require developers to implement dependency graphs, which can become much more as the amount of shared information and sequence of operations increases. Techniques such as locks and barriers, critical sections, semaphores, monitors, RPC and rendezvous have been proposed to aid in the design of multi threaded and distributed. In Parallel and Distributed processing, intelligent task design attempts to eliminate as many synchronization points as possible, but some will still be required. Patterns such as “Master/Worker” and “Producer/Consumer” are different patterns that developers can use to implement parallel thread processing.

MapReduce provides a programming model which abstracts many of the aforementioned complexities of parallel processing from the software engineer. The MapReduce implementation performs much of the “wiring” associated with parallel processing, leaving the developer to implement relatively simple methods. The use of MapReduce does come with some constraints, making it less appropriate for some tasks. MapReduce models are optimized for tasks where a large number of key*value input lists must be processed somewhat independently. MapReduce map() method must be commutative, in order for the MapReduce implementation to make use of parallelization. MapReduce enables the parallelization across hundreds and even thousands of CPU’s.

Views: 56

Reply to This

Take our poll!

Take our poll!

Latest Activity

Adam Fleming posted a blog post

Trial Registration and Results Reporting on ClinicalTrials.gov

The Final Rule for Clinical Trials Registration and Results Information Submission (42 CFR Part 11) clarifies and expands the regulatory requirements and procedures for submitting registration and results information for certain clinical trials to ClinicalTrials.gov, in accordance with Section 801 of the Food and Drug Administration Amendments Act (FDAAA 801).The Final Rule has been in effect since January 18, 2017. The International Committee of Medical Journal Editors (ICMJE) requires trial…See More
12 hours ago
Traininngdotcom-LLC posted a blog post

An Introductory overview of the FDA and EMA Requirements

The aim of this webinar is to provide an introductory overview of the requirements demanded by the FDA and EMA (arguably the two most important regulatory authorities for drug developers) for EIM (the IND and IMPD) and to provide an brief review of where the agencies differ as well the commonalities of approach for their EIM dossiers.Although each regulatory authority publishes guidelines and instructions regarding the requirements for the Entry Into Man (EIM) of a new medication, such…See More
13 hours ago
Training Doyens posted events
14 hours ago
Martin walker is now a member of Online Safety Community
Friday

Forum

Technology and Safety 1 Reply

What are the most useful tools to have at your fingertips? -Toolbox Talks-JSA-JHA-Daily Reports, etc. What is falling through the cracks that could be an easy fix? Safety Managers, Coordinators and…Continue

Tags: safety

Started by Drew stone. Last reply by Jen McDade on Friday.

Important of Warning sign 1 Reply

Warning sign is a type of traffic sign that guide a hazard ahead on the road. Having proper warning sign on the road provide a healthy environment.Continue

Tags: Signs, Workplace, Safety, Sign, Warning

Started by healthandsafetysigns. Last reply by Jen McDade Jan 14.

Workers paticipation in safety management 2 Replies

Workers paticipation in safety management is the aspect which is required to be implemented in the OHSAS 18001 2007 version. , I invite our experience community members to share their views on the…Continue

Tags: management, safety, in, paticipation, Workers

Started by SafetyRaja. Last reply by Tara safe Dec 27, 2018.

How to improve safety culture of factories 4 Replies

How to improve safety culture of factories having mostly contract and casual ever changing workers for whom training and monitoring both are major issues. Such qorkers are mainly meeting accidents in…Continue

Started by Harkant Dave. Last reply by Jen McDade Dec 24, 2018.

[General Industry] What is your workplace's policy on headphones? Working on one currently. 1 Reply

I have been tasked to create a headphones (and cell phone) policy for my employer. I am relatively new to this company, but so far they've let everyone listen to headphones and mess around with their…Continue

Tags: general, industry, distraction, music, phone

Started by Kyle C. Johnson. Last reply by Jen McDade Dec 19, 2018.

Badge

Loading…

© 2019   Created by Safety Community.   Powered by

Badges  |  Report an Issue  |  Terms of Service