Big Data: Overview of apache Hadoop

What YARN Does

  • Scalability The processing power in data centers continues to grow quickly. Because YARN ResourceManager focuses exclusively on scheduling, it can manage those larger clusters much more easily.
  • Compatibility with MapReduce Existing MapReduce applications and users can run on top of YARN without disruption to their existing processes.
  • Improved cluster utilization. The ResourceManager is a pure scheduler that optimizes cluster utilization according to criteria such as capacity guarantees, fairness, and SLAs. Also, unlike before, there are no named map and reduce slots, which helps to better utilize cluster resources.
  • Support for workloads other than MapReduceAdditional programming models such as graph processing and iterative modeling are now possible for data processing. These added models allow enterprises to realize near real-time processing and increased ROI on their Hadoop investments.
  • AgilityWith MapReduce becoming a user-land library, it can evolve independently of the underlying resource manager layer and in a much more agile manner.

How YARN Works

  • a global ResourceManager
  • a per-application ApplicationMaster.
  • a per-node slave NodeManager and
  • a per-application Container running on a NodeManager

--

--

--

Big Data,ios,android,Spark

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Notes on further wallet refactoring

Reduce Cost and Increase Productivity with Value Added IT Services from buzinessware — {link} -

3 WAYS AZURE DATABRICKS CAN MAKE YOUR LIFE EASIER

Using Cron Jobs and How to Create Schedule Job in WLSDM

Turn your Python code into GUI’s easily!

Apex WTF #01

PyBlog — Project Structure

Wacky Sorting Algorithms

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
MultiTech

MultiTech

Big Data,ios,android,Spark

More from Medium

Data vs Big Data: Know the difference

Converting JD Edwards Julian Dates to Gregorian Using Spark SQL Function in Databricks

Convert Julain to Gregorian 1900's

Data Persistence