Understand the process of configuring Spark Application

  • Apache Spark Application Name
  • Number of Apache Spark Driver Cores
  • Driver’s Maximum Result Size
  • Driver’s Memory
  • Executors’ Memory
  • Spark’s Extra Listeners
  • Local Directory
  • Log Spark Configuration
  • Spark Master
  • Deploy Mode of Spark Driver
  • Log App Information
  • Spark Driver Supervise Action
  • Name of the Property: spark.driver.cores
  • Default value: 01
  • Exception: This property is considered only within-cluster mode.
  • Name of the property: spark.driver.maxResultSize
  • Default value: 1 GB
  • Exception: Min value 1MB
  • Property Name : spark.driver.memory
  • Default value: Its 1g or 1 GB
  • Exception: In case, the spark application is yielded in client mode, the property has to be set through the command line option –driver-memory.

--

--

--

Big Data,ios,android,Spark

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

CookieSwap has been reviewed by RugDoc!

Learning To Code: The 5 Stages of Progress

What You Actually Need To Become A Real Full Stack Developer

Smart Moving Mod 1.16.4–1.14.4–1.12.2–1.10.2–1.7.10— Sprinting key — Wminecraft.net

Smart Moving Mod 1.16.4–1.14.4–1.12.2–1.10.2–1.7.10

Road to OSCP 6: Active HackTheBox

The roadmap of Livesola

Voicemod license key reddit

My first open-source contribution….

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
MultiTech

MultiTech

Big Data,ios,android,Spark

More from Medium

Big Data and Hadoop

What is Apache Spark? A Beginners Guide

Understanding Apache Hive Transforms

Spark Applications Overview | Use Cases of Apache Spark