Apache Spark & Scala Training in Hyderabad @ ORIENIT

Apache Spark & Scala Content

Introduction to Big Data

What is Big Data

Challenges with Big Data

Batch Vs. Real Time Big Data Analytics

Batch Analytics

• Hadoop Ecosystem Overview

Real Time Analytics

• Streaming Data - Storm
• In Memory Data - Spark

Introduction of Spark

What is Spark

Why Spark

Who Uses Spark

Brief History of Spark

Storage Layers for Spark

Unified Stack of Spark

• Spark Core
• Spark Sql
• Spark Streaming
• MLlib
• GraphX

Modes of Spark Installation

Learn Spark more ..

Installation Spark in different modes

• Local mode
• Pseudo mode
• Cluster mode

Spark Architecture explanation

• Master Slave architecture
• Spark Driver
• Workers
• Executors

Working with Spark in different programming languages

Python

• How to use 'pyspark'
• Practical examples on spark in python

Scala

• How to use 'spark-shell'
• Practical examples on spark in Scala

Java

• How to use 'Java'
• Practical examples on spark in Java

R

• How to use 'SparkR'
• Practical examples on spark in R

Creating the Spark Context

Configuring Spark Context with SparkConf

Caching Overview

Distributed Persistence

Combine scala and java seamlessly

Deploying Applications with spark-submit

Verify spark jobs in Spark Web UI

Installing sbt

Building a Spark Project with sbt

Running Spark Project with sbt

Installing maven

Building a Spark Project with maven

Running Spark Project with maven

Resilient Distributed Dataset (RDD)

What is RDD

Creating RDDs

RDD Operations

• Transformations
• Actions
• Lazy Evaluation

Passing Functions to Spark

• Python, Java, Scala

Working with Key/Value Pairs

Creating Pair RDDs

Transformations on Pair RDDs

• Aggregations
• Grouping Data
• Joins
• Sorting Data

Data Partitioning

• Determining an RDD’s Partitioner
• Custom Partitioners

Loading and Saving Your Data

File Formats

• Text, Json, csv, tsv, Object files
• Hadoop Input and Output Formats

Loading Data using RDD

Saving Data using RDD

MapReduce and Pair RDD Operations

Scala and Hadoop Integrations

Broadcast and Accumulators

Accumulators

• Introduction to Accumulators
• Practical Examples on Accumulators
• Creating Custom Accumulators

Broadcast variables

• Introduction to Broadcast variables
• Practical Examples on Broadcast variables
• Optimizing Broadcasts

Apache Spark SQL

Hive and Spark SQL Architecture explanation

Working with Spark SQL DataFrames

Using Spark SQL Context

Practical examples on Spark SQL

Integrating hive and Spark SQL

Creating & Using SQL Hive Context

Hive Queries through Spark

Processing Text, JSON and Parquet Files using in Spark

Spark SQL UDFs

Spark SQL Performance Tuning Options

JDBC/ODBC Server

Apache Spark Streaming

Spark Streaming Architecture explanation

Transformations

Output Operations

Streaming UI explanation

Performance Considerations

Practical examples on Spark Streaming

Apache Spark MLlib

Machine Learning Basics

Machine Learning Algorithms

• Classification
• Clustering
• Collaborative Filtering

         Performance Considerations

Practical examples on Spark MLlib

Apache Mesos

Introduction to Apache Mesos

Apache Mesos Architecture explanation

Practical Examples on Apache Mesos

Apache Mahout

Introduction to Apache Mahout

Apache Mahout Architecture explanation

Practical Examples on Apache Mahout

Apache Storm

Introduction to Apache Storm

Apache Storm Architecture explanation

Practical Examples on Apache Storm

Apache Kafka

Introduction to Apache Kafka

Apache Kafka Architecture explanation

Practical Examples on Apache Kafka

Introduction of Scala

What is Scala?

Why Scala?

Advantages of Scala?

Using the Scala REPL(Read Evaluate print loop)

What is Type Inference

Interoperability between Scala and Java

Scala using Command Line

Installing Java & Scala

Interactive Scala

Writing Scala Scripts

Compiling Scala Programs

Basics of Scala

Defining Variables

Defining Functions

String Interpolation

IDE for Scala

Scala Type Less, Do More

Semicolons

Variable Declarations

Method Declarations

Type Inference

Immutability

Reserved Words

Operators

Precedence Rules

Literals

Options

Arrays, Lists, Ranges, Tuples

Expressions and Conditionals

If expressions

If-Else expressions

Match Expressions

For Loops

While Loops

Do-While Loops

Conditional Operators

Enumerations

Pattern Matching

Using try, catch, and finally Clauses

Functional Programming in Scala

What is Functional Programming?

Functional Literals and Closures

Recursions

Tail Calls

Currying

Functional Data Structures

• Sequences,Maps,Sets

Traversing

• Traversal, Mapping, Filtering, Folding and Reducing

Implicit Function Parameters

Call by Name, Call by Value

Object-Oriented Programming in Scala

Class and Object Basics

Value Classes

Parent Types

Constructors in Scala

Fields in Classes

Nested Types

Traits as Mixins

Stackable Traits

Creating Traits

Visibility Rules

Scala for Big Data

Improving MapReduce with Scala

Moving Beyond MapReduce

Categories for Mathematics

A List of Scala-Based Data Tools

What we are offering to you:

• Hands on Practice on Spark & Scala Real-Time Examples.
• Providing 1 Major project on Spark.
• Providing 2 Mini projects on Spark.
• Hands on installation Spark and it's relative software's in your laptop.
• Well documented Spark & Scala material with all the topics covering in the course.
• Well documented Spark blog contains frequent interview questions along with the answers and latest updates on BigData technology.
• Discussing about Spark & Scala interview questions daily base.
• Resume preparation with POC's or Project's based on your experience.