PySpark Overview — PySpark 4.1.1 documentation
PySpark Overview # Date: Jan 02, 2026 Version: 4.1.1 Useful links: Live Notebook | GitHub | Issues | Examples | Community | Stack Overflow | Dev Mailing List | User Mailing List
Apache Spark™
Apache Spark is a multi-language engine for executing data engineering, data science, and machine learning on single-node machines or clusters.
Spark SQL and DataFrames
Spark SQL, DataFrames and Datasets Guide Spark SQL is a Spark module for structured data processing. Unlike the basic Spark RDD API, the interfaces provided by Spark SQL provide Spark
Documentation | Apache Spark
Hands-On Exercises Hands-on exercises from Spark Summit 2014. These let you install Spark on your laptop and learn basic concepts, Spark SQL, Spark Streaming, GraphX and MLlib. Hands-on
Examples | Apache Spark
Apache Spark ™ examples This page shows you how to use different Apache Spark APIs with simple examples. Spark is a great engine for small and large datasets. It can be used with single
Spark Streaming
Spark Streaming is an extension of the core Spark API that enables scalable, high-throughput, fault-tolerant stream processing of live data streams. Data can be ingested from many sources like Kafka,
Spark SQL & DataFrames | Apache Spark
Spark SQL is Spark''s module for working with structured data, either within Spark programs or through standard JDBC and ODBC connectors.
Spark Energy Solar Panels Online Store | SPARK ENERGY
Shop Spark Energy for reliable solar panels and solar energy solutions that brighten your home and save on power bills.
It Only Takes a Spark to Start a Revolution!
Most solar companies want to sell you the largest system possible. We take a different approach. By reducing your energy consumption first, we can install a
Overview
Spark Connect is a new client-server architecture introduced in Spark 3.4 that decouples Spark client applications and allows remote connectivity to Spark clusters.
Quick Start
Quick Start Interactive Analysis with the Spark Shell Basics More on Dataset Operations Caching Self-Contained Applications Where to Go from Here This tutorial provides a quick introduction to using
Configuration
The Spark shell and spark-submit tool support two ways to load configurations dynamically. The first is command line options, such as --master, as shown above. spark-submit can accept any Spark