DEV Community

#spark

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Como aprendi Apache Spark — Parte 1: Por que o Spark mudou o processamento de Big Data
renanpyd profile

Como aprendi Apache Spark — Parte 1: Por que o Spark mudou o processamento de Big Data

#spark #dataengineering #bigdata #python
1
Comments
7 min read
Como aprendi Apache Spark: revisitando uma jornada pela Engenharia de Dados
renanpyd profile

Como aprendi Apache Spark: revisitando uma jornada pela Engenharia de Dados

#spark #bigdata #dataengineering #opensource
1
Comments
4 min read
How I Built a Two-Tier Data Platform in 9 Phases (Spark, Kafka, dbt, Kubernetes)
adnanphp profile
Adnan

How I Built a Two-Tier Data Platform in 9 Phases (Spark, Kafka, dbt, Kubernetes)

#dataengineering #spark #kafka #kubernetes
Comments
6 min read
What Actually Happens When You Call spark.read? One Line of Python, a Thousand Tasks
anikethsdeshpande profile

What Actually Happens When You Call spark.read? One Line of Python, a Thousand Tasks

#spark #dataengineering #python #distributedsystems
Comments
42 min read
The Distributed Compute Playground: Everything You Can Plug In and Tune in an Apache Spark Job
anikethsdeshpande profile

The Distributed Compute Playground: Everything You Can Plug In and Tune in an Apache Spark Job

#spark #dataengineering #python #distributedsystems
2
Comments 1
50 min read
Spark Structured Streaming Can’t Save You From Bad Architecture
aniketsoni profile

Spark Structured Streaming Can’t Save You From Bad Architecture

#spark #architecture #data #engineering
Comments
5 min read
Delta Lake ACID vs Apache Spark DataFrames: What the Databricks Data Engineer Exam Really Tests
palak_mazumdar_f5431445f6 profile

Delta Lake ACID vs Apache Spark DataFrames: What the Databricks Data Engineer Exam Really Tests

#databricks #dataengineering #spark #certification
Comments
5 min read
Fabric Runtime 2.0 is becoming the default. Here is what actually breaks.
firfircelik profile

Fabric Runtime 2.0 is becoming the default. Here is what actually breaks.

#fabric #spark #dataengineering #azure
Comments 2
7 min read
Don't Let Unity Catalog Migration Become Your Resume Generating Event
aniketsoni profile

Don't Let Unity Catalog Migration Become Your Resume Generating Event

#databricks #spark #engineering #migration
Comments
5 min read
Spark Is a Smart Engine. So Why Doesn't It Cache Automatically?
mohamedsaid25 profile

Spark Is a Smart Engine. So Why Doesn't It Cache Automatically?

#dataengineering #spark #caching
Comments
3 min read
Is Your Data Lake Actually A Landfill?
aniketsoni profile

Is Your Data Lake Actually A Landfill?

#iceberg #data #engineering #spark
Comments
5 min read
Why are you still debugging Spark on your host machine?
aniketsoni profile

Why are you still debugging Spark on your host machine?

#spark #docker #data #engineering
Comments
4 min read
Cutting a production AWS Glue pipeline's runtime by about half, plus two bugs that taught me more
kiran_gunturu_022085015ec profile

Cutting a production AWS Glue pipeline's runtime by about half, plus two bugs that taught me more

#aws #dataengineering #spark #performance
Comments
4 min read
Spark's coupling was thinner than it looked
mykola_melnyk_ml profile

Spark's coupling was thinner than it looked

#opensource #spark #webdev #documentprocessing
1
Comments
5 min read
Microsoft Fabric Now Encrypts Spark Temporary Data with Customer-Managed Keys;What It Actually Means
gilbert_lelon_8352bf20997 profile

Microsoft Fabric Now Encrypts Spark Temporary Data with Customer-Managed Keys;What It Actually Means

#fabric #spark #dataengineering #azure
Comments
5 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.