Prashant Pandey

@prashant_pdy

Big Data Consultant @ Tata Consultancy Services Ltd. Creating Big Data Tutorials @ youtube.com/learningjourna…

Bangalore

learningjournal.guru

Joined April 2015

137 Following

285 Followers

Replies Reposts Media

Prashant Pandey
@prashant_pdy
May 12, 2024
Spark performance tuning scenario-based interview questions for data engineers and architects. Complete course available here. scholarnest.in/courses/apache…
Prashant Pandey
@prashant_pdy
Jul 11, 2023
How to grow from data engineer to a solution architect? Do you know the answer or know the career path? I know one thing. To be a solution architect, you need to start being a solution architect. No one is offering you a role that you are not already…
#data #cloud #architect | Prashant Kumar Pandey
From linkedin.com
Prashant Pandey
@prashant_pdy
Jul 10, 2023
Big data, data lake, data ingestion and lakehouse architects! Here is a design scenario and a bunch of interview questions for data ingestion. You have data arriving in a Kafka cluster and you are asked to design a data ingestion pipeline to bring data…
#bigdata #dataengineering #data #cloud | Prashant Kumar Pandey
From linkedin.com
Prashant Pandey
@prashant_pdy
Jul 7, 2023
What are the 5-S of Spark performance tuning? These are the critical areas of performance bottlenecks in a Spark application and are subject to tuning and need special care and follow best practices. Here is the list. 1. Skew 2. Spill 3. Shuffle 4. Stora…
#dataengineering #bigdata #data | Prashant Kumar Pandey
From linkedin.com
Prashant Pandey
@prashant_pdy
Jul 6, 2023
Common misconception. Spark Dataframe supports structured and semi-structured data. But for unstructured data, you must use RDD APIs. Here is another way of saying the same statement. Spark Dataframe supports CSV and JSON. But for reading a text and imag…
#dataanalytics #bigdata #dataengineering #python #java #video #streaming #aws #computerprogramming...
From linkedin.com