Big Data Analyst Course

Lesson 1 • Defining Big Data Concepts

Introduces the 5 Vs framework and distinguishes big data from traditional data. Establishes vocabulary used throughout the course.

Lesson 2 • Data Roles and Analyst Responsibilities

Differentiates data engineer, data scientist, and data analyst roles within a big data team. Clarifies the analyst's scope of ownership and collaboration points.

Lesson 3 • Business Drivers and Use Cases

Examines why organisations invest in big data and which industries derive the most value. Grounds technical learning in real-world business outcomes.

Lesson 1 • NoSQL Database Fundamentals

Surveys key-value, document, column-family, and graph databases and their appropriate use cases. Equips analysts to query and interpret data stored in non-relational systems.

Lesson 2 • Data Warehouses and OLAP

Explains columnar storage, star and snowflake schemas, and OLAP cube design for analytical queries. Connects warehouse design decisions to query performance.

Lesson 3 • Data Governance and Quality

Introduces data cataloging, lineage tracking, and quality dimensions critical for trustworthy analytics. Establishes governance practices that underpin all analytical work.

Lesson 4 • Data Lakes and Lakehouses

Defines data lake architecture, its advantages over rigid schemas, and the lakehouse pattern that merges lake flexibility with warehouse governance. Prepares students for modern hybrid storage environments.

Lesson 1 • Distributed Query Engines

Explains how engines like Hive, Presto, and Spark SQL execute queries across distributed storage. Connects query syntax to underlying execution plans.

Lesson 2 • Advanced SQL for Analytics

Covers window functions, CTEs, and complex aggregations essential for analytical workloads. Builds on basic SQL knowledge to handle multi-step analytical logic.

Lesson 3 • Data Profiling and Exploration

Uses SQL-based profiling to assess data distributions, nulls, and anomalies before deeper analysis. Establishes a systematic exploration workflow for new datasets.

Lesson 1 • Spark SQL and DataFrames

Applies Spark SQL and DataFrame APIs to perform large-scale data wrangling and aggregation. Bridges SQL skills from Chapter 3 with distributed processing.

Lesson 2 • Spark Structured Streaming

Demonstrates building streaming pipelines with Spark Structured Streaming using the unified DataFrame API. Connects stream processing theory to practical implementation.

Lesson 3 • Apache Spark Core Concepts

Covers RDDs, DataFrames, and the Spark execution model including DAG scheduling and lazy evaluation. Forms the primary processing skill set for the course.

Lesson 4 • MapReduce Programming Model

Explains the map, shuffle, and reduce phases as the conceptual basis for distributed computation. Provides the mental model needed to understand higher-level frameworks.

Lesson 1 • Data Cleaning Strategies

Addresses missing values, duplicates, inconsistent formats, and erroneous records using scalable methods. Directly improves the reliability of all downstream analytical outputs.

Lesson 2 • Data Integration and Joining

Merges data from heterogeneous sources using deterministic and fuzzy matching techniques. Enables analysts to build unified datasets from fragmented enterprise data.

Lesson 3 • Feature Engineering Techniques

Transforms raw variables into informative features through encoding, binning, and interaction creation. Prepares data for both analytical reporting and machine learning pipelines.

Lesson 4 • Sampling and Data Reduction

Applies stratified, reservoir, and systematic sampling to reduce dataset size while preserving statistical properties. Balances analytical accuracy with computational efficiency.

Lesson 1 • Descriptive Statistics at Scale

Computes and interprets central tendency, dispersion, and shape metrics on large distributed datasets. Establishes the statistical baseline for all analytical and modeling work.

Lesson 2 • Correlation and Regression Analysis

Measures variable relationships using correlation coefficients and builds linear regression models for prediction. Bridges statistical analysis with predictive analytical tasks.

Lesson 3 • Hypothesis Testing Methods

Applies t-tests, chi-square tests, and ANOVA to validate analytical claims on large samples. Connects statistical rigor to business decision-making.

Lesson 4 • A/B Testing and Experimentation

Designs controlled experiments, calculates sample sizes, and interprets results to measure causal impact. Applies hypothesis testing directly to product and business experiments.

Lesson 1 • Model Evaluation and Selection

Uses cross-validation, confusion matrices, ROC curves, and information criteria to compare and select models. Ensures analytical conclusions rest on rigorously validated models.

Lesson 2 • Clustering and Segmentation

Applies k-means, hierarchical, and density-based clustering to discover natural groupings in large datasets. Enables customer segmentation and anomaly detection use cases.

Lesson 3 • Machine Learning Fundamentals

Defines supervised, unsupervised, and reinforcement learning paradigms and the ML workflow. Establishes conceptual grounding before hands-on model building.

Lesson 4 • Classification and Regression Models

Builds decision trees, random forests, and gradient boosting models for classification and regression tasks. Applies models to structured big data using Spark MLlib.

Lesson 1 • Visualisation Design Principles

Applies perceptual and cognitive principles to choose chart types, colour palettes, and layouts that minimise misinterpretation. Establishes the design foundation for all visual outputs.

Lesson 2 • Interactive Dashboards

Builds interactive dashboards with filters, drill-downs, and KPI cards to support self-service analytics. Extends static visualisation skills to dynamic, user-driven reporting.

Lesson 3 • Data Storytelling and Presentation

Structures analytical findings into a narrative arc with a clear problem, evidence, and recommendation. Translates technical outputs into executive-ready communication.

Lesson 4 • Geospatial and Advanced Visuals

Creates choropleth maps, network graphs, and Sankey diagrams for complex multi-dimensional data. Expands the analyst's visual toolkit for specialised analytical scenarios.