Apache NiFi Course

Lesson 1 • What Is Apache NiFi

Covers NiFi's origin, design philosophy, and primary use cases in data flow automation. Establishes context for all subsequent architectural and hands-on content.

Lesson 2 • Installing and Configuring NiFi

Walks through standalone installation, directory layout, and essential configuration files. Students gain a working local environment for all lab exercises.

Lesson 3 • Navigating the NiFi Web UI

Introduces the canvas, toolbar, component palette, and summary views. Students can orient themselves and perform basic canvas operations after this section.

Lesson 1 • Essential Ingestion Processors

Covers GetFile, GenerateFlowFile, ListenHTTP, and similar source processors for common ingestion patterns. Students build flows that pull data from files and HTTP endpoints.

Lesson 2 • Outputting and Delivering Data

Covers PutFile, PutHTTP, LogAttribute, and similar sink processors for delivering processed data. Completes the end-to-end flow pattern introduced in this chapter.

Lesson 3 • Routing and Filtering FlowFiles

Introduces RouteOnAttribute, RouteOnContent, and FilterAttribute for conditional flow logic. Students implement branching flows based on data content and metadata.

Lesson 1 • FlowFile Attributes Deep Dive

Examines core and custom FlowFile attributes, how they are created, and how they propagate. Provides the attribute foundation required for all EL exercises.

Lesson 2 • Dynamic Property Configuration with EL

Shows how to embed EL in processor properties to create data-driven, reusable flows. Reduces hardcoded values and increases flow maintainability.

Lesson 3 • Expression Language Syntax and Functions

Covers EL syntax, operator types, and the most-used built-in functions for string, maths, and date operations. Students write and test expressions directly in processor properties.

Lesson 1 • Record-Oriented Processing Model

Introduces NiFi's Record API, schema registries, and Reader/Writer controllers for structured data. Establishes the record model that underpins all transformation processors.

Lesson 2 • Splitting and Merging Records

Explains SplitRecord, SplitJson, MergeRecord, and MergeContent for batch and streaming patterns. Students control FlowFile granularity to optimise throughput and downstream processing.

Lesson 3 • Data Enrichment and Lookup

Covers LookupRecord, LookupAttribute, and controller-service-backed enrichment patterns. Students add contextual data to FlowFiles from databases, files, and in-memory caches.

Lesson 4 • Content Transformation Processors

Teaches ReplaceText, JoltTransformJSON, and XSLT-based transformation for content-level changes. Students apply structural and value-level transformations to FlowFile content.

Lesson 1 • Database Connectivity

Covers DBCPConnectionPool, ExecuteSQL, PutSQL, and QueryDatabaseTable for relational database integration. Students build flows that read from and write to SQL databases.

Lesson 2 • Cloud Storage Integration

Introduces processors for object storage services including S3-compatible, Azure Blob, and GCS patterns. Students list, fetch, and put objects in cloud storage within a dataflow.

Lesson 3 • REST API and HTTP Integration

Covers InvokeHTTP, ListenHTTP, and HandleHttpRequest for bidirectional HTTP communication. Students build flows that call external APIs and expose NiFi as an HTTP endpoint.

Lesson 4 • Messaging System Integration

Teaches Kafka, JMS, and AMQP processor configuration for event-driven pipeline integration. Students produce and consume messages from enterprise messaging platforms.

Lesson 1 • Process Groups and Ports

Explains how Process Groups encapsulate logic and how Input/Output Ports enable inter-group communication. Students refactor flat flows into hierarchical, navigable structures.

Lesson 2 • Templates and Flow Versioning

Covers template export/import and NiFi Registry integration for version-controlled flow management. Students commit, diff, and revert flows using the Registry workflow.

Lesson 3 • Controller Services and Shared Resources

Covers controller service scoping, enabling, and disabling across Process Groups. Students centralise shared resources like connection pools and schema registries for reuse.

Lesson 1 • Authorisation and Multi-Tenancy

Explains NiFi's policy-based authorisation model, user groups, and resource-level access control. Students grant least-privilege access to users and groups across Process Groups.

Lesson 2 • Sensitive Data and Encryption

Covers NiFi's sensitive property encryption, EncryptContent processor, and secrets management integration. Students protect credentials and PII within flows and configuration files.

Lesson 3 • Error Handling and Dead-Letter Patterns

Teaches retry loops, failure routing, and dead-letter queue patterns for resilient flow design. Students build flows that recover from transient errors without data loss.

Lesson 4 • Monitoring Flows and System Health

Covers the Summary page, Bulletin Board, Data Provenance, and reporting tasks for operational visibility. Students detect bottlenecks and failures using built-in monitoring tools.

Lesson 1 • NiFi Cluster Architecture

Explains the cluster coordinator, primary node, ZooKeeper roles, and heartbeat mechanics. Students understand how NiFi distributes work and maintains cluster state.

Lesson 2 • Deployment Automation and Containerisation

Covers Docker images, Kubernetes StatefulSets, and Helm charts for containerised NiFi deployment. Students automate cluster provisioning using infrastructure-as-code approaches.

Lesson 3 • Upgrading and Maintaining NiFi

Explains rolling upgrades, backup strategies, and flow migration between NiFi versions. Students execute a zero-downtime upgrade and validate flow integrity post-upgrade.

Lesson 4 • Performance Tuning and Sizing

Covers JVM heap, thread pool, repository storage, and back-pressure tuning for high-throughput flows. Students apply tuning parameters to meet defined throughput and latency targets.