Advanced Developer Training

StreamSets Advanced Developer Training

Labs

Review lesson slides and run labs at your own pace

Live Instructor Led

Take the 2-day course live with a Certified Instructor

StreamSets Advanced Developer Course Overview

High-performance deployments deserve high-caliber training. The fastest, most reliable way to build proven skills in StreamSets is via expert instructor-led hands-on classroom training in a structured learning environment.

This 2-day hands-on advanced developer course dives into solving use cases that span across data ingest (Data Collector) and data engineering (Transformer). The course also covers deeper topics such as advanced expression language, CI/CD, pipeline testing using the StreamSets SDK, complex pipeline development, custom processors, and best practices.

Requirements

Students preferably should have a general knowledge of operating systems, networking, programming concepts, and databases. Students should also have a solid understanding of the objectives covered in the following fundamentals courses:
Control Hub Fundamentals
Data Collector Fundamentals
Transformer Fundamentals

Audience

The course is designed for data engineers, developers, data flow engineers, and systems architects who will be architecting and developing advanced pipelines.

Objectives

Introduction
Lab environment
Course Resources

Quick Review
Control Hub
Data Collector
Transformer

Expression Language
Constants
Literals
Variables & Fields
Functions & Operators
Runtime EL
Regular expressions

SDC Evaluator Processors
Record processing with scripts
Individual record inspection
Logging and error handling

Transformer Spark Processors & Evaluators
Spark SQL Query Processor
Spark SQL Expression Processor
PySpark Processor

End to End Pipelines
Data ingest & transformation use cases
Complex pipelines
Advanced parameterization
Connection Catalog
Orchestration pipelines
Topologies
Custom component development
Kafka Pipelines
CDC Pipelines

Pipeline Testing & Automation
The Pipeline Repository
Pipeline development lifecycle
Testing pipelines
Pipeline CI/CD
SCH REST APIs

Capstone Project
Develop an end-to-end solution given a complex use case
Optimize & tune solution