Virtual Round Table · Jul 22

View the event
Apache Flink 2.3 Turns Streaming SQL Into a Production Discipline
Data Engineering

Apache Flink 2.3 Turns Streaming SQL Into a Production Discipline

The June 25 release adds changelog conversion operators, brings materialized tables to feature parity with regular tables, and ships a native S3 filesystem with zero Hadoop dependencies. Flink is quietly closing the gap between a stream and a governed table.

PublishedJuly 18, 2026
Read time5 min read
Share

A Maintenance Release That Is Anything But Minor

The Apache Flink PMC announced 2.3.0 on June 25, 2026, and on the surface it reads like a routine point release in the 2.x series. Read the changelog and a different picture emerges. The release, in the project's own words, "significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem," and adds application-level lifecycle management. Roughly 180 contributors are credited, which is a healthy signal for a project this deep into a major version line.

We flag this one because the theme running through it is operability. Each headline feature reduces the amount of bespoke engineering a team needs to run streaming pipelines in production. Flink has long been the most capable open stream processor, and just as often the most demanding to operate. Version 2.3 is a deliberate step toward making streaming SQL something a general data engineering team can own, rather than a specialty that lives with a handful of experts who understand the internals.

Changelog Operators Close a Real Gap

The most consequential additions are the FROM_CHANGELOG and TO_CHANGELOG SQL functions. They let engineers convert between append-only streams and dynamic changelog tables directly in SQL, and they support custom change-data-capture formats rather than only the handful Flink understood natively. Anyone who has piped Debezium or a homegrown CDC feed into a streaming job knows how much glue code this normally requires. Moving that logic into SQL operators makes the pipeline legible to a much wider set of engineers.

There is a sharp edge worth naming. As the release notes put it, "by default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy." That is stricter behavior than before, and it will surface latent bugs in existing jobs the moment you upgrade. We read that as a good default, because a pipeline that silently reconciles mismatched keys is a data-quality incident waiting to happen. Plan for the migration work, but treat the failure as the feature.

Materialized Tables Grow Up

Flink 2.3 brings materialized tables to feature parity with regular tables, and this is the change most likely to alter how teams design pipelines. CREATE MATERIALIZED TABLE now accepts explicit column definitions, including watermarks and primary keys. ALTER MATERIALIZED TABLE gains the full set of DDL verbs, ADD, MODIFY, DROP, and RENAME TO, so a materialized table can evolve through the same workflow teams already use for ordinary Flink tables. Schema evolution stops being a reason to avoid the feature.

The other addition here is the START_MODE clause, which gives granular control over reprocessing when a materialized table's query changes. Instead of an all-or-nothing rebuild, engineers can choose exactly where the refresh pipeline begins. For anyone who has watched a query edit trigger a full backfill across months of data, this is a meaningful cost and latency lever. It moves materialized tables from a promising abstraction toward something you can safely operate on live production data with predictable reprocessing behavior.

A Native S3 Filesystem Sheds Hadoop

The new flink-s3-fs-native plugin is built from the ground up on AWS SDK v2, with non-blocking I/O and native AWS integration including IAM Roles for Service Accounts. Critically, it carries zero Hadoop dependencies. For teams running Flink on Kubernetes, the Hadoop dependency chain has been a persistent source of classpath conflicts, bloated images, and security-patch churn that had nothing to do with stream processing itself. Removing it is the kind of unglamorous cleanup that pays off every single deployment.

The feature ships as experimental, so we would not point production checkpoints at it on day one. The direction is clear, though. Flink is trimming the legacy big-data baggage that made it heavy to run in cloud-native environments, and aligning its storage layer with how modern teams actually deploy. Combined with the application-level lifecycle management that replaces the older cluster-job model, 2.3 reads as a release focused on the boring, decisive details of running Flink reliably at scale.

Why This Matters for the Lakehouse

Flink does not live in isolation. The broader story of 2026 is streaming data landing directly in open table formats, with Confluent's Tableflow materializing Kafka topics as Iceberg tables and multiple ingestion paths competing on latency. Flink sits at the center of that picture as the processing engine that shapes streams before they become governed tables. The event stream processing market is on track to pass fourteen billion dollars, and the practical question for most teams is no longer whether to stream, but how cheaply and reliably they can.

Every feature in 2.3 nudges that answer in the right direction. Changelog operators make CDC pipelines maintainable in SQL. Materialized tables give teams a governed, evolvable target with controlled refresh. The native S3 filesystem makes cloud deployment lighter. Taken together, they lower the total cost of running streaming as a standard part of the data platform rather than a specialist annex. That is the shift worth planning around.

What to Do Before You Upgrade

Start by auditing existing streaming SQL jobs for upsert and primary-key mismatches, because 2.3 will now fail them at planning time. Better to find those in staging than in a 3 a.m. page. Inventory any custom CDC handling you built by hand and evaluate whether the new changelog operators can replace it, which would shrink your maintenance surface considerably. Treat that as a concrete refactoring backlog rather than a someday item.

Then decide where materialized tables now fit in your reference architecture. With schema evolution and START_MODE reprocessing control in place, several patterns that previously demanded custom Flink jobs may collapse into declarative SQL. Pilot the native S3 filesystem in a non-critical pipeline to measure the deployment simplification before you commit checkpoints to it. The upgrade rewards teams that treat streaming as core infrastructure and plan the migration deliberately, rather than bolting the new version onto old assumptions.

Tagged#news#data#data-engineering#databases#analytics#lakehouse#streaming#apache-flink#stream-processing#streaming-sql#cdc#materialized-tables