Spark 4.x support in Spark accelerators: where each one stands in September 2026

The Apache Spark news page lists these releases so far in 2026: 4.1.1 on 9 January, 3.5.8 on 15 January, 4.0.2 on 5 February, 4.1.2 on 21 May, 4.0.3 on 11 June, 4.2.0 on 14 July, 4.1.3 and 4.0.4 on 15 July, and 3.5.9 on 16 July, with five 4.2.0 previews between 11 January and 1 May. Spark 4.1.0 came out on 16 December 2025.

The 4.2.0 release notes highlight GEOMETRY and GEOGRAPHY types, a SQL CHANGES clause for change data capture, Arrow-optimized Python UDFs switched on by default, vectorized data loading improvements in the Parquet reader and shuffle checksum validation.

Accelerators hook into Spark below its public API. The RAPIDS Accelerator documentation describes replacing "parts of the physical plan that Apache Spark considers internal", and Gluten, Comet and Auron all rewrite physical plans and replace the shuffle manager (see how accelerators plug in). Internal interfaces change between Spark releases, so each accelerator ships builds for specific versions. When you upgrade Spark, check that your accelerator lists your exact version, not only the minor line.

Managed platforms add a second layer. On Amazon EMR or Databricks, the Spark version comes with the platform release you choose, so the accelerator has to support the Spark version inside that release. That is why the table below lists Photon against Databricks Runtime versions, and why EMR users should look up the Spark version of their EMR release in the AWS release guide before matching it to an accelerator's list.

Databricks' release notes page for Runtime 19 gives general availability as 23 July 2026. Comet 1.0.0 also deprecates JDK 11, with removal planned in 1.1.0. Gluten's 1.7.0 release adds Delta 4 native write for Spark 4.0.

• Is my exact Spark patch version supported, experimental or deprecated in the accelerator release I would run?
• Which operators fall back to standard Spark on the new version that did not before?
• Does the new release change the JDK requirement?
• On a managed platform such as Amazon EMR or Databricks, which platform release carries the Spark version, and does the accelerator list that release?
• Will you rerun the baseline after the upgrade, before comparing accelerator results?

The last point matters because Spark itself changes between releases. A speedup measured on Spark 3.5 against a Spark 3.5 baseline does not tell you what you will see on 4.2. Flarion's August 2026 post on upgrades makes a related point: validating that output data is unchanged is the hard part of moving Spark versions, and it applies to accelerator trials too. Our proof of concept guide covers the output checks.