Rankings · Updated 13 October 2026
Spark ETL accelerators: what each one publishes, criterion by criterion
ETL Compare staff · Figures checked September 2026
In brief
This page lists the published facts behind each criterion for all 9 accelerators, in alphabetical order, without scores or ranks. The scored ranking answers one buyer question: Which accelerator speeds up the Spark ETL you already run, on the platform you already use, without code changes or a dedicated tuning team?
Lead ranking: speeding up the Spark ETL you already run without a tuning team.
How much work to run an existing Spark ETL job on it, starting from the platform you already use? No code changes, few setup steps and no platform move score high.
| Vendor | What the vendor or project publishes | Source |
|---|---|---|
| Apache Auron (formerly Blaze) | No code changes, but you enable it through Spark settings and a custom shuffle manager and install it yourself. | Auron review |
| Apache DataFusion Comet | No code changes; add the Comet jar from Maven Central, enable the extension and configure off-heap memory and the Comet shuffle manager. | Comet review |
| Apache Gluten (with Velox) | No application code changes; you add the Gluten JAR, set spark.plugins, enable off-heap memory and switch the shuffle manager, per the Velox getting-started page. | Gluten + Velox review |
| Databricks Photon | One checkbox (or on by default on serverless) with no code changes if you already run on Databricks; for a team on EMR or self-managed Spark it means moving to Databricks first. | Photon review |
| DataPelago Accelerator for Spark | DataPelago states existing Spark code runs unchanged with no data migration, configured through standard Spark settings and bought through the Google Cloud or AWS marketplace. | DataPelago review |
| DualBird | DualBird states two steps: change the EC2 instance type and add the DualBird Spark plugin, with no code changes and no platform move for EMR and EKS users. | DualBird review |
| Flarion | Flarion states "zero code and infrastructure changes" and a 6-minute setup by adding configuration parameters. | Flarion review |
| RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF for Apache Spark) | No code changes, but the job has to move to NVIDIA GPU instances and the cluster needs GPU-specific configuration. | RAPIDS Accelerator review |
| Speedata | Speedata states Dash drops into existing Spark environments with zero code changes, but the job has to move onto servers fitted with Speedata APU cards, bought pre-configured or through OEM partners such as HPE. | Speedata review |
Which stages of an ETL job (read, transform, shuffle, spill, write) does the vendor's own documentation say it addresses, and what happens to unsupported operations?
| Vendor | What the vendor or project publishes | Source |
|---|---|---|
| Apache Auron (formerly Blaze) | Native vectorized execution on DataFusion, compacted shuffle formats and multi-level memory management are all documented. | Auron review |
| Apache DataFusion Comet | Native Parquet scans (including for Iceberg), native operators and native or columnar shuffle; native operators can spill, bounded per task; write path not described on the pages we reviewed. | Comet review |
| Apache Gluten (with Velox) | Offloads execution to the Velox native engine with a columnar shuffle manager; spilling is documented as experimental. | Gluten + Velox review |
| Databricks Photon | Documents native scans (Parquet, Delta, CSV, JSON), filters, joins, aggregates, windows and writes, with fallback to the Spark runtime; no UDF, RDD or stateful streaming support. | Photon review |
| DataPelago Accelerator for Spark | Its page names good fits (compute-intensive jobs, Parquet, ORC, SQL and DataFrame) and says to avoid heavy RDD and UDF use; shuffle, spill and write behavior are not described. | DataPelago review |
| DualBird | The only vendor in this set whose product page names all three ETL pain points directly: disk spills, data skew and shuffle data, plus an end-to-end claim. | DualBird review |
| Flarion | Describes a DataFusion-based, Arrow-native execution engine plus caching; shuffle, spill and write behavior are not described on the pages we reviewed. | Flarion review |
| RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF for Apache Spark) | Documents GPU execution for group by, joins, sorts and windows, Parquet and ORC writing, CSV reading and a RAPIDS Shuffle Manager; unsupported operations fall back to CPU. | RAPIDS Accelerator review |
| Speedata | The product page says Dash offloads compute-intensive work and that non-accelerable elements such as UDFs fall back to CPU; no operator list and no shuffle, spill or write detail are published. | Speedata review |
After install, how much memory sizing, configuration and troubleshooting falls on your team, and who supports you when a job fails?
| Vendor | What the vendor or project publishes | Source |
|---|---|---|
| Apache Auron (formerly Blaze) | Self-managed with community support through the Apache mailing list. | Auron review |
| Apache DataFusion Comet | The tuning guide says memory accounting "isn't 100% accurate" and describes pool fractions, batch size and spill limits to set; community support. | Comet review |
| Apache Gluten (with Velox) | You size off-heap memory yourself (the docs example uses 20g) and rely on community channels; no vendor support contract is described on the project pages. | Gluten + Velox review |
| Databricks Photon | Managed by Databricks inside its runtime, so there is no plugin for your team to install or version. | Photon review |
| DataPelago Accelerator for Spark | A vendor-run accelerator configured with standard Spark settings; tuning guidance and support terms are not published on the pages we reviewed. | DataPelago review |
| DualBird | DualBird states "no more performance tuning and troubleshooting" and the product is vendor-supported; this is a vendor claim we could not check. | DualBird review |
| Flarion | A commercial product with performance monitoring and anomaly detection described; support terms are not published. | Flarion review |
| RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF for Apache Spark) | Qualification and Profiling tools help, but GPU sizing and plugin configuration are your team's job. | RAPIDS Accelerator review |
| Speedata | A vendor product with a Workload Analyzer that reads your Spark logs; you run dedicated hardware in your own data center, and support terms are not published. | Speedata review |
How many Spark platforms is it documented for, and does it run on the instance types you already use?
| Vendor | What the vendor or project publishes | Source |
|---|---|---|
| Apache Auron (formerly Blaze) | Runs on standard CPU instances and supports JDK 8, 11, 17 and 21; the project says it is adapted to Spark mainline versions without listing them on the page we reviewed. | Auron review |
| Apache DataFusion Comet | Supports Spark 3.5, 4.0 and 4.1 (3.4 deprecated) with prebuilt Linux amd64 and arm64 jars on commodity hardware; you install it yourself on any platform. | Comet review |
| Apache Gluten (with Velox) | Supports Spark 3.4, 3.5, 4.0 and 4.1 on x86_64 and aarch64 Linux on standard CPU instances; managed-platform guides are not listed, so you install it yourself. | Gluten + Velox review |
| Databricks Photon | Available only inside Databricks (on AWS, Azure and Google Cloud). | Photon review |
| DataPelago Accelerator for Spark | Supports multiple Spark versions on Google Cloud and AWS and any hardware from CPU to GPU, by the vendor's statement; specific managed services such as Amazon EMR, Dataproc or Databricks are not named. | DataPelago review |
| DualBird | Documented for Apache Spark, Amazon EMR and Amazon EKS on AWS only, and it runs on Amazon EC2 F2 instances rather than the instance types most jobs use today. | DualBird review |
| Flarion | Lists Databricks, AWS EMR, GCP Dataproc, Azure HDInsight, Spark on Kubernetes and on-premises, on standard instances. | Flarion review |
| RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF for Apache Spark) | The widest platform list in this set (EMR, Databricks, Dataproc, GKE, Azure Synapse, Kubernetes, on-premises, OCI), held back because every one of them needs GPU instances. | RAPIDS Accelerator review |
| Speedata | Documented for on-premises servers with C200 cards, plus a February 2026 sovereign-cloud partnership with Nebul; Amazon EMR, EKS, Databricks and Kubernetes are not mentioned on the product page. | Speedata review |
Can you work out what it costs before a sales call: license or fee, plus any change in instance price?
| Vendor | What the vendor or project publishes | Source |
|---|---|---|
| Apache Auron (formerly Blaze) | Apache License 2.0, no license fee. | Auron review |
| Apache DataFusion Comet | Apache License 2.0, no license fee. | Comet review |
| Apache Gluten (with Velox) | Apache License 2.0, no license fee; cost is your existing compute plus engineering time. | Gluten + Velox review |
| Databricks Photon | Databricks publishes DBU rates, but Photon compute consumes DBUs at different rates, so the net effect has to be modelled per job. | Photon review |
| DataPelago Accelerator for Spark | Pay as you go through cloud marketplaces; the AWS Marketplace listing shows a vCPU-hour dimension and a $100,000 per month contract entry, but no per-job rate you can model without a quote. | DataPelago review |
| DualBird | No pricing on the pages we reviewed; buyers need a quote to compare the fee plus F2 instance cost against current spend. | DualBird review |
| Flarion | Listed on AWS Marketplace; no price on the homepage. | Flarion review |
| RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF for Apache Spark) | The plugin is Apache 2.0 with no fee; the cost question becomes GPU instance price against runtime saved, which you can model from public cloud prices. | RAPIDS Accelerator review |
| Speedata | No pricing page and no public price for the software or the APU servers; buyers need a demo and a quote. | Speedata review |
Are performance figures published with enough setup detail to reproduce, and has anyone other than the vendor published results?
| Vendor | What the vendor or project publishes | Source |
|---|---|---|
| Apache Auron (formerly Blaze) | The project states about 2x faster than Spark 3.5 on TPC-DS with about 50% cluster resources saved; project-published. | Auron review |
| Apache DataFusion Comet | Publishes a TPC-DS at 1 TB benchmark with a per-query breakdown in its Benchmarking Guide. | Comet review |
| Apache Gluten (with Velox) | Publishes TPC-H and TPC-DS results (Velox backend 2.71x overall, tested June 2023) with the benchmark named; results come from the project. | Gluten + Velox review |
| Databricks Photon | Databricks documents where Photon does not help (queries under two seconds, UDFs); performance comparisons outside Databricks are not possible because it only runs there. | Photon review |
| DataPelago Accelerator for Spark | Vendor claims of 10x faster at 80% cost reduction, customer quotes with figures (Akad Seguros, ShareChat) and a benchmark page against NVIDIA cuDF, all vendor-published. | DataPelago review |
| DualBird | Publishes an Iceberg compaction benchmark with dataset shape and cost per TB, written by DualBird staff; cluster sizes and Spark versions are not stated and no third-party results were found. | DualBird review |
| Flarion | Headline figures (3x performance, 60% cost reduction) without a published methodology on the pages we reviewed. | Flarion review |
| RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF for Apache Spark) | The Qualification Tool estimates fit from your own event logs; headline benchmark figures were not reviewed for this edition. | RAPIDS Accelerator review |
| Speedata | Vendor figures include 62.7x on production Spark workloads and 37 servers consolidated to 3; the site also cites a SIGMOD 2026 Best Paper whose Microsoft CoddSpeed benchmark features the APU. Setup detail for the vendor figures is not published. | Speedata review |
How long has it been in use, who contributes, and how visible is its production track record?
| Vendor | What the vendor or project publishes | Source |
|---|---|---|
| Apache Auron (formerly Blaze) | Still incubating at the Apache Software Foundation (latest listed release v8.0.0-incubating), with a history under the Blaze name. | Auron review |
| Apache DataFusion Comet | A subproject of Apache DataFusion that has reached a 1.x release line; younger than Gluten and RAPIDS. | Comet review |
| Apache Gluten (with Velox) | Apache top-level project since March 2026, started by Intel and Kyligence in 2022, with contributors including Alibaba Cloud, Meituan, Microsoft, IBM and Google. | Gluten + Velox review |
| Databricks Photon | The default engine on Databricks serverless and SQL warehouses, backed by Databricks. | Photon review |
| DataPelago Accelerator for Spark | The Accelerator for Spark launched on 5 August 2025, on the Nucleus engine unveiled in October 2024; offices in Mountain View and Hyderabad; no open-source community of its own. | DataPelago review |
| DualBird | A young commercial product (it raised $25M in February 2026) with a short public track record and no open-source community. | DualBird review |
| Flarion | Company details such as founding date and funding are not published on the pages we reviewed. | Flarion review |
| RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF for Apache Spark) | A long-running NVIDIA project with more than 9,000 commits on main, now published as NVIDIA cuDF for Apache Spark. | RAPIDS Accelerator review |
| Speedata | A commercial hardware vendor with Israeli founders; TechCrunch reported a $44M Series B in June 2025 (secondary source). Founding year is not stated and there is no open-source community. | Speedata review |