Known 2.2 image version issues and limitations:
Legacy agents are not installed in 2.2 image version clusters. monitoring-agent-defaults are not available unless the Ops Agent is installed. Note: OSS metrics and logging are available for Managed Service for Apache Spark components.
Logging or monitoring for third-party applications are not available unless the Ops Agent is installed.
Notes:
2.2.x-*-armimages support only the pre-installed components and the following optional components. The other 2.2 optional components and all initialization actions aren't supported:- Apache Hive WebHCat
- Apache Pig (starting with
2.2.75-ubuntu22-arm) - Docker
- Zeppelin
- Zookeeper (installed in high availability clusters; optional component in other clusters)
The source code to image 2.2 libraries that are licensed under Reciprocal and Restricted licenses is available at the
/usr/local/share/google/managed-spark/third-party-sourcespath on Managed Service for Apache Spark cluster VMs.The following Hudi procedures are known to not work on a Hudi table backed by the Cloud Storage file system:
run_clusteringrun_compaction
The default resource calculator for YARN has been changed from DefaultResourceCalculator to DominantResourceCalculator, which uses the dominant-resource concept to determine resource allocation, such as Memory and CPU allocation. This change impacts Autoscaler, which scales based on the dominant resource usage of the cluster.
Apache Iceberg version: In 2.2 images, Apache Iceberg is available in versions
1.6.1and1.10. The default version is1.6.1. To specify the Iceberg version on any 2.2 cluster, set thedataproc:dataproc.iceberg.versioncluster property to1.10(or1.6) during cluster creation.The
2.2.85and later image versions do not have preconfigured Conda channels. Packages can't be installed using Conda unless channels are manually configured during cluster initialization. Also,2.2image version clusters that have preconfigured Conda channels, including2.2.84and earlier image versions, can't be used after August 25, 2026.- You may need to delete and replace existing clusters. After August 25, 2026, existing clusters that were created with images that have pre-configured Conda channels, even if cluster jobs don't use Conda to install packages, must be deleted and replaced with new clusters created or recreated with images that don't have preconfigured Conda channels.