Ganglia is included in Databricks Cloud
Metrics: All, if possible. But these are the most important: cpu_*, dfs*, disk_*, disk_total, jvm.*, load_*, network_report,.
This use case is to help us determine:
1) The optimal set of configuration options.
2) The optimal size for the cluster.
3) Potential bugs in the code.
The first 2 points are interrelated.
You can view cluster metrics through Ganglia UI starting from August 2017. See more here: https://docs.databricks.com/user-guide/clusters/metrics.html#ganglia-metrics