Keep common python packages up-to-date in default machine image
Some of the common python packages (e.g., scipy, numpy, and scikit-learn) came with the Databricks default machine image were out-of-date. It will be great if Databricks can keep these commonly used python packages up-to-date in the default machine image.
We have now substantially upgraded the software on the images. In fact, we regularly update various software installed on the clusters. The release notes covers the exact version of all the installed software. Here is an example of this: