Product Feedback

  • Hot ideas
  • Top ideas
  • New ideas
  • My feedback
  1. We need the ability to mask/obfuscate sensitive data in DataBricks notebooks.

    We sensitive data on S3 that we'd like to be able to mask/obfuscate. Traditional databases have the functionality to mask column based on security permissions. Below is how SQL Server handles it. For example, only being able to see the last 4 digits of a ssn. We need the same functionality when view data in DataBricks notebooks.

    https://docs.microsoft.com/en-us/sql/relational-databases/security/dynamic-data-masking?view=sql-server-2017

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  2. Timer for currently running cell

    It would be really handy to have a running timer for currently processing jobs. It would help users identify cells which have been over running due to some user error, for example running a job with exponential complexity that would take hours whereas the usual process only takes a few seconds.

    7 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Notebooks  ·  Flag idea as inappropriate…  ·  Admin →
  3. "Single Node TensorFlow to Distributed DL" example worked

    I can't finish https://docs.databricks.com/applications/deep-learning/distributed-training/mnist-tensorflow.html. My cluster is vanilla 5.3 ML image, standard nc12 gpu. On "prepare single node code", I get from "train(learning_rate=.01, batch_size=64)" an "UnknownError: /dbfs/horovod_tf_keras//MNISTDemo/train/1556245977.6539524/checkpoint.tmpa6724f77bca94128bd6ee8f49cbbd5ba; Input/output error". And under "Migrate to HorovodRunner" I get from "hr.run(train_hvd, learning_rate=0.01, batch_size=64)" the error "org.apache.spark.SparkException: Job aborted due to stage failure: Could not recover from a failed barrier ResultStage. Most recent failure reason: Stage failed because barrier task ResultTask(1, 0) finished unsuccessfully.". I realize I probably do something silly. Advice is appreciated. :-)

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Notebooks  ·  Flag idea as inappropriate…  ·  Admin →
  4. Allow management of Groups through the UI

    Allow management of Groups through the UI rather than just through the Groups API.

    Only being able to Add/Remove/List Users in a Group through the Groups API is inconvenient and restrictive.

    The current state of Permissions management for administrators of Databricks is not a great experience.

    When a new User is granted access, the Administrators need to go and manually grant the User Permission to all of the Libraries/Folders/Jobs/Clusters etc.

    This manual process has resulted in Users being granted Permissions that they shouldn't have or not being granted Permissions that they should have.

    We want to use Groups to help…

    35 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    3 comments  ·  Navigation UI  ·  Flag idea as inappropriate…  ·  Admin →
  5. 3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  External libraries / applications  ·  Flag idea as inappropriate…  ·  Admin →
  6. Clusters created with python3 used pip3

    I created a cluster using python3, but pip failed to install a package made for python3, and pip3 is not present.

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  External libraries / applications  ·  Flag idea as inappropriate…  ·  Admin →
  7. there is a detailed description of mlflow exceptions along with their resolutions on mlflow website.

    While trying out the various functionality of mlflow lifecycle, I experienced multiple mlflow exceptions like 'Could not find a registered tracking store ' etc. which was not explained anywhere on mlflow website or even in Azure Databricks documentation. It will be good to have a dedicated page explaining the reasons of the exceptions that usually occur along with solution to resolve them similar to what Microsoft does for Windows exceptions.

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  8. gitlab integration

    adding gitlab support as github so I can store them my shit and optionallya add some auto export when committing in pdf so I can get them in the same location easiily

    28 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Notebooks  ·  Flag idea as inappropriate…  ·  Admin →
  9. dbutils.fs.ls would return the lastmodified timestamp of files in addition to the size.

    The dbutils.fs.ls command returns the path, filename and size of the files it lists. With the timestamp the input files can be processed in the proper sequence.

    5 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Other  ·  Flag idea as inappropriate…  ·  Admin →
  10. Upload library for all clusters through API

    If a library has been set to be installed on all clusters, is_library_for_all_clusters will be true. Right now, this can only be done through the home page. It would be great if this could be done through the API or CLI.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  REST API  ·  Flag idea as inappropriate…  ·  Admin →
  11. The job list page sorted by job id in descending order

    When I navigate to the job list page (https://###.cloud.databricks.com/#joblist) it would be nice if the jobs were sorted by job id in descending order so the latest job was at the top.

    I use Hadoop and HUE and they do this and I find it very less user friendly that the Databricks job list doesn't do this.

    Also, if I look at the details of a job and then navigate back to the job list the sort I had selected (job id desc) is lost and I have to resort.

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Navigation UI  ·  Flag idea as inappropriate…  ·  Admin →
  12. i am unable to create Cluster using trial version , its showing error "This account may not have enough cores to satisfy the request"

    I am trying to create cluster using trial account however getting error. I am unable to set the configuration as its allowing enter min and max workers. 1 worker is adding 4 cores and I have max 4 cores allocated. I cannot provide 0 as min value and max should be greater than min. So technically I am unable to create the cluster, please help.

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  13. Can list what users have create cluster permission through api

    I would like to be able to list what users have create cluster permission through the rest api.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. Style of widgets

    Customize the style of any widgets in azure Databricks is not possible

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Notebooks  ·  Flag idea as inappropriate…  ·  Admin →
  15. use VSTS git repo with databricks on AWS

    We use VSTS for CI/CD and AWS for our cloud infrastructure. It would be nice to access VSTS from databricks on AWS.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. Display Function in Notebooks could be used to generate multiple plots

    I would like to be able to generate a set of visuals from a single display function call or SQL query output without having to rerun the same chunk multiple times.

    13 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Notebooks  ·  Flag idea as inappropriate…  ·  Admin →
  17. Databricks could integrate better with a Python IDE

    Similar to the RStudio Server integration which has become availble recently, it would be great to have an integration with at least one Python IDE (PyCharm, Spyder, etc.). Is such an integration planned for the near future? The development of code in notebooks can be rather cumbersome for larger projects.

    26 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Other  ·  Flag idea as inappropriate…  ·  Admin →
  18. Support of Shapely values prediction in MLlib boosted tree models.

    Shapely Feature Contributions at row low level for tree ensembles. This allows explainable at the row level of why things are being predicted. There is already a method that can be build into tree ensembles (like xboost) to calculated shapely very fast using the tree nodes
    https://www.groundai.com/project/consistent-feature-attribution-for-tree-ensembles/4

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  External libraries / applications  ·  Flag idea as inappropriate…  ·  Admin →
  19. Ablity to Download pickled files to a local machine, created in a notebook.

    I need a simply way to download pickled model files created by notebooks and on the cluster driver. This doesn't appear to be possible with no public IP set.

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Data import / export  ·  Flag idea as inappropriate…  ·  Admin →
  20. 3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Feedback and Knowledge Base