This post is how to use the Databricks Python SDK.
Install the Package
- pip install databricks-sdk
Update Package
- pip install databricks-sdk --upgrade
Check Package Version
- pip show databricks-sdk | grep -oP '(?<=Version: )\S+'
Setup WorkspaceClient
- from databricks.sdk import WorkspaceClient
- secret = dbutils.secrets.get(scope = "<SCOPE>", key = "<KEY>")
- w = WorkspaceClient(
- host = 'https://<URL>/'
- azure_workspace_resource_id = '<RESOURCE_ID_OF_DATABRICKS>',
- azure_tenant_id = '<TENANT_ID>',
- azure_client_id = '<CLIENT_ID>',
- azure_client_secret = secret
- )
Setup AccountClient
You can get the account_id from the databricks account portal. By your id in the top right hand corner.
- from databricks.sdk import AccountClient
- secret = dbutils.secrets.get(scope = "<SCOPE>", key = "<KEY>")
- a = AccountClient(
- host = 'https://accounts.azuredatabricks.net'
- account_id = '<ACCOUNT_ID>'
- azure_tenant_id = '<TENANT_ID>',
- azure_client_id = '<CLIENT_ID>',
- azure_client_secret = secret
- )
List Workspace Groups
NOTE: You must also setup the workspaceclient to do this.
- w.groups.list()
List Account Groups
NOTE: You must also setup the accountclient to do this. You must also be account admin.
- a.groups.list()
Create Storage Credential
NOTE: Your SPN must be account admin to do this. You must also setup the workspaceclient to do this.
- from databricks.sdk.service.catalog import AzureManagedIdentity
- storage_credential_name = '<CREDENTIAL_NAME>'
- comment = '<COMMENT>'
- connector_id = '<DATABRICKS_ACCESS_CONNECTOR>'
- az_mi = AzureManagedIdentity(access_connector_id = connector_id)
- w.storage_credenditals.create(
- name = storage_credential_name,
- azure_managed_identity = az_mi
- comment = comment
- )