databricks-install-auth

# Databricks Install & Auth ## Overview Set up Databricks CLI and SDK with authentication credentials. ## Prerequisites - Python 3.8+ with pip - Databricks workspace URL (e.g., `https://adb-1234567890.1.azuredatabricks.net`) - Personal Access Token or OAuth credentials - Admin access for Unity Catalog setup ## Instructions ### Step 1: Install Databricks CLI and SDK ```bash # Install CLI v2 (recommended) pip install databricks-cli # Install Python SDK pip install databricks-sdk # Install dbx for local development pip install dbx ``` ### Step 2: Configure Authentication #### Option A: Personal Access Token (Simplest) ```bash # Interactive configuration databricks configure --token # Enter workspace URL: https://adb-1234567890.1.azuredatabricks.net # Enter token: dapi1234567890abcdef... ``` #### Option B: Environment Variables ```bash # Set environment variables export DATABRICKS_HOST="https://adb-1234567890.1.azuredatabricks.net" export DATABRICKS_TOKEN="dapi1234567890abcdef..." # Or create .env file cat > .env << 'EOF' DATABRICKS_HOST=https://adb-1234567890.1.azuredatabricks.net DATABRICKS_TOKEN=dapi1234567890abcdef... EOF ``` #### Option C: OAuth (Recommended for Production) ```bash # Using OAuth U2M (user-to-machine) databricks configure --oauth # Using OAuth M2M (service principal) export DATABRICKS_HOST="https://adb-1234567890.1.azuredatabricks.net" export DATABRICKS_CLIENT_ID="your-client-id" export DATABRICKS_CLIENT_SECRET="your-client-secret" ``` ### Step 3: Configure Profiles (Multi-Workspace) ```ini # ~/.databrickscfg [DEFAULT] host = https://adb-dev-1234567890.1.azuredatabricks.net token = dapi_dev_token... [staging] host = https://adb-staging-1234567890.1.azuredatabricks.net token = dapi_staging_token... [production] host = https://adb-prod-1234567890.1.azuredatabricks.net token = dapi_prod_token... ``` ### Step 4: Verify Connection ```bash # Test CLI connection databricks workspace list / # Test with specific profile databricks workspace list / --profile staging # Test SDK connection python -c " from databricks.sdk import WorkspaceClient w = WorkspaceClient() print(f'Connected to: {w.config.host}') print(f'Clusters: {len(list(w.clusters.list()))}') " ``` ## Output - Databricks CLI installed and configured - Python SDK installed - Authentication credentials stored - Connection verified to workspace ## Error Handling | Error | Cause | Solution | |-------|-------|----------| | `Invalid token` | Expired or incorrect token | Generate new PAT in User Settings | | `Workspace not found` | Wrong host URL | Verify workspace URL format | | `Permission denied` | Token lacks permissions | Request admin access or correct scopes | | `Connection refused` | Network/firewall issue | Check VPN, firewall rules | | `SSL certificate error` | Corporate proxy | Set `--insecure` or configure proxy | ## Examples ### Python SDK Initialization ```python from databricks.sdk import WorkspaceClient # Auto-detect from environment/config file w = WorkspaceClient() # Explicit configuration w = WorkspaceClient( host="https://adb-1234567890.1.azuredatabricks.net", token="dapi1234567890abcdef..." ) # With profile w = WorkspaceClient(profile="production") ``` ### Service Principal Authentication ```python from databricks.sdk import WorkspaceClient from databricks.sdk.config import Config config = Config( host="https://adb-1234567890.1.azuredatabricks.net", client_id="your-client-id", client_secret="your-client-secret" ) w = WorkspaceClient(config=config) ``` ### Azure AD Authentication ```python from databricks.sdk import WorkspaceClient from azure.identity import DefaultAzureCredential # Uses Azure AD managed identity w = WorkspaceClient( host="https://adb-1234567890.1.azuredatabricks.net", credentials_provider=DefaultAzureCredential() ) ``` ## Resources - [Databricks CLI Documentation](https://docs.databricks.com/dev-tools/cli/index.html) - [Databricks SDK for Python](https://docs.databricks.com/dev-tools/sdk-python.html) - [Authentication Methods](https://docs.databricks.com/dev-tools/auth.html) - [Personal Access Tokens](https://docs.databricks.com/dev-tools/auth.html#personal-access-tokens) ## Next Steps After successful auth, proceed to `databricks-hello-world` for your first cluster and notebook.