{"id":840,"date":"2025-09-07T14:34:04","date_gmt":"2025-09-07T14:34:04","guid":{"rendered":"https:\/\/dataopsschool.com\/blog\/?p=840"},"modified":"2025-09-07T14:34:06","modified_gmt":"2025-09-07T14:34:06","slug":"databricks-user-management-in-databricks","status":"publish","type":"post","link":"https:\/\/dataopsschool.com\/blog\/databricks-user-management-in-databricks\/","title":{"rendered":"Databricks: User Management in Databricks"},"content":{"rendered":"\n<p><\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h1 class=\"wp-block-heading\">Introduction<\/h1>\n\n\n\n<p>In Databricks, identities (users, groups, service principals) live at the <strong>account<\/strong> level and can be assigned to one or more <strong>workspaces<\/strong>. For Unity Catalog (UC), principals must exist at the <strong>account<\/strong> level to be granted data privileges, and Databricks recommends provisioning via <strong>SCIM<\/strong> from your IdP (Microsoft Entra ID). (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/data-governance\/unity-catalog\/best-practices?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/p>\n\n\n\n<p><strong>Roles you typically need<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Cloud admin<\/strong> (Azure) to create users\/service principals in Entra ID.<\/li>\n\n\n\n<li><strong>Account admin<\/strong> to add principals to the Databricks <strong>account<\/strong> and assign to workspaces.<\/li>\n\n\n\n<li><strong>Workspace admin<\/strong> to manage <strong>workspace<\/strong> entitlements (personas) and workspace-local settings. (<a href=\"https:\/\/docs.databricks.com\/aws\/en\/admin\/?utm_source=chatgpt.com\">Databricks Documentation<\/a>)<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h1 class=\"wp-block-heading\">How to add\/create new users in Microsoft Entra ID (Azure AD)<\/h1>\n\n\n\n<ol class=\"wp-block-list\">\n<li>In the Azure portal, open <strong>Microsoft Entra ID<\/strong> \u2192 <strong>Users<\/strong> \u2192 <strong>New user<\/strong>.<\/li>\n\n\n\n<li>Set <strong>User principal name (UPN)<\/strong> and <strong>Display name<\/strong>, create an initial password, and <strong>Create<\/strong>.<\/li>\n\n\n\n<li>(Optional) Add the user to Entra <strong>groups<\/strong> you\u2019ll sync to Databricks later via SCIM.<\/li>\n<\/ol>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>Entra \u201cusers\u201d become principals you can <strong>provision<\/strong> into Databricks with SCIM. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/admin\/users-groups\/scim\/aad?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/p>\n<\/blockquote>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h1 class=\"wp-block-heading\">How to add\/create new users in the <strong>Databricks Account console<\/strong><\/h1>\n\n\n\n<p>\u2705 Best practice: Use <strong>SCIM<\/strong> to sync identities automatically (next section).<br>If you must add manually:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Go to <strong>Account console<\/strong> \u2192 <strong>User management<\/strong> \u2192 <strong>Users<\/strong> \u2192 <strong>Add user<\/strong>.<\/li>\n\n\n\n<li>Enter the user\u2019s email\/UPN and <strong>Add<\/strong>.<\/li>\n\n\n\n<li>(Optional) Assign <strong>account admin<\/strong> or other <strong>account-level<\/strong> roles if needed (use sparingly). (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/admin\/users-groups\/users?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/li>\n<\/ol>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>Account admins can also <strong>assign users to workspaces<\/strong> from the account console (assuming identity federation). (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/admin\/users-groups\/users?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/p>\n<\/blockquote>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h1 class=\"wp-block-heading\">What is <strong>SCIM<\/strong> in Databricks?<\/h1>\n\n\n\n<p><strong>SCIM<\/strong> (System for Cross-domain Identity Management) is the standard used to <strong>auto-provision<\/strong> users and groups from your IdP (Microsoft Entra ID) into the <strong>Databricks account<\/strong>. Once configured, new Entra users\/groups sync automatically into Databricks (no manual adds), and removals\/updates propagate. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/admin\/users-groups\/scim\/?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/p>\n\n\n\n<p><strong>High-level setup (Entra \u2192 Databricks account)<\/strong><\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>In Entra, create\/select the <strong>Azure Databricks (SCIM)<\/strong> enterprise app (account-level).<\/li>\n\n\n\n<li>Under <strong>Provisioning<\/strong>, set <strong>Provisioning Status = On<\/strong>, configure the Databricks <strong>SCIM endpoint + token<\/strong>, and <strong>Save<\/strong>.<\/li>\n\n\n\n<li>Add <strong>Users\/Groups<\/strong> to the enterprise app (assignments). Entra will sync them on its schedule.<\/li>\n\n\n\n<li>(Recommended) In <strong>Properties<\/strong>, set <strong>Assignment required = No<\/strong> if you want all IdP users to sign in to the Databricks account. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/admin\/users-groups\/scim\/aad?utm_source=chatgpt.com\">Microsoft Learn<\/a>, <a href=\"https:\/\/docs.databricks.com\/aws\/en\/admin\/users-groups\/scim\/enable-all-users?utm_source=chatgpt.com\">Databricks Documentation<\/a>)<\/li>\n<\/ol>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>If you previously synced identities to <strong>individual workspaces<\/strong>, disable those connectors when enabling <strong>account-level SCIM<\/strong>. (<a href=\"https:\/\/docs.databricks.com\/aws\/en\/admin\/users-groups\/scim\/aad?utm_source=chatgpt.com\">Databricks Documentation<\/a>)<\/p>\n<\/blockquote>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h1 class=\"wp-block-heading\">Create and Use <strong>Groups<\/strong> in Databricks<\/h1>\n\n\n\n<p><strong>Why groups?<\/strong> Grant permissions to a <strong>group<\/strong> (e.g., <code>data-analysts<\/code>, <code>data-engineers<\/code>) instead of individual users. This simplifies workspace access, persona\/entitlements, and data permissions in Unity Catalog. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/admin\/users-groups\/?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/p>\n\n\n\n<p><strong>Create groups<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>If you use <strong>SCIM<\/strong>, create and manage groups in Entra and let SCIM sync them to Databricks.<\/li>\n\n\n\n<li>If manual: Account console \u2192 <strong>User management \u2192 Groups \u2192 Add group<\/strong>; add members. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/admin\/users-groups\/?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/li>\n<\/ul>\n\n\n\n<p><strong>Use cases<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>data-analysts<\/code>: Databricks <strong>SQL<\/strong> persona only.<\/li>\n\n\n\n<li><code>data-engineers<\/code>: <strong>Workspace<\/strong> persona (DE\/ML), plus <strong>Cluster create<\/strong> ability.<\/li>\n\n\n\n<li><code>platform-admins<\/code>: Workspace admins.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h1 class=\"wp-block-heading\">Assign users (or groups) to a <strong>Workspace<\/strong><\/h1>\n\n\n\n<p>From the <strong>Account console<\/strong>:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Workspaces<\/strong> \u2192 choose a workspace \u2192 <strong>Permissions<\/strong>.<\/li>\n\n\n\n<li><strong>Add permissions<\/strong> \u2192 search your <strong>group<\/strong> (preferred) or individual user \u2192 assign <strong>User<\/strong> or <strong>Workspace admin<\/strong> \u2192 <strong>Save<\/strong>. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/admin\/users-groups\/users?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/li>\n<\/ol>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>Assign <strong>groups<\/strong> instead of individual users so new team members inherit access automatically.<\/p>\n<\/blockquote>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h1 class=\"wp-block-heading\">Workspace-level access or <strong>Persona<\/strong> in Databricks (Entitlements)<\/h1>\n\n\n\n<p>Within each <strong>workspace<\/strong>, a principal needs the right <strong>entitlements<\/strong> (\u201cpersonas\u201d):<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Workspace access<\/strong>: access to Data Engineering \/ ML UI (core workspace).<\/li>\n\n\n\n<li><strong>Databricks SQL access<\/strong>: access to SQL persona (warehouses, dashboards).<\/li>\n\n\n\n<li><strong>Allow cluster creation<\/strong>: permission to create clusters (DE\/ML persona).<br>Entitlements are available in <strong>Premium<\/strong> or above and managed by a <strong>workspace admin<\/strong>. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/security\/auth\/entitlements?utm_source=chatgpt.com\">Microsoft Learn<\/a>, <a href=\"https:\/\/docs.databricks.com\/aws\/en\/security\/auth\/entitlements?utm_source=chatgpt.com\">Databricks Documentation<\/a>)<\/li>\n<\/ul>\n\n\n\n<p><strong>Where to set (workspace)<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Workspace <strong>Settings<\/strong> \u2192 <strong>Identity and access<\/strong> \u2192 <strong>Manage<\/strong> (Users\/Groups\/Service principals) \u2192 <strong>Entitlements<\/strong> \u2192 toggle <strong>Workspace access<\/strong>, <strong>Databricks SQL access<\/strong>, <strong>Allow cluster creation<\/strong>. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/security\/auth\/entitlements?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/li>\n<\/ul>\n\n\n\n<p><strong>Examples<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Data Analyst group<\/strong>: enable <strong>Databricks SQL access<\/strong>, disable <strong>Workspace access<\/strong> and <strong>Allow cluster creation<\/strong> \u2192 gives just SQL persona (warehouses), no clusters.<\/li>\n\n\n\n<li><strong>Data Engineer group<\/strong>: enable <strong>Workspace access<\/strong> and <strong>Allow cluster creation<\/strong>; optionally disable <strong>Databricks SQL access<\/strong> if they don\u2019t need warehouses. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/security\/auth\/entitlements?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h1 class=\"wp-block-heading\"> How to create a <strong>Service Principal<\/strong> in Microsoft Azure<\/h1>\n\n\n\n<p>A <strong>service principal<\/strong> (SP) is a non-human identity you use for <strong>automation<\/strong> (Jobs, pipelines, CI\/CD, API\/CLI). You can create:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>A Databricks-managed SP in the <strong>Account console<\/strong>, or<\/li>\n\n\n\n<li>A Microsoft <strong>Entra ID<\/strong> app registration (recommended for Azure automation outside Databricks).<\/li>\n<\/ul>\n\n\n\n<p><strong>Create Entra service principal (app registration)<\/strong><\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Entra ID \u2192 <strong>App registrations<\/strong> \u2192 <strong>New registration<\/strong> \u2192 Name (e.g., <code>etl-automation-sp<\/code>) \u2192 <strong>Register<\/strong>.<\/li>\n\n\n\n<li>Copy <strong>Application (client) ID<\/strong> and <strong>Directory (tenant) ID<\/strong>.<\/li>\n\n\n\n<li>Create a <strong>client secret<\/strong> under <strong>Certificates &amp; secrets<\/strong>; copy the secret <strong>value<\/strong>. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/entra\/identity-platform\/quickstart-register-app?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/li>\n<\/ol>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>You\u2019ll use these when granting the SP access to Databricks and other Azure resources.<\/p>\n<\/blockquote>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h1 class=\"wp-block-heading\">Add Service Principal to the <strong>Databricks Workspace<\/strong><\/h1>\n\n\n\n<p><strong>At the account level<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Account console \u2192 <strong>User management \u2192 Service principals \u2192 Add<\/strong>.\n<ul class=\"wp-block-list\">\n<li>Choose <strong>Microsoft Entra ID managed<\/strong> and paste the <strong>Application (client) ID<\/strong>, or<\/li>\n\n\n\n<li>Create a <strong>Databricks-managed<\/strong> service principal. (<a href=\"https:\/\/docs.databricks.com\/aws\/en\/admin\/users-groups\/manage-service-principals?utm_source=chatgpt.com\">Databricks Documentation<\/a>)<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<p><strong>Assign the SP to workspaces<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Account console \u2192 <strong>Workspaces \u2192 [Workspace] \u2192 Permissions \u2192 Add permissions<\/strong> \u2192 select the SP \u2192 choose <strong>User<\/strong> (or <strong>Workspace admin<\/strong>, rarely) \u2192 <strong>Save<\/strong>.<\/li>\n\n\n\n<li>In the workspace, a workspace admin can set SP <strong>entitlements<\/strong> and who can <strong>manage\/use<\/strong> the SP (Permissions tab on the SP). (<a href=\"https:\/\/docs.databricks.com\/aws\/en\/admin\/users-groups\/service-principals?utm_source=chatgpt.com\">Databricks Documentation<\/a>)<\/li>\n<\/ul>\n\n\n\n<p><strong>Use cases<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Run as<\/strong> a service principal for <strong>Jobs\/Workflows<\/strong> to avoid dependency on human accounts.<\/li>\n\n\n\n<li>Use OAuth (M2M) or PATs scoped to the SP for API\/CLI automations. (<a href=\"https:\/\/docs.databricks.com\/aws\/en\/dev-tools\/auth\/oauth-m2m?utm_source=chatgpt.com\">Databricks Documentation<\/a>)<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Handy snippets &amp; checklists<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Databricks CLI (optional) quick checks<\/h3>\n\n\n\n<pre class=\"wp-block-code\"><code># List account\/workspace scopes or principals (run in the right context\/profile)\ndatabricks secrets list-scopes\ndatabricks groups list        # list workspace groups (if using workspace API)\ndatabricks service-principals list\n<\/code><\/pre>\n\n\n\n<p>(Use the <strong>Account<\/strong> vs <strong>Workspace<\/strong> CLI context appropriately.)<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Who can manage what?<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Account admin<\/strong>: add users\/groups\/SPs to the <strong>account<\/strong>, assign to workspaces, and configure identity federation\/SCIM. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/admin\/users-groups\/users?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/li>\n\n\n\n<li><strong>Workspace admin<\/strong>: grant <strong>entitlements<\/strong> (personas), manage local settings, optionally make a principal a <strong>workspace admin<\/strong>. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/security\/auth\/entitlements?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Provision via SCIM<\/strong> from Entra ID; manage group membership in the IdP. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/admin\/users-groups\/scim\/?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/li>\n\n\n\n<li>Grant access to <strong>groups<\/strong>, not individuals. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/admin\/users-groups\/?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/li>\n\n\n\n<li>Keep <strong>Workspace admin<\/strong> role minimal; use personas (entitlements) tailored to job function. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/security\/auth\/entitlements?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/li>\n\n\n\n<li>For UC permissions, always ensure principals exist at the <strong>account<\/strong> level. (<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/databricks\/data-governance\/unity-catalog\/best-practices?utm_source=chatgpt.com\">Microsoft Learn<\/a>)<\/li>\n\n\n\n<li>Use <strong>service principals<\/strong> for jobs\/automation (\u201cRun as\u201d SP), not human users. (<a href=\"https:\/\/docs.databricks.com\/aws\/en\/admin\/users-groups\/service-principals?utm_source=chatgpt.com\">Databricks Documentation<\/a>)<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Introduction In Databricks, identities (users, groups, service principals) live at the account level and can be assigned to one or more workspaces. For Unity Catalog (UC), principals&#8230; <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-840","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/840","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/comments?post=840"}],"version-history":[{"count":1,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/840\/revisions"}],"predecessor-version":[{"id":841,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/840\/revisions\/841"}],"wp:attachment":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/media?parent=840"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/categories?post=840"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/tags?post=840"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}