Access model in depth
A plain-language description of what Lakemine does inside your Databricks account, who needs which permissions, and why the access model is built the way it is.
For the short overview, see How Lakemine works.
For the step-by-step install, see Quickstart. For the exact permission statements, see the Connecting Databricks section.
The shape of it
Architecture diagram: the Electron installer signing in to Lakemine and to the
customer’s Databricks account, deploying the lakemine-scan job and the
“Databricks Cost Optimization” dashboard into the hub workspace, the job reading
system.* and writing the results schema, the dashboard reading the results
schema, and the scan job’s outbound calls to the Lakemine portal (subscription check,
token refresh, usage report).
Everything lands under one hub workspace you pick at install:
- A scheduled job,
lakemine-scan, which runs the compiled analysis wheel. - A dashboard, “Databricks Cost Optimization”, with two pages.
- The compiled wheel itself, under
/Workspace/Applications/lakemine/artifacts/. - A results schema in the catalog you chose, default schema name
lakemine, created by the scan on its first run.
The results schema holds seven small tables that the dashboard reads: run_summary,
opportunities, cost_by_workspace, workspaces, cost_by_day, cost_by_user, and
cost_by_tag. Every one of them has install_id as its first column. They are rebuilt
on each run. The dashboard queries these tables, never system.*, so opening it is
cheap.
Who can deploy it
Installing is a one-time administrative act. It needs three permissions, which one person often holds together.
| Permission | Why it is needed during install |
|---|---|
| Account admin | To sign in at the account level and list the workspaces, so you can choose where the appliance lives. |
| Workspace admin on the chosen hub | To create the appliance’s service principal, deploy the scheduled job and the dashboard, and give the appliance access to the SQL warehouse. |
| Metastore admin | To let the appliance read the Databricks system usage tables. Account admin and workspace admin do not grant this. It is a separate role in Unity Catalog. |
If the person running the installer is not a metastore admin, everything else still completes and the installer prints the exact commands for a metastore admin to run. Until those are applied the scheduled scan cannot read your usage data, so this is the one step that genuinely blocks results.
After install, the scheduled scan needs none of these permissions. Re-running the install wizard repeats the account admin step. Update on the installer’s Deploys screen needs no account-level sign-in, but you still need permission to deploy to that workspace. See Why re-deploying is safe.
Who can see the dashboard
Everyone in the viewers group chosen at install, and nobody else.
During install you pick a group. An existing one such as your BI team, your FinOps
group, or your workspace admins, or a dedicated lakemine-viewers group the installer
creates for the purpose. That group’s membership is the entire access control for the
cost data.
- Add someone to the group and they see the dashboard.
- Remove them and they do not.
- No database permissions, no SQL, no ticket to a data platform team.
Viewers need no access of their own to the result tables, to the system usage tables, or to the catalog. The next section explains how that works.
Two identities, clearly separated
Lakemine splits who installs it from who runs it.
You deploy it. The install is done under the admin’s own identity. That is what makes it auditable, and it is why the admin permissions above are needed.
A dedicated service principal runs it. Every scheduled scan from then on runs as a
machine identity created at install, named lakemine-scan, never as a person.
An earlier design ran the scan as whoever installed it. That tied a permanent, daily business process to one employee’s account. Their permissions changing, their leaving the company, or a colleague simply re-running the installer would silently break the scan or hand it different access. Splitting the two removes that dependency entirely.
The service principal owns the results and answers the dashboard’s questions
Because the scheduled job runs as the service principal:
- It reads the Databricks system usage tables, using the access granted at install.
- It writes and owns the result tables, so it can always read back what it wrote.
- It publishes the dashboard under its own credentials. Every chart you open runs its queries as the service principal, not as you.
That third point is what makes the access model simple. A viewer opening the dashboard is not querying your data warehouse under their own name. The appliance’s identity answers on their behalf. So a viewer needs no permission on anything: not the result tables, not the catalog, not the system usage tables.
The only question Databricks asks about a viewer is whether they are in the group.
Why re-deploying is safe
Because the appliance’s identity is fixed and independent of people, an upgrade or re-install by a different administrator changes nothing about how the appliance runs. The same service principal keeps the same access, keeps owning the same tables, and keeps answering the dashboard’s queries. Nothing re-points at the new deployer, and no viewer loses access.
This matters more than it sounds. It means the appliance survives staff changes, delegation, and routine upgrades without anyone re-granting anything.
What this buys you
- A single control point for access. One group membership list, owned by you, with no data-platform involvement to add or remove a viewer.
- No standing dependency on an individual. No key-person risk in a daily process.
- Least privilege for people. Viewers see cost findings without being granted access to underlying usage data they do not need.
- Auditability. The install is attributable to a named admin. Everything after it is attributable to one clearly named machine identity.
When it runs
- Daily at 06:00 UTC by default. You can choose every 12 hours, weekly on Mondays, or paused for manual runs only, at install time.
- Each run scans a trailing 30-day window by default, configurable between 7 and 90 days, across every workspace the run identity can see, in one pass.
- Each run overwrites the results snapshot. The dashboard always shows the latest scan.
Daily rather than hourly keeps the compute you pay for modest. The dashboard reads the
small results tables rather than system.*, so reading it costs almost nothing.
Credentials, and where they live
The install writes two values into a lakemine secret scope in your workspace: an
opaque subscription key and the metering credential. Both are readable by the scan
identity and workspace administrators.
Neither is a bundle variable or a job parameter. Those are stored in plaintext in the job spec, and anything an admin can edit there is not a credential.
What leaves your environment
Your usage data and findings do not. Lakemine receives two kinds of data, install metadata and aggregate usage figures. The full list, and what each call carries, is in What is Lakemine.
Restricted-egress and air-gapped workspaces
The appliance needs outbound access to the Lakemine portal. If your workspace blocks it:
- Metering simply skips. It is best effort and never fails a scan.
- The subscription check runs on its last successful answer for 7 days. Past that the scan stops and the job reports why, and your dashboard keeps its last snapshot.
A fully air-gapped install therefore needs an allowlist entry for the portal host. Talk to us before deploying into one.