Security overview
Lakemine runs inside your own Databricks workspace. This page summarizes where your data stays, what we receive, and what access the installer requests.
In one line. Your table data, queries, and results never leave your Databricks account. We receive install metadata and aggregate usage figures, listed in What is Lakemine.
How your data is handled
- All connections use HTTPS. Every connection between the installer, your Databricks workspace, and Lakemine is encrypted in transit.
- The product runs in your workspace. The analysis job runs on your own Databricks compute and reads your usage data in place. Table data, queries, and results stay in your Databricks account.
- Results are written to your catalog. Findings are stored in a schema in your own catalog, under your own governance and retention. They are not sent to us.
- What we receive is limited and listed. Install metadata to register your installs, and aggregate usage figures for licensing and metering. We receive no table data, no query content, and no Databricks user identities. The full list is in What is Lakemine.
How the pieces connect
Everything in the middle box stays inside your Databricks account. What reaches Lakemine is listed in What is Lakemine.
What gets deployed to your workspace
- Scan job
lakemine-scan: runs the scan on the schedule you choose. - Results schema
<catalog>.lakemine, in a catalog you choose (the schema name is editable): holds the findings tables. - Dashboard “Lakemine Cost Optimization Dashboard”: shows the findings to your viewers group.
- Service principal
lakemine-sp: the identity the scan job runs as. - SQL warehouse
lakemine-wh(optional): created only if you choose a dedicated one. - Viewers group
lakemine-viewers(optional): created only if you choose a dedicated one.
Permissions
The installer asks for access in two stages, and shows you each permission with a plain-language reason before it applies anything. It checks what is already in place first and only adds what is missing.
During setup: the installer
- How you sign in. You connect to Databricks with OAuth browser sign-in (SSO), once for your account and once for the hub workspace you choose. Your Databricks credentials are never sent to us.
- Why it is needed. To create the pieces Lakemine runs on: a results schema, a scheduled job, a dashboard, a secret scope, and a dedicated service identity.
- Write scope. Write access is used once, during setup, to create those resources.
After setup: the scheduled job
The job runs under its own dedicated service principal (lakemine-sp), not yours.
The scan’s service principal is granted exactly these:
| Grant | On | Why |
|---|---|---|
| USE CATALOG | The system catalog | Reach the system schemas. Reading a table needs USE CATALOG on its parent. |
| USE SCHEMA, SELECT | system.billing, system.compute, system.lakeflow, system.query, system.access | Read usage, compute, job and query data in place. system.access supplies workspace names only. |
| USE CATALOG | The results catalog you choose | Reach the results schema. |
| USE SCHEMA, SELECT, MODIFY, CREATE TABLE, MANAGE | The results schema | Write and replace the findings snapshot. MANAGE is included because the scan replaces its tables on every run, and replacing a table owned by an earlier run identity needs MANAGE on the schema. |
| CAN USE | The SQL warehouse you choose | Run the scan’s queries. |
| WRITE | The lakemine secret scope | Read the subscription key and the metering credential, and store the rotated metering credential back after each use. |
workspace-access and databricks-sql-access entitlements | The service principal itself | Use the workspace and open a SQL warehouse connection. The catalog grants and the warehouse permission do not cover these. |
| CAN_MANAGE | The install folder, /Workspace/Applications/lakemine | The admin who deploys owns the bundle files, so the job running as the service principal needs this to read and install the compiled wheel. |
Separately, the person who runs the installer is given the User role on the service principal, which Databricks requires in order to bind it as the job’s run identity.
Each grant is checked first and applied only if it is missing. If the installer cannot
apply one, such as the system grants when you are not a metastore admin, it prints the
exact statement for someone who can. The scan also keeps a small table,
_lakemine_activation, in the results schema. It records the last subscription check,
including a hash of the key rather than the key itself, which is what lets a scan
continue for 7 days if the portal cannot be reached.
Read access to usage system tables is granted by Databricks across your whole account, not per workspace. Write access to your data is limited to the results schema. The other write-type grants cover only Lakemine’s own secret scope and install folder.
Removing Lakemine
Uninstalling removes the job, dashboard, results schema, secret scope, and service identity that the installer created.