Immutable folder support in DABs by andrewnester · Pull Request #5254 · databricks/cli

andrewnester · 2026-05-15T09:56:18Z

Changes

Added support for deploying bundles to immutable folders in the workspace

When experimental.immutable_folder: true is set, the CLI packages all bundle source files and built artifacts into a single zip archive and uploads it as a content-addressed, immutable snapshot. The snapshot path is derived from the content hash of the zip, so identical code always lands at the same location and re-deploying an unchanged bundle is a no-op at the storage layer.

This mode is currently supported with the direct deployment engine only (DATABRICKS_BUNDLE_ENGINE=direct).

Example databricks.yml:

bundle:
  name: my-bundle

experimental:
  immutable_folder: true

resources:
  jobs:
    my_job:
      name: my-job
      tasks:
        - task_key: main
          python_script_task:
            python_file: ./src/main.py

After bundle deploy, the job's python_file resolves to a path like:

/Workspace/Users/[UUID]/.snapshots///src/files/src/main.py

Tests

Added an acceptance tests

eng-dev-ecosystem-bot · 2026-05-28T11:17:06Z

Integration test report

Commit: 60e41d3

Run: 28107757408

	Env	❌FAIL	🔄flaky	💚RECOVERED	🙈SKIP	✅pass	🙈skip	Time
💚	aws linux			7	13	244	1028	5:25
💚	aws windows			7	13	246	1026	6:28
💚	aws-ucws linux			7	13	334	944	6:54
💚	aws-ucws windows			7	13	336	942	7:23
❌	azure linux	3		1	15	244	1026	15:11
❌	azure windows	2	4		15	244	1024	16:02
❌	azure-ucws linux	3		1	15	336	940	11:11
🔄	azure-ucws windows		4		15	338	938	9:35
💚	gcp linux			1	15	246	1028	4:48
💚	gcp windows			1	15	248	1026	5:30

25 interesting tests: 13 SKIP, 6 RECOVERED, 3 flaky, 3 FAIL

	Test Name	aws linux	aws windows	aws-ucws linux	aws-ucws windows	azure linux	azure windows	azure-ucws linux	azure-ucws windows	gcp linux	gcp windows
🔄	TestAccept	💚R	💚R	💚R	💚R	💚R	🔄f	💚R	🔄f	💚R	💚R
🔄	TestAccept/bundle/generate/alert	✅p	✅p	✅p	✅p	✅p	🔄f	✅p	🔄f	✅p	✅p
🔄	TestAccept/bundle/generate/alert/DATABRICKS_BUNDLE_ENGINE=terraform	✅p	✅p	✅p	✅p	✅p	🔄f	✅p	🔄f	✅p	✅p
🙈	TestAccept/bundle/invariant/no_drift	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/permissions	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
💚	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions	💚R	💚R	💚R	💚R	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
💚	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct	💚R	💚R	💚R	💚R
💚	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform	💚R	💚R	💚R	💚R
💚	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions	💚R	💚R	💚R	💚R	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
💚	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct	💚R	💚R	💚R	💚R
💚	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform	💚R	💚R	💚R	💚R
🙈	TestAccept/bundle/resources/postgres_branches/basic	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/recreate	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/replace_existing	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/update_protected	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/without_branch_id	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_endpoints/basic	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_projects/update_display_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/synced_database_tables/basic	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/ssh/connection	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
❌	TestFetchRepositoryInfoAPI_FromRepo	✅p	✅p	✅p	✅p	❌F	❌F	❌F	🔄f	✅p	✅p
❌	TestFetchRepositoryInfoAPI_FromRepo/root	✅p	✅p	✅p	✅p	❌F	❌F	❌F	✅p	✅p	✅p
❌	TestFetchRepositoryInfoAPI_FromRepo/subdir	✅p	✅p	✅p	✅p	❌F	🔄f	❌F	✅p	✅p	✅p

Top 22 slowest tests (at least 2 minutes):

duration	env	testname
3:33	aws windows	TestAccept
3:28	azure windows	TestAccept/bundle/generate/auto-bind/DATABRICKS_BUNDLE_ENGINE=terraform
3:23	azure windows	TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=terraform
3:18	azure windows	TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=direct
3:07	aws-ucws windows	TestAccept
3:05	gcp windows	TestAccept
2:59	azure windows	TestFilerReadWrite/workspace_files
2:50	azure windows	TestAccept/bundle/deployment/bind/job/generate-and-bind/DATABRICKS_BUNDLE_ENGINE=terraform
2:45	azure windows	TestAccept/bundle/destroy/jobs-and-pipeline/DATABRICKS_BUNDLE_ENGINE=terraform
2:45	azure windows	TestAccept/bundle/deployment/bind/job/generate-and-bind/DATABRICKS_BUNDLE_ENGINE=direct
2:39	azure linux	TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=direct
2:30	azure linux	TestAccept/bundle/run_as/job_default/DATABRICKS_BUNDLE_ENGINE=direct
2:30	azure-ucws windows	TestAccept/bundle/deployment/bind/job/generate-and-bind/DATABRICKS_BUNDLE_ENGINE=terraform
2:23	azure linux	TestFilerReadWrite/workspace_files_extensions
2:22	azure windows	TestAccept/bundle/run_as/job_default/DATABRICKS_BUNDLE_ENGINE=direct
2:20	azure linux	TestFilerWorkspaceFilesExtensionsReadDir
2:20	azure linux	TestAccept/bundle/generate/auto-bind/DATABRICKS_BUNDLE_ENGINE=terraform
2:11	azure linux	TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=terraform
2:09	azure-ucws windows	TestAccept/bundle/generate/auto-bind/DATABRICKS_BUNDLE_ENGINE=terraform
2:09	azure windows	TestLock
2:04	azure linux	TestAccept/bundle/deployment/bind/job/generate-and-bind/DATABRICKS_BUNDLE_ENGINE=direct
2:01	azure windows	TestImportDirWithOverwriteFlag

github-actions · 2026-06-01T11:35:34Z

Approval status: pending

`/acceptance/bundle/` - needs approval

29 files changed
Suggested: @denik
Also eligible: @pietern, @janniklasrose, @shreyas-goenka, @lennartkats-db, @anton-107

`/bundle/` - needs approval

23 files changed
Suggested: @denik
Also eligible: @pietern, @janniklasrose, @shreyas-goenka, @lennartkats-db, @anton-107

`/libs/sync/` - needs approval

Files: libs/sync/sync.go
Suggested: @simonfaltum
Also eligible: @tanmay-db, @Divyansh-db, @renaudhartert-db, @parthban-db, @hectorcast-db, @tejaskochar-db, @mihaimitrea-db, @chrisst, @rauchy

General files (require maintainer)

Files: acceptance/bin/print_requests.py, libs/testserver/handlers.go
Based on git history:

@denik -- recent work in bundle/direct/, bundle/phases/, bundle/config/mutator/

_{Any maintainer (@anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db) can approve all areas.

See OWNERS for ownership rules.}

shreyas-goenka

Minor comments - other than the bit where we use metadata.json

shreyas-goenka · 2026-06-02T16:30:01Z

@@ -15,6 +15,9 @@ type Bundle struct {

 type Workspace struct {
 	FilePath string `json:"file_path"`
+	// SnapshotPath is the workspace path of the immutable snapshot uploaded
+	// during deployment. Only populated for bundles with bundle.immutable = true.
+	SnapshotPath string `json:"snapshot_path,omitempty"`


Will the UI use the immutable snapshot path? In that case we'll need to add it to DMS as well.

pietern · 2026-06-24T08:15:46Z

+		// Allows running the full test suite against the immutable folder code path without
+		// modifying any databricks.yml files.
+		mutator.OverrideImmutableFolder(),
+


Is this test only?

Yes, this is only used for test

pietern · 2026-06-24T08:26:28Z

+		})
+	}
+	return acl
+}


Nit: can live in a different file.

Immutable folder support in DABs

b685ec2

andrewnester requested a review from pietern May 15, 2026 09:56

andrewnester temporarily deployed to test-trigger-is May 15, 2026 09:56 — with GitHub Actions Inactive

andrewnester added 2 commits May 28, 2026 11:03

remove unused snapshot path method

a429b26

added an acceptance test

e7c1968

andrewnester temporarily deployed to test-trigger-is May 28, 2026 10:01 — with GitHub Actions Inactive

fix for notebook import

67914d0

andrewnester temporarily deployed to test-trigger-is May 28, 2026 10:19 — with GitHub Actions Inactive

removed unused function

549492a

andrewnester temporarily deployed to test-trigger-is May 28, 2026 10:41 — with GitHub Actions Inactive

andrewnester marked this pull request as ready for review June 1, 2026 11:34

fix schema + unit test

aedfdb0

andrewnester temporarily deployed to test-trigger-is June 1, 2026 11:39 — with GitHub Actions Inactive

andrewnester requested a review from denik June 2, 2026 10:01

shreyas-goenka reviewed Jun 2, 2026

View reviewed changes

andrewnester and others added 2 commits June 9, 2026 12:49

use immutable_folder config

eddec61

Merge branch 'main' into demo-immutable

27a6b02

andrewnester temporarily deployed to test-trigger-is June 9, 2026 10:51 — with GitHub Actions Inactive

remove merge conflict

4a9bcd9

andrewnester temporarily deployed to test-trigger-is June 9, 2026 10:53 — with GitHub Actions Inactive

fix empty artifact path + tests

ebd26ea

andrewnester requested a review from pietern June 23, 2026 11:00

fixed app deploying

b9f67f7

andrewnester temporarily deployed to test-trigger-is June 23, 2026 11:48 — with GitHub Actions Inactive

use snapshot_path veriable prefixes and move to experimetal

01136f5

andrewnester temporarily deployed to test-trigger-is June 23, 2026 15:49 — with GitHub Actions Inactive

pietern reviewed Jun 24, 2026

View reviewed changes

Merge branch 'main' into demo-immutable

6e9aa77

andrewnester temporarily deployed to test-trigger-is June 24, 2026 09:33 — with GitHub Actions Inactive

addressed feedback

e26f085

andrewnester temporarily deployed to test-trigger-is June 24, 2026 11:11 — with GitHub Actions Inactive

simplify and clean up

e2a5b2a

andrewnester temporarily deployed to test-trigger-is June 24, 2026 13:27 — with GitHub Actions Inactive

update out.test.toml

6f3f077

andrewnester temporarily deployed to test-trigger-is June 24, 2026 13:30 — with GitHub Actions Inactive

comment

5cefc2f

andrewnester temporarily deployed to test-trigger-is June 24, 2026 13:32 — with GitHub Actions Inactive

andrewnester requested a review from pietern June 24, 2026 13:32

fix test on windows

55acaf4

andrewnester temporarily deployed to test-trigger-is June 24, 2026 14:11 — with GitHub Actions Inactive

added client side validation for size of upload

60e41d3

andrewnester temporarily deployed to test-trigger-is June 24, 2026 14:56 — with GitHub Actions Inactive

Uh oh!

Conversation

andrewnester commented May 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Tests

Uh oh!

eng-dev-ecosystem-bot commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Integration test report

Uh oh!

github-actions Bot commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Approval status: pending

/acceptance/bundle/ - needs approval

/bundle/ - needs approval

/libs/sync/ - needs approval

General files (require maintainer)

Uh oh!

shreyas-goenka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

shreyas-goenka Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pietern Jun 24, 2026

Choose a reason for hiding this comment

Uh oh!

andrewnester Jun 24, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pietern Jun 24, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

andrewnester commented May 15, 2026 •

edited

Loading

eng-dev-ecosystem-bot commented May 28, 2026 •

edited

Loading

github-actions Bot commented Jun 1, 2026 •

edited

Loading

`/acceptance/bundle/` - needs approval

`/bundle/` - needs approval

`/libs/sync/` - needs approval