diff --git a/dev/breeze/README.md b/dev/breeze/README.md index d0e7244a3ada9..f7cc35c1d59cb 100644 --- a/dev/breeze/README.md +++ b/dev/breeze/README.md @@ -46,6 +46,15 @@ worktree's sources. Because the shim is a real file on `PATH`, subprocesses (pre hooks, CI scripts, dev tools) see it just like a `uv tool`-installed binary. See [ADR 0017](doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md) for the rationale. +When invoked from outside any Airflow worktree — for example from an SVN release checkout +(`asf-dist`) during a provider release — the shim falls back to, in order: the worktree +pointed at by `$AIRFLOW_REPO_ROOT` (which the release docs export to the repo root, so breeze +resolves the same way across every release process), then the `dev/breeze` of the worktree it +was installed from (baked in at install time). This keeps release commands such as +`breeze release-management clean-old-provider-artifacts --directory ` working +from the SVN tree. The fallbacks never override a real worktree, so per-worktree isolation is +preserved wherever it matters. + The `scripts/tools/setup_breeze` script installs the shim for you. If you previously installed breeze globally via `uv tool install -e ./dev/breeze` or `pipx install -e ./dev/breeze`, remove that install first — both write to `~/.local/bin/breeze` and would conflict: @@ -60,25 +69,37 @@ To install the shim manually, write this file to `~/.local/bin/breeze` and `chmo #!/usr/bin/env bash # Apache Airflow breeze shim — managed by scripts/tools/setup_breeze (ADR 0017). set -e -repo_root=$(git rev-parse --show-toplevel 2>/dev/null) || { - echo "breeze: not inside a git repository — cd into an Airflow worktree first" >&2 - exit 1 -} -if [ ! -d "${repo_root}/dev/breeze" ]; then - echo "breeze: ${repo_root} is not an Airflow worktree (no dev/breeze)" >&2 +# Install-time fallback: the Airflow sources 'scripts/tools/setup_breeze' was run +# from. Used only when the current directory is not an Airflow worktree. +fallback_root="/abs/path/to/airflow" # baked in by setup_breeze (= the worktree it ran from) +repo_root=$(git rev-parse --show-toplevel 2>/dev/null) || repo_root="" +if [ -n "${repo_root}" ] && [ -d "${repo_root}/dev/breeze" ]; then + breeze_root="${repo_root}" +elif [ -n "${AIRFLOW_REPO_ROOT:-}" ] && [ -d "${AIRFLOW_REPO_ROOT}/dev/breeze" ]; then + breeze_root="${AIRFLOW_REPO_ROOT}" +elif [ -d "${fallback_root}/dev/breeze" ]; then + breeze_root="${fallback_root}" +else + echo "breeze: not inside an Airflow worktree, AIRFLOW_REPO_ROOT is unset or not an Airflow worktree, and the install-time fallback '${fallback_root}/dev/breeze' is missing — re-run scripts/tools/setup_breeze" >&2 exit 1 fi -exec env AIRFLOW_ROOT_PATH="${repo_root}" SKIP_BREEZE_SELF_UPGRADE_CHECK=1 \ - uvx --from "${repo_root}/dev/breeze" --quiet breeze "$@" +exec env AIRFLOW_ROOT_PATH="${breeze_root}" SKIP_BREEZE_SELF_UPGRADE_CHECK=1 \ + uvx --from "${breeze_root}/dev/breeze" --quiet breeze "$@" ``` -Then `breeze` invoked from any Airflow checkout uses that checkout's source. The first call in -a fresh worktree pays a one-time `uvx` resolve/install; subsequent calls hit the cache. +Then `breeze` invoked from any Airflow checkout uses that checkout's source, and from +anywhere else it uses `$AIRFLOW_REPO_ROOT` or the baked-in fallback. The first call in a +fresh worktree pays a one-time `uvx` resolve/install; subsequent calls hit the cache. The legacy global-install path (`uv tool install -e ./dev/breeze --force` or `pipx install -e ./dev/breeze --force`) still works for users who explicitly want a single shared install, but it is no longer the recommended approach. +The shim carries a `# breeze-shim-version: N` marker. On startup breeze compares it with the +version the current sources would install and, if your installed shim is older (or you are still +on a legacy global install), prints a warning telling you to re-run `scripts/tools/setup_breeze` +(after uninstalling the global install, if any). + You can read more about Breeze in the [documentation](https://github.com/apache/airflow/blob/main/dev/breeze/doc/README.rst) This README file contains automatically generated hash of the `pyproject.toml` files that were diff --git a/dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md b/dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md index de24c41da05ac..017fd6d1d1854 100644 --- a/dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md +++ b/dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md @@ -90,17 +90,32 @@ The recommended way to run breeze is via a small **shim script** at # Runs breeze from the dev/breeze folder of the current git worktree via 'uvx', # so each worktree (e.g. parallel agentic runs) gets its own ephemerally-installed # breeze tied to that worktree's source. +# +# Resolution order for the Airflow sources breeze runs from: +# 1. the current git worktree (per-worktree isolation — see above); +# 2. $AIRFLOW_REPO_ROOT, if exported and pointing at an Airflow worktree — the +# release docs export this, so breeze resolves the same way across every +# release process regardless of where the shim was installed from; +# 3. the install-time fallback baked in below (the worktree setup_breeze ran from). +# Steps 2 and 3 apply only when the current directory is not an Airflow worktree, +# so the fallbacks never override a real worktree and isolation is preserved. set -e -repo_root=$(git rev-parse --show-toplevel 2>/dev/null) || { - echo "breeze: not inside a git repository — cd into an Airflow worktree first" >&2 - exit 1 -} -if [ ! -d "${repo_root}/dev/breeze" ]; then - echo "breeze: ${repo_root} is not an Airflow worktree (no dev/breeze)" >&2 +# Install-time fallback: the Airflow sources 'scripts/tools/setup_breeze' was run +# from. Used only when the current directory is not an Airflow worktree. +fallback_root="/abs/path/to/airflow" # baked in by setup_breeze (= AIRFLOW_SOURCES) +repo_root=$(git rev-parse --show-toplevel 2>/dev/null) || repo_root="" +if [ -n "${repo_root}" ] && [ -d "${repo_root}/dev/breeze" ]; then + breeze_root="${repo_root}" +elif [ -n "${AIRFLOW_REPO_ROOT:-}" ] && [ -d "${AIRFLOW_REPO_ROOT}/dev/breeze" ]; then + breeze_root="${AIRFLOW_REPO_ROOT}" +elif [ -d "${fallback_root}/dev/breeze" ]; then + breeze_root="${fallback_root}" +else + echo "breeze: not inside an Airflow worktree, AIRFLOW_REPO_ROOT is unset or not an Airflow worktree, and the install-time fallback '${fallback_root}/dev/breeze' is missing — re-run scripts/tools/setup_breeze" >&2 exit 1 fi -exec env AIRFLOW_ROOT_PATH="${repo_root}" SKIP_BREEZE_SELF_UPGRADE_CHECK=1 \ - uvx --from "${repo_root}/dev/breeze" --quiet breeze "$@" +exec env AIRFLOW_ROOT_PATH="${breeze_root}" SKIP_BREEZE_SELF_UPGRADE_CHECK=1 \ + uvx --from "${breeze_root}/dev/breeze" --quiet breeze "$@" ``` ``scripts/tools/setup_breeze`` writes this file (replacing any previous @@ -150,6 +165,13 @@ behaviour, but they are no longer the recommended path. * **Subprocess-safe.** The shim is a real binary on ``PATH``, so anything that shells out to ``breeze`` — pre-commit hooks, CI helpers, dev scripts — resolves it exactly like a ``uv tool`` install did. +* **Self-detecting staleness.** The shim carries a ``# breeze-shim-version: N`` + marker that ``setup_breeze`` bumps whenever the shim body changes. On startup + breeze compares the installed shim's version against the version the current + sources would install and warns the user to re-run ``setup_breeze`` if the + installed shim is older (or predates versioning). The same startup check also + detects a leftover legacy global ``uv tool`` / ``pipx`` install and nudges the + user to migrate to the shim. **Costs** @@ -160,11 +182,19 @@ behaviour, but they are no longer the recommended path. runs ``git rev-parse`` and ``uvx`` for every invocation. Negligible at the command line, but noticeable inside tight loops or shell completion that re-invokes ``breeze`` many times. -* **Requires a git checkout.** ``breeze`` invoked outside a git tree errors - out with a clear message rather than running. This matches actual usage — - breeze is meaningless outside an Airflow source tree — but is a behavioural - change from the global install, which would silently run against whatever - tree it was last installed from. +* **Resolution is current-worktree-first, with two fallbacks.** ``breeze`` + invoked from inside an Airflow worktree runs that worktree's breeze. Invoked + from anywhere else (a non-Airflow git tree, or no git tree at all — e.g. an + ``asf-dist`` SVN release checkout), it falls back to, in order: the worktree + pointed at by ``$AIRFLOW_REPO_ROOT`` (which the release docs export to the + repo root, so breeze resolves the same way across every release process), then + the ``dev/breeze`` of the worktree ``setup_breeze`` was last run from, baked + into the shim at install time. This keeps release commands such as + ``breeze release-management clean-old-provider-artifacts --directory `` + working from the SVN tree. Only if the current worktree, ``$AIRFLOW_REPO_ROOT``, + and the baked-in fallback are all missing ``dev/breeze`` does the shim error + out with a clear message. The fallbacks never override a real worktree, so + per-worktree isolation is preserved wherever it matters. * **One-time migration.** Users who previously installed breeze with ``uv tool install`` need to ``uv tool uninstall apache-airflow-breeze`` before installing the shim, otherwise both write to ``~/.local/bin/breeze`` diff --git a/dev/breeze/doc/images/output_workflow-run_publish-docs.svg b/dev/breeze/doc/images/output_workflow-run_publish-docs.svg index 6a8e55584df3a..48033b138d1af 100644 --- a/dev/breeze/doc/images/output_workflow-run_publish-docs.svg +++ b/dev/breeze/doc/images/output_workflow-run_publish-docs.svg @@ -1,4 +1,4 @@ - +