Skip to content

fix: reduce galaxy requirements check noise and de-duplicate path checks#3901

Open
ksrp1984 wants to merge 1 commit into
semaphoreui:developfrom
ksrp1984:fix/galaxy-requirements-noise
Open

fix: reduce galaxy requirements check noise and de-duplicate path checks#3901
ksrp1984 wants to merge 1 commit into
semaphoreui:developfrom
ksrp1984:fix/galaxy-requirements-noise

Conversation

@ksrp1984
Copy link
Copy Markdown

Description

Problem

On every Ansible task run, Semaphore probes a fixed set of candidate
requirements.yml locations and logs a line for each one that is missing:

No .../playbooks/collections/requirements.yml file found. Skip galaxy install process.
No .../playbooks/requirements.yml file found. Skip galaxy install process.
No .../collections/requirements.yml file found. Skip galaxy install process.
No .../requirements.yml file found. Skip galaxy install process.
No .../playbooks/roles/requirements.yml file found. Skip galaxy install process.
No .../playbooks/requirements.yml file found. Skip galaxy install process.
No .../roles/requirements.yml file found. Skip galaxy install process.
No .../requirements.yml file found. Skip galaxy install process.

That is up to 8 lines per run, and some are exact duplicates (the shared
<dir>/requirements.yml is probed once for roles and once for collections).
When collections/roles are managed globally and no requirements files are
expected in the repo, this is pure noise that hides the meaningful task output.

Related: #2982, #3111.

Behavior change — full algorithm

Requirements resolution is reworked into a collect → check → install flow.
Candidate paths are classified into two kinds, each with its own rule.

Search locations

Two base directories are searched:

  • the playbook directory (GetPlaybookDir())
  • the repository root (getRepoPath())

If these two resolve to the same directory (e.g. the playbook lives at the repo
root), it is searched once — duplicate base directories are removed up front.

Kind 1 — type-specific subdirectory files

<base>/roles/requirements.yml (roles only)
<base>/collections/requirements.yml (collections only)

Situation Action
roles/ (or collections/) directory does not exist Skip silently — nothing to install, no log
Directory exists, but requirements.yml is missing Log one warning (likely a misconfiguration)
Directory exists and requirements.yml is present Queue the file for installation

Kind 2 — shared root file

<base>/requirements.yml

This file may contain both roles: and collections: sections (the
canonical Ansible layout), so when present it is installed twice — once with
ansible-galaxy role install and once with ansible-galaxy collection install.

Situation Action
At least one shared requirements.yml exists (any base dir) Use the existing one(s); say nothing about missing locations
No shared requirements.yml exists in any base dir Log a single message listing all paths that were searched

What gets de-duplicated vs. what is intentionally kept

  • Removed: repeated checks/log lines for the same physical path, and the
    duplicate base directory when playbook dir == repo root.
  • Kept (by design): a present shared requirements.yml is still processed
    for both roles and collections — this is not a duplicate, the two
    ansible-galaxy subcommands read different sections of the same file.

Before / after (example: repo with no requirements files, playbook at repo root)

Before — 8 lines (with duplicates):

No <root>/collections/requirements.yml file found. Skip galaxy install process.
No <root>/requirements.yml file found. Skip galaxy install process.
No <root>/collections/requirements.yml file found. Skip galaxy install process.
No <root>/requirements.yml file found. Skip galaxy install process.
No <root>/roles/requirements.yml file found. Skip galaxy install process.
No <root>/requirements.yml file found. Skip galaxy install process.
No <root>/roles/requirements.yml file found. Skip galaxy install process.
No <root>/requirements.yml file found. Skip galaxy install process.

After — 1 line:

No requirements.yml found. Skip galaxy install process. Searched:
  <root>/requirements.yml

(the roles/ and collections/ subdirectories don't exist, so they are skipped silently)

Implementation notes

  • New resolveGalaxyRequirements() performs collect/check/logging and returns
    two slices of existing paths (roles, collections).
  • installGalaxyRequirementsFile() no longer does existence checks or
    "not found" logging — it is only ever called with files that exist. Its md5
    change-detection ("has no changes. Skip…") is unchanged.
  • installRolesRequirements() / installCollectionsRequirements() are kept and
    reused; they now take a pre-resolved []string of existing paths.
  • Public InstallRequirements() signature is unchanged.
  • No API changes (no Swagger/dredd impact).

Closes #2982
Closes #3111

Skip silently when roles/ or collections/ subdirectory doesn't exist.
Warn when subdirectory exists but has no requirements.yml.
Log a single message listing searched paths when no shared
requirements.yml is found, instead of one line per checked path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Question: Skip galaxy install process. Question: Not run galaxy install process when executing a Ansible task template?

1 participant