Adding Offload Test into DXC build pipeline by joaosaffran · Pull Request #8584 · microsoft/DirectXShaderCompiler

joaosaffran · 2026-06-23T02:04:29Z

This patch adds a new stage into DXC build pipeline, it runs the offload-test agains the built binaries, it uses WARP for DXIL and lava-pipe for SPIRV builds.

Part of: microsoft/hlsl-specs#886

damyanp · 2026-06-24T22:24:05Z

+  variables:
+    LAVAPIPE_VERSION: '26.1.2'
+    WARP_VERSION: '1.0.19'
+    VS_DEV: 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvars64.bat'


We shouldn't hardcode this string. We should be able to reuse some of the logic in the other scripts to find this.

damyanp · 2026-06-24T22:30:15Z

+    - task: BatchScript@1
+      displayName: 'Set up MSVC environment'
+      inputs:
+        filename: '$(VS_DEV)'


Does hctstart.cmd actually do this for you? Might be a way to avoid hard coding the script path?

That would require cloning DXC repo in the offload-tests worker which I was avoiding doing... I've modified the script to find MSVC variables instead.

damyanp · 2026-06-24T22:32:41Z

      displayName: 'DXIL Compat Suite Tests (1.8 point release)'
      condition: succeededOrFailed()
-
+    - script: |


For PR builds, I'm wondering if we should only do this if the PR has a particular label on it? I don't think we'll want to gate all PRs on this since it'll add significant time to the builds.

alsepkow · 2026-06-24T22:58:07Z

+        $dst = "$(Build.ArtifactStagingDirectory)\dxc"
+        New-Item -ItemType Directory -Force -Path $dst | Out-Null
+        foreach ($f in @("dxc.exe","dxv.exe","dxcompiler.dll","dxil.dll")) {
+          if (Test-Path "$bin\$f") { Copy-Item "$bin\$f" $dst } else { Write-Warning "missing $f" }


Instead of logging a warning, should we fail if one of the files doesn't exist?

alsepkow · 2026-06-24T22:58:55Z

+  condition: succeededOrFailed()
+  variables:
+    LAVAPIPE_VERSION: '26.1.2'
+    WARP_VERSION: '1.0.19'


Are hard coded versions intentional? I worry about needing to consistently bump these versions.

joaosaffran · 2026-06-25T03:57:17Z

/azp run

azure-pipelines · 2026-06-25T03:57:25Z

Azure Pipelines successfully started running 1 pipeline(s).

damyanp · 2026-06-25T17:15:43Z


    - pwsh: |
-        $ver = "$(LAVAPIPE_VERSION)"
+        $ver = (Invoke-RestMethod "https://api.github.com/repos/pal1000/mesa-dist-win/releases/latest" -Headers @{ "User-Agent" = "azure-pipelines" }).tag_name


Actually, I think picking the version of WARP / lava pipe is something we want to be able to do manually so we can choose the correct last known good combination of these tools - and also get reproducible builds.

damyanp

I think this is basically good to go. It's unfortunate that we need to jump through hoops to deal with ADO pipeline integration not being as feature rich as github actions.

Icohedron · 2026-06-25T21:23:36Z

+    - task: UsePythonVersion@0
+      inputs:
+        versionSpec: '3.11'
+    - script: choco install ninja -y


We install ninja using chocolatey as opposed to winget?

In previous jobs, we used hctstart, which is only available inside DXC repo. I am avoiding cloning that in this stage of the pipeline. Not sure how hctstart gets ninja installed

Is Chocolatey a standard piece of software that's available on all the Windows VMs?
I was thinking WinGet would be more preferrable than Chocolatey, since Chocolatey is a third-party piece of software whereas WinGet is included with Windows.

Chocolatey is available as part of the 1ES VM images, which we use in this build

alsepkow

Took a look with @inbelic @Icohedron and added a few comments.

alsepkow · 2026-06-25T21:34:07Z

        testResultsFiles: '**/testresults.xunit.xml'
      condition: succeededOrFailed()
+
+- stage: OffloadGate


Is this stage neccessary? Seems wasteful to spin up a VM to check labels. I think we could just do that at the end of the build job instead? Or is there a reason you chose to do this in its own stage.

ADO doesn't have access to github labels, the way I came up to do that was to use github api. This is required to block the offload-test execution unless a PR opts in to it by explicitly setting the run-offload-tests in GitHub. This happen before the tests and in parallel with build stage, It must be before testing in order to block it

Why is it a requirement that the label checking is done in parallel with the build stage instead of as part of the build stage? The concern alex raised is about spinning an entire VM just to check labels and nothing else, when it could be merged with the build stage.

Adding the check to the previous stage would make the check run for all the builds we do. Which is not ideal, since it would hit the GitHub API multiple times. I personally prefer to spawn a new VM for such a check; in my tests, this didn't took long to do compared to being rate limited by github

alsepkow · 2026-06-25T21:39:53Z

+  dependsOn:
+  - Build
+  - OffloadGate
+  condition: and(succeededOrFailed(), eq(dependencies.OffloadGate.outputs['gate.decide.run'], 'true'))


Was the succeedOrFailed() intentional? Wondering if it would make more sense to have this as succeeded.
Is there value in running the tests if the build failed?

We only need one stage in the build to pass and emit binaries to run the offload-tests. Adding succeededOrFailed() makes sure tests are run even if an unrelated stage fails.

Do DXC PR builds often fail due to "unrelated" things? I know the offload test suite often has unrelated fails because of constant updates to llvm and the machines' drivers and such, but what about DXC?

"unrelated" in that scenario is a failure in a build that doesn't produce a binary, currently, that is the case for all of them except one. So if any build that doesn't produce a binary fails, we will not test this. This pipeline will be part of VK Release, therefore I don't want to block a VK release because a MacOS or Linux build is failling.

alsepkow · 2026-06-25T21:45:22Z

+    strategy:
+      matrix:
+        spirv_release:
+          artifact: 'dxc_spirv_release'


Whats the reasoning for this? If we just default this to true all the time then we could remove the branhcing logic and just do that every run instead?

Not all emitted binaries support SPIR-V codegen; in those cases, we don't want to test on lavapipe.

This matrix only has one element: spirv_release. Is it planned to add more configurations to this matrix in the future? If not, we could fold that runLavapipe check into the steps instead of as part of the matrix.

We don't have current plans to add more configurations, although I personally think we should. I personally think that leaving the matrix with one element helps clarify that this is a testing job and that we could add more binaries to be tests easily to that stage

joaosaffran · 2026-06-26T17:28:04Z

/azp run

azure-pipelines · 2026-06-26T17:28:14Z

Azure Pipelines successfully started running 1 pipeline(s).

damyanp

This LGTM, but please make sure you've addressed everyone else's feedback before merging.

github-project-automation Bot added this to HLSL Roadmap Jun 23, 2026

github-project-automation Bot moved this to New in HLSL Roadmap Jun 23, 2026

add vk to release pipeline

1cab50c

joaosaffran force-pushed the vk-release/build-test-pipeline branch 6 times, most recently from 2959f34 to 6c3d8b4 Compare June 24, 2026 20:32

clean up

fad2cb9

joaosaffran force-pushed the vk-release/build-test-pipeline branch from 6c3d8b4 to fad2cb9 Compare June 24, 2026 20:32

joaosaffran changed the title ~~[DO NOT REVIEW] Adding VK into nightly build~~ Adding Offload Test into DXC build pipeline Jun 24, 2026

joaosaffran marked this pull request as ready for review June 24, 2026 20:39

damyanp requested changes Jun 24, 2026

View reviewed changes

github-project-automation Bot moved this from New to In progress in HLSL Roadmap Jun 24, 2026

alsepkow reviewed Jun 24, 2026

View reviewed changes

address comments, test changes

aba9720

joaosaffran added the run-offload-tests trigger offload-tests on a given pr label Jun 25, 2026

joaosaffran requested review from alsepkow and damyanp June 25, 2026 16:34

damyanp reviewed Jun 25, 2026

View reviewed changes

damyanp approved these changes Jun 25, 2026

View reviewed changes

remove release-dxil from test

418b47c

Icohedron reviewed Jun 25, 2026

View reviewed changes

alsepkow reviewed Jun 25, 2026

View reviewed changes

joaosaffran removed the run-offload-tests trigger offload-tests on a given pr label Jun 26, 2026

joaosaffran requested a review from damyanp June 26, 2026 19:53

damyanp approved these changes Jun 26, 2026

View reviewed changes

add parameters

5c25c24

joaosaffran requested a review from damyanp June 26, 2026 21:28

damyanp approved these changes Jun 26, 2026

View reviewed changes

joaosaffran enabled auto-merge (squash) June 26, 2026 21:56

joaosaffran merged commit 02e491e into microsoft:main Jun 26, 2026
14 checks passed

github-project-automation Bot moved this from In progress to Done in HLSL Roadmap Jun 26, 2026

Uh oh!

Conversation

joaosaffran commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joaosaffran Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joaosaffran commented Jun 25, 2026

Uh oh!

azure-pipelines Bot commented Jun 25, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

damyanp left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alsepkow left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joaosaffran Jun 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Icohedron Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joaosaffran commented Jun 26, 2026

Uh oh!

azure-pipelines Bot commented Jun 26, 2026

Uh oh!

damyanp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

joaosaffran commented Jun 23, 2026 •

edited

Loading

joaosaffran Jun 25, 2026 •

edited

Loading

joaosaffran Jun 27, 2026 •

edited

Loading

Icohedron Jun 26, 2026 •

edited

Loading