You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Extract shared C# result downloading, parsing, publishing, transport, and metrics for the Helix SDK and Job Monitor.
Publish inline Helix results from the build after work-item completion; keep Job Monitor as the exclusive publisher when enabled.
Collect raw xUnit/JUnit XML and TRX on workers using PowerShell/shell payloads; remove the Python Azure Pipelines reporter and its JSON/pickle protocol.
Preserve run naming, queue mapping, attachment policies, and work-item execution results; add explicit handling for incomplete downloads/uploads and malformed inline results.
Add orchestration, target-ordering, collector, failure, cancellation, and concurrency tests; update reporting and migration documentation.
Compatibility
EnableAzurePipelinesReporter remains the inline reporting switch. Local/disabled reporting is unchanged; Job Monitor mode never also publishes inline.
Custom JSON/pickle report producers must emit supported raw XML/TRX instead.
Inline malformed results now fail reporting rather than silently omitting tests. Job Monitor retains its existing parsing policy.
.\build.cmd -configuration Release -test -pack — failed in two unchanged tests; compilation/package generation and Helix SDK tests passed:
CentralPackageManagementTests.ImplicitPackageReferences_ShouldNotConflictWithPackageVersionEntries: existing implicit Microsoft.VisualStudioEng.MicroBuild.Plugins.SwixBuild / central package-version conflict.
MsiTests.ItCanBuildATemplatePackMsi: Visual Studio MSBuild\Current\Bin\MSBuild.exe was not found on this machine.
Isolated dotnet/arcade-validation checkout, pinned to the locally built 12.0.0-dev packages and using a separate NuGet package directory:
& D:\repos\arcade\.dotnet\dotnet.exe build .\src\Validation\src\Validation.csproj --configuration Release — passed, no warnings/errors.
& D:\repos\arcade\.dotnet\dotnet.exe msbuild .\HelixReporting.proj /t:VerifyPackagedReporting /p:HelixTargetQueues=package-smoke /v:minimal — passed. Session-only smoke project checked reporting imports, both collector payloads, the shared result-processing assembly, and absence of the Python reporter; synthetic token/queue, no service requests.
Collect raw XML results on workers, download and publish them before closing test runs, and preserve monitor ownership and existing failure policies. Add task, target ordering, collection, and incomplete-upload tests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Place the collector under tools/test-results and suppress only NuGet's installation-script warning for this intentional runtime payload.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Qualify TRX display names only for inline publishing, keeping Job Monitor naming and xUnit custom-display identities unchanged. Cover class collisions, existing prefixes, and data rows through parsing, aggregation, and publication.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This migration also bypasses the removed inline xUnit parser's _unescape_xunit_message step. xUnit encodes control characters in failure messages as sequences such as \n and \xNNNN; the shared reader currently publishes failure/message verbatim, so inline failures will display the escape sequences instead of the original text. Preserve that decoding in the shared reader and add a representative compatibility test.
Switching inline reporting to LocalTestResultsReader drops the reason for skipped TRX tests. The removed TRX parser read Output/StdOut for NotExecuted, but the shared reader sets SkipReason from ErrorInfo/Message; skipped TRX entries normally have no ErrorInfo, and Failed attachment mode also discards their stdout. As a result, Azure Pipelines will show these tests as skipped with no reason. Preserve the former StdOut mapping (and add a compatibility assertion) before routing inline reporting through this processor.
The new inline path also changes JUnit failure data: the removed parser mapped the <failure message> attribute to the error message and the element text to the stack trace, while LocalTestResultsReader.ReadJUnitResult ignores the attribute and always sets StackTrace to null. Inline JUnit failures will therefore lose their structured message/stack trace in Azure Pipelines. Port those mappings to the shared reader and cover them with an inline compatibility test.
Keep Microsoft.Build for reporting target tests while accepting the upstream removal of coverlet.collector.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e810b258-0322-45b2-a0d3-b12937362f90
Disable Job Monitor jobs and override EnableHelixJobMonitor for unit and XHarness submissions. Revert this commit before merging the PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e810b258-0322-45b2-a0d3-b12937362f90
find -path interprets its argument as a glob even when quoted. With an upload directory under run[1]/upload, the collector does not prune it and copies upload/already.trx to upload/upload/already.trx. Both artifacts can then be parsed as separate attempts. Compare directory paths literally to avoid copying files already in the upload directory.
find -path treats its argument as a glob even when shell-quoted. With a working directory named work[1] and an upload directory inside it, the collector fails to prune that directory: an existing upload/already.trx is copied to upload/upload/already.trx. The duplicate can then be downloaded and aggregated as another attempt. Compare directory paths literally, and add a bracket-containing root to the deduplication test.
Read nested test results in postorder with output scoped to each result. Publish work-item execution results and close test runs after reporting failures without retrying uploads or hiding errors. Fix both collectors for bracketed working directories and cover the failure and publication paths.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e810b258-0322-45b2-a0d3-b12937362f90
Nested TRX rows are parsed/published, with each result retaining only its own diagnostics.
Failed inline result uploads now still publish completed work-item execution results and close runs, preserving the reporting error and without retrying the failed uploader.
Also fixed the collector issue from the review summary: the shell collector prunes upload directories using literal path equality rather than find glob matching. The same bracketed-directory regression exposed Windows PowerShell resetting its provider location at process startup; the PowerShell collector now uses the inherited process working directory. Both collectors keep existing uploads intact and avoid duplicates.
The xUnit decoding finding was already fixed in aebb18f. Temporary pipeline overrides remain intentionally in place for the requested inline-flow trial and must be reverted before merge.
.\build.cmd -configuration Release -test -pack — compilation/package generation and all 400 Helix tests passed; overall failed only on the same two known tests documented earlier: implicit SwixBuild central-package conflict and missing Visual Studio MSBuild for the MSI test.
Independent Git Bash collector scenarios — passed for normal/bracketed roots, upload roots inside/outside the working directory, quoted nested paths, and two collection passes without duplication.
Rebuilt SDK package contents — shared result-processing DLL, reporting targets, and both collector payloads match the updated build/source files.
Let healthy in-flight and queued uploads finish before reporting collected download, parse, or publication failures. Preserve the original exception for single failures and caller cancellation for the batch. Add mixed-stage regression coverage beyond the parallelism limit.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e810b258-0322-45b2-a0d3-b12937362f90
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Compatibility and download-isolation issues remain unresolved, and the cross-component migration still needs human review and live pipeline validation.
The inline uploader processes up to 16 work items concurrently, but sanitizing their names does not give each download a distinct directory. On Windows, work:item and work?item both become work-item; submission only replaces / (Sdk/Helpers.cs:17–20). If both contain testResults.xml, their downloads can overwrite one another before parsing or fail with sharing violations. Give each download an isolated directory and add a regression with colliding names. The suggested change adds a unique parent directory while preserving the final directory name used by the reader's JUnit fallback.
Show high-importance download, parsing, and publication stages with per-work-item file/result counts and a final summary. Explain the all-jobs completion gate in build output and document logging behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e810b258-0322-45b2-a0d3-b12937362f90
Added high-importance inline reporting progress logs in 4c80639, visible in normal pipeline output:
The build explicitly says it is waiting for all submitted Helix jobs before result processing, then announces work-item discovery.
Each work item logs download start, downloaded file count (or no supported result files), parsing/aggregation completion, publication start, and completed-work-item progress.
A final summary reports successful/failed work-item counts and the total published results. Existing per-work-item failure isolation and cancellation semantics are unchanged.
The supplied excerpt still has the Windows job outstanding; the existing all-jobs completion gate explains why result processing has not started yet. This change makes that gate visible without changing execution order.
Validation:
.\build.cmd -configuration Release -test -projects src\Microsoft.DotNet.Helix\Sdk.Tests\Microsoft.DotNet.Helix.Sdk.Tests\Microsoft.DotNet.Helix.Sdk.Tests.csproj — passed, 406 tests. Added stage-order/visibility/count assertions for populated and empty results, plus mixed-success/failure summary assertions.
.\build.cmd -configuration Release -test -pack — compilation/package generation and all 406 Helix tests passed; overall failed in unchanged SDK/workload tests: the known implicit SwixBuild central-package conflict and missing Visual Studio MSBuild for the MSI test, plus CreateVisualStudioWorkloadTests.ItCanCreateWorkloadsThatSupportArm64InVisualStudio (Assert.True failure) in this run.
Rebuilt SDK package contains the matching task assembly and MultiQueue targets; git diff --check passed.
The -type f predicate excludes symbolic links to result files, unlike the removed Python reader and previous monitor collector. With testResults.xml symlinked to a runner's output outside the working directory, the collector exits successfully but stages nothing; reporting then treats the work item as having no tests. Include symlinks whose targets are regular files without following directory symlinks, and add a collector regression for this case.
Keep batch boundaries and one outcome summary per work item visible by default. Move processing stages to detailed verbosity and replace repeated generic results links with test run IDs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e810b258-0322-45b2-a0d3-b12937362f90
Reduced inline result-processing log noise in 8715cb8:
Default output retains batch start/end and one summary per work item identifying its Helix job, published result count, top-level outcome breakdown, and completed/total work-item progress.
Download, parsing/aggregation, publication-stage messages and shared informational diagnostics now require detailed/diagnostic MSBuild verbosity. Warnings and errors stay visible.
Test-run closing messages show the run ID instead of repeating the same build-wide Results tab URL.
Validation: .\build.cmd -configuration Release -test -projects src\Microsoft.DotNet.Helix\Sdk.Tests\Microsoft.DotNet.Helix.Sdk.Tests\Microsoft.DotNet.Helix.Sdk.Tests.csproj passed (406 tests), including assertions for concise default messages and detailed-only stages. .\build.cmd -configuration Release -test -pack was started but interrupted before a final outcome; its Helix SDK tests passed and it reported the previously observed unrelated CentralPackageManagementTests implicit SwixBuild PackageVersion conflict. This is not a passing full-suite validation.
The paused retry/quarantine design and other outstanding review fixes are unchanged.
Retain UploadHelixTestResults without the reverted Architecture workaround and update upstream blob client tests for the shared publisher namespace.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ca6021bb-42a2-4bc8-9962-8d30c3e875e7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Compatibility
EnableAzurePipelinesReporterremains the inline reporting switch. Local/disabled reporting is unchanged; Job Monitor mode never also publishes inline.Validation
.\build.cmd -configuration Release -test -projects src\Microsoft.DotNet.Helix\Sdk.Tests\Microsoft.DotNet.Helix.Sdk.Tests\Microsoft.DotNet.Helix.Sdk.Tests.csproj— passed: 366 tests..\build.cmd -configuration Release -test -pack— failed in two unchanged tests; compilation/package generation and Helix SDK tests passed:CentralPackageManagementTests.ImplicitPackageReferences_ShouldNotConflictWithPackageVersionEntries: existing implicitMicrosoft.VisualStudioEng.MicroBuild.Plugins.SwixBuild/ central package-version conflict.MsiTests.ItCanBuildATemplatePackMsi: Visual StudioMSBuild\Current\Bin\MSBuild.exewas not found on this machine.dotnet/arcade-validationcheckout, pinned to the locally built12.0.0-devpackages and using a separate NuGet package directory:& D:\repos\arcade\.dotnet\dotnet.exe build .\src\Validation\src\Validation.csproj --configuration Release— passed, no warnings/errors.& D:\repos\arcade\.dotnet\dotnet.exe msbuild .\HelixReporting.proj /t:VerifyPackagedReporting /p:HelixTargetQueues=package-smoke /v:minimal— passed. Session-only smoke project checked reporting imports, both collector payloads, the shared result-processing assembly, and absence of the Python reporter; synthetic token/queue, no service requests.git diff --check— passed.To double check: