Retry plugin causes paths longer than 256 characters for trx and coverage files. Please document or fix.

[turbobobbytraykov]

Summary

When using the Microsoft.Testing.Extensions.Retry plugin, trx and coverage file paths become longer than Windows default.
This caused a an error for a dotnet 4.6.2 execution, but it was handled internallyfor the dotnet 6 and 8 runs of the same tests.

Background and Motivation

Prior to using the Retry plugin, the file path length was within the default Windows limits.
For dotnet 8 and 6, this is not a problem, but for dotnet 4.6.2, one needs to apply both the registry key 'LongPathsEnabled' and add an app.manifest file to the tests project with the longPathAware setting set to true

Microsoft.Testing.Extensions.Retry = 1.4.1
Executing using 'dotnet run' in the UseDotNet task (v = 2.247.3 ) in Azure DevOps on on-prem agents.

Without the plugin, the file paths in my usecase look like so:
D:\agents\1_work\1\s\Product\CS\CP\Source\DataLayer\NetCore\Tests\DataLayerTests.NetCore\bin\DataLayerTests.NetCore\Release\net8.0\win-x64\TestResults\username_PRODUCTEST1_2024-10-26_01_20_58.262.trx
D:\agents\1_work\1\s\Product\CS\CP\Source\DataLayer\NetCore\Tests\DataLayerTests.NetCore\bin\DataLayerTests.NetCore\Release\net8.0\win-x64\TestResults\fa27f53a-ceb6-488d-b36d-c32375117c40.coverage

When using retries, these change to:
D:\agents\1_work\2\s\Product\CS\CP\Source\DataLayer\NetCore\Tests\DataLayerTests.NetCore\bin\DataLayerTests.NetCore\Release\net8.0\win-x64\TestResults\Retries\cbdb6a7c2aae469989a17c72d6a83a87\4\username_PRODUCTEST1_2024-11-04_15_12_34.959.trx
D:\agents\1_work\2\s\Product\CS\CP\Source\DataLayer\NetCore\Tests\DataLayerTests.NetCore\bin\DataLayerTests.NetCore\Release\net8.0\win-x64\TestResults\Retries\cbdb6a7c2aae469989a17c72d6a83a87\4\username_PRODUCTEST1_2024-11-
04_15_12_34.959\In\PRODUCTEST1\98290b44-a6e4-47dd-9430-7479de5b3119.coverage

The error in the dotnet 4.6.2 execution:

Unhandled Exception: System.IO.PathTooLongException: The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.Directory.InternalCreateDirectory(String fullPath, String path, Object dirSecurityObj, Boolean checkHost)
at System.IO.Directory.InternalCreateDirectoryHelper(String path, Boolean checkHost)
at Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.CreateOrGetTrxArtifactDirectory(String runDeploymentRoot)
at Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.d__31.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.d__29.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.d__30.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.<>c__DisplayClass26_0.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.d__27.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.d__26.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportGenerator.d__42.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.Testing.Platform.Hosts.CommonTestHost.d__16.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Testing.Platform.Hosts.CommonTestHost.d__12.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Testing.Platform.Hosts.ConsoleTestHost.d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.Testing.Platform.Hosts.ConsoleTestHost.d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Testing.Platform.Hosts.CommonTestHost.d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Testing.Platform.Hosts.CommonTestHost.d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.Testing.Platform.Hosts.CommonTestHost.d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Testing.Platform.Hosts.TestHostControlledHost.d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Testing.Platform.Builder.TestApplication.d__16.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at TestingPlatformEntryPoint.

d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at TestingPlatformEntryPoint.(String[] args)

[+2402/x0/?0] Company.Product.DataLayer.NetCoreTests.exe (net462|x64)(2h 08m 26s)

Test application process didn't exit gracefully, exit code is '7'
Test suite failed with and exit code different that 2 (failed tests). Failure related to an unexpected condition. Exit code '7'

Proposed Feature

Either document that the plugin requires changes to the registry and the tests project's app.manifest in order to accomodate long paths, or change the paths for the retries' files to be within the default limit.

For example, shortening the UUID to a smaller reasonable char count may help a lot.

Any alternative options are also welcome.

Alternative Designs

N/A

[nohwnd]

@Evangelink are you okay with changing the guid to the short random id e.g. fUfko we use in our tests as well? We can check for the directory, and create it to reserve it, so there is not really much concern around colliding ids, as long as they are reasonably random.

https://github.com/microsoft/testfx/blob/main/test/Utilities/Microsoft.Testing.TestInfrastructure/RandomId.cs

Or do you consider this a breaking change?

[Evangelink]

Good suggestion and question. I guess that ideally we should consider this a breaking change as it's likely that users will have to update custom scripts but given we have very few users at the moment I am fine to consider the change for v1.5.0.

Note that it's still possible that the generated path exceed the 256 characters and there is nothing we can do about that. I don't see any .NET tool mentioning this so I don't think we should either.