By the time I pushed the OrphanedSid branch to GitHub, it had passed 2,412 tests on both shells locally and a full run against a lab domain controller. The quality-gates workflow was copied from my other Gallery modules, so I wasn’t expecting it to find much.

This was a way to see my old style of thinking against my new one: Find-UnknownSID was the first script I tried Pester on, and the quality-gates workflow has been polished by everything I’ve learned since.

It has three jobs. Tests and analysis runs PSScriptAnalyzer and Pester with coverage on PowerShell 7. Windows PowerShell 5.1 runs the suite again under 5.1. Command help runs on Ubuntu and checks that the compiled help matches its Markdown. The first run, at 21:42 UTC, came back like this:

1
2
3

Windows PowerShell 5.1 => failure [Pester under Windows PowerShell 5.1]
Command help => failure [Fail on stale committed MAML]
Tests and analysis => failure [Pester with coverage]

The two Pester jobs had 44 and 55 failures:

1
2

Tests Passed: 2370, Failed: 44, Skipped: 1, Inconclusive: 0, NotRun: 0
Tests Passed: 2359, Failed: 55, Skipped: 1, Inconclusive: 0, NotRun: 0

Help files with nothing in them

The help job failed on its last check:

1

throw 'en-US/OrphanedSid-Help.xml is out of date. Run ./Build/Build-Help.ps1 and commit the result.'

The command help comes from PlatyPS Markdown under docs/OrphanedSid/, compiled to MAML in en-US/. All four Markdown files had been committed at zero bytes. A script that rewrote the troubleshooting paths in them had assigned the result of a regex call that threw, and then written $null back to each file.

Build-Help.ps1 passed on my machine anyway, for two reasons. Measure-PlatyPSMarkdown classifies an empty file as nothing at all, so the structure, placeholder and link checks had nothing to reject. And the compile step copied whatever was in maml/:

1
2
3

$built = Join-Path $mamlRoot $moduleName $helpFileName
# ...
Copy-Item -LiteralPath $built -Destination $cultureDir -Force

Locally, maml/ still held a build from before the files were emptied, so the copy found a real help file and the build looked fine. The Ubuntu job started from a clean checkout, where there was nothing to copy.

The Markdown was regenerated from the three public functions’ original comment-based help. Build-Help.ps1 now deletes maml/ before it compiles:

1
2
3
4
5

# A maml/ left by an earlier run would be copied below as if this run had produced it, hiding a
# compile that produced nothing. Start from nothing every time.
if (Test-Path $mamlRoot) {
    Remove-Item -LiteralPath $mamlRoot -Recurse -Force
}

It also counts what PlatyPS recognized against what the module exports, so a folder of empty files can’t pass again:

1
2
3
4
5

$exportedCount = @((Get-Module $moduleName).ExportedFunctions.Keys).Count
if (@($structure).Count -lt $exportedCount) {
    $empty = @(Get-ChildItem $markdown | Where-Object { $_.Length -eq 0 }).Name -join ', '
    throw "Only $(@($structure).Count) of $exportedCount command help files were recognised as PlatyPS Markdown. Empty files: $(if ($empty) { $empty } else { 'none' })."
}

Shuffled test order

Both Pester jobs shuffle the suite and print the seed, so you can replay any failing order with Run.ShuffleSeed:

1
2
3
4

$seed = Get-Random -Minimum 1 -Maximum 2147483647
$cfg.Run.Shuffle = $true
$cfg.Run.ShuffleSeed = $seed
Write-Host "Pester shuffle seed: $seed"

I’d run the suite in its normal order at home. On the runners, a lot of the failures looked like this:

1
2

[-] Should accept optional SecurityContext parameter 75ms
 CommandNotFoundException: Could not find Command Write-StructuredLog

Pester can only mock a command that exists. Four test files mocked Write-StructuredLog, and a fifth mocked Write-SecurityStructuredLogEntry, without defining either one. In declaration order an earlier file had already left a definition lying around, so the mock had something to attach to. Shuffled, the file sometimes ran first.

Other tests had quieter dependencies on order. The end-to-end discovery tests let one test’s scenario reach the next, a performance context reused an object built in another context, and the logging configuration tests checked a file-load state that other tests in the same file had changed. Each fix was small. The rule they add up to is in the commit: every test file passes alone in a fresh process with shuffling on, and the full suite passes on both shells with the seeds from the failed runs.

I’ve been enjoying shuffle since it showed up in Pester 6.1 (still marked experimental). It’s been a great way to find out whether my tests have hidden dependencies on each other.

TEMP on the runner is a short path

Several Test-PathTraversal tests failed only on GitHub. Their setup built paths from $env:TEMP, and on the Windows runners that’s an 8.3 short path. The first run’s log shows it:

1

C:\Users\RUNNER~1\AppData\Local\Temp\PathTraversalTests\invalid|file.txt

The function under test returns resolved, long paths. If your test builds its expected value from $env:TEMP, you’re comparing C:\Users\RUNNER~1\... against C:\Users\runneradmin\..., two spellings of the same folder. Get-Item expands the short form:

1
2
3

# $env:TEMP is an 8.3 short path (RUNNER~1) on the GitHub runners; Get-Item expands it so the
# paths the functions return, which are long, compare equal to the ones built here
$script:TestBasePath = Join-Path (Get-Item -LiteralPath $env:TEMP).FullName "PathTraversalTests"

The next run’s log has the long name, C:\Users\runneradmin\AppData\Local\Temp\..., in the same place.

Stop, twice

The rest of the Pester failures came from $ErrorActionPreference. The workflow set it to Stop before calling Invoke-Pester, which seemed like a reasonable thing to do in CI. The tests inherit it, though, and so does every function they call. A function that reports one bad item with Write-Error and carries on becomes a function that stops at the first bad item.

ScriptConfiguration.ValidateConfiguration() was one of those. It reported each finding with Write-Error and then returned $false, and its tests checked for the $false:

1
2

[-] Should return false for invalid BatchSize 12ms
 WriteErrorException: Configuration validation failed: BatchSize out of valid range (1-1000): 0

Under Stop, the Write-Error threw before the method could return. The method answers with $false and lets the caller decide, so its findings are warnings now:

1
2

Write-Warning "Invalid protected SID format: $sid"
return $false

The workflow lost its Stop line in the third run. One test still failed, and only under 5.1:

1
2

[-] Should maintain correlation tracking during errors 19ms
 WriteErrorException: Error validating path C:\Users\runneradmin\AppData\Local\Temp\PathTraversalTests\invalid|file.txt : Illegal characters in path.

That test passes a path with a pipe in it. On Windows PowerShell 5.1, .NET Framework rejects the pipe outright, so Test-PathTraversal lands in its catch, reports the problem with Write-Error, and moves on. PowerShell 7 on .NET doesn’t complain about the pipe at all, which is why the test only failed in one job. The same path on both shells on my machine:

1

[IO.Path]::GetFullPath('C:\Temp\PathTraversalTests\invalid|file.txt')

1
2
3
4

=== 5.1
FAILED: Illegal characters in path.
=== 7
C:\Temp\PathTraversalTests\invalid|file.txt

So the Write-Error was expected, and something was still turning it into a stop. The answer is in the runner’s design notes: for shell: pwsh and shell: powershell, the runner prepends $ErrorActionPreference = 'stop' to every step’s script. Deleting my line changed nothing. If you run Pester from a pwsh or powershell step, your tests are running under Stop whether your script says so or not.

A function that reports a problem and keeps going behaves like this with and without that preference:

1
2
3
4

--- default
carried on
--- stop
TERMINATED: Error validating path: bad path

Both Pester steps now set it back before the suite runs:

1
2
3
4
5

# The pwsh and powershell shells on GitHub Actions prepend $ErrorActionPreference = 'stop' to
# every step. The tests inherit it, and a function under test that reports a per-item
# problem with Write-Error then terminates instead of continuing, which fails tests that
# pass everywhere else. Set it back before the suite runs; the gates below throw explicitly.
$ErrorActionPreference = 'Continue'

The gates after the run check FailedCount and coverage themselves and throw, so the step still fails when it should.

A .gitignore rule one folder too wide

Fixing the restore path in the live run had come with a new test file, Tests/Unit/Private/Backup/Restore-IndividualObject.Tests.ps1. It wasn’t in the repository. The .gitignore had this:

1
2

Logs/
Backup/

Those are meant for the folders a run writes into the current directory. A bare Backup/ matches a folder named Backup at any depth, so it matched Tests/Unit/Private/Backup/ too. Older files in that folder were already tracked, and git kept tracking them, so nothing looked wrong. Only the new file was hidden. Anchoring both rules to the repository root fixed it:

1
2
3
4

/Logs/
# The runtime backup folder at the repository root only. A bare Backup/ also hid
# Tests/Unit/Private/Backup/, so a new test file there was never committed.
/Backup/

The third run is the first with that file in it, so its pass count went from 2,413 to 2,418.

The coverage gate

The second run’s PowerShell 7 job passed every test and failed anyway:

1
2

Covered 62.28% / 80%. 5,170 analyzed Commands in 60 Files.
Code coverage 62.28% is below the 80% gate.

The 80% gate came with the workflow, and this suite had never been near it. For now it became a ratchet at 60%, with a note in CLAUDE.md that the number only moves up.

The fourth run, at 23:34 on a595f92, was the first green one: all three jobs, both shells, shuffled.