I published CaOutcome 0.6.0 to the PowerShell Gallery in September with 204 passing tests and a README whose Maester example had been returning the wrong answer since the day I wrote it. It ran, it returned objects of the right type, and every one of them was wrong. The fix is in 0.6.1, and the bug has nothing to do with Maester or Conditional Access. It’s a PowerShell pipeline rule I already knew, and I’d bet the same line is sitting in a module of yours. It was in mine for two months.

I still can’t believe I let this slip past me before publish, but we all make mistakes even with AI and unit tests.

Thirteen answers to one question

The README shows this one-liner for using the module with Maester:

1
2

Test-MtConditionalAccessWhatIf -UserId $u -IncludeApplications $app -AllResults |
    ConvertTo-CaOutcome

It should produce one CaOutcome.Result for the sign-in: the controls the tenant enforces today, the controls it would enforce once the report-only policies are promoted, and the delta between the two. Against my tenant’s thirteen policies it produced thirteen results. Each was folded from a single policy, so Current and Projected on every one of them described a sign-in nobody simulated, and Delta compared two worlds that didn’t exist. It raised no error and no warning, just thirteen objects of the right type.

Hand the same collection over by parameter and you get the right answer:

1
2

$results = Test-MtConditionalAccessWhatIf -UserId $u -IncludeApplications $app -AllResults
ConvertTo-CaOutcome -WhatIfResult $results

One result, and it’s correct. That gap between the two calls is where the bug lives, and it had been there since the first release that mentioned Maester.

Why 204 tests didn’t notice

Every unit test for ConvertTo-CaOutcome feeds the response in by parameter. The one that covers Maester’s shape looks like this:

1
2
3
4

It 'accepts the already-unwrapped collection, as Maester hands it back' {
    (ConvertTo-CaOutcome -WhatIfResult $script:Fixture.whatIf.value).Current.RequiredControls |
        Should-BeCollection $script:Expected
}

There was a pipeline test too. It piped $script:Fixture.whatIf, the whole OData envelope, which is a single object with a value property. One object in, one object out, assertion passes. Nothing in the suite piped the bare array, and the bare array is what Maester returns.

The rule I took away: once a function’s input can be a collection, a parameter test and a pipeline test of that function run different code. Binding by parameter hands process the whole collection as one argument. Binding from the pipeline hands process each element, one call at a time. If ValueFromPipeline is on the parameter, you need both tests, and the pipeline one has to pipe the shape your real upstream produces.

What PowerShell does with a piped array

PowerShell unrolls. Write an array to the pipeline and the next command’s process block runs once per element. That’s the behavior that makes Get-ChildItem | Where-Object work, and it doesn’t switch off because the downstream parameter is typed [object]. Parameter binding does no such thing. -WhatIfResult $array arrives in process as a single argument holding the array, and process runs once.

So the same data behaved two ways depending on who fetched it. Invoke-MgGraphRequest hands back the OData envelope, { value = [...] }, as one object whether you ask for a hashtable, a PSObject or a JSON string. Pipe that and process runs once with the whole response. Maester’s Test-MtConditionalAccessWhatIf returns the value array already unwrapped, as a convenience. Pipe that and process runs thirteen times with one policy each. Same evaluation, same thirteen policy results, two different functions on the receiving end.

There are ways to stop the unrolling from the sending side. Write-Output -NoEnumerate sends an array down the pipeline as one object, and the unary comma, ,$array, wraps it in an outer array that unrolls once and delivers the inner one intact. Neither helps here, because the sending command is Maester’s. You don’t control the upstream, so the receiving function has to cope.

Collect in process, fold in end

The fix is the standard shape for a function that reduces a collection to one result: collect in process, do the work in end.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

begin {
    $loosePolicies = [System.Collections.Generic.List[object]]::new()
}

process {
    if ($isLonePolicy) {
        $loosePolicies.Add($WhatIfResult)
    } else {
        Get-CaFoldedOutcome -Response $WhatIfResult @foldParameters
    }
}

end {
    if ($loosePolicies.Count -gt 0) {
        Get-CaFoldedOutcome -Response $loosePolicies.ToArray() @foldParameters
    }
}

The part that took the time was $isLonePolicy. The simple version, collect everything and fold once in end, breaks a case that already worked: piping several whole responses has to give one outcome per response, because Invoke-CaScenarioMatrix does that for a whole matrix of personas. That means process has to tell a lone policy result, which it collects, from a whole response arriving as one object, which it folds on the spot.

Three input shapes have to stay distinct for that to work:

  • A JSON string, from Invoke-MgGraphRequest -OutputType Json or a fixture read from disk. A string is IEnumerable, of characters, so it has to be excluded before any enumerable check or it’s treated as a collection and folded as a bag of chars. A string is always a whole response.
  • A hashtable, from -OutputType HashTable. Also enumerable, as key-value pairs, and also one object. It’s a whole response if it has a value key and a lone policy result if it doesn’t.
  • A PSObject, from -OutputType PSObject or ConvertFrom-Json. Not enumerable at all. Here a policy result and the OData envelope look identical to a type check, because both are single non-enumerable objects. The envelope carries a value property, and a policy result never does, so that property is the discriminator.
1
2
3
4
5
6
7
8
9

$isLonePolicy = $null -ne $WhatIfResult -and $WhatIfResult -isnot [string] -and
    ($WhatIfResult -is [hashtable] -or $WhatIfResult -isnot [System.Collections.IEnumerable])
if ($isLonePolicy) {
    if ($WhatIfResult -is [hashtable]) {
        $isLonePolicy = -not $WhatIfResult.ContainsKey('value')
    } else {
        $isLonePolicy = -not $WhatIfResult.PSObject.Properties['value']
    }
}

An array passed by parameter is enumerable and isn’t a string, so it fails the first line and folds immediately, which is the path every existing test was already exercising. The fold itself moved into a private function, Get-CaFoldedOutcome, because it now runs from two places in the command and I wanted one definition of the output object.

The test that would have caught it

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

It 'folds an unwrapped collection piped in as one response, as Maester pipes it' {
    $outcome = @($script:Fixture.whatIf.value | ConvertTo-CaOutcome)

    $outcome.Count | Should-Be 1
    $outcome[0].PolicyCount | Should-Be @($script:Fixture.whatIf.value).Count
    $outcome[0].Current.RequiredControls | Should-BeCollection $script:Expected
}

It 'still folds each whole response piped in on its own' {
    $outcome = @($script:Fixture.whatIf, $script:Fixture.whatIf | ConvertTo-CaOutcome)

    $outcome.Count | Should-Be 2
}

The first fails on the old code with a count of 13 against the thirteen-policy fixture. The second pins the behavior the simple fix would have broken.

The @() around each pipeline is the same unrolling rule biting inside the test. Without it, one outcome comes back as a bare object and thirteen come back as an array, and .Count on the bare object only reads 1 because PowerShell bolts a scalar Count onto single objects. Wrapping the pipeline makes the assertion mean the same thing either way, and the second test depends on it: two outcomes have to be two elements, not a pair of objects I happen to be comparing against a number.

How it surfaced

Not from the test suite. I was checking the module against the Maester 3.0 release candidate on a dev tenant, ran the README one-liner live, and the custom test’s detail line printed System.Object[] where $outcome.Delta.Summary should have been a sentence. Thirteen outcomes, thirteen summaries, stringified as an array. Fixtures replayed by parameter would never have shown that, no matter how many of them I added. On 3.0 itself there’s little to say: verified against rc26, the integration is unchanged, and the one practical difference is that 3.0 no longer installs Pester, so install 5.7.1 or later where Maester runs. The changelog has the rest.

What I check now

  • If a parameter has ValueFromPipeline, I test with $collection | Verb-Noun and with -Param $collection. They’re two different paths once the input is enumerable.
  • If the function folds a collection into one result, I decide up front what a single element arriving alone means, and write that decision into the function.
  • I exclude [string] before any IEnumerable check.
  • A README example is a test case I haven’t written yet.

Sources