I have a script called Convert-RC4toAES.ps1. It finds accounts whose msDS-SupportedEncryptionTypes sits at zero or at RC4 only and widens them to RC4 plus AES128 plus AES256. To test it I made a handful of accounts in the lab domain by hand, ran the script, and read the output. Clean run, every time, for a good while.

Two of the accounts it was supposed to catch were invisible to it, and my fixture caused both.

The first was a filter I had written to find DES accounts, comparing the attribute against 3, which is both DES bits set together. An account offering only DES-CBC-CRC carries 1. One offering only DES-CBC-MD5 carries 2. Neither satisfies a test that demands both bits, and a single bit is the more common misconfiguration in a real domain. Every fixture I had made carried 3, because 3 is what I typed when I sat down to invent example accounts, so the case the filter got wrong was the one case my lab did not contain.

The second is the one I find funnier now. To build the “nothing set, RC4 by default” case I cleared the attribute, because clearing it felt like the same thing as zero. The script selected on -eq 0. An absent attribute does not satisfy -eq 0. The fixture that existed specifically to prove that branch worked was a fixture the branch could never see, and the test passed because there was nothing to find.

Hand-built lab data is written by the same person, on the same afternoon, holding the same assumptions as the script it is meant to test. It agrees with the code because both came out of one head. That is the failure, and it does not announce itself: everything goes green.

So I built the data properly, and then I kept building it. TestEnvironment is where it ended up: one PowerShell module that seeds a realistic identity environment into Entra ID, Active Directory, Okta, Authentik or FreeIPA, lets you point scripts at it, and tears it down again proving ownership of every object before it deletes anything. It is on the PowerShell Gallery now, and publishing it got me writing this.

This post is the overall shape of it and what I learned about making seed data useful. Each provider gets its own post after this one.

From two modules to one

Some of this is already on the blog in pieces. I wrote up the Active Directory seeder when it populated a lab domain with a thousand-odd users, and later the Okta one, where the whole design problem was a free org’s ten-user ceiling. I started working on an Entra one too, unpublished. Three separate modules, three repositories, three sets of instructions.

What made me merge them was not tidiness. It was noticing what they had each learned separately.

All three needed to keep a credential somewhere, so all three had SecretStore handling. All three generated passwords. Two of them persisted certificates. Each implementation had independently discovered that a locked vault is detected by the specific error it throws, since no state you can query reports it, and that a private key survives a round trip only as a PFX. They learned the same lessons at different times, at different levels of quality, and a fix to one never reached the others. The AD module’s password generator was the weakest of the three and nobody had noticed, because nothing compared them.

Those live in Core now and are learned once. Everything specific to a directory keeps its own folder under Providers, and providers are found by looking:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

foreach ($providerFolder in (Get-ChildItem -Path "$ModuleRoot\Providers" -Directory -ErrorAction SilentlyContinue)) {
    $providerName = $providerFolder.Name

    foreach ($subFolder in 'Private', 'Public') {
        foreach ($function in (Get-ChildItem -Path "$($providerFolder.FullName)\$subFolder\*.ps1" -ErrorAction SilentlyContinue)) {
            . $function.FullName
        }
    }

    $script:TestEnvironmentProvider[$providerName] = @{
        Name     = $providerName
        Root     = $providerFolder.FullName
        DataPath = Join-Path $providerFolder.FullName 'Data'
    }
}

A provider appears by existing. No registration list, nothing to forget to update. Authentik and FreeIPA both went in after the split and neither needed a line of code outside its own folder.

The session is four steps in every directory, and only the connect line differs:

1
2
3
4

Connect-TestEnvironment -Provider Entra -TenantId <tenant-guid> -CertificateThumbprint <thumb>
New-TestEnvironment
Get-TestEnvironmentReport
Remove-TestEnvironment -Force

Dispatch that does not swallow -WhatIf

New-TestEnvironment forwards to whichever provider is connected. The interesting problem with a layer like that is parameters. Restating each provider’s parameters by hand means every provider change needs a matching edit in the dispatcher, and the edit somebody forgets fails by silently not forwarding. ValueFromRemainingArguments accepts everything and binds nothing, so a typo becomes a runtime surprise and tab completion stops working.

The target command’s real parameters get reflected over and re-declared:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

foreach ($entry in $command.Parameters.GetEnumerator()) {
    if ($common -contains $entry.Key) { continue }
    if ($Exclude -contains $entry.Key) { continue }

    $attributes = [System.Collections.ObjectModel.Collection[System.Attribute]]::new()

    # Parameter attributes are copied rather than recreated, so mandatory-ness, parameter
    # sets and pipeline binding all survive. Validation attributes come with them, which is
    # what keeps a bad value failing at the dispatcher rather than inside the provider.
    foreach ($attribute in $entry.Value.Attributes) {
        $attributes.Add($attribute)
    }

    $dictionary.Add($entry.Key,
        [System.Management.Automation.RuntimeDefinedParameter]::new(
            $entry.Key, $entry.Value.ParameterType, $attributes))
}

Copying the attribute objects carries mandatory-ness, parameter sets, pipeline binding and validation across; building them fresh would drop all four. A bad value fails at the dispatcher with the provider’s own message.

One detail in the same area cost me a while to reason about. New-TestEnvironment declares SupportsShouldProcess and never calls ShouldProcess:

1

[CmdletBinding(SupportsShouldProcess = $true)]

Each half of that is deliberate. Declaring it is what makes -WhatIf and -Confirm bind at all, because they are optional common parameters: a wrapper that omits the declaration rejects -WhatIf as an unknown parameter and never forwards it, and a dispatch layer built that way quietly removes -WhatIf from every command behind it. Not calling it keeps the prompt in one place. The provider function knows what it is about to touch and carries its own ConfirmImpact, and a wrapper that prompted as well would ask twice for one action.

What a seed has to be before a script can be tested against it

Designed rows and volume rows are different jobs. Every provider’s seed data has two tiers. Core is hand-written, and each row exists because it breaks something specific. Bulk is three hundred people with departments, titles, offices and manager chains. Volume does not make any designed case more likely to be found, so -Tier Core builds the awkward rows alone in seconds and is the loop you want while you are iterating. Volume proves the thing you wrote for nine users survives three hundred, and makes transitive group expansion cost something real: one bulk group has 17 direct members and 87 transitive ones.

The generated half has to be deterministic. Everything derived in the bulk tier, compliance state, usage location, platform, clearance level, comes from a stable SHA-256 hash of the object’s own key. Not Get-Random. Regenerating the seed files produces byte-identical output, so a diff shows only real changes, and a test regenerates them into a temporary folder and fails if the committed copies have drifted. A hand edit to generated data, or a tool change committed without its output, gets caught before it ships. This sounds fussy until you have reviewed a pull request where 296 CSV rows changed and none of them meaningfully.

The people are the same people everywhere. The Entra, Authentik and FreeIPA bulk tiers are all the AD provider’s directory mapped across. Priya Raghunathan exists in all of them, in the same department, reporting to the same manager. Hybrid identity matching is testable because of that, and so is anything that reconciles two directories against each other. Phones and printers stay behind in the FreeIPA mapping, because those do not enroll in an identity domain, and workstations and servers cross over.

Then there are negative controls. I added those last and would now start with them. The Kerberos edge cases are the clearest example, and they are a direct descendant of the fixture that beat me:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

# The values matter individually.
#
#   0  no explicit value beyond the default, which is the RC4-only state
#      an RC4-to-AES migration looks for
#   1  DES-CBC-CRC alone
#   2  DES-CBC-MD5 alone
#   3  both DES types
#   24 AES128+AES256, already remediated - the negative control
#
# 1 and 2 exist specifically because an AD filter written as
# "-band 3" requires BOTH bits and silently passes over an account
# offering only one, which is the more common misconfiguration and was a
# real defect in the DES remediation script.

And the second mistake, fixed in the way I should have built it originally:

1
2
3
4
5
6
7

# Written as a literal 0, not cleared. An absent attribute and an
# attribute set to 0 are different things to an LDAP filter: the
# RC4-to-AES script selects on "-eq 0", which an absent attribute
# does not satisfy, so clearing it produced a fixture that script
# could never see.
Set-ADUser -Identity $account.Sam `
    -Replace @{ 'msDS-SupportedEncryptionTypes' = $account.Enc }

The AES-only account at 24 is there to be ignored. A remediation script that selects it is broken in the direction nobody tests for, and without a row that must not be chosen, there is no way to tell selecting-correctly from selecting-everything.

Not a reference implementation, and not a baseline

I want to be blunt about this, because I don’t want to be misunderstood.

Nothing in this module is a model of a well-configured identity provider. A fair amount of it is a model of a badly configured one, because that is what scripts have to survive. A partial list of what gets seeded on purpose:

  • A fine-grained password policy with complexity off and reversible encryption on. No review should find either of those enabled anywhere.
  • A service account whose SPN points at a DNS alias while the host behind it has none. Kerberos breaks the day the alias moves.
  • A group whose members include a computer and a contact as well as a user. The contact has no SID at all, so Add-ADGroupMember refuses it and the membership has to be written to the member attribute directly.
  • Access control entries for SIDs that resolve to nothing, sitting on real groups. Well-formed, belonging to the domain, pointing at a RID the domain never issued. Residue, the way a deleted account leaves it.
  • Two groups in one domain sharing a display name, in different containers.
  • Acme, Inc Contractors and a user called Nakamura, Yuki, so the escaped comma in a distinguished name has something to escape. Code that recovers a name with Split(',')[0] returns Nakamura\ and does not throw.
  • A sudo rule in FreeIPA that grants vim to contractors with no password, which is a shell escape wearing an editor’s clothes.
  • An HBAC rule that allows everything, switched off, because a real realm keeps those around.
  • DNS records that do not reconcile: an address with no computer behind it, an alias whose target does not exist, a reverse record with no forward name, one name with two addresses.
  • A user who holds the same licence twice, once directly and once inherited from a group.
  • An external identity converted to a member, so that filtering on userType eq 'Guest' misses a genuine outsider.

If you copy any of that into a production tenant as a template, you have carefully built a bad directory. Some of it will fail an audit, and it should. The data exists so your script has something to be wrong about.

Seeded objects are also frequently not the same kind of thing as the real ones they resemble. A device object created through Graph has no certificate and no hardware identity, because a real one is created by a device joining a domain, which produces both. A seeded one is enough to be found by a report, counted in an inventory, owned by a user and matched by a membership rule. Signing in from it is impossible, and a Conditional Access evaluation will never see one as the device in play. Every seeded FreeIPA host is added with force and never enrolls, so has_keytab is how an inventory tells a machine from a record. Authenticator devices are absent from the Authentik provider entirely: the admin endpoints for TOTP, static and WebAuthn devices create them for the caller only, and the owner field is read-only, so there is no way through the API to put MFA state onto a seeded user.

Wrong in the data, inert in the directory

The counterweight to all of that is a small set of properties with no parameter attached to them.

A seeded Conditional Access policy is never enforcing. Ten of the eleven are report-only and one is deliberately disabled, because real tenants keep retired policies and an inventory script must not count one as active. enabled is not reachable: a seed row asking for it is refused, there is no -State, -Enabled or -Enforce parameter, and a contract test asserts the absence of all three. A report-only policy is fully evaluated and fully logged, appears in sign-in logs, and is returned by What If, so a tool like CaOutcome has real input and nothing gets denied.

The same bargain repeats in each directory. A seeded PIM role eligibility is eligible and never active, and a second test asserts nothing is ever posted to the endpoint that would make an assignment standing. A seeded Authentik flow never becomes anyone’s default, because the seed never writes to the brand and never touches a flow whose slug lacks the prefix. FreeIPA’s stock rules, allow_all, global_policy, the Default Trust View, the default automount location, are never created, modified, enabled or disabled, and the tests pin that no request reaches one. Unconstrained delegation is never seeded in Active Directory and no switch asks for it, since that one is a live weakness rather than inert test data. A review of a seeded domain should find the constrained delegation and nothing worse.

Nothing leaves a tenant either. The four Entra guests are invited with sendInvitationMessage false, on example.com, which RFC 2606 reserves and nobody can register. Named locations use the RFC 5737 documentation ranges, since a lab location containing a real routable range is a policy that could lock somebody out for real. There is no -SendInvitationMessage, no -Notify, and no path to making the module send mail.

One dynamic group rule taught me why that stance has to be built into the code. An early run used (user.userType -eq "Guest") and Entra immediately put two real external accounts into a seeded group. Careful was not enough; every dynamic rule is now scoped to the seed prefix and the module refuses one that is not.

Fixtures for what an API gives back

I did not expect to care about response shapes, and I use them constantly now. A seeded object produces a known response from the API, and a known response is something you can write an assertion against.

Licensing is my favourite. One seeded user holds the same SKU twice, directly and inherited from a group. Graph reports both identically in assignedLicenses: same skuId, listed once. Only licenseAssignmentStates distinguishes them, where the inherited entry carries the group’s object id in assignedByGroup. A script that reads assignedLicenses and stops cannot tell “remove this user from the group” from “remove the licence from this user”, and there is no way to discover that without an object where both are true at the same time.

Three seeded PIM eligibilities produce a different kind of fixture: an authoritative empty result. GET /roleManagement/directory/roleAssignments returns nothing for any of the three custom roles, so a standing-privilege report that reads that endpoint and stops concludes the roles are held by nobody, while three principals are one activation away from holding them. The report prints RoleEligibilities and RoleAssignments side by side and the second number is measured. If it is ever non-zero, a human made a standing assignment by hand.

Directory extensions were chosen for type coverage over realism, and they earn it. labRiskScore is an integer, so a row with a value of zero gets dropped by any check written as if ($value). labHeadcount is a 64-bit integer that loses precision through a double. labBadgePhoto is binary and breaks anything assuming a profile is printable. A schema of nothing but strings will not tell you that your export flattened a binary value.

The one I like best is extensionName on the governance groups, which is deliberately not in the recommended attribute map:

1
2
3
4
5

# extensionName as well, and it is NOT in the recommended map above.
# It is multi-valued, and an export that writes an attribute straight
# into a CSV cell renders a multi-valued one as the literal string
# "Microsoft.ActiveDirectory.Management.ADPropertyValueCollection"
# rather than its contents.

Anyone who has done AD reporting has shipped that string to a stakeholder at least once. Having it reproducible on demand, in a fixture, is better than rediscovering it in a spreadsheet somebody else is reading.

Non-ASCII names sit in the same category. Windows PowerShell writes CSV as ASCII unless told otherwise and replaces those characters with ?, silently, so Zoë Müller and José Niño exist to make that data loss visible. And an employeeId of 0007, since any numeric cast turns it into 7 and nothing says so.

One prefix, one tag, five places to put it

Everything gets named with ZZ-TEST-, and the reason for the exact string is not aesthetic:

1
2
3
4
5

# ZZ- leads so that seeded objects sort to the bottom of an alphabetical console listing, out of
# the way in a directory that also holds real work. No asterisks or other wildcard characters:
# they have to be escaped in an LDAP distinguished name and are rejected outright in an Entra
# group's mailNickname.
$script:TestEnvironmentDefaultPrefix = 'ZZ-TEST-'

The tag value is ZZ-TEST-seed everywhere, derived from the prefix, because two settings that have to agree for teardown to work are one setting too many. Where it gets stored differs, since each directory has a different native place for it. Active Directory uses adminDescription: base schema, present since Windows 2000, defined on top so one attribute covers every class, hidden from the default ADUC view, and written by nothing else.

1

Get-ADObject -LDAPFilter '(adminDescription=ZZ-TEST-seed)'

Extending the schema was rejected outright. An attribute can be deactivated and never deleted, it needs Schema Admins, and it replicates forest-wide permanently. A module whose entire promise is that teardown removes everything it made has no business leaving a permanent mark on your forest schema.

Entra puts the tag inside a sentence in description so it still reads like a description in the portal, and also uses a directory extension, because that is the only writable, queryable marker on a user. employeeType, companyName and every extensionAttribute are writable and not filterable, and Graph rejects all of them in $filter with Request_UnsupportedQuery even with ConsistencyLevel: eventual. Okta defines a labSeedTag profile attribute. Authentik puts it in the free-form attributes and additionally gives every seeded user a path of its own. FreeIPA uses userclass on users and hosts, which user-find --class filters on server-side, and a bracketed marker in the description of everything else, since human logins carry no prefix and the class is the whole proof for a user.

Teardown asks the container first and then proves ownership, and the rule the AD provider states is the one every provider follows: being inside OU=TestData makes an object a candidate, carrying the tag makes it a target. Anything in the seeded containers without the tag is left standing and reported, because a test OU is exactly the kind of place somebody parks a real object for a minute. The asymmetry is deliberate. The tag can spare an object or claim one the module made, and it can never cause something untagged to be deleted.

-WhatIf beats -Force on every destructive command, pinned by tests in every teardown suite. Somebody passing both is asking what would happen, not asking to be spared the question. -Force defeating -WhatIf was the worst defect an earlier version of mine ever shipped.

Running it

1
2
3
4

Install-PSResource -Name TestEnvironment

# or, on a host that still only has PowerShellGet v2
Install-Module -Name TestEnvironment -Scope CurrentUser

RequiredModules is empty and a contract test keeps it that way. Importing the module installs nothing and requires nothing, because the providers do not share a platform: four of them reach a REST API from any host, and the AD provider needs RSAT’s ActiveDirectory and GroupPolicy modules. Declaring those in the manifest would impose a Windows-only, RSAT-only dependency on somebody who only wanted to seed an Okta org from a container, so the AD provider imports them at connect time and names them clearly when they are absent.

The Pester suite mocks every Graph call, every RSAT cmdlet and every Okta, Authentik and FreeIPA request, so it reaches no tenant, no domain, no org and no realm, and runs on a workstation in about a minute. The AD suites bind against generated stubs under Tests/Stubs, appended to PSModulePath, so they run on a host with no RSAT at all.

1

Invoke-Pester -Path .\Tests

I point it only at a tenant, domain or org I can afford to reseed. The licence comes with no warranty, and that reads literally for a tool whose job is creating and deleting directory objects.

The five providers

Each of these gets a post of its own, because what is interesting about them is what is specific to them, and there is a lot.

Active Directory connects with nothing, because it uses the caller’s own Windows identity. Around 1,100 objects under OU=TestData, all 688 devices resolving forward and reverse in two zones the seed owns, and a live run that went wrong in a way I could not have designed: a lab domain registered two controllers and one had been switched off for weeks, discovery handed back the dead one part way through a seed, and three steps succeeded while everything after them failed identically.

Entra ID has no long-lived API key a human can generate, so the bootstrap credential is a human, signed in for exactly as long as it takes to create an application that can act on its own. Roughly 1,150 objects held in administrative units, no Graph SDK anywhere, and a long list of things Graph does that no documentation prepared me for. Replication lag reports itself three different ways, and one of them is a read.

Okta is a design exercise in a ten-user ceiling. It seeds breadth over volume: group rules, custom user types, network zones, sign-on policies, trusted origins, event hooks and linked objects, in an org that will not let you have an eleventh person.

Authentik is self-hosted, so there is no cap to design around, and the seed goes after the layers above the directory: flows, policies, bindings, entitlements, scope mappings and outposts. A policy binds to a UUID that is not the application’s primary key, and getting that wrong creates a policy that reports fine and enforces nothing.

FreeIPA is the largest provider and the one that taught me the most, because a POSIX directory with Kerberos manages things the cloud providers have no equivalent for: HBAC, sudo rules, ID views, automount, SELinux user maps, OTP tokens, and real certificates issued by the realm’s own CA. A CA never forgets, so teardown revokes rather than deletes, and a realm that has been seeded carries that history.

The other direction I keep coming back to is the one I started with. Every one of these providers exists because I had a script I could not prove anything about. The seed data is the part of testing I used to skip, and every nasty defect I have found in my own work since has come out of a row that was written to disagree with me.