Seeding FreeIPA, Where Teardown Starts Revoking

Teardown in this module has one promise: everything the seed made, removed, with ownership proved before anything is deleted. Five providers keep it exactly. FreeIPA cannot.

The realm has its own certificate authority, the seed asks it for ten real certificates, and a CA has no delete. Teardown revokes every one that is still valid, and deleting the entries revokes the rest, but the serials stay in the CA’s records as revoked forever. A realm that has been seeded and torn down carries that history the way any CA carries its own.

That single property changed how I think about the whole module. Up to then, “put it back” was a thing I could keep absolutely. Here it is a thing with a footnote, and the footnote is the correct behaviour, not a shortcoming to engineer away.

This is part six of the series, after what makes seed data useful, Active Directory, Entra, Okta and Authentik. FreeIPA is the largest provider in it, and the one that taught me the most.

What a POSIX realm manages that a tenant does not

FreeIPA is a directory with Kerberos attached, and what it manages is what a fleet of Linux hosts asks it about. HBAC answers who may log in where. Sudo rules cover who may run what. An ID view sets the UID a person has on a given machine, automount decides which NFS export lands at /home, and roles reaching privileges reaching permissions say who administers the lot.

None of those has a cloud equivalent. A Conditional Access policy is not an HBAC rule, an Entra role is not a sudo rule, and nothing in a tenant answers the question ipa hbactest answers.

Twelve minutes to seed, eight to tear down. Most of that is the realm’s own work per object.

Four states an account can be in

FreeIPA accounts are not enabled or disabled. They are active, disabled, staged or preserved, and the seed has one of each because the difference decides what an audit of the realm can see.

A disabled account keeps every membership intact, so it still appears in the HBAC rule and the sudo rule that name it. A staged account is invisible to user-find entirely, and that is the point of staging. A preserved account has had every membership stripped and sits in a different container.

A report that reads user-find and stops has missed two of the four. Teardown has the same problem from the other direction, so it asks for staged and preserved users in separate calls, since the listing that finds the other two lists neither.

The rest of the core tier is arranged to break assumptions one at a time. A contractor whose Kerberos principal expired two weeks ago while the account stayed enabled. A user whose authentication type demands an OTP token that nobody enrolled, so the account is healthy and cannot log in. A user with no private group, whose primary GID belongs to a shared group. A non-POSIX group nested inside a POSIX one, so membership resolves and no GID does. An external group that can hold only trusted-domain SIDs, wrapped in the POSIX group a trust would grant a GID through, holding nothing because the lab has no trust.

Nine of the people are written in Han, Cyrillic, Greek, Arabic and Devanagari, with a decomposed name, a Turkish dotless i, an eszett and a surname above the basic multilingual plane, each on a plain POSIX username. Same nine as everywhere else in the module, from the same shared file.

Every password an administrator sets is already expired

This one shapes the bootstrap, and it is FreeIPA behaving exactly as designed.

Any password an administrator sets is expired the moment it is set. The user has to change it before it can be used for anything. So creating a service account and writing its password into a record produces a credential that cannot authenticate, and nothing about the creation call says so.

The way out is the change-password endpoint. It needs no session, because it authenticates with the old password:

1
2

Set-FreeIPAPassword -Connection $connection -Username 'zz-test-automation' `
    -OldPassword $random -NewPassword $fresh

It answers in headers, not in a status code. X-IPA-Pwchange-Result is ok, invalid-password, policy-error or error, and a policy failure explains itself in X-IPA-Pwchange-Policy-Error. Anything but ok is thrown with that explanation, because a caller that carried on would store and report a password that does not work.

The seed uses that endpoint three times over: to make the service account’s password current right after the bootstrap set it, to rotate it when the realm’s policy has expired it, and to give a seeded user whose row says Current a password that is. A user whose row says MustChange keeps the administrator-set one, expired on the spot, because that state belongs in the data too.

Two more consequences fall out of the same rule. A fresh administrator credential that has never logged in is in the same position, so the connect takes -NewPassword to do the change for you. And the realm’s own password policy will expire the service account on its own schedule, so the bootstrap pushes that expiry ten years out, and a connect that meets an expired one anyway rotates it and rewrites the record with a warning.

The realm’s server certificate is from its own CA, so -CertificateAuthorityPath takes that PEM and the connection trusts exactly it and nothing else, with nothing installed on the machine. The bootstrap writes the CA into the record, so later connects need no path.

Records, not machines

Four hundred and thirteen hosts, and not one of them will ever enrol. Every seeded host is added with force, so the realm’s DNS is never consulted or written by the host step, and no seeded host has a keytab.

That makes has_keytab the property that tells a machine from a record, and an inventory that does not read it will report four hundred Linux servers that do not exist. One host record carries nothing at all: no description, no operating system, no group, no rule.

The hosts still have to resolve, because a review reconciles a host list against DNS. Writing those records into the realm’s own zone would put four hundred lab entries into somebody’s production DNS with only their names to say they were ours, so the seed keeps zones of its own: a forward zone carrying the prefix under the realm’s domain, and a reverse zone for a private /16 nothing real should be using.

A reverse zone’s name cannot carry a prefix, so ownership rests on something else:

1
2
3
4

Forward  zz-test-lab.ipa.example.com
Reverse  213.10.in-addr.arpa.
Subnet   10.213.0.0/16
Contact  hostmaster.zz-test-lab.ipa.example.com.

The SOA contact is the fingerprint. The seed writes that exact administrator address into each zone it creates, and a zone is the seed’s only when it carries it. A zone with the seed’s name and another contact is refused and never adopted. One comparison closes off the case where somebody made a zone by that name first.

Inside those zones sit the records that deliberately do not reconcile: an address with no host behind it, an alias whose target does not exist, a reverse record with no forward name, one name with two addresses, beside the ordinary aliases, a mail exchanger and a service record.

Access that does not add up

The access layers are where the seed spends most of its design effort, because HBAC and sudo are what a Linux fleet enforces, and both are easy to misread.

One user is admitted to payroll twice, by name and through her group, so anything counting grants finds two and anything deduplicating finds one. A disabled user is still named in an HBAC rule and in a sudo rule, since disabling an account does not remove it from anything. An allow-everything HBAC rule exists and is switched off, because real realms keep those around after a migration. One rule of each kind is bound to nothing.

The sudo rules go after reach, not counts. A rule with a host category of all and one command. A rule with a user category of all carrying both an allow and a deny. A run-as of root with no password, granted through a non-POSIX group three levels down the nesting chain. A run-as of a local account that is not an IPA user at all, stored as external because FreeIPA has nowhere else to put it.

And the one I would put in front of anyone writing a privilege review: a sudo rule that grants vim to contractors, with no password. That is a root shell wearing an editor’s name badge, and it is the kind of row a report has to flag and not merely list.

ipa hbactest and ipa sudorule-show answer for all of it the way they would in a real realm, which is the point of seeding the rules at all.

Delegation runs the other way. Roles reach privileges reach permissions, and the single permission that writes is scoped by the seed tag. A host holds a role, which FreeIPA allows and most reports forget is possible. One seeded role carries a stock read-only privilege, and teardown leaves that privilege exactly where it found it. One role has a privilege and no holder.

Certificates the realm really issued

Ten certificates, all real, all signed by the realm’s CA against a request the module builds with a key it generates and throws away. Nothing ever needs to use them; things need to see them on the entries and in the CA.

The request is built with in-box .NET, so neither edition of PowerShell needs a module or openssl:

1

System.Security.Cryptography.X509Certificates.CertificateRequest

That type is in .NET Framework 4.7.2 and every .NET Core, and a machine older than that gets a plain error and not a type resolution failure.

Each certificate is added to the entry it was issued to, so the chain works end to end: a user carries one in userCertificate, the realm-CA mapping rule maps it back to them, and ipa certmap-match answers with the right login. A service or host carries one with its name in the subject alternative name. The CA lists each with its serial, validity and status.

The states are the ones a review has to tell apart. A valid certificate on a disabled account. A revoked one sitting beside its replacement on the same user. One on certificate hold. A service certificate revoked as ceased while the service lives on. Only the seeded CA ACL permits the user certificates to be issued at all; the paused pilot rule is disabled and grants nothing.

Then teardown, where this provider stops resembling the others:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# A certificate is revoked, one serial at a time, because a CA has no delete: the record
# stays, marked ceased. Only a valid one is touched; a revoked one is already what
# teardown would make it. The serial goes as the hex form the CA returned, which is text
# in both editions, never the decimal number.
$held = @(Get-FreeIPASeededObject -Type Certificates -Connection $connection |
        Where-Object { (& $first $_.status) -eq 'VALID' })
foreach ($cert in $held) {
    ...
    $null = Invoke-FreeIPARequest -Method 'cert_revoke' -Arguments $serial `
        -Options @{ revocation_reason = 5 } -Connection $connection
}

Reason 5 is cessation of operation, which describes a lab certificate being torn down accurately enough. The serial travels as the hex string the CA returned rather than the decimal number, because the decimal form is a big integer that both editions of PowerShell will happily mangle on the way through.

Never touching what the realm shipped with

A seeding tool pointed at a realm somebody depends on has to be incapable of editing the realm’s own objects, and incapable is a stronger word than careful.

allow_all, allow_systemd-user, global_policy, the stock privileges, the ipa-http-delegation rule, the Default Trust View, the default automount location and the automember default groups are never created, modified, enabled or disabled. There is no switch that would, and tests pin that no request ever reaches one.

A seeded rule may still reference a stock object, because an HBAC rule that cannot name sshd is not a useful HBAC rule. That reference is written with a marker:

1

builtin:sshd

A key written that way names an object FreeIPA created at install. The seed may attach to it and never creates or changes it, and the key resolves to the bare name. The seed data tests list exactly which stock names may appear behind that marker and refuse every other reference to a stock object, because a row naming admins, allow_all or global_policy would mean the seed writing to it.

Automember gets the same treatment from a different angle. The five rules agree with every membership the data already lists, and the seeded users and hosts are rebuilt against them by name after creation, which changes nothing and proves it. A rebuild of the whole realm is never issued, since that would reprocess every real account against every real rule.

Fifty at a time

FreeIPA answers one command in well under a second, and the seed has hundreds of users, hundreds of hosts and hundreds of DNS records to create. The round trip is where the time went.

The realm’s JSON-RPC batch method takes a list of commands in one request and answers with one result per command, in order, so a step decides every object individually and sends them fifty at a time. Each answer carries its command back with Success, Ignored, the result envelope, and the error name and message when there is one.

Two failure semantics decide how you build on it. A command the realm refuses fails alone and the others in its chunk stand. A chunk the realm could not take at all, a transport failure or a session that could not be renewed, fails every command in it with the same message, so no command is ever left silently unanswered.

Teardown batches too, because FreeIPA takes a list of names per delete and four hundred hosts one at a time is four hundred round trips. Each object is still confirmed individually, so -WhatIf still lists every one of them by name.

Names, the prefix and the tag

FreeIPA lower-cases a login, and a group, host group or netgroup name has to match [a-zA-Z0-9_.][a-zA-Z0-9_.-]*, so the prefix is applied in its lower-case form and the seed files hold bare keys. all-staff becomes zz-test-all-staff. web01 becomes zz-test-web01.zz-test-lab.<domain>, because a FreeIPA host is its fully qualified name and a seeded host resolves in a zone the seed owns.

Human logins carry no prefix at all, deliberately: a real realm’s logins look like logins. The tag in userclass is what marks them, beside the class their row names, and user-find --class and host-find --class filter on it server-side.

Everything else carries whatever evidence its type can hold. Groups and host groups take a bracketed marker at the end of their description. A password policy is keyed by its group and is the seed’s because the group is. A service belongs to its host. Permissions, delegation rules and targets have no description at all and carry the prefix alone.

Sudo commands are the interesting exception. A sudo command is named by its path, /usr/bin/vim, and cannot be prefixed. Ownership rests entirely on the marker in the description, so a command that already existed without one is reused as it is and left behind at teardown. Anything else would mean a lab tool deleting /usr/bin/vim from somebody’s sudo configuration.

The seed files never hold the prefix or the tag as a literal. Where one is needed inside a value, an automount key pointing at the seeded NFS host or a permission filter scoped to the tag, it is written as {prefix} or {tag} and substituted at creation.

Deliberately absent

Four things the provider will not seed, and the reason for each.

Subordinate ID ranges, because FreeIPA can generate one per user and by design can never delete one. Seeding them would leave the realm permanently changed after a teardown that reported success.

Vaults, because they need a KRA that a realm may not have installed.

Host DNS records written by the host step, because a host added with force never consults or writes the realm’s DNS. The seed’s own zones carry the records instead.

Trust members, because the external group’s members would be SIDs from a trusted domain the lab does not have. The group exists and holds nothing, which is a state in its own right.

And no enrolment, ever. The seed makes records, not machines.

Checking it afterwards

Test-TestEnvironment reads the users, groups and hosts the module owns, the same way teardown finds them, and compares them with the seed files. It asks for the staged and preserved accounts separately for the same reason teardown does, so a realm that lost one to a failed step says so rather than reporting a clean directory that is missing two states.

Names are compared by codepoint. The nine people in other writing systems are here on POSIX usernames, and a realm that folded one of them would satisfy a -eq comparison against the wrong string. Memberships are judged on what is missing only, because the automember rules add memberships the data does not list, and reporting those as unexpected would mean reporting the rules working as a fault.

Repair-TestEnvironment re-runs the seed steps that own whatever failed, which on this provider is the difference between fixing one step and reseeding twelve minutes of realm. Compare-TestEnvironment matches this realm’s people against another connected provider’s by their shared login key, which is how the same person in the domain and in the realm gets checked against each other.

Running it

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# First run, as an administrator. The realm's CA is at /etc/ipa/ca.crt on any enrolled host.
Connect-TestEnvironment -Provider FreeIPA -BaseUrl https://ipa.example.com `
    -Credential (Get-Credential admin) -CertificateAuthorityPath .\ca.crt
New-TestServiceApp

# Afterwards
Connect-TestEnvironment -Provider FreeIPA -BaseUrl https://ipa.example.com -ServiceAccount
New-TestEnvironment -ShowProgress
Get-TestEnvironmentReport
Test-TestEnvironment
Remove-TestEnvironment -WhatIf

-Tier Core on any of the seed commands builds the designed rows alone in seconds, and the bulk tier is the AD provider’s people, service accounts, groups and devices mapped across, so the same person and the same machine exist in this realm as in the domain. Phones and printers stay behind, since those do not enrol in an identity domain; workstations and servers cross over, each joining a host group for its kind and one for its office.

Twenty-two of the twenty-six seed files are written by hand. Providers\FreeIPA\Tools\New-FreeIPATestSeedData.ps1 regenerates the other four, deterministically, taking the shared names from Core\Data\SeedPeople.csv, and a test regenerates them into a temporary folder and fails if the committed copies have drifted.

A realm installed without DNS gets a warning and hosts without addresses. Everything else holds against a stock FreeIPA with its own CA, which is the only configuration I have tested against. The full inventory, with every object type and what each row is shaped to expose, is on the provider’s page: Providers/FreeIPA/README.md.

Next, and last in the provider series: PingOne, where the population is built from the same people as everything else and the environment stores no display name at all, so a cross-directory comparison has to match on the parts.

Related reading

• Seeding Authentik, Where the Directory Is Only the Floor
• Standing Up FreeIPA When Everything in the Lab Is Linux
• Test Data That Breaks Scripts on Purpose