Seeding FreeIPA, Where Teardown Starts Revoking

Teardown in this module makes one promise: everything the seed made gets removed, with ownership established before anything is deleted. Five providers keep that promise to the letter. FreeIPA can’t.

The realm has its own certificate authority, the seed asks it for ten real certificates, and a CA has no delete. Teardown revokes every one that is still valid, and deleting the entries revokes the rest, but the serials stay in the CA’s records as revoked forever. A realm that has been seeded and torn down carries that history the way any CA carries its own.

That one property changed how I think about the whole module. Up to then I could treat “put it back” as absolute. With FreeIPA it comes with a footnote, and I decided the footnote is correct behavior for a CA and left it alone.

This is part six of the series, after what makes seed data useful, Active Directory, Entra, Okta and Authentik. FreeIPA is the largest provider in the module, and the one I learned the most from building.

What a POSIX realm manages

FreeIPA is a directory with Kerberos attached, and what it manages is whatever a fleet of Linux hosts asks it about. HBAC answers who may log in where. Sudo rules cover who may run what. An ID view sets the UID a person has on a given machine, automount decides which NFS export lands at /home, and roles reaching privileges reaching permissions say who administers the lot.

The cloud providers have nothing that maps onto these. A Conditional Access policy and an HBAC rule answer different questions, an Entra role grants nothing on a Linux box, and no tenant can answer what ipa hbactest answers.

A seed takes twelve minutes and a teardown eight, and most of that is the realm’s own per-object work.

Four states an account can be in

A FreeIPA account is active, disabled, staged or preserved. I seed one of each, because the difference decides what an audit of the realm can see.

A disabled account keeps every membership intact, so it still appears in the HBAC rule and the sudo rule that name it. A staged account doesn’t show up in user-find, which is what staging is for. A preserved account has had every membership stripped and sits in a different container.

A report that reads user-find and stops has missed two of the four. Teardown has the same problem from the other direction, so it asks for staged and preserved users in separate calls, since the listing that finds the other two lists neither.

The rest of the core tier is arranged to break assumptions one at a time. A contractor whose Kerberos principal expired two weeks ago while the account stayed enabled. A user whose authentication type demands an OTP token that nobody enrolled, so the account is healthy and can’t log in. A user with no private group, whose primary GID belongs to a shared group. A non-POSIX group nested inside a POSIX one, so membership resolves and no GID does. An external group that can hold only trusted-domain SIDs, wrapped in the POSIX group a trust would grant a GID through, holding nothing because the lab has no trust.

Nine people carry the shared set of hard names, each on a plain POSIX username.

Every password an administrator sets is already expired

This one shaped the bootstrap, and it’s FreeIPA working as designed.

Any password an administrator sets is expired the moment it is set, and the user has to change it before it works for anything. Creating a service account and writing its password into a record therefore produces a credential that can’t authenticate, and the creation call gives no hint of it.

The way out is the change-password endpoint. It needs no session, because it authenticates with the old password:

1
2

Set-FreeIPAPassword -Connection $connection -Username 'zz-test-automation' `
    -OldPassword $random -NewPassword $fresh

It reports the outcome in headers, so a status-code check misses it entirely. X-IPA-Pwchange-Result is ok, invalid-password, policy-error or error, and a policy failure explains itself in X-IPA-Pwchange-Policy-Error. I throw anything other than ok with that explanation attached, because a caller that carried on would store and report a password that doesn’t work.

The seed uses that endpoint three times over: to make the service account’s password current right after the bootstrap sets it, to rotate it when the realm’s policy has expired it, and to give a seeded user whose row says Current a password that is current. A user whose row says MustChange keeps the administrator-set one, expired on the spot, because that state belongs in the data too.

Two more consequences come from the same rule. A fresh administrator credential that has never logged in is in the same position, so the connect takes -NewPassword to do the change for you. The realm’s password policy will also expire the service account on its own schedule, so the bootstrap pushes that expiry ten years out, and a connect that meets an expired one anyway rotates it and rewrites the record with a warning.

The realm’s server certificate comes from its own CA. -CertificateAuthorityPath takes that PEM, and the connection trusts that one CA with nothing installed on the machine. The bootstrap writes the CA into the record, so later connects need no path.

Four hundred hosts with no keytab

None of the 413 hosts will ever enroll. Every seeded host is added with force, so the host step never consults or writes the realm’s DNS, and no seeded host has a keytab.

That leaves has_keytab as the property that tells a machine from a record, and an inventory that skips it will report four hundred Linux servers that don’t exist. One host record carries nothing at all: no description, no operating system, no group, no rule.

The hosts still have to resolve, because a review reconciles a host list against DNS. Writing those records into the realm’s own zone would put four hundred lab entries into somebody’s production DNS, with only their names to mark them as mine. The seed keeps zones of its own instead: a forward zone carrying the prefix under the realm’s domain, and a reverse zone for a private /16 nothing real should be using.

A reverse zone’s name has no room for a prefix, so ownership rests on something else:

1
2
3
4

Forward  zz-test-lab.ipa.example.com
Reverse  213.10.in-addr.arpa.
Subnet   10.213.0.0/16
Contact  hostmaster.zz-test-lab.ipa.example.com.

The SOA contact is the fingerprint. The seed writes that administrator address into each zone it creates, and it treats a zone as its own only when the contact matches. A zone with the seed’s name and some other contact is refused and never adopted, which closes off the case where somebody made a zone by that name first.

Inside those zones sit records built to disagree with each other: an address with no host behind it, an alias whose target doesn’t exist, a reverse record with no forward name, and one name with two addresses. Around them are ordinary aliases, a mail exchanger and a service record.

Access designed to be miscounted

Most of my design effort on this provider went into the access layers. HBAC and sudo are what a Linux fleet enforces, and both are easy to misread.

One user is admitted to payroll twice, by name and through her group, so anything counting grants finds two and anything deduplicating finds one. A disabled user is still named in an HBAC rule and in a sudo rule, since disabling an account removes it from nothing. An allow-everything HBAC rule exists and is switched off, because real realms keep those around after a migration. One rule of each kind is bound to nothing.

The sudo rules go after reach. A rule with a host category of all and one command. A rule with a user category of all carrying both an allow and a deny. A run-as of root with no password, granted through a non-POSIX group three levels down the nesting chain. A run-as of a local account that is not an IPA user at all, stored as external because FreeIPA has nowhere else to put it.

The one I would put in front of anyone writing a privilege review is a sudo rule that grants vim to contractors, with no password. That’s a root shell wearing an editor’s name badge, and a report that lists it without flagging it has failed.

ipa hbactest and ipa sudorule-show answer for all of it the way they would in a real realm, which is why I seed the rules at all.

Delegation runs the other way. Roles reach privileges reach permissions, and the single permission that writes is scoped by the seed tag. A host holds a role, which FreeIPA allows and most reports forget is possible. One seeded role carries a stock read-only privilege, and teardown leaves that privilege where it found it. One role has a privilege and no holder.

Certificates the realm really issued

All ten certificates are real, signed by the realm’s CA against a request the module builds with a key it generates and throws away. Nothing ever needs to use them. They exist to be seen on the entries and in the CA.

I build the request with in-box .NET, so neither edition of PowerShell needs a module or openssl:

1

System.Security.Cryptography.X509Certificates.CertificateRequest

That type is in .NET Framework 4.7.2 and every .NET Core, and a machine older than that gets a plain error in place of a confusing type resolution failure.

Each certificate is added to the entry it was issued to, so the chain works end to end: a user carries one in userCertificate, the realm-CA mapping rule maps it back to them, and ipa certmap-match answers with the right login. A service or host carries one with its name in the subject alternative name. The CA lists each with its serial, validity and status.

The states are the ones a review has to tell apart. A valid certificate on a disabled account. A revoked one sitting beside its replacement on the same user. One on certificate hold. A service certificate revoked as ceased while the service lives on. Only the seeded CA ACL permits the user certificates to be issued at all; the paused pilot rule is disabled and grants nothing.

Then teardown, where this provider stops resembling the others:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# A certificate is revoked, one serial at a time, because a CA has no delete: the record
# stays, marked ceased. Only a valid one is touched; a revoked one is already what
# teardown would make it. The serial goes as the hex form the CA returned, which is text
# in both editions, never the decimal number.
$held = @(Get-FreeIPASeededObject -Type Certificates -Connection $connection |
        Where-Object { (& $first $_.status) -eq 'VALID' })
foreach ($cert in $held) {
    ...
    $null = Invoke-FreeIPARequest -Method 'cert_revoke' -Arguments $serial `
        -Options @{ revocation_reason = 5 } -Connection $connection
}

Reason 5 is cessation of operation, which describes a lab certificate being torn down accurately enough. The serial travels as the hex string the CA returned. The decimal form is a big integer, and both editions of PowerShell will happily mangle it on the way through.

The realm’s stock objects

A seeding tool pointed at a realm somebody depends on has to be incapable of editing the realm’s own objects. Careful is a weaker guarantee than incapable.

allow_all, allow_systemd-user, global_policy, the stock privileges, the ipa-http-delegation rule, the Default Trust View, the default automount location and the automember default groups are never created, modified, enabled or disabled. No switch exists that would, and tests pin that no request ever reaches one.

A seeded rule may still reference a stock object, because an HBAC rule that can’t name sshd is useless. I write that reference with a marker:

1

builtin:sshd

A key written that way names an object FreeIPA created at install. The seed may attach to it but doesn’t create or change it, and the key resolves to the bare name. The seed data tests list which stock names may appear behind that marker and refuse every other reference to a stock object, because a row naming admins, allow_all or global_policy would mean the seed writing to it.

Automember gets the same treatment from a different angle. The five rules agree with every membership the data already lists, and after creation the seeded users and hosts are rebuilt against them by name, which changes nothing and confirms the rules match the data. A rebuild of the whole realm is never issued, since that would reprocess every real account against every real rule.

Batching over JSON-RPC

FreeIPA answers one command in well under a second, and the seed has hundreds of users, hundreds of hosts and hundreds of DNS records to create. The round trips were where my time went.

The realm’s JSON-RPC batch method takes a list of commands in one request and answers with one result per command, in order. Each step decides every object individually and sends them fifty at a time. Each answer carries its command back with Success, Ignored, the result envelope, and the error name and message when there’s one.

Two failure semantics decide how you build on it. A command the realm refuses fails alone, and the others in its chunk stand. A chunk the realm couldn’t take at all, whether from a transport failure or a session that couldn’t be renewed, fails every command in it with the same message, so no command is ever left without an answer.

Teardown batches too, because FreeIPA takes a list of names per delete, and four hundred hosts one at a time is four hundred round trips. Each object is still confirmed individually, so -WhatIf still lists every one of them by name.

Names, the prefix and the tag

FreeIPA lower-cases a login, and a group, host group or netgroup name has to match [a-zA-Z0-9_.][a-zA-Z0-9_.-]*, so the prefix is applied in its lower-case form and the seed files hold bare keys. all-staff becomes zz-test-all-staff. web01 becomes zz-test-web01.zz-test-lab.<domain>, because a FreeIPA host is its fully qualified name and a seeded host resolves in a zone the seed owns.

Human logins carry no prefix at all, since a real realm’s logins look like logins. The tag in userclass marks them, beside the class their row names, and user-find --class and host-find --class filter on it server-side.

Everything else carries whatever evidence its type can hold. Groups and host groups take a bracketed marker at the end of their description. A password policy is keyed by its group and belongs to the seed because the group does. A service belongs to its host. Permissions, delegation rules and targets have no description field and carry the prefix alone.

Sudo commands are the odd one out. A sudo command is named by its path, /usr/bin/vim, and can’t be prefixed, so ownership rests entirely on the marker in the description. A command that already existed without one is reused as it is and left behind at teardown. I would much rather leave a stray sudo command than have a lab tool delete /usr/bin/vim from somebody’s sudo configuration.

The seed files don’t hold the prefix or the tag as a literal. Where one is needed inside a value, such as an automount key pointing at the seeded NFS host or a permission filter scoped to the tag, it’s written as {prefix} or {tag} and substituted at creation.

What the provider leaves out

Subordinate ID ranges are out because FreeIPA can generate one per user and by design can never delete one. Seeding them would leave the realm permanently changed after a teardown that reported success.

Vaults need a KRA, and a realm may not have one installed.

Host DNS records from the host step are out for the reason covered above, and the seed’s own zones carry them instead.

Trust members would be SIDs from a trusted domain the lab doesn’t have. The external group exists and holds nothing, which is a state in its own right.

On top of those four, nothing ever enrolls. Every host in the seed is a record.

Verifying the realm

Test-TestEnvironment reads the users, groups and hosts the module owns through the same lookups teardown uses, and compares them with the seed files. It asks for staged and preserved accounts separately for the same reason teardown does, so a realm that lost one to a failed step reports the loss. Without that, the verifier would call a directory clean while two states were missing.

Names are compared by codepoint. The nine people in other writing systems sit on POSIX usernames here, and a realm that folded one of them would satisfy a -eq comparison against the wrong string. The automember rules add memberships the data never lists, so the verifier only fails a membership when something the data names is absent.

Repair-TestEnvironment re-runs the seed steps that own whatever failed. On this provider that means fixing one step in place of reseeding twelve minutes of realm. Compare-TestEnvironment lines this realm’s people up against another connected provider the same way as for PingOne.

First connect, then seed

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# First run, as an administrator. The realm's CA is at /etc/ipa/ca.crt on any enrolled host.
Connect-TestEnvironment -Provider FreeIPA -BaseUrl https://ipa.example.com `
    -Credential (Get-Credential admin) -CertificateAuthorityPath .\ca.crt
New-TestServiceApp

# Afterwards
Connect-TestEnvironment -Provider FreeIPA -BaseUrl https://ipa.example.com -ServiceAccount
New-TestEnvironment -ShowProgress
Get-TestEnvironmentReport
Test-TestEnvironment
Remove-TestEnvironment -WhatIf

-Tier Core on any of the seed commands builds the designed rows alone in seconds. The bulk tier is the AD provider’s people, service accounts, groups and devices mapped across, so the same person and the same machine exist in this realm as in the domain. Phones and printers stay behind, since they don’t enroll in an identity domain. Workstations and servers cross over, each joining a host group for its kind and one for its office.

I wrote twenty-two of the twenty-six seed files by hand. Providers\FreeIPA\Tools\New-FreeIPATestSeedData.ps1 regenerates the other four deterministically, taking the shared names from Core\Data\SeedPeople.csv, under the same drift test every provider uses.

A realm installed without DNS gets a warning and hosts without addresses. Everything else holds against a stock FreeIPA with its own CA, which is the only configuration I have tested against. The full inventory, with every object type and what each row is shaped to expose, is on the provider’s page: Providers/FreeIPA/README.md.

Related reading

• A Directory Full of Question Marks: Seeding PingOne
• Seeding Authentik, Where the Directory Is Only the Floor
• Standing Up FreeIPA When Everything in the Lab Is Linux