A Directory Full of Question Marks: Seeding PingOne

The first full PingOne seed I ran from Windows PowerShell 5.1 produced a directory where José was Jos<?>, the Han names were rows of question marks, and one surname had lost a character entirely. The same seed from PowerShell 7, against the same environment, stored every name perfectly.

PingOne was blameless. Windows PowerShell sends a string request body as ISO-8859-1 when no charset is named, whatever the machine’s code page. An accented e goes out as the lone byte E9 and comes back stored as U+FFFD. A Han character goes out as ?, and that’s what the directory keeps.

I would have filed it as a footnote, except that PingOne was the sixth provider, written from scratch after the other five, and all six had the same exposure. The fix is now one function in Core that every provider sends through:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# The body goes out as UTF-8 bytes with the charset named, on both editions. A string
# body is sent by 5.1 as ISO-8859-1 when no charset is named, whatever the machine's code
# page: observed against PingOne, an accented e went out as the lone byte E9 and was
# stored as U+FFFD, and a Han character as '?'.
elseif ($Body -is [string]) {
    $bytes = [System.Text.Encoding]::UTF8.GetBytes($Body)
}
else {
    $bytes = [System.Text.Encoding]::UTF8.GetBytes(($Body | ConvertTo-Json -Depth $JsonDepth -Compress))
}

Responses are decoded from their raw bytes for the mirror-image reason. Windows PowerShell decodes by the declared charset and falls back to Latin-1, and at least one provider’s API declares none.

This is part seven, after what makes seed data useful, Active Directory, Entra, Okta, Authentik and FreeIPA.

Four populations and 319 people

A full seed of 319 users takes four minutes, teardown a minute, and the report ten seconds. -Tier Core seeds every object type with the nineteen designed users alone in about twenty-five seconds, so while I’m working on behavior the loop costs less than reading the output.

Populations carry more weight here than containers do in the other providers, because they’re the containment mechanism and PingOne gives a user one of them. I seed four. The first holds most of the directory and is the one the dynamic filter keys on. A second exists so that anything assuming a single population is wrong. A third is where the population-scoped group is bound. The fourth stays empty, since plenty of code has never seen an empty container.

Everything the seed creates carries the prefix on its name and the tag somewhere it can be checked, and nothing else in the environment is created, changed or removed. The list of what it leaves alone runs long and is on the provider’s page: sign-on policies, MFA and risk policies, administrator roles, certificates, branding, gateways, the environment’s default population, PingOne’s own applications, and any object it didn’t create, including a trial’s sample data.

A token from the wrong environment

The first thing that cost me an evening has nothing to do with seeding.

A PingOne worker application gets its token from the environment it lives in, which may be a different one from the environment it manages. A trial hands you an Administrators environment and a sandbox, and the console’s application list opens on Administrators, so that’s where most workers end up. Asking the sandbox’s token endpoint for a token with that worker is refused with invalid_client.

A disabled application produces that same message, and so does a mistyped secret. Three quite different problems share one answer, and the answer names none of them.

1
2
3

Connect-TestEnvironment -Provider PingOne -EnvironmentId <sandbox guid> `
    -ClientId <worker guid> -ClientSecret $secret `
    -AuthEnvironmentId <administrators guid> -SaveSecret

-AuthEnvironmentId says where the application lives. It defaults to -EnvironmentId, which is correct when the two are the same and harmless to state when they differ.

The region has the same shape of problem. PingOne serves North America, Europe, Canada, Asia-Pacific and Australia from different hostnames, and an environment id gives no hint which. Ask the wrong regional host and you get a 404 that reads as a missing environment. -Region has to be named, the module never guesses it, and the error says which host was tried. The console’s own hostname is the reliable way to tell.

The connection is then validated before it is stored, by reading the environment once. Without that, a credential accepted at connect time and refused on the first real call would hand somebody an error about users when the problem is the credential.

-SaveSecret writes the worker secret to ~/.testenvironment/<environment>.pingone.secret once the connection has worked, and -UseStoredSecret reads it back later.

A user with nowhere to put a tag

On every earlier provider I found a user field to hold the seed’s marker, either one that already existed or one the platform let me add.

A PingOne user has account, address, email, enabled, identityProvider, lifecycle, mfaEnabled, name, population, username and verifyStatus. None of them is free text I could claim without stepping on something real.

So the provider makes its own field. It creates a custom attribute, zzTestSeedTag, and writes the tag into it on every user it creates. Nothing else writes that attribute, because the module created it.

Ownership then has two routes, and the primary one is containment:

1
2
3
4
5
6

Users          in a seeded population, or carrying the tag attribute
Populations    tag in the description AND prefix on the name
Groups         tag in the description AND prefix on the name
Applications   tag in the description AND prefix on the name
Resources      tag in the description AND prefix on the name
Attributes     schemaType CUSTOM AND named in this provider's data file

Every seeded user sits in a seeded population, so teardown enumerates those populations and removes what is in them. The tag attribute is the fallback, for a user somebody moved out of a seeded population by hand, and on its own it is enough.

The other types need both pieces. The tag alone could be pasted into a real object’s description by accident, and the prefix alone would be name matching, which this module refuses to do anywhere.

PingOne’s own applications and built-in resources are refused by type whatever their description says, so renaming one in the console can’t bring it within teardown’s reach. Custom attributes are claimed only when they are CUSTOM and declared in the provider’s data file, since a CORE or STANDARD attribute belongs to the platform however it is named.

Ask the schema before you filter on it

Finding the tagged users means filtering on the custom attribute, and filtering on an attribute the schema lacks is refused with HTTP 400 REQUEST_FAILED. That code is broad enough that ignoring it would hide real failures too.

The attribute is absent in two situations, and both are ordinary: a fresh environment before the first seed, and an environment after teardown, since teardown removes the attribute last. A report run before seeding anything and a teardown run twice both used to hit it.

1
2
3
4
5
6
7

$attributeName = $script:PingOneSeedAttributeName
$attributeExists = $false
foreach ($schema in (Invoke-PingOneRequest -Method GET -Path 'schemas' -Paginate -Connection $Connection)) {
    $match = @(Invoke-PingOneRequest -Method GET -Path "schemas/$($schema.id)/attributes" -Paginate -Connection $Connection |
            Where-Object { $_.name -eq $attributeName })
    if ($match) { $attributeExists = $true; break }
}

Asking the schema is deterministic and needs no guess about which error code means “no such attribute”. Two extra calls buy a discovery path that works on an empty environment.

The filter that follows carries a guard that looks paranoid until you read the comment:

1
2
3
4

# Nulls filtered out explicitly, so no path can ever hand teardown a user with no
# id - which would become "DELETE users/" with nothing after the slash.
$tagged = @(Invoke-PingOneRequest -Method GET -Path 'users' -Paginate -Connection $Connection `
        -Query @{ filter = $filter } | Where-Object { $null -ne $_ -and $_.id })

An ignored error in this provider’s request helper returns $null, and @($null) in PowerShell is a one-element array holding nothing. Unguarded, a foreach over it runs once, with a user object that has no id, and builds a delete path ending in a slash. I had no wish to find out what that would hit on a live environment.

Nineteen designed users

The designed users carry their real given and family names. Only their usernames and emails take the prefix, and the emails sit under a reserved domain that can’t receive mail. Each row has a job.

A disabled user still holds every group membership, because disabling an account removes it from nothing and membership reports routinely count it as active. One user has the MFA-enabled flag off where most of the directory has it on, with no MFA device or policy created behind it, since the flag is a field and enrollment is a separate thing. A badge value is one PingOne enforces as unique, so a second user with the same one is refused outright.

Nine of them carry the shared set of hard names, on plain ASCII usernames. They’re the only coverage this module has for string handling going wrong, and the first PingOne seed showed why they stay.

STRING or JSON, so the boolean is text

PingOne refuses a custom attribute of any type but STRING or JSON. Ask for BOOLEAN and you get INVALID_DATA on type: must be STRING or JSON.

The contractor flag is therefore the text "true" or "false", and that puts a nasty row in the seed data for free. In PowerShell the string "false" is truthy. Any code that does this:

1

if ($user.contractor) { ... }

reads every user in the directory as a contractor, including the twelve whose flag says otherwise. I kept the attribute in the seed because that mistake is easy and common, and it produces a confidently wrong report with no error at all.

One of the five attributes is declared unique, and PingOne enforces it properly: creating a second user with the same badge value is refused with INVALID_DATA. Another is multivalued, and one is JSON, so an export that flattens structured values has something to flatten wrongly.

Two user states are absent because the management API can’t set them. verifyStatus is NOT_INITIATED on every user it creates, and an account lock is a separate operation rather than a field on the user.

Groups that disagree with the data

There are eleven groups, and the interesting ones are the disagreements.

The Contractors group holds the Contractors population, and the contractor attribute disagrees with it: twelve members are flagged false, and one partner flagged true sits outside the group entirely. Any report that treats group membership and an attribute as two views of one fact gets two different answers.

A population-scoped group refuses a user from outside its population, a constraint most group code has never met. The static chain runs three deep, so membership has to resolve transitively, and memberOfGroups on a group is itself transitive: Team Platform, nested only in Engineering, reports both Engineering and All Staff as parents.

Then there are two dynamic groups, the part of the design I like most. A dynamic group reports zero members immediately after a seed, because PingOne evaluates filters asynchronously, and that’s the state the row exists to capture. Beside it sits a second dynamic group whose filter is valid and matches nobody, and from the outside the two look identical. Telling a group that hasn’t been evaluated yet from one that will never match anybody is a real problem, and the seed puts both in front of you.

Two APIs and six clients

Above the directory sit the things that consume it. Two custom resources stand in for internal APIs, each with its own audience and token lifetime: an orders API with read, write and admin scopes on a one-hour token, and a reports API with a single read scope on fifteen minutes. A report that assumes one lifetime across an environment has two to reconcile.

Six applications cover the shapes an inventory has to handle. An OIDC web client with a secret, granted a scope on the orders API and open to all staff. A second web client restricted to the one group with a single member and granted the reports scope, the pairing an access review should be able to trace end to end. A single-page client with no secret. A native client with a custom-scheme redirect and a refresh token. A SAML application that is issued no access tokens and so holds no scopes, which trips up anything expecting every application to have some.

The sixth is a disabled OIDC client with no group and no scopes. A disabled application still appears in every listing, and an inventory that counts applications without reading their state overstates what the environment can do by one.

Redirect and ACS URLs are written under the connection’s email domain, so nothing in the seed points at a host somebody owns.

The default population, platform objects and PKCE

The provider has no parameter for any of these three.

No seeded population ever becomes the environment’s default, since the default decides where every user created without a population lands. No platform application or built-in resource is created, edited or deleted. And no public client is created without PKCE:

1
2

# A public client runs safely only with PKCE, so it is never seeded without it.
if ($row.TokenAuth -eq 'NONE') { $body['pkceEnforcement'] = 'S256_REQUIRED' }

Tests assert each of the three, so if somebody adds a -Default or a -DisablePkce switch as a convenience, the suite catches it as a regression. I use the same pattern for the Entra provider’s Conditional Access rule and the Authentik provider’s flows. A parameter that was never written is the one guard a hurried afternoon can’t talk its way past.

Teardown in the order PingOne allows

Applications, resources, users, groups, populations, and custom attributes last. The platform dictates each position: an attribute can’t go while any user still holds a value in it, and a population can’t go while it holds a user.

That ordering also explains a parameter that would otherwise look inconsistent. -Keep Users also keeps the populations and the attributes, because PingOne deletes only an empty population and an unreferenced attribute. Keeping the users keeps the three things that depend on them.

The confirmation is asked once, in the body of the command, where a refusal stops the work. A session that can’t answer the prompt counts as a refusal, so an unattended teardown has to pass -Force. -WhatIf always wins: -Force -WhatIf removes nothing and prints a line for every object it would have removed. Running teardown again on an already-clean environment removes nothing and reports no error, the property the schema check above exists to protect.

Comparing on the parts

Compare-TestEnvironment matches the people in two connected providers by their shared login key, and PingOne is where that ran into something none of the others had.

PingOne keeps a given name and a family name, and no display name at all. Composing one would be easy and wrong:

1
2
3
4

# PingOne keeps a given name and a family name and no display name, so the display name is
# left empty and the comparison falls back to the parts rather than composing a name the
# environment never stored: a composed one would put the family name last for a person whose
# name puts it first.

The nine people in other writing systems are in this directory too, and at least one of them writes the family name first. A comparison that composed given + " " + family would report a mismatch against the directory that stored the real display name, and the mismatch would come from the comparison itself, with both directories correct.

Leaving the display name empty and comparing the parts gives the right answer for everybody. When I ran it against the lab Entra tenant and this sandbox from one session, 329 people matched on the login key, 328 names compared as given name and surname with none differing, and one account existed only in PingOne.

Verifying the environment

Test-TestEnvironment reads the populations, users, groups, applications, resources and attributes the module owns and compares them with the seed files: every seeded name present, nothing owned that the data doesn’t describe, every given and family name matching by codepoint, and every group a user row lists holding that user.

Names are compared ordinally because -eq treats a decomposed and a precomposed name as equal while the wire treats them as different. A mangled name is the fault this provider shipped once, and an equality check would have passed against the wrong string. Usernames are compared case-insensitively, since PingOne folds them.

Memberships are read one user at a time. The dynamic group’s filter and the nested chain both add members the data never lists, so a membership only fails when one the data names is absent. -SkipMembership drops those reads, at one call per seeded user that has any.

Repair-TestEnvironment re-runs the seed steps that own whatever failed, which here usually means re-running one object type in place of four minutes of environment.

Commands

1
2
3
4
5
6
7
8
9

$secret = Read-Host 'Worker secret' -AsSecureString
Connect-TestEnvironment -Provider PingOne -EnvironmentId <sandbox> `
    -ClientId <worker> -ClientSecret $secret -AuthEnvironmentId <administrators> -SaveSecret

New-TestEnvironment -WhatIf
New-TestEnvironment -ShowProgress
Get-TestEnvironmentReport
Test-TestEnvironment
Remove-TestEnvironment -Force

Tools\New-PingOneTestSeedData.ps1 generates the three hundred bulk users. It’s an authoring tool the module never calls, reading the same shared people the other five providers read and assigning populations, titles, departments and group memberships here. The committed CSVs are the artifact, and the drift test every provider carries fails if they diverge from what the tool produces. The nineteen designed rows live in the tool verbatim and are never generated.

The worker needs Identity Data Admin, Environment Admin and Client Application Developer on the environment being seeded. Nothing else gets installed: Windows PowerShell 5.1 or PowerShell 7, and the module.

I verified it against a North America trial sandbox from both editions. The other four regions haven’t been exercised, and I would rather say so than imply otherwise.

The provider’s page, with the full inventory and every behavior above in longer form, is at Providers/PingOne/README.md.

That made six providers: Active Directory, Entra, Okta, Authentik, FreeIPA and this one. Six directories share one set of people and the same nineteen names, so a script that reconciles two of them has two populations that correspond.

Writing the sixth provider from scratch did more for the other five than anything else I worked on. It found the encoding fault at the top of this post, it produced the shared request helper, and it put the differences between the six side by side where I could no longer ignore them, which is how most of the shared surface came to exist.

Related reading

• A Role With Nothing but a Name: Seeding OneLogin
• Seeding FreeIPA, Where Teardown Starts Revoking
• Test Data That Breaks Scripts on Purpose