SteadyLatch

Guides · Data for reports

PowerShell: Fail When an API Property Is Null, Empty or Missing

Published · Tested with PowerShell 7.6.6 and Windows PowerShell 5.1 (build 26100) on Windows 11 Pro

A script reads orders from an API and builds a report. One order has no customer, another has null for the total, a third has an empty string. The script finishes, the report has blank cells and zeros, and a real order of 0 looks exactly like an order whose total never arrived. Nothing in PowerShell complains. This guide reproduces that against a small local server, shows what null, an empty string, an empty list and a missing property do in PowerShell 7 and Windows PowerShell 5.1, and ends with a check that names the property and the record and exits with code 1.

Before you start

The report that looks fine

The server returns this list (shortened to the properties that matter; each order also has currency and, except 1008, a lines list):

The eight orders the test server returns
idcustomertotallines
1001"Ana Example"120.51 line
1002null801 line
1003""351 line
1004not sent451 line
1005"Ben Sample"null1 line
1006"Cleo Test"01 line
1007"Dan Demo"60[]
1008"Eva Sample"""not sent

Order 1006 is a real zero (a free replacement, say). The script that builds the report is the one most of us would write:

[Threading.Thread]::CurrentThread.CurrentCulture = 'en-US'
$r = Invoke-RestMethod http://127.0.0.1:8765/orders
$rows = foreach ($o in $r.data) { [pscustomobject]@{ Id = $o.id; Customer = $o.customer; Total = [decimal] $o.total; Lines = @($o.lines).Count } }
$rows | Format-Table -AutoSize | Out-String -Width 120
"Sum of Total: " + ($rows | Measure-Object Total -Sum).Sum
$rows | Export-Csv naive.csv -NoTypeInformation
"`$? = $?"
Get-Content naive.csv

In PowerShell 7.6.6:

  Id Customer      Total Lines
  -- --------      ----- -----
1001 Ana Example 120.500     1
1002              80.000     1
1003              35.000     1
1004              45.000     1
1005 Ben Sample    0.000     1
1006 Cleo Test     0.000     1
1007 Dan Demo     60.000     0
1008 Eva Sample    0.000     1
Sum of Total: 340.5
$? = True
"Id","Customer","Total","Lines"
"1001","Ana Example","120.5","1"
"1002",,"80","1"
"1003","","35","1"
"1004",,"45","1"
"1005","Ben Sample","0","1"
"1006","Cleo Test","0","1"
"1007","Dan Demo","60","0"
"1008","Eva Sample","0","1"

Windows PowerShell 5.1 printed the same values (the table columns are narrower) and the same CSV file. What the output shows:

Why strict mode is not enough

The usual advice is Set-StrictMode -Version Latest, so that a missing property is an error. It helps with exactly one of the cases. The same loop, with strict mode and $ErrorActionPreference = 'Stop':

[Threading.Thread]::CurrentThread.CurrentCulture = 'en-US'; [Threading.Thread]::CurrentThread.CurrentUICulture = 'en-US'
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
$r = Invoke-RestMethod http://127.0.0.1:8765/orders
foreach ($o in $r.data) {
    "{0}: customer=[{1}] total=[{2}]" -f $o.id, $o.customer, $o.total
}
1001: customer=[Ana Example] total=[120.5]
1002: customer=[] total=[80]
1003: customer=[] total=[35]
The property 'customer' cannot be found on this object. Verify that the property exists.

Windows PowerShell 5.1 gave the same four lines (the last one inside its longer error block), and both versions ended with exit code 1. Orders 1002 and 1003 went through: null and an empty string are not missing properties, and the null total of order 1005 would have gone through too. The script also stops at the first missing property, so you learn about 1004 and not about 1008. And the protection lives in the code that reads the data: each new property you read is one more place where you have to remember what a missing value would mean.

Four states, and a zero that is a value

The check has to ask separate questions: is the property there, is it null, is it empty. In an object from ConvertFrom-Json or Invoke-RestMethod, $record.PSObject.Properties['customer'] answers the first one: it is $null when the property does not exist, and it is a property whose .Value may be $null when the API sent null. For each required property the check names one of these states:

The property lines shows why PowerShell hides the difference: a property sent as [] reads back as an empty array in both versions (Object[], count 0). It is not $null ($null -eq $e.lines is False), but if ($e.lines) is false, the same as for null.

The check

Save this as Test-ApiRecord.ps1. Test-ApiRecord checks one record against the list of required properties and their types and returns one line per problem. Test-ApiResponse checks every record and puts the record's id and its position in the list in front of each line:

# Test-ApiRecord.ps1: check that the required properties of an API record are really there.
# Tells absent, null, empty and zero apart. Windows PowerShell 5.1 and PowerShell 7.
# Dot-source it, then call Test-ApiRecord (one record) or Test-ApiResponse (a list of records).

function Get-PropertyState {
    # absent | null | empty | zero | value. Works on objects from ConvertFrom-Json and on hashtables (-AsHashtable).
    param($Record, [string] $Name)
    if ($Record -is [System.Collections.IDictionary]) {
        if (-not $Record.Contains($Name)) { return 'absent' }
        $value = $Record[$Name]
    }
    else {
        $property = $Record.PSObject.Properties[$Name]
        if ($null -eq $property) { return 'absent' }
        $value = $property.Value
    }
    if ($null -eq $value) { return 'null' }
    if ($value -is [string]) { if ([string]::IsNullOrWhiteSpace($value)) { return 'empty' } else { return 'value' } }
    if ($value -is [System.Collections.IList]) { if ($value.Count -eq 0) { return 'empty' } else { return 'value' } }
    if ($value -is [ValueType] -and $value -isnot [bool] -and $value -eq 0) { return 'zero' }
    return 'value'
}

function Test-ApiRecord {
    # One line per problem: "<property> is absent", "is null", "is empty" or "is <type>, expected <type>".
    # A 0 is a value for a number, so it is not a problem; an empty string or empty list never is a value.
    param(
        $Record,
        [Parameter(Mandatory)] [System.Collections.IDictionary] $Required   # name = 'string' | 'number' | 'array'
    )
    foreach ($name in $Required.Keys) {
        $state = Get-PropertyState $Record $name
        if ($state -in 'absent', 'null', 'empty') { "$name is $state"; continue }
        if ($Record -is [System.Collections.IDictionary]) { $value = $Record[$name] } else { $value = $Record.PSObject.Properties[$name].Value }
        $ok = switch ($Required[$name]) {
            'string' { $value -is [string] -or $value -is [datetime] }   # PowerShell 7 reads timestamp strings as dates
            'number' { $value -is [ValueType] -and $value -isnot [bool] -and $value -isnot [datetime] }
            'array'  { $value -is [System.Collections.IList] }
        }
        if (-not $ok) { "$name is $($value.GetType().Name), expected $($Required[$name])" }
    }
}

function Test-ApiResponse {
    # Checks every record. Returns the problems as "record <id> (item n of N): <problem>".
    param(
        [Parameter(Mandatory)] [AllowEmptyCollection()] [object[]] $Records,
        [Parameter(Mandatory)] [System.Collections.IDictionary] $Required,
        [string] $KeyName = 'id'
    )
    for ($i = 0; $i -lt $Records.Count; $i++) {
        $key = if ((Get-PropertyState $Records[$i] $KeyName) -in 'value', 'zero') { "$KeyName=" + $(if ($Records[$i] -is [System.Collections.IDictionary]) { $Records[$i][$KeyName] } else { $Records[$i].$KeyName }) } else { "(no $KeyName)" }
        foreach ($problem in Test-ApiRecord $Records[$i] $Required) {
            "record $key (item $($i + 1) of $($Records.Count)): $problem"
        }
    }
}

The caller that reads the API and decides the exit code is a second file, check-orders.ps1. It uses three exit codes: 0 when every record is valid, 1 when some record is not, and 2 when the response cannot be used at all (HTTP error, no connection, or no data list):

# check-orders.ps1: read the orders from the API and stop when a required property is absent, null or empty.
# Exit codes: 0 all records valid, 1 invalid records, 2 the response cannot be used.
param([string] $Url = 'http://127.0.0.1:8765/orders')
. "$PSScriptRoot\Test-ApiRecord.ps1"
$required = [ordered]@{ id = 'number'; customer = 'string'; total = 'number'; currency = 'string'; lines = 'array' }

try { $response = Invoke-RestMethod $Url -TimeoutSec 20 }
catch { Write-Error "Could not read ${Url}: $($_.Exception.Message)"; exit 2 }
if ($response -isnot [pscustomobject] -or $null -eq $response.PSObject.Properties['data'] -or $response.data -isnot [System.Collections.IList]) {
    Write-Error "The response from $Url has no 'data' list."; exit 2
}
$records = @($response.data)
if ($records.Count -eq 0) { Write-Error "The response from $Url has 0 records."; exit 1 }

$problems = @(Test-ApiResponse $records $required)
if ($problems.Count -gt 0) {
    $problems | Select-Object -First 20 | ForEach-Object { Write-Host $_ }
    Write-Error "$($problems.Count) problem(s) in $($records.Count) records."; exit 1
}
"OK: $($records.Count) records, all required properties present"
exit 0

Run it

The test server, server.ps1. Start it in a second window with pwsh -File server.ps1. It serves the list above on /orders and two clean orders (1001 and 1006) on /orders-ok:

# Local test server: GET /orders returns eight invented orders with the defects; GET /orders-ok returns two clean ones.
param([int] $Port = 8765)
$bad = @'
{"data":[
 {"id":1001,"customer":"Ana Example","total":120.5,"currency":"EUR","lines":[{"sku":"A-100","qty":2}]},
 {"id":1002,"customer":null,"total":80,"currency":"EUR","lines":[{"sku":"B-200","qty":1}]},
 {"id":1003,"customer":"","total":35,"currency":"EUR","lines":[{"sku":"A-100","qty":1}]},
 {"id":1004,"total":45,"currency":"EUR","lines":[{"sku":"C-300","qty":3}]},
 {"id":1005,"customer":"Ben Sample","total":null,"currency":"EUR","lines":[{"sku":"B-200","qty":1}]},
 {"id":1006,"customer":"Cleo Test","total":0,"currency":"EUR","lines":[{"sku":"A-100","qty":1}]},
 {"id":1007,"customer":"Dan Demo","total":60,"currency":"EUR","lines":[]},
 {"id":1008,"customer":"Eva Sample","total":"","currency":"EUR"}
]}
'@
$ok = '{"data":[{"id":1001,"customer":"Ana Example","total":120.5,"currency":"EUR","lines":[{"sku":"A-100","qty":2}]},{"id":1006,"customer":"Cleo Test","total":0,"currency":"EUR","lines":[{"sku":"A-100","qty":1}]}]}'
$l = [System.Net.HttpListener]::new()
$l.Prefixes.Add("http://127.0.0.1:$Port/")
$l.Start()
"listening $Port"
while ($l.IsListening) {
    $c = $l.GetContext()
    $body = switch ($c.Request.Url.AbsolutePath) { '/orders' { $bad } '/orders-ok' { $ok } default { $null } }
    if ($null -eq $body) { $c.Response.StatusCode = 404; $body = '{"error":"not found"}' }
    $bytes = [Text.Encoding]::UTF8.GetBytes($body)
    $c.Response.ContentType = 'application/json; charset=utf-8'
    $c.Response.OutputStream.Write($bytes, 0, $bytes.Length)
    $c.Response.Close()
}

The check against /orders, with the same lines in PowerShell 7.6.6 and in Windows PowerShell 5.1 (the final error line is longer in 5.1, which adds the script path and a category), and exit code 1 in both:

record id=1002 (item 2 of 8): customer is null
record id=1003 (item 3 of 8): customer is empty
record id=1004 (item 4 of 8): customer is absent
record id=1005 (item 5 of 8): total is null
record id=1007 (item 7 of 8): lines is empty
record id=1008 (item 8 of 8): total is empty
record id=1008 (item 8 of 8): lines is absent
Write-Error: 7 problem(s) in 8 records.

Seven problems in eight records. Orders 1001 and 1006 are not in the list: the zero is a value. Against /orders-ok the script printed OK: 2 records, all required properties present and exited with 0. Against a path the server does not have (/missing), it exited with 2; PowerShell 7 said Could not read http://127.0.0.1:8765/missing: Response status code does not indicate success: 404 (Not Found)., and Windows PowerShell 5.1 gave the same exit code with its own message text. With nothing listening on the port, the exit code was 2 too. In a scheduled job, let that code reach the task; the Task Scheduler guide linked above shows how.

Other shapes, and hash tables

A second list, run through Test-ApiResponse with customer, total and lines required, shows what the type check adds. In order 1 the customer is " " and the total is the text "120.5"; in order 2 the customer is 0, the total is true and lines is an object; orders 3 and 4 have a total of 0 and 0.0:

record id=1 (item 1 of 4): customer is empty
record id=1 (item 1 of 4): total is String, expected number
record id=2 (item 2 of 4): customer is Int64, expected string
record id=2 (item 2 of 4): total is Boolean, expected number
record id=2 (item 2 of 4): lines is PSCustomObject, expected array

Windows PowerShell 5.1 gave the same lines except for one type name: Int32 for the number 0 where PowerShell 7 said Int64. Orders 3 and 4 passed in both: a number that is 0 is a value.

PowerShell 7 can also read the JSON as hash tables. ConvertFrom-Json “Converts the JSON to a hash table object” with -AsHashtable, which “was introduced in PowerShell 6.0” (Microsoft Learn, ConvertFrom-Json), so Windows PowerShell 5.1 does not have it: in our run it stopped with A parameter cannot be found that matches parameter name 'AsHashtable'. With a hash table, $h['zzz'] is $null for a missing key just as for a null value, and $h.ContainsKey('zzz') is the question to ask (False for a missing key, True for a key whose value is null in our test). Test-ApiRecord handles both kinds of record; the hash-table run gave the same problems as above, with lines is OrderedHashtable for the object.

One difference to know about: hash-table keys keep their case (ContainsKey('customer') was False for a key sent as Customer), while the properties of an object from plain ConvertFrom-Json do not (PSObject.Properties['customer'] found Customer). If your API may change the case of a name, the two readers answer differently.

Limits

Just want the scripts? The Scripts Pack · €12 has the code of this guide and the other guides ready to run, with examples and tests. The code on this page stays free.

The code on this page was written for this guide and tested on the versions above. The test scripts also set the culture and the UI culture to en-US, so that messages and numbers came out in English. The data is invented.