Reading Time: 5 minutes

Repo Link: Link

Introduction

PingCastle is one of the most well known tools for monitoring the security of your Active Directory domain. There is plenty of documentation around for it and is one of the quickest ways of prioritising quick win security improvements. There are various licence levels, but the Free version is the one we are focusing on currently.

A recurring issue is that organisations run PingCastle on an irregular basis (often run via a third party consultancy), but there is very limited trend analysis or forward thinking what the data may mean. As it is a manual task, it inevitably gets forgotten or missed. The most effective way to address this is automation, both of the running of PingCastle, and the transmission of results. We can pair this with (internal) instances of AI LLMs to track trends, analysis and so forth, especially when paired with secondary tools such as Dump-DomainInfo.ps1

Running PingCastle via SchTask

A simple way of making sure that you and your team are able to track the output of PingCastle trends is to simply run it via a Scheduled Task on an appropriately located server. PingCastle only needs Domain User privileges to run, and we can protect those credentials using DPAPI.

First, create the credential object. The password will be DPAPI encrypted to whichever user this script is run as.

$SecurePassword = ConvertTo-SecureString "Ad@udit2025!" -AsPlainText -Force
$Credential = New-Object System.Management.Automation.PSCredential("domain.local\adaudit", $SecurePassword)
$Credential | Export-Clixml -Path "C:\Scripts\PingCastleCred.xml"

Next create a simple Powershell script called Run-PingCastle.ps1 (GitHub) that will run PingCastle with the correct arguments needed to give coverage. You’ll need to update the path to the PingCastle binary to reflect where you placed it on disk. It is important this script is stored in a location that is only accessible by authorised users. You can change the -Method parameter to Slack or MSTeams as fits your use case. The secondary script Compare-PingCastle.ps1 that is called by the first script is also hosted here. These could obviously be combined into a single script, but are kept separate here so you can see the sequential process, or just use certain scripts if you don’t care about the comparisons.

$timestamp = Get-Date -Format "yyyyMMdd-HHmmss"
$reportDir = "C:\scripts\PingCastle\reports"
$logDir = "C:\scripts\PingCastle\logs"

$transcriptLog = Join-Path $logDir "RunPingCastle-$timestamp.transcript.txt"
$stdoutLog = Join-Path $logDir "PingCastle-$timestamp.stdout.txt"
$stderrLog = Join-Path $logDir "PingCastle-$timestamp.stderr.txt"

# Create directories
foreach ($dir in @($reportDir, $logDir)) {
    if (-not (Test-Path $dir)) {
        New-Item -ItemType Directory -Path $dir -Force | Out-Null
    }
}

Start-Transcript -Path $transcriptLog -Append

Write-Host "[$(Get-Date)] Starting Run-PingCastle.ps1"
Write-Host "Running as: $env:USERNAME"
Write-Host "Setting current directory to: $reportDir"

# PingCastle doesn't support specifying an output dir, so change here.
Set-Location -Path $reportDir

Write-Host "Launching PingCastle..."
$process = Start-Process -FilePath "C:\scripts\PingCastle\PingCastle.exe" `
  -ArgumentList @(
      "--healthcheck",
      "--level", "Full",
      "--no-enum-limit",
      "--datefile"
  ) `
  -RedirectStandardOutput $stdoutLog `
  -RedirectStandardError $stderrLog `
  -NoNewWindow `
  -Wait `
  -PassThru

Write-Host "PingCastle exited with code: $($process.ExitCode)"

Stop-Transcript
 
# Run secondary PowerShell script that does analysis and webhook.

powershell.exe -ExecutionPolicy Bypass -File "C:\Scripts\Compare-PingCastleScores.ps1" -Method Slack

Now create a Scheduled Task that will run your PS1 on whichever schedule you feel is appropriate, you will need to have permissions to create Scheduled Tasks on that machine. In this example, I am logged in as domain.local\adaudit. Remember that passwords passed on the command line may be ingested into SIEMs or EDR tooling.

$action = New-ScheduledTaskAction -Execute "powershell.exe" -Argument "-File 'C:\Scripts\Run-PingCastle.ps1'"
$trigger = New-ScheduledTaskTrigger -Weekly -DaysOfWeek Monday -At 7:00am
Register-ScheduledTask -Action $action -Trigger $trigger -TaskName "WeeklyPingCastleWithAlert" -Description "Runs PingCastle and sends webhooks"

This will take care of PingCastle running, as well as calling the Compare-PingCastleScores.ps1 secondary script which does the heavy lifting (details below).

Parsing Output

We need to correctly parse through the output XMLs to determine if the Risk Score is increasing or decreasing, and send Slack notifications so that we can keep on top of any issues. We can easily do this using Powershell and it’s XML parsing abilities.

The full script is located on GitHub here, with the functionality summarised below.

• Retrieves the webhook URL from DPAPI.
• Creates an archive subdirectory for reports older than six weeks and moves reports older into that subdirectory.
• Checks the last two XML reports for Risk Scores and the top three Risk Factors.
• Sends webhooks to Slack / MSTeams with suitable text and links to the reports.

The next step is to communicate the output to the security team. The webhook URL for the script to send to should also be securely stored within DPAPI.

$webhookUrl = "https://slack.com/webhook/123456abcd"  # Or MSTeams webhook URL
$secure = $webhookUrl | ConvertTo-SecureString -AsPlainText -Force
$secure | ConvertFrom-SecureString | Set-Content "C:\Scripts\PingCastleWebhook.txt"

Sample Output

You should see output similar to the below (lab data).

Note: The link to the filesystem location doesn’t render in the message as I am using MacOS rather than Windows.

AI Analysis

As AI is the current solution-to-all problems, if you have a suitable and authorised LLM you wish to supply your PingCastle outputs to, you can also get some introductory analysis and trends. You could obviously do this manually just by dragging and dropping the XML / HTML into the same chat each week, but that would be an extremely manual process.

OpenAI have a Python based SDK that we can use, which can be installed via pip (after installing Python whereever you want the script to run from).

pip install openai requests

It is also necessary to create a thread, which our later Python script will reference. This can be done with the below code (or GitHub here), which will write the thread ID to a text file. You need to create an API key in OpenAI Platform (and fund some tokens).

import openai
import os

openai.api_key = os.getenv("OPENAI_API_KEY") or input("Enter your OpenAI API Key: ") or "hardcode_key_here"

# Create a thread
thread = openai.beta.threads.create()
thread_id = thread.id

print(f"Your thread ID is: {thread_id}")

with open("C:\\Scripts\\pingcastle_thread.txt", "w") as f:
    f.write(thread_id)

Once done, create an Assistant within Platform and supply it with the prompt that is listed in GitHub here. I used gpt-4-turbo with File Search enabled. (This is also why we use a HTML rather than XML upload, as File Search supports it directly).

Credential Storage

Again, we’ll need to securely store API tokens and other credential material, as shown below. You could also export environment variables, or simply hardcode the details into this script, run it, then delete the script. Again, SIEM or EDR may store these variables.

$secrets = @{
    OPENAI_API_KEY   = Read-Host -Prompt "Enter your OpenAI API key"
    ASSISTANT_ID     = Read-Host -Prompt "Enter your Assistant ID (e.g., asst_abc123)"
    THREAD_ID        = Read-Host -Prompt "Enter your persistent Thread ID (e.g., thread_xyz456)"
    ANALYSIS_WEBHOOK = Read-Host -Prompt "Enter webhook URL for AI analysis response"
}

$secure = $secrets | ConvertTo-Json | ConvertTo-SecureString -AsPlainText -Force
$secure | ConvertFrom-SecureString | Set-Content "C:\Scripts\OpenAISecrets.sec"
Write-Host "OpenAI secrets securely stored."

Scheduled Task to Run Analysis

$action = New-ScheduledTaskAction -Execute "python.exe" -Argument "C:\Scripts\Analyse_PingCastle_OpenAI.py"
$trigger = New-ScheduledTaskTrigger -Weekly -DaysOfWeek Monday -At 7:45am
Register-ScheduledTask -Action $action -Trigger $trigger -TaskName "PingCastleAIAnalysis" `
  -User "domain.local\\adaudit" -Password "Ad@udit2025!" 
  -Description "Uploads latest PingCastle HTML reports to OpenAI and sends analysis to webhook"

Python Script

The full script can be seen on GitHub here, but the high level functionality is:

• Retrieve secrets needed at runtime.
• Retrieve the previous four HTML reports and upload to OpenAI.
• Retrieve analysis and send to webhook.

You can customise the prompt as needed to help fine tune the output.

Note: As of June 2025, OpenAI assistants do not support continual memory for particular threads. This means that each message to the thread (and thus the assistant) does not have any knowledge of what has gone before. Hence the need to upload the previous week’s HTML reports on each run. Once in-thread-memory is deployed by OpenAI, this won’t be needed.

Sample AI Analysis Output

You should see analysis similar to the below.