DudeWheresMyLogs: An Azure Diagnostic Settings Audit Tool

The Backstory

A while back I was working at a place that had several Azure tenants and dozens of subscriptions across each of them. There was already Azure Policies in place as a standard way to deploy diagnostic settings across resources, which is good, that’s the right way to do it.

The problem was that over time, different teams had requested their own Log Analytics Workspaces for whatever reason. So new policies were added, they did their thing, and now resources were having their diagnostic settings sent to multiple workspaces. Nobody went back and cleaned up the old settings, nobody checked if anyone was actually using those destinations, and the bill just kept growing.

On top of that, Sentinel was enabled on a bunch of these workspaces. Instances with default data connectors turned on, no analytics rules, no one running queries, no dashboards, no investigations. Just added ingestion costs piling up. Millions of dollars a year, all in the name of CoMpLiAnCe.

The Audit

I started poking around the Sentinel instances first, since that was the obvious waste. Going through each workspace, checking if anyone had actually done anything with it. Most of them were ghost towns. Easy enough to fix.

The diagnostic settings were a bigger headache though. Azure lets you stack multiple diagnostic settings on a resource - each one can point to a Log Analytics workspace, Storage Account, Event Hub, or partner solution. That’s fine if you’re doing it on purpose, like sending security logs to your SIEM and operational metrics to storage account. But when it’s many policies from many teams all deploying the same settings to many workspaces, you’re just multiplying ingestion costs for no reason.

There were thousands of resources across dozens of subscriptions, too many to review. So I wrote a Python script.

The Script

The first version was pretty rough. Used the Azure SDK to list resources, pull diagnostic settings for each one, and flag anything that had the same destination type more than once. Dumped it all to a CSV to filter and review.

It got the job done. We found the overlap, consolidated workspaces, and turned off the Sentinel instances nobody was using. The script did what I needed and I kind of forgot about it.

The Tool

After I moved on from that role, I came back to the idea. I figured I probably wasn’t the only person dealing with this - Azure Policy making it really easy to deploy diagnostic settings everywhere means it’s also really easy to end up with duplicates when multiple teams or initiatives are involved.

I cleaned the script up. Rewrote the scanning to run in parallel so it wouldn’t take too long on large environments. Added retry logic for ARM throttling. Built an HTML report with collapsible sections, summary cards, and a destination map that shows which resources are streaming where. Something you could actually hand to someone and say “here’s the problem.” It evolved into a diagnostic health check tool, something you can use to quickly scan your Azure environment and determine your log health at a glance.

So, here’s DudeWheresMyLogs.

What It Does

The tool scans your Azure subscriptions and checks every resource for diagnostic settings. For each one you get:

• Missing - no diagnostic settings at all
• Duplicates - multiple settings shipping to the same destination type (like two different Log Analytics workspaces)
• Healthy - configured, no duplicates
• Not Supported - resource types that don’t support diagnostic settings

It also checks storage account sub-services (blob, queue, table, file) separately, since those have their own diagnostic settings apart from the parent account.

Usage

Install it in a venv and authenticate however you normally do, az login, managed identity, service principal, whatever DefaultAzureCredential supports:

git clone https://github.com/KernelCaleb/DudeWheresMyLogs.git
cd DudeWheresMyLogs
python3 -m venv .venv
source .venv/bin/activate
pip install .

Then run it:

# Interactive subscription picker
DudeWheresMyLogs

# Scan all accessible subscriptions
DudeWheresMyLogs -a

# Scan specific subscriptions
DudeWheresMyLogs -s <subscription-id> -s <subscription-id>

# CSV instead of HTML
DudeWheresMyLogs -f csv

# More parallel workers (default is 10)
DudeWheresMyLogs -w 20

The HTML report is self-contained, just inline CSS and native <details> elements. Open it in a browser, share it, attach it to a ticket, store it locally and never look at it, whatever.

How Duplicate Detection Works

This was the whole reason the tool exists, so it’s worth explaining. It looks at how many diagnostic settings per resource point to the same destination type. A resource sending logs to a Log Analytics workspace and a Storage Account is probably fine, those are different destination types serving different purposes. But if a resource is sending logs to two or three different Log Analytics workspaces, that’s likely duplicate settings from overlapping policies and your org is paying for that extra ingestion.

The tool flags those and shows you which settings are causing it, so you can figure out what to clean up.

Wrapping Up

Diagnostic settings through Azure Policy are one of those set-and-forget things in Azure. Someone writes the policy, it rolls out, resources get their settings, and then nobody thinks about it again until the bill gets weird, if you’re like my last org nobody looks at the bill either. And if multiple teams did that at different times, you end up with a mess.

Azure doesn’t really give you a good way to see all of this at once. There’s no portal view that says “hey, this resource is shipping logs to three places.” You have to go resource by resource or build something yourself.

So that’s what this is. Check it out on GitHub if you’re dealing with something similar.

Hope it helps and thanks for stopping by.