My First 3 Splunk Queries: Finding the Loudest IP on the Network

I loaded 109,864 events into Splunk and went looking for the noisiest host on the network. What I found taught me more about SOC work than the search syntax did — and the syntax turned out to be something I already knew.

Last time, I fought my way through installing Splunk on a Mac — four errors, an OS mismatch, and a forced restart. That post ended with a running SIEM and an empty search bar. This is what happened when I finally typed something into it.

Spoiler: my first real query found a suspect. My second and third proved the suspect was innocent. Somewhere in the middle, I realized I'd been writing this language for ten years without knowing its name.

First, why any of this matters

Before the searches — a quick word on why Splunk keeps showing up in the job descriptions I'm targeting.

Splunk is the market-leading SIEM, and a SIEM is the tool a SOC analyst actually sits in front of all day. When 109 million logs a day pour in from firewalls, servers, endpoints, and cloud services, no human triages that by hand. The SIEM funnels it — ingest, index, correlate — down to a handful of things worth a person's attention. The analyst's job is to ask the right questions of that funnel and interpret what comes back.

That's why "Splunk" or "SIEM experience" sits on nearly every SOC Analyst and Security Monitoring req. It's not a nice-to-have; it's the daily instrument. Learning to search it is learning the core motion of the job. So let's do the core motion.

Getting data in

You can't search an empty SIEM. Splunk ships a free tutorial dataset — web access logs from a fictional shop called Buttercup Games, plus some vendor sales logs — so I uploaded that. Settings → Add Data → Upload, point it at the zip, set the host to pull from the file path, and submit.

109,864 events. Here's the first thing that clicked, straight out of my data-analyst brain: look at the left rail after the data lands. Splunk had already pulled clientip, status, bytes, host — structured fields — out of raw log text. Each event is a row. Each field is a column. The indexes I'd installed were, functionally, database tables. I hadn't defined a schema. Splunk extracted the fields at search time from the raw text. The mental model was instantly familiar.

Query 1: Who's the loudest IP on the network?

The classic first threat-hunting question: which host is generating the most traffic? Here's what I typed:

source="tutorialdata.zip:*" | stats count by clientip | sort -count

And there it was — 87.194.216.51, with 1,036 requests, well ahead of the next host at 736. Splunk crunched 109,864 events into a ranked table of 182 unique IPs in about a second.

Look at what I actually typed, because this is the moment the whole thing cracked open for me.

Wait — this is basically SELECT

I've written SQL for a decade in healthcare analytics. This SPL query is, line for line, a SELECT statement I've written a thousand times:

What I typed in SPL	What it is in SQL	What it does
source="tutorialdata.zip:*"	FROM tutorialdata	picks the data source
status=403	WHERE status = 403	filters to matching events
stats count by clientip	SELECT clientip, COUNT(*) GROUP BY clientip	groups and counts
sort -count	ORDER BY count DESC	ranks highest first
| (the pipe)	— no clean equivalent —	feeds each stage into the next

Four of those five map almost perfectly. Filter, group, sort — every instinct I built in SQL transferred on day one. Even Splunk's editor gives it away: it syntax-highlights commands, functions, and keywords in different colors, exactly like a SQL editor does.

The one genuinely new thing is the pipe (|). SQL nests — subqueries inside subqueries, read inside-out. SPL flows left to right: each pipe hands its output to the next command. It's closer to a Unix pipeline (grep | sort | uniq) than a SELECT statement. Once I reframed the pipe as "chained subqueries, but readable in order," SPL stopped feeling foreign.

Honest caveat — "for now":

SPL meets your SQL instincts where they already are, then keeps going into territory SQL never had — time as a first-class citizen, schema-on-read, and stateful correlation across events (the "event A then event B within 5 minutes" logic a flat query can't do). You're not starting over. You're extending a foundation you already own.

Query 2: What are the errors telling me?

A loud IP is a lead, not a verdict. Before chasing it, I wanted the lay of the land — what's failing on this server? Web logs carry a status field (200 = OK, 404 = not found, 403 = forbidden), so:

source="tutorialdata.zip:*" | stats count by status | sort -count

The 200s dominate — that's normal, successful traffic, my baseline. Underneath: 690 404s (requests for pages that don't exist — a lot of these from one source can mean scanning) and 228 403s (forbidden — someone trying to reach things they're not allowed to). The 5xx codes are server errors, an ops signal more than a security one.

The SOC instinct here: the 403s and 404s are where you look, not because the numbers are big, but because those codes mean "someone tried something they shouldn't." So I went hunting.

Query 3: Who's generating the forbidden requests?

source="tutorialdata.zip:*" status=403 | stats count by clientip | sort -count

Same query shape as the loudest-IP hunt, with one addition: status=403 right after the source — a filter, exactly like SQL's WHERE. Take only the forbidden requests, then count them by IP.

Here's where it got interesting — by not being interesting. The 403s were spread across 126 different IPs, top offender with just 7, then a long tail of 4s and 3s. No single host dominating. 228 forbidden requests scattered across 126 sources.

// Analyst Take

This is the lesson no query tutorial teaches: the search is easy. The interpretation is the job.

If one IP had generated 200 of those 403s, that's someone hammering restricted resources — a real hunt. But 7-6-4-4-3 spread across 126 IPs is what the internet does to a public server: bots, stale bookmarks, casual probing. The shape of the result — concentrated versus spread — is what separates "I found a threat" from "this is background noise."

Same instinct I used reading weather data for a decade. One station reporting an anomaly could be a broken sensor. The same anomaly across a spatial cluster is a real front moving through. Distribution tells you signal from noise.

Back to the suspect: was the loudest IP a threat?

Now I could investigate 87.194.216.51 properly. Two questions: does its traffic succeed or fail, and what is it actually requesting?

source="tutorialdata.zip:*" clientip="87.194.216.51" | stats count by status

894 of its 1,036 requests were 200 OK — about 86% successful. A scanner or attacker shows the opposite shape: mostly errors, because they're probing for things that don't exist. This IP mostly succeeds, which means it's requesting real, valid pages. Then the clincher — what pages?

source="tutorialdata.zip:*" clientip="87.194.216.51" | stats count by uri | sort -c

/product.screen. /cart.do?action=purchase. /cart/success.do. /category.screen. That's not an attack — that's shopping. Browsing products, adding to cart, completing purchases, spread across nearly a thousand distinct URLs hit two or three times each. A real person (or a price crawler) moving naturally through a catalog.

The loudest IP on the network turned out to be the best customer.

High volume ≠ threat.

The noisiest thing isn't automatically the dangerous thing. You have to look at what the noise is, not just how much of it there is. Most beginners find the outlier and cry wolf. The actual skill is the second step — investigating whether abnormal is actually bad — and being honest when it isn't.

Then I made it a chart, and saved it

Two more things a SOC analyst does with a search: visualize it and save it.

Clicking the Visualization tab turned my status breakdown into a column chart — one tall bar for the 894 successes towering over a row of tiny error stubs. That shape is the "legit user" verdict; anyone glancing at it sees "this host mostly succeeds" in half a second. And it works exactly like Tableau or Power BI: the chart is driven by the query, updates live when the search changes, same chart-type picker and formatting. The only difference is the data source underneath.

Then Save As → Report, named it "Loudest IPs," and it's permanent — rerunnable on demand, schedulable, or droppable into a dashboard. Which is, functionally, the same saved-report workflow I've used in healthcare systems for years: build the query once, rerun it whenever. A saved report on a schedule is one small step from a standing detection rule — a query that never stops running.

The real takeaway isn't "learn Splunk"

Same frame as last time, because it holds for every tool on the skills-gap list:

Why / where / how

• Why it's on the job req: Splunk is the market-leading SIEM — the daily instrument of SOC work. Searching it is the core motion of the role.

  
  

• Where to learn it free: Splunk Free, the official docs, and the tutorial dataset I used here. No bootcamp, no license.

  
  

• How to prove it: not "I watched a Splunk course," but "here's a threat hunt I ran, here's the loudest IP, and here's how I knew it was nothing." I saved that as a report. It's portfolio evidence.

That's the difference between a skill you can name and one you can show. In one sitting, I loaded real data, ran a threat hunt, cleared a false lead, visualized it, and saved the artifact — with SQL instincts I already had, in a tool the job actually uses.

The syntax was familiar. The interpretation was the new muscle. And the loudest thing on the network turned out to be perfectly innocent — which is its own lesson worth carrying into a SOC.

The tool changes. The question doesn't: what deserves attention?