---

🔍 Real attack patterns. Zero real-world risk. Every log file blends normal background noise with deliberately embedded attack vectors — so you can practice detection without needing an actual breach.

🤔 Why This Exists

Finding realistic log data for threat hunting practice is surprisingly hard. Clean, normal-activity logs teach you nothing about detection. Massive raw dumps from public breach archives are unstructured, unlabeled, and overwhelming to parse. Real incident data is — rightly — locked down.

This repository bridges that gap. It provides structured, purpose-built log files across five environments — Applications, Cloud, Email & DLP, Endpoints, and Network — each containing specific, identifiable attack techniques hidden inside realistic background traffic.

If you're a student, a SOC analyst, or a threat hunter, this dataset is built for you to practice SIEM ingestion, query writing, and log-based detection — safely.

[!NOTE] Dataset Architecture & Log Schemas:

• Synthetic Educational Datasets: The .zip archives in the primary folders use a simplified, uniform key-value text format with explicit ground-truth labels (attack="SIMULATED_*"). This simplified schema is intentionally designed for beginner-to-intermediate grep, regex, and baseline log parsing exercises without the overhead of full enterprise XML/JSON schemas.
• Production Telemetry Samples: For authentic, production-grade schemas, explore the real_lab_samples/ directory containing raw, unmodified logs exported directly from live SOC lab environments: Microsoft Sysmon EID 1 Process Creation JSON (windows_sysmon_process_creation.json), Windows Security Event 4625/4624 Audit JSON (windows_security_logon_4625.json), and Suricata NIDS eve.json (suricata_nids_eve.json).

📋 Table of Contents

• ✨ Key Features
• 📂 What's Inside
• 🗂️ Repository Structure
• 🚀 Getting Started
• 🕵️ Threat Hunting Playbook
• 🎓 Who This Is For
• 🤝 Contributing
• 🗺️ Roadmap
• ❓ FAQ
• ⚠️ Disclaimer
• 📄 License
• ⭐ Support This Project

---

✨ Key Features

• 🎯 Targeted, not random — every file is built around specific, well-known attack techniques (SQLi, XSS, brute force, C2 beaconing, DNS tunneling, IAM abuse, and more), not random noise.
• 🌍 Five real-world domains — Applications, Cloud, Email & DLP, Endpoints, and Network, so you can practice across the environments a real SOC actually monitors.
• 🧩 Realistic background noise — attacks are buried inside normal-looking traffic, just like production, so you have to actually hunt for them.
• 🔌 SIEM-agnostic — plain text and JSONL formats that ingest cleanly into Splunk, the Elastic Stack, Microsoft Sentinel, or your own scripts.
• 🆓 Free and open — MIT licensed, free to use, modify, and redistribute for educational or commercial purposes.

📂 What's Inside

Domain	Files	Format	Hidden Attack Vectors
Applications	app_sec_audit.log.zip web_server_access.log.zip	Raw text	SQL injection, cross-site scripting (XSS), directory traversal
Cloud	cloud_activity_audit.log.zip	Raw text	Unauthorized access, IAM privilege escalation, anomalous resource deletion
Email & DLP	dlp_events.log.zip email_gateway_events.jsonl.zip	Text / JSONL	Phishing payloads, malicious attachments, data exfiltration
Endpoints	linux_syslog.log.zip windows_security_events.log.zip	Raw text	Brute-force attacks, privilege escalation, suspicious process execution
Network	dns_query_logs.jsonl.zip firewall_traffic.log.zip ids_ips_alerts.log.zip proxy_access.log.zip vpn_access.log.zip	Text / JSONL	DNS tunneling, C2 beaconing, port scanning, unauthorized VPN access

All files are zipped to keep the repository lightweight — unzip before ingesting.

🗂️ Repository Structure

SOC-Log-Analysis-Datasets/
├── Applications/
│   ├── app_sec_audit.log.zip
│   └── web_server_access.log.zip
├── Cloud/
│   └── cloud_activity_audit.log.zip
├── Email_and_DLP/
│   ├── dlp_events.log.zip
│   └── email_gateway_events.jsonl.zip
├── Endpoints/
│   ├── linux_syslog.log.zip
│   └── windows_security_events.log.zip
├── Network/
│   ├── dns_query_logs.jsonl.zip
│   ├── firewall_traffic.log.zip
│   ├── ids_ips_alerts.log.zip
│   ├── proxy_access.log.zip
│   └── vpn_access.log.zip
├── real_lab_samples/                        # Authentic enterprise JSON logs from live labs
│   ├── windows_sysmon_process_creation.json # Sysmon Event ID 1 (T1059.001)
│   ├── windows_security_logon_4625.json     # Windows Event ID 4625/4624 (T1110.001)
│   ├── suricata_nids_eve.json               # Suricata NIDS EVE JSON (T1046)
│   └── README.md                            # Telemetry schema guide
├── LICENSE
└── README.md

🚀 Getting Started

1. Clone the repository

git clone https://github.com/mahialjaber/SOC-Log-Analysis-Datasets.git
cd SOC-Log-Analysis-Datasets

2. Unzip the dataset(s) you want to work with

Linux / macOS:

unzip Network/firewall_traffic.log.zip -d Network/

Windows (PowerShell):

Expand-Archive -Path Network/firewall_traffic.log.zip -DestinationPath Network/

3. Ingest into your tool of choice

Tool	How
Splunk	Use "Add Data" and point it at the unzipped file. JSONL sources auto-extract fields.
Elastic Stack (ELK)	Feed through Filebeat/Logstash; use a json codec for .jsonl files.
Microsoft Sentinel	Upload via a custom log table or ingest through the Log Analytics API.
Command line	Use grep, awk, sed, and jq for rapid manual triage.
Custom scripts	Great for testing Python/PowerShell log-parsing and automation.

🕵️ Threat Hunting Playbook

Sample queries to get you started hunting for the attack vectors embedded in each dataset. Treat these as starting points — field names may need to match your ingestion schema.

1. SQL Injection & XSS

Target: Applications/web_server_access.log

Splunk SPL

index=web sourcetype=access_combined
| search uri_query="*UNION*" OR uri_query="*SELECT*" OR uri_query="*<script>*"
| stats count by src_ip, uri_query, status
| sort - count

CLI

grep -iE "union.*select|%27|<script>" web_server_access.log

2. Cloud IAM Privilege Escalation

Target: Cloud/cloud_activity_audit.log

Splunk SPL

index=cloud sourcetype=cloud_audit
| search event_name IN ("PrivilegeEscalation", "CreateAccessKey", "AssumeRole", "StopLogging")
| stats count by actor, event_name, source_ip
| where count > 3

CLI

grep -iE "PrivilegeEscalation|CreateAccessKey|AssumeRole|StopLogging" cloud_activity_audit.log

3. Phishing & Data Exfiltration

Target: Email_and_DLP/dlp_events.log, Email_and_DLP/email_gateway_events.jsonl

Splunk SPL

index=dlp sourcetype=email_gateway
| search action="blocked" OR category="phishing" OR category="exfiltration"
| stats count by sender, recipient, attachment_name
| sort - count

CLI

grep -iE "phishing|exfiltration|blocked" dlp_events.log

4. Windows Authentication Brute Force

Target: Endpoints/windows_security_events.log

Splunk SPL

index=windows event_id=4625
| stats count by user, source_ip
| where count > 10

Microsoft Sentinel (KQL)

SecurityEvent
| where EventID == 4625
| summarize count() by TargetUserName, IpAddress
| where count_ > 10

CLI

grep -i "4625" windows_security_events.log

5. C2 Beaconing

Target: Network/firewall_traffic.log

Splunk SPL

index=firewall action=ALLOW dst_port!=80 dst_port!=443
| stats sum(bytes) as total_outbound by src_ip, dst_ip, dst_port
| where total_outbound > 50000

CLI

awk '$6 != "dst_port=80" && $6 != "dst_port=443" && $9 == "action=ALLOW" {print $3, $5, $6}' firewall_traffic.log | sort | uniq -c | sort -nr

6. DNS Tunneling

Target: Network/dns_query_logs.jsonl

Splunk SPL

index=dns sourcetype=dns_query
| eval query_length=len(query)
| where query_length > 50
| stats count by client_ip, query
| sort - count

CLI (jq)

jq -r 'select((.query | length) > 50) | [.client_ip, .query] | @tsv' dns_query_logs.jsonl

💡 Tip: Run each query against its target file, then tune the thresholds. Real detection engineering is as much about cutting false positives as it is about finding the signal.

🎓 Who This Is For

• 🧑‍🎓 Students studying for certifications like Security+, CySA+, GCIH, or OSCP
• 🛡️ SOC analysts (Tier 1/2) building and sharpening triage skills
• 🔧 Detection engineers prototyping and testing correlation rules
• 🟣 Purple/blue teams running tabletop exercises or workshops
• 👨‍🏫 Instructors building hands-on cybersecurity curricula

🤝 Contributing

Contributions are welcome.

1. Fork the repository
2. Create a branch for your addition (git checkout -b feature/new-attack-scenario)
3. Add your log data or query following the existing structure and format
4. Open a pull request describing what you added and why

Ideas for contributions: new attack scenarios, additional log formats (Azure/GCP audit logs, EDR telemetry, Kubernetes audit logs), Sigma or YARA rule mappings, or fixes to existing queries.

🗺️ Roadmap

• Detailed threat hunting solutions & ground truth answer keys
• Sigma rule repository mapped to each simulated attack vector
• Cloud audit expansion (Azure Activity Logs & Google Cloud Audit Logs)
• Pre-configured Docker Compose environment for automated log ingestion (Splunk & Elastic)

❓ FAQ

Is this real attack data? No. Everything is synthetic and simulated, purpose-built for training. No real breach data, credentials, or PII is included.

Why are the log files provided as .zip archives? The raw log files contain tens of thousands of event records (multiple megabytes uncompressed). Compressing them keeps repository download fast and lightweight while preserving the raw formatting when extracted.

What SIEM do I need? None, strictly speaking. The formats work with Splunk, the Elastic Stack, Microsoft Sentinel, or nothing more than grep/awk/jq on the command line.

Can I use this for a course or workshop I'm running? Yes — it's MIT licensed, so educational and commercial use are both fine. A credit or link back is appreciated but not required.

How do I know if I found the "right" answer? Start with the sample queries in the Threat Hunting Playbook above and tune from there. An answer-key branch is on the roadmap.

⚠️ Disclaimer

All datasets in this repository are fully synthetic and simulated. They do not contain real user data, credentials, PII, or data from any actual breach or incident. They exist solely for education, training, and lab/detection-engineering use. Use responsibly and only against systems and SIEMs you own or are authorized to use.

📄 License

Released under the MIT License — free to use, modify, and distribute for educational or commercial purposes.

⭐ Support This Project

If this dataset helped with your studies, certifications, or lab setup:

• ⭐ Star this repository so it reaches other analysts who need it
• 👤 Follow @mahialjaber for future defensive security projects and SOC tooling
• 🍴 Fork it and share what you build

Building resources like this for the community takes time — every star genuinely helps it reach the next analyst who needs it.

---