How do you monitor your homelab network for internal attackers? E.g. you have a publicly available service and theres a vulnerability that you miss or you pull a bad update and suddenly someone has access to your VM/machine/container. How could you increase the chances of automatically detecting that?
The built in IDS in opnsense seems pretty useless, and doesn’t really help detect if e.g. someone is trying to exploit services between your vlans (I could be using it wrong though).
Crowdsec in opnsense is nice but it seems to also be primarily for protecting from malicious actors coming from the WAN.
I’ve heard about the opnsense zenarmor plugin but you have to agree to a privacy policy to use it?
Another option I guess would be collecting firewall logs and making custom notifications for things that you think would be suspicious on your network.
I also know update cooldowns and not exposing anything could largely solve this too, but the monitoring and alerting question really interests me.
You must log in or register to comment.
If you can swing compatible hardware, Security Onion is pretty great. It’s not necessary to buy their solution, it’s FOSS and there are instructions on getting it to work on proxmox.
(But you need to install agents on all your VMs and servers, and ideally do port mirroring so all traffic can make it to the Security Onion box.
Very cool thanks for the recommendation.
I’m in the process of building a monitoring system with grafana stack.
Right now I have monitoring panels for some common metrics and logs. I am yet to set up alerts.
The idea being that if something goes wrong some metric will grow up unexpectedly, for instance network traffic. And I would get a notification.
What I’m still considering is what would I consider abnormal behavior, so I could set up the thresholds.
Yeah that’s kind of what I was originally thinking to do too.
One thing you could deploy throughout all your internal networks are honeypots that send alerts upon ANY network activity. Nobody should be hitting those at all.
Do you have any recommended tools or ways to set that up?
Oh yeah that’s a good idea. Thank you.
Segment the network as much as feasible, forbid the communication between the segments via FW rules, and set an alert when those rules are triggered.
For example: your dmz should never initiate any type of communication with your lan segment, your lan segment should not try to access services outside ports 80/443, your dns should log all resolutions performed and it would be nice to have at least a black list.
None of them should have dns over tls, and for specific hosts and networks segments, new domains with very looong active but idle connections should trigger an alert.
My personal opinion is that for a homelab is not realistic to perform a dpi to check that there is not an active attack ongoing, neither from the raw processing power, either from the human effort side, your best chance is to alert when something unusual is happening and then adjust your rules of the are false positives
Thanks for your helpful answer.
Do you do all of that on a separate log sever? Or on your firewall? I haven’t found a good way to do that on opnsense that doesn’t feel hacky.
My setup is “simple” and all these monitoring functions are performed in my opnsense box with the telegram plugin.
Most of the alerts are pretty basic and are done into the FW level or the outbound basic logging. So opnsense with the basic tooling is just enough.
I have in my todo to connect the logging system from opnsense to a proper Prometheus/grafana system to really have proper log of several days without having an impact on the FW but I never find the time to do it (lazyness problem)
So, I have a standalone pFsense box running Suricata (IDS/IPS) on WAN & LAN, pfblockerng (filtering), unbound, ntopng (traffic analysis), and Tailscale as an overlay on the firewall. The servers are on their separate VLANs, IoT, phone, laptops and other devices are also isolated. On the server I run Cloudflare Tunnels/Zero Trust, UFW, Tailscale as an overlay, and Fail2Ban. Everything is ‘default deny until something complains’. I’ve never run oPensense so I can’t speak to their software, but pFsense has been rock solid for me for years. I utilize Lnav to review logs. It’s lean, easy to install, and it just works.
To be quite honest about updates like Docker container updates, I wait until all the early adopters work out all the bugs for me and update when I feel comfortable. Updates to Ubuntu Jammy are usually straight forward and I can’t say I’ve had many issues over the years with Ubuntu Jammy updates. There may be a rare occasion, but nothing really stands out at this moment. The only issue I can think of here recently is when Docker updated but Portainer hadn’t caught up, but that was about a 30 minute fix.
It took me a while to get everything lined out from the modem to the servers, but my track record makes me feel pretty comfortable with how I have everything set up. In fact, literally most of the time, I just enjoy the set up and scout for the next project.
How did you monitor your vlans with suricata? I have enough trouble tuning a couple subnets, never mind tuning for correlation between 3 or 4. This assumes you have different subnets per vlan, of course.
How did you monitor your vlans with suricata?
At present I only monitor the VLAN serving the ‘computer room’ which includes the servers. It’s where all the ‘stuff’ happens so I figured I’d start with the most important. However I am in the process of learning how to SPAN (Switch Port Mirroring) where you configure your managed switch to copy traffic from specific VLAN port or all VLANs ports to a dedicated ‘monitoring port’ where a micro server is running Suricata. The mirrored traffic will retain their VLAN tags and Suricata can parse these tags.
Those are the lofty plans. LOL I’m still fiddling around with it and Suricata because you have to actually set Suricata up to be able to do that. It doesn’t right out of the box.
The mirrored traffic will retain their VLAN tags and Suricata can parse these tags.
I’m not sure how far down this path you’ve gone, but suricata will not automatically correlate primitives into actual alerts from different vlans without transforms, which are cpu-intensive for what they do.
You may want to pull your tap/span/mirror from a point where they converge, like internal side of network egress.
but suricata will not automatically correlate primitives into actual alerts from different vlans without transforms, which are cpu-intensive for what they do.
It is possible to offload the correlation to a downstream SIEM or log aggregator like Wazuh or ELK. Again, it’s something I’m currently trying to spool up on. I know it can be done, I’m just trying different things until I do get it right. I appreciate any input.
Oh, yeah, absolutely. Suricata was created not long after snort, in the days when an ids did the gathering and the correlation.
You’re totally right, the way most people and orgs do it today is to ship ids logs to a siem for the correlation, overall easier to manage. ELK is the go-to for most, not sure about wazuh, I’ve only seen it in the homelab space, but it might work.
There is a distro (not totally open source) called SELKS, which sets up suricata, elastic and some other tooling (kibana) in a commonly-used setup. I deploy it a lot because it saves time with the non-security setup with dB’s and such. Pretty easy to point syslog to it and you can see alerts right away and start tuning.
I’m envious of your position, I learned a lot setting this stuff up.
There is a distro (not totally open source) called SELKS, which sets up suricata, elastic and some other tooling (kibana) in a commonly-used setup.
I did not know about SELKS. I will definitely check it out. Thank you for the tip.
