Hey All,
Having a bit of trouble with my network setup. I’m mostly a noob with very light understanding of what I’m working with, so bear with me.
I’ve got my opnsense box setup with mostly defaults set for rules. The opnsense box is hooked directly to an MB8611 modem on the WAN interface, with LAN interface running to an 8 port managed netgear switch on port 1 of the switch. I have port 2 of the switch hooked up to a vlan-aware access point. Other ports are occupied by physical links to some servers.
Everything on LAN works fine, including the access point. I have the access point setup with 3 SSIDs, all that can connect clients no problem. Some of the clients on the network are game consoles/gaming PCs that run into connectivity issues with some titles, I believe because of a strict NAT. Rather than just assigning outbound rules by static addresses, I opted to create a VLAN to house all gaming devices and segment them from the network. I don’t need them to talk to each other or other devices.
I have created VLAN10, assigned it a gateway address of 192.168.10.0/24, setup DHCP and assigned the LAN as the parent interface. I created a new SSID on the access point and gave it the VLAN 10 tag. All of the ports on the switch are now assigned to default vlan1 with untagged traffic, and then ports 1 and 2 are assigned to vlan 10 with tagged traffic. Testing with an iPhone, this works totally fine. I get assigned the correct leases and can make outbound connections to the internet. Testing on a windows 11 gaming PC, previously connected to an untagged SSID and now switched over to the tagged SSID, I am unable to make outbound connections to the internet. Another android device that I connected with to the new SSID also worked fine, so I’m not sure what’s up with this PC. I haven’t tested other gaming devices yet.
Any ideas on what I’m doing wrong here?
Your switch and AP configuration seem to be fine, so I would guess that the issue is on the routing/firewall side in OPNsense. Do I understand correctly that you assigned 192.168.10.0 as the IP address for OPNsense on the VLAN10 interface?
That might pose an issue, since in a /24 sized subnet in IPv4, the .0 address is the network identifier. Some software historically would wrongly disallow using this as an IP address, either as a source or as a destination. You might try changing your address to 192.168.10.1/24 to see if that works for your devices.
BTW: do you plan to enable IPv6 as well?
Hey! Thanks so much for the response.
So correction again, I do have vlan10 assigned with an IP of 192.168.10.1/24, so that does appear correct.
I have enabled ipv6 on both the vlan and the main LAN. I get assigned leases on both with the correct prefix I have set, and I have a requested prefix delegation of /60 on the WAN side, which also appears to have applied correctly. LAN I can pass all ipv6 tests, but the vlan I’m never able to pass any of the devices.
Are you familiar with using Wireshark for traffic analysis? I think the next step is to figure out what is getting through and what isn’t, to the Windows machine to start with.
Focusing on IPv4 for now, I would hope the network trace shows the DHCP request being sent out, the DHCP response with an IP for the Windows machine, and then some outbound web TCP traffic (eg google.com), followed by some sort of TCP response. But since it’s not working, I imagine the latter would be replaced by – ideally – ICMP error messages that will describe the problem.
I’m familiar with wireshark, but don’t have so much hands on experience with it. I’ll give it a shot and see the type of responses I’m getting back from the afflicted machine.
In a the meantime, here’s some of the firewall rules I have set on the interface itself as well as some floating rules. I’m following the recent guide from home network guy to set this up.
Looking at the firewall config, nothing stands out to me as unusual. On the gaming rules page, can you include the 16 autogenerated rules? I don’t imagine that’s where the issue is, but it might be worth a look.
When your Windows machine is attached on the VLAN network, you said it is successfully assigned an IPv4 address using DHCP, right? Is it able to ping the router? Can it ping anything successfully?
Really appreciate your help on this!
I’ve been messing with wireshark, but I’ll admit I’m not super sure how to interpret it all. Biggest thing standing out is some TCP retransmission packets, but nothing jumping out as an immediate failure. I realized I’m having similar difficulties across devices I test on the vlan. I’ve been using my laptop, and I can ping things like google.com or just the DNS of 8.8.8.8 no problem. I can’t ping the static router address of 192.168.10.1, but I think that’s because of the rule I have in place that includes all private networks, which includes the vlan net. I also realized that on the interfaces overview section, I’ve got 1 collision error on the LAN, and 2 in/out errors on the vlan on the out side, but I’m not sure how to assess those. Also correct that I am getting the expected DHCP assignments on the vlan side.
Np, it helps me keep my networking skills fresh and relevant.
I can ping things like google.com or just the DNS of 8.8.8.8 no problem
When you ping google.com, does this resolve as Google’s v4 or V6 address? In either case, this at least proves that the VLAN routing is enough to: 1) reach the system’s configured DNS server, 2) receive the DNS record, 3) send an ICMP (v6?) Echo to the default gateway, and 4) receive the ICMP Reply in response. If this works on v6, that makes sense since you have a rule explicitly for v6 ICMP to pass through. If this works on v4, I’m slightly confused why this works but nothing else does.
I can’t ping the static router address of 192.168.10.1, but I think that’s because of the rule I have in place that includes all private networks
Which rule was this? But more importantly, in the Wireshark trace, does any traffic at all from 192.168.10.1 show up as a source IP? The pings from earlier, they only need the MAC address of the gateway. But the DHCP responses should be coming from 192.168.10.1. Does anything else come from that IP? On a related note, do you see any ARP broadcasts originating from your laptop asking for any addresses on the network, such as 192.168.10.1? I’m trying to rule out certain odd situations.
I’ve got 1 collision error on the LAN, and 2 in/out errors on the vlan on the out side
While collisions are unexpected in today’s point-to-point switching topologies, if it’s just in the single digits and the vast, vast number of total frames are passing through without issue, then this is not a cause for great concern about your L2 network. To be clear, are you running 1 Gbps on the OPNSense interface and on all the switch ports?
When you ping google.com, does this resolve as Google’s v4 or V6 address
It’s definitely returning the v4 address each ping.
Which rule was this? But more importantly, in the Wireshark trace, does any traffic at all from 192.168.10.1 show up as a source IP?
The “only allow access to internet” rule on the gaming interface which encapsulates the firewall alias I set as “privatenetworks” that included the LAN and gaming nets. As far as wireshark, I do see traffic from 192.168.10.1 as a source! Being totally fresh with you on the ARP broadcasts, with my current understanding, I don’t know if I’m picking it out right. I do see broadcast requests coming from my laptop to 192.168.10.1 via DNS with responses of AAAA ipv4only.arpa.
To be clear, are you running 1 Gbps on the OPNSense interface and on all the switch ports?
OPNSense has a 2.5 Gbps connection to from the modem to 2.5 Gbps port on the box itself. Then the switch that is connected to the LAN on both the OPNSense interface and the switch port are both 2.5 Gbps. The remainder of the ports on the switch are all also 2.5 Gbps capable, but there are some ports occupied by devices that only support a max of 1 Gbps.
I did test the vlan by disabling ipv6 entirely and bam! All traffic flows no problem. Certainly a quick fix, but for no reason other than looking to understand and learn, I do want to get it working. I’ve got both LAN and the vlan set to track interface, and originally, both to allow manual adjustment of DHCPv6 and router advertisements. That seems to work no problem on the LAN with a prefix ID of 1, passes all ipv6 tests. On the vlan, though, prefix ID of 2, I do get the expected ipv6 leases with the corresponding ID, but it can never pass the ipv6 tests.
Edit: so correction, the android and iphones can resolve certain domains, but I get timeouts with others. I’m running unbound as as the local resolver and have set the rule to allow traffic from the vlan to the DNS port.