networking

I often get into discussions about "next-generation firewalls" versus "traditional firewalls". All firewall vendors are moving in this same direction, adding deeper inspection to their devices.
For sales, it's a very easy pitch. Everyone is annoyed with the limited visibility L4 firewalls have and proxies have always been a necessary evil. Along comes this one new box that will fix all these issues. Who wouldn't want one?

Marketing will say you no longer need to worry about TCP or UDP ports, just applications. But if you think about this for a few seconds, you'll know this isn't true. Applications can only be identified once a connection has been established and a certain amount of traffic has been passed back and forth. So if you want to allow HTTP on any port, you need to allow TCP connections to any port and block them once they turn out to be something different than HTTP. You just opened your entire network to portscans and all sorts of badness. This is a very easy configuration error to make - I've seen it happen several times already.

I was going to write about some of the interesting things you could do to trick the application-identification, but found an excellent presentation which pretty much sums up my thoughts:
DEFCON19 Presentation Slides
DEFCON19 Presentation Video
You may recognize the CLI - but this applies to all vendors. It's about the technology, not the individual products.

In summary:

• application-identification only looks at the first bytes and can be tricked.
• application-identification caches can be poisoned.
• its not all unicorns and rainbows, sorry.

Conclusion:
Does this mean application firewalls are useless? Absolutely not. You simply need to be aware of its limitations.

In my opinion, L7 firewalls are an additional layer of defense, not a replacement for the stateful firewalls. You still need a stateful L4 box in front to attach the network segments to that don't require application-identification or need better performance. This also reduces the risk of configuration errors on the L7 device.
On the L7 firewall you need to remember to only allow applications on the ports they need and that application-identification is an expensive operation, especially for certain UDP based protocols.

As everyone will already know by now, tomorrow is World IPv6 Day.

Below are a list of resources that may be useful.

• Test your IPv4/IPv6 connectivity to diagnose problems
• RIPE IPv6 Day measurements
• IPv6 Eye Chart - test connectivity to several IPv6 sites at once
• Akamai IPv6 statistics
• Netalyzr network diagnostics
• What will most likely be a huge flood of tweets
• Problems that could occur (a lot)

Good luck everyone. Like most of Europe, I'll sleep through the first few hours of World IPv6 Day (as it starts at midnight UTC).

I have been experimenting with ECMP to balance outgoing connections over multiple internet connections. Configuring this is quite simple but it does have quite some side effects you might not immediately think about.

• The one thing that I expected to be a major problem is HTTP/HTTPS traffic. And this turned out to be the case. Several websites lock a session to a specific IP address. When requests are balanced over two entirely different public IP addresses, you can expect to see some very bizar behavior. This includes websites of banks and the management interfaces of several devices that I need to use. So it didn't take long before all web traffic was forced back over one internet connection.
• The result of a traceroute is completely useless. Each packet it sends is seen as a new session and they are sent out over both internet connections.
• One system in the DMZ was somewhat slow. Tests quickly revealed that it was a DNS problem (surprise, surprise). While most systems use our own resolvers, this system was configured to use the DNS servers of both internet providers. Combined with ECMP this resulted in the following behavior:
  • Server sends (recursive) DNS Query to providerA. Firewall decides to send this packet via providerB. The DNS server of providerA won't allow the recursive query because it arrived from an address owned by providerB.
  • Server repeats the DNS Query, this time sending it to providerB. Firewall sends the packet via providerA. Same problem
  • Server tries again and gets lucky, query is sent to providerA and the firewall also routes the packet via providerA.
• ...

That was all for today. I'm sure there will be other issues. Upgrading to a faster internet connection is so much easier, but a lot less fun ;)

update: I ran into another problem when I enabled anti-spoofing functionality on the external interfaces. When a packet arrives on interface ext0, the firewall looks up the route back to the source-ip. When a route exists via ext0, the packet is accepted. When the route points to another interface, this is detected as spoofing and the packet is dropped. The problem occurs when someone on the internal network tries to access a system in the DMZ. Systems in the DMZ have private addresses with 1-to-1 NAT on the firewall and users usually connect to the public IP because thats how it is registered in DNS. Actually, the systems have two public addresses, one on interface ext0 and one on interface ext1 and both are configured as round robbin DNS entries.
What happens is when user connects to a public address on ext1? Because the firewall doesn't know it "owns" the destination ip, this is simply configured as a NAT rule on ext1, it routes the traffic to the internet. If ext1 is chosen as the external interface, the firewall will detect that the packet loops back to itself and it will be delivered without a problem. If ext0 is chosen however, the packet will be sent to the internet, eventually arriving back on ext1. At this point the anti-spoofing functionality kicks in and drops the packet because the source-ip is that of ext0 and according to the routing tables, that address is located on ext0.

I am still using an old 486 as firewall. Several times I have said that I was going to replace it "the next time it reboots", but the system is so rock stable that it never had to be rebooted. However, the system is too slow and can't handle a large number of connections. Time for a replacement, and this time I mean it!

Instead of installing yet another debian system, I decided it is time for something new. For a firewall the obvious choice would be OpenBSD. It has most of the firewalling features that I need, uses sane default values (in most cases) and has some interesting features that I want to experiment with (carp/pfsync).

Aside from firewalling, I also want to configure two IPSec VPN tunnels to remote devices. The remote devices are netscreen firewalls, they are quite flexible, i have configured VPNs with all kinds of funky systems ranging from checkpoint clusters to cheap soho routers. So I don't really expect any problems here.
The idea is to run OSPF on top of that. One of the VPN tunnels is meant as a backup in case the other tunnel fails. Adding multiple routes with different metrics is a possibility but I don't like having to add them manually every time new subnets are created or linked with the central site.

Oh, and of course my IPv6 tunnels need to be configured as well, but i'm not expecting any issues there either.