Wednesday, 27 April 2011

Defaultroute why not? Part 1

Whats the big deal about a defaultroute? The defaultroute is the last resort path your IP traffic takes to get out of your local network if it doesn’t have a more specific route in it’s routing table.

Well you know, the current internet routing table is hitting around 370,000 entries or so today. Having all of those routes sat on your PC is not going to be a good thing for your RAM or CPU when you have to store and lookup those routes. So what do we do, we have a default route pointing at a gateway device on our network. So, that gateway might be a layer 3 switch where the gateway address is a virtual switch interface (VSI). Do you think your switch was built to hold 370k+ routes? No sir you are correct. Most likely therefore your first hop gateway device will also have a default gateway configured pointing to an upstream device like your firewall. So, that firewall, is that likely to be fantastic enough to hold these routes to the internet? You guessed right, no way. So your firewall has a default route to another upstream gateway like you CPE (Customer/Provider Edge) router. Now finally this is the most likely place you’ll not find a default route. If you are taking a full BGP feed from your ISP then you don’t need a default route because your router learns all of the networks out there. So do you need a full routing table here? If you are connected to one ISP with one link...nope you do NOT need the full routing table and a default route from the next hop (the ISP neighbor router) is enough.

Now we see the defaultroute and it’s importance it’s important to understand why it’s a bad thing for network security. If you give your devices access to a network which contains a default route then it’s something which an attacker can use to ‘get around’. Without a route to a destination packets are dropped - just think about that - network security without a firewall - awesome.

So if we want a default route how can we have one?

We’re going to start our ramble into default routes using the humble and powerful but ultimately high maintenance static route.

  • Static route - easiest to configure but least scalable. In this example we’ll walk through a defaultroute to get traffic via from the source interface of Loopback0 on R1 to the destination interface of Loopback0 on R4.

  • Here is the topology:

  • Screen shot 2011-05-03 at 22.48.25


  • Some topology information:
  • R1 Fa1/0 10.1.13.1/24 <-> R3 Fa1/0 10.1.13.3/24
  • R1 Fa0/0 10.1.12.1/24 <-> R2 Fa0/0 10.1.12.2/24
  • R3 Fa0/0 10.1.34.3/24 <-> R4 Fa0/0 10.1.34.4/24
  • R2 Fa1/0 10.1.24.2/24 <-> R4 Fa1/0 10.1.24.4/24
  • R1 Loopback0 1.1.1.1/32
  • R2 Loopback0 2.2.2.2/32
  • R3 Loopback0 3.3.3.3/32
  • R4 Loopback0 4.4.4.4/32
Screen shot 2011-05-03 at 23.02.10

Screen shot 2011-05-03 at 23.02.22

Screen shot 2011-05-03 at 23.02.34

Screen shot 2011-05-03 at 23.02.45

I’ve configured R2, R3 and R4 to ‘know’ how to reach the loopback interface of R1. This is key to the topology and you should try to think of R2 and R3 as intermediate systems rather like those of the internet. On the internet there are man hops in the network between you and your destination...just look at this traceroute for an example. I’ve removed the IP information from the first 4 hops to anonymise the result but you can see that my packets hop across 9 nodes before they get to the www.google.com server. In our example we’re going to hop twice to get to R4.

Screen shot 2011-05-03 at 23.11.21

So lets start with showing the routing table on R1

Screen shot 2011-05-03 at 23.03.55

So we see that there are 3 networks listed 1.0.0.0/32 is the loopback0 interface and the other two are the Fast Ethernet ports. Each interface is listed as code ‘C’ which means connected (Juniper call this ‘direct’). If we try to ping the remote loopback interface on R4 we get a fail because R4 doesn’t know how to get to the R4 network of 4.4.4.4/32.

Screen shot 2011-05-03 at 23.18.29

Simple enough so far and hopefully no worries.

So we want to get to google.com and in our wisdom we’ve put into two links to the internet. We’re going to use the link between R1 and R2 as ‘way out’ and we’re going to trust that R2 knows how to get to google.com. Lets put in our default route now, point it at R2.

Screen shot 2011-05-03 at 23.20.49

The first command we put in adds the default route. The IP address of 0.0.0.0 and netmask of all zeros means any IP address basically. It can sometimes be shown as 0.0.0.0/0. The last IP address is the address of the next hop from us. R1 has a connected network link between it’s Fa0/0 interface and R2 using network 10.1.12.0/24. R2’s IP address on this link is 10.1.12.2 so we point our defaultroute at that.

The result of the ‘show ip route’ now has a new route at the bottom prefaced with the code of ’S*’ which means a static route. So now we have a defaultroute pointing at the next hop on R2. R2 knows how to get to the Lo0 interface of R4...lets try a ping now.

Screen shot 2011-05-03 at 23.28.35
  • Great stuff we did it - we can now ping - lets see the path we took.
Screen shot 2011-05-03 at 23.30.13

So, as we know our packets went via R2. But hey you’ve bought two uplinks to the internet, one via R2 and one via R3...can we have two default routes? Well if your have one ISP and two uplinks to that ISP then the answer is yes. If you have two uplinks but are using different ISP’s on each link then the answer is a vague maybe and we’re not getting into the reasons why here.

So lets add another default route into R1 via R3.
Screen shot 2011-05-03 at 23.32.23

This time the ‘show ip route’ has two lines under 0.0.0.0/0. This is IOS telling you there are two equal cost routes to the destination. This means your packets can take either way to get there. We do a traceroute...

Screen shot 2011-05-03 at 23.34.17

Check out how the ICMP packets are telling us that sometime the first hop is 10.1.13.3 then 10.1.12.2? This is your packets being ‘load balanced’ in a round robin way. The second hop is either the network between R3 and R4 10.1.34.4 or the network between R2 and R4 10.1.24.4. Perfect. Just what we wanted...right? So what happens no if the link between R1 and R3 fails...we’ll administratively shutdown the interface for the demo.

Screen shot 2011-05-03 at 23.37.12
Screen shot 2011-05-03 at 23.37.34

Now lets ping the R4 loopback0 interface. What has happened now is that every other packet is failing to get to the 4.4.4.4 address. Thats because we effectively load balanced the traffic and now one of those links is down...hence the loss.

Screen shot 2011-05-03 at 23.42.56

So what can we do? Well in most cases you wouldn’t use static routes you’d be using dynamic routing protocols, some sort of first hop redundancy protocol and certainly be tracking upstream connectivity to failover and load balance properly but we’ll leave that for another time. For our brief demo we’ll concentrate on weighted static routes sometimes called ‘floating static routes’.

Floating static routes allow you to put a ‘primary route’ into the table but in situations where that route is no longer available a second route with a higher metric can be brought into service. Here we’re testing everything is good...and we’ve brought the link between R1 and R3 back up for this.

Screen shot 2011-05-03 at 23.47.38

The second configuration command basically takes out with a ‘no’ the second load balanced default route to R4 via R3. So now lets see the routing table...

Screen shot 2011-05-03 at 23.48.57

Ok you can see we’ve got one route now via 10.1.12.2. Lets add in our floating static route. Now by default a static route to a next hop gets a metric of 1. You can see the metric in the above ‘show ip route’ as the [1/0] after the route 0.0.0.0/0. We’ll add a ‘floating’ route with a metric of 200 but it can be anything between 2 and 255.

Screen shot 2011-05-03 at 23.50.42

So the route is in but crucially you WON’T see this in the routing table. It will still look like it did before we added the floating route...see...

Screen shot 2011-05-03 at 23.51.12

The floating route is only used when the lower metric route is lost from the table. To demonstrate it we’ll shutdown the interface on R1 connected to the link between R1 and R2.

Screen shot 2011-05-03 at 23.52.47

Check out the ‘show ip route’ now. You see that the defaultroute now points to 10.1.13.3 which before was not shown. R1 has realised that the next hop of the previous route is no longer valid so it drops that route from the table. The new defaultroute has a metric of 200 which is what we configured earlier. job done.

Thanks for reading.

Tuesday, 26 April 2011

Juniper SSG Firewall IPv6 Tunnel

Fact is that despite the hype and buzz the uptake of IPv6 has been slow, real slow. If IPv6 was having a race with the turtle it would lose and it won’t need a rest and 40 winks either. Despite US government mandated to introduce IPv6 throughout by June 2008 we’ve not seen huge uptake around the world. World IPv6 day in 2011 will be used by big brands such as Google to demonstrate that there is a framework for using IPv6 across the internet.

So how do you use it? The IPv6 protocol is really smart and collaboration with IPv4 networks was designed in from the start. This brief article is all about how we get IPv6 over an IPv4 network using an IPv6 tunnel broker. We recommend the TE guys - so go ahead, signup for an account, get your /40 IPv6 allocation and come back when you are ready.

You have it now? Great here we go.

The Juniper SSG firewall has IPv6 support from like ScreenOS version 5.3 or so. The trouble is that it isn’t enabled and you can only ‘turn it on’ via the CLI. I’m doing this configuration using ScreenOS 6.2

> set envar ipv6=yes

Now we have to do a reboot...I know...shocking. Anyway reboot and when you get your box back we’ll go with the next bits.

Create a tunnel interface, bang it into the’ Untrust zone’.

set interface tunnel.1 zone Untrust
set interface tunnel.1 ipv6 mode host


So now we enable the tunnel interface with IPv6 and give it your end of the IPv6 network you’ve been allocated for the point-to-point tunnel (/64)

set interface tunnel.1 ipv6 ip
set interface tunnel.1 ipv6 enable


We set the encapsulation to a 6in4 or IPv6 encapsulated into IPv4 packets

set interface tunnel.1 tunnel encap ip6in4 manual

The tunnel is built and pointed at the other end (IPv4 now because we are going across the normal internet)

set interface tunnel.1 tunnel local-if untrust dst-ip

Turn off neighbor discovery - you don’t need it because you’re not running IPv6 with your ISP...probably

unset interface tunnel.1 ipv6 nd nud
set interface tunnel.1 ipv6 nd dad-count 0


Finally you need another defaultroute for your IPv6 traffic using your broker IPv6 address as the next hop

set route ::/0 interface tunnel.1 gateway

So thats all you need for the IPv6 tunnel but you want your LAN hosts to go across this right so we need to enable IPv6 on your LAN interface. The following configuration enables it into a bridge interface on my SSG but you can do the same config for any number of ports

set interface bgroup0 ipv6 mode router
set interface bgroup0 ipv6 ip
set interface bgroup0 ipv6 enable
unset interface bgroup0 ipv6 ra link-address


So we enable ra (Router Advertisements) for our LAN hosts to learn the IPv6 gateway

set interface bgroup0 ipv6 ra transmit

We really want to enable neighbor discovery now because we are runnign IPv6 on our LAN and our hosts will understand this (if they are dual stack of course...which most modern OS’s are).

set interface bgroup0 ipv6 nd nud

So thats it - if you hop into the GUI now and go to interfaces you should see the state of the tunnel.1 as ‘Active’ or ‘Ready’. If your LAN machine is IPv6 aware it should be able to learn an IPv6 address using EUI-64 and will learn the SSG as the gateway to the internet.

You’ll need so configure a policy in the SSG to allow outbound IPv6 from Trust to Untrust or it won’t let you out! Have a look at your policy and you’ll see two distinct entries for IPv4 and IPv6...all good stuff.

Now fire up a browser and go to
test-ipv6.com to see if it’s all s-a-weet.

Good Luck, Happy IPv6 Day.

Wednesday, 20 April 2011

Spanning Tree Part 1

Here is the basic topology.

R1 and R3 are connected using port FastEthernet0/0
R1 and R2 are connected using port FastEthernet0/6
R2 and R3 are connected using port FastEthernet0/12

Screen shot 2011-04-20 at 22.14.15

Lets create a new VLAN 100 on each ‘switch’

Screen shot 2011-04-20 at 22.19.53

So now lets run through all the switches setting up each connected port as a trunk. We don’t need to set the encapsulation since these only support 802.1q but we’ll do it anyway...

Screen shot 2011-04-20 at 22.22.33

When all the ports are up lets wait for spanning-tree to converge.

Screen shot 2011-04-20 at 22.25.37

On R3 the port FastEthernet0/12 is ‘blocking’.

Screen shot 2011-04-20 at 22.26.03

So let’s move the blocked port to R2’s FastEthernet0/12 interface. To do that we need to make the cost of ‘getting’ to R3 easier than getting to R2. So lets lower the port cost on the interface closest to the root switch.

Screen shot 2011-04-20 at 22.59.02


The default port cost for FastEthernet is 19 so a cost of 1 is preferred. Now looking at R2 we’ve got the result we wanted.

Screen shot 2011-04-20 at 22.59.30

Monday, 18 April 2011

Removing and resetting a Juniper EX Virtual Chassis node

We’ve broken up the 5 node cluster and re-racked two of the nodes into a new stack. The following screen shot shows the two nodes as ‘Inactive’ and node members FPC 1 and FPC 4. Both are Linecards because the Master and Backup switches were nodes 0 and 3 in the pre-existing cluster. You can also see that only the vcp-0 virtual chassis cable has been connected (the vcp-1 cable is disconnected for no reason other than we didn’t do it).

Screen shot 2011-04-18 at 21.59.44

First things first, lets kill off the VCP ports to disable the VC traffic.

Screen shot 2011-04-18 at 22.04.16

Now go into config mode by typing ‘configure’. and then we’ll load the factory default settings.

Screen shot 2011-04-18 at 22.05.20

Before we can commit the blank configuration we need to set the root password...it won’t save without it...give it a go if you want.

Screen shot 2011-04-18 at 22.06.02

OK so now commit the blank configuration

Screen shot 2011-04-18 at 22.07.01

Back into EXEC mode we’ll take a look at the new cluster status. Note we can’t see the other node because the VCP ports are disabled.

Screen shot 2011-04-18 at 22.06.34

Lets run through the same process on the other node and commit. Now back in EXEC mode we turn the VCP ports back on. By NOT putting the keyword ‘disable’ on the end they are enabled...I know thats a bit poor but there you go.

> request virtual-chassis vc-port set interface vcp-0
> request virtual-chassis vc-port set interface vcp-1

So now we check the status of the node...all good. We have a Master and Backup in a two node cluster with FPC 0 and FPC1. The mastership priorities are both the default of 128.

Screen shot 2011-04-18 at 22.16.25

Tuesday, 12 April 2011

Managing remote service dependancies with IP SLA

A simple yet effective way to manage remote service dependancies is with IP SLA.

Imagine you have two routers connected to one ISP, behind those routers you have a shared ethernet LAN in which you have a firewall. The ISP offers you eBGP services and you receive a full routing table from them into each of your CPE routers. Your CPE facing LAN is configured over ethernet and you are running VRRP between your two routers and the firewall in this LAN (212.1.30.0/24). Devices in the CPE LAN all point to the VRRP virtual IP address for service to the Internet.

Screen shot 2011-04-13 at 20.59.09

So, everything is great, VRRP is up and running as master on RouterA. Now you’ve been extra sensible and setup VRRP to ‘track’ an object on each router to preference the VRRP master device. In our case we are tracking object 1, the object is looking for route 1.2.3.4/32 in the routing table.

Screen shot 2011-04-13 at 21.08.53

If the route exists then our priority s the default VRRP priority of 100 however if the route is lost we decrement by 10 and RouterB (configured with a priority of 95 and to preempt) takes over.

Screen shot 2011-04-13 at 21.08.22

You figure if the eBGP neighbor goes down you’ll lose the routing table so you be able to get out into the internet so you should maybe fail over the VRRP address to the other node (hopefully he’s got a full routing table).

Now, what happens if there is a problem on the ISP network so you are receiving a full routing table into both of the routers and yet behind one of the PE routers there is no upstream service and your data is being blackholed there. This is bad. Your VRRP service does NOT fail over because it was tracking a local route and the provider eBGP neighbor is still sending you a full table. Without manual intervention you are out of service.

So this is where ‘ip sla’ takes over. Cisco ip sla stands for IP Service Level Agreements and is used to provide telemetry information about service availability and capability for a number of network streams such as FTP, HTTP, Voice Jitter.

First lets setup a simple SLA object. For this example we’ll do an HTTP GET request against the http://www.defaultroute.co.uk website but htere are a number of different types of SLA (Jitter, ping,FTP etc). The response time from the site will be a deterministic value for GOOD or BAD status. Clearly a DOWN website might be for reasons other than a lost route but it’s just an example.

Screen shot 2011-04-13 at 21.13.44

Lets check to see if it’s running

Screen shot 2011-04-13 at 21.16.35

Looks good. We’re getting a round trip time (RTT) of 5060ms, the DNS lookup is around 52ms. The return code is OK which is what we are looking for. So now we’ll bind this sla monitor to a track object so we can add it to the VRRP tracking.

Screen shot 2011-04-13 at 21.13.35

Lets confirm thats in

Screen shot 2011-04-13 at 21.13.13

So now we’ll add the new track object to theVRRP configuration and we’ll add another 10 to the increment. So now the outcome of losing the route AND connectivity to the web server will reduce the value to 80. However the key here is that losing connectivity to the monitored web server OR the existence of the route to 1.2.3.4/32 will cause the VRRP service to failover.

Screen shot 2011-04-13 at 21.13.24

Job done - experiment and have fun

Sunday, 10 April 2011

When you need log data you need good timekeeping

When there is a technical error with the device or you need to troubleshoot a failure the most important thing is time. Recently we were onsite troubleshooting a denial of service attack (DoS) on a web server for a client. They informed us that it had taken place sometime between 22:00 and 22:10 on a certain date. We asked for the web logs and then for the router/firewall logs for the period concerned. We spent time looking though the logs and noticed no commonality with the customers experience and the logs we were looking at. Indeed there appeared to be some difference even with the log files we had been given. For example a very clear HTTP GET request on the web server had no matching event on the firewall...

You guessed it, not only was the time wrong on the web server but it was also wrong on the firewall. Worse still the timezone was wrong so even if the time setting was correct we were at best an hour out as we were operating in BST.

Setting the time on the web server to point to the cisco switch (Catalyst 4500) to obtain it’s time was the first stage. The timezone was setup correctly and we were to use the windows time service using NTP. Here are the steps we took to setup the Cisco switch to set it’s timezone, to setup the correct British Summer Time skew plus the NTP daemon function so that the Cisco device to collect the correct time information from the internet and also allow the windows server to collect the time information.

First we setup the timezone so that the switch knew where in the world it was:

Screen shot 2011-04-10 at 21.24.37

Now we need to setup the NTP daemon itself. We need to find an NTP source out there to deal with us. There are different tiers of NTP service called ‘strata’. We don’t really need to be hugely accurate so we chose a stratum 2 NTP source. Here are a list of the UK NTP servers we used.

Screen shot 2011-04-10 at 21.28.18

The prefer keyword is just to say to IOS “if this one is available then we’d like you to trust this one most”. We also want to make sure our log data on the switch is given a timestamp.

Screen shot 2011-04-10 at 21.24.47

So lets just make sure it’s all working. Type “show ntp associations”

Screen shot 2011-04-10 at 21.25.16