Although this has been well blogged before, as we’ll be rolling DMVPN out where I work I’d like to record the steps I took to configure and verify a working example in GNS3.
The first hurdle I came across is that the image I’d upgraded my virtual Cisco 3725s to (
c3725-adventerprisek9-mz124-25.bin, in order to support EIGRP for IPv6) doesn’t have everything you need for DMVPN. Although it seems to be possible to get things working, the ‘show dmvpn’ command in missing. I rolled back to c3725-adventerprisek9-mz.124-15.T14.bin and all was well.
Here is a screenshot of my GNS3 topology. Linknets between routers are shown in green, client networks in dark blue. I’ve included a box showing the multipoint GRE tunnel peers in light blue.
I’ve attached full configs to the post. Prior to configuring DMVPN I set up OSPF on all routers. The intention here is to mimic each sites’ Internet connection. I probably should have used BGP or a static route to 0.0.0.0/0 via R5 on each router but at this stage I simply wanted all routers to be aware of all green subnets so that the mGRE tunnel would come up.
interface FastEthernet0/0 description R1 ip address 192.168.15.2 255.255.255.252 ! interface FastEthernet0/1 description R2 ip address 192.168.25.2 255.255.255.252 ! interface FastEthernet1/0 description R3 ip address 192.168.35.2 255.255.255.252 ! interface FastEthernet2/0 description R4 ip address 192.168.45.2 255.255.255.252 ! router ospf 100 router-id 188.8.131.52 network 0.0.0.0 255.255.255.255 area 0 default-information originate ! ip route 0.0.0.0 0.0.0.0 Null0
A sister config was added to each router (without the default-information originate), lighting up the protocol on fa 0/0 only, that is matching the local green subnet. A quick look at the OSPF entires in R1s routing table shows that we’re ready:
R1#show ip route ospf 192.168.45.0/30 is subnetted, 1 subnets O 192.168.45.0 [110/11] via 192.168.15.2, 01:47:25, FastEthernet0/0 192.168.25.0/30 is subnetted, 1 subnets O 192.168.25.0 [110/20] via 192.168.15.2, 01:47:25, FastEthernet0/0 192.168.35.0/30 is subnetted, 1 subnets O 192.168.35.0 [110/11] via 192.168.15.2, 01:47:25, FastEthernet0/0 O*E2 0.0.0.0/0 [110/1] via 192.168.15.2, 01:47:25, FastEthernet0/0
Multipoint GRE Tunnel
I’ve selected 172.16.0.0/24 for my tunnel interfaces. The hub and all spokes will need a logical interface in this subnet to act as the local tunnel ingress/egress. Each router gets a config like this, the example here being R1. Note that there is no tunnel destination on any router.
interface Tunnel0 ip address 172.16.0.1 255.255.255.0 tunnel source 192.168.15.1 tunnel mode gre multipoint
R2 would have a tunnel source of 192.168.25.1 etc.
The multipoint GRE tunnel is treated like a Non Broadcast Multi Access (NBMA) network, much like point-to-multipoint frame-relay.
The point of DMVPN is that any spoke should be able to bring up a tunnel to any other spoke on demand. We need a mechanism to define our hub router, and for spoke routers to be able to resolve the physical IP address in use by another spoke, enter Next Hop Resolution Protocol. Hub routers – NHRP clients – query the spoke router or Next Hop Server (NHS) to learn the address of another hub router with which they need to bring up a tunnel. It is notable that although each router in the topology has an interface in 172.16.0.0/24, this network is not known by R5 (my ‘Internet’ router). Instead, NHRP maps interfaces in this subnets to real interfaces in ‘globally routable’ subnets (inverted commas as I’ve used RFC1918 address in this example).
There a four commands needed to configure NHRP, although only two are used on the hub.
The ‘ip nhrp network-id’ must be the same across the DMVPN instance; it is quite possible and sometimes desirable to run multiple concurrent DMVPN overlays. For this lab, one is enough.
The ‘ip nhrp map multicast’ command sets up a mapping so that our NBMA network can process multicast and unknown unicast packets. We then have two options:
R1(config-if)#ip nhrp map multicast ? A.B.C.D IP NBMA address dynamic Dynamically learn destinations from client registrations on hub
On a spoke, we enter the IP NBMA address of our hub so that broadcast and unknown unicast traffic is sent there. We need to do that so that we can run an IGP over our mGRE network later. In my example that would be 192.168.15.1. On our hub we use the dynamic keyword.
Spokes need an additional map entry to link the hub’s tunnel address to its physical address, ‘ip nhrp map 172.16.0.1 192.168.15.1’ in our example.
Finally, for each spoke we need to define its NHS using the ‘ip nhrp nhs 172.16.0.1’ commend
Bringing it all together, to define our hub we add two commands to Tun0 on R1:
interface Tunnel0 ip nhrp map multicast dynamic ip nhrp network-id 1
To define a spoke, we add four commends to Tun0 on R2, R3 and R4:
interface Tunnel0 ip nhrp map 172.16.0.1 192.168.15.1 ip nhrp map multicast 192.168.15.1 ip nhrp network-id 1 ip nhrp nhs 172.16.0.1
Like many things, DMVPN relies on a sensible routing table on each router. Here is the logical topology:
We need each router to have routes to the other routers’ client networks. EIGRP lends itself well to this. We configure it to match 172.16.0.0/24 on each router, as well as the local client networks. We’ll make the client interfaces passive to stop our routers peering with anything downstream. Finally we’ll need to tweak the defaults on the hub so that:
* split-horizon doesn’t prevent us advertising subnets back out to the other spokes
* we advertise the next hop IP as that of the originating router
Here is R1’s eigrp config, the only change on the other routers is to match their local client networks as they all use the same local physical interfaces in my lab.
router eigrp 1 passive-interface FastEthernet0/1 network 10.10.10.0 0.0.0.255 network 172.16.0.0 0.0.0.255 no auto-summary
On R1 we also add:
interface Tunnel0 ! ip address 172.16.0.1 255.255.255.0 no ip next-hop-self eigrp 1 no ip split-horizon eigrp 1
First, let’s check the routing – each hub and spoke should know about all the other client networks via the mGRE tunnel:
R1#show ip route eigrp 184.108.40.206/24 is subnetted, 1 subnets D 220.127.116.11 [90/297270016] via 172.16.0.2, 02:58:14, Tunnel0 18.104.22.168/30 is subnetted, 1 subnets D 22.214.171.124 [90/297270016] via 172.16.0.4, 02:58:16, Tunnel0 126.96.36.199/24 is subnetted, 1 subnets D 188.8.131.52 [90/297270016] via 172.16.0.3, 02:58:16, Tunnel0 R4#show ip route eigrp 184.108.40.206/24 is subnetted, 1 subnets D 220.127.116.11 [90/310070016] via 172.16.0.2, 02:58:06, Tunnel0 10.0.0.0/24 is subnetted, 1 subnets D 10.10.10.0 [90/297270016] via 172.16.0.1, 02:58:06, Tunnel0 18.104.22.168/24 is subnetted, 1 subnets D 22.214.171.124 [90/310070016] via 172.16.0.3, 02:58:06, Tunnel0
On the hub, we should see our three peers:
R1#show dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incompletea N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer Tunnel0, Type:Hub, NHRP Peers:3, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 192.168.25.1 172.16.0.2 UP never D 1 192.168.35.1 172.16.0.3 UP never D 1 192.168.45.1 172.16.0.4 UP never D
On each spoke, we’ll just see the hub:
R4#show dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incompletea N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer Tunnel0, Type:Spoke, NHRP Peers:1, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 192.168.15.1 172.16.0.1 UP 02:56:54 S
Now, let’s try and ping 126.96.36.199 (R3’s client interface) from 188.8.131.52 (R4’s client interface):
R4#ping 184.108.40.206 source fa 0/1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 220.127.116.11, timeout is 2 seconds: Packet sent with a source address of 18.104.22.168 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 24/34/64 ms
Great, it worked. You can see that the dynamic tunnel has come up:
R4#show dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incompletea N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer Tunnel0, Type:Spoke, NHRP Peers:2, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 192.168.15.1 172.16.0.1 UP 03:00:22 S 1 192.168.35.1 172.16.0.3 UP never D
I’ve uploaded a packet capture taken during the above test (on the R4-R5 link) to cloudshark.
Click on that link and you’ll see the capture filtered to NHRP traffic only. Note that packets 16 and 19 flow via the hub, but 20 and 21 are spoke-to-spoke. If you open up the NHRP Mandatory Part in packet 16 you’ll see that the source and destination protocol addresses are the mGRE tunnel endpoints and the source NBMA address is the physical interface of the router.
In packet 20, NHRP provides R4 with R3’s physical address:
In packets 20 and 21, open the client information entry section to see that NHRP has returned 192.168.35.1, the physical address of R3 and endpoint for our dynamic, spoke-to-spoke tunnel.
Finally, let’s have a look at the pings. The first thing I should point out is that we are tunnelling the traffic, so there are two IP headers. In real life I would drop the IP MTU on each Tu interface to allow for the GRE encapsulation and extra IP header, plus any crypto.
Packet 15’s outer IP headers have source and destination physical addresses of R4 and R1. The inner headers are the local client subnets foreach router. This continues until packet 22 (right after NHRP resolved the true address). From this point, the traffic is direct. Note that in this case, the request went via R1, but the reply came direct as the spoke-to-spoke tunnel had come up.
In this post I’ve described and implemented a simple (crypto-free) DMVPN topology. I may expand on this in future to add encryption.
Update: I’ve covered how to add encryption here.