DMVPN - phase one - OSPF
Today I would like to start with very known and widely spread technology - Dynamic Multipoint Virtual Private Network (DMVPN). It was invented more than 10 years ago. DMVPN is very popular because it allows us to create point-to-point or full mesh secure connections between many hosts. The main problem with point-to-point VPN is its lack of scalability in terms of configuration management with more than few point-to-point tunnels.
DMVPN is not one protocol, it is collections of protocols, frameworks: IPsec, multipoint GRE (mGRE), Next Hop Resolution Protocol (NHRP).
Let’s look on the below scenario where you can see one hub (for example HQ) and two spokes (branches). You can try to compare it with GET VPN but between them are more differences than similarities. General speaking DMVPN consist of one hub router (it has to have static IP) and many spokes (static IP is not required). From configuration perspective spokes know IP of the hub router but IPs of spokes is not required and it can by dynamic.
In this post I describe OSPF protocol but you need to remember it is not widely used due to area issue (can be only one) and its scalability. More preferable protocols are EIGRP and BGP.
I put ASA in the middle to better understand what protocols, packets are exchanged between them.
Let’s start to configure basic IP connection and allow ping between all physical interfaces.
R1 (hub):
!
interface FastEthernet0/0
ip address 5.5.5.1 255.255.255.0
speed auto
duplex auto
!
interface Loopback0
ip address 11.11.11.11 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 5.5.5.2
!
R2 (spoke1):
!
interface FastEthernet0/0
ip address 6.6.6.1 255.255.255.0
speed auto
duplex auto
!
interface Loopback0
ip address 22.22.22.22 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 6.6.6.2
!
R3 (spoke2):
!
interface FastEthernet0/0
ip address 7.7.7.1 255.255.255.0
speed auto
duplex auto
!
interface Loopback0
ip address 33.33.33.33 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 7.7.7.2
ASA:
!
interface GigabitEthernet0
nameif hub
security-level 100
ip address 5.5.5.2 255.255.255.0
!
interface GigabitEthernet1
nameif spoke1
security-level 70
ip address 6.6.6.2 255.255.255.0
!
interface GigabitEthernet2
nameif spoke2
security-level 50
ip address 7.7.7.2 255.255.255.0
!
fixup protocol icmp
!
Now we can test it:
The R1 can ping both spokes without any rule on ASA because it has the highest security level:
R1#ping 6.6.6.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 6.6.6.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 32/63/132 ms
R1#ping 7.7.7.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 7.7.7.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 32/60/76 ms
R1#
The R2 can’t ping the hub but can spoke2 (security level):
R2#ping 5.5.5.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 5.5.5.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
R2#
R2#
R2#
R2#ping 7.7.7.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 7.7.7.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 24/46/56 ms
R2#
I need to add one access list:
%ASA-3-106014: Deny inbound icmp src spoke1:6.6.6.1 dst hub:5.5.5.1 (type 8, code 0)
access-list SPOKE1 ext permit icmp host 6.6.6.1 host 5.5.5.1 echo
access-group SPOKE1 in interface spoke1
One more test:
R2#ping 5.5.5.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 5.5.5.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 40/46/48 ms
R2#
Ok, I try now from R3:
R3#ping 5.5.5.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 5.5.5.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
R3#
R3#ping 6.6.6.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 6.6.6.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
R3#
as you see I can’t ping any of them because r3 is on the interface with the lowest security level:
%ASA-3-106014: Deny inbound icmp src spoke2:7.7.7.1 dst hub:5.5.5.1 (type 8, code 0)
%ASA-3-106014: Deny inbound icmp src spoke2:7.7.7.1 dst spoke1:6.6.6.1 (type 8, code 0)
Let’s fix it:
access-list SPOKE2 exte permit icmp host 7.7.7.1 host 5.5.5.1 echo
access-list SPOKE2 exte permit icmp host 7.7.7.1 host 6.6.6.1 echo
access-group SPOKE2 in interface SPOKE2
R3#ping 5.5.5.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 5.5.5.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 24/42/60 ms
R3#ping 6.6.6.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 6.6.6.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 40/50/64 ms
R3#
Ok, now we have connectivity and it’s time to configure tunnel interfaces. I prefer to enable tunnels first without any protection (IPsec profile) to make my life simpler in case something doesn’t work.
You don’t need to remember all the commands because it is available here (even during your ccie lab):
for hub:
for spoke:
R1 (hub):
interface Tunnel0
ip address 10.10.10.1 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication cisco
ip nhrp map multicast dynamic
ip nhrp network-id 12
tunnel source FastEthernet0/0
tunnel destination dynamic
tunnel mode gre multipoint
tunnel key 12
as you see some settings here are dynamic without static mapping (NHRP, destination).
R2 (spoke1):
interface Tunnel0
ip address 10.10.10.2 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication cisco
ip nhrp map multicast 5.5.5.1
ip nhrp map 10.10.10.1 5.5.5.1
ip nhrp network-id 12
ip nhrp nhs 10.10.10.1
tunnel source FastEthernet0/0
tunnel mode gre multipoint
tunnel key 12
for spokes we need to do mapping for NHRP for the tunnel and the physical interfaces.
R3 (spoke2):
!
interface Tunnel0
ip address 10.10.10.3 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication cisco
ip nhrp map multicast 5.5.5.1
ip nhrp map 10.10.10.1 5.5.5.1
ip nhrp network-id 12
ip nhrp nhs 10.10.10.1
tunnel source FastEthernet0/0
tunnel mode gre multipoint
tunnel key 12
Ok, all tunnels are up, let’s check what we see on the ASA:
%ASA-4-106023: Deny protocol 47 src spoke1:6.6.6.1 dst hub:5.5.5.1 by access-group "SPOKE1" [0x0, 0x0]
%ASA-4-106023: Deny protocol 47 src spoke2:7.7.7.1 dst hub:5.5.5.1 by access-group "SPOKE2" [0x0, 0x0]
We see packets from both spokes to the hub (protocol 47 - GRE) have been denied . I fix it now:
access-list SPOKE1 ext permit gre host 6.6.6.1 host 5.5.5.1
access-list SPOKE2 ext permit gre host 7.7.7.1 host 5.5.5.1
I test the connectivity between all tunnel interfaces:
R1#ping 10.10.10.3 source tun0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.3, timeout is 2 seconds:
Packet sent with a source address of 10.10.10.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 40/76/104 ms
R1#
R1#ping 10.10.10.2 source tun0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.2, timeout is 2 seconds:
Packet sent with a source address of 10.10.10.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 68/82/108 ms
R1#
R2#ping 10.10.10.1 source tun0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.1, timeout is 2 seconds:
Packet sent with a source address of 10.10.10.2
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 60/81/104 ms
R2#ping 10.10.10.3 source tun0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.3, timeout is 2 seconds:
Packet sent with a source address of 10.10.10.2
.....
Success rate is 0 percent (0/5)
R2#
The connection between spokes is not allowed:
%ASA-4-106023: Deny protocol 47 src spoke1:6.6.6.1 dst spoke2:7.7.7.1 by access-group "SPOKE1" [0x0, 0x0]
I need to add one entry to the SPOKE1 acl:
access-list SPOKE1 ext permit gre host 6.6.6.1 host 7.7.7.1
It solved the problem:
R2#ping 10.10.10.3 source tun0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.3, timeout is 2 seconds:
Packet sent with a source address of 10.10.10.2
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 64/82/92 ms
R2#
and last one from R3:
R3#ping 10.10.10.1 source tun0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.1, timeout is 2 seconds:
Packet sent with a source address of 10.10.10.3
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 48/78/96 ms
R3#ping 10.10.10.2 source tun0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.2, timeout is 2 seconds:
Packet sent with a source address of 10.10.10.3
.....
Success rate is 0 percent (0/5)
R3#
I have the same problem as with R2:
%ASA-4-106023: Deny protocol 47 src spoke2:7.7.7.1 dst spoke1:6.6.6.1 by access-group "SPOKE2" [0x0, 0x0]
I add following entry:
access-list SPOKE2 ext permit gre host 7.7.7.1 host 6.6.6.1
and the last test:
R3#ping 10.10.10.2 source tun0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.2, timeout is 2 seconds:
Packet sent with a source address of 10.10.10.3
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 76/80/92 ms
R3#
Ok, all tunnels can communicate with each other. Let’s check the connection status on the ASA:
asa1# sh conn
6 in use, 6 most used
GRE spoke2 7.7.7.1:0 hub 5.5.5.1:0, idle 0:00:08, bytes 1080, flags E
GRE spoke2 7.7.7.1:0 spoke1 6.6.6.1:0, idle 0:00:05, bytes 2517, flags E
GRE spoke1 6.6.6.1:0 hub 5.5.5.1:0, idle 0:00:03, bytes 1080, flags E
GRE spoke2 7.7.7.1:0 spoke1 6.6.6.1:0, idle 0:00:05, bytes 1620, flags E
GRE spoke2 7.7.7.1:0 hub 5.5.5.1:0, idle 0:00:08, bytes 1080, flags E
GRE spoke1 6.6.6.1:0 hub 5.5.5.1:0, idle 0:00:04, bytes 1080, flags E
asa1#
Now it’s time to add the tunnel protection. I’m going to add following IPsec (phase 1 and 2) configuration on all three routers:
crypto isakmp policy 10
encr 3des
authentication pre-share
group 2
crypto isakmp key cisco address 0.0.0.0
!
!
crypto ipsec transform-set TS esp-3des esp-sha-hmac
mode tunnel
!
crypto ipsec profile IPSEC-PRF
set transform-set TS
!
interface Tunnel0
tunnel protection ipsec profile IPSEC-PRF
!
Once I added above config on the ASA I see:
%ASA-4-106023: Deny protocol 50 src spoke1:6.6.6.1 dst hub:5.5.5.1 by access-group "SPOKE1" [0x0, 0x0]
%ASA-4-106023: Deny udp src spoke1:6.6.6.1/500 dst spoke2:7.7.7.1/500 by access-group "SPOKE1" [0x0, 0x0]
%ASA-4-106023: Deny udp src spoke1:6.6.6.1/500 dst spoke2:7.7.7.1/500 by access-group "SPOKE1" [0x0, 0x0]
%ASA-4-106023: Deny udp src spoke2:7.7.7.1/500 dst spoke1:6.6.6.1/500 by access-group "SPOKE2" [0x0, 0x0]
%ASA-4-106023: Deny udp src spoke1:6.6.6.1/500 dst spoke2:7.7.7.1/500 by access-group "SPOKE1" [0x0, 0x0]
%ASA-4-106023: Deny protocol 50 src spoke2:7.7.7.1 dst hub:5.5.5.1 by access-group "SPOKE2" [0x0, 0x0]
%ASA-4-106023: Deny udp src spoke2:7.7.7.1/500 dst spoke1:6.6.6.1/500 by access-group "SPOKE2" [0x0, 0x0]
%ASA-4-106023: Deny udp src spoke1:6.6.6.1/500 dst spoke2:7.7.7.1/500 by access-group "SPOKE1" [0x0, 0x0]
%ASA-4-106023: Deny protocol 50 src spoke1:6.6.6.1 dst spoke2:7.7.7.1 by access-group "SPOKE1" [0x0, 0x0]
%
access-list SPOKE1 ext permit esp host 6.6.6.1 host 5.5.5.1
access-list SPOKE2 ext permit esp host 7.7.7.1 host 5.5.5.1
access-list SPOKE1 ext permit esp host 6.6.6.1 host 7.7.7.1
access-list SPOKE2 ext permit esp host 7.7.7.1 host 6.6.6.1
access-list SPOKE1 ext permit udp host 6.6.6.1 host 7.7.7.1 eq 500
access-list SPOKE2 ext permit udp host 7.7.7.1 host 6.6.6.1 eq 500
access-list SPOKE2 ext permit udp host 7.7.7.1 host 5.5.5.1 eq 500
Let’s check the connection table on the ASA:
asa1(config)# sh conn
6 in use, 6 most used
ESP spoke2 7.7.7.1 hub5.5.5.1, idle 0:00:05, bytes 780
ESP spoke2 7.7.7.1 spoke16.6.6.1, idle 0:00:11, bytes 1560
ESP spoke1 6.6.6.1 hub5.5.5.1, idle 0:00:19, bytes 780
ESP spoke2 7.7.7.1 spoke16.6.6.1, idle 0:00:11, bytes 1560
ESP spoke2 7.7.7.1 hub5.5.5.1, idle 0:00:05, bytes 780
ESP spoke1 6.6.6.1 hub5.5.5.1, idle 0:00:19, bytes 780
asa1(config)#
We can’t see GRE anymore only ESP. What does it mean? It means the tunnel protection works fine and now all GRE packets are encrypted by ESP. For security reason I’m going to remove all permit GRE access list on ASA. It protects us against unencrypted communication in case of IPsec failure. I remember one discussion on one Cisco forum about GRE and IPsec. There was a topic if one connection is ‘IPsec over GRE’ or ‘GRE over IPsec’. In this case we see we have ‘GRE over IPsec’ because we can’t see GRE anymore, only ESP.
no access-list SPOKE1 extended permit gre host 6.6.6.1 host 5.5.5.1
no access-list SPOKE1 extended permit gre host 6.6.6.1 host 7.7.7.1
no access-list SPOKE2 extended permit gre host 7.7.7.1 host 5.5.5.1
no access-list SPOKE2 extended permit gre host 7.7.7.1 host 6.6.6.1
The last step is adding OSPF routing protocol to allow connect remote LANs.
R1:
router ospf 1
network 10.10.10.0 0.0.0.255 area 0
network 11.11.11.0 0.0.0.255 area 0
!
int tun0
ip ospf network point-to-multipoint
R2:
router ospf 1
network 10.10.10.0 0.0.0.255 area 0
network 22.22.22.0 0.0.0.255 area 0
!
int tun0
ip ospf network point-to-multipoint
R3:
router ospf 1
network 10.10.10.0 0.0.0.255 area 0
network 33.33.33.0 0.0.0.255 area 0
!
int tun0
ip ospf network point-to-multipoint
Now, I do the test:
R1:
R1#ping 22.22.22.22 source loop0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 22.22.22.22, timeout is 2 seconds:
Packet sent with a source address of 11.11.11.11
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 96/123/152 ms
R1#ping 33.33.33.33 source loop0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 33.33.33.33, timeout is 2 seconds:
Packet sent with a source address of 11.11.11.11
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 104/127/144 ms
R1#
R2:
R2#ping 11.11.11.11 source loop0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 11.11.11.11, timeout is 2 seconds:
Packet sent with a source address of 22.22.22.22
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 104/120/132 ms
R2#ping 33.33.33.33 source loop0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 33.33.33.33, timeout is 2 seconds:
Packet sent with a source address of 22.22.22.22
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 132/160/192 ms
R2#
R3:
R3#ping 11.11.11.11 so
R3#ping 11.11.11.11 source loop0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 11.11.11.11, timeout is 2 seconds:
Packet sent with a source address of 33.33.33.33
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 104/127/148 ms
R3#ping 22.22.22.22 source loop0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 22.22.22.22, timeout is 2 seconds:
Packet sent with a source address of 33.33.33.33
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 156/166/180 ms
R3#
As you see everything works fine. To be sure the traffic is encrypted I do one more test:
R1#ping 33.33.33.33 source loop0 repeat 100
Type escape sequence to abort.
Sending 100, 100-byte ICMP Echos to 33.33.33.33, timeout is 2 seconds:
Packet sent with a source address of 11.11.11.11
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (100/100), round-trip min/avg/max = 92/127/172 ms
R1#
R1#sh crypto session detail
Crypto session current status
Code: C - IKE Configuration mode, D - Dead Peer Detection
K - Keepalives, N - NAT-traversal, T - cTCP encapsulation
X - IKE Extended Authentication, F - IKE Fragmentation
Interface: Tunnel0
Uptime: 00:00:27
Session status: UP-ACTIVE
Peer: 7.7.7.1 port 500 fvrf: (none) ivrf: (none)
Phase1_id: 7.7.7.1
Desc: (none)
IKEv1 SA: local 5.5.5.1/500 remote 7.7.7.1/500 Active
Capabilities:(none) connid:1017 lifetime:23:59:32
IKEv1 SA: local 5.5.5.1/500 remote 7.7.7.1/500 Inactive
Capabilities:(none) connid:1015 lifetime:0
IKEv1 SA: local 5.5.5.1/500 remote 7.7.7.1/500 Inactive
Capabilities:(none) connid:1013 lifetime:0
IPSEC FLOW: permit 47 host 5.5.5.1 host 7.7.7.1
Active SAs: 4, origin: crypto map
Inbound: #pkts dec'ed 103 drop 0 life (KB/Sec) 4233144/3573
Outbound: #pkts enc'ed 102 drop 0 life (KB/Sec) 4233144/3573
Interface: Tunnel0
Uptime: 00:00:23
Session status: UP-ACTIVE
Peer: 6.6.6.1 port 500 fvrf: (none) ivrf: (none)
Phase1_id: 6.6.6.1
Desc: (none)
IKEv1 SA: local 5.5.5.1/500 remote 6.6.6.1/500 Active
Capabilities:(none) connid:1018 lifetime:23:59:35
IKEv1 SA: local 5.5.5.1/500 remote 6.6.6.1/500 Inactive
Capabilities:(none) connid:1016 lifetime:0
IKEv1 SA: local 5.5.5.1/500 remote 6.6.6.1/500 Inactive
Capabilities:(none) connid:1014 lifetime:0
IPSEC FLOW: permit 47 host 5.5.5.1 host 6.6.6.1
Active SAs: 4, origin: crypto map
Inbound: #pkts dec'ed 3 drop 0 life (KB/Sec) 4207370/3576
Outbound: #pkts enc'ed 1 drop 0 life (KB/Sec) 4207371/3576
R1#
As you see I sent 100 pings and for following flow we can see more than 100 packets:
IPSEC FLOW: permit 47 host 5.5.5.1 host 7.7.7.1
Active SAs: 4, origin: crypto map
Inbound: #pkts dec'ed 103 drop 0 life (KB/Sec) 4233144/3573
Outbound: #pkts enc'ed 102 drop 0 life (KB/Sec) 4233144/3573
The last thing to check is connection between spokes. As per title ‘DMVPN phase one’ I need to explain what does it mean. There are three phases of DMVPN. They were developed to improve existing phase one. The difference between them is possibility of direct connection between spokes in DMVPN phase 2 and 3 (I will speak about it in my next posts). For phase one, all traffic between spokes needs to be sent via the hub. What, of course, has impact on its performance, etc.
Let’s check traceroute from R2 to R3:
R2#traceroute 33.33.33.33 source 22.22.22.22
Type escape sequence to abort.
Tracing the route to 33.33.33.33
VRF info: (vrf in name/id, vrf out name/id)
1 10.10.10.1 92 msec 120 msec 128 msec
2 10.10.10.3 148 msec 160 msec 140 msec
R2#
At the end I check NHRP:
R1#sh ip nhrp
10.10.10.2/32 via 10.10.10.2
Tunnel0 created 01:04:13, expire 01:54:48
Type: dynamic, Flags: unique registered
NBMA address: 6.6.6.1
10.10.10.3/32 via 10.10.10.3
Tunnel0 created 01:04:16, expire 01:46:20
Type: dynamic, Flags: unique registered
NBMA address: 7.7.7.1
R1#
R2#sh ip nhrp
10.10.10.1/32 via 10.10.10.1
Tunnel0 created 02:17:11, never expire
Type: static, Flags: used
NBMA address: 5.5.5.1
R2#
R3#sh ip nhrp
10.10.10.1/32 via 10.10.10.1
Tunnel0 created 02:14:03, never expire
Type: static, Flags: used
NBMA address: 5.5.5.1
R3#
In my next post I will test phase 2.