DMVPN - phase three - OSPF

The third version of DMVPN is the improved version of phase 2. To be more specific there are two modes of the phase 3: early and new implementation. The main difference between them is not how it works but how you can check what is the real next-hop. The new commands (rather new parameter) is:

sh ip route next-hop-override

You can find there new sub-entries which show you the real next hop. The new version is available on: ASR1K, 15.2(1)T - ISR, 7200 and I don’t have chance to test it (on 7200 even with higher version it didn’t work, there was a problem with NHRP protocol). I was able to test only the early mode on 12.4.

Both versions support spoke-to-spoke communication but the version 3 was improved NHRP shortcut and redirection feature. Let’s do some tests.

dmvpn-1-1.jpg

R2#sh ver | i Ver
Cisco IOS Software, 7200 Software (C7200-ADVIPSERVICESK9-M), Version 12.4(11)T1, RELEASE SOFTWARE (fc5)
BOOTLDR: 7200 Software (C7200-ADVIPSERVICESK9-M), Version 12.4(11)T1, RELEASE SOFTWARE (fc5)
6 slot VXR midplane, Version 2.1
R2#

Before I send traffic from spoke1 to spoke2 I check CEF and NHRP tables:

R2#sh ip cef | i 33
33.33.33.33/32      10.10.10.1           Tunnel0
R2#
R2#sh ip nhrp
10.10.10.1/32 via 10.10.10.1, Tunnel0 created 00:01:16, never expire
  Type: static, Flags: nat used
  NBMA address: 5.5.5.1
R2#

Routing for LAN3 (33.33.33.33) is via the hub router:

R2#sh ip route 33.33.33.33
Routing entry for 33.33.33.33/32
  Known via "ospf 1", distance 110, metric 22223, type intra area
  Last update from 10.10.10.1 on Tunnel0, 00:01:03 ago
  Routing Descriptor Blocks:
  * 10.10.10.1, from 33.33.33.33, 00:01:03 ago, via Tunnel0
      Route metric is 22223, traffic share count is 1

R2#

Let’s check the status of DMVPN tunnels:

R2#sh dmvpn
Legend: Attrb --> S - Static, D - Dynamic, I - Incompletea
        N - NATed, L - Local, X - No Socket
        # Ent --> Number of NHRP entries with same NBMA peer

Tunnel0, Type:Spoke, NHRP Peers:1,
 # Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1         5.5.5.1      10.10.10.1    UP 00:01:34 S

R2#sh dmvpn details
Legend: Attrb --> S - Static, D - Dynamic, I - Incompletea
        N - NATed, L - Local, X - No Socket
        # Ent --> Number of NHRP entries with same NBMA peer

 -------------- Interface Tunnel0 info: --------------
Intf. is up, Line Protocol is up, Addr. is 10.10.10.2
   Source addr: 6.6.6.1, Dest addr: MGRE
  Protocol/Transport: "multi-GRE/IP", Protect "IPSEC-PRF",
Tunnel VRF "", ip vrf forwarding ""

NHRP Details: NHS:         10.10.10.1 RE

Type:Spoke, NBMA Peers:1
# Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb    Target Network
----- --------------- --------------- ----- -------- ----- -----------------
    1         5.5.5.1      10.10.10.1    UP 00:01:40 S         10.10.10.1/32

  IKE SA: local 6.6.6.1/500 remote 5.5.5.1/500 Active
  Crypto Session Status: UP-ACTIVE
  fvrf: (none)
  IPSEC FLOW: permit 47 host 6.6.6.1 host 5.5.5.1
        Active SAs: 2, origin: crypto map
   Outbound SPI : 0x1951C898, transform : esp-3des esp-sha-hmac
    Socket State: Open

Pending DMVPN Sessions:

Ok, now I send traffic to check how it will be processed by the hub:

R2#traceroute 33.33.33.33 source 22.22.22.22

Type escape sequence to abort.
Tracing the route to 33.33.33.33

  1 10.10.10.1 80 msec 76 msec 108 msec
  2 10.10.10.3 88 msec 124 msec 156 msec
R2#

R2#traceroute 33.33.33.33 source 22.22.22.22

Type escape sequence to abort.
Tracing the route to 33.33.33.33

  1 10.10.10.3 60 msec 108 msec 44 msec
R2#

The second traceroute shows direct connection between spokes. On the hub you can redirection process:

R1#
*Dec 25 12:47:44.803: NHRP: inserting (6.6.6.1/33.33.33.33) in redirect table
*Dec 25 12:47:44.819: NHRP: Attempting to send packet via DEST 22.22.22.22
*Dec 25 12:47:44.823: NHRP: Encapsulation succeeded.  Tunnel IP addr 6.6.6.1
*Dec 25 12:47:44.823: NHRP: Send Traffic Indication via Tunnel0 vrf 0, packet size: 97
*Dec 25 12:47:44.823:       src: 10.10.10.1, dst: 22.22.22.22
*Dec 25 12:47:44.823: NHRP: 97 bytes out Tunnel0
*Dec 25 12:47:44.851: NHRP: inserting (7.7.7.1/22.22.22.22) in redirect table
*Dec 25 12:47:44.871: NHRP: Attempting to send packet via DEST 10.10.10.3
*Dec 25 12:47:44.875: NHRP: Encapsulation succeeded.  Tunnel IP addr 7.7.7.1
*Dec 25 12:47:44.875: NHRP: Send Traffic Indication via Tunnel0 vrf 0, packet size: 97
*Dec 25 12:47:44.879:       src: 10.10.10.1, dst: 10.10.10.3
*Dec 25 12:47:44.883: NHRP: 97 bytes out Tunnel0
*Dec 25 12:47:44.967: NHRP: Receive Resolution Request via Tunnel0 vrf 0, packet size: 85
*Dec 25 12:47:44.971: NHRP: netid_in =
R1# 12, to_us = 0
*Dec 25 12:47:44.971: NHRP: nhrp_rtlookup yielded Tunnel0
*Dec 25 12:47:44.975: NHRP: netid_out 12, netid_in 12
*Dec 25 12:47:44.975: NHRP: nhrp_cache_lookup_comp returned 0x0
*Dec 25 12:47:44.975: NHRP: Attempting to send packet via DEST 33.33.33.33
*Dec 25 12:47:44.975: NHRP: Encapsulation succeeded.  Tunnel IP addr 7.7.7.1
*Dec 25 12:47:44.975: NHRP: Forwarding Resolution Request via Tunnel0 vrf 0, packet size: 105
*Dec 25 12:47:44.975:       src: 10.10.10.1, dst: 33.33.33.33
*Dec 25 12:47:44.979: NHRP: 105 bytes out Tunnel0
*Dec 25 12:47:45.011: NHRP: Receive Resolution Request via Tunnel0 vrf 0, packet size: 85
*Dec 25 12:47:45.019: NHRP: netid_in = 12, to_us = 0
*Dec 25 12:47:45.019: NHRP: nhrp_rtlookup yielded Tunnel0
*Dec 25 12:47:45.023: NHRP: netid_out 12, netid_in 12
*Dec 25 12:47:45.027: NHRP: nhrp_cache_lookup_comp returned 0x0
*Dec 25 12:47:45.027: NHRP: Attempting to send packet via DEST 22.22.22.22
*Dec 25 12:47:45.031: NHRP: Encapsulatio
R1#n succeeded.  Tunnel IP addr 6.6.6.1
*Dec 25 12:47:45.031: NHRP: Forwarding Resolution Request via Tunnel0 vrf 0, packet size: 105
*Dec 25 12:47:45.031:       src: 10.10.10.1, dst: 22.22.22.22
*Dec 25 12:47:45.031: NHRP: 105 bytes out Tunnel0
*Dec 25 12:47:45.567: NHRP: Receive Resolution Reply via Tunnel0 vrf 0, packet size: 133
*Dec 25 12:47:45.571: NHRP: netid_in = 0, to_us = 0
*Dec 25 12:47:45.575: NHRP: Finding next idb with in_pak id: 0
*Dec 25 12:47:45.575: NHRP: Attempting to send packet via DEST 10.10.10.2
*Dec 25 12:47:45.579: NHRP: Encapsulation succeeded.  Tunnel IP addr 6.6.6.1
*Dec 25 12:47:45.583: NHRP: Forwarding Resolution Reply via Tunnel0 vrf 0, packet size: 153
*Dec 25 12:47:45.587:       src: 10.10.10.1, dst: 10.10.10.2
*Dec 25 12:47:45.591: NHRP: 153 bytes out Tunnel0
*Dec 25 12:47:45.647: NHRP: Receive Resolution Reply via Tunnel0 vrf 0, packet size: 133
*Dec 25 12:47:45.651: NHRP: netid_in = 0, to_us = 0
*Dec 25 12:47:45.655: NHRP: Finding next
R1#idb with in_pak id: 0
*Dec 25 12:47:45.659: NHRP: Attempting to send packet via DEST 10.10.10.3
*Dec 25 12:47:45.663: NHRP: Encapsulation succeeded.  Tunnel IP addr 7.7.7.1
*Dec 25 12:47:45.663: NHRP: Forwarding Resolution Reply via Tunnel0 vrf 0, packet size: 153
*Dec 25 12:47:45.667:       src: 10.10.10.1, dst: 10.10.10.3
*Dec 25 12:47:45.671: NHRP: 153 bytes out Tunnel0
R1#

Spokes receive NHRP redirect message from their hub and on both: the hub and spokes you can find new dynamic entries:

R1#sh ip nhrp
10.10.10.2/32 via 10.10.10.2, Tunnel0 created 00:29:20, expire 01:30:39
  Type: dynamic, Flags: unique nat registered
  NBMA address: 6.6.6.1
10.10.10.3/32 via 10.10.10.3, Tunnel0 created 00:27:23, expire 01:32:36
  Type: dynamic, Flags: unique nat registered
  NBMA address: 7.7.7.1
R1#
R1#sh ip nhrp
10.10.10.2/32 via 10.10.10.2, Tunnel0 created 00:31:01, expire 01:28:58
  Type: dynamic, Flags: unique nat registered
  NBMA address: 6.6.6.1
10.10.10.3/32 via 10.10.10.3, Tunnel0 created 00:29:04, expire 01:30:55
  Type: dynamic, Flags: unique nat registered
  NBMA address: 7.7.7.1
22.22.22.0/24 via 10.10.10.2, Tunnel0 created 00:01:05, expire 01:58:53
  Type: dynamic, Flags: router nat
  NBMA address: 6.6.6.1
    (no-socket)
33.33.33.0/24 via 10.10.10.3, Tunnel0 created 00:01:06, expire 01:58:54
  Type: dynamic, Flags: router nat
  NBMA address: 7.7.7.1
    (no-socket)
R1#

As you see CEF entry is still the same pointing 33.33.33.33 via the HUB:

R2#sh ip cef | i 33
33.33.33.33/32      10.10.10.1           Tunnel0
R2#

On the R2 you can find new dynamic tunnels:

R2#sh ip nhrp
10.10.10.1/32 via 10.10.10.1, Tunnel0 created 00:02:22, never expire
  Type: static, Flags: nat used
  NBMA address: 5.5.5.1
10.10.10.3/32 via 10.10.10.3, Tunnel0 created 00:00:31, expire 01:59:29
  Type: dynamic, Flags: router nat implicit
  NBMA address: 7.7.7.1
22.22.22.0/24 via 10.10.10.2, Tunnel0 created 00:00:30, expire 01:59:29
  Type: dynamic, Flags: router unique nat local
  NBMA address: 6.6.6.1
    (no-socket)
33.33.33.0/24 via 10.10.10.3, Tunnel0 created 00:00:30, expire 01:59:29
  Type: dynamic, Flags: router nat used
  NBMA address: 7.7.7.1
R2#

Routing is still the same (in Phase 2 the routing for LAN3 was via R3):

R2#sh ip route 33.33.33.33
Routing entry for 33.33.33.33/32
  Known via "ospf 1", distance 110, metric 22223, type intra area
  Last update from 10.10.10.1 on Tunnel0, 00:02:10 ago
  Routing Descriptor Blocks:
  * 10.10.10.1, from 33.33.33.33, 00:02:10 ago, via Tunnel0
      Route metric is 22223, traffic share count is 1

R2#

So, only DMVPN tunnels show us how the traffic is sent:

R2#sh dmvpn
Legend: Attrb --> S - Static, D - Dynamic, I - Incompletea
        N - NATed, L - Local, X - No Socket
        # Ent --> Number of NHRP entries with same NBMA peer

Tunnel0, Type:Spoke, NHRP Peers:2,
 # Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1         5.5.5.1      10.10.10.1    UP 00:02:40 S
     2         7.7.7.1      10.10.10.3    UP 00:00:48 D
R2#sh dmvpn de
Legend: Attrb --> S - Static, D - Dynamic, I - Incompletea
        N - NATed, L - Local, X - No Socket
        # Ent --> Number of NHRP entries with same NBMA peer

 -------------- Interface Tunnel0 info: --------------
Intf. is up, Line Protocol is up, Addr. is 10.10.10.2
   Source addr: 6.6.6.1, Dest addr: MGRE
  Protocol/Transport: "multi-GRE/IP", Protect "IPSEC-PRF",
Tunnel VRF "", ip vrf forwarding ""

NHRP Details: NHS:         10.10.10.1 RE

Type:Spoke, NBMA Peers:3
# Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb    Target Network
----- --------------- --------------- ----- -------- ----- -----------------
    1         5.5.5.1      10.10.10.1    UP 00:02:47 S         10.10.10.1/32

  IKE SA: local 6.6.6.1/500 remote 5.5.5.1/500 Active
  Crypto Session Status: UP-ACTIVE
  fvrf: (none)
  IPSEC FLOW: permit 47 host 6.6.6.1 host 5.5.5.1
        Active SAs: 2, origin: crypto map
   Outbound SPI : 0x1951C898, transform : esp-3des esp-sha-hmac
    Socket State: Open
# Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb    Target Network
----- --------------- --------------- ----- -------- ----- -----------------
    2         7.7.7.1      10.10.10.3    UP 00:00:59 D         10.10.10.3/32
                                         UP 00:00:58 D         33.33.33.0/24

  IKE SA: local 6.6.6.1/500 remote 7.7.7.1/500 Active
  IKE SA: local 6.6.6.1/500 remote 7.7.7.1/500 Active
  Crypto Session Status: UP-ACTIVE
  fvrf: (none)
  IPSEC FLOW: permit 47 host 6.6.6.1 host 7.7.7.1
        Active SAs: 2, origin: crypto map
   Outbound SPI : 0xE2AC64A5, transform : esp-3des esp-sha-hmac
    Socket State: Open
# Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb    Target Network
----- --------------- --------------- ----- -------- ----- -----------------
    1         6.6.6.1      10.10.10.2    UP 00:00:58 DLX       22.22.22.0/24


Pending DMVPN Sessions:

R2#

Comparing to the phase 2, the new version (3) for OSPF was improved by removing limit of two hubs (DR and BDR) - you need to only change OSPF network type from ‘broadcast’ to ‘point -to-multipoint’.

 
6
Kudos
 
6
Kudos

Now read this

FlexVPN - troubleshooting

I would like to share with you very useful commands which can helpful you during your troubleshooting. Instead of checking (sometimes very long) configuration, you can check/compare the same block of configuration between your peers. 1)... Continue →