IPSEC VPN results between ERL and ERPoE
-
I was recently asked how the VPN throughput was for the EdgeRouters. Both of these routers are running the current standard v1.7 firmware. One is an ER-PoE and the other is an ERL.
Here are the results of iperf3 from one user desktop in St Louis to a user in Quincy.
The St. Louis connection has a traffic shaper policy in place but is set to 95mbit. I will remove it and test again to be sure.St Louis: ER-PoE 100/100 fiber from AT&T
Quincy: ERL 50/50 fiber from AdamsC:\iperf3>iperf3 -c 10.202.30.35 -p 9676 -F office2013.iso -t 120 Connecting to host 10.202.30.35, port 9676 [ 4] local 10.202.1.126 port 63430 connected to 10.202.30.35 port 9676 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.01 sec 384 KBytes 3.10 Mbits/sec [ 4] 1.01-2.01 sec 768 KBytes 6.30 Mbits/sec [ 4] 2.01-3.01 sec 1.38 MBytes 11.6 Mbits/sec [ 4] 3.01-4.01 sec 1.88 MBytes 15.8 Mbits/sec [ 4] 4.01-5.01 sec 2.50 MBytes 21.0 Mbits/sec [ 4] 5.01-6.01 sec 2.62 MBytes 22.1 Mbits/sec [ 4] 6.01-7.00 sec 2.12 MBytes 17.9 Mbits/sec [ 4] 7.00-8.00 sec 2.62 MBytes 22.1 Mbits/sec [ 4] 8.00-9.00 sec 2.38 MBytes 20.0 Mbits/sec [ 4] 9.00-10.02 sec 2.25 MBytes 18.6 Mbits/sec [ 4] 10.02-11.01 sec 2.75 MBytes 23.1 Mbits/sec [ 4] 11.01-12.01 sec 2.62 MBytes 22.1 Mbits/sec [ 4] 12.01-13.01 sec 2.25 MBytes 18.9 Mbits/sec [ 4] 13.01-14.01 sec 2.62 MBytes 22.1 Mbits/sec [ 4] 14.01-15.01 sec 2.00 MBytes 16.8 Mbits/sec [ 4] 15.01-16.01 sec 2.12 MBytes 17.9 Mbits/sec [ 4] 16.01-17.00 sec 1.50 MBytes 12.6 Mbits/sec [ 4] 17.00-18.00 sec 2.25 MBytes 18.9 Mbits/sec [ 4] 18.00-19.00 sec 2.75 MBytes 23.1 Mbits/sec [ 4] 19.00-20.01 sec 1.88 MBytes 15.5 Mbits/sec [ 4] 20.01-21.01 sec 2.50 MBytes 21.0 Mbits/sec [ 4] 21.01-22.01 sec 2.38 MBytes 20.0 Mbits/sec [ 4] 22.01-23.01 sec 2.00 MBytes 16.8 Mbits/sec [ 4] 23.01-24.01 sec 2.62 MBytes 22.1 Mbits/sec [ 4] 24.01-25.01 sec 2.00 MBytes 16.8 Mbits/sec [ 4] 25.01-26.01 sec 1.50 MBytes 12.6 Mbits/sec [ 4] 26.01-27.00 sec 1.62 MBytes 13.7 Mbits/sec [ 4] 27.00-28.00 sec 2.38 MBytes 20.0 Mbits/sec [ 4] 28.00-29.00 sec 2.38 MBytes 20.0 Mbits/sec [ 4] 29.00-30.01 sec 640 KBytes 5.17 Mbits/sec [ 4] 30.01-31.01 sec 1.12 MBytes 9.45 Mbits/sec [ 4] 31.01-32.01 sec 1.75 MBytes 14.7 Mbits/sec [ 4] 32.01-33.01 sec 2.38 MBytes 20.0 Mbits/sec [ 4] 33.01-34.01 sec 2.50 MBytes 21.0 Mbits/sec [ 4] 34.01-35.01 sec 2.00 MBytes 16.8 Mbits/sec [ 4] 35.01-36.01 sec 2.62 MBytes 22.1 Mbits/sec [ 4] 36.01-37.00 sec 2.25 MBytes 18.9 Mbits/sec [ 4] 37.00-38.00 sec 1.12 MBytes 9.45 Mbits/sec [ 4] 38.00-39.00 sec 1.88 MBytes 15.8 Mbits/sec [ 4] 39.00-40.01 sec 2.50 MBytes 20.7 Mbits/sec [ 4] 40.01-41.01 sec 2.38 MBytes 20.0 Mbits/sec [ 4] 41.01-42.01 sec 2.00 MBytes 16.8 Mbits/sec [ 4] 42.01-43.01 sec 2.62 MBytes 22.1 Mbits/sec [ 4] 43.01-44.01 sec 2.00 MBytes 16.8 Mbits/sec [ 4] 44.01-45.01 sec 2.25 MBytes 18.9 Mbits/sec [ 4] 45.01-46.00 sec 2.62 MBytes 22.1 Mbits/sec [ 4] 46.00-47.00 sec 1.25 MBytes 10.5 Mbits/sec [ 4] 47.00-48.00 sec 1.50 MBytes 12.6 Mbits/sec [ 4] 48.00-49.02 sec 2.12 MBytes 17.6 Mbits/sec [ 4] 49.02-50.01 sec 2.75 MBytes 23.1 Mbits/sec [ 4] 50.01-51.01 sec 1.38 MBytes 11.6 Mbits/sec [ 4] 51.01-52.01 sec 1.38 MBytes 11.6 Mbits/sec [ 4] 52.01-53.01 sec 2.00 MBytes 16.8 Mbits/sec [ 4] 53.01-54.01 sec 2.50 MBytes 21.0 Mbits/sec [ 4] 54.01-55.01 sec 1.75 MBytes 14.7 Mbits/sec [ 4] 55.01-56.00 sec 2.25 MBytes 18.9 Mbits/sec [ 4] 56.00-57.00 sec 2.88 MBytes 24.2 Mbits/sec [ 4] 57.00-58.00 sec 1.12 MBytes 9.45 Mbits/sec [ 4] 58.00-59.02 sec 1.62 MBytes 13.4 Mbits/sec [ 4] 59.02-60.01 sec 1.88 MBytes 15.8 Mbits/sec [ 4] 60.01-61.01 sec 1.00 MBytes 8.40 Mbits/sec [ 4] 61.01-62.01 sec 1.25 MBytes 10.5 Mbits/sec [ 4] 62.01-63.01 sec 1.12 MBytes 9.45 Mbits/sec [ 4] 63.01-64.01 sec 1.62 MBytes 13.7 Mbits/sec [ 4] 64.01-65.01 sec 2.25 MBytes 18.9 Mbits/sec [ 4] 65.01-66.00 sec 2.62 MBytes 22.1 Mbits/sec [ 4] 66.00-67.00 sec 1.88 MBytes 15.8 Mbits/sec [ 4] 67.00-68.00 sec 2.25 MBytes 18.9 Mbits/sec [ 4] 68.00-69.01 sec 1.62 MBytes 13.4 Mbits/sec [ 4] 69.01-70.01 sec 2.25 MBytes 18.9 Mbits/sec [ 4] 70.01-70.86 sec 2.25 MBytes 22.4 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-70.86 sec 142 MBytes 16.8 Mbits/sec sender Sent 142 MByte / 1.39 GByte (10%) of office2013.iso [ 4] 0.00-70.86 sec 142 MBytes 16.8 Mbits/sec receiver iperf Done. -
Here is a sample of ping time. The ping was started before the tunnel was up. that is where the packet loss came from.
--- 10.202.1.1 ping statistics --- 1940 packets transmitted, 1750 received, 9% packet loss, time 1941474ms rtt min/avg/max/mdev = 43.059/44.636/55.135/0.855 ms -
Here is the result with 2 streams.
I do not have my buffer set high in command prompt but no one needs to see ever bit anyway.
C:\iperf3>iperf3 -c 10.202.30.35 -p 9676 -F office2013.iso -t 120 -P 2 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 71.01-72.01 sec 1.25 MBytes 10.5 Mbits/sec [ 7] 71.01-72.01 sec 1.25 MBytes 10.5 Mbits/sec [SUM] 71.01-72.01 sec 2.50 MBytes 21.0 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 72.01-73.01 sec 1.12 MBytes 9.45 Mbits/sec [ 7] 72.01-73.01 sec 1.00 MBytes 8.40 Mbits/sec [SUM] 72.01-73.01 sec 2.12 MBytes 17.9 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 73.01-74.01 sec 1.38 MBytes 11.6 Mbits/sec [ 7] 73.01-74.01 sec 1.38 MBytes 11.6 Mbits/sec [SUM] 73.01-74.01 sec 2.75 MBytes 23.1 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 74.01-75.01 sec 1.88 MBytes 15.8 Mbits/sec [ 7] 74.01-75.01 sec 1.75 MBytes 14.7 Mbits/sec [SUM] 74.01-75.01 sec 3.62 MBytes 30.5 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 75.01-76.00 sec 1.75 MBytes 14.7 Mbits/sec [ 7] 75.01-76.00 sec 1.62 MBytes 13.7 Mbits/sec [SUM] 75.01-76.00 sec 3.38 MBytes 28.4 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 76.00-77.00 sec 1.88 MBytes 15.8 Mbits/sec [ 7] 76.00-77.00 sec 1.75 MBytes 14.7 Mbits/sec [SUM] 76.00-77.00 sec 3.62 MBytes 30.5 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 77.00-78.00 sec 2.00 MBytes 16.8 Mbits/sec [ 7] 77.00-78.00 sec 2.00 MBytes 16.8 Mbits/sec [SUM] 77.00-78.00 sec 4.00 MBytes 33.6 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 78.00-79.01 sec 1.75 MBytes 14.5 Mbits/sec [ 7] 78.00-79.01 sec 1.75 MBytes 14.5 Mbits/sec [SUM] 78.00-79.01 sec 3.50 MBytes 29.0 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 79.01-80.01 sec 2.25 MBytes 18.9 Mbits/sec [ 7] 79.01-80.01 sec 1.75 MBytes 14.7 Mbits/sec [SUM] 79.01-80.01 sec 4.00 MBytes 33.6 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 80.01-81.01 sec 2.62 MBytes 22.1 Mbits/sec [ 7] 80.01-81.01 sec 1.62 MBytes 13.7 Mbits/sec [SUM] 80.01-81.01 sec 4.25 MBytes 35.7 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 81.01-82.01 sec 1.75 MBytes 14.7 Mbits/sec [ 7] 81.01-82.01 sec 2.25 MBytes 18.9 Mbits/sec [SUM] 81.01-82.01 sec 4.00 MBytes 33.6 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 82.01-83.01 sec 896 KBytes 7.35 Mbits/sec [ 7] 82.01-83.01 sec 1.75 MBytes 14.7 Mbits/sec [SUM] 82.01-83.01 sec 2.62 MBytes 22.1 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 83.01-84.01 sec 640 KBytes 5.25 Mbits/sec [ 7] 83.01-84.01 sec 640 KBytes 5.25 Mbits/sec [SUM] 83.01-84.01 sec 1.25 MBytes 10.5 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 84.01-85.00 sec 1.25 MBytes 10.5 Mbits/sec [ 7] 84.01-85.00 sec 1.25 MBytes 10.5 Mbits/sec [SUM] 84.01-85.00 sec 2.50 MBytes 21.0 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 85.00-85.33 sec 640 KBytes 16.0 Mbits/sec [ 7] 85.00-85.33 sec 640 KBytes 16.0 Mbits/sec [SUM] 85.00-85.33 sec 1.25 MBytes 32.0 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-85.33 sec 133 MBytes 13.1 Mbits/sec sender Sent 133 MByte / 1.39 GByte (9%) of office2013.iso [ 4] 0.00-85.33 sec 133 MBytes 13.0 Mbits/sec receiver [ 7] 0.00-85.33 sec 142 MBytes 14.0 Mbits/sec sender Sent 142 MByte / 1.39 GByte (10%) of office2013.iso [ 7] 0.00-85.33 sec 142 MBytes 14.0 Mbits/sec receiver [SUM] 0.00-85.33 sec 275 MBytes 27.0 Mbits/sec sender [SUM] 0.00-85.33 sec 275 MBytes 27.0 Mbits/sec receiver iperf Done. -
Here is a sample with 4 threads.
C:\iperf3>iperf3 -c 10.202.30.35 -p 9676 -F office2013.iso -t 120 -P 4 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 111.01-112.01 sec 1.25 MBytes 10.5 Mbits/sec [ 7] 111.01-112.01 sec 1.00 MBytes 8.40 Mbits/sec [ 10] 111.01-112.01 sec 768 KBytes 6.30 Mbits/sec [ 13] 111.01-112.01 sec 1.12 MBytes 9.45 Mbits/sec [SUM] 111.01-112.01 sec 4.12 MBytes 34.7 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 112.01-113.01 sec 1.25 MBytes 10.5 Mbits/sec [ 7] 112.01-113.01 sec 1.25 MBytes 10.5 Mbits/sec [ 10] 112.01-113.01 sec 640 KBytes 5.25 Mbits/sec [ 13] 112.01-113.01 sec 896 KBytes 7.35 Mbits/sec [SUM] 112.01-113.01 sec 4.00 MBytes 33.6 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 113.01-114.01 sec 1.00 MBytes 8.40 Mbits/sec [ 7] 113.01-114.01 sec 1.75 MBytes 14.7 Mbits/sec [ 10] 113.01-114.01 sec 896 KBytes 7.35 Mbits/sec [ 13] 113.01-114.01 sec 1.00 MBytes 8.40 Mbits/sec [SUM] 113.01-114.01 sec 4.62 MBytes 38.9 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 114.01-115.00 sec 1.00 MBytes 8.40 Mbits/sec [ 7] 114.01-115.00 sec 1.00 MBytes 8.40 Mbits/sec [ 10] 114.01-115.00 sec 640 KBytes 5.25 Mbits/sec [ 13] 114.01-115.00 sec 1.50 MBytes 12.6 Mbits/sec [SUM] 114.01-115.00 sec 4.12 MBytes 34.7 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 115.00-116.00 sec 384 KBytes 3.15 Mbits/sec [ 7] 115.00-116.00 sec 896 KBytes 7.35 Mbits/sec [ 10] 115.00-116.00 sec 256 KBytes 2.10 Mbits/sec [ 13] 115.00-116.00 sec 768 KBytes 6.30 Mbits/sec [SUM] 115.00-116.00 sec 2.25 MBytes 18.9 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 116.00-117.00 sec 640 KBytes 5.25 Mbits/sec [ 7] 116.00-117.00 sec 1.38 MBytes 11.6 Mbits/sec [ 10] 116.00-117.00 sec 768 KBytes 6.30 Mbits/sec [ 13] 116.00-117.00 sec 1.12 MBytes 9.45 Mbits/sec [SUM] 116.00-117.00 sec 3.88 MBytes 32.6 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 117.00-118.01 sec 768 KBytes 6.20 Mbits/sec [ 7] 117.00-118.01 sec 1.25 MBytes 10.3 Mbits/sec [ 10] 117.00-118.01 sec 896 KBytes 7.24 Mbits/sec [ 13] 117.00-118.01 sec 1.62 MBytes 13.4 Mbits/sec [SUM] 117.00-118.01 sec 4.50 MBytes 37.2 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 118.01-119.01 sec 512 KBytes 4.20 Mbits/sec [ 7] 118.01-119.01 sec 1.50 MBytes 12.6 Mbits/sec [ 10] 118.01-119.01 sec 640 KBytes 5.25 Mbits/sec [ 13] 118.01-119.01 sec 1.25 MBytes 10.5 Mbits/sec [SUM] 118.01-119.01 sec 3.88 MBytes 32.6 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 119.01-120.01 sec 896 KBytes 7.35 Mbits/sec [ 7] 119.01-120.01 sec 1.25 MBytes 10.5 Mbits/sec [ 10] 119.01-120.01 sec 768 KBytes 6.30 Mbits/sec [ 13] 119.01-120.01 sec 896 KBytes 7.35 Mbits/sec [SUM] 119.01-120.01 sec 3.75 MBytes 31.5 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-120.01 sec 127 MBytes 8.85 Mbits/sec sender Sent 127 MByte / 1.39 GByte (8%) of office2013.iso [ 4] 0.00-120.01 sec 127 MBytes 8.84 Mbits/sec receiver [ 7] 0.00-120.01 sec 127 MBytes 8.87 Mbits/sec sender Sent 127 MByte / 1.39 GByte (8%) of office2013.iso [ 7] 0.00-120.01 sec 127 MBytes 8.86 Mbits/sec receiver [ 10] 0.00-120.01 sec 118 MBytes 8.26 Mbits/sec sender Sent 118 MByte / 1.39 GByte (8%) of office2013.iso [ 10] 0.00-120.01 sec 118 MBytes 8.25 Mbits/sec receiver [ 13] 0.00-120.01 sec 120 MBytes 8.41 Mbits/sec sender Sent 120 MByte / 1.39 GByte (8%) of office2013.iso [ 13] 0.00-120.01 sec 120 MBytes 8.40 Mbits/sec receiver [SUM] 0.00-120.01 sec 492 MBytes 34.4 Mbits/sec sender [SUM] 0.00-120.01 sec 491 MBytes 34.3 Mbits/sec receiver iperf Done. -
As you can see with a 50/50 pipe as my limiter on one end the performance seems very reasonable..
-
For comparison here is a session going over OpenVPN to another site with an 80/5 cable modem service.
Maxing under 8mbit on average.
C:\iperf3>iperf3 -c 10.202.10.49 -p 9676 -F office2013.iso -t 120 -P 4 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 113.01-114.01 sec 128 KBytes 1.05 Mbits/sec [ 7] 113.01-114.01 sec 384 KBytes 3.15 Mbits/sec [ 10] 113.01-114.01 sec 256 KBytes 2.10 Mbits/sec [ 13] 113.01-114.01 sec 128 KBytes 1.05 Mbits/sec [SUM] 113.01-114.01 sec 896 KBytes 7.35 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 114.01-115.00 sec 256 KBytes 2.10 Mbits/sec [ 7] 114.01-115.00 sec 384 KBytes 3.15 Mbits/sec [ 10] 114.01-115.00 sec 256 KBytes 2.10 Mbits/sec [ 13] 114.01-115.00 sec 256 KBytes 2.10 Mbits/sec [SUM] 114.01-115.00 sec 1.12 MBytes 9.45 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 115.00-116.00 sec 256 KBytes 2.10 Mbits/sec [ 7] 115.00-116.00 sec 512 KBytes 4.20 Mbits/sec [ 10] 115.00-116.00 sec 128 KBytes 1.05 Mbits/sec [ 13] 115.00-116.00 sec 0.00 Bytes 0.00 bits/sec [SUM] 115.00-116.00 sec 896 KBytes 7.35 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 116.00-117.00 sec 256 KBytes 2.10 Mbits/sec [ 7] 116.00-117.00 sec 384 KBytes 3.15 Mbits/sec [ 10] 116.00-117.00 sec 0.00 Bytes 0.00 bits/sec [ 13] 116.00-117.00 sec 0.00 Bytes 0.00 bits/sec [SUM] 116.00-117.00 sec 640 KBytes 5.25 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 117.00-118.01 sec 256 KBytes 2.07 Mbits/sec [ 7] 117.00-118.01 sec 384 KBytes 3.10 Mbits/sec [ 10] 117.00-118.01 sec 128 KBytes 1.03 Mbits/sec [ 13] 117.00-118.01 sec 128 KBytes 1.03 Mbits/sec [SUM] 117.00-118.01 sec 896 KBytes 7.24 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 118.01-119.01 sec 384 KBytes 3.15 Mbits/sec [ 7] 118.01-119.01 sec 384 KBytes 3.15 Mbits/sec [ 10] 118.01-119.01 sec 128 KBytes 1.05 Mbits/sec [ 13] 118.01-119.01 sec 128 KBytes 1.05 Mbits/sec [SUM] 118.01-119.01 sec 1.00 MBytes 8.40 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 119.01-120.01 sec 384 KBytes 3.15 Mbits/sec [ 7] 119.01-120.01 sec 128 KBytes 1.05 Mbits/sec [ 10] 119.01-120.01 sec 128 KBytes 1.05 Mbits/sec [ 13] 119.01-120.01 sec 256 KBytes 2.10 Mbits/sec [SUM] 119.01-120.01 sec 896 KBytes 7.35 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-120.01 sec 27.5 MBytes 1.92 Mbits/sec sender Sent 27.5 MByte / 1.39 GByte (1%) of office2013.iso [ 4] 0.00-120.01 sec 27.3 MBytes 1.91 Mbits/sec receiver [ 7] 0.00-120.01 sec 30.1 MBytes 2.11 Mbits/sec sender Sent 30.1 MByte / 1.39 GByte (2%) of office2013.iso [ 7] 0.00-120.01 sec 30.0 MBytes 2.09 Mbits/sec receiver [ 10] 0.00-120.01 sec 25.6 MBytes 1.79 Mbits/sec sender Sent 25.6 MByte / 1.39 GByte (1%) of office2013.iso [ 10] 0.00-120.01 sec 25.5 MBytes 1.78 Mbits/sec receiver [ 13] 0.00-120.01 sec 25.1 MBytes 1.76 Mbits/sec sender Sent 25.1 MByte / 1.39 GByte (1%) of office2013.iso [ 13] 0.00-120.01 sec 24.9 MBytes 1.74 Mbits/sec receiver [SUM] 0.00-120.01 sec 108 MBytes 7.58 Mbits/sec sender [SUM] 0.00-120.01 sec 108 MBytes 7.53 Mbits/sec receiver iperf Done.
