The sixth part of my ongoing series of posts on Ansible for Networking will cover VyOS. You can view the other posts in the series below: -

All the playbooks, roles and variables used in this article are available in my Network Automation with Ansible repository.

Why VyOS?

VyOS is a fork of Vyatta. Vyatta was started in 2006 to compete with the likes of Cisco and Juniper by providing a Linux-based (Debian specifically) network platform which can be run on bare metal, or in virtual machines. It can be used anywhere from a home router, in the cloud to the edge of a service provider network and everywhere in between.

When Vyatta was acquired in by Brocade in 2012, a group of developers forked the last community release of Vyatta and created the VyOS project. Also, Ubiquiti Networks forked Vyatta themselves, creating EdgeOS. While VyOS and EdgeOS no longer share the same codebase, there are enough similarities that switching between the two is not too difficult.

VyOS is well suited to use either in virtualization environments, labs, clouds, low powered hardware, or as the primary routing platform in a network. This versatility is why I have chosen to cover it.

Configuration Style

The configuration approach within VyOS is similar to Juniper’s JunOS. For example, to configure an interface in JunOS, you would use something like: -

[email protected]# set interfaces fxp0 unit 0 family inet address 10.15.30.33/24

The equivalent in VyOS is: -

[email protected]# set interfaces ethernet eth1 address '10.15.30.63/24'

Also, VyOS uses a similar candidate configuration system, where changes are made to the candidate configuration and then committed to the running configuration. You can check what changes will be made using show | compare (just like in JunOS).

The VyOS syntax does have many similarities to IOS though. For example, VyOS uses route maps (like IOS) rather than route policies (like JunOS).

Many of the verification commands are identical to Cisco: -

[email protected]:~$ show ip bgp summary

IPv4 Unicast Summary:
BGP router identifier 192.0.2.105, local AS number 65105 vrf-id 0
BGP table version 8
RIB entries 15, using 2760 bytes of memory
Peers 2, using 41 KiB of memory

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
10.100.105.254  4      65430      36      36        0    0    0 00:30:08            1
192.0.2.205     4      65105      32      35        0    0    0 00:29:19            0

Total number of neighbors 2
[email protected]:~$ show ip ospf neighbor

Neighbor ID     Pri State           Dead Time Address         Interface                        RXmtL RqstL DBsmL
192.0.2.205       1 Full/DR           33.863s 10.100.205.253  eth2.205:10.100.205.254              0     0     0

If you are well versed in both IOS and JunOS, VyOS will feel very familiar.

Objectives

For each vendor, I will be using Ansible to configure two routers/switches/firewalls/appliances.

One will serve as the Edge router, connecting to the Internet and also via BGP to the Net Server. The Net Server is a CentOS 8 Virtual Machine acting as a route server, syslog collector and TACACS+ server (detailed in The Lab Environment)

The other will be an internal router, performing core functions (i.e. internal routing rather than external).

This lab is based upon the rolling release version of VyOS.

Edge router

The edge router will run the following: -

  • External BGP (eBGP) to the Net Server
    • Advertising internal networks
  • Internal BGP (iBGP) to the Internal router
    • Advertising any routes received from the Net Server
    • Advertising a default route (for internet access)
  • OSPF
    • Advertising loopbacks and internal networks between both routers
  • IPv4 and IPv6 routing
    • Using OSPFv3 (for IPv6 support)
    • Using the IPv6 Address Family for BGP
  • SNMPv3 for monitoring
  • IPv4 NAT to allow internet access
    • I cannot run IPv6 for internet access, as my current ISP does not support IPv6
  • Logging via Syslog to the Net Server
  • Authentication, Authorization and Accounting (AAA) via TACACS+ RADIUS to the Net Server
  • Zones to place interfaces in, for zone-based firewalling
  • Firewall Rules to allow traffic to/from the Net Server, and between the two rotuers

Notice in the AAA objective, we are using RADIUS, rather than TACACS+. This is because VyOS (like RouterOS) does not support TACACS+.

Internal router

The internal router runs a subset of the functions that the edge router does: -

  • Internal BGP (iBGP) to the Edge router
    • Receiving any routes received from the Net Server
    • Receiving a default route (for internet access)
  • OSPF
    • Advertising loopbacks and internal networks between both routers
  • IPv4 and IPv6 routing
    • Using OSPFv3 (for IPv6 support)
    • Using the IPv6 Address Family for BGP
  • SNMPv3 for monitoring
  • Logging via Syslog to the Net Server
  • Authentication, Authorization and Accounting (AAA) via TACACS+ RADIUS to the Net Server

Prerequisites

To manage a VyOS device with Ansible, the following steps are required. We also make some changes to the default Ansible connection configuration.

Ansible Configuration

The following defaults are required to use Ansible with VyOS: -

nsible_connection: network_cli
ansible_network_os: vyos
ansible_user: vyos

As with JunOS, there is no enable or privileged mode within VyOS. Instead, the user privileges determine whether a user can run commands or make changes.

We do not need to supply a password, as authentication is via SSH public keys.

VyOS Configuration

To allow Ansible access to the VyOS routers, either use an existing SSH key pair, or create a new SSH key pair. Take the contents of the public key, and add them to the routers like so: -

# View SSH key
$ less ~/.ssh/id_ed25519.pub 
ssh-ed25519 ###REDACTED### [email protected]

# Go into configuration mode on the router
[email protected]:~$ configure

[edit]
[email protected]# set system login user vyos authentication public-keys [email protected] key '###REDACTED###'

[edit]
[email protected]# set system login user vyos authentication public-keys [email protected] type 'ssh-ed25519'

# Set the management IP
[edit]
[email protected]# set interfaces ethernet eth1 address '10.15.30.63/24'

[edit]
[email protected]# set interfaces ethernet eth1 description 'Management'

You should now be able to SSH into the router without a password.

Our inventory file looks like the below: -

[vyos]
vyos-01 ansible_host=10.15.30.63
vyos-02 ansible_host=10.15.30.64

Verification

Can we contact both devices?

$ ansible vyos -m vyos_facts --ask-vault-pass | grep -i hostname
Vault password:
        "ansible_net_hostname": "vyos-01",
        "ansible_net_hostname": "vyos-02",

Setup

The setup is identical to the IOS and Juniper lab, with a management interface to access the devices, a VLAN bridge for inter-device communication, and an interface on the edge router attached to the KVM NAT bridge for DHCP/Internet access. Unlike the Arista vEOS images, VLANs work correctly in the lab.

VLANs, IP addressing and Autonomous System numbers

The ID chosen for VyOS is 05.

VLANs

The VLANs used will be: -

  • VLAN105 between the edge router and netsvr-01
  • VLAN205 between the edge router and internal router

IP Addressing

  • IPv4 Subnet on VLAN105: 10.100.105.0/24
    • edge router - 10.100.105.253/24
    • netsvr-01 - 10.100.105.254/24
  • IPv4 Subnet on VLAN205: 10.100.205.0/24
    • edge router - 10.100.205.254/24
    • internal router - 10.100.205.253/24
  • IPv6 Subnet on VLAN105: 2001:db8:105::/64
    • edge router - 2001:db8:105::f/64
    • netsvr-01 - 2001:db8:105:ffff/64
  • IPv6 Subnet on VLAN205: 2001:db8:205::/64
    • edge router - 2001:db8:205::a/64
    • internal router - 2001:db8:205:f/64
  • IPv4 Loopback Addressing
    • edge router - 192.0.2.105/32
    • internal router - 192.0.2.205/32
  • IPv6 Loopback Address
    • edge router - 2001:db8:905:beef::1/128
    • internal router - 2001:db8:905:beef::2/128

BGP Autonomous System

The BGP Autonomous System number will be AS65105.

Configuration

Unlike RouterOS, there are quite a few Ansible modules for VyOS. It does not have as many as something like Cisco NX-OS, but has more than RouterOS or Extreme EXOS.

System Tasks

As noted in previous parts, the system tasks setup basic logging, banners and the hostname.

Playbook

The contents of the playbook are below: -

---
# tasks file for system
- name: Set hostname
  vyos_system:
    host_name: "{{ inventory_hostname }}"

- name: Remove unneeded banners
  vyos_banner:
    banner: "{{ item }}"
    state: absent
  loop:
  - post-login

- name: Update login banner
  vyos_banner:
    banner: pre-login
    text: |
      ----------------------------------------
      |
      | This banner was generated by Ansible
      |
      ----------------------------------------
      |
      | You are logged into {{ inventory_hostname }}
      |
      ----------------------------------------
      |
    state: present

- name: Configure syslog
  vyos_config:
    lines:
      - "set system syslog host {{ log_host }} facility all level info"
Setting Hostname

Ansible module: vyos_system

This task sets the hostname of the device. The generated configuration is below: -

set system host-name vyos-01
Removing unneeded banners

Ansible module: vyos_banner

This removes the post-login banner, as we only use a pre-login banner.

Update the login banner

Ansible module: vyos_banner

This task generates a banner for when you login to a device, which references the hostname. As in the Cisco IOS version, you could use a template file to generate this, especially if you have to provide specific information for legal/compliance reasons.

The generated configuration looks like the below: -

set system login banner pre-login '----------------------------------------\n|\n| This banner was generated by Ansible \n|\n----------------------------------------\n|\n| You are logged into vyos-01\n| \n----------------------------------------\n|'

When you login to the device, this looks like: -

$ ssh [email protected]
----------------------------------------
|
| This banner was generated by Ansible
|
----------------------------------------
|
| You are logged into vyos-01
|
----------------------------------------
|
Linux vyos-01 4.19.136-amd64-vyos #1 SMP Sat Aug 1 08:40:04 UTC 2020 x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.

Last login: Mon Aug  3 19:02:18 2020 from 10.15.30.1
[email protected]:~$

You also get the standard Debian MOTD banner.

Configure syslog

Ansible module: vyos_config

Unlike JunOS, there is no vyos_logging module, so we use vyos_config. Like in IOS, JunOS and EOS, vyos_config can apply lines of configuration (using the set syntax) or it can apply commands from a template.

As we are only apply one line of configuration, we use the lines option to apply it. The generated configuration is below: -

set system syslog host 10.100.105.254 facility all level 'info'

Interfaces

This role configures all the interfaces being used on the VyOS device. VyOS calls VLAN subinterfaces vifs, rather than units (like in JunOS) or subinterfaces (like in IOS/EOS).

Playbook

The contents of the Playbook are: -

---
# tasks file for interfaces
- name: Configure interfaces - Status and Descriptions
  vyos_interfaces:
    config:
      - name: "{{ item.vyos_if }}"
        description: "{{ item.desc }}"
        enabled: "{{ item.enabled }}"
  when: item.vif is not defined
  loop: "{{ interfaces }}"

- name: Configure interfaces - Status and Descriptions - vifs
  vyos_interfaces:
    config:
      - name: "{{ item.vyos_if }}"
        vifs:
          - description: "{{ item.desc }}"
            enabled: "{{ item.enabled }}"
            vlan_id: "{{ item.vif }}"
  when: item.vif is defined
  loop: "{{ interfaces }}"

- name: Configure interfaces - L3 IPv4
  vyos_l3_interfaces:
    config:
      - name: "{{ item.vyos_if }}"
        ipv4:
        - address: "{{ item.ipv4_addr }}"
  when:
    - item.ipv4_addr is defined
    - item.vif is not defined
  loop: "{{ interfaces }}"

- name: Configure interfaces - L3 IPv4 - vifs
  vyos_l3_interfaces:
    config:
      - name: "{{ item.vyos_if }}"
        vifs:
          - ipv4:
            - address: "{{ item.ipv4_addr }}"
            vlan_id: "{{ item.vif }}"
  when:
    - item.ipv4_addr is defined
    - item.vif is defined
  loop: "{{ interfaces }}"

- name: Configure interfaces - L3 IPv6
  vyos_l3_interfaces:
    config:
      - name: "{{ item.vyos_if }}"
        ipv6:
        - address: "{{ item.ipv6_addr }}"
  when:
    - item.ipv6_addr is defined
    - item.vif is not defined
  loop: "{{ interfaces }}"

- name: Configure interfaces - L3 IPv6 - vifs
  vyos_l3_interfaces:
    config:
      - name: "{{ item.vyos_if }}"
        vifs:
          - ipv6:
             - address: "{{ item.ipv6_addr }}"
            vlan_id: "{{ item.vif }}"
  when:
    - item.ipv6_addr is defined
    - item.vif is defined
  loop: "{{ interfaces }}"

Unlike in JunOS, we can configure all VLAN interfaces and non-VLAN interfaces using Ansible modules. Both vyos_interfaces and vyos_l3_interfaces support vif configuration.

Configure Interfaces - Status And Descriptions

Ansible module: vyos_interfaces

This task configures the descriptions and status (i.e. enabled or disabled) of each non-VLAN interface. For any interface without the vif value defined, the description and status will be updated. For example, the below is a summarized version of the host_vars for vyos-01: -

interfaces:
  - vyos_if: "eth1"
    desc: "Management"
    enabled: "true"
  - vyos_if: "eth2"
    desc: "VLAN Bridge"
    enabled: "true"
  - vyos_if: "eth2"
    vif: 105
    desc: "To netsvr"
    enabled: "true"
  - vyos_if: "eth2"
    vif: 205
    desc: "To vyos-02"
    enabled: "true"
  - vyos_if: "eth0"
    desc: "To the Internet"
    enabled: "true"
  - vyos_if: "lo"
    desc: "Loopback"
    enabled: "true"

This task would apply to: -

  • eth0
  • eth1
  • eth2
  • lo (the loopback interface)

The following configuration is generated: -

set interfaces ethernet eth0 description 'To the Internet'
set interfaces ethernet eth1 description 'Management'
set interfaces ethernet eth2 description 'VLAN Bridge'
set interfaces loopback lo description 'Loopback'
Configure Interfaces - Status And Descriptions (VLANs)

Ansible module: vyos_interfaces

This task is the same as the previous task, except it applies the changes for vif interfaces (i.e. VLAN interfaces). Based upon the previous host_vars, we would generate configuration for: -

  • eth2.105
  • eth2.205

The following configuration is generated: -

set interfaces ethernet eth2 vif 105 description 'To netsvr'
set interfaces ethernet eth2 vif 205 description 'To vyos-02'
Configure interfaces - L3 IPv4

Ansible module: vyos_l3_interfaces

This task configures the IPv4 addressing of each non-VLAN interface. This goes through the host_vars, and for every interface that has an IPv4 address (and no vif value), an IPv4 address is configured. The relevant host_vars are: -

  - vyos_if: "eth1"
    desc: "Management"
    enabled: "true"
    ipv4_addr: "10.15.30.63/24"
  - vyos_if: "eth2"
    desc: "VLAN Bridge"
    enabled: "true"
  - vyos_if: "eth2"
    vif: 105
    desc: "To netsvr"
    enabled: "true"
    ipv4_addr: "10.100.105.253/24"
  - vyos_if: "eth2"
    vif: 205
    desc: "To vyos-02"
    enabled: "true"
    ipv4_addr: "10.100.205.254/24"
  - vyos_if: "eth0"
    desc: "To the Internet"
    ipv4_addr: "dhcp"
    enabled: "true"
  - vyos_if: "lo"
    desc: "Loopback"
    enabled: "true"
    ipv4_addr: "192.0.2.105/32"

This task would then configure: -

  • eth0 (using DHCP)
  • eth1
  • lo

This generates the following configuration: -

set interfaces ethernet eth0 address 'dhcp'
set interfaces ethernet eth1 address '10.15.30.63/24'
set interfaces loopback lo address '192.0.2.105/32'
Configure interfaces - L3 IPv4 (VLANs)

Ansible module: vyos_l3_interfaces

This task is the same as the previous one, except it applies to VLAN interfaces (vif). Based upon the previous host_vars, the following interfaces would be configured: -

  • eth2.105
  • eth2.205

This generates the following configuration: -

set interfaces ethernet eth2 vif 105 address '10.100.105.253/24'
set interfaces ethernet eth2 vif 205 address '10.100.205.254/24'
Configure interfaces - L3 IPv6

Ansible module: vyos_l3_interfaces

This task is identical to the one for IPv4, except it applies IPv6 addresses instead. The following are the relevant host_vars

  - vyos_if: "eth1"
    desc: "Management"
    enabled: "true"
    ipv4_addr: "10.15.30.63/24"
  - vyos_if: "eth2"
    desc: "VLAN Bridge"
    enabled: "true"
  - vyos_if: "eth2"
    vif: 105
    desc: "To netsvr"
    enabled: "true"
    ipv4_addr: "10.100.105.253/24"
    ipv6_addr: "2001:db8:105::f/64"
  - vyos_if: "eth2"
    vif: 205
    desc: "To vyos-02"
    enabled: "true"
    ipv4_addr: "10.100.205.254/24"
    ipv6_addr: "2001:db8:205::a/64"
  - vyos_if: "eth0"
    desc: "To the Internet"
    ipv4_addr: "dhcp"
    enabled: "true"
  - vyos_if: "lo"
    desc: "Loopback"
    enabled: "true"
    ipv4_addr: "192.0.2.105/32"
    ipv6_addr: "2001:db8:905:beef::1/128"

We would then configure the following interfaces: -

  • lo

This would generate the following configuration :-

set interfaces loopback lo address '2001:db8:905:beef::1/128'
Configure interfaces - L3 IPv6 (VLANs)

Ansible module: vyos_l3_interfaces

As with the previous task, this is identical to the IPv4 task, except applying IPv6 addresses. Based upon the previous host_vars, we would configure: -

  • eth2.105
  • eth2.205

This would generate the following configuration :-

set interfaces ethernet eth2 vif 105 address '2001:db8:105::f/64'
set interfaces ethernet eth2 vif 205 address '2001:db8:205::a/64'

Verification

vyos-01

! Show IPs (IPv4 and IPv6), interface status and descriptions
[email protected]:~$ show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0             192.168.122.182/24                u/u  To the Internet
eth1             10.15.30.63/24                    u/u  Management
eth2             -                                 u/u  VLAN Bridge
eth2.105         10.100.105.253/24                 u/u  To netsvr
                 2001:db8:105::f/64
eth2.205         10.100.205.254/24                 u/u  To vyos-02
                 2001:db8:205::a/64
eth3             192.168.30.10/24                  u/u
lo               127.0.0.1/8                       u/u  Loopback
                 192.0.2.105/32
                 2001:db8:905:beef::1/128
                 ::1/128

! Ping to netsvr-01 on IPv4 and IPv6
[email protected]:~$ ping 10.100.105.254
PING 10.100.105.254 (10.100.105.254) 56(84) bytes of data.
64 bytes from 10.100.105.254: icmp_seq=1 ttl=64 time=0.580 ms
64 bytes from 10.100.105.254: icmp_seq=2 ttl=64 time=0.881 ms
^C
--- 10.100.105.254 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 7ms
rtt min/avg/max/mdev = 0.580/0.730/0.881/0.152 ms
[email protected]:~$ ping 2001:db8:105::ffff
PING 2001:db8:105::ffff(2001:db8:105::ffff) 56 data bytes
64 bytes from 2001:db8:105::ffff: icmp_seq=1 ttl=64 time=0.518 ms
64 bytes from 2001:db8:105::ffff: icmp_seq=2 ttl=64 time=1.07 ms
64 bytes from 2001:db8:105::ffff: icmp_seq=3 ttl=64 time=1.10 ms
^C
--- 2001:db8:105::ffff ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 28ms
rtt min/avg/max/mdev = 0.518/0.898/1.103/0.270 ms

! Ping to vyos-02 on IPv4 and IPv6
[email protected]:~$ ping 10.100.205.253
PING 10.100.205.253 (10.100.205.253) 56(84) bytes of data.
64 bytes from 10.100.205.253: icmp_seq=1 ttl=64 time=0.557 ms
64 bytes from 10.100.205.253: icmp_seq=2 ttl=64 time=0.955 ms
^C
--- 10.100.205.253 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 20ms
rtt min/avg/max/mdev = 0.557/0.756/0.955/0.199 ms
[email protected]:~$ ping 2001:db8:205::f
PING 2001:db8:205::f(2001:db8:205::f) 56 data bytes
64 bytes from 2001:db8:205::f: icmp_seq=1 ttl=64 time=0.845 ms
64 bytes from 2001:db8:205::f: icmp_seq=2 ttl=64 time=1.05 ms
64 bytes from 2001:db8:205::f: icmp_seq=3 ttl=64 time=0.988 ms
^C
--- 2001:db8:205::f ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 5ms
rtt min/avg/max/mdev = 0.845/0.962/1.053/0.086 ms

vyos-02

! Show IPs (IPv4 and IPv6), interface status and descriptions
[email protected]:~$ show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0             -                                 u/u
eth1             10.15.30.64/24                    u/u  Management
                 10.15.30.34/24
eth2             -                                 u/u  VLAN Bridge
eth2.205         10.100.205.253/24                 u/u  To vyos-01
                 2001:db8:205::f/64
lo               127.0.0.1/8                       u/u  Loopback
                 192.0.2.205/32
                 2001:db8:905:beef::2/128
                 ::1/128

! Ping to vyos-01 on IPv4 and IPv6
[email protected]:~$ ping 10.100.205.254
PING 10.100.205.254 (10.100.205.254) 56(84) bytes of data.
64 bytes from 10.100.205.254: icmp_seq=1 ttl=64 time=0.545 ms
64 bytes from 10.100.205.254: icmp_seq=2 ttl=64 time=1.05 ms
64 bytes from 10.100.205.254: icmp_seq=3 ttl=64 time=0.880 ms
^C
--- 10.100.205.254 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 5ms
rtt min/avg/max/mdev = 0.545/0.826/1.054/0.212 ms
[email protected]:~$ ping 2001:db8:205::a
PING 2001:db8:205::a(2001:db8:205::a) 56 data bytes
64 bytes from 2001:db8:205::a: icmp_seq=1 ttl=64 time=0.570 ms
64 bytes from 2001:db8:205::a: icmp_seq=2 ttl=64 time=0.850 ms
^C
--- 2001:db8:205::a ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 6ms
rtt min/avg/max/mdev = 0.570/0.710/0.850/0.140 ms

Looking good so far!

Firewall

Like JunOS, VyOS can be used as a zone-based firewall. It can also work in stateless or stateful modes, meaning you either need to match traffic in both directions (i.e. any reply traffic needs to be matched by a separate rule, stateless), or you can define rules only in one direction (stateful).

Playbook

The contents of the playbook are below: -

---
# tasks file for firewall
#
- name: Allow all local ICMP
  vyos_config:
    lines:
      - set firewall all-ping enable
  tags:
    - firewall

- name: Allow stateful traffic
  vyos_config:
    lines:
      - set firewall state-policy established action accept
      - set firewall state-policy related action accept
  tags:
    - firewall

- name: Define Addresses
  vyos_config:
    src: addressbook.j2
  when:
    - fw_addresses is defined
  tags:
    - firewall

- name: Define zones
  vyos_config:
    src: zones.j2
  when:
    - zones is defined
  tags:
    - firewall

- name: Define Zone Policies - IPv4
  vyos_config:
    src: policy_v4.j2
  when:
    - fw_policies is defined
    - fw_policies.ipv4 is defined
  tags:
    - firewall

- name: Define Zone Policies - IPv6
  vyos_config:
    src: policy_v6.j2
  when:
    - fw_policies is defined
    - fw_policies.ipv6 is defined
  tags:
    - firewall

There are no Ansible modules for VyOS firewalling, so we use the vyos_config module for all of the configuration.

Allowing ICMP

Ansible module: vyos_config

This task allows all local ping traffic. Without this, any pings to addresses configured on the router will be dropped.

Allow stateful traffic

Ansible module: vyos_config

This task allows VyOS to work in stateful mode across all firewall policies, rather than needing to apply it on a per-zone/policy basis. You could also set all stateful traffic to be rejected by default, and then allow it on a per zone/policy basis.

Define addresses

Ansible module: vyos_config

This task defines address groups. Address groups are a way of grouping together multiple hosts so that we can apply the same firewall rules and policies to them. Unlike JunOS, address groups can only be defined globally.

The template used is below: -

{% for address in fw_addresses %}
{% if address['groups'] is defined %}
{% for group in address['groups'] %}
{% if address['ip'] is defined %}
set firewall group address-group {{ group }} address {{ address['ip'] }}
{% endif %}
{% if address['ipv6'] is defined %}
set firewall group ipv6-address-group {{ group }} address {{ address['ipv6'] }}
{% endif %}
{% endfor %}
{% endif %}
{% endfor %}

IPv4 and IPv6 address groups use different syntax, so we build them separately. The relevant host_vars are: -

fw_addresses:
  - name: netsvr
    ip: 10.100.105.254
    groups:
      - external-bgp-peers-v4
      - netsvr-direct-v4
  - name: netsvr-v6
    ipv6: "2001:db8:105::ffff"
    groups:
      - external-bgp-peers-v6
      - netsvr-direct-v6
  - name: netsvr-lo
    ip: 192.0.2.1
    groups:
    - netsvr-loop-v4
  - zone: edge
    name: netsvr-lo-v6
    ipv6: "2001:db8:999:beef::1"
    groups:
    - netsvr-loop-v6
  - name: internal-rtr
    ip: 192.0.2.205
    groups:
    - internal-bgp-peers-v4
    - internal-rtr-loop-v4
  - name: internal-rtr-v6
    ipv6: "2001:db8:905:beef::2"
    groups:
    - internal-bgp-peers-v6
    - internal-rtr-loop-v6

This would then generate the following configuration: -

set firewall group address-group external-bgp-peers-v4 address '10.100.105.254'
set firewall group address-group internal-bgp-peers-v4 address '192.0.2.205'
set firewall group address-group internal-rtr-loop-v4 address '192.0.2.205'
set firewall group address-group netsvr-direct-v4 address '10.100.105.254'
set firewall group address-group netsvr-loop-v4 address '192.0.2.1'
set firewall group ipv6-address-group external-bgp-peers-v6 address '2001:db8:105::ffff'
set firewall group ipv6-address-group internal-bgp-peers-v6 address '2001:db8:905:beef::2'
set firewall group ipv6-address-group internal-rtr-loop-v6 address '2001:db8:905:beef::2'
set firewall group ipv6-address-group netsvr-direct-v6 address '2001:db8:105::ffff'
set firewall group ipv6-address-group netsvr-loop-v6 address '2001:db8:999:beef::1'
Define zones

Ansible module: vyos_config

In this section, we define the zones that our interfaces are placed in. We also define our management configuration. This is because if we create the zones and apply them without any rules, traffic to that zone is dropped by default. It would cause Ansible to lose connectivity during configuration, and also drop any SSH access to be able to rectify the issue.

The template used is below: -

{% for zone in zones %}
set zone-policy zone {{ zone['name'] }} description "{{ zone['description'] }}"
{% if zone['local'] is defined %}
set zone-policy zone {{ zone['name'] }} local-zone
{% endif %}
{% endfor %}
{% for interface in interfaces %}
{% if interface['zone'] is defined %}
{% if interface['vif'] is defined %}
set zone-policy zone {{ interface['zone'] }} interface {{ interface['vyos_if'] }}.{{ interface['vif'] }}
{% else %}
set zone-policy zone {{ interface['zone'] }} interface {{ interface['vyos_if'] }}
{% endif %}
{% endif %}
{% endfor %}
{% if fw_policies['mgmt'] is defined %}
{% for policy in fw_policies['mgmt']['ipv4'] %}
{% for rule in policy['rules'] %}
{% if policy['source_group'] is defined %}
set firewall name {{ policy['name'] }} rule {{ loop.index }} source group address-group {{ rule['source_group'] }}
{% endif %}
{% if rule['dest_group'] is defined %}
set firewall name {{ policy['name'] }} rule {{ loop.index }} destination group address-group {{ rule['dest_group'] }}
{% endif %}
{% if rule['protocol'] is defined %}
set firewall name {{ policy['name'] }} rule {{ loop.index }} protocol {{ rule['protocol'] }}
{% endif %}
{% if rule['port'] is defined %}
set firewall name {{ policy['name'] }} rule {{ loop.index }} destination port {{ rule['port'] }}
{% endif %}
{% if rule['state'] is defined %}
set firewall name {{ policy['name'] }} rule {{ loop.index }} state {{ rule['state'] }}
{% endif %}
{% if rule['action'] is defined %}
set firewall name {{ policy['name'] }} rule {{ loop.index }} action {{ rule['action'] }}
{% endif %}
{% endfor %}
set zone-policy zone {{ policy['zones']['to'] }} from {{ policy['zones']['from'] }} firewall name {{ policy['name'] }}
{% endfor %}
{% endif %}

This template can be seen as two parts. Everything before {% if fw_policies['mgmt'] is defined %} creates zones based upon the following host_vars: -

zones:
  - name: external
    description: "External facing interfaces"
  - name: internal
    description: "Internal facing interfaces"
  - name: local
    description: "Local router zone"
    local: true
  - name: mgmt
    description: "Management zone"
interfaces:
  - vyos_if: "eth1"
    zone: "mgmt"
  - vyos_if: "eth2"
    zone: "external"
  - vyos_if: "eth2"
    zone: "internal"
  - vyos_if: "eth0"
  - vyos_if: "lo"

This will then generate the following configuration: -

set zone-policy zone external default-action 'drop'
set zone-policy zone external description 'External facing interfaces'
set zone-policy zone external interface 'eth2.105'
set zone-policy zone internal default-action 'drop'
set zone-policy zone internal description 'Internal facing interfaces'
set zone-policy zone internal interface 'eth2.205'
set zone-policy zone local default-action 'drop'
set zone-policy zone local description 'Local router zone'
set zone-policy zone local local-zone
set zone-policy zone mgmt default-action 'drop'
set zone-policy zone mgmt description 'Management zone'
set zone-policy zone mgmt interface 'eth1'

The default-action for each zone can either be reject or drop (i.e. we cannot accept by default). When configuring a local-zone, this matches all traffic destined to/from this router. This means that the moment this zone is applied, all traffic (including SSH traffic) is dropped. This presents a problem when configuring with Ansible, as it uses SSH to configure the router.

To avoid this, we also apply our management policy at the same time. This uses the following host_vars: -

fw_policies:
  mgmt:
    ipv4:
      - name: mgmt_to_local_ipv4
        zones:
          from: mgmt
          to: local
        rules:
          - protocol: tcp
            port: 22
            action: accept
          - protocol: udp
            port: 161
            action: accept

This will then generate the following: -

set firewall name mgmt_to_local_ipv4 default-action 'drop'
set firewall name mgmt_to_local_ipv4 rule 1 action 'accept'
set firewall name mgmt_to_local_ipv4 rule 1 destination port '22'
set firewall name mgmt_to_local_ipv4 rule 1 protocol 'tcp'
set firewall name mgmt_to_local_ipv4 rule 2 action 'accept'
set firewall name mgmt_to_local_ipv4 rule 2 destination port '161'
set firewall name mgmt_to_local_ipv4 rule 2 protocol 'udp'

This is enough to allow Ansible to continue configuring the routers. It also allows us to use the management address for SNMP later in this post.

Define Zone Policies - IPv4

Ansible module: vyos_config

This task configures the IPv4 firewall policies in VyOS, and applies them to zones.

The template used is below: -

{% for policy in fw_policies['ipv4'] %}
{% for rule in policy['rules'] %}
{% if rule['source_groups'] is defined %}
set firewall name {{ policy['name'] }} rule {{ loop.index }} source group address-group {{ rule['source_groups'] }}
{% endif %}
{% if rule['dest_groups'] is defined %}
set firewall name {{ policy['name'] }} rule {{ loop.index }} destination group address-group {{ rule['dest_groups'] }}
{% endif %}
{% if rule['protocol'] is defined %}
set firewall name {{ policy['name'] }} rule {{ loop.index }} protocol {{ rule['protocol'] }}
{% endif %}
{% if rule['port'] is defined %}
set firewall name {{ policy['name'] }} rule {{ loop.index }} destination port {{ rule['port'] }}
{% endif %}
{% if rule['state'] is defined %}
set firewall name {{ policy['name'] }} rule {{ loop.index }} state {{ rule['state'] }} {{ rule['action'] }}
{% endif %}
{% if rule['action'] is defined %}
set firewall name {{ policy['name'] }} rule {{ loop.index }} action {{ rule['action'] }}
{% endif %}
{% endfor %}
set zone-policy zone {{ policy['zones']['to'] }} from {{ policy['zones']['from'] }} firewall name {{ policy['name'] }}
{% endfor %}

This template does the following: -

  • It goes through the firewall policies for IPv4 and looks to see if there are
    • Source Groups for the policy
    • Destination Groups for the policy
    • Checks if a protocol is defined
    • Checks if a port is defined
    • Checks if we expect the policy to apply based upon a certain TCP state
    • Defines an action
  • It also applies this zone between the zones referenced
    • The to zone is the destination of traffic
    • The from zone is the source of traffic

We also use {{ loop.index }}. This starts at 1 and increments for every time the loop is iterated.

The following host_vars are relevant to this: -

fw_policies:
  ipv4:
    - name: external_to_local_ipv4
      zones:
        from: external
        to: local
      rules:
        - source_groups: external-bgp-peers-v4
          protocol: tcp
          port: 179
          action: accept
        - protocol: icmp
          action: accept
        - protocol: all
          action: reject
    - name: internal_to_local_ipv4
      zones:
        from: internal
        to: local
      rules:
        - source_groups: internal-bgp-peers-v4
          protocol: tcp
          port: 179
          action: accept
        - protocol: ospf
          action: accept
        - protocol: icmp
          action: accept
        - protocol: all
          action: reject
    - name: external_to_internal_ipv4
      zones:
        from: external
        to: internal
      rules:
        - protocol: icmp
          action: accept
        - protocol: all
          action: reject
    - name: local_to_external_ipv4
      zones:
        from: local
        to: external
      rules:
        - dest_groups: external-bgp-peers-v4
          protocol: tcp
          port: 179
          action: accept
        - dest_groups: netsvr-direct-v4
          protocol: udp
          port: 514
          action: accept
        - dest_groups: netsvr-loop-v4
          protocol: tcp_udp
          port: 1812-1813
          action: accept
        - protocol: icmp
          action: accept
        - protocol: all
          action: reject
    - name: local_to_internal_ipv4
      zones:
        from: local
        to: internal
      rules:
        - dest_groups: internal-bgp-peers-v4
          protocol: tcp
          port: 179
          action: accept
        - protocol: ospf
          action: accept
        - protocol: icmp
          action: accept
        - protocol: all
          action: reject
    - name: internal_to_external_ipv4
      zones:
        from: internal
        to: external
      rules:
        - dest_groups: netsvr-direct-v4
          protocol: udp
          port: 514
          action: accept
        - dest_groups: netsvr-loop-v4
          protocol: tcp_udp
          port: 1812-1813
          action: accept
        - protocol: icmp
          action: accept
        - protocol: all
          action: reject

This generates the following configuration: -

set firewall name external_to_internal_ipv4 rule 1 action 'accept'
set firewall name external_to_internal_ipv4 rule 1 protocol 'icmp'
set firewall name external_to_internal_ipv4 rule 2 action 'reject'
set firewall name external_to_internal_ipv4 rule 2 protocol 'all'
set firewall name external_to_local_ipv4 rule 1 action 'accept'
set firewall name external_to_local_ipv4 rule 1 destination port '179'
set firewall name external_to_local_ipv4 rule 1 protocol 'tcp'
set firewall name external_to_local_ipv4 rule 1 source group address-group 'external-bgp-peers-v4'
set firewall name external_to_local_ipv4 rule 2 action 'accept'
set firewall name external_to_local_ipv4 rule 2 protocol 'icmp'
set firewall name external_to_local_ipv4 rule 3 action 'reject'
set firewall name external_to_local_ipv4 rule 3 protocol 'all'
set firewall name internal_to_external_ipv4 rule 1 action 'accept'
set firewall name internal_to_external_ipv4 rule 1 destination group address-group 'netsvr-direct-v4'
set firewall name internal_to_external_ipv4 rule 1 destination port '514'
set firewall name internal_to_external_ipv4 rule 1 protocol 'udp'
set firewall name internal_to_external_ipv4 rule 2 action 'accept'
set firewall name internal_to_external_ipv4 rule 2 destination group address-group 'netsvr-direct-v4'
set firewall name internal_to_external_ipv4 rule 2 destination port '49'
set firewall name internal_to_external_ipv4 rule 2 protocol 'tcp_udp'
set firewall name internal_to_external_ipv4 rule 3 action 'accept'
set firewall name internal_to_external_ipv4 rule 3 protocol 'icmp'
set firewall name internal_to_external_ipv4 rule 4 action 'reject'
set firewall name internal_to_external_ipv4 rule 4 protocol 'all'
set firewall name internal_to_local_ipv4 rule 1 action 'accept'
set firewall name internal_to_local_ipv4 rule 1 destination port '179'
set firewall name internal_to_local_ipv4 rule 1 protocol 'tcp'
set firewall name internal_to_local_ipv4 rule 1 source group address-group 'internal-bgp-peers-v4'
set firewall name internal_to_local_ipv4 rule 2 action 'accept'
set firewall name internal_to_local_ipv4 rule 2 protocol 'ospf'
set firewall name internal_to_local_ipv4 rule 3 action 'accept'
set firewall name internal_to_local_ipv4 rule 3 protocol 'icmp'
set firewall name internal_to_local_ipv4 rule 4 action 'reject'
set firewall name internal_to_local_ipv4 rule 4 protocol 'all'
set firewall name local_to_external_ipv4 rule 1 action 'accept'
set firewall name local_to_external_ipv4 rule 1 destination group address-group 'external-bgp-peers-v4'
set firewall name local_to_external_ipv4 rule 1 destination port '179'
set firewall name local_to_external_ipv4 rule 1 protocol 'tcp'
set firewall name local_to_external_ipv4 rule 2 action 'accept'
set firewall name local_to_external_ipv4 rule 2 destination group address-group 'netsvr-direct-v4'
set firewall name local_to_external_ipv4 rule 2 destination port '514'
set firewall name local_to_external_ipv4 rule 2 protocol 'udp'
set firewall name local_to_external_ipv4 rule 3 action 'accept'
set firewall name local_to_external_ipv4 rule 3 destination group address-group 'netsvr-direct-v4'
set firewall name local_to_external_ipv4 rule 3 destination port '49'
set firewall name local_to_external_ipv4 rule 3 protocol 'tcp_udp'
set firewall name local_to_external_ipv4 rule 4 action 'accept'
set firewall name local_to_external_ipv4 rule 4 protocol 'icmp'
set firewall name local_to_external_ipv4 rule 5 action 'reject'
set firewall name local_to_external_ipv4 rule 5 protocol 'all'
set firewall name local_to_internal_ipv4 rule 1 action 'accept'
set firewall name local_to_internal_ipv4 rule 1 destination group address-group 'internal-bgp-peers-v4'
set firewall name local_to_internal_ipv4 rule 1 destination port '179'
set firewall name local_to_internal_ipv4 rule 1 protocol 'tcp'
set firewall name local_to_internal_ipv4 rule 2 action 'accept'
set firewall name local_to_internal_ipv4 rule 2 protocol 'ospf'
set firewall name local_to_internal_ipv4 rule 3 action 'accept'
set firewall name local_to_internal_ipv4 rule 3 protocol 'icmp'
set firewall name local_to_internal_ipv4 rule 4 action 'reject'
set firewall name local_to_internal_ipv4 rule 4 protocol 'all'
set zone-policy zone external from internal firewall name 'internal_to_external_ipv4'
set zone-policy zone external from local firewall name 'local_to_external_ipv4'
set zone-policy zone internal from external firewall name 'external_to_internal_ipv4'
set zone-policy zone internal from local firewall name 'local_to_internal_ipv4'
set zone-policy zone local from external firewall name 'external_to_local_ipv4'
set zone-policy zone local from internal firewall name 'internal_to_local_ipv4'
Define Zone Policies - IPv6

Ansible module: vyos_config

This task is identical to the previous, except that it applies to IPv6.

The template used is below: -

{% for policy in fw_policies['ipv6'] %}
{% for rule in policy['rules'] %}
{% if rule['source_groups'] is defined %}
set firewall ipv6-name {{ policy['name'] }} rule {{ loop.index }} source group address-group {{ rule['source_groups'] }}
{% endif %}
{% if rule['dest_groups'] is defined %}
set firewall ipv6-name {{ policy['name'] }} rule {{ loop.index }} destination group address-group {{ rule['dest_groups'] }}
{% endif %}
{% if rule['protocol'] is defined %}
set firewall ipv6-name {{ policy['name'] }} rule {{ loop.index }} protocol {{ rule['protocol'] }}
{% endif %}
{% if rule['port'] is defined %}
set firewall ipv6-name {{ policy['name'] }} rule {{ loop.index }} destination port {{ rule['port'] }}
{% endif %}
{% if rule['state'] is defined %}
set firewall ipv6-name {{ policy['name'] }} rule {{ loop.index }} state {{ rule['state'] }} {{ rule['action'] }}
{% endif %}
{% if rule['action'] is defined %}
set firewall ipv6-name {{ policy['name'] }} rule {{ loop.index }} action {{ rule['action'] }}
{% endif %}
{% endfor %}
set zone-policy zone {{ policy['zones']['to'] }} from {{ policy['zones']['from'] }} firewall ipv6-name {{ policy['name'] }}
{% endfor %}

This is the same as the previous template, except it uses ipv6-name in place of name.

The following host_vars are relevant to this: -

fw_policies:
  ipv6:
    - name: external_to_local_ipv6
      zones:
        from: external
        to: local
      rules:
        - source_groups: external-bgp-peers-v6
          protocol: tcp
          port: 179
          action: accept
        - protocol: icmpv6
          action: accept
        - protocol: all
          action: reject
    - name: internal_to_local_ipv6
      zones:
        from: internal
        to: local
      rules:
        - source_groups: internal-bgp-peers-v6
          protocol: tcp
          port: 179
          action: accept
        - protocol: ospf
          action: accept
        - protocol: icmpv6
          action: accept
        - protocol: all
          action: reject
    - name: external_to_internal_ipv6
      zones:
        from: external
        to: internal
      rules:
        - protocol: icmpv6
          action: accept
        - protocol: all
          action: reject
    - name: local_to_external_ipv6
      zones:
        from: local
        to: external
      rules:
        - dest_groups: external-bgp-peers-v6
          protocol: tcp
          port: 179
          action: accept
        - protocol: ospf
          action: accept
        - protocol: icmpv6
          action: accept
        - protocol: all
          action: reject
    - name: local_to_internal_ipv6
      zones:
        from: local
        to: internal
      rules:
        - dest_groups: internal-bgp-peers-v6
          protocol: tcp
          port: 179
          action: accept
        - protocol: ospf
          action: accept
        - protocol: icmpv6
          action: accept
        - protocol: all
          action: reject
    - name: internal_to_external_ipv6
      zones:
        from: internal
        to: external
      rules:
        - protocol: icmpv6
          action: accept
        - protocol: all
          action: reject

This generates the following configuration: -

set firewall ipv6-name external_to_internal_ipv6 default-action 'drop'
set firewall ipv6-name external_to_internal_ipv6 rule 1 action 'accept'
set firewall ipv6-name external_to_internal_ipv6 rule 1 protocol 'icmpv6'
set firewall ipv6-name external_to_internal_ipv6 rule 2 action 'reject'
set firewall ipv6-name external_to_internal_ipv6 rule 2 protocol 'all'
set firewall ipv6-name external_to_local_ipv6 rule 1 action 'accept'
set firewall ipv6-name external_to_local_ipv6 rule 1 destination port '179'
set firewall ipv6-name external_to_local_ipv6 rule 1 protocol 'tcp'
set firewall ipv6-name external_to_local_ipv6 rule 1 source group address-group 'external-bgp-peers-v6'
set firewall ipv6-name external_to_local_ipv6 rule 2 action 'accept'
set firewall ipv6-name external_to_local_ipv6 rule 2 protocol 'icmpv6'
set firewall ipv6-name external_to_local_ipv6 rule 3 action 'reject'
set firewall ipv6-name external_to_local_ipv6 rule 3 protocol 'all'
set firewall ipv6-name internal_to_external_ipv6 rule 1 action 'accept'
set firewall ipv6-name internal_to_external_ipv6 rule 1 protocol 'icmpv6'
set firewall ipv6-name internal_to_external_ipv6 rule 2 action 'reject'
set firewall ipv6-name internal_to_external_ipv6 rule 2 protocol 'all'
set firewall ipv6-name internal_to_local_ipv6 rule 1 action 'accept'
set firewall ipv6-name internal_to_local_ipv6 rule 1 destination port '179'
set firewall ipv6-name internal_to_local_ipv6 rule 1 protocol 'tcp'
set firewall ipv6-name internal_to_local_ipv6 rule 1 source group address-group 'internal-bgp-peers-v6'
set firewall ipv6-name internal_to_local_ipv6 rule 2 action 'accept'
set firewall ipv6-name internal_to_local_ipv6 rule 2 protocol 'ospf'
set firewall ipv6-name internal_to_local_ipv6 rule 3 action 'accept'
set firewall ipv6-name internal_to_local_ipv6 rule 3 protocol 'icmpv6'
set firewall ipv6-name internal_to_local_ipv6 rule 4 action 'reject'
set firewall ipv6-name internal_to_local_ipv6 rule 4 protocol 'all'
set firewall ipv6-name local_to_external_ipv6 rule 1 action 'accept'
set firewall ipv6-name local_to_external_ipv6 rule 1 destination group address-group 'external-bgp-peers-v6'
set firewall ipv6-name local_to_external_ipv6 rule 1 destination port '179'
set firewall ipv6-name local_to_external_ipv6 rule 1 protocol 'tcp'
set firewall ipv6-name local_to_external_ipv6 rule 2 action 'accept'
set firewall ipv6-name local_to_external_ipv6 rule 2 protocol 'ospf'
set firewall ipv6-name local_to_external_ipv6 rule 3 action 'accept'
set firewall ipv6-name local_to_external_ipv6 rule 3 protocol 'icmpv6'
set firewall ipv6-name local_to_external_ipv6 rule 4 action 'reject'
set firewall ipv6-name local_to_external_ipv6 rule 4 protocol 'all'
set firewall ipv6-name local_to_internal_ipv6 rule 1 action 'accept'
set firewall ipv6-name local_to_internal_ipv6 rule 1 destination group address-group 'internal-bgp-peers-v6'
set firewall ipv6-name local_to_internal_ipv6 rule 1 destination port '179'
set firewall ipv6-name local_to_internal_ipv6 rule 1 protocol 'tcp'
set firewall ipv6-name local_to_internal_ipv6 rule 2 action 'accept'
set firewall ipv6-name local_to_internal_ipv6 rule 2 protocol 'ospf'
set firewall ipv6-name local_to_internal_ipv6 rule 3 action 'accept'
set firewall ipv6-name local_to_internal_ipv6 rule 3 protocol 'icmpv6'
set firewall ipv6-name local_to_internal_ipv6 rule 4 action 'reject'
set firewall ipv6-name local_to_internal_ipv6 rule 4 protocol 'all'
set zone-policy zone external from internal firewall ipv6-name 'internal_to_external_ipv6'
set zone-policy zone external from local firewall ipv6-name 'local_to_external_ipv6'
set zone-policy zone internal from external firewall ipv6-name 'external_to_internal_ipv6'
set zone-policy zone internal from local firewall ipv6-name 'local_to_internal_ipv6'
set zone-policy zone local from external firewall ipv6-name 'external_to_local_ipv6'
set zone-policy zone local from internal firewall ipv6-name 'internal_to_local_ipv6'

Verification

Now we can verify whether the firewall rules work: -

[email protected]:~$ show firewall statistics

------------------------
Firewall Global Settings
------------------------

Firewall state-policy for all IPv4 and Ipv6 traffic

state           action   log
-----           ------   ---
established     accept   disabled
related         accept   disabled

-----------------------------
Rulesets Information
-----------------------------
--------------------------------------------------------------------------------
IPv4 Firewall "external_to_internal_ipv4":

 Active on traffic to -
  zone [internal] from zone [external]

rule  packets   bytes     action  source              destination
----  -------   -----     ------  ------              -----------
1     0         0         ACCEPT  0.0.0.0/0           0.0.0.0/0
2     0         0         REJECT  0.0.0.0/0           0.0.0.0/0
10000 0         0         DROP    0.0.0.0/0           0.0.0.0/0

--------------------------------------------------------------------------------
IPv4 Firewall "external_to_local_ipv4":

 Active on traffic to -
  zone [local] from zone [external]

rule  packets   bytes     action  source              destination
----  -------   -----     ------  ------              -----------
1     0         0         ACCEPT  0.0.0.0/0           0.0.0.0/0
2     0         0         ACCEPT  0.0.0.0/0           0.0.0.0/0
3     6         1.90K     REJECT  0.0.0.0/0           0.0.0.0/0
10000 0         0         DROP    0.0.0.0/0           0.0.0.0/0

--------------------------------------------------------------------------------
IPv4 Firewall "internal_to_external_ipv4":

 Active on traffic to -
  zone [external] from zone [internal]

rule  packets   bytes     action  source              destination
----  -------   -----     ------  ------              -----------
1     0         0         ACCEPT  0.0.0.0/0           0.0.0.0/0
2     0         0         ACCEPT  0.0.0.0/0           0.0.0.0/0
3     0         0         ACCEPT  0.0.0.0/0           0.0.0.0/0
4     0         0         REJECT  0.0.0.0/0           0.0.0.0/0
10000 0         0         DROP    0.0.0.0/0           0.0.0.0/0

--------------------------------------------------------------------------------
IPv4 Firewall "internal_to_local_ipv4":

 Active on traffic to -
  zone [local] from zone [internal]

rule  packets   bytes     action  source              destination
----  -------   -----     ------  ------              -----------
1     1         60        ACCEPT  0.0.0.0/0           0.0.0.0/0
2     16        1.12K     ACCEPT  0.0.0.0/0           0.0.0.0/0
3     8         672       ACCEPT  0.0.0.0/0           0.0.0.0/0
4     0         0         REJECT  0.0.0.0/0           0.0.0.0/0
10000 0         0         DROP    0.0.0.0/0           0.0.0.0/0

--------------------------------------------------------------------------------
IPv4 Firewall "local_to_external_ipv4":

 Active on traffic to -
  zone [external] from zone [local]

rule  packets   bytes     action  source              destination
----  -------   -----     ------  ------              -----------
1     1         60        ACCEPT  0.0.0.0/0           0.0.0.0/0
2     166       20.52K    ACCEPT  0.0.0.0/0           0.0.0.0/0
3     0         0         ACCEPT  0.0.0.0/0           0.0.0.0/0
4     0         0         ACCEPT  0.0.0.0/0           0.0.0.0/0
5     23        2.09K     REJECT  0.0.0.0/0           0.0.0.0/0
10000 0         0         DROP    0.0.0.0/0           0.0.0.0/0

--------------------------------------------------------------------------------
IPv4 Firewall "local_to_internal_ipv4":

 Active on traffic to -
  zone [internal] from zone [local]

rule  packets   bytes     action  source              destination
----  -------   -----     ------  ------              -----------
1     1         60        ACCEPT  0.0.0.0/0           0.0.0.0/0
2     14        1.04K     ACCEPT  0.0.0.0/0           0.0.0.0/0
3     9         756       ACCEPT  0.0.0.0/0           0.0.0.0/0
4     4         160       REJECT  0.0.0.0/0           0.0.0.0/0
10000 0         0         DROP    0.0.0.0/0           0.0.0.0/0

--------------------------------------------------------------------------------
IPv4 Firewall "mgmt_to_local_ipv4":

 Active on traffic to -
  zone [local] from zone [mgmt]

rule  packets   bytes     action  source              destination
----  -------   -----     ------  ------              -----------
1     1         64        ACCEPT  0.0.0.0/0           0.0.0.0/0
10000 0         0         DROP    0.0.0.0/0           0.0.0.0/0

--------------------------------------------------------------------------------
IPv6 Firewall "external_to_internal_ipv6":

 Active on traffic to -
  zone [internal] from zone [external]

rule  packets   bytes     action  source              destination
----  -------   -----     ------  ------              -----------
1     0         0         ACCEPT  ::/0                ::/0
2     0         0         REJECT  ::/0                ::/0
10000 0         0         DROP    ::/0                ::/0

--------------------------------------------------------------------------------
IPv6 Firewall "external_to_local_ipv6":

 Active on traffic to -
  zone [local] from zone [external]

rule  packets   bytes     action  source              destination
----  -------   -----     ------  ------              -----------
1     0         0         ACCEPT  ::/0                ::/0
2     8         488       ACCEPT  ::/0                ::/0
3     0         0         REJECT  ::/0                ::/0
10000 0         0         DROP    ::/0                ::/0

--------------------------------------------------------------------------------
IPv6 Firewall "internal_to_external_ipv6":

 Active on traffic to -
  zone [external] from zone [internal]

rule  packets   bytes     action  source              destination
----  -------   -----     ------  ------              -----------
1     0         0         ACCEPT  ::/0                ::/0
2     0         0         REJECT  ::/0                ::/0
10000 0         0         DROP    ::/0                ::/0

--------------------------------------------------------------------------------
IPv6 Firewall "internal_to_local_ipv6":

 Active on traffic to -
  zone [local] from zone [internal]

rule  packets   bytes     action  source              destination
----  -------   -----     ------  ------              -----------
1     1         80        ACCEPT  ::/0                ::/0
2     16        1.57K     ACCEPT  ::/0                ::/0
3     2         136       ACCEPT  ::/0                ::/0
4     0         0         REJECT  ::/0                ::/0
10000 0         0         DROP    ::/0                ::/0

--------------------------------------------------------------------------------
IPv6 Firewall "local_to_external_ipv6":

 Active on traffic to -
  zone [external] from zone [local]

rule  packets   bytes     action  source              destination
----  -------   -----     ------  ------              -----------
1     1         80        ACCEPT  ::/0                ::/0
2     0         0         ACCEPT  ::/0                ::/0
3     17        1.45K     ACCEPT  ::/0                ::/0
4     0         0         REJECT  ::/0                ::/0
10000 0         0         DROP    ::/0                ::/0

--------------------------------------------------------------------------------
IPv6 Firewall "local_to_internal_ipv6":

 Active on traffic to -
  zone [internal] from zone [local]

rule  packets   bytes     action  source              destination
----  -------   -----     ------  ------              -----------
1     1         80        ACCEPT  ::/0                ::/0
2     14        1.43K     ACCEPT  ::/0                ::/0
3     14        1.25K     ACCEPT  ::/0                ::/0
4     0         0         REJECT  ::/0                ::/0
10000 0         0         DROP    ::/0                ::/0

! What happens if we try to SSH from the netsvr?

[[email protected] ~] $ ssh [email protected]
ssh: connect to host 10.100.105.253 port 22: Connection refused

! Can we still ping it?
[[email protected] ~] $ ping 10.100.105.253
PING 10.100.105.253 (10.100.105.253) 56(84) bytes of data.
64 bytes from 10.100.105.253: icmp_seq=1 ttl=64 time=0.837 ms
64 bytes from 10.100.105.253: icmp_seq=2 ttl=64 time=0.854 ms
^C
--- 10.100.105.253 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 36ms
rtt min/avg/max/mdev = 0.837/0.845/0.854/0.030 ms

Routing

For routing, we use BGP for external network connectivity, OSPF for internal IPv4 routing and OSPFv3 for internal IPv6 routing. Unlike other vendors that have IPv4 support for OSPFv3, VyOS currently only support IPv6 routes and addressing in OSPFv3. This is why I have chosen to configure both in each article in this series, so that those using mixed-vendor networks can integrate them together without needing to reconfigure their core routing protocols.

Main Playbook

As with the others in this series, the main playbook is used to include other playbooks: -

---
## tasks file for routing
##
- name: Include OSPF routing
  include: ospf.yml

- name: Include OSPFv3 routing
  include: ospfv3.yml

- name: Include BGP routing
  include: bgp.yml

OSPF Playbook

The contents of the OSPF Playbook are below: -

---
# tasks file for routing
#
- name: OSPF Process - Router ID
  vyos_config:
    lines:
      - "set protocols ospf parameters router-id  {{ router_id }}"
  tags:
    - ospf
    - ospf_v4

- name: OSPF Interfaces
  vyos_config:
    lines:
      - set protocols ospf area {{ item.ospf.area }} network {{ item.ipv4_addr | ipaddr('network/prefix') }}
  when:
    - item.ospf is defined
    - item.ipv4_addr is not search("dhcp")
  loop: "{{ interfaces }}"
  tags:
    - ospf
    - ospf_v4

- name: OSPF Interfaces - Passive
  vyos_config:
    lines:
      - set protocols ospf passive-interface {{ item.vyos_if }}
  when:
    - item.ospf is defined
    - item.ospf.passive is defined
    - item.vif is not defined
  loop: "{{ interfaces }}"
  tags:
    - ospf
    - ospf_v4

- name: OSPF Interfaces - Passive - vif
  vyos_config:
    lines:
      - set protocols ospf passive-interface {{ item.vyos_if }}.{{ item.vif }}
  when:
    - item.ospf is defined
    - item.ospf.passive is defined
    - item.vif is defined
  loop: "{{ interfaces }}"
  tags:
    - ospf
    - ospf_v4

This has some similarities to the JunOS OSPF playbook, except that we must enable passive interfaces for standard interfaces and Virtual Interfaces separately.

Router ID

Ansible module: vyos_config

The Router ID task picks up the router_id variable from host_vars, and applies it as part of the OSPF configuration.

The following host_vars are relevant: -

router_id: 192.0.2.105

This generates the following configuration: -

set protocols ospf parameters router-id 192.0.2.105
OSPF Interfaces

Ansible module: vyos_config`

This task goes through our list of interfaces, discovers the OSPF area they belong to, and uses the ipaddr feature within Jinja2/Ansible to discover the network address for the subnet the interface is in. For example, an interface with the IP 192.168.1.222/24, the network address would be 192.168.1.0/24.

The relevant host_vars for this are: -

interfaces:
  - vyos_if: "eth2"
    vif: 105
    desc: "To netsvr"
    enabled: "true"
    ipv4_addr: "10.100.105.253/24"
    zone: "external"
    ospf:
      area: "0.0.0.0"
  - vyos_if: "eth2"
    vif: 205
    desc: "To vyos-02"
    enabled: "true"
    zone: "internal"
    ipv4_addr: "10.100.205.254/24"
    ospf:
      area: "0.0.0.0"
  - vyos_if: "eth0"
    desc: "To the Internet"
    ipv4_addr: "dhcp"
    enabled: "true"
    ospf:
      area: "0.0.0.0"
  - vyos_if: "lo"
    desc: "Loopback"
    enabled: "true"
    ipv4_addr: "192.0.2.105/32"
    ospf:
      area: "0.0.0.0"

This generates the following configuration: -

set protocols ospf area 0.0.0.0 network '10.100.105.0/24'
set protocols ospf area 0.0.0.0 network '10.100.205.0/24'
set protocols ospf area 0.0.0.0 network '192.0.2.105/32'
OSPF Interfaces - Passive

Ansible module: vyos_config`

This is similar to the above, except for any interface with the passive field under OSPF, we also generate the passive configuration.

The host_vars relevant to this are: -

interfaces:
  - vyos_if: "eth2"
    vif: 105
    desc: "To netsvr"
    enabled: "true"
    ipv4_addr: "10.100.105.253/24"
    ipv6_addr: "2001:db8:105::f/64"
    zone: "external"
    ospf:
      area: "0.0.0.0"
      passive: true
  - vyos_if: "eth2"
    vif: 205
    desc: "To vyos-02"
    enabled: "true"
    zone: "internal"
    ipv4_addr: "10.100.205.254/24"
    ospf:
      area: "0.0.0.0"
  - vyos_if: "eth0"
    desc: "To the Internet"
    ipv4_addr: "dhcp"
    enabled: "true"
    ospf:
      area: "0.0.0.0"
      passive: true
  - vyos_if: "lo"
    desc: "Loopback"
    enabled: "true"
    ipv4_addr: "192.0.2.105/32"
    ospf:
      area: "0.0.0.0"
      passive: true

This would then generate the following configuration: -

set protocols ospf passive-interface 'eth0'
set protocols ospf passive-interface 'lo'

This task is only for non-VIF interfaces, meaning that it only applies to eth0 and lo.

OSPF Interfaces - Passive (VIFs)

Ansible module: vyos_config`

This is almost identical to the previous task, except that it applies to VIFs.

The previous host_vars apply, and generates the following configuration: -

set protocols ospf passive-interface 'eth2.105'
Verification

After this, we should be able to see OSPF routes on both the edge router and the internal router: -

vyos-01

! Show OSPF interfaces
[email protected]:~$ show ip ospf interface
eth2.105 is up
  ifindex 6, MTU 1500 bytes, BW 1000 Mbit <UP,BROADCAST,RUNNING,MULTICAST>
  Internet Address 10.100.105.253/24, Broadcast 10.100.105.255, Area 0.0.0.0
  MTU mismatch detection: enabled
  Router ID 192.0.2.105, Network Type BROADCAST, Cost: 100
  Transmit Delay is 1 sec, State DR, Priority 1
  No backup designated router on this network
  Multicast group memberships: <None>
  Timer intervals configured, Hello 10s, Dead 40s, Wait 40s, Retransmit 5
    No Hellos (Passive interface)
  Neighbor Count is 0, Adjacent neighbor count is 0
eth2.205 is up
  ifindex 7, MTU 1500 bytes, BW 1000 Mbit <UP,BROADCAST,RUNNING,MULTICAST>
  Internet Address 10.100.205.254/24, Broadcast 10.100.205.255, Area 0.0.0.0
  MTU mismatch detection: enabled
  Router ID 192.0.2.105, Network Type BROADCAST, Cost: 100
  Transmit Delay is 1 sec, State Backup, Priority 1
  Backup Designated Router (ID) 192.0.2.105, Interface Address 10.100.205.254
  Multicast group memberships: OSPFAllRouters OSPFDesignatedRouters
  Timer intervals configured, Hello 10s, Dead 40s, Wait 40s, Retransmit 5
    Hello due in 3.154s
  Neighbor Count is 1, Adjacent neighbor count is 1
lo is up
  ifindex 1, MTU 65536 bytes, BW 0 Mbit <UP,LOOPBACK,RUNNING>
  Internet Address 192.0.2.105/32, Broadcast 192.0.2.105, Area 0.0.0.0
  MTU mismatch detection: enabled
  Router ID 192.0.2.105, Network Type LOOPBACK, Cost: 10
  Transmit Delay is 1 sec, State Loopback, Priority 1
  No backup designated router on this network
  Multicast group memberships: <None>
  Timer intervals configured, Hello 10s, Dead 40s, Wait 40s, Retransmit 5
    No Hellos (Passive interface)
  Neighbor Count is 0, Adjacent neighbor count is 0

! Show OSPF neighbours
[email protected]:~$ show ip ospf neighbor

Neighbor ID     Pri State           Dead Time Address         Interface                        RXmtL RqstL DBsmL
192.0.2.205       1 Full/DR           38.604s 10.100.205.253  eth2.205:10.100.205.254              0     0     0

! Show routing table
[email protected]:~$ show ip route ospf
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

O   10.100.105.0/24 [110/100] is directly connected, eth2.105, 00:07:39
O   10.100.205.0/24 [110/100] is directly connected, eth2.205, 00:07:38
O   192.0.2.105/32 [110/0] is directly connected, lo, 00:07:44
O>* 192.0.2.205/32 [110/100] via 10.100.205.253, eth2.205, 00:06:52

! Can we ping?
[email protected]:~$ ping 192.0.2.205
PING 192.0.2.205 (192.0.2.205) 56(84) bytes of data.
64 bytes from 192.0.2.205: icmp_seq=1 ttl=64 time=0.359 ms
^C
--- 192.0.2.205 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.359/0.359/0.359/0.000 ms

vyos-02

! Show OSPF interfaces
[email protected]:~$ show ip ospf interface
eth2.205 is up
  ifindex 5, MTU 1500 bytes, BW 4294967295 Mbit <UP,BROADCAST,RUNNING,MULTICAST>
  Internet Address 10.100.205.253/24, Broadcast 10.100.205.255, Area 0.0.0.0
  MTU mismatch detection: enabled
  Router ID 192.0.2.205, Network Type BROADCAST, Cost: 1
  Transmit Delay is 1 sec, State DR, Priority 1
  Backup Designated Router (ID) 192.0.2.105, Interface Address 10.100.205.254
  Multicast group memberships: OSPFAllRouters OSPFDesignatedRouters
  Timer intervals configured, Hello 10s, Dead 40s, Wait 40s, Retransmit 5
    Hello due in 2.643s
  Neighbor Count is 1, Adjacent neighbor count is 1
lo is up
  ifindex 1, MTU 65536 bytes, BW 0 Mbit <UP,LOOPBACK,RUNNING>
  Internet Address 192.0.2.205/32, Broadcast 192.0.2.205, Area 0.0.0.0
  MTU mismatch detection: enabled
  Router ID 192.0.2.205, Network Type LOOPBACK, Cost: 10
  Transmit Delay is 1 sec, State Loopback, Priority 1
  No backup designated router on this network
  Multicast group memberships: <None>
  Timer intervals configured, Hello 10s, Dead 40s, Wait 40s, Retransmit 5
    No Hellos (Passive interface)
  Neighbor Count is 0, Adjacent neighbor count is 0

! Show OSPF neighbours
[email protected]:~$ show ip ospf neighbor

Neighbor ID     Pri State           Dead Time Address         Interface                        RXmtL RqstL DBsmL
192.0.2.105       1 Full/Backup       38.921s 10.100.205.254  eth2.205:10.100.205.253              0     0     0

! Show routing table
[email protected]:~$ show ip route ospf
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

O>* 10.100.105.0/24 [110/101] via 10.100.205.254, eth2.205, 00:09:21
O   10.100.205.0/24 [110/1] is directly connected, eth2.205, 00:10:18
O>* 192.0.2.105/32 [110/1] via 10.100.205.254, eth2.205, 00:09:21
O   192.0.2.205/32 [110/0] is directly connected, lo, 00:10:26

! Can we ping?
[email protected]:~$ ping 192.0.2.105
PING 192.0.2.105 (192.0.2.105) 56(84) bytes of data.
64 bytes from 192.0.2.105: icmp_seq=1 ttl=64 time=0.454 ms
64 bytes from 192.0.2.105: icmp_seq=2 ttl=64 time=0.798 ms
^C
--- 192.0.2.105 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 18ms
rtt min/avg/max/mdev = 0.454/0.626/0.798/0.172 ms

[email protected]:~$ ping 10.100.105.253
PING 10.100.105.253 (10.100.105.253) 56(84) bytes of data.
64 bytes from 10.100.105.253: icmp_seq=1 ttl=64 time=0.452 ms
64 bytes from 10.100.105.253: icmp_seq=2 ttl=64 time=0.831 ms
64 bytes from 10.100.105.253: icmp_seq=3 ttl=64 time=0.861 ms
^C
--- 10.100.105.253 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 14ms
rtt min/avg/max/mdev = 0.452/0.714/0.861/0.188 ms

All looks good!

OSPFv3 Playbook

As noted, we are using OSPFv3 for IPv6 internal routing.

---
# tasks file for routing
#

- name: OSPFv3 Interfaces
  vyos_config:
    lines:
      - set protocols ospfv3 area {{ item.ospf.area }} interface {{ item.vyos_if }}
  when:
    - item.ospfv3 is defined
    - item.vif is not defined
  loop: "{{ interfaces }}"
  tags:
    - ospf
    - ospf_v6

- name: OSPFv3 Interfaces - vif
  vyos_config:
    lines:
      - set protocols ospfv3 area {{ item.ospf.area }} interface {{ item.vyos_if }}.{{ item.vif }}
  when:
     - item.ospfv3 is defined
     - item.vif is defined
  loop: "{{ interfaces }}"
  tags:
    - ospf
    - ospf_v6

- name: OSPFv3 Interfaces - Passive
  vyos_config:
    src: ospfv3_passive.j2
  tags:
    - ospf
    - ospf_v6

We allow OSPFv3 to dynamically discover the router ID in this case, although you may prefer to set it statically.

The first two tasks are fundamentally the same as for OSPF, except we use the ospfv3 keyword rather than ospf and that we apply OSPFv3 to an interface rather than using a network to match interfaces.

However we use a different task for setting passive interfaces. This is because you need to define whether an interface is passive using the set interface $MEDIA $INTERFACE ipv6 ospf passive syntax, which differs for the three kinds of interfaces to apply this to (Ethernet, VIFs and Loopbacks). Rather than creating three separate tasks for this, we use a template instead.

This template looks like the below: -

{% for interface in interfaces %}
  {% if interface.ospfv3 is defined %}
    {% if interface.ospfv3.passive is defined %}
      {% if "eth" in interface.vyos_if %}
        {% if interface.vif is defined %}
interfaces ethernet {{ interface.vyos_if }} vif {{ interface.vif }} ipv6 ospfv3 passive
        {% else %}
interfaces ethernet {{ interface.vyos_if }} ipv6 ospfv3 passive
        {% endif %}
      {% elif "lo" in interface.vyos_if %}
interfaces loopback {{ interface.vyos_if }} ipv6 ospfv3 passive
      {% endif %}
   {% endif %}
  {% endif %}
{% endfor %}

As you can see, we are doing the following: -

  • Looping through our interfaces…
  • If the interface has an OSPFv3 section then…
  • If the interface has a VIF then apply the correct passive command to it else…
  • If the interface does not have a VIF, apply the passive command directly to the interface, else…
  • If the interface is a loopback, use the correct command for the loopback

The use of the media type of the interface, as well as VIFs requiring slightly different syntax makes applying passive OSPFv3 commands slightly more complex.

The host_vars that are relevant to this playbook are: -

interfaces:
  - vyos_if: "eth1"
    desc: "Management"
    enabled: "true"
    ipv4_addr: "10.15.30.63/24"
    zone: "mgmt"
  - vyos_if: "eth2"
    desc: "VLAN Bridge"
    enabled: "true"
  - vyos_if: "eth2"
    vif: 105
    desc: "To netsvr"
    enabled: "true"
    ipv4_addr: "10.100.105.253/24"
    ipv6_addr: "2001:db8:105::f/64"
    zone: "external"
    ospf:
      area: "0.0.0.0"
      passive: true
    ospfv3:
      area: "0.0.0.0"
      passive: true
  - vyos_if: "eth2"
    vif: 205
    desc: "To vyos-02"
    enabled: "true"
    zone: "internal"
    ipv4_addr: "10.100.205.254/24"
    ipv6_addr: "2001:db8:205::a/64"
    ospf:
      area: "0.0.0.0"
    ospfv3:
      area: "0.0.0.0"
  - vyos_if: "eth0"
    desc: "To the Internet"
    ipv4_addr: "dhcp"
    enabled: "true"
    ospf:
      area: "0.0.0.0"
      passive: true
    nat:
      role: "outside"
  - vyos_if: "lo"
    desc: "Loopback"
    enabled: "true"
    ipv4_addr: "192.0.2.105/32"
    ipv6_addr: "2001:db8:905:beef::1/128"
    ospf:
      area: "0.0.0.0"
      passive: true
    ospfv3:
      area: "0.0.0.0"
      passive: true

This then generates the following configuration: -

set interfaces ethernet eth2 vif 105 ipv6 ospfv3 passive
set interfaces loopback lo ipv6 ospfv3 passive
set protocols ospfv3 area 0.0.0.0 interface 'lo'
set protocols ospfv3 area 0.0.0.0 interface 'eth2.105'
set protocols ospfv3 area 0.0.0.0 interface 'eth2.205'
Verification

We’ll follow the same steps as we did for OSPF: -

vyos-01

! Show OSPFv3 interfaces
[email protected]:~$ show ipv6 ospfv3 interface
eth0 is up, type BROADCAST
  Interface ID: 2
   OSPF not enabled on this interface
eth1 is up, type BROADCAST
  Interface ID: 3
   OSPF not enabled on this interface
eth2 is up, type BROADCAST
  Interface ID: 5
   OSPF not enabled on this interface
eth2.105 is up, type BROADCAST
  Interface ID: 6
  Internet Address:
    inet : 10.100.105.253/24
    inet6: 2001:db8:105::f/64
    inet6: fe80::5054:ff:fe23:2eac/64
  Instance ID 0, Interface MTU 1500 (autodetect: 1500)
  MTU mismatch detection: enabled
  Area ID 0.0.0.0, Cost 1
  State DR, Transmit Delay 1 sec, Priority 1
  Timer intervals configured:
   Hello 10, Dead 40, Retransmit 5
  DR: 192.0.2.105 BDR: 0.0.0.0
  Number of I/F scoped LSAs is 1
    0 Pending LSAs for LSUpdate in Time 00:00:00 [thread off]
    0 Pending LSAs for LSAck in Time 00:00:00 [thread off]
eth2.205 is up, type BROADCAST
  Interface ID: 7
  Internet Address:
    inet : 10.100.205.254/24
    inet6: fe80::5054:ff:fe23:2eac/64
    inet6: 2001:db8:205::a/64
  Instance ID 0, Interface MTU 1500 (autodetect: 1500)
  MTU mismatch detection: enabled
  Area ID 0.0.0.0, Cost 100
  State BDR, Transmit Delay 1 sec, Priority 1
  Timer intervals configured:
   Hello 10, Dead 40, Retransmit 5
  DR: 192.0.2.205 BDR: 192.0.2.105
  Number of I/F scoped LSAs is 2
    0 Pending LSAs for LSUpdate in Time 00:00:00 [thread off]
    0 Pending LSAs for LSAck in Time 00:00:00 [thread off]
eth3 is up, type BROADCAST
  Interface ID: 4
   OSPF not enabled on this interface
lo is up, type LOOPBACK
  Interface ID: 1
  Internet Address:
    inet : 192.0.2.105/32
    inet6: 2001:db8:905:beef::1/128
    inet6: fe80::200:ff:fe00:0/64
  Instance ID 0, Interface MTU 65536 (autodetect: 65536)
  MTU mismatch detection: enabled
  Area ID 0.0.0.0, Cost 1
  State DR, Transmit Delay 1 sec, Priority 1
  Timer intervals configured:
   Hello 10, Dead 40, Retransmit 5
  DR: 192.0.2.105 BDR: 0.0.0.0
  Number of I/F scoped LSAs is 1
    0 Pending LSAs for LSUpdate in Time 00:00:00 [thread off]
    0 Pending LSAs for LSAck in Time 00:00:00 [thread off]

! Show OSPFv3 neighbours
Neighbor ID     Pri    DeadTime    State/IfState         Duration I/F[State]
192.0.2.205       1    00:00:38     Full/DR              00:24:06 eth2.205[BDR]

! Show routing table
[email protected]:~$ show ipv6 ospfv3 neighbor
Neighbor ID     Pri    DeadTime    State/IfState         Duration I/F[State]
192.0.2.205       1    00:00:38     Full/DR              00:24:06 eth2.205[BDR]
[email protected]:~$ show ipv6 route ospfv3
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

O   2001:db8:105::/64 [110/1] is directly connected, eth2.105, 00:24:59
O   2001:db8:205::/64 [110/100] is directly connected, eth2.205, 00:24:23
O   2001:db8:905:beef::1/128 [110/1] is directly connected, lo, 00:24:59
O>* 2001:db8:905:beef::2/128 [110/101] via fe80::5054:ff:fe50:4198, eth2.205, 00:24:18

! Ping!
[email protected]:~$ ping 2001:db8:905:beef::2
PING 2001:db8:905:beef::2(2001:db8:905:beef::2) 56 data bytes
64 bytes from 2001:db8:905:beef::2: icmp_seq=1 ttl=64 time=0.836 ms
64 bytes from 2001:db8:905:beef::2: icmp_seq=2 ttl=64 time=0.615 ms
^C
--- 2001:db8:905:beef::2 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 29ms
rtt min/avg/max/mdev = 0.615/0.725/0.836/0.113 ms

vyos-02

! Show OSPFv3 interfaces
[email protected]:~$ show ipv6 ospfv3 interface
eth0 is up, type BROADCAST
  Interface ID: 2
   OSPF not enabled on this interface
eth1 is up, type BROADCAST
  Interface ID: 3
   OSPF not enabled on this interface
eth2 is up, type BROADCAST
  Interface ID: 4
   OSPF not enabled on this interface
eth2.205 is up, type BROADCAST
  Interface ID: 5
  Internet Address:
    inet : 10.100.205.253/24
    inet6: fe80::5054:ff:fe50:4198/64
    inet6: 2001:db8:205::f/64
  Instance ID 0, Interface MTU 1500 (autodetect: 1500)
  MTU mismatch detection: enabled
  Area ID 0.0.0.0, Cost 1
  State DR, Transmit Delay 1 sec, Priority 1
  Timer intervals configured:
   Hello 10, Dead 40, Retransmit 5
  DR: 192.0.2.205 BDR: 192.0.2.105
  Number of I/F scoped LSAs is 2
    0 Pending LSAs for LSUpdate in Time 00:00:00 [thread off]
    0 Pending LSAs for LSAck in Time 00:00:00 [thread off]
lo is up, type LOOPBACK
  Interface ID: 1
  Internet Address:
    inet : 192.0.2.205/32
    inet6: 2001:db8:905:beef::2/128
    inet6: fe80::200:ff:fe00:0/64
  Instance ID 0, Interface MTU 65536 (autodetect: 65536)
  MTU mismatch detection: enabled
  Area ID 0.0.0.0, Cost 1
  State DR, Transmit Delay 1 sec, Priority 1
  Timer intervals configured:
   Hello 10, Dead 40, Retransmit 5
  DR: 192.0.2.205 BDR: 0.0.0.0
  Number of I/F scoped LSAs is 1
    0 Pending LSAs for LSUpdate in Time 00:00:00 [thread off]
    0 Pending LSAs for LSAck in Time 00:00:00 [thread off]

! Show OSPFv3 neighbours
[email protected]:~$ show ipv6 ospfv3 neighbor
Neighbor ID     Pri    DeadTime    State/IfState         Duration I/F[State]
192.0.2.105       1    00:00:39     Full/BDR             00:26:09 eth2.205[DR]

! Show routing table
[email protected]:~$ show ipv6 ospfv3 neighbor
Neighbor ID     Pri    DeadTime    State/IfState         Duration I/F[State]
192.0.2.105       1    00:00:39     Full/BDR             00:26:09 eth2.205[DR]
[email protected]:~$ show ipv6 route ospfv3
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

O>* 2001:db8:105::/64 [110/2] via fe80::5054:ff:fe23:2eac, eth2.205, 00:26:18
O   2001:db8:205::/64 [110/1] is directly connected, eth2.205, 00:26:23
O>* 2001:db8:905:beef::1/128 [110/2] via fe80::5054:ff:fe23:2eac, eth2.205, 00:26:18
O   2001:db8:905:beef::2/128 [110/1] is directly connected, lo, 00:27:10

! Ping!
[email protected]:~$ ping 2001:db8:905:beef::1
PING 2001:db8:905:beef::1(2001:db8:905:beef::1) 56 data bytes
64 bytes from 2001:db8:905:beef::1: icmp_seq=1 ttl=64 time=0.803 ms
64 bytes from 2001:db8:905:beef::1: icmp_seq=2 ttl=64 time=0.757 ms
^C
--- 2001:db8:905:beef::1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 6ms
rtt min/avg/max/mdev = 0.757/0.780/0.803/0.023 ms

[email protected]:~$ ping 2001:db8:105::f
PING 2001:db8:105::f(2001:db8:105::f) 56 data bytes
64 bytes from 2001:db8:105::f: icmp_seq=1 ttl=64 time=0.517 ms
64 bytes from 2001:db8:105::f: icmp_seq=2 ttl=64 time=0.712 ms
64 bytes from 2001:db8:105::f: icmp_seq=3 ttl=64 time=0.563 ms
^C
--- 2001:db8:105::f ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 35ms
rtt min/avg/max/mdev = 0.517/0.597/0.712/0.085 ms

Looks good to me!

BGP Playbook

The BGP playbook is where we configure our internal and external BGP peers. There are no BGP Ansible modules for VyOS, so we use vyos_config instead.

We are also applying prefix lists and route maps to only allow certain routes to be advertised and received. In day to day BGP configuration, you would configure these all the time, so it makes sense to include tasks to configure them.

The playbook itself looks like the below: -

---
- name: Configure Prefix Lists - IPv4
  vyos_config:
    src: prefixlists_v4.j2
  when:
    - route_maps is defined
    - route_maps.prefix_lists is defined
    - route_maps.prefix_lists.ipv4 is defined
  tags:
    - bgp
    - bgp_v4

- name: Configure Prefix Lists - IPv6
  vyos_config:
    src: prefixlists_v6.j2
  when:
    - route_maps is defined
    - route_maps.prefix_lists is defined
    - route_maps.prefix_lists.ipv6 is defined
  tags:
    - bgp
    - bgp_v6

- name: Configure Route Maps
  vyos_config:
    src: routemap.j2
  when:
    - route_maps is defined
    - route_maps.rm is defined
  tags:
    - bgp
    - bgp_v4
    - bgp_v6

- name: Configure BGP Peers
  vyos_config:
    src: bgp.j2
  when:
    - bgp is defined
  tags:
    - bgp
    - bgp_v4
    - bgp_v6
Prefix Lists - IPv4

Ansible module: vyos_config

Prefix lists are used to match a range of IP addresses. They can be exact matches (i.e. 192.168.0.0/24), or you can match on a longer or shorter prefix length.

The configuration template looks like the below: -

{% for pfx_list in route_maps['prefix_lists']['ipv4'] %}
{% for address in pfx_list['addresses'] %}
set policy prefix-list {{ pfx_list['name'] }} rule {{ loop.index }} prefix {{ address }}
set policy prefix-list {{ pfx_list['name'] }} rule {{ loop.index }} action {{ pfx_list['action'] }}
{% endfor %}
{% endfor %}

As you can see, we use the {{ loop.index }} value again, which increments on every iteration. This means that each rule has a number to say what order it will be evaluated in.

The relevant host_vars for this are: -

route_maps:
  prefix_lists:
    ipv4:
      - name: internal-nets-v4
        addresses:
           - 192.0.2.105/32
           - 192.0.2.205/32
           - 10.100.205.0/24
        action: permit
      - name: external-nets-v4
        addresses:
           - 192.0.2.1/32
           - 10.100.105.0/24
        action: permit

This generates the following configuration: -

set policy prefix-list external-nets-v4 rule 1 action 'permit'
set policy prefix-list external-nets-v4 rule 1 prefix '192.0.2.1/32'
set policy prefix-list external-nets-v4 rule 2 action 'permit'
set policy prefix-list external-nets-v4 rule 2 prefix '10.100.105.0/24'
set policy prefix-list internal-nets-v4 rule 1 action 'permit'
set policy prefix-list internal-nets-v4 rule 1 prefix '192.0.2.105/32'
set policy prefix-list internal-nets-v4 rule 2 action 'permit'
set policy prefix-list internal-nets-v4 rule 2 prefix '192.0.2.205/32'
set policy prefix-list internal-nets-v4 rule 3 action 'permit'
set policy prefix-list internal-nets-v4 rule 3 prefix '10.100.205.0/24'
Prefix Lists - IPv6

Ansible module: vyos_config

This task performs the same function as the previous one, except for IPv6 prefixes.

The configuration template looks like the below: -

{% for pfx_list in route_maps['prefix_lists']['ipv6'] %}
{% for address in pfx_list['addresses'] %}
set policy prefix-list6 {{ pfx_list['name'] }} rule {{ loop.index }} prefix {{ address }}
set policy prefix-list6 {{ pfx_list['name'] }} rule {{ loop.index }} action {{ pfx_list['action'] }}
{% endfor %}
{% endfor %}

The only difference here is that we use prefix-list6 instead of prefix-list.

The relevant host_vars for this are: -

route_maps:
  prefix_lists:
    ipv6:
      - name: internal-nets-v6
        addresses:
           - "2001:db8:905:beef::1/128"
           - "2001:db8:905:beef::2/128"
           - "2001:db8:905::/64"
        action: permit
      - name: external-nets-v6
        addresses:
           - "2001:db8:999:beef::1/128"
        action: permit

This generates the following configuration: -

set policy prefix-list6 external-nets-v6 rule 1 action 'permit'
set policy prefix-list6 external-nets-v6 rule 1 prefix '2001:db8:999:beef::1/128'
set policy prefix-list6 internal-nets-v6 rule 1 action 'permit'
set policy prefix-list6 internal-nets-v6 rule 1 prefix '2001:db8:905:beef::1/128'
set policy prefix-list6 internal-nets-v6 rule 2 action 'permit'
set policy prefix-list6 internal-nets-v6 rule 2 prefix '2001:db8:905:beef::2/128'
set policy prefix-list6 internal-nets-v6 rule 3 action 'permit'
set policy prefix-list6 internal-nets-v6 rule 3 prefix '2001:db8:905::/64'
Route Maps

Ansible module: vyos_config

Route maps are used to apply our chosen actions to the routes advertised to, or received from, our BGP neighbours.

Our template looks like the below: -

{% for route_map in route_maps['rm'] %}
{% for rule in route_map['rules'] %}
{% if rule['pfx_list'] is defined %}
set policy route-map {{ route_map['name'] }} rule {{ loop.index }} match ip address prefix-list {{ rule['pfx_list'] }}
{% endif %}
{% if rule['pfx_list6'] is defined %}
set policy route-map {{ route_map['name'] }} rule {{ loop.index }} match ipv6 address prefix-list {{ rule['pfx_list6'] }}
{% endif %}
set policy route-map {{ route_map['name'] }} rule {{ loop.index }} action {{ rule['action'] }}
{% endfor %}
{% endfor %}

In this, we loop through the route_maps section of our host_vars. If the prefix list used to match is in the pfx_list section, we use match ip address prefix-list $PREFIX_LIST_NAME (i.e. for IPv4). If it is in the pfx_list6 section, we use match ipv6 address prefix-list $IPv6_PREFIX_LIST_NAME (i.e. for IPv6).

The relevant host_vars for this are: -

route_maps:
  rm:
    - name: external-networks-v4
      rules:
        - pfx_list: external-nets-v4
          action: permit
        - action: deny
    - name: internal-networks-v4
      rules:
        - pfx_list: internal-nets-v4
          action: permit
        - action: deny
    - name: external-networks-v6
      rules:
        - pfx_list6: external-nets-v6
          action: permit
        - action: deny
    - name: internal-networks-v6
      rules:
        - pfx_list6: internal-nets-v6
          action: permit
        - action: deny

The generated configuration looks like the below: -

set policy route-map external-networks-v4 rule 1 action 'permit'
set policy route-map external-networks-v4 rule 1 match ip address prefix-list 'external-nets-v4'
set policy route-map external-networks-v4 rule 2 action 'deny'
set policy route-map external-networks-v6 rule 1 action 'permit'
set policy route-map external-networks-v6 rule 1 match ipv6 address prefix-list 'external-nets-v6'
set policy route-map external-networks-v6 rule 2 action 'deny'
set policy route-map internal-networks-v4 rule 1 action 'permit'
set policy route-map internal-networks-v4 rule 1 match ip address prefix-list 'internal-nets-v4'
set policy route-map internal-networks-v4 rule 2 action 'deny'
set policy route-map internal-networks-v6 rule 1 action 'permit'
set policy route-map internal-networks-v6 rule 1 match ipv6 address prefix-list 'internal-nets-v6'
set policy route-map internal-networks-v6 rule 2 action 'deny'
Configuring BGP peers

Ansible module: vyos_config

This task creates the BGP peers. The template is quite complex, so I will explain each section in detail: -

set protocols bgp {{ bgp['local_as'] }} parameters router-id {{ router_id }}
set protocols bgp {{ bgp['local_as'] }} parameters default no-ipv4-unicast
{% if bgp['redistribute'] is defined %}
{% if bgp['redistribute']['ospf'] is defined %}
set protocols bgp {{ bgp['local_as'] }} address-family ipv4-unicast redistribute ospf
{% endif %}
{% endif %}
{% if bgp['redistribute'] is defined %}
{% if bgp['redistribute']['ospfv3'] is defined %}
set protocols bgp {{ bgp['local_as'] }} address-family ipv6-unicast redistribute ospfv3
{% endif %}
{% endif %}
{% if bgp['neighbours']['ipv4'] is defined %}
{% for neighbour in bgp['neighbours']['ipv4'] %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} remote-as {{ neighbour['remote_as'] }}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} address-family ipv4-unicast
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} description "{{ neighbour['desc'] }}"
{% if neighbour['loc_ip'] is defined %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} update-source {{ neighbour['loc_ip'] }}
{% endif %}
{% if neighbour['default_originate'] is defined %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} address-family ipv4-unicast default-originate
{% endif %}
{% if neighbour['route_map'] is defined %}
{% if neighbour['route_map']['in'] is defined %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} address-family ipv4-unicast route-map import {{ neighbour['route_map']['in'] }}
{% endif %}
{% if neighbour['route_map']['out'] is defined %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} address-family ipv4-unicast route-map export {{ neighbour['route_map']['out'] }}
{% endif %}
{% endif %}
{% endfor %}
{% endif %}
{% if bgp['neighbours']['ipv6'] is defined %}
{% for neighbour in bgp['neighbours']['ipv6'] %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} remote-as {{ neighbour['remote_as'] }}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} address-family ipv6-unicast
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} description "{{ neighbour['desc'] }}"
{% if neighbour['loc_ip'] is defined %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} update-source {{ neighbour['loc_ip'] }}
{% endif %}
{% if neighbour['route_map'] is defined %}
{% if neighbour['route_map']['in'] is defined %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} address-family ipv6-unicast route-map import {{ neighbour['route_map']['in'] }}
{% endif %}
{% if neighbour['route_map']['out'] is defined %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} address-family ipv6-unicast route-map export {{ neighbour['route_map']['out'] }}
{% endif %}
{% endif %}
{% endfor %}
{% endif %}

There are multiple different sections here, and we also combine redistribution and IPv4 and IPv6 in the same template.

set protocols bgp {{ bgp['local_as'] }} parameters router-id {{ router_id }}
set protocols bgp {{ bgp['local_as'] }} parameters default no-ipv4-unicast

These two lines set the router ID for BGP, and also disable IPv4 unicast by default. The reason for the latter is that IPv4 unicast would be enabled for every peer, including IPv6 peers. This means that we would advertise IPv4 routes over an IPv6 session, making our route-maps applied to the IPv4 peers superfluous.

{% if bgp['redistribute'] is defined %}
{% if bgp['redistribute']['ospf'] is defined %}
set protocols bgp {{ bgp['local_as'] }} address-family ipv4-unicast redistribute ospf
{% endif %}
{% endif %}

This section enables the redistribution of OSPF routes into BGP if it has been defined in our host_vars.

{% if bgp['redistribute'] is defined %}
{% if bgp['redistribute']['ospfv3'] is defined %}
set protocols bgp {{ bgp['local_as'] }} address-family ipv6-unicast redistribute ospfv3
{% endif %}
{% endif %}

This is the same as above, except for IPv6.

{% if bgp['neighbours']['ipv4'] is defined %}
{% for neighbour in bgp['neighbours']['ipv4'] %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} remote-as {{ neighbour['remote_as'] }}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} address-family ipv4-unicast
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} description "{{ neighbour['desc'] }}"
{% if neighbour['loc_ip'] is defined %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} update-source {{ neighbour['loc_ip'] }}
{% endif %}
{% if neighbour['default_originate'] is defined %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} address-family ipv4-unicast default-originate
{% endif %}
{% if neighbour['route_map'] is defined %}
{% if neighbour['route_map']['in'] is defined %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} address-family ipv4-unicast route-map import {{ neighbour['route_map']['in'] }}
{% endif %}
{% if neighbour['route_map']['out'] is defined %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} address-family ipv4-unicast route-map export {{ neighbour['route_map']['out'] }}
{% endif %}
{% endif %}

In this section, we loop through any IPv4 BGP peers we have defined in our host_vars and then: -

  • We configure the remote autonomous system number, enable IPv4 unicast on a per peer basis, and apply a description to the peer
  • If we have defined a local IP to source BGP from for this peer (potentially the loopback IP), we set it as an update source
  • If we want to unconditionally advertise a default route to the peer, we enable it with default-originate
  • If any route maps are defined, we apply them either in the inbound (import) or outbound (export) direction
{% if bgp['neighbours']['ipv6'] is defined %}
{% for neighbour in bgp['neighbours']['ipv6'] %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} remote-as {{ neighbour['remote_as'] }}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} address-family ipv6-unicast
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} description "{{ neighbour['desc'] }}"
{% if neighbour['loc_ip'] is defined %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} update-source {{ neighbour['loc_ip'] }}
{% endif %}
{% if neighbour['route_map'] is defined %}
{% if neighbour['route_map']['in'] is defined %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} address-family ipv6-unicast route-map import {{ neighbour['route_map']['in'] }}
{% endif %}
{% if neighbour['route_map']['out'] is defined %}
set protocols bgp {{ bgp['local_as'] }} neighbor {{ neighbour['peer'] }} address-family ipv6-unicast route-map export {{ neighbour['route_map']['out'] }}
{% endif %}
{% endif %}
{% endfor %}
{% endif %}

This section does fundmentally the same as the above except: -

  • It enables the IPv6 address family (for IPv6 peers)
  • We do not use the default-originate command, as we have no IPv6 internet to test it with

The relevant host_vars for this are: -

bgp:
  local_as: 65105
  redistribute:
    ospf: true
    ospfv3: true
  neighbours:
    ipv4:
      - peer: 10.100.105.254
        remote_as: 65430
        desc: "netsvr-01 IPv4"
        route_map:
          in: external-networks-v4
          out: internal-networks-v4
      - peer: 192.0.2.205
        loc_ip: 192.0.2.105
        default_originate: true
        desc: "vyos-02 IPv4"
        remote_as: 65105
        route_map:
          out: external-networks-v4
          in: internal-networks-v4
    ipv6:
      - peer: "2001:db8:105::ffff"
        remote_as: 65430
        desc: "netsvr-01 IPv6"
        route_map:
          in: external-networks-v6
          out: internal-networks-v6
      - peer: "2001:db8:905:beef::2"
        loc_ip: "2001:db8:905:beef::1"
        desc: "vyos-02 IPv6"
        remote_as: 65105
        route_map:
          out: external-networks-v6
          in: internal-networks-v6

This generates the following configuration: -

set protocols bgp 65105 address-family ipv4-unicast redistribute ospf
set protocols bgp 65105 address-family ipv6-unicast redistribute ospfv3
set protocols bgp 65105 neighbor 10.100.105.254 address-family ipv4-unicast route-map export 'internal-networks-v4'
set protocols bgp 65105 neighbor 10.100.105.254 address-family ipv4-unicast route-map import 'external-networks-v4'
set protocols bgp 65105 neighbor 10.100.105.254 description 'netsvr-01 IPv4'
set protocols bgp 65105 neighbor 10.100.105.254 remote-as '65430'
set protocols bgp 65105 neighbor 192.0.2.205 address-family ipv4-unicast default-originate
set protocols bgp 65105 neighbor 192.0.2.205 address-family ipv4-unicast route-map export 'external-networks-v4'
set protocols bgp 65105 neighbor 192.0.2.205 address-family ipv4-unicast route-map import 'internal-networks-v4'
set protocols bgp 65105 neighbor 192.0.2.205 description 'vyos-02 IPv4'
set protocols bgp 65105 neighbor 192.0.2.205 remote-as '65105'
set protocols bgp 65105 neighbor 192.0.2.205 update-source '192.0.2.105'
set protocols bgp 65105 neighbor 2001:db8:105::ffff address-family ipv6-unicast route-map export 'internal-networks-v6'
set protocols bgp 65105 neighbor 2001:db8:105::ffff address-family ipv6-unicast route-map import 'external-networks-v6'
set protocols bgp 65105 neighbor 2001:db8:105::ffff description 'netsvr-01 IPv6'
set protocols bgp 65105 neighbor 2001:db8:105::ffff remote-as '65430'
set protocols bgp 65105 neighbor 2001:db8:905:beef::2 address-family ipv6-unicast route-map export 'external-networks-v6'
set protocols bgp 65105 neighbor 2001:db8:905:beef::2 address-family ipv6-unicast route-map import 'internal-networks-v6'
set protocols bgp 65105 neighbor 2001:db8:905:beef::2 description 'vyos-02 IPv6'
set protocols bgp 65105 neighbor 2001:db8:905:beef::2 remote-as '65105'
set protocols bgp 65105 neighbor 2001:db8:905:beef::2 update-source '2001:db8:905:beef::1'
set protocols bgp 65105 parameters default no-ipv4-unicast
set protocols bgp 65105 parameters router-id '192.0.2.105'
Verification

After all of the above has run, we should have BGP sessions up over IPv4 and IPv6, as well as routes received and sent to netsvr-01 BGP route server.

vyos-01

! Show IPv4 BGP neighbours
[email protected]:~$ show ip bgp summary

IPv4 Unicast Summary:
BGP router identifier 192.0.2.105, local AS number 65105 vrf-id 0
BGP table version 8
RIB entries 15, using 2760 bytes of memory
Peers 2, using 41 KiB of memory

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
10.100.105.254  4      65430      62      62        0    0    0 00:56:42            1
192.0.2.205     4      65105      58      61        0    0    0 00:55:54            0

Total number of neighbors 2

! Show IPv4 BGP routes
[email protected]:~$ show ip bgp
BGP table version is 8, local router ID is 192.0.2.105, vrf id 0
Default local pref 100, local AS 65105
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 10.15.30.0/24    0.0.0.0                  0         32768 ?
*> 10.100.105.0/24  0.0.0.0                  0         32768 ?
*> 10.100.205.0/24  0.0.0.0                  0         32768 ?
*> 192.0.2.1/32     10.100.105.254           0             0 65430 i
*> 192.0.2.105/32   0.0.0.0                  0         32768 ?
*> 192.0.2.205/32   10.100.205.253         100         32768 ?
*> 192.168.30.0/24  0.0.0.0                  0         32768 ?
*> 192.168.122.0/24 0.0.0.0                  0         32768 ?

! Ping the IPv4 netsvr Loopback (192.0.2.1)
[email protected]:~$ ping 192.0.2.1
PING 192.0.2.1 (192.0.2.1) 56(84) bytes of data.
64 bytes from 192.0.2.1: icmp_seq=1 ttl=64 time=0.585 ms
64 bytes from 192.0.2.1: icmp_seq=2 ttl=64 time=0.473 ms
^C
--- 192.0.2.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 25ms
rtt min/avg/max/mdev = 0.473/0.529/0.585/0.056 ms

! Show IPv6 BGP neighbours
[email protected]:~$ show ipv6 bgp summary

IPv6 Unicast Summary:
BGP router identifier 192.0.2.105, local AS number 65105 vrf-id 0
BGP table version 2
RIB entries 3, using 552 bytes of memory
Peers 2, using 41 KiB of memory

Neighbor             V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
2001:db8:105::ffff   4      65430      66      65        0    0    0 01:00:11            1
2001:db8:905:beef::2 4      65105      62      63        0    0    0 00:59:28            0

Total number of neighbors 2

! Show IPv6 BGP routes
[email protected]:~$ show ipv6 bgp
BGP table version is 2, local router ID is 192.0.2.105, vrf id 0
Default local pref 100, local AS 65105
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 2001:db8:905:beef::2/128
                    fe80::5054:ff:fe50:4198
                                           101         32768 ?
*> 2001:db8:999:beef::1/128
                    fe80::5fe6:3105:856b:b294
                                             0             0 65430 i

Displayed  2 routes and 2 total paths

! Ping the IPv6 netsvr Loopback (2001:DB8:999:BEEF::1)
[email protected]:~$ ping 2001:DB8:999:BEEF::1
PING 2001:DB8:999:BEEF::1(2001:db8:999:beef::1) 56 data bytes
64 bytes from 2001:db8:999:beef::1: icmp_seq=1 ttl=64 time=0.454 ms
64 bytes from 2001:db8:999:beef::1: icmp_seq=2 ttl=64 time=0.874 ms
^C
--- 2001:DB8:999:BEEF::1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 33ms
rtt min/avg/max/mdev = 0.454/0.664/0.874/0.210 ms

vyos-02

! Show IPv4 BGP neighbours
[email protected]:~$ show ip bgp summary

IPv4 Unicast Summary:
BGP router identifier 192.0.2.205, local AS number 65105 vrf-id 0
BGP table version 3
RIB entries 3, using 552 bytes of memory
Peers 1, using 20 KiB of memory

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
192.0.2.105     4      65105      67      64        0    0    0 01:01:51            3

Total number of neighbors 1

! Show IPv4 BGP routes
[email protected]:~$ show ip bgp
BGP table version is 3, local router ID is 192.0.2.205, vrf id 0
Default local pref 100, local AS 65105
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*>i0.0.0.0/0        192.0.2.105                   100      0 i
*>i10.100.105.0/24  192.0.2.105              0    100      0 ?
*>i192.0.2.1/32     10.100.105.254           0    100      0 65430 i

Displayed  3 routes and 3 total paths

! Ping the IPv4 netsvr Loopback (192.0.2.1)
[email protected]:~$ ping 192.0.2.1
PING 192.0.2.1 (192.0.2.1) 56(84) bytes of data.
64 bytes from 192.0.2.1: icmp_seq=1 ttl=63 time=0.797 ms
64 bytes from 192.0.2.1: icmp_seq=2 ttl=63 time=1.20 ms
64 bytes from 192.0.2.1: icmp_seq=3 ttl=63 time=1.53 ms
^C
--- 192.0.2.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 9ms
rtt min/avg/max/mdev = 0.797/1.175/1.525/0.299 ms

! Show IPv6 BGP neighbours
[email protected]:~$ show ipv6 bgp summary

IPv6 Unicast Summary:
BGP router identifier 192.0.2.205, local AS number 65105 vrf-id 0
BGP table version 1
RIB entries 1, using 184 bytes of memory
Peers 1, using 20 KiB of memory

Neighbor             V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
2001:db8:905:beef::1 4      65105      66      65        0    0    0 01:02:36            1

Total number of neighbors 1

! Show IPv6 BGP routes
[email protected]:~$ show ipv6 bgp
BGP table version is 1, local router ID is 192.0.2.205, vrf id 0
Default local pref 100, local AS 65105
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*>i2001:db8:999:beef::1/128
                    2001:db8:105::ffff
                                             0    100      0 65430 i

Displayed  1 routes and 1 total paths

! Ping the IPv6 netsvr Loopback (2001:DB8:999:BEEF::1)
[email protected]:~$ ping 2001:db8:999:beef::1 interface 2001:db8:905:beef::2
PING 2001:db8:999:beef::1(2001:db8:999:beef::1) from 2001:db8:905:beef::2 : 56 data bytes
64 bytes from 2001:db8:999:beef::1: icmp_seq=1 ttl=63 time=0.815 ms
64 bytes from 2001:db8:999:beef::1: icmp_seq=2 ttl=63 time=1.09 ms
64 bytes from 2001:db8:999:beef::1: icmp_seq=3 ttl=63 time=1.62 ms
^C
--- 2001:db8:999:beef::1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 29ms
rtt min/avg/max/mdev = 0.815/1.177/1.624/0.336 ms

All looking good!

SNMP

In this section we enable SNMP so that we can monitor the routers. Again, no native Ansible module exists, so we use the vyos_config module.

Playbook

The contents of the playbook are below: -

---
# tasks file for snmp
- name: Enable SNMPv3
  vyos_config:
    src: snmpv3.j2
  tags:
    - snmp

Template

The template for this module looks like the below: -

delete service snmp
set service snmp contact "{{ snmp['contact'] }}"
set service snmp location "{{ snmp['location'] }}"
set service snmp v3 engineid '000000000000000000000002'
set service snmp v3 group default mode 'ro'
set service snmp v3 group default view 'default'
set service snmp v3 user {{ snmp['user'] }} auth plaintext-password {{ snmp['auth_key'] }}
set service snmp v3 user {{ snmp['user'] }} auth type 'sha'
set service snmp v3 user {{ snmp['user'] }} group 'default'
set service snmp v3 user {{ snmp['user'] }} privacy plaintext-password {{ snmp['priv_key'] }}
set service snmp v3 user {{ snmp['user'] }} privacy type 'aes'
set service snmp v3 view default oid 1

We don’t need to use any loops for this, so the template is almost identical to the generated configuration (other than some variables that will be replaced). If you wanted more than one SNMPv3 user and/or group, then you would need to add loops to this.

In a production scenario, you would want to randomly generate the engine IDs so that they are not identical on every router, but for the purposes of this lab it demonstrates the functionality.

We use group_vars for this, rather than host_vars, as the same SNMPv3 user is used on both the edge router and the internal router

snmp:
  location: Yeti Home
  contact: The Hairy One
  user: yetiops
  group: yetiops_group
  auth_key: !vault |
            $ANSIBLE_VAULT;1.1;AES256
            383###REDACTED###############################################################566
            363###REDACTED###############################################################366
            343###REDACTED###############################################################565
            326###REDACTED###############################################################362
            3431
  priv_key: !vault |
            $ANSIBLE_VAULT;1.1;AES256
            383###REDACTED###############################################################566
            363###REDACTED###############################################################366
            343###REDACTED###############################################################565
            326###REDACTED###############################################################362
            3431

We use Ansible Vault again, so that we can store our credentials in version control, without storing them unencrypted.

The generated configuration looks like the below: -

set service snmp contact 'The Hairy One'
set service snmp location 'Yeti Home'
set service snmp v3 engineid '000000000000000000000002'
set service snmp v3 group default mode 'ro'
set service snmp v3 group default view 'default'
set service snmp v3 user yetiops auth encrypted-password '###PASSWORD###'
set service snmp v3 user yetiops auth type 'sha'
set service snmp v3 user yetiops group 'default'
set service snmp v3 user yetiops privacy encrypted-password '###PASSWORD###'
set service snmp v3 user yetiops privacy type 'aes'
set service snmp v3 view default oid 1

Verification

To check whether this is working, you will need either some form of monitoring system, or you can use something like snmpwalk to check: -

! snmpwalk to vyos-01
$ snmpwalk -v3 -u yetiops -a SHA -A ###AUTH-KEY### -x AES -X ###PRIV-KEY### -l authPriv 10.15.30.63 
iso.3.6.1.2.1.1.1.0 = STRING: "VyOS 1.3-rolling-202008020117"
iso.3.6.1.2.1.1.2.0 = OID: iso.3.6.1.4.1.44641
iso.3.6.1.2.1.1.3.0 = Timeticks: (19787) 0:03:17.87
iso.3.6.1.2.1.1.4.0 = STRING: "The Hairy One"
iso.3.6.1.2.1.1.5.0 = STRING: "vyos-01"
iso.3.6.1.2.1.1.6.0 = STRING: "Yeti Home"
iso.3.6.1.2.1.1.7.0 = INTEGER: 14
iso.3.6.1.2.1.1.8.0 = Timeticks: (1) 0:00:00.01
iso.3.6.1.2.1.1.9.1.2.1 = OID: iso.3.6.1.6.3.11.3.1.1
iso.3.6.1.2.1.1.9.1.2.2 = OID: iso.3.6.1.6.3.15.2.1.1
iso.3.6.1.2.1.1.9.1.2.3 = OID: iso.3.6.1.6.3.10.3.1.1
iso.3.6.1.2.1.1.9.1.2.4 = OID: iso.3.6.1.6.3.1
iso.3.6.1.2.1.1.9.1.2.5 = OID: iso.3.6.1.6.3.16.2.2.1
iso.3.6.1.2.1.1.9.1.2.6 = OID: iso.3.6.1.2.1.49

! snmpwalk to vyos-02
$ snmpwalk -v3 -u yetiops -a SHA -A ###AUTH-KEY### -x AES -X ###PRIV-KEY### -l authPriv 10.15.30.64
iso.3.6.1.2.1.1.1.0 = STRING: "VyOS 1.3-rolling-202008020117"
iso.3.6.1.2.1.1.2.0 = OID: iso.3.6.1.4.1.44641
iso.3.6.1.2.1.1.3.0 = Timeticks: (618) 0:00:06.18
iso.3.6.1.2.1.1.4.0 = STRING: "The Hairy One"
iso.3.6.1.2.1.1.5.0 = STRING: "vyos-02"
iso.3.6.1.2.1.1.6.0 = STRING: "Yeti Home"
iso.3.6.1.2.1.1.7.0 = INTEGER: 14
iso.3.6.1.2.1.1.8.0 = Timeticks: (0) 0:00:00.00
iso.3.6.1.2.1.1.9.1.2.1 = OID: iso.3.6.1.6.3.11.3.1.1
iso.3.6.1.2.1.1.9.1.2.2 = OID: iso.3.6.1.6.3.15.2.1.1
iso.3.6.1.2.1.1.9.1.2.3 = OID: iso.3.6.1.6.3.10.3.1.1
iso.3.6.1.2.1.1.9.1.2.4 = OID: iso.3.6.1.6.3.1
iso.3.6.1.2.1.1.9.1.2.5 = OID: iso.3.6.1.6.3.16.2.2.1
iso.3.6.1.2.1.1.9.1.2.6 = OID: iso.3.6.1.2.1.49
iso.3.6.1.2.1.1.9.1.2.7 = OID: iso.3.6.1.2.1.4
iso.3.6.1.2.1.1.9.1.2.8 = OID: iso.3.6.1.2.1.50
iso.3.6.1.2.1.1.9.1.2.9 = OID: iso.3.6.1.6.3.13.3.1.3
iso.3.6.1.2.1.1.9.1.2.10 = OID: iso.3.6.1.2.1.92
iso.3.6.1.2.1.1.9.1.3.1 = STRING: "The MIB for Message Processing and Dispatching."
iso.3.6.1.2.1.1.9.1.3.2 = STRING: "The management information definitions for the SNMP User-based Security Model."

NAT

In this section we are allowing the internal router to reach the internet via the edge router. The edge router has a default route to the internet that is learned from DHCP.

Playbook

The playbook looks like the below: -

---
## tasks file for nat
- name: Apply NAT Overload
  vyos_config:
    src: nat-overload.j2
  tags:
  - nat

Again, we are using vyos_config for this, as no Ansible module exists for NAT on VyOS.

Template

The template looks like the below: -

{% if 'edge' in rtr_role %}
delete nat
{% for interface in interfaces %}
{% if 'nat' in interface %}
{% if 'outside' in interface['nat']['role'] %}
set nat source rule 100 source address '0.0.0.0/0'
set nat source rule 100 outbound-interface '{{ interface['vyos_if'] }}'
set nat source rule 100 translation address 'masquerade'
{% endif %}
{% endif %}
{% endfor %}
{% endif %}

The above template removes all the existing NAT configuration so that we start fresh on every apply. This NAT rule allows any traffic leaving via a certain interface (the interface learning a default route from DHCP) to be subject to source NAT.

Our host_vars for this looks like the below: -

  - vyos_if: "eth0"
    desc: "To the Internet"
    ipv4_addr: "dhcp"
    enabled: "true"
    ospf:
      area: "0.0.0.0"
      passive: true
    nat:
      role: "outside"

This then generates the following configuration: -

set nat source rule 100 outbound-interface 'eth0'
set nat source rule 100 source address '0.0.0.0/0'
set nat source rule 100 translation address 'masquerade'

Verification

We can now test from the internal router, to see if it can reach the internet: -

! Can we reach the internet?
[email protected]:~$ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=55 time=15.10 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=55 time=16.7 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=55 time=17.1 ms
^C
--- 1.1.1.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 5ms
rtt min/avg/max/mdev = 15.967/16.601/17.126/0.502 ms
 
[email protected]:~$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=114 time=19.7 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=114 time=21.7 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=114 time=29.4 ms
^C
--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 6ms
rtt min/avg/max/mdev = 19.714/23.600/29.388/4.173 ms

! What does this look like on the edge router?
[email protected]:~$ show nat source statistics

rule      pkts        bytes   interface
----      ----        -----   ---------
100          2          168   eth0

All looking good!

AAA

The final task is AAA (Authentication, Authorization and Accounting). Like MikroTik, VyOS does not support TACACS+ so we configure the routers to authenticate against freeradius running on netsvr-01: -

Playbook

The contents of the playbook are: -

---
# tasks file for aaa
- name: Enable RADIUS
  vyos_config:
    src: radius.j2
  tags:
  - aaa

This uses vyos_config with a Jinja2 template to configure AAA.

Template

The contents of the template are: -

set system login radius server {{ tacacs['ipv4'] }} key {{ radius['secret'] }}
set system login radius source-address {{ router_id }}

As with the MikroTik RADIUS configuration, we use the tacacs IPv4 address because at some point VyOS may support TACACS+, thus making transitioning between the two easier.

The relevant host_vars are: -

router_id: 192.0.2.105
radius:
  secret: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          356431################REDACTED###########################31313136626333
          623664################REDACTED###########################65366437633463
          623135################REDACTED###########################33346233346665
          633265################REDACTED###########################63333834396361
          333835################REDACTED###########################13936

The rest of our variables come from our group_vars: -

tacacs:
  ipv4: 192.0.2.1

The above generates the following configuration: -

set system login radius server 192.0.2.1 key '###RADIUS_SECRET###'
set system login radius source-address '192.0.2.105'

Verification

! Can we login with the yetiops user?
$ ssh [email protected]
----------------------------------------
|
| This banner was generated by Ansible
|
----------------------------------------
|
| You are logged into vyos-02
|
----------------------------------------
|
[email protected]'s password:
Creating directory '/home/yetiops'.
Linux vyos-02 4.19.136-amd64-vyos #1 SMP Sat Aug 1 08:40:04 UTC 2020 x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.


[email protected]:~$

! What about a user that doesn't exist?
$ ssh [email protected]
----------------------------------------
|
| This banner was generated by Ansible
|
----------------------------------------
|
| You are logged into vyos-02
|
----------------------------------------
|
[email protected]'s password:
Permission denied, please try again.
[email protected]'s password:

[email protected]:~$ show log | grep -i jeff
Sep 24 18:25:38 vyos-02 sshd[2901]: Failed password for jeff from 10.15.30.253 port 63877 ssh2
Sep 24 18:25:39 vyos-02 sshd[2901]: Connection closed by authenticating user jeff 10.15.30.253 port 63877 [preauth]

! What do see in our radius log?
Thu Sep 24 19:25:36 2020 : Auth: (12) Login incorrect (No Auth-Type found: rejecting the user via Post-Auth-Type = Reject): [jeff/jkslfjlsfd] (from client vyos-02 port 2901 cli 10.15.30.253)

! What if freeradius goes away?
$ sudo systemctl stop radiusd

$ systemctl status radiusd
  radiusd.service - FreeRADIUS high performance RADIUS server.
   Loaded: loaded (/usr/lib/systemd/system/radiusd.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since Thu 2020-09-24 19:28:38 BST; 14s ago
  Process: 1824 ExecStart=/usr/sbin/radiusd -d /etc/raddb (code=exited, status=0/SUCCESS)
  Process: 1539 ExecStartPre=/usr/sbin/radiusd -C (code=exited, status=0/SUCCESS)
  Process: 1528 ExecStartPre=/bin/chown -R radiusd.radiusd /var/run/radiusd (code=exited, status=0/SUCCESS)
 Main PID: 1826 (code=exited, status=0/SUCCESS)

Sep 24 19:13:09 netsvr-01 systemd[1]: Starting FreeRADIUS high performance RADIUS server....
Sep 24 19:13:13 netsvr-01 systemd[1]: Started FreeRADIUS high performance RADIUS server..
Sep 24 19:28:38 netsvr-01 systemd[1]: Stopping FreeRADIUS high performance RADIUS server....
Sep 24 19:28:38 netsvr-01 systemd[1]: Stopped FreeRADIUS high performance RADIUS server..

$ ssh [email protected]
----------------------------------------
|
| This banner was generated by Ansible
|
----------------------------------------
|
| You are logged into vyos-02
|
----------------------------------------
|
[email protected]'s password:
Permission denied, please try again.

Success!

Parent playbook

The parent playbook (i.e. the playbook that brings all the roles together) is below: -

---
- hosts: vyos
  gather_facts: false
  tasks:
    - import_role:
        name: system
    - import_role:
        name: interfaces
    - import_role:
        name: firewall
    - import_role:
        name: routing
    - import_role:
        name: snmp
    - import_role:
        name: nat
    - import_role:
        name: aaa
    - name: Save configuration
      vyos_config:
        save: true

Like IOS and EOS, we need to save the configuration at the end.

Any changes made before this will be committed to the running/in-memory configuration, but will not be saved to disk until the final Save configuration task.

This can be a little confusing if you are familiar with JunOS (and IOS-XR) as the commit operation makes configuration persist after a reboot, whereas in VyOS it does not. If anything the commit operation in VyOS is used so you can stage your configuration changes, and then apply them all at once (rather than instantly applying each change like in Cisco IOS).

Role Order

The order the roles are applied is identical to JunOS. The justification for the role order is detailed within Part 3 - Cisco IOS. To summarize: -

  • system - Sets up logging, hostnames and banners
    • This ensure we have logging ready for if any of the other roles fail (that the Ansible debug output cannot help with)
  • interfaces - This is a prerequisite for most of the following tasks
  • firewall - Apply before routing so that the device is not open to the world when publiccly routable
  • routing - Routing is required for NAT and AAA to function
  • snmp - No dependency on any service, so this can go anywhere
  • nat - Apply this after routing, otherwise the internal router has no default route to reach external destinations anyway
  • aaa - It depends upon routing, and if configured incorrectly it can break login sessions

Artifacts

The final directory structure looks like the below: -

$ tree -L 2
.
├── ansible.cfg
├── ansible.log
├── group_vars
│   └── vyos
├── host_vars
│   ├── vyos-01.yml
│   └── vyos-02.yml
├── inventory
├── roles
│   ├── aaa
│   ├── firewall
│   ├── interfaces
│   ├── nat
│   ├── routing
│   ├── snmp
│   └── system
└── vyos.yaml

10 directories, 7 files

The final contents of our group_vars are: -

ansible_connection: network_cli
ansible_network_os: vyos
ansible_user: vyos
log_host: 10.100.105.254
tacacs:
  ipv4: 192.0.2.1
  secret: supersecret
snmp:
  location: Yeti Home
  contact: The Hairy One
  user: yetiops
  group: yetiops_group
  auth_key: !vault |
            $ANSIBLE_VAULT;1.1;AES256
            383###REDACTED###############################################################566
            363###REDACTED###############################################################366
            343###REDACTED###############################################################565
            326###REDACTED###############################################################362
            3431
  priv_key: !vault |
            $ANSIBLE_VAULT;1.1;AES256
            386###REDACTED###############################################################764
            613###REDACTED###############################################################630
            646###REDACTED###############################################################331
            376###REDACTED###############################################################137
            3563

The final contents of our host_vars are: -

vyos-01.yaml

router_id: 192.0.2.105
rtr_role: edge
fw_addresses:
  - name: netsvr
    ip: 10.100.105.254
    groups:
      - external-bgp-peers-v4
      - netsvr-direct-v4
  - name: netsvr-v6
    ipv6: "2001:db8:105::ffff"
    groups:
      - external-bgp-peers-v6
      - netsvr-direct-v6
  - name: netsvr-lo
    ip: 192.0.2.1
    groups:
    - netsvr-loop-v4
  - zone: edge
    name: netsvr-lo-v6
    ipv6: "2001:db8:999:beef::1"
    groups:
    - netsvr-loop-v6
  - name: internal-rtr
    ip: 192.0.2.205
    groups:
    - internal-bgp-peers-v4
    - internal-rtr-loop-v4
  - name: internal-rtr-v6
    ipv6: "2001:db8:905:beef::2"
    groups:
    - internal-bgp-peers-v6
    - internal-rtr-loop-v6
fw_policies:
  mgmt:
    ipv4:
      - name: mgmt_to_local_ipv4
        zones:
          from: mgmt
          to: local
        rules:
          - protocol: tcp
            port: 22
            action: accept
          - protocol: udp
            port: 161
            action: accept
  ipv4:
    - name: external_to_local_ipv4
      zones:
        from: external
        to: local
      rules:
        - source_groups: external-bgp-peers-v4
          protocol: tcp
          port: 179
          action: accept
        - protocol: icmp
          action: accept
        - protocol: all
          action: reject
    - name: internal_to_local_ipv4
      zones:
        from: internal
        to: local
      rules:
        - source_groups: internal-bgp-peers-v4
          protocol: tcp
          port: 179
          action: accept
        - protocol: ospf
          action: accept
        - protocol: icmp
          action: accept
        - protocol: all
          action: reject
    - name: external_to_internal_ipv4
      zones:
        from: external
        to: internal
      rules:
        - protocol: icmp
          action: accept
        - protocol: all
          action: reject
    - name: local_to_external_ipv4
      zones:
        from: local
        to: external
      rules:
        - dest_groups: external-bgp-peers-v4
          protocol: tcp
          port: 179
          action: accept
        - dest_groups: netsvr-direct-v4
          protocol: udp
          port: 514
          action: accept
        - dest_groups: netsvr-loop-v4
          protocol: tcp_udp
          port: 1812-1813
          action: accept
        - protocol: icmp
          action: accept
        - protocol: all
          action: reject
    - name: local_to_internal_ipv4
      zones:
        from: local
        to: internal
      rules:
        - dest_groups: internal-bgp-peers-v4
          protocol: tcp
          port: 179
          action: accept
        - protocol: ospf
          action: accept
        - protocol: icmp
          action: accept
        - protocol: all
          action: reject
    - name: internal_to_external_ipv4
      zones:
        from: internal
        to: external
      rules:
        - dest_groups: netsvr-direct-v4
          protocol: udp
          port: 514
          action: accept
        - dest_groups: netsvr-loop-v4
          protocol: tcp_udp
          port: 1812-1813
          action: accept
        - protocol: icmp
          action: accept
        - protocol: all
          action: reject
  ipv6:
    - name: external_to_local_ipv6
      zones:
        from: external
        to: local
      rules:
        - source_groups: external-bgp-peers-v6
          protocol: tcp
          port: 179
          action: accept
        - protocol: icmpv6
          action: accept
        - protocol: all
          action: reject
    - name: internal_to_local_ipv6
      zones:
        from: internal
        to: local
      rules:
        - source_groups: internal-bgp-peers-v6
          protocol: tcp
          port: 179
          action: accept
        - protocol: ospf
          action: accept
        - protocol: icmpv6
          action: accept
        - protocol: all
          action: reject
    - name: external_to_internal_ipv6
      zones:
        from: external
        to: internal
      rules:
        - protocol: icmpv6
          action: accept
        - protocol: all
          action: reject
    - name: local_to_external_ipv6
      zones:
        from: local
        to: external
      rules:
        - dest_groups: external-bgp-peers-v6
          protocol: tcp
          port: 179
          action: accept
        - protocol: ospf
          action: accept
        - protocol: icmpv6
          action: accept
        - protocol: all
          action: reject
    - name: local_to_internal_ipv6
      zones:
        from: local
        to: internal
      rules:
        - dest_groups: internal-bgp-peers-v6
          protocol: tcp
          port: 179
          action: accept
        - protocol: ospf
          action: accept
        - protocol: icmpv6
          action: accept
        - protocol: all
          action: reject
    - name: internal_to_external_ipv6
      zones:
        from: internal
        to: external
      rules:
        - protocol: icmpv6
          action: accept
        - protocol: all
          action: reject
route_maps:
  prefix_lists:
    ipv4:
      - name: internal-nets-v4
        addresses:
           - 192.0.2.105/32
           - 192.0.2.205/32
           - 10.100.205.0/24
        action: permit
      - name: external-nets-v4
        addresses:
           - 192.0.2.1/32
           - 10.100.105.0/24
        action: permit
    ipv6:
      - name: internal-nets-v6
        addresses:
           - "2001:db8:905:beef::1/128"
           - "2001:db8:905:beef::2/128"
           - "2001:db8:905::/64"
        action: permit
      - name: external-nets-v6
        addresses:
           - "2001:db8:999:beef::1/128"
        action: permit
  rm:
    - name: external-networks-v4
      rules:
        - pfx_list: external-nets-v4
          action: permit
        - action: deny
    - name: internal-networks-v4
      rules:
        - pfx_list: internal-nets-v4
          action: permit
        - action: deny
    - name: external-networks-v6
      rules:
        - pfx_list6: external-nets-v6
          action: permit
        - action: deny
    - name: internal-networks-v6
      rules:
        - pfx_list6: internal-nets-v6
          action: permit
        - action: deny
bgp:
  local_as: 65105
  redistribute:
    ospf: true
    ospfv3: true
  neighbours:
    ipv4:
      - peer: 10.100.105.254
        remote_as: 65430
        desc: "netsvr-01 IPv4"
        route_map:
          in: external-networks-v4
          out: internal-networks-v4
      - peer: 192.0.2.205
        loc_ip: 192.0.2.105
        default_originate: true
        desc: "vyos-02 IPv4"
        remote_as: 65105
        route_map:
          out: external-networks-v4
          in: internal-networks-v4
    ipv6:
      - peer: "2001:db8:105::ffff"
        remote_as: 65430
        desc: "netsvr-01 IPv6"
        route_map:
          in: external-networks-v6
          out: internal-networks-v6
      - peer: "2001:db8:905:beef::2"
        loc_ip: "2001:db8:905:beef::1"
        desc: "vyos-02 IPv6"
        remote_as: 65105
        route_map:
          out: external-networks-v6
          in: internal-networks-v6
interfaces:
  - vyos_if: "eth1"
    desc: "Management"
    enabled: "true"
    ipv4_addr: "10.15.30.63/24"
    zone: "mgmt"
  - vyos_if: "eth2"
    desc: "VLAN Bridge"
    enabled: "true"
  - vyos_if: "eth2"
    vif: 105
    desc: "To netsvr"
    enabled: "true"
    ipv4_addr: "10.100.105.253/24"
    ipv6_addr: "2001:db8:105::f/64"
    zone: "external"
    ospf:
      area: "0.0.0.0"
      passive: true
    ospfv3:
      area: "0.0.0.0"
      passive: true
  - vyos_if: "eth2"
    vif: 205
    desc: "To vyos-02"
    enabled: "true"
    zone: "internal"
    ipv4_addr: "10.100.205.254/24"
    ipv6_addr: "2001:db8:205::a/64"
    ospf:
      area: "0.0.0.0"
    ospfv3:
      area: "0.0.0.0"
  - vyos_if: "eth0"
    desc: "To the Internet"
    ipv4_addr: "dhcp"
    enabled: "true"
    ospf:
      area: "0.0.0.0"
      passive: true
    nat:
      role: "outside"
  - vyos_if: "lo"
    desc: "Loopback"
    enabled: "true"
    ipv4_addr: "192.0.2.105/32"
    ipv6_addr: "2001:db8:905:beef::1/128"
    ospf:
      area: "0.0.0.0"
      passive: true
    ospfv3:
      area: "0.0.0.0"
      passive: true
zones:
  - name: external
    description: "External facing interfaces"
  - name: internal
    description: "Internal facing interfaces"
  - name: local
    description: "Local router zone"
    local: true
  - name: mgmt
    description: "Management zone"
radius:
  secret: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          356###REDACTED###############################################################33
          623###REDACTED###############################################################63
          623###REDACTED###############################################################65
          633###REDACTED###############################################################61
          333###REDACTED###########################################36

vyos-02.yaml

router_id: 192.0.2.205
rtr_role: internal
route_maps:
  prefix_lists:
    ipv4:
      - name: internal-nets-v4
        addresses:
           - 192.0.2.105/32
           - 192.0.2.205/32
           - 10.100.205.0/24
        action: permit
      - name: external-nets-v4
        addresses:
           - 192.0.2.1/32
           - 10.100.105.0/24
        action: permit
      - name: default-route-v4
        addresses:
           - 0.0.0.0/0
        action: permit
    ipv6:
      - name: internal-nets-v6
        addresses:
           - "2001:db8:905:beef::1/128"
           - "2001:db8:905:beef::2/128"
           - "2001:db8:905::/64"
        action: permit
      - name: external-nets-v6
        addresses:
           - "2001:db8:999:beef::1/128"
        action: permit
  rm:
    - name: external-networks-v4
      rules:
        - pfx_list: external-nets-v4
          action: permit
        - pfx_list: default-route-v4
          action: permit
        - action: deny
    - name: internal-networks-v4
      rules:
        - pfx_list: internal-nets-v4
          action: permit
        - action: deny
    - name: external-networks-v6
      rules:
        - pfx_list6: external-nets-v6
          action: permit
        - action: deny
    - name: internal-networks-v6
      rules:
        - pfx_list6: internal-nets-v6
          action: permit
        - action: deny
bgp:
  local_as: 65105
  neighbours:
    ipv4:
      - peer: 192.0.2.105
        loc_ip: 192.0.2.205
        desc: "vyos-02 IPv4"
        remote_as: 65105
        route_map:
          in: external-networks-v4
          out: internal-networks-v4
    ipv6:
      - peer: "2001:db8:905:beef::1"
        loc_ip: "2001:db8:905:beef::2"
        desc: "vyos-02 IPv6"
        remote_as: 65105
        route_map:
          in: external-networks-v6
          out: internal-networks-v6
interfaces:
  - vyos_if: "eth1"
    desc: "Management"
    enabled: "true"
    ipv4_addr: "10.15.30.34/24"
  - vyos_if: "eth2"
    desc: "VLAN Bridge"
    enabled: "true"
  - vyos_if: "eth2"
    vif: 205
    desc: "To vyos-01"
    enabled: "true"
    ipv4_addr: "10.100.205.253/24"
    ipv6_addr: "2001:db8:205::f/64"
    ospf:
      area: "0.0.0.0"
    ospfv3:
      area: "0.0.0.0"
  - vyos_if: "lo"
    desc: "Loopback"
    enabled: "true"
    ipv4_addr: "192.0.2.205/32"
    ipv6_addr: "2001:db8:905:beef::2/128"
    ospf:
      area: "0.0.0.0"
      passive: true
    ospfv3:
      area: "0.0.0.0"
      passive: true
radius:
  secret: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          356###REDACTED###############################################################33
          623###REDACTED###############################################################63
          623###REDACTED###############################################################65
          633###REDACTED###############################################################61
          333###REDACTED###########################################36

As with JunOS, there are quite a few variables, especially for building firewalls. However remember this is all defined in a static configuration file. If you use another system like Netbox as a source of truth and data source for Ansible, you can manage many of these variables there instead. This lowers the barrier of entry for network configuration, meaning that those without access to the routers themselves would potentially be able to provision services for customers.

Running the playbooks

Below is an Asciinema output of my terminal when running the playbooks, so you can see them being applied: -

A few tasks are marked as changed, as we have seen previously in the IOS, JunOS and EOS posts. As Ansible is reading the configuration and checking the output matches the input, there are times where what is supplied is not identical to the configuration.

Again, using something like changed_when could tidy this up significantly.

Native modules versus vyos_config

Below is a summary of how many different modules are used, and also how many in total were native modules (compared to using vyos_config).

Module Used
vyos_config 21
vyos_l3_interfaces 4
vyos_interfaces 2
vyos_banner 2
vyos_system 1

Compared to IOS and JunOS, the majority of the tasks use the vyos_config module. As with JunOS, no modules exist for any routing protocols, firewalling or similar.

The time to configure these two VyOS routers is quite small, both complete in under 2 minutes, which is similar to JunOS. If you ran more VyOS routers and used the parallel execution features within Ansible, you can conceivably configure an entire core network, edge router network or VPN concentrator cluster in a similar amount of time. This significantly reduces the time required to make and implement changes.

Thoughts on VyOS

VyOS is an interesting mixture of IOS and JunOS, bringing about many advantages of both.

For those coming from an IOS background (even most Juniper, Nokia or Extreme engineers will have some experience in IOS), most of the verification commands will be familiar to you. Everything from show ip bgp summary to show mpls ldp binding are close to (or the same as) what you would use in IOS.

If you come from a Juniper background, then the configuration approach will be immediately familiar, along with the commit-style approach to staging multiple changes into one “apply”.

VyOS is in a constant state of improvement, and with the ability to run it on commodity hardware, virtualized, in the cloud or anywhere you see fit, it is an incredible powerful option that I wouldn’t hesitate to use and recommend it..

Summary

Using Ansible with VyOS, or any configuration management solution for your network hardware brings huge benefits over manually managing your core infrastructure. Version controlled configuration changes, parallel execution, configuration consistency and using a source of truth to define the network (rather than network being the source of truth) can reduce mistakes and give more time for engineers to investigate new products, designs and improvements to their infrastructure.

If you are interested in seeing how others approach configuration management with VyOS, I would highly recommend the following from Faelix: -

Future parts of this series

For those who have followed this series so far, you will probably have noticed a few things: -

  • The first 6 parts of the series took over 3 months to put together
    • This includes building the labs, preparing the roles, testing them, and then writing the posts
  • This post comes out over 4 months after the last post
  • Many of the sections are almost word-for-word identical to the same sections in other posts

In some cases, especially the MikroTik lab, it would take days and even weeks to create the correct roles. Additionally, writing the posts to describe these roles can often take as much (if not more) time as creating the roles themselves.

My career focus over the past few years has been more towards DevOps and Site Reliability Engineering, meaning the time spent on creating these Ansible roles is becoming less and less relevant to my day-to-day work.

Because of this, I have decided that rather than building individual posts for other vendors I am going to: -

  • Build the Ansible roles for a number of other vendors
    • Cumulus
    • Extreme EXOS
    • OpenBSD
    • HP Procurve
    • HPE/H3C Comware/Huawei
    • Nokia (ex-Alcatel)
    • Cisco IOS-XR
    • Cisco NX-OS
    • Check Point GAiA firewalls
    • Fortinet Fortigate firewalls
  • Commit them to my Ansible Network Automation Repository on Gitlab
  • Create a anthology/compendium post that covers any quirks, gotchyas or other interesting parts of working with across all the vendors rather than single posts per vendor

With many of the tasks being almost identical except for vendor syntax changes, it makes more sense to provide a single repository to allow people to compare the differences for themselves.

I have enjoyed putting this series together so far and I now feel that the series has achieved it’s goal of showing how to use Ansible to manage network infrastructure. Any differences in future roles will be down to vendor-specific syntax rather than with any significant changes to the Ansible roles themselves (in terms of logic or approach).

If you have found these posts useful, let me know if you’d like to see other vendors covered in the Gitlab repository, any questions you have or any improvements you think could be made across all of them (pull/merge requests always welcome!).