85 minutes
Ansible for Networking - Part 6: MikroTik RouterOS
The sixth part of my ongoing series of posts on Ansible for Networking will cover Mikrotik’s RouterOS. You can view the other posts in the series below: -
- Part 1 - Start of the series
- Part 2 - The Lab Environment
- Part 3 - Cisco IOS
- Part 4 - Juniper JunOS
- Part 5 - Arista EOS
- Part 7 - VyOS
All the playbooks, roles and variables used in this article are available in my Network Automation with Ansible repository.
Why RouterOS?
MikroTik is a Latvian company who provide routing, switching, wireless and other networking devices. Their operating system (RouterOS) is built upon Linux, but unlike Arista EOS (or the BSD base of JunOS), you don’t typically have access to a Linux shell itself.
MikroTik are often significantly lower in price than what you’d find from other vendors. For example, I have a MikroTik RB4011 for my home router that has 10 single gigabit ports and 1 ten gigabit port. You’d typically be looking in the multiple hundreds or thousands of pounds for a similar offering from other vendors. The price? £180.
MikroTiks have a reputation for being a networking Swiss army knife. Even on their hAP Lite (a £20 access point and router), they support packet captures, BGP, stateful firewalling, IPSec VPNs and more. You are likely to see high resource usage and performance impact when enabling some of these features, but the fact they are available at this price level is astounding.
Because of the price and flexibility of MikroTik devices, they are a very popular option for smaller ISPs and WISPs (Wireless ISPs). I have used the extensively in my career, in everything from VPN concentration to regional layer 2 extensions.
RouterOS CLI
RouterOS’s command line interface is unique in the networking world. For example, to add an IP address to an interface on a MikroTik, you would do the following: -
[admin@routeros-01] > /ip address add interface=test address=192.168.89.1/24
[admin@routeros-01] >
To remove it, you wouldn’t run /ip address remove interface=test address=192.168.89.1/24. Instead, you’ll need to do one of the following: -
Find out the ID/number of the interface and remove
! Print IP addressing
[admin@routeros-01] > /ip address print
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK INTERFACE
0 ;;; Management
10.15.30.53/24 10.15.30.0 ether1
1 ;;; Loopback
192.0.2.104/32 192.0.2.104 loopback0
2 ;;; To netsvr
10.100.104.253/24 10.100.104.0 vlan104
3 ;;; To routeros-02
10.100.204.254/24 10.100.204.0 vlan204
4 D 192.168.122.208/24 192.168.122.0 ether3
5 192.168.89.1/24 192.168.89.0 test
! Remove the IP address
[admin@routeros-01] > /ip address remove 5
! Check it is removed
[admin@routeros-01] > /ip address print
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK INTERFACE
0 ;;; Management
10.15.30.53/24 10.15.30.0 ether1
1 ;;; Loopback
192.0.2.104/32 192.0.2.104 loopback0
2 ;;; To netsvr
10.100.104.253/24 10.100.104.0 vlan104
3 ;;; To routeros-02
10.100.204.254/24 10.100.204.0 vlan204
4 D 192.168.122.208/24 192.168.122.0 ether3
Using find
! Print IP addressing
[admin@routeros-01] > /ip address print
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK INTERFACE
0 ;;; Management
10.15.30.53/24 10.15.30.0 ether1
1 ;;; Loopback
192.0.2.104/32 192.0.2.104 loopback0
2 ;;; To netsvr
10.100.104.253/24 10.100.104.0 vlan104
3 ;;; To routeros-02
10.100.204.254/24 10.100.204.0 vlan204
4 D 192.168.122.208/24 192.168.122.0 ether3
5 192.168.89.1/24 192.168.89.0 test
! Using the find command
[admin@routeros-01] > /ip address remove [find where address="192.168.89.1/24" and interface="test"]
! Check it is removed
[admin@routeros-01] > /ip address print
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK INTERFACE
0 ;;; Management
10.15.30.53/24 10.15.30.0 ether1
1 ;;; Loopback
192.0.2.104/32 192.0.2.104 loopback0
2 ;;; To netsvr
10.100.104.253/24 10.100.104.0 vlan104
3 ;;; To routeros-02
10.100.204.254/24 10.100.204.0 vlan204
4 D 192.168.122.208/24 192.168.122.0 ether3
If you want to change the address rather than deleting it, you would use commands like /ip address set 5 address=X.X.X.X/24 or /ip address set [find where address="192.168.89.1/24" and interface="test"] address=X.X.X.X/24.
Also, the ID numbers are not always consistent. For example if you deleted the second IP address (192.0.2.104/32) in the list above, this will move the third IP address to second in the list. However if you then tried to run /ip address remove 2, the command line would return “no such item” (as you have already removed the second item in the list). You need to run /ip address print to “regenerate” the list, and then you can remove the second address,
Finally, some settings do not have ID numbers. A good example would be the hostname of the device (/system identity set name=$HOSTNAME). With these settings, you cannot add or delete settings, only set (i.e. updating/changing them).
All of this makes automating the configuration with Ansible (and other tools) more complex. Because you cannot reference elements of configuration directly, and IDs can change between commands, it isn’t quite as straightforward as something like IOS or JunOS. Also, because some commands use set for configuration and others require add (and then later using set to change them), it requires a lot of trial and error to build a list of commands that produces the same configuration every time they are applied (i.e. idempotence).
Objectives
For each vendor, I will be using Ansible to configure two routers/switches/firewalls/appliances.
One will serve as the Edge router, connecting to the Internet and also via BGP to the Net Server. The Net Server is a CentOS 8 Virtual Machine acting as a route server, syslog collector and TACACS+ server (detailed in The Lab Environment)
The other will be an internal router, performing core functions (i.e. internal routing rather than external).
This lab is based upon the MikroTik CHR (Cloud Hosted Router) platform.
Edge router
The edge router will run the following: -
- External BGP (eBGP) to the Net Server
- Advertising internal networks
- Internal BGP (iBGP) to the Internal router
- Advertising any routes received from the Net Server
- Advertising a default route (for internet access)
- OSPF
- Advertising loopbacks and internal networks between both routers
- IPv4 and IPv6 routing
- Using OSPFv3 (for IPv6 support)
- Using the IPv6 Address Family for BGP
- SNMPv3 for monitoring
- IPv4 NAT to allow internet access
- I cannot run IPv6 for internet access, as my current ISP does not support IPv6
- Logging via Syslog to the Net Server
- Authentication, Authorization and Accounting (AAA) via
TACACS+RADIUS to the Net Server - Zones to place interfaces in, for zone-based firewalling
- Firewall Rules to allow traffic to/from the Net Server
Notice in the AAA objective, we are using RADIUS, rather than TACACS+. This is because MikroTik devices do not support TACACS+.
Internal router
The internal router runs a subset of the functions that the edge router does: -
- Internal BGP (iBGP) to the Edge router
- Receiving any routes received from the Net Server
- Receiving a default route (for internet access)
- OSPF
- Advertising loopbacks and internal networks between both routers
- IPv4 and IPv6 routing
- Using OSPFv3 (for IPv6 support)
- Using the IPv6 Address Family for BGP
- SNMPv3 for monitoring
- Logging via Syslog to the Net Server
- Authentication, Authorization and Accounting (AAA) via
TACACS+RADIUS to the Net Server - Zones to place interfaces in, for zone-based firewalling
- Firewall Rules to allow traffic to/from the Net Server
Notice in the AAA objective, we are using RADIUS, rather than TACACS+. This is because MikroTik devices do not support TACACS+.
RADIUS
As RADIUS requires a new role on netsvr-01, I will update the original post with the details on how we configure this using Ansible. Support for TACACS+ has been suggested as a feature request on RouterOS, but at the time of writing it is not available.
Prerequisites
To manage a MikroTik device with Ansible, the following steps are required. We also make some changes to the default Ansible connection configuration.
Ansible Configuration
The following defaults are required to use Ansible with MikroTik RouterOS: -
ansible_user: ansible
ansible_connection: network_cli
ansible_network_os: routeros
ansible_ssh_pass: !vault |
$ANSIBLE_VAULT;1.1;AES256
383###REDACTED###############################################################566
363###REDACTED###############################################################366
343###REDACTED###############################################################565
326###REDACTED###############################################################362
3431
As with JunOS, there is no enable or privileged mode within RouterOS. Instead, the user privileges determine whether a user can run commands or make changes.
MikroTik Configuration
To allow Ansible access to the MikroTik routers, they need an account creating with full privileges (the default RouterOS administrator group). We will also set the IP address and the hostname.
Below shows how to enable all of this: -
## Add the user
[admin@mikrotik] > /user add name=ansible group=full password=###PASSWORD###
## Add an IP to the management interface
[admin@mikrotik] > /ip address add address=10.15.30.53/24 comment=Management interface=ether1
## Verify the IP address is applied
[admin@mikrotik] > /ip address print where comment=Management
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK INTERFACE
0 ;;; Management
10.15.30.53/24 10.15.30.0 ether1
## Set the hostname
[admin@mikrotik] > /system identity set name=routeros-01
[admin@routeros-01] >
Additionally, you can also set up SSH keys to allow SSH access without using a password. To do this, first you must transfer your public key to the device. I do this with scp ~/.ssh/id_ed25519.pub ansible@$ROUTER-IP. After this, add the key to user like so: -
## Verify the key exists on the router
[admin@routeros-01] > /file print where name ~"id.*pub"
# NAME TYPE IZE CREATION-TIME
0 id_ed25519.pub ssh key 98 apr/12/2020 19:14:11
## Add it to the Ansible user
[admin@routeros-01] > /user ssh-keys import public-key-file=id_ed25519.pub user=ansible
At this point, you should be able to SSH to the router without using a password.
As you can see above, we already have multiple different ways of applying configuration: -
/ip address add- Add an IP address/system identity set- Set an identity/user ssh-keys import- Import a key
When we start looking at the playbooks, you’ll notice this again and again!
Our inventory file looks like the below: -
[mikrotik]
routeros-01 ansible_host=10.15.30.53
routeros-02 ansible_host=10.15.30.54
Verification
Can we contact both devices?
$ ansible mikrotik -m routeros_facts --ask-vault-pass | grep -i hostname
Vault password:
"ansible_net_hostname": "routeros-01",
"ansible_net_hostname": "routeros-02",
Setup
The setup is identical to the IOS and Juniper lab, with a management interface to access the devices, a VLAN bridge for inter-device communication, and an interface on the edge router attached to the KVM NAT bridge for DHCP/Internet access. Unlike the Arista vEOS images, VLANs work correctly in the lab.
VLANs, IP addressing and Autonomous System numbers
The ID chosen for MikroTik RouterOS is 04.
VLANs
The VLANs used will be: -
- VLAN104 between the edge router and netsvr-01
- VLAN204 between the edge router and internal router
IP Addressing
- IPv4 Subnet on VLAN104:
10.100.104.0/24- edge router -
10.100.104.253/24 - netsvr-01 -
10.100.104.254/24
- edge router -
- IPv4 Subnet on VLAN204: 10.100.204.0/24
- edge router -
10.100.204.254/24 - internal router -
10.100.204.253/24
- edge router -
- IPv6 Subnet on VLAN104:
2001:db8:104::/64- edge router -
2001:db8:104::f/64 - netsvr-01 -
2001:db8:104:ffff/64
- edge router -
- IPv6 Subnet on VLAN204: 2001:db8:204::/64
- edge router -
2001:db8:204::a/64 - internal router -
2001:db8:204:f/64
- edge router -
- IPv4 Loopback Addressing
- edge router -
192.0.2.104/32 - internal router -
192.0.2.204/32
- edge router -
- IPv6 Loopback Address
- edge router -
2001:db8:904:beef::1/128 - internal router -
2001:db8:904:beef::2/128
- edge router -
BGP Autonomous System
The BGP Autonomous System number will be AS65104.
Configuration
Before we start looking at the Ansible playbooks themselves, there is one major point to address when managing RouterOS with Ansible. At the time of writing only two modules exist for RouterOS, routeros_facts and routeros_command. We can either gather information from the device itself, or we can run RouterOS commands.
Our playbooks consist of raw RouterOS commands, with no inbuilt idempotence or abstraction. To make configuration apply the same way twice, we use RouterOS features like find where (i.e. conditional logic) and in some cases command parsing and regular expressions.
System tasks
This role sets the hostname, sets the login banner, and enables remote logging to syslog on the netsvr-01 machine.
Playbook
The contents of the playbook are as follows: -
---
## tasks file for system
- name: Set hostname
routeros_command:
commands:
- /system identity set name="{{ inventory_hostname }}"
- name: Update login banner
routeros_command:
commands:
- /system note set show-at-login=yes
- >
/system note set note="\
\n ----------------------------------------\
\n |\
\n | This banner was generated by Ansible\
\n |\
\n ----------------------------------------\
\n |\
\n | You are logged into {{ inventory_hostname }}\
\n |\
\n ----------------------------------------"
- name: Configure syslog
routeros_command:
commands:
- /system logging remove [find where action=netsvr]
- /system logging action set [find where action=remote] remote="{{ log_host }}"
- /system logging action remove [find where name=netsvr]
- /system logging action add name="netsvr" remote="{{ log_host }}" remote-port=514 syslog-facility=local7 syslog-severity=auto target=remote
- /system logging add action=netsvr disabled=no prefix={{ inventory_hostname }} topics=system,info
- /system logging add action=netsvr disabled=no prefix={{ inventory_hostname }} topics=warning
- /system logging add action=netsvr disabled=no prefix={{ inventory_hostname }} topics=critical
- /system logging add action=netsvr disabled=no prefix={{ inventory_hostname }} topics=error,!ospf,!route
tags:
- logging
As you can see, every command is a native RouterOS command. We also see our first occurrence of find where.
find where and idempotence
The find where syntax is used to find any configuration items that match a certain value.
For example, in the above we have /system logging remove [find where action=netsvr]. This line looks for any configuration in the /system logging context that has an action that equals netsvr, and removes it. Similarly, the line /system logging action remove [find where name=netsvr] looks for any configuration in the /system logging action context with a name of netsvr.
The reason we cannot just remove configuration lines is due to the aforementioned issue where certain configuration has an id. This id changes based upon how many other elements of configuration exist within this context: -
[admin@routeros-01] /system logging action> print
Flags: * - default
0 * name="memory" target=memory memory-lines=1000 memory-stop-on-full=no
1 * name="disk" target=disk disk-file-name="log" disk-lines-per-file=1000 disk-file-count=2 disk-stop-on-full=no
2 * name="echo" target=echo remember=yes
3 * name="remote" target=remote remote=192.0.2.1 remote-port=514 src-address=0.0.0.0 bsd-syslog=no
syslog-time-format=bsd-syslog syslog-facility=daemon syslog-severity=auto
4 name="netsvr" target=remote remote=10.100.104.254 remote-port=514 src-address=0.0.0.0 bsd-syslog=no
syslog-time-format=bsd-syslog syslog-facility=local7 syslog-severity=auto
In the above, if we had another line before netsvr, the netsvr action would have an id of 5. This makes the id unpredictable, so we cannot build playbooks that make assumptions on what id number a line of configuration will have.
Additionally we cannot assume that the configuration already exists, meaning that we may need to use either add (i.e. adding new configuration) or set (updating existing configuration).
As in previous parts of this series, we will sometimes remove some existing configuration and rebuild it. This allows the configuration to reflect the variables we have supplied, without orphaned configuration.
Generated configuration
The variables we use for this configuration are inventory_hostname (an in-built Ansible variable) and log_host, which exists in our group_vars: -
$ cat group_vars/junos | grep -i log
log_host: 10.100.101.254
The actual generated configuration from the above is: -
/system identity
set name=routeros-01
/system note
set show-at-login=yes
set note="\
\n ----------------------------------------\
\n |\
\n | This banner was generated by Ansible\
\n |\
\n ----------------------------------------\
\n |\
\n | You are logged into routeros-01\
\n |\
\n ----------------------------------------"
/system logging action
set 3 remote=192.0.2.1
add name=netsvr remote=10.100.104.254 syslog-facility=local7 target=remote
/system logging
add action=netsvr prefix=routeros-01 topics=system,info
add action=netsvr prefix=routeros-01 topics=warning
add action=netsvr prefix=routeros-01 topics=critical
add action=netsvr prefix=routeros-01 topics=error,!ospf,!route
Verification
Can we see log messages getting to netsvr-01?
$ tail -n 4 /var/log/remote/10.100.104.253
2020-05-05T19:45:16+01:00 10.100.104.253: system,info,account routeros-01: user ansible logged in from 10.15.30.1 via ssh
2020-05-05T19:45:28+01:00 10.100.104.253: system,info,account routeros-01: user ansible logged out from 10.15.30.1 via ssh
2020-05-05T19:45:42+01:00 10.100.104.253: system,info,account routeros-01: user ansible logged in from 10.15.30.1 via ssh
2020-05-05T19:45:53+01:00 10.100.104.253: system,info,account routeros-01: user ansible logged out from 10.15.30.1 via ssh
$ tail -n 4 /var/log/remote/10.100.204.253
2020-05-05T19:45:28+01:00 10.100.204.253: system,info,account routeros-02: user ansible logged out from 10.15.30.1 via ssh
2020-05-05T19:45:42+01:00 10.100.204.253: system,info,account routeros-02: user ansible logged in from 10.15.30.1 via ssh
2020-05-05T19:45:53+01:00 10.100.204.253: system,info,account routeros-02: user ansible logged out from 10.15.30.1 via ssh
2020-05-05T19:49:26+01:00 10.100.204.253: system,info,account routeros-02: user admin logged in from 10.15.30.1 via ssh
Looks good!
Interfaces
This role configures all of the interfaces, including VLANs, creating “loopbacks” and IP addressing (IPv4 and IPv6).
Playbook
The contents of the playbook are below: -
---
## tasks file for interfaces
##
- name: Configure VLANs
routeros_command:
commands:
- /interface vlan add name="vlan{{ item.vlan_id }}" interface="{{ item.interface }}" comment="To {{ item.name }}" vlan-id="{{ item.vlan_id }}"
when:
- vlans is defined
loop: "{{ vlans }}"
tags:
- vlans
- name: Create Loopbacks
routeros_command:
commands:
- /interface bridge add name="{{ item.routeros_if }}" comment="{{ item.desc }}"
when:
- item.routeros_if is match('loopback.*')
loop: "{{ interfaces }}"
tags:
- loopback
- name: Configure interfaces
routeros_command:
commands:
- /interface set comment="{{ item.desc }}" [find where name="{{ item.routeros_if }}"]
loop: "{{ interfaces }}"
tags:
- interfaces
- name: Configure IPv4 addressing
routeros_command:
commands:
- /ip address add interface="{{ item.routeros_if }}" address="{{ item.ipv4 }}" comment="{{ item.desc }}"
when:
- item.ipv4 is defined
- '"dhcp" not in item.ipv4'
loop: "{{ interfaces }}"
tags:
- ipv4
- name: Configure IPv4 addressing - DHCP
routeros_command:
commands:
- /ip dhcp-client add interface="{{ item.routeros_if }}" disabled=no comment="{{ item.desc }}"
when:
- item.ipv4 is defined
- '"dhcp" in item.ipv4'
loop: "{{ interfaces }}"
tags:
- ipv4
- name: Configure IPv6 addressing
routeros_command:
commands:
- /ipv6 address add interface="{{ item.routeros_if }}" address="{{ item.ipv6 }}" comment="{{ item.desc }}"
when: item.ipv6 is defined
loop: "{{ interfaces }}"
tags:
- ipv6
- name: Interface cleanup
routeros_command:
commands:
- /ip address remove [find where invalid]
tags:
- ipv4
- ipv6
Again, no Ansible RouterOS Ansible modules exist outside of routeros_command (for configuration changes). We are using parametrized RouterOS commands, the variables being sourced from our host_vars or group_vars.
Configuring VLANs
There are many different ways to configure VLANs in RouterOS, in part due to the different switch chips MikroTik have used across their devices (historically and currently). Some benefit from VLAN tags being switched/evaluated in hardware, whereas others require software bridging.
The approach I am using is the /interface vlan add command, as we are using the RouterOS images for routing rather than switching. If we were switching (rather than routing) traffic, we would probably need to use one of the bridge vlan options (see here).
This task checks to see if any VLANs are defined. If any are, it loops through and adds them (configuring the name, ID and interface they are tagged on).
The relevant host_vars that we create VLANs with are: -
vlans:
- name: netsvr-01
vlan_id: 104
interface: ether2
- name: routeros-02
vlan_id: 204
interface: ether2
This generates the following configuration: -
/interface vlan add comment="To netsvr" interface=ether2 name=vlan104 vlan-id=104
/interface vlan add comment="To routeros-02" interface=ether2 name=vlan204 vlan-id=204
We can verify this with: -
[admin@routeros-01] > /interface vlan print
Flags: X - disabled, R - running
# NAME MTU ARP VLAN-ID INTERFACE
0 R ;;; To netsvr
vlan104 1500 enabled 104 ether2
1 R ;;; To routeros-02
vlan204 1500 enabled 204 ether2
Creating Loopbacks
RouterOS does not have any pre-defined loopback interfaces (like in JunOS), or an interface type of “Loopback” (e.g. LoopbackX in IOS). To have something like a loopback within MikroTik, we create a bridge interface with no physical ports bridged to it. The interface still comes up, and is routable, without the need to bind it to any other interface.
We need to create these interfaces, rather than running something like interface Loopback0 (as we would in IOS). To do so, we loop through our interfaces defined in host_vars and use a small regular expression to match against the names of our defined interfaces. If any have “loopback” in the name, we create a bridge with that name.
For example, in our host_vars we have: -
interfaces:
- routeros_if: "ether1"
- routeros_if: "ether2"
- routeros_if: "vlan104"
- routeros_if: "vlan204"
- routeros_if: "ether3"
- routeros_if: "loopback0"
When evaluated against our Ansible match function (item.routeros_if is match('loopback.*'), the only one that matches our expression is loopback0.
The generated configuration is therefore: -
/interface bridge add comment=Loopback name=loopback0
We can verify this has been created with: -
[admin@routeros-01] /interface bridge> print brief
Flags: X - disabled, R - running
# NAME MTU ACTUAL-MTU L2MTU
0 R ;;; Loopback
loopback0 auto 1500 65535
Configure IPv4 Addressing
This task loops through our interface host_vars, and when it has an IPv4 address (that is not dhcp), it adds them to the device. Unlike IOS, JunOS or EOS, DHCP is configured in a different configuration context (rather than something like set interface fxp0 unit 0 family inet address dhcp).
The relevant host_vars are below: -
- routeros_if: "ether1"
desc: "Management"
ipv4: "10.15.30.53/24"
- routeros_if: "ether2"
desc: "VLAN Bridge"
- routeros_if: "vlan104"
desc: "To netsvr"
ipv4: "10.100.104.253/24"
- routeros_if: "vlan204"
desc: "To routeros-02"
ipv4: "10.100.204.254/24"
- routeros_if: "ether3"
desc: "To the Internet"
ipv4: "dhcp"
- routeros_if: "loopback0"
desc: "Loopback"
ipv4: "192.0.2.104/32"
This generates the following configuration: -
/ip address add address=10.15.30.53/24 comment=Management interface=ether1 network=10.15.30.0
/ip address add address=192.0.2.104 comment=Loopback interface=loopback0 network=192.0.2.104
/ip address add address=10.100.104.253/24 comment="To netsvr" interface=vlan104 network=10.100.104.0
/ip address add address=10.100.204.254/24 comment="To routeros-02" interface=vlan204 network=10.100.204.0
As noted, no configuration is generated for the DHCP interface.
We can verify this with: -
[admin@routeros-01] > /ip address print
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK INTERFACE
0 ;;; Management
10.15.30.53/24 10.15.30.0 ether1
1 ;;; Loopback
192.0.2.104/32 192.0.2.104 loopback0
2 ;;; To netsvr
10.100.104.253/24 10.100.104.0 vlan104
3 ;;; To routeros-02
10.100.204.254/24 10.100.204.0 vlan204
Configure IPv4 DHCP
This task loops through our interface host_vars. If the interface has an IPv4 address field that contains the word dhcp, then we will configure that interface as a DHCP client.
The same host_vars from the previous task are relevant here. This generates the following configuration: -
/ip dhcp-client add comment="To the Internet" dhcp-options=hostname,clientid disabled=no interface=ether3
The dhcp-options field is a default value. You can add or remove options from this if you choose.
We can verify this with: -
[admin@routeros-01] /ip dhcp-client> print
Flags: X - disabled, I - invalid, D - dynamic
# INTERFACE USE-PEER-DNS ADD-DEFAULT-ROUTE STATUS ADDRESS
0 ;;; To the Internet
ether3 yes yes bound 192.168.122.208/24
[admin@routeros-01] > /ip dns print
servers:
dynamic-servers: 192.168.122.1
allow-remote-requests: no
max-udp-packet-size: 4096
query-server-timeout: 2s
query-total-timeout: 10s
max-concurrent-queries: 100
max-concurrent-tcp-sessions: 20
cache-size: 2048KiB
cache-max-ttl: 1w
cache-used: 17KiB
[admin@routeros-01] > /ip route print where dst-address=0.0.0.0/0
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 ADS 0.0.0.0/0 192.168.122.1 1
Configure IPv6 addressing
This task is identical to the IPv4 addressing task, except that it applies IPv6 addresses instead. The relevant host_vars are below: -
- routeros_if: "vlan104"
desc: "To netsvr"
ipv6: "2001:db8:104::f/64"
- routeros_if: "vlan204"
desc: "To routeros-02"
ipv6: "2001:db8:204::a/64"
- routeros_if: "loopback0"
desc: "Loopback"
ipv6: "2001:db8:904:beef::1/128"
This generates the following configuration: -
/ipv6 address add address=2001:db8:104::f comment="To netsvr" interface=vlan104
/ipv6 address add address=2001:db8:204::a comment="To routeros-02" interface=vlan204
/ipv6 address add address=2001:db8:904:beef::1/128 advertise=no comment=Loopback interface=loopback0
We can verify this with: -
[admin@routeros-01] > /ipv6 address print
Flags: X - disabled, I - invalid, D - dynamic, G - global, L - link-local
# ADDRESS FROM-POOL INTERFACE ADVERTISE
0 G ;;; To netsvr
2001:db8:104::f/64 vlan104 yes
1 G ;;; To routeros-02
2001:db8:204::a/64 vlan204 yes
2 G ;;; Loopback
2001:db8:904:beef::1/128 loopback0 no
The advertise=no option is because loopback0 is not a physical interface. This means that we do not require IPv6 Neighbour Discovery.
Verification
routeros-01
! Show IPs (IPv4 and IPv6)
[admin@routeros-01] > /ip address print
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK INTERFACE
0 ;;; Management
10.15.30.53/24 10.15.30.0 ether1
1 ;;; Loopback
192.0.2.104/32 192.0.2.104 loopback0
2 ;;; To netsvr
10.100.104.253/24 10.100.104.0 vlan104
3 ;;; To routeros-02
10.100.204.254/24 10.100.204.0 vlan204
4 D 192.168.122.208/24 192.168.122.0 ether3
[admin@routeros-01] > /ipv6 address print
Flags: X - disabled, I - invalid, D - dynamic, G - global, L - link-local
# ADDRESS FROM-POOL INTERFACE ADVERTISE
0 G ;;; To netsvr
2001:db8:104::f/64 vlan104 yes
1 G ;;; To routeros-02
2001:db8:204::a/64 vlan204 yes
2 G ;;; Loopback
2001:db8:904:beef::1/128 loopback0 no
! Show interface statuses and descriptions
[admin@routeros-01] > /interface print
Flags: D - dynamic, X - disabled, R - running, S - slave
# NAME TYPE ACTUAL-MTU L2MTU MAX-L2MTU MAC-ADDRESS
0 R ;;; Management
ether1 ether 1500 52:54:00:CA:DA:15
1 R ;;; VLAN Bridge
ether2 ether 1500 52:54:00:C0:5D:55
2 R ;;; To the Internet
ether3 ether 1500 52:54:00:30:EC:65
3 R ;;; Loopback
loopback0 bridge 1500 65535 42:E5:10:16:E9:17
4 R ;;; To netsvr
vlan104 vlan 1500 52:54:00:C0:5D:55
5 R ;;; To routeros-02
vlan204 vlan 1500 52:54:00:C0:5D:55
! Ping to netsvr-01 on IPv4 and IPv6
[admin@routeros-01] > /ping 10.100.104.254
SEQ HOST SIZE TTL TIME STATUS
0 10.100.104.254 56 64 0ms
1 10.100.104.254 56 64 0ms
sent=2 received=2 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms
[admin@routeros-01] > /ping 2001:db8:104::ffff
SEQ HOST SIZE TTL TIME STATUS
0 2001:db8:104::ffff 56 64 0ms echo reply
1 2001:db8:104::ffff 56 64 0ms echo reply
sent=2 received=2 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms
! Ping to routeros-02 on IPv4 and IPv6
[admin@routeros-01] > /ping 10.100.204.253
SEQ HOST SIZE TTL TIME STATUS
0 10.100.204.253 56 64 0ms
1 10.100.204.253 56 64 0ms
2 10.100.204.253 56 64 0ms
sent=3 received=3 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms
[admin@routeros-01] > /ping 2001:db8:204::f
SEQ HOST SIZE TTL TIME STATUS
0 2001:db8:204::f 56 64 0ms echo reply
1 2001:db8:204::f 56 64 0ms echo reply
2 2001:db8:204::f 56 64 0ms echo reply
sent=3 received=3 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms
routeros-02
! Show IPs (IPv4 and IPv6)
[admin@routeros-02] > /ip address print
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK INTERFACE
0 ;;; Management
10.15.30.54/24 10.15.30.0 ether1
1 ;;; Loopback
192.0.2.204/32 192.0.2.204 loopback0
2 ;;; To routeros-01
10.100.204.253/24 10.100.204.0 vlan204
[admin@routeros-02] > /ipv6 address print
Flags: X - disabled, I - invalid, D - dynamic, G - global, L - link-local
# ADDRESS FROM-POOL INTERFACE ADVERTISE
0 G ;;; Loopback
2001:db8:904:beef::2/128 loopback0 no
1 G ;;; To routeros-01
2001:db8:204::f/64 vlan204 yes
! Show interface statuses and descriptions
[admin@routeros-01] > /interface print
Flags: D - dynamic, X - disabled, R - running, S - slave
# NAME TYPE ACTUAL-MTU L2MTU MAX-L2MTU MAC-ADDRESS
0 R ;;; Management
ether1 ether 1500 52:54:00:CA:DA:15
1 R ;;; VLAN Bridge
ether2 ether 1500 52:54:00:C0:5D:55
2 R ;;; To the Internet
ether3 ether 1500 52:54:00:30:EC:65
3 R ;;; Loopback
loopback0 bridge 1500 65535 42:E5:10:16:E9:17
4 R ;;; To netsvr
vlan104 vlan 1500 52:54:00:C0:5D:55
5 R ;;; To routeros-02
vlan204 vlan 1500 52:54:00:C0:5D:55
! Ping to routeros-01 on IPv4 and IPv6
[admin@routeros-01] > /ping 10.100.104.254
SEQ HOST SIZE TTL TIME STATUS
0 10.100.104.254 56 64 0ms
1 10.100.104.254 56 64 0ms
sent=2 received=2 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms
[admin@routeros-01] > /ping 2001:db8:204::a
SEQ HOST SIZE TTL TIME STATUS
0 2001:db8:204::a 56 64 0ms echo reply
1 2001:db8:204::a 56 64 0ms echo reply
2 2001:db8:204::a 56 64 0ms echo reply
sent=3 received=3 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms
Firewall
The next role goes through applying firewall filters between the edge router and the netsvr-01 machine. RouterOS are well known for being capable firewall devices, with a lot of flexibility.
The firewall within RouterOS is stateful, so if you apply a rule in one direction, the return flow of traffic should also be matched by the same rule.
Playbook
The contents of the playbook are below: -
---
## tasks file for acl
- name: Firewall Address Lists - BGP
routeros_command:
commands:
- /ip firewall address-list add address="{{ item.peer }}" comment="{{ item.name }}" list="{{ item.acl }}"
when:
- rtr_role is search("edge")
- bgp is defined
- bgp.neighbors is defined
- bgp.neighbors.ipv4 is defined
- item.acl is defined
loop: "{{ bgp.neighbors.ipv4 }}"
tags:
- acl
- acl_ipv4
- name: Firewall Address Lists - BGPv6
routeros_command:
commands:
- /ipv6 firewall address-list add address="{{ item.peer }}" comment="{{ item.name }}" list="{{ item.acl }}"
when:
- rtr_role is search("edge")
- bgp is defined
- bgp.neighbors is defined
- bgp.neighbors.ipv6 is defined
- item.acl is defined
loop: "{{ bgp.neighbors.ipv6 }}"
tags:
- acl
- acl_ipv6
- name: Firewall Address Lists - Services
routeros_command:
commands:
- /ip firewall address-list add address="{{ log_host }}" comment="netsvr-01-syslog" list="syslog"
- /ip firewall address-list add address="{{ tacacs['ipv4'] }}" comment="netsvr-01-aaa" list="aaa"
when:
- rtr_role is search("edge")
tags:
- acl
- acl_ipv4
- name: Firewall Filters - IPv4
routeros_command:
commands:
- /ip firewall filter remove [find where comment~"ICMP"]
- /ip firewall filter add action=accept chain=input protocol=icmp comment="ICMP In"
- /ip firewall filter add action=accept chain=output protocol=icmp comment="ICMP Out"
- /ip firewall filter add action=accept chain=forward protocol=icmp comment="ICMP Forward"
- /ip firewall filter remove [find where comment="BGP IPv4 Peers Inbound"]
- /ip firewall filter remove [find where comment="BGP IPv4 Peers Outbound"]
- /ip firewall filter add action=accept chain=input in-interface="{{ item.routeros_if }}" src-address-list="bgp-ipv4-peers" port="179" protocol="tcp" comment="BGP IPv4 Peers Inbound"
- /ip firewall filter add action=accept chain=output out-interface="{{ item.routeros_if }}" dst-address-list="bgp-ipv4-peers" port="179" protocol="tcp" comment="BGP IPv4 Peers Outbound"
- /ip firewall filter remove [find where comment~"RADIUS"]
- /ip firewall filter remove [find where comment~"Syslog"]
- /ip firewall filter add action=accept chain=input in-interface="{{ item.routeros_if }}" src-address-list="aaa" port="1812-1813" protocol="udp" comment="RADIUS Inbound"
- /ip firewall filter add action=accept chain=output out-interface="{{ item.routeros_if }}" dst-address-list="syslog" port="514" protocol="udp" comment="Syslog Outbound"
- /ip firewall filter add action=accept chain=output out-interface="{{ item.routeros_if }}" src-address-list="aaa" port="1812-1813" protocol="udp" comment="RADIUS Outbound"
- /ip firewall filter add action=accept chain=forward in-interface="{{ item.routeros_if }}" src-address-list="aaa" port="1812-1813" protocol="udp" comment="RADIUS Forward Inbound"
- /ip firewall filter add action=accept chain=forward out-interface="{{ item.routeros_if }}" dst-address-list="syslog" port="514" protocol="udp" comment="Syslog Forward Outbound"
- /ip firewall filter add action=accept chain=forward out-interface="{{ item.routeros_if }}" dst-address-list="aaa" port="1812-1813" protocol="udp" comment="RADIUS Forward Outbound"
- /ip firewall filter remove [find where comment~"Drop all"]
- /ip firewall filter add action drop in-interface="{{ item.routeros_if }}" chain=input comment="Drop all inbound"
- /ip firewall filter add action drop out-interface="{{ item.routeros_if }}" chain=output comment="Drop all outbound"
when:
- rtr_role is search("edge")
- item.acl is defined
loop: "{{ interfaces }}"
tags:
- acl
- acl_ipv4
- name: Firewall Filters - IPv6
routeros_command:
commands:
- /ipv6 firewall filter remove [find where comment~"ICMP"]
- /ipv6 firewall filter add action=accept chain=input protocol=icmpv6 comment="ICMPv6 In"
- /ipv6 firewall filter add action=accept chain=output protocol=icmpv6 comment="ICMPv6 Out"
- /ipv6 firewall filter add action=accept chain=forward protocol=icmpv6 comment="ICMPv6 Forward"
- /ipv6 firewall filter remove [find where comment="BGP IPv6 Peers Inbound"]
- /ipv6 firewall filter remove [find where comment="BGP IPv6 Peers Outbound"]
- /ipv6 firewall filter add action=accept chain=input in-interface="{{ item.routeros_if }}" src-address-list="bgp-ipv6-peers" port="179" protocol="tcp" comment="BGP IPv6 Peers Inbound"
- /ipv6 firewall filter add action=accept chain=output out-interface="{{ item.routeros_if }}" dst-address-list="bgp-ipv6-peers" port="179" protocol="tcp" comment="BGP IPv6 Peers Outbound"
- /ipv6 firewall filter remove [find where comment~"Drop all"]
- /ipv6 firewall filter add action drop in-interface="{{ item.routeros_if }}" chain=input comment="Drop all inbound"
- /ipv6 firewall filter add action drop out-interface="{{ item.routeros_if }}" chain=output comment="Drop all outbound"
when:
- rtr_role is search("edge")
- item.acl is defined
loop: "{{ interfaces }}"
tags:
- acl
- acl_ipv6
This role is the a good example of running a set of commands from the playbook. Other than defining a few variables, most of the commands are as you would apply them yourself to RouterOS.
Firewall Address Lists - BGP
A firewall address list in RouterOS is a list of addresses (or subnets) that are logically grouped together. This allows you to apply rules to a group of addresses, negating the need to duplicate them for every host/range.
This task goes through our BGP peers. For every IPv4 BGP peer we have defined that has the acl field, it will add them to the address list named in the acl field. The relevant host_vars are below: -
bgp:
ipv4:
- peer: 10.100.104.254
acl: bgp-ipv4-peers
name: netsvr-01-v4
- peer: 192.0.2.204
name: routeros-02-v4
As we can see above, only one peer has the acl field defined. The generated configuration is therefore: -
/ip firewall address-list add address=10.100.104.254 comment=netsvr-01-v4 list=bgp-ipv4-peers
We can verify this with: -
[admin@routeros-01] > /ip firewall address-list print
Flags: X - disabled, D - dynamic
# LIST ADDRESS CREATION-TIME TIMEOUT
0 ;;; netsvr-01-v4
bgp-ipv4-peers 10.100.104.254 apr/26/2020 22:54:50
We only have one peer in here, but if you had multiple peers (say, on an Internet Exchange port) using lists makes firewall rules much simpler and easier to maintain.
Firewall Address Lists - BGP IPv6
This task is identical to the above, except it is for IPv6 BGP peers.
The relevant host_vars are below: -
bgp:
ipv6:
- peer: "2001:db8:104::ffff"
acl: bgp-ipv6-peers
name: netsvr-01-v6
- peer: "2001:db8:904:beef::2"
name: routeros-02-v6
Again, only one peer has the acl field defined. The generated configuration is therefore: -
/ipv6 firewall address-list add address=2001:db8:104::ffff/128 comment=netsvr-01-v6 list=bgp-ipv6-peers
We can verify this with: -
[admin@routeros-01] > /ipv6 firewall address-list print
Flags: X - disabled, D - dynamic
# LIST ADDRESS TIMEOUT
0 ;;; netsvr-01-v6
bgp-ipv6-peers 2001:db8:104::ffff/128
Firewall Address Lists - Services
This address list configures the Syslog and RADIUS/AAA address lists. In our scenario, RADIUS and Syslog are on the same virtual machine, but in most networks they would typically be separated.
We retrieve the Syslog and RADIUS host from our group_vars, which is: -
log_host: 10.100.104.254
tacacs:
ipv4: 192.0.2.1
Why are we using the tacacs variable for our RADIUS host? Because if and when RouterOS supports TACACS+, this would be the preferred method. Rather than changing our variables, we will keep them as generic as possible, and change the roles at a later date to accommodate TACACS+.
This generates the following configuration: -
/ip firewall address-list add address=10.100.104.254 comment=netsvr-01-syslog list=syslog
/ip firewall address-list add address=192.0.2.1 comment=netsvr-01-aaa list=aaa
We can verify this with: -
[admin@routeros-01] > /ip firewall address-list print
Flags: X - disabled, D - dynamic
# LIST ADDRESS CREATION-TIME TIMEOUT
0 ;;; netsvr-01-v4
bgp-ipv4-peers 10.100.104.254 apr/26/2020 22:54:50
1 ;;; netsvr-01-syslog
syslog 10.100.104.254 apr/26/2020 23:00:52
2 ;;; netsvr-01-aaa
aaa 192.0.2.1 apr/26/2020 23:00:52
Firewall Filters - IPv4
This task builds the firewall filters for the IPv4 traffic between the edge router and the netsvr-01 machine. To do this, it loops through our interfaces, and if they have the acl field defined, rules are created bound to that interface (as well as some general rules).
The relevant host_vars are below: -
interfaces:
- routeros_if: "ether1"
- routeros_if: "ether2"
- routeros_if: "vlan104"
acl:
ipv4:
- bgp-ipv4-peers
- syslog
- aaa
ipv6:
- bgp-ipv6-peers
- routeros_if: "vlan204"
- routeros_if: "ether3"
- routeros_if: "loopback0"
The host_vars do reference the names of filters being applied, but I did not build the logic to build the filters per interface. This is something which I will improve upon at a later date.
Also notice that we are removing rules and then reapplying them. This is so that the rules are up to date, and also so that rules do not get placed after the “drop all” rule and never take effect.
The generated configuration looks like the below: -
/ip firewall filter add action=accept chain=input comment="ICMP In" protocol=icmp
/ip firewall filter add action=accept chain=output comment="ICMP Out" protocol=icmp
/ip firewall filter add action=accept chain=forward comment="ICMP Forward" protocol=icmp
/ip firewall filter add action=accept chain=input comment="BGP IPv4 Peers Inbound" in-interface=vlan104 port=179 protocol=tcp src-address-list=bgp-ipv4-peers
/ip firewall filter add action=accept chain=output comment="BGP IPv4 Peers Outbound" dst-address-list=bgp-ipv4-peers out-interface=vlan104 port=179 protocol=tcp
/ip firewall filter add action=accept chain=input comment="RADIUS Inbound" in-interface=vlan104 port=1812-1813 protocol=udp src-address-list=aaa
/ip firewall filter add action=accept chain=output comment="Syslog Outbound" dst-address-list=syslog out-interface=vlan104 port=514 protocol=udp
/ip firewall filter add action=accept chain=output comment="RADIUS Outbound" out-interface=vlan104 port=1812-1813 protocol=udp src-address-list=aaa
/ip firewall filter add action=accept chain=forward comment="RADIUS Forward Inbound" in-interface=vlan104 port=1812-1813 protocol=udp src-address-list=aaa
/ip firewall filter add action=accept chain=forward comment="Syslog Forward Outbound" dst-address-list=syslog out-interface=vlan104 port=514 protocol=udp
/ip firewall filter add action=accept chain=forward comment="RADIUS Forward Outbound" dst-address-list=aaa out-interface=vlan104 port=1812-1813 protocol=udp
/ip firewall filter add action=drop chain=input comment="Drop all inbound" in-interface=vlan104
/ip firewall filter add action=drop chain=output comment="Drop all outbound" out-interface=vlan104
We can verify this with: -
[admin@routeros-01] > /ip firewall filter print
Flags: X - disabled, I - invalid, D - dynamic
0 ;;; ICMP In
chain=input action=accept protocol=icmp
1 ;;; ICMP Out
chain=output action=accept protocol=icmp
2 ;;; ICMP Forward
chain=forward action=accept protocol=icmp
3 ;;; BGP IPv4 Peers Inbound
chain=input action=accept protocol=tcp src-address-list=bgp-ipv4-peers in-interface=vlan104 port=179
4 ;;; BGP IPv4 Peers Outbound
chain=output action=accept protocol=tcp dst-address-list=bgp-ipv4-peers out-interface=vlan104 port=179
5 ;;; RADIUS Inbound
chain=input action=accept protocol=udp src-address-list=aaa in-interface=vlan104 port=1812-1813
6 ;;; Syslog Outbound
chain=output action=accept protocol=udp dst-address-list=syslog out-interface=vlan104 port=514
7 ;;; RADIUS Outbound
chain=output action=accept protocol=udp src-address-list=aaa out-interface=vlan104 port=1812-1813
8 ;;; RADIUS Forward Inbound
chain=forward action=accept protocol=udp src-address-list=aaa in-interface=vlan104 port=1812-1813
9 ;;; Syslog Forward Outbound
chain=forward action=accept protocol=udp dst-address-list=syslog out-interface=vlan104 port=514
10 ;;; RADIUS Forward Outbound
chain=forward action=accept protocol=udp dst-address-list=aaa out-interface=vlan104 port=1812-1813
11 ;;; Drop all inbound
chain=input action=drop in-interface=vlan104
12 ;;; Drop all outbound
chain=output action=drop out-interface=vlan104
Firewall Filters - IPv6
This task is identical to the above task, except it is for IPv6 firewall filters. The same host_vars are used too. The major difference is that we do not talk to RADIUS and Syslog over IPv6, so we do not create filters for them. The generated configuration looks like the below: -
/ipv6 firewall filter add action=accept chain=input comment="ICMPv6 In" protocol=icmpv6
/ipv6 firewall filter add action=accept chain=output comment="ICMPv6 Out" protocol=icmpv6
/ipv6 firewall filter add action=accept chain=forward comment="ICMPv6 Forward" protocol=icmpv6
/ipv6 firewall filter add action=accept chain=input comment="BGP IPv6 Peers Inbound" in-interface=vlan104 port=179 protocol=tcp src-address-list=bgp-ipv6-peers
/ipv6 firewall filter add action=accept chain=output comment="BGP IPv6 Peers Outbound" dst-address-list=bgp-ipv6-peers out-interface=vlan104 port=179 protocol=tcp
/ipv6 firewall filter add action=drop chain=input comment="Drop all inbound" in-interface=vlan104
/ipv6 firewall filter add action=drop chain=output comment="Drop all outbound" out-interface=vlan104
We can verify this with: -
[admin@routeros-01] > /ipv6 firewall filter print
Flags: X - disabled, I - invalid, D - dynamic
0 ;;; ICMPv6 In
chain=input action=accept protocol=icmpv6
1 ;;; ICMPv6 Out
chain=output action=accept protocol=icmpv6
2 ;;; ICMPv6 Forward
chain=forward action=accept protocol=icmpv6
3 ;;; BGP IPv6 Peers Inbound
chain=input action=accept protocol=tcp src-address-list=bgp-ipv6-peers in-interface=vlan104 port=179
4 ;;; BGP IPv6 Peers Outbound
chain=output action=accept protocol=tcp dst-address-list=bgp-ipv6-peers out-interface=vlan104 port=179
5 ;;; Drop all inbound
chain=input action=drop in-interface=vlan104
6 ;;; Drop all outbound
chain=output action=drop out-interface=vlan104
Verification
! Show the hit count on the policies
[admin@routeros-01] > /ip firewall filter print stats
Flags: X - disabled, I - invalid, D - dynamic
# CHAIN ACTION BYTES PACKETS
0 ;;; ICMP In
input accept 0 0
1 ;;; ICMP Out
output accept 0 0
2 ;;; ICMP Forward
forward accept 159 1
3 ;;; BGP IPv4 Peers Inbound
input accept 790 13
4 ;;; BGP IPv4 Peers Outbound
output accept 809 13
5 ;;; RADIUS Inbound
input accept 0 0
6 ;;; Syslog Outbound
output accept 1 270 15
7 ;;; RADIUS Outbound
output accept 0 0
8 ;;; RADIUS Forward Inbound
forward accept 386 3
9 ;;; Syslog Forward Outbound
forward accept 424 4
10 ;;; RADIUS Forward Outbound
forward accept 820 5
11 ;;; Drop all inbound
input drop 0 0
12 ;;; Drop all outbound
output drop 0 0
[admin@routeros-01] > /ipv6 firewall filter print stats
Flags: X - disabled, I - invalid, D - dynamic
# CHAIN ACTION BYTES PACKETS
0 ;;; ICMPv6 In
input accept 2 504 40
1 ;;; ICMPv6 Out
output accept 2 080 29
2 ;;; ICMPv6 Forward
forward accept 0 0
3 ;;; BGP IPv6 Peers Inbound
input accept 1 122 14
4 ;;; BGP IPv6 Peers Outbound
output accept 1 160 14
5 ;;; Drop all inbound
input drop 0 0
6 ;;; Drop all outbound
output drop 0 0
! What happens if we try to SSH from the netsvr?
[stuh84@netsvr-01 ~] $ ssh [email protected]
! Can we still ping it?
[stuh84@netsvr-01 ~] $ ping 10.100.104.253
PING 10.100.104.253 (10.100.104.253) 56(84) bytes of data.
64 bytes from 10.100.104.253: icmp_seq=1 ttl=64 time=0.591 ms
64 bytes from 10.100.104.253: icmp_seq=2 ttl=64 time=0.670 ms
64 bytes from 10.100.104.253: icmp_seq=3 ttl=64 time=0.675 ms
^C
--- 10.100.104.253 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 41ms
rtt min/avg/max/mdev = 0.591/0.645/0.675/0.043 ms
All looks good!
Routing
For routing, we are using BGP for external network connectivity, as well as OSPF for IPv4 and OSPFv3 for IPv6. Unlike other vendors that have IPv4 support for OSPFv3, MikroTik only support IPv6 routes and addressing in OSPFv3. This is why I have chosen to configure both in each article in this series, so that those using mixed-vendor networks can integrate them together without needing to reconfigure their core routing protocols.
Main Playbook
As with the others in this series, the main playbook is used to include other playbooks: -
---
## tasks file for routing
##
- name: Include OSPF routing
include: ospf.yml
- name: Include OSPFv3 routing
include: ospfv3.yml
- name: Include BGP routing
include: bgp.yml
OSPF Playbook
The contents of the OSPF Playbook are below: -
---
## tasks file for routing
##
- name: OSPF Process - Router ID
routeros_command:
commands:
- /routing ospf instance set 0 router-id="{{ router_id }}"
tags:
- ospf
- name: OSPF Interfaces - Networks
routeros_command:
commands:
- /routing ospf network add network="{{ item.ipv4 | ipaddr('network/prefix') }}" area=backbone
when: item.ospf is defined
loop: "{{ interfaces }}"
tags:
- ospf
- name: OSPF Interfaces - Passive
routeros_command:
commands:
- /routing ospf interface add passive=yes interface="{{ item.routeros_if }}"
when:
- item.ospf is defined
- item.ospf.passive is defined
loop: "{{ interfaces }}"
tags:
- ospf
- name: OSPF Interfaces - Non-passive
routeros_command:
commands:
- /routing ospf interface set [find where interface="{{ item.routeros_if }}"] passive=no
when:
- item.ospf is defined
- item.ospf.passive is not defined
loop: "{{ interfaces }}"
tags:
- ospf
## Remove undefined networks
- name: Retrieve configured OSPF networks
routeros_command:
commands:
- /routing ospf network print
register: ospf_network_result
tags:
- ospf
- name: Generate list of configured OSPF networks
set_fact:
ospf_configured_networks: "{{ ospf_configured_networks|default([]) + [ item | regex_findall('\\b(?:[0-9]{1,3}\\.){3}[0-9]{1,3}/[1-3]{0,1}[0-9]\\b') ] }}"
loop: "{{ ospf_network_result.stdout_lines[0] | reject('search', 'Flags') | reject('search', 'AREA') | list }}"
tags:
- ospf
- name: Generate list of defined OSPF networks
set_fact:
ospf_defined_networks: "{{ ospf_defined_networks|default([]) + [ item.ipv4 | ipaddr('network/prefix') ] }}"
when: item.ospf is defined
loop: "{{ interfaces }}"
tags:
- ospf
- name: Create list of networks to delete
set_fact:
ospf_delete_networks: "{{ ospf_delete_networks|default([]) + [ item ] }}"
when: ospf_defined_networks is not search(item)
loop: "{{ ospf_configured_networks | flatten }}"
tags:
- ospf
- name: Delete undefined networks
routeros_command:
commands:
- /routing ospf network remove [find where network="{{ item }}"]
when: ospf_delete_networks is defined
loop: "{{ ospf_delete_networks }}"
tags:
- ospf
## Remove undefined interfaces
- name: Gather facts from Router
routeros_facts:
gather_subset:
- interfaces
register: mikro_facts
tags:
- ospf
- name: Find configured interfaces
set_fact:
configured_interfaces: "{{ configured_interfaces| default([]) + [ item.key ] }}"
loop: "{{ lookup('dict', mikro_facts.ansible_facts.ansible_net_interfaces) }}"
tags:
- ospf
- name: List of defined OSPF interfaces
set_fact:
ospf_interfaces: "{{ ospf_interfaces| default([]) + [ item.routeros_if ]}}"
when:
- item.ospf is defined
loop: "{{ interfaces }}"
tags:
- ospf
- name: List of non-OSPF interfaces
set_fact:
non_ospf_interfaces: "{{ non_ospf_interfaces|default([]) + [ item ] }}"
when:
- ospf_interfaces is not search(item)
loop: "{{ configured_interfaces }}"
tags:
- ospf
- name: Remove OSPF from non-OSPF interfaces
routeros_command:
commands:
- /routing ospf interface remove [find where interface="{{ item }}"]
loop: "{{ non_ospf_interfaces }}"
tags:
- ospf
Compared to some of the other vendors, there are significantly more tasks in this playbook. Again, because no modules other than routeros_command exist for RouterOS, any idempotence (i.e. repeatable configuration) is achieved through RouterOS CLI features and commands.
OSPF Router ID
In this task, we set the OSPF Router ID. This is so that the router has a static (and predictable) Router ID. The importance of this is that changing the Router ID of an OSPF process can, in some cases, reset OSPF neighbourships. Router IDs are usually determined by the highest number IP address on an active interface. If the interface with that IP becomes inactive, the Router ID could change.
The host_vars we use for this are: -
router_id: 192.0.2.104
This generates the following configuration: -
/routing ospf instance set [ find default=yes ] router-id=192.0.2.104
We can verify this with: -
[admin@routeros-01] > /routing ospf instance print value-list
name: default
router-id: 192.0.2.104
OSPF Interfaces - Networks
This task goes through our list of interfaces, and for any that have the OSPF field, we take their IPv4 address, convert it into a network address (i.e. the first address in the subnet) and then add it to OSPF. Unlike BGP, networks in OSPF are used to include interfaces in your OSPF routing domain, rather than generating routes for advertisement. For example, you could have a network statement of 0.0.0.0/0 which would include all of your interfaces (because this covers every subnet), but it would not advertise out 0.0.0.0/0 as a route.
The relevant host_vars for this task are: -
interfaces:
- routeros_if: "ether1"
- routeros_if: "ether2"
- routeros_if: "vlan104"
ipv4: "10.100.104.253/24"
ospf:
area: "0.0.0.0"
- routeros_if: "vlan204"
ipv4: "10.100.204.254/24"
ospf:
area: "0.0.0.0"
- routeros_if: "ether3"
ipv4: "dhcp"
- routeros_if: "loopback0"
ipv4: "192.0.2.104/32"
ospf:
area: "0.0.0.0"
In the above, the interfaces that have an OSPF field are vlan104, vlan204 and loopback0. Therefore the configuration that is generated is: -
/routing ospf network add area=backbone network=10.100.104.0/24
/routing ospf network add area=backbone network=10.100.204.0/24
/routing ospf network add area=backbone network=192.0.2.104/32
We can verify this with: -
[admin@routeros-01] > /routing ospf network print
Flags: X - disabled, I - invalid
# NETWORK AREA
0 10.100.104.0/24 backbone
1 10.100.204.0/24 backbone
2 192.0.2.104/32 backbone
The backbone area is the default OSPF area (known as either area 0 in IOS or area 0.0.0.0 on most other vendors). This task does not configure interfaces outside of the backbone area, as for our purposes we are not using them.
OSPF Interfaces - Passive
This task configures which interfaces will run in passive mode (i.e. the subnet they have is advertised, but no OSPF neighbours will be negotiated over this interface). The relevant host_vars for this task are: -
interfaces:
- routeros_if: "vlan104"
ipv4: "10.100.104.253/24"
ospf:
area: "0.0.0.0"
passive: true
- routeros_if: "vlan204"
ipv4: "10.100.204.254/24"
ospf:
area: "0.0.0.0"
- routeros_if: "loopback0"
ipv4: "192.0.2.104/32"
ospf:
area: "0.0.0.0"
passive: true
This will generated the following configuration: -
/routing ospf interface add interface=loopback0 passive=yes
/routing ospf interface add interface=vlan104 passive=yes
We can verify this with: -
[admin@routeros-01] > /routing ospf interface print
Flags: X - disabled, I - inactive, D - dynamic, P - passive
# INTERFACE COST PRIORITY NETWORK-TYPE AUTHENTICATION AUTHENTICATION-KEY
0 P loopback0 10 1 default none
1 P vlan104 10 1 default none
OSPF Interfaces - Non-Passive
This task does the same as the above, except that it configures all interfaces that are not passive. This is done because if we change an interface from passive to non-passive, we cannot just remove the passive declaration. Instead it must be set explicitly.
This does mean that all interfaces running OSPF will have a declaration in /routing ospf interfaces, even though technically not all are required. Without this, we cannot change the passive state of an interface.
The same host_vars from the previous task apply, which generate the following configuration: -
/routing ospf interface add interface=vlan204 passive=no
We can verify this with: -
[admin@routeros-01] > /routing ospf interface print where passive=no
Flags: X - disabled, I - inactive, D - dynamic, P - passive
# INTERFACE COST PRIORITY NETWORK-TYPE AUTHENTICATION AUTHENTICATION-KEY
0 vlan204 10 1 default none
Removing Undefined Networks
To remove undefined networks from MikroTiks, we use a series of tasks to gather information from our host_vars and the running configuration. The two are then compared. Any network statements that are configured on the router but not defined in our host_vars are removed.
Retrieve configured OSPF networks
The first task runs /routing ospf network print and stores it in a variable called ospf_network_result. The output of /routing ospf network print looks something like the below: -
[admin@routeros-01] > /routing ospf network print
Flags: X - disabled, I - invalid
# NETWORK AREA
0 10.100.104.0/24 backbone
1 10.100.204.0/24 backbone
2 192.0.2.104/32 backbone
3 169.254.1.1/32 backbone
For demonstration purposes I have added the 169.254.1.1/32 network, which does not exist in our host_vars.
Generate list of configured OSPF networks
Firstly, this task generates a list for us to loop through, by doing the following: -
- Looks at the output of
ospf_network_result, looking for the fieldstdout_lines, retrieving only the first value in the list - If any line contains the word flags, we ignore it (i.e. the
Flags: X - disabled, I - invalidline) - If any line contains the word area, we also ignore it (i.e. the
# NETWORK AREAline) - Converts the results to a list
Once the above is complete, we loop through the results, and do the following on each iteration: -
- Create the list
ospf_configured_networkswith a default value of[](i.e. an empty list), ifospf_configured_networksdoes not already exist - If it does already exist, then retrieve the list from the previous iteration of the loop
- Run a regular expression to find an IPv4 network address in each item in our loop, and add it to the
ospf_configured_networkslist
If we do not supply a default value for the ospf_configured_networks, it will not exist on the first iteration of the loop. This will cause the task will fail. Subsequent iterations will not need a default value, as the list already exists.
All of the above doesn’t make a lot of sense without seeing it in action, so the below shows what happens at each state: -
Contents the ospf_network_result variable
ok: [routeros-01] => {
"ospf_network_result": {
"changed": false,
"failed": false,
"stdout": [
"Flags: X - disabled, I - invalid \n # NETWORK AREA \n 0 10.100.104.0/24 backbone \n 1 10.100.204.0/24 backbone \n 2 192.0.2.104/32 backbone \n 3 169.254.1.1/32 backbone"
],
"stdout_lines": [
[
"Flags: X - disabled, I - invalid ",
" # NETWORK AREA ",
" 0 10.100.104.0/24 backbone ",
" 1 10.100.204.0/24 backbone ",
" 2 192.0.2.104/32 backbone ",
" 3 169.254.1.1/32 backbone"
]
]
}
}
Retrieving the stdout_lines section
ok: [routeros-01] => {
"ospf_network_result.stdout_lines": [
[
"Flags: X - disabled, I - invalid ",
" # NETWORK AREA ",
" 0 10.100.104.0/24 backbone ",
" 1 10.100.204.0/24 backbone ",
" 2 192.0.2.104/32 backbone ",
" 3 169.254.1.1/32 backbone"
]
]
}
Notice in the above we have a list within a list. We do not require a nested list here, so we retrieve the first element of it
Retrieve first element in the list
ok: [routeros-01] => {
"ospf_network_result.stdout_lines[0]": [
"Flags: X - disabled, I - invalid ",
" # NETWORK AREA ",
" 0 10.100.104.0/24 backbone ",
" 1 10.100.204.0/24 backbone ",
" 2 192.0.2.104/32 backbone ",
" 3 169.254.1.1/32 backbone"
]
}
Ignore lines with the word Flags
ok: [routeros-01] => {
"ospf_network_result.stdout_lines[0] | reject('search', 'Flags') | list": [
" # NETWORK AREA ",
" 0 10.100.104.0/24 backbone ",
" 1 10.100.204.0/24 backbone ",
" 2 192.0.2.104/32 backbone ",
" 3 169.254.1.1/32 backbone"
]
}
Ignore lines with the word AREA
ok: [routeros-01] => {
"ospf_network_result.stdout_lines[0] | reject('search', 'Flags') | reject('search', 'AREA') | list": [
" 0 10.100.104.0/24 backbone ",
" 1 10.100.204.0/24 backbone ",
" 2 192.0.2.104/32 backbone ",
" 3 169.254.1.1/32 backbone"
]
}
Our regular expression that matches the networks (in CIDR format) is \\b(?:[0-9]{1,3}\\.){3}[0-9]{1,3}/[1-3]{0,1}[0-9]\\b. To explain what this does: -
\\b- Whole words/strings only, not a match within a word- The
(?:and)section creates a non-capturing group, allowing us to repeat the expression inside it without needing to necessarily “save” the results- A capturing group could be used to replace sections later if required, but we do not need this behaviour
- The
[0-9]{1,3}section says that we are looking for numbers between 0-9 between 1 and 3 times. This will match up to 255 - Technically this could also match up to 999, but IP addresses do not go that high!
- The
{3}matches the previous expression exactly 3 times - The second use of
[0-9]{1,3}matches the last octet of an IP address - The
/[1-3]{0,1}[0-9]matches the subnet mask, saying that the we could have a number from 1 to 3, either zero or once, and any number from 0 to 9- This is because subnet masks can be anything from
/0to/32
- This is because subnet masks can be anything from
Thankfully, this expression was already available within the Ansible Regular Expression Filters documentation. Without it, I would have probably used something much simpler (and potentially less likely to match the values!).
To sum up, this regular expression extracts only the subnet addresses that are in the list generated when we loop through our ospf_network_result. While we could run this without ignoring the lines containing AREA or Flags, this would add empty values to our list, which are unnecessary and could lead to commands failing later with empty values.
The list generated by this task is: -
ok: [routeros-01] => {
"ospf_configured_networks": [
[
"10.100.104.0/24"
],
[
"10.100.204.0/24"
],
[
"192.0.2.104/32"
],
[
"169.254.1.1/32"
]
]
}
Gathering list of defined networks
This task loops through our host_vars and generates a list of the OSPF networks we have defined. This would then generate: -
ok: [routeros-01] => {
"ospf_defined_networks": [
"10.100.104.0/24",
"10.100.204.0/24",
"192.0.2.104/32"
]
}
Create list of networks to delete
First, this task takes our list of networks that are configured on the router and flattens it. Flattening a list means that rather than having lists inside of lists (nested list), every value is part one combined list instead. This is required because the task to generate the ospf_configured_networks variable created a list of lists.
We then loop through the results of the flattened list. For each entry, we check to see if it is in the ospf_defined_networks list. If it isn’t, we add it to a variable called ospf_delete_networks.
The output of this is therefore: -
ok: [routeros-01] => {
"ospf_delete_networks": [
"169.254.1.1/32"
]
}
Delete the networks
Our final part of this section is removing the non-defined networks. This takes our ospf_delete_networks list, loops through it, and runs /routing ospf network remove [find where network="{{ item }}"], {{ item }} being the OSPF networks found in our ospf_delete_networks list.
Before running this, we see this in our OSPF networks list: -
[admin@routeros-01] > /routing ospf network print
Flags: X - disabled, I - invalid
# NETWORK AREA
0 10.100.104.0/24 backbone
1 10.100.204.0/24 backbone
2 192.0.2.104/32 backbone
3 169.254.1.1/32 backbone
After this task, we see: -
[admin@routeros-01] > /routing ospf network print
Flags: X - disabled, I - invalid
# NETWORK AREA
0 10.100.104.0/24 backbone
1 10.100.204.0/24 backbone
2 192.0.2.104/32 backbone
This task will only run if the ospf_delete_networks variable exists. If we tried to run it regardless, the task would fail if no undefined networks were configured.
Removing undefined interfaces
Similar to removing undefined OSPF networks, this requires a set of tasks, rather than a single task.
Gathering facts from the router
The first task gathers facts (i.e. details about the devices) from the routers. You could use this to discover the version of RouterOS running, the hostname, or more usefully in our case, the list of interfaces (physical and logical) that exist on the router.
The below shows what we retrieve from a RouterOS device when using routeros_facts. We use the gather_subset option to limit to the interface facts: -
ok: [routeros-01] => {
"mikro_facts": {
"ansible_facts": {
"ansible_net_all_ipv4_addresses": [
"10.15.30.53",
"192.0.2.104",
"10.100.104.253",
"10.100.204.254",
"192.168.122.208"
],
"ansible_net_all_ipv6_addresses": [
"fe80::5054:ff:fec0:5d55",
"fe80::5054:ff:fec0:5d55",
"fe80::308a:89ff:fe50:7b25",
"fe80::5054:ff:feca:da15",
"fe80::5054:ff:fec0:5d55",
"fe80::5054:ff:fe30:ec65"
],
"ansible_net_gather_subset": [
"default",
"interfaces"
],
"ansible_net_hostname": "routeros-01",
"ansible_net_interfaces": {
"ether1": {
"actual-mtu": "1500",
"default-name": "ether1",
"ipv4": [
{
"address": "10.15.30.53",
"subnet": "24"
}
],
"ipv6": [
{
"address": "fe80::5054:ff:feca:da15",
"subnet": "64"
}
],
"last-link-up-time": "may/17/2020 06:03:35",
"link-downs": "0",
"mac-address": "52:54:00:CA:DA:15",
"mtu": "1500",
"name": "ether1",
"type": "ether"
},
"ether2": {
"actual-mtu": "1500",
"default-name": "ether2",
"ipv6": [
{
"address": "fe80::5054:ff:fec0:5d55",
"subnet": "64"
}
],
"last-link-up-time": "may/17/2020 06:03:35",
"link-downs": "0",
"mac-address": "52:54:00:C0:5D:55",
"mtu": "1500",
"name": "ether2",
"type": "ether"
},
"ether3": {
"actual-mtu": "1500",
"default-name": "ether3",
"ipv4": [
{
"address": "192.168.122.208",
"subnet": "24"
}
],
"ipv6": [
{
"address": "fe80::5054:ff:fe30:ec65",
"subnet": "64"
}
],
"last-link-up-time": "may/17/2020 06:03:35",
"link-downs": "0",
"mac-address": "52:54:00:30:EC:65",
"mtu": "1500",
"name": "ether3",
"type": "ether"
},
"loopback0": {
"actual-mtu": "1500",
"ipv4": [
{
"address": "192.0.2.104",
"subnet": "32"
}
],
"ipv6": [
{
"address": "fe80::308a:89ff:fe50:7b25",
"subnet": "64"
}
],
"l2mtu": "65535",
"last-link-up-time": "may/17/2020 06:03:25",
"link-downs": "0",
"mac-address": "32:8A:89:50:7B:25",
"mtu": "auto",
"name": "loopback0",
"type": "bridge"
},
"vlan104": {
"actual-mtu": "1500",
"ipv4": [
{
"address": "10.100.104.253",
"subnet": "24"
}
],
"ipv6": [
{
"address": "fe80::5054:ff:fec0:5d55",
"subnet": "64"
}
],
"last-link-up-time": "may/17/2020 06:03:35",
"link-downs": "0",
"mac-address": "52:54:00:C0:5D:55",
"mtu": "1500",
"name": "vlan104",
"type": "vlan"
},
"vlan204": {
"actual-mtu": "1500",
"ipv4": [
{
"address": "10.100.204.254",
"subnet": "24"
}
],
"ipv6": [
{
"address": "fe80::5054:ff:fec0:5d55",
"subnet": "64"
}
],
"last-link-up-time": "may/17/2020 06:03:35",
"link-downs": "0",
"mac-address": "52:54:00:C0:5D:55",
"mtu": "1500",
"name": "vlan204",
"type": "vlan"
}
},
"ansible_net_model": null,
"ansible_net_neighbors": {
"ether1": {
"address": "10.15.30.54",
"address4": "10.15.30.54",
"address6": "fe80::5054:ff:fe09:3b0",
"age": "5s",
"board": "CHR",
"identity": "routeros-02",
"interface": "ether1",
"interface-name": "ether1",
"ipv6": "yes",
"mac-address": "52:54:00:09:03:B0",
"platform": "MikroTik",
"software-id": "OKX",
"unpack": "none",
"uptime": "24m8s",
"version": "6.45.8"
},
"ether2": {
"address": "fe80::5054:ff:fefc:cd86",
"address6": "fe80::5054:ff:fefc:cd86",
"age": "5s",
"board": "CHR",
"identity": "routeros-02",
"interface": "ether2",
"interface-name": "ether2",
"ipv6": "yes",
"mac-address": "52:54:00:FC:CD:86",
"platform": "MikroTik",
"software-id": "OKX",
"unpack": "none",
"uptime": "24m8s",
"version": "6.45.8"
},
"vlan204": {
"address": "10.100.204.253",
"address4": "10.100.204.253",
"address6": "2001:db8:204::f",
"age": "5s",
"board": "CHR",
"identity": "routeros-02",
"interface": "vlan204",
"interface-name": "vlan204",
"ipv6": "yes",
"mac-address": "52:54:00:FC:CD:86",
"platform": "MikroTik",
"software-id": "OKX",
"unpack": "none",
"uptime": "24m8s",
"version": "6.45.8"
}
},
"ansible_net_serialnum": null,
"ansible_net_version": "6.45.8 (long-term)"
},
"changed": false,
"failed": false
}
}
As you can see, we have a lot of information returned. We do not need all of this information, but it does mean we do not need to do any regular expressions or parsing to discover what interfaces exist.
Find configured interfaces
This task goes through the facts we just retrieved. Rather than looping through a list, we use it as a dictionary (i.e. key/values). We can then retrieve all the keys from the ansible_net_interfaces section. We then generate a list of the keys, which looks like: -
ok: [routeros-01] => {
"configured_interfaces": [
"ether1",
"ether2",
"ether3",
"loopback0",
"vlan104",
"vlan204"
]
}
Defined OSPF interfaces
This task goes through our host_vars, and generates a list of interfaces that have the ospf field. This results in: -
ok: [routeros-01] => {
"ospf_interfaces": [
"vlan104",
"vlan204",
"loopback0"
]
}
List of non-ospf interfaces
To generate the list of interfaces that OSPF should not be running on, we do similar to what we did in the OSPF network tasks. We loop through our list of interfaces gathered from the routeros_facts module and check if each item appears in the ospf_interfaces variable. If it doesn’t, we add it to non_ospf_interfaces.
This generates the following: -
ok: [routeros-01] => {
"non_ospf_interfaces": [
"ether1",
"ether2",
"ether3"
]
}
Remove OSPF from non-OSPF interfaces
This task loops through our non_ospf_interfaces variable. For each item, it runs /routing ospf interface remove [find where interface="{{ item }}"].
In the above, we would run the following: -
/routing ospf interface remove [find where interface="ether1"]
/routing ospf interface remove [find where interface="ether2"]
/routing ospf interface remove [find where interface="ether3"]
Verification
After this, we should be able to see OSPF routes on both the edge router and the internal router: -
routeros-01
! Show OSPF interfaces
[admin@routeros-01] > /routing ospf interface print
Flags: X - disabled, I - inactive, D - dynamic, P - passive
# INTERFACE COST PRIORITY NETWORK-TYPE AUTHENTICATION AUTHENTICATION-KEY
0 P loopback0 10 1 default none
1 P vlan104 10 1 default none
2 vlan204 10 1 default none
! Show OSPF neighbours
[admin@routeros-01] > /routing ospf neighbor print brief
# ROUTER-ID ADDRESS STATE STATE-CHANGES
0 192.0.2.204 10.100.204.253 Full 6
! Show routing table
[admin@routeros-01] > /ip route print where ospf
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 ADo 192.0.2.204/32 10.100.204.253 110
! Can we ping?
[admin@routeros-01] > /ping 192.0.2.204
SEQ HOST SIZE TTL TIME STATUS
0 192.0.2.204 56 64 0ms
1 192.0.2.204 56 64 0ms
sent=2 received=2 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms
routeros-02
! Show OSPF interfaces
[admin@routeros-02] > /routing ospf interface print
Flags: X - disabled, I - inactive, D - dynamic, P - passive
# INTERFACE COST PRIORITY NETWORK-TYPE AUTHENTICATION AUTHENTICATION-KEY
0 P loopback0 10 1 default none
1 vlan204 10 1 default none
! Show OSPF neighbours
[admin@routeros-02] > /routing ospf neighbor print brief
# ROUTER-ID ADDRESS STATE STATE-CHANGES
0 192.0.2.104 10.100.204.254 Full 5
! Show routing table
[admin@routeros-02] > /ip route print where ospf
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 ADo 10.100.104.0/24 10.100.204.254 110
1 ADo 192.0.2.104/32 10.100.204.254 110
! Can we ping
[admin@routeros-02] > /ping 192.0.2.104
SEQ HOST SIZE TTL TIME STATUS
0 192.0.2.104 56 64 0ms
1 192.0.2.104 56 64 0ms
sent=2 received=2 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms
[admin@routeros-02] > /ping 10.100.104.253
SEQ HOST SIZE TTL TIME STATUS
0 10.100.104.253 56 64 0ms
sent=1 received=1 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms
All looking good!
OSPFv3 Playbook
The contents of the OSPFv3 Playbook are below: -
---
## tasks file for routing
##
- name: OSPFv3 Process - Router ID
routeros_command:
commands:
- /routing ospf-v3 instance set 0 router-id="{{ router_id }}"
tags:
- ospfv3
- name: OSPFv3 Interfaces
routeros_command:
commands:
- /routing ospf-v3 interface add interface="{{ item.routeros_if }}" area=backbone
when: item.ospfv3 is defined
loop: "{{ interfaces }}"
tags:
- ospfv3
- name: OSPFv3 Interfaces - Passive
routeros_command:
commands:
- /routing ospf-v3 interface set passive=yes [find where interface="{{ item.routeros_if }}"]
when:
- item.ospfv3 is defined
- item.ospfv3.passive is defined
loop: "{{ interfaces }}"
tags:
- ospfv3
- name: OSPFv3 Interfaces - Non-Passive
routeros_command:
commands:
- /routing ospf-v3 interface set passive=no [find where interface="{{ item.routeros_if }}"]
when:
- item.ospfv3 is defined
- item.ospfv3.passive is not defined
loop: "{{ interfaces }}"
tags:
- ospfv3
- name: Remove Inactive Interface
routeros_command:
commands:
- /router ospf-v3 interface remove [find where inactive]
tags:
- ospfv3
## Remove undefined interfaces
- name: Gather facts from Router
routeros_facts:
gather_subset:
- interfaces
register: mikro_facts
tags:
- ospfv3
- name: Find configured interfaces
set_fact:
configured_interfaces: "{{ configured_interfaces| default([]) + [ item.key ] }}"
loop: "{{ lookup('dict', mikro_facts.ansible_facts.ansible_net_interfaces) }}"
tags:
- ospfv3
- name: List of defined OSPFv3 interfaces
set_fact:
ospfv3_interfaces: "{{ ospfv3_interfaces| default([]) + [ item.routeros_if ]}}"
when:
- item.ospfv3 is defined
loop: "{{ interfaces }}"
tags:
- ospfv3
- name: List of non-OSPFv3 interfaces
set_fact:
non_ospfv3_interfaces: "{{ non_ospfv3_interfaces|default([]) + [ item ] }}"
when:
- ospfv3_interfaces is not search(item)
loop: "{{ configured_interfaces }}"
tags:
- ospfv3
- name: Remove OSPFv3 from non-OSPFv3 interfaces
routeros_command:
commands:
- /routing ospf-v3 interface remove [find where interface="{{ item }}"]
loop: "{{ non_ospfv3_interfaces }}"
tags:
- ospfv3
This task is very similar to the OSPF playbook, except: -
- OSPFv3 does not use network statements in RouterOS
- We need to remove inactive interfaces
- The word
ospf-v3is used instead ofospf
Removing inactive interfaces is used because sometimes when adding an ospf-v3 interface, it can add configuration identical to what already exists, and then remove the interface from the existing configuration. This task removes any inactive (i.e. potentially broken) configuration.
[admin@routeros-02] /routing ospf-v3 interface> print
Flags: X - disabled, I - inactive, D - dynamic, P - passive
# INTERFACE AREA COST PRIORITY NETWORK-TYPE
0 I *E backbone 10 1 default
1 P loopback0 backbone 10 1 default
2 I *F backbone 10 1 default
3 vlan204 backbone 10 1 default
As we can see in the above, two of the interfaces are inactive. The task above then removes them, so that we are only left with: -
[admin@routeros-02] /routing ospf-v3 interface> print
Flags: X - disabled, I - inactive, D - dynamic, P - passive
# INTERFACE AREA COST PRIORITY NETWORK-TYPE
0 P loopback0 backbone 10 1 default
1 vlan204 backbone 10 1 default
Other than that, this set of tasks takes the same approach as we would for OSPF.
##### Verification
We’ll follow the same steps as we did for OSPF: -
routeros-01
! Show OSPFv3 interfaces
[admin@routeros-01] > /routing ospf-v3 interface print
Flags: X - disabled, I - inactive, D - dynamic, P - passive
# INTERFACE AREA COST PRIORITY NETWORK-TYPE
0 P loopback0 backbone 10 1 default
1 P vlan104 backbone 10 1 default
2 vlan204 backbone 10 1 default
! Show OSPFv3 neighbours
[admin@routeros-01] > /routing ospf-v3 neighbor print brief
# ROUTER-ID ADDRESS STATE STATE-CHANGES
0 192.0.2.204 fe80::5054:ff:fefc:cd86 Full 6
! Show routing table
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, o - ospf, b - bgp, U - unreachable
# DST-ADDRESS GATEWAY DISTANCE
0 ADo 2001:db8:904:beef::2/128 fe80::5054:ff:fefc:cd... 110
! Ping!
[admin@routeros-01] > /ping 2001:db8:904:beef::2
SEQ HOST SIZE TTL TIME STATUS
0 2001:db8:904:beef::2 56 64 1ms echo reply
1 2001:db8:904:beef::2 56 64 0ms echo reply
sent=2 received=2 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=1ms
routeros-02
! Show OSPFv3 interfaces
[admin@routeros-02] /routing ospf-v3 interface> print
Flags: X - disabled, I - inactive, D - dynamic, P - passive
# INTERFACE AREA COST PRIORITY NETWORK-TYPE
0 P loopback0 backbone 10 1 default
1 vlan204 backbone 10 1 default
! Show OSPFv3 neighbours
[admin@routeros-02] > /routing ospf-v3 neighbor print brief
# ROUTER-ID ADDRESS STATE STATE-CHANGES
0 192.0.2.104 fe80::5054:ff:fec0:5d55 Full 5
! Show routing table
[admin@routeros-02] > /ipv6 route print where ospf
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, o - ospf, b - bgp, U - unreachable
# DST-ADDRESS GATEWAY DISTANCE
0 ADo 2001:db8:104::/64 fe80::5054:ff:fec0:5d... 110
1 ADo 2001:db8:904:beef::1/128 fe80::5054:ff:fec0:5d... 110
! Ping!
[admin@routeros-02] > /ping 2001:db8:904:beef::1
SEQ HOST SIZE TTL TIME STATUS
0 2001:db8:904:beef::1 56 64 1ms echo reply
1 2001:db8:904:beef::1 56 64 0ms echo reply
sent=2 received=2 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=1ms
[admin@routeros-02] > /ping 2001:db8:104::f
SEQ HOST SIZE TTL TIME STATUS
0 2001:db8:104::f 56 64 0ms echo reply
1 2001:db8:104::f 56 64 0ms echo reply
2 2001:db8:104::f 56 64 0ms echo reply
sent=3 received=3 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms
All looks good!
BGP Playbook
The BGP playbook is where we configure our internal and external BGP peers. This playbook is potentially the most complex playbook in the series so far. We are not only adding peers, but also checking for the existence of peers, updating them if they exist, creating them if they do not, and also using regular expressions and command parsing to remove peers that are not defined any longer.
---
## tasks file for routing
##
##
- name: Configure BGP - Instance details
routeros_command:
commands:
- /routing bgp instance set 0 as="{{ bgp.local_as }}" router-id="{{ router_id }}"
tags:
- bgp
- name: Retrieve current BGP peers
routeros_command:
commands:
- /routing bgp peer print
register: bgp_peer_result
no_log: True
tags:
- bgp
- bgp_v4
- bgp_v6
- name: Clean the command output - BGP Peers
set_fact:
bgp_peer_result_output: "{{ bgp_peer_result_output|default([]) + [ item ] }}"
no_log: True
loop: "{{ bgp_peer_result.stdout_lines }}"
tags:
- bgp
- bgp_v4
- bgp_v6
- name: Check if the peer exists - IPv4
set_fact:
peer_exists: "{{ peer_exists|default([]) + [ item.peer ] }}"
when: bgp_peer_result_output is search(item.peer)
loop: "{{ bgp.neighbors.ipv4 }}"
tags:
- bgp
- bgp_v4
- name: Check if the peer exists - IPv6
set_fact:
peer_exists: "{{ peer_exists|default([]) + [ item.peer ] }}"
when: bgp_peer_result_output is search(item.peer)
loop: "{{ bgp.neighbors.ipv6 }}"
tags:
- bgp
- bgp_v6
- name: Configure BGP - Add eBGP v4 peers if they are not configured
routeros_command:
commands:
- /routing bgp peer add remote-as="{{ item.remote_as }}" remote-address="{{ item.peer }}" name="{{ item.name }}"
when:
- bgp is defined
- bgp.neighbors is defined
- bgp.neighbors.ipv4 is defined
- item.ebgp is defined
- peer_exists is not defined or peer_exists is not search(item.peer)
loop: "{{ bgp.neighbors.ipv4 }}"
tags:
- bgp
- bgp_v4
- name: Configure BGP - Update eBGP v4 peers if they are configured
routeros_command:
commands:
- /routing bgp peer set [find where remote-address="{{ item.peer }}"] remote-as="{{ item.remote_as }}" name="{{ item.name }}"
when:
- bgp is defined
- bgp.neighbors is defined
- bgp.neighbors.ipv4 is defined
- item.ebgp is defined
- peer_exists is defined
- peer_exists is search(item.peer)
loop: "{{ bgp.neighbors.ipv4 }}"
tags:
- bgp
- bgp_v4
- name: Configure BGP - Add eBGP v6 peers if they are not configured
routeros_command:
commands:
- /routing bgp peer add remote-as="{{ item.remote_as }}" remote-address="{{ item.peer }}" address-families="ipv6" name="{{ item.name }}"
when:
- bgp is defined
- bgp.neighbors is defined
- bgp.neighbors.ipv6 is defined
- item.ebgp is defined
- peer_exists is not defined or peer_exists is not search(item.peer)
loop: "{{ bgp.neighbors.ipv6 }}"
tags:
- bgp
- bgp_v6
- name: Configure BGP - Update eBGP v6 peers if they are configured
routeros_command:
commands:
- /routing bgp peer set [find where remote-address="{{ item.peer }}"] remote-as="{{ item.remote_as }}" address-families="ipv6" name="{{ item.name }}"
when:
- bgp is defined
- bgp.neighbors is defined
- bgp.neighbors.ipv6 is defined
- item.ebgp is defined
- peer_exists is defined
- peer_exists is search(item.peer)
loop: "{{ bgp.neighbors.ipv6 }}"
tags:
- bgp
- bgp_v6
- name: Configure BGP - Add iBGP v4 peers if they are not configured
routeros_command:
commands:
- /routing bgp peer add remote-as="{{ item.remote_as }}" remote-address="{{ item.peer }}" update-source="{{ item.update_source }}" name="{{ item.name }}"
when:
- bgp is defined
- bgp.neighbors is defined
- bgp.neighbors.ipv4 is defined
- item.ibgp is defined
- peer_exists is not defined or peer_exists is not search(item.peer)
loop: "{{ bgp.neighbors.ipv4 }}"
tags:
- bgp
- bgp_v4
- name: Configure BGP - Update iBGP v4 peers if they are configured
routeros_command:
commands:
- /routing bgp peer set [find where remote-address="{{ item.peer }}"] remote-as="{{ item.remote_as }}" update-source="{{ item.update_source }}" name="{{ item.name }}"
when:
- bgp is defined
- bgp.neighbors is defined
- bgp.neighbors.ipv4 is defined
- item.ibgp is defined
- peer_exists is defined
- peer_exists is search(item.peer)
loop: "{{ bgp.neighbors.ipv4 }}"
tags:
- bgp
- bgp_v4
- name: Configure BGP - Add iBGP v6 peers if they are not configured
routeros_command:
commands:
- /routing bgp peer add remote-as="{{ item.remote_as }}" remote-address="{{ item.peer }}" update-source="{{ item.update_source }}" address-families="ipv6" name="{{ item.name }}"
when:
- bgp is defined
- bgp.neighbors is defined
- bgp.neighbors.ipv6 is defined
- item.ibgp is defined
- peer_exists is not defined or peer_exists is not search(item.peer)
loop: "{{ bgp.neighbors.ipv6 }}"
tags:
- bgp
- bgp_v6
- name: Configure BGP - Update iBGP v6 peers if they are configured
routeros_command:
commands:
- /routing bgp peer set [find where remote-address="{{ item.peer }}"] remote-as="{{ item.remote_as }}" update-source="{{ item.update_source }}" address-families="ipv6" name="{{ item.name }}"
when:
- bgp is defined
- bgp.neighbors is defined
- bgp.neighbors.ipv6 is defined
- item.ibgp is defined
- peer_exists is defined
- peer_exists is search(item.peer)
loop: "{{ bgp.neighbors.ipv6 }}"
tags:
- bgp
- bgp_v6
- name: Configure BGP - iBGP v4 Default Originate
routeros_command:
commands:
- /routing bgp peer set [find where remote-address="{{ item.peer }}"] default-originate=if-installed
when:
- bgp is defined
- bgp.neighbors is defined
- bgp.neighbors.ipv4 is defined
- item.ibgp is defined
- item.default_originate is defined
loop: "{{ bgp.neighbors.ipv4 }}"
tags:
- bgp
- bgp_v4
- name: Configure BGP - Redistribute OSPF, connected and static
routeros_command:
commands:
- /routing bgp instance set 0 redistribute-ospf=yes redistribute-connected=yes redistribute-static=yes
when:
- bgp is defined
- bgp.redist is defined
- bgp.redist.ospf is defined
tags:
- bgp
- bgp_v4
- bgp_v6
#### Remove undefined peers
- name: Retrieve IPs of configured peers - IPv4
set_fact:
v4_configured_peers: "{{ v4_configured_peers|default([]) + [ item | regex_findall('\\b(?:[0-9]{1,3}\\.){3}[0-9]{1,3}\\b') ] }}"
loop: "{{ bgp_peer_result.stdout_lines[0] | reject('search', 'Flags') | reject('search', 'INSTANCE') | list }}"
tags:
- bgp
- bgp_v4
- name: Retrieve IPs of configured peers - IPv6
set_fact:
v6_configured_peers: "{{ v6_configured_peers|default([]) + [ item | regex_findall('[0-9a-fA-F]{1,4}:.*:[0-9a-fA-F]{1,4}') ] }}"
loop: "{{ bgp_peer_result.stdout_lines[0] | reject('search', 'Flags') | reject('search', 'INSTANCE') | list }}"
tags:
- bgp
- bgp_v6
- name: Combine the configured peer lists
set_fact:
configured_peers: '{{ v4_configured_peers|flatten|default([]) + v6_configured_peers|flatten|default([])}}'
tags:
- bgp
- bgp_v4
- bgp_v6
- name: Create list of defined peers
set_fact:
defined_peers: "{{ defined_peers|default([]) + [ item.peer ] }}"
loop: "{{ bgp.neighbors.ipv4 + bgp.neighbors.ipv6 }}"
tags:
- bgp
- bgp_v4
- bgp_v6
- name: Create list of peers to delete
set_fact:
delete_peers: "{{ delete_peers|default([]) + [ item ] }}"
when:
- defined_peers is not search(item)
loop: "{{ configured_peers }}"
tags:
- bgp
- bgp_v4
- bgp_v6
- name: Delete the peers
routeros_command:
commands:
- /routing bgp peer remove [find where remote-address="{{ item }}"]
when: delete_peers is defined
loop: "{{ delete_peers }}"
tags:
- bgp
- bgp_v4
- bgp_v6
Setting BGP instance details
The first task updates the main BGP instance, setting the router ID and our local autonomous system number. These variables are sourced from our host_vars: -
router_id: 192.0.2.104
bgp:
local_as: 65104
This generates the following configuration: -
/routing bgp instance set default as=65104 router-id=192.0.2.104
RouterOS automatically translates the instance with an ID of 0 to default.
We can verify this with: -
[admin@routeros-01] > /routing bgp instance print
Flags: * - default, X - disabled
0 * name="default" as=65104 router-id=192.0.2.104 redistribute-connected=no redistribute-static=no redistribute-rip=no redistribute-ospf=no redistribute-other-bgp=no out-filter="" client-to-client-reflection=no
ignore-as-path-len=no routing-table=""
Building peers
The following set of tasks are used to determine whether a BGP peer is already defined. If it isn’t, it will create a new peer. If it is, it will update the existing peer. Because the syntax used to add a peer and configure an existing peer are not consistent, this is a good workaround to maintain idempotence.
Get the list of peers
First, we get our list of BGP peers. This includes both IPv4 and IPv6 peers. We also use the no_log: True option, so that we are not presented with pages of information on every run through of the playbook. If you need to enable it for debugging, you would change this to false, or remove it entirely.
Once we have the list of peers, we store them in the bgp_peer_result variable: -
ok: [routeros-01] => {
"bgp_peer_result": {
"changed": false,
"failed": false,
"stdout": [
"Flags: X - disabled, E - established \n # INSTANCE REMOTE-ADDRESS REMOTE-AS \n 0 E default 2001:db8:904:beef::2 65104 \n 1 E default 10.100.104.254 65430 \n 2 E default 2001:db8:104::ffff 65430 \n 3 E default 192.0.2.204 65104"
],
"stdout_lines": [
[
"Flags: X - disabled, E - established ",
" # INSTANCE REMOTE-ADDRESS REMOTE-AS ",
" 0 E default 2001:db8:904:beef::2 65104 ",
" 1 E default 10.100.104.254 65430 ",
" 2 E default 2001:db8:104::ffff 65430 ",
" 3 E default 192.0.2.204 65104"
]
]
}
}
Cleaning the output
The next task takes the output above, and creates a list from the entries in stdout_lines.
ok: [routeros-01] => {
"bgp_peer_result_output": [
[
"Flags: X - disabled, E - established ",
" # INSTANCE REMOTE-ADDRESS REMOTE-AS ",
" 0 E default 2001:db8:904:beef::2 65104 ",
" 1 E default 10.100.104.254 65430 ",
" 2 E default 2001:db8:104::ffff 65430 ",
" 3 E default 192.0.2.204 65104"
]
]
}
While this doesn’t do much, it does make the output cleaner and easier to work with.
Checking if the peer exists - IPv4
This task goes through the list of peers defined in our host_vars. If the peer exists in the bgp_peer_result_output variable, it adds it to peer_exists variable. If peer_exists isn’t already defined, it creates an empty list.
ok: [routeros-01] => {
"peer_exists": [
"10.100.104.254",
"192.0.2.204"
]
}
Checking if the peer exists - IPv6
This task does the same as the above, except for IPv6: -
ok: [routeros-01] => {
"peer_exists": [
"10.100.104.254",
"192.0.2.204",
"2001:db8:104::ffff",
"2001:db8:904:beef::2"
]
}
For simplicity, we do not create a new list for IPv6.
Adding eBGP peers if they do not exist - IPv4
This task will go through our list of peers in our host_vars. If they are IPv4, and if they are not already configured (i.e. they are not in the peer_exists variable, or no peers are configured), it will attempt to add them.
The host_vars that are relevant are: -
bgp:
local_as: 65104
neighbors:
ipv4:
- peer: 10.100.104.254
remote_as: 65430
ebgp: true
name: netsvr-01-v4
- peer: 192.0.2.204
remote_as: 65104
ibgp: true
update_source: loopback0
name: routeros-02-v4
As we can see, only one peer here is eBGP, so the configuration we would generate is: -
/routing bgp peer add name=netsvr-01-v4 remote-address=10.100.104.254 remote-as=65430
Update existing eBGP peers if they do not exist - IPv4
This task is like the previous, except rather than attempting to add peers, it updates existing peer configuration. This task is dependent on the peer being defined as an eBGP peer in our host_vars, and being in the peer_exists variable.
The same host_vars are used, but this time we generate the following configuration: -
/routing bgp peer set [find where remote-address="10.100.104.254"] remote-as="65430" name="netsvr-01-v4"
Adding eBGP peers if they do not exist - IPv6
This task is the same as the IPv4 version, except it is for IPv6 peers.
The host_vars that are relevant are: -
bgp:
local_as: 65104
neighbors:
ipv6:
- peer: "2001:db8:104::ffff"
remote_as: 65430
ebgp: true
name: netsvr-01-v6
- peer: "2001:db8:904:beef::2"
remote_as: 65104
ibgp: true
update_source: loopback0
name: routeros-02-v6
This then generates the following configuration: -
/routing bgp peer add address-families=ipv6 name=netsvr-01-v6 remote-address=2001:db8:104::ffff remote-as=65430
In addition, we restrict this to the IPv6 address family, to ensure it does not attempt to advertise/receive IPv4 routes over this session. This is supported on RouterOS, but not all vendors supports this.
Update existing eBGP peers if they do exist - IPv6
Again, this is like the IPv4 task, except it runs only if the peer is in the peer_exists variable.
The same host_vars are used, but this time we generate the following configuration: -
/routing bgp peer set [find where remote-address="2001:db8:104::ffff"] remote-as="65430" address-families="ipv6" name="netsvr-01-v6"
iBGP tasks for IPv4 and IPv6
We have an almost identical set of tasks, except for iBGP rather than eBGP. The difference between eBGP and iBGP is that for eBGP, we do not set a source interface for the BGP peer. For iBGP we do, because this peer may be available over multiple interfaces.
If the peers do not exist (i.e. not already configured), this configuration is generated: -
/routing bgp peer add address-families=ipv6 name=routeros-02-v6 remote-address=2001:db8:904:beef::2 remote-as=65104 update-source=loopback0
/routing bgp peer add name=routeros-02-v4 remote-address=192.0.2.204 remote-as=65104 update-source=loopback0
If the peers do not exist, this is applied instead: -
/routing bgp peer set [find where remote-address="2001:db8:904:beef::2"] address-families=ipv6 name=routeros-02-v6 remote-as=65104 update-source=loopback0
/routing bgp peer set [find where remote-address="192.0.2.204"] name=routeros-02-v4 remote-as=65104 update-source=loopback0
Default Originate on the iBGP IPv4 session
This task will go through the BGP neighbours in our host_vars. If the neighbour has the default_originate option, it will update that peer. This will enable it to advertise a default route.
The host_vars that are relevant are: -
bgp:
local_as: 65104
neighbors:
ipv4:
- peer: 192.0.2.204
remote_as: 65104
ibgp: true
update_source: loopback0
default_originate: true
name: routeros-02-v4
This will generate the following: -
/routing bgp peer set [find where remote-address="{{ item.peer }}"] default-originate=if-installed
We can verify this with: -
[admin@routeros-01] > /routing bgp peer print detail where remote-address=192.0.2.204
Flags: X - disabled, E - established
0 E name="routeros-02-v4" instance=default remote-address=192.0.2.204 remote-as=65104 tcp-md5-key="" nexthop-choice=default multihop=no route-reflect=no hold-time=3m ttl=255 in-filter="" out-filter="" address-families=ip
update-source=loopback0 default-originate=if-installed remove-private-as=no as-override=no passive=no use-bfd=no
Redistributing OSPF, connected and static routes
This task is used to advertise our OSPF routes, connected routes and also static routes (including our DHCP-received default route). The latter is required because the if-installed option of BGP’s default-originate requires the route to be part of the BGP process (even if through redistribution). This task will be applied if we have the redist option set in the bgp section of our host_vars.
This generates the following configuration: -
/routing bgp instance set 0 redistribute-ospf=yes redistribute-connected=yes redistribute-static=yes
We can verify this with: -
[admin@routeros-01] > /routing bgp instance print
Flags: * - default, X - disabled
0 * name="default" as=65104 router-id=192.0.2.104 redistribute-connected=yes redistribute-static=yes redistribute-rip=no redistribute-ospf=yes redistribute-other-bgp=no out-filter="" client-to-client-reflection=yes
ignore-as-path-len=no routing-table=""
Removing undefined peers
This set of tasks retrieves the list of peers configured on the routers. If they do not exist in our host_vars, they are removed.
This means that peers removed from our host_vars will also be removed from the routers too.
Similar to the OSPF playbook, we use regular expressions to parse commands. We then use variables (dynamically generated/discovered, and our host_vars) to remove the undefined peers.
Retrieve IPs of configured peers - IPv4
This task takes the output of bgp_peer_result (created earlier in the playbook), and gathers the first entry in the stdout_lines section. This takes the below output: -
ok: [routeros-01] => {
"bgp_peer_result": {
"changed": false,
"failed": false,
"stdout": [
"Flags: X - disabled, E - established \n # INSTANCE REMOTE-ADDRESS REMOTE-AS \n 0 E default 2001:db8:904:beef::2 65104 \n 1 E default 10.100.104.254 65430 \n 2 E default 2001:db8:104::ffff 65430 \n 3 E default 192.0.2.204 65104 \n 4 default 192.0.2.255 65001"
],
"stdout_lines": [
[
"Flags: X - disabled, E - established ",
" # INSTANCE REMOTE-ADDRESS REMOTE-AS ",
" 0 E default 2001:db8:904:beef::2 65104 ",
" 1 E default 10.100.104.254 65430 ",
" 2 E default 2001:db8:104::ffff 65430 ",
" 3 E default 192.0.2.204 65104 ",
" 4 default 192.0.2.255 65001"
]
]
}
}
and turns it into: -
ok: [routeros-01] => {
"bgp_peer_result_output": [
[
"Flags: X - disabled, E - established ",
" # INSTANCE REMOTE-ADDRESS REMOTE-AS ",
" 0 E default 2001:db8:904:beef::2 65104 ",
" 1 E default 10.100.104.254 65430 ",
" 2 E default 2001:db8:104::ffff 65430 ",
" 3 E default 192.0.2.204 65104 ",
" 4 default 192.0.2.255 65001"
]
]
}
After this, we use reject filters to remove the lines Flags: X - disabled, E - established and # INSTANCE REMOTE-ADDRESS REMOTE-AS. This leaves us with: -
[
" 0 E default 2001:db8:904:beef::2 65104 ",
" 1 E default 10.100.104.254 65430 ",
" 2 E default 2001:db8:104::ffff 65430 ",
" 3 E default 192.0.2.204 65104 ",
" 4 default 192.0.2.255 65001"
]
Once this is done, we convert it to a list again. This is because reject filters turn the list into something called a generator object. A generator is an object that can be iterated over (like a list) but is not human readable. If you are familiar with Python, you will likely be familiar with generators. For our purposes, a generator is not useful, so we transform it back into a list.
We then loop through the output and extract the IPv4 peer IPs using regular expressions.
The regular expression is \\b(?:[0-9]{1,3}\\.){3}[0-9]{1,3}\\b. This does the following: -
\\b- Whole words/strings only, not a match within a word- The
(?:and)section creates a non-capturing group, allowing us to repeat the expression inside it without needing to necessarily “save” the results - The
[0-9]{1,3}says that we are looking for numbers between 0-9 between 1 and 3 times. This will match up to 255 - Technically this could also match up to 999, but IP addresses do not go that high!
- The
{3}matches the previous expression exactly 3 times - The second use of
[0-9]{1,3}matches the last octet of an IP address
This is similar to what we used for OSPF networks, except there are no network masks to match.
We then create a list called v4_configured_peers. Every IPv4 peer we find using the above regular expression is added to it: -
ok: [routeros-01] => {
"v4_configured_peers": [
[],
[
"10.100.104.254"
],
[],
[
"192.0.2.204"
],
[
"192.0.2.255"
]
]
}
Retrieve IPs of configured peers - IPv6
We do the same as above for IPv6. The only difference is the regular expression. The regular expression is [0-9a-fA-F]{1,4}:.*:[0-9a-fA-F]{1,4}. This is not as comprehensive as the IPv4 regular expression, and could miss certain kinds of IPv6 addresses. It will match any BGP peer we’d configure using IPv6, so this is good enough.
This generates the v6_configured_peers list, which looks like the below: -
ok: [routeros-01] => {
"v6_configured_peers": [
[
"2001:db8:904:beef::2"
],
[],
[
"2001:db8:104::ffff"
],
[],
[]
]
}
Combining the configured peer lists
This task takes the two lists generated (v4_configured_peers and v6_configured_peers) and combines them. It also flattens the lists. Rather than lists within lists, all elements are part of one list instead.
The output of this is: -
ok: [routeros-01] => {
"configured_peers": [
"10.100.104.254",
"192.0.2.204",
"192.0.2.255",
"2001:db8:904:beef::2",
"2001:db8:104::ffff"
]
}
Creating list of defined peers
This task goes through our host_vars, and generates a list of the BGP peers we have defined. Our host_vars are our source of truth, meaning the only peers that should be configured are those in our host_vars.
The below host_vars are relevant: -
bgp:
neighbors:
ipv4:
- peer: 10.100.104.254
- peer: 192.0.2.204
ipv6:
- peer: "2001:db8:104::ffff"
- peer: "2001:db8:904:beef::2"
This generates the following list: -
ok: [routeros-01] => {
"defined_peers": [
"10.100.104.254",
"192.0.2.204",
"2001:db8:104::ffff",
"2001:db8:904:beef::2"
]
}
Creating list of peers to delete
This task loops through the configured_peers variable, and checks if each peer exists in the defined_peers variable. If it doesn’t, it is added to the variable delete_peers.
This generates something like the below: -
ok: [routeros-01] => {
"delete_peers": [
"192.0.2.255"
]
}
Deleting the undefined peers
This task loops through the delete_peers variable. For every peer, it will remove them from the configuration (IPv4 or IPv6). Based upon the last task, this would then generate the following command: -
/routing bgp peer remove [find where remote-address="192.0.2.255"]
Before this, our peers looked like: -
[admin@routeros-01] /routing bgp peer> print brief
Flags: X - disabled, E - established
# INSTANCE REMOTE-ADDRESS REMOTE-AS
0 E default 2001:db8:904:beef::2 65104
1 E default 10.100.104.254 65430
2 E default 2001:db8:104::ffff 65430
3 E default 192.0.2.204 65104
4 default 192.0.2.255 65001
After this, they look like: -
[admin@routeros-01] /routing bgp peer> print brief
Flags: X - disabled, E - established
# INSTANCE REMOTE-ADDRESS REMOTE-AS
0 E default 2001:db8:904:beef::2 65104
1 E default 10.100.104.254 65430
2 E default 2001:db8:104::ffff 65430
3 E default 192.0.2.204 65104
Summary
As mentioned, this is probably the most complex playbook in the series so far. As we have no modules to rely on, we achieve idempotence using RouterOS CLI features, regular expressions and complex list operations.
This does go some way to showing the power of Ansible, and in many ways the flexibility of RouterOS as well. You may not be benefiting from having any abstraction away from RouterOS, but it does allow you to make use of list comparisons, variables, regular expressions, looping and more.
Verification
At the end of this playbook, we should have BGP sessions established over IPv4 and IPv6, as well as routes received and sent to the netsvr-01 BGP route server.
routeros-01
! Show BGP neighbours on IPv4 and IPv6
[admin@routeros-01] > /routing bgp peer print
Flags: X - disabled, E - established
# INSTANCE REMOTE-ADDRESS REMOTE-AS
0 E default 2001:db8:904:beef::2 65104
1 E default 10.100.104.254 65430
2 E default 2001:db8:104::ffff 65430
3 E default 192.0.2.204 65104
! Show BGP routes
[admin@routeros-01] > /ip route print where bgp
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 ADb 192.0.2.1/32 10.100.104.254 20
[admin@routeros-01] > /ipv6 route print where bgp
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, o - ospf, b - bgp, U - unreachable
# DST-ADDRESS GATEWAY DISTANCE
0 ADb 2001:db8:999:beef::1/128 fe80::33a6:26b8:1b0e:... 20
! Ping the netsvr Loopback (192.0.2.1 and 2001:DB8:999:BEEF::1)
[admin@routeros-01] > /ping 192.0.2.1
SEQ HOST SIZE TTL TIME STATUS
0 192.0.2.1 56 64 0ms
sent=1 received=1 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms
[admin@routeros-01] > /ping 2001:db8:999:beef::1
SEQ HOST SIZE TTL TIME STATUS
0 2001:db8:999:beef::1 56 64 1ms echo reply
1 2001:db8:999:beef::1 56 64 0ms echo reply
sent=2 received=2 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=1ms
routeros-02
! Show BGP neighbours on IPv4 and IPv6
[admin@routeros-02] > /routing bgp peer print
Flags: X - disabled, E - established
# INSTANCE REMOTE-ADDRESS REMOTE-AS
0 E default 2001:db8:904:beef::1 65104
1 E default 192.0.2.104 65104
! Show BGP routes
[admin@routeros-02] > /ip route print where bgp
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 ADb 0.0.0.0/0 192.0.2.104 200
1 Db 10.15.30.0/24 192.0.2.104 200
2 Db 10.100.104.0/24 192.0.2.104 200
3 Db 10.100.204.0/24 192.0.2.104 200
4 ADb 192.0.2.1/32 10.100.104.254 200
5 Db 192.0.2.104/32 192.0.2.104 200
6 Db 192.0.2.204/32 192.0.2.104 200
7 ADb 192.168.122.0/24 192.0.2.104 200
[admin@routeros-02] > /ipv6 route print where bgp
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, o - ospf, b - bgp, U - unreachable
# DST-ADDRESS GATEWAY DISTANCE
0 Db 2001:db8:104::/64 2001:db8:904:beef::1 200
1 Db 2001:db8:204::/64 2001:db8:904:beef::1 200
2 Db 2001:db8:904:beef::1/128 2001:db8:904:beef::1 200
3 Db 2001:db8:904:beef::2/128 2001:db8:904:beef::1 200
4 Db 2001:db8:999:beef::1/128 2001:db8:904:beef::1 200
! Ping the netsvr Loopback (192.0.2.1 and 2001:DB8:999:BEEF::1)
[admin@routeros-02] > /ping 192.0.2.1
SEQ HOST SIZE TTL TIME STATUS
0 192.0.2.1 56 63 0ms
1 192.0.2.1 56 63 1ms
2 192.0.2.1 56 63 1ms
sent=3 received=3 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=1ms
[admin@routeros-02] > /ping 2001:db8:999:beef::1
SEQ HOST SIZE TTL TIME STATUS
0 no route to host
1 no route to host
sent=2 received=0 packet-loss=100%
Hmm, the last ping isn’t working. Why is that? Lets have a look at the routing table: -
! Check the route
[admin@routeros-02] /routing bgp instance> /ipv6 route print where dst-address=2001:db8:999:beef::1/128
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, o - ospf, b - bgp, U - unreachable
# DST-ADDRESS GATEWAY DISTANCE
0 Db 2001:db8:999:beef::1/128 2001:db8:904:beef::1 200
! Not active? Why?
[admin@routeros-02] /routing bgp instance> /ipv6 route print detail where dst-address=2001:db8:999:beef::1/128
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, o - ospf, b - bgp, U - unreachable
0 Db dst-address=2001:db8:999:beef::1/128 gateway=2001:db8:904:beef::1 gateway-status=2001:db8:904:beef::1 unreachable distance=200 scope=40 target-scope=30 bgp-as-path="65430" bgp-local-pref=100 bgp-med=0 bgp-origin=igp
received-from=routeros-01-v6
Unreachable? Lets have a look at that gateway (i.e. the next-hop)
[admin@routeros-02] /routing bgp instance> /ipv6 route print where dst-address=2001:db8:904:beef::1
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, o - ospf, b - bgp, U - unreachable
# DST-ADDRESS GATEWAY DISTANCE
0 ADo 2001:db8:904:beef::1/128 fe80::5054:ff:fec0:5d... 110
1 Db 2001:db8:904:beef::1/128 2001:db8:904:beef::1 200
So the next-hop is received via OSPF (v3). Why is it being marked as unreachable?
Unfortunately this is because of a bug in RouterOS. It is not able to use next-hops that are link-local for BGP next-hop recursion. This MikroTik forum post (with over a hundred replies) details the issue further.
This makes the internal IPv6 BGP peering unusable. There are ways of statically setting next hops, or using filtering to override next hops, but this goes against using a dynamic internal routing protocol.
SNMP
In this section we enable SNMP for monitoring the routers.
Playbook
The contents of the playbook are below: -
---
## tasks file for snmp
- name: Enable SNMPv3
routeros_command:
commands:
- /snmp set contact="{{ snmp['contact'] }}" location="{{ snmp['location'] }}"
- /snmp community set 0 name="{{ snmp['user'] }}"
- /snmp community set 0 encryption-protocol="AES" encryption-password="{{ snmp['priv_key'] }}"
- /snmp community set 0 authentication-protocol="SHA1" authentication-password="{{ snmp['auth_key'] }}"
- /snmp community set 0 security=privacy
- /snmp set enabled=yes
tags:
- snmp
We gather our SNMP details and credentials from host_vars: -
snmp:
location: Yeti Home
contact: The Hairy One
user: yetiops
group: yetiops_group
auth_key: !vault |
$ANSIBLE_VAULT;1.1;AES256
383###REDACTED###############################################################566
363###REDACTED###############################################################366
343###REDACTED###############################################################565
326###REDACTED###############################################################362
3431
priv_key: !vault |
$ANSIBLE_VAULT;1.1;AES256
383###REDACTED###############################################################566
363###REDACTED###############################################################366
343###REDACTED###############################################################565
326###REDACTED###############################################################362
3431
This generates the following configuration: -
/snmp set contact="The Hairy One" location="Yeti Home"
/snmp community set 0 name="yetiops"
/snmp community set 0 encryption-protocol="AES" encryption-password="###PRIV_KEY###"
/snmp community set 0 authentication-protocol="SHA1" authentication-password="###AUTH_KEY###"
/snmp community set 0 security=privacy
/snmp set enabled=yes
We can verify this with: -
admin@routeros-01] > /snmp community print detail
Flags: * - default
0 * name="yetiops" addresses=::/0 security=none read-access=yes write-access=no authentication-protocol=SHA1 encryption-protocol=AES
authentication-password="###AUTH_KEY###" encryption-password="###PRIV_KEY###"
[admin@routeros-01] > /snmp print
enabled: yes
contact: The Hairy One
location: Yeti Home
engine-id:
src-address: ::
trap-target:
trap-community: yetiops
trap-version: 1
trap-generators: temp-exception
Verification
To check whether this is working, you will need either some form of monitoring system, or you can use something like snmpwalk to check: -
! snmpwalk to routeros-01
$ snmpwalk -v3 -u yetiops -a SHA -A ###AUTH-KEY### -x AES -X ###PRIV-KEY### -l authPriv 10.15.30.53
iso.3.6.1.2.1.1.1.0 = STRING: "RouterOS CHR"
iso.3.6.1.2.1.1.2.0 = OID: iso.3.6.1.4.1.14988.1
iso.3.6.1.2.1.1.3.0 = Timeticks: (369200) 1:01:32.00
iso.3.6.1.2.1.1.4.0 = STRING: "The Hairy One"
iso.3.6.1.2.1.1.5.0 = STRING: "routeros-01"
iso.3.6.1.2.1.1.6.0 = STRING: "Yeti Home"
iso.3.6.1.2.1.1.7.0 = INTEGER: 78
iso.3.6.1.2.1.2.1.0 = INTEGER: 6
iso.3.6.1.2.1.2.2.1.1.1 = INTEGER: 1
iso.3.6.1.2.1.2.2.1.1.2 = INTEGER: 2
iso.3.6.1.2.1.2.2.1.1.3 = INTEGER: 3
iso.3.6.1.2.1.2.2.1.1.12 = INTEGER: 12
iso.3.6.1.2.1.2.2.1.1.30 = INTEGER: 30
iso.3.6.1.2.1.2.2.1.1.32 = INTEGER: 32
[...]
! snmpwalk to routeros-02
$ snmpwalk -v3 -u yetiops -a SHA -A ###AUTH-KEY### -x AES -X ###PRIV-KEY### -l authPriv 10.15.30.54
iso.3.6.1.2.1.1.1.0 = STRING: "RouterOS CHR"
iso.3.6.1.2.1.1.2.0 = OID: iso.3.6.1.4.1.14988.1
iso.3.6.1.2.1.1.3.0 = Timeticks: (378200) 1:03:02.00
iso.3.6.1.2.1.1.4.0 = STRING: "The Hairy One"
iso.3.6.1.2.1.1.5.0 = STRING: "routeros-02"
iso.3.6.1.2.1.1.6.0 = STRING: "Yeti Home"
iso.3.6.1.2.1.1.7.0 = INTEGER: 78
iso.3.6.1.2.1.2.1.0 = INTEGER: 4
iso.3.6.1.2.1.2.2.1.1.1 = INTEGER: 1
iso.3.6.1.2.1.2.2.1.1.2 = INTEGER: 2
iso.3.6.1.2.1.2.2.1.1.7 = INTEGER: 7
iso.3.6.1.2.1.2.2.1.1.16 = INTEGER: 16
[...]
All looking good!
NAT
In this section, we allow the internal router to reach the internet via the edge router. The edge router has a default route to the internet that is learned from DHCP. NAT is used because the internet does not know about our private IP addresses, requiring us to masquerade behind a public IP address (or addresses) to route to and from the internet.
Playbook
The playbook looks like the below: -
---
## tasks file for nat
- name: Apply NAT Masquerade
routeros_command:
commands:
- /ip firewall nat remove [find where out-interface="{{ item.routeros_if }}"]
- /ip firewall nat add chain=srcnat action=masquerade out-interface="{{ item.routeros_if }}"
when:
- item.nat is defined
- item.nat.role is defined
- item.nat.role is match('outside')
loop: "{{ interfaces }}"
tags:
- nat
In the above, we are looking for whether we have a the nat.role variable on any interface in our host_vars, and if it is set to outside.
The relevant host_vars are: -
interfaces:
- routeros_if: "ether1"
- routeros_if: "ether2"
- routeros_if: "vlan104"
- routeros_if: "vlan204"
nat:
role: inside
- routeros_if: "ether3"
nat:
role: outside
- routeros_if: "loopback0"
In the above, only the ether3 interface has the outside NAT role, so we generate the following configuration: -
/ip firewall nat remove [find where out-interface="ether3"]
/ip firewall nat add chain=srcnat action=masquerade out-interface="ether3"
We remove the existing configuration first, to ensure that any manual changes are rectified, as well as not creating multiple identical configuration items. Without this, you would see the line for NAT via ether3 configured again every time you run the playbook.
We can verify this with: -
[admin@routeros-01] /routing bgp peer> /ip firewall nat print
Flags: X - disabled, I - invalid, D - dynamic
0 chain=srcnat action=masquerade out-interface=ether3
Verification
We can now test from the internal router, to see if it can reach the internet: -
! Can we reach the internet?
[admin@routeros-02] > /ping 8.8.8.8
SEQ HOST SIZE TTL TIME STATUS
0 8.8.8.8 56 52 21ms
1 8.8.8.8 56 52 20ms
sent=2 received=2 packet-loss=0% min-rtt=20ms avg-rtt=20ms max-rtt=21ms
[admin@routeros-02] >
[admin@routeros-02] > /ping 1.1.1.1
SEQ HOST SIZE TTL TIME STATUS
0 1.1.1.1 56 55 14ms
1 1.1.1.1 56 55 14ms
2 1.1.1.1 56 55 13ms
sent=3 received=3 packet-loss=0% min-rtt=13ms avg-rtt=13ms max-rtt=14ms
[admin@routeros-02] > /ping 1.1.1.1 src-address=192.0.2.204
SEQ HOST SIZE TTL TIME STATUS
0 1.1.1.1 56 55 17ms
1 1.1.1.1 56 55 15ms
2 1.1.1.1 56 55 12ms
sent=3 received=3 packet-loss=0% min-rtt=12ms avg-rtt=14ms max-rtt=17ms
[admin@routeros-02] > /ping 8.8.8.8 src-address=192.0.2.204
SEQ HOST SIZE TTL TIME STATUS
0 8.8.8.8 56 52 19ms
1 8.8.8.8 56 52 19ms
sent=2 received=2 packet-loss=0% min-rtt=19ms avg-rtt=19ms max-rtt=19ms
! What does this look like on the edge router?
[admin@routeros-01] > /ip firewall nat print stats
Flags: X - disabled, I - invalid, D - dynamic
# CHAIN ACTION BYTES PACKETS
0 srcnat masquerade 5 662 94
All looking good!
AAA
The final task is AAA (Authentication, Authorization and Accounting). As noted, MikroTik does not work with TACACS+, so we use RADIUS instead. This performs AAA against freeradius running on the netsvr-01 machine.
This will allow central management of our users, as well as providing logs of commands being run.
Playbook
The contents of the playbook are: -
---
## tasks file for aaa
- name: Enable RADIUS
routeros_command:
commands:
- /radius remove [find where address="{{ tacacs['ipv4'] }}"]
- /radius add service=login address="{{ tacacs['ipv4'] }}" secret="{{ radius['secret'] }}" src-address="{{ router_id }}" comment="netsvr-01"
- /user aaa set use-radius=yes
tags:
- aaa
While it may seem a little strange using the tacacs variables for the server address, if MikroTik do support TACACS+ in the future, it will make transitioning between the two easier.
The relevant host_vars are: -
router_id: 192.0.2.104
radius:
secret: !vault |
$ANSIBLE_VAULT;1.1;AES256
356431################REDACTED###########################31313136626333
623664################REDACTED###########################65366437633463
623135################REDACTED###########################33346233346665
633265################REDACTED###########################63333834396361
333835################REDACTED###########################13936
The rest of our variables come from our group_vars: -
tacacs:
ipv4: 192.0.2.1
The above generates the following configuration: -
/radius remove [find where address="192.0.2.1"]
/radius add service=login address="192.0.2.1" secret="###RADIUS_SECRET###" src-address="192.0.2.104" comment="netsvr-01"
/user aaa set use-radius=yes
Again, we remove the existing configuration to ensure that any manual changes are overridden and to avoid generating duplicate configuration.
We can verify this with: -
[admin@routeros-01] > /radius print
Flags: X - disabled
# SERVICE CALLED-ID DOMAIN ADDRESS SECRET
0 ;;; netsvr-01
login 192.0.2.1 $RADIUS_SECRET
Verification
! Can we login with the yetiops user?
ssh [email protected]
[email protected]'s password:
MMM MMM KKK TTTTTTTTTTT KKK
MMMM MMMM KKK TTTTTTTTTTT KKK
MMM MMMM MMM III KKK KKK RRRRRR OOOOOO TTT III KKK KKK
MMM MM MMM III KKKKK RRR RRR OOO OOO TTT III KKKKK
MMM MMM III KKK KKK RRRRRR OOO OOO TTT III KKK KKK
MMM MMM III KKK KKK RRR RRR OOOOOO TTT III KKK KKK
MikroTik RouterOS 6.45.8 (c) 1999-2020 http://www.mikrotik.com/
[?] Gives the list of available commands
command [?] Gives help on the command and list of arguments
[Tab] Completes the command/word. If the input is ambiguous,
a second [Tab] gives possible options
/ Move up to base level
.. Move up one level
/command Use command at the base level
----------------------------------------
|
| This banner was generated by Ansible
|
----------------------------------------
|
| You are logged into routeros-02
|
----------------------------------------
[yetiops@routeros-02] >
! What about a user that doesn't exist?
$ ssh [email protected]
[email protected]'s password:
Permission denied, please try again.
!! This attempt was logged to the router and syslog !!
[admin@routeros-02] >
23:01:53 echo: system,error,critical login failure for user jeff from 10.15.30.1 via ssh
! What do see in our radius log?
Tue May 19 00:00:45 2020 : Auth: (2) Login OK: [yetiops/<via Auth-Type = mschap>] (from client routeros-02 port 0 cli 10.15.30.1)
Tue May 19 00:01:50 2020 : Auth: (5) Login incorrect (mschap: FAILED: No NT/LM-Password. Cannot perform authentication): [jeff/<via Auth-Type = mschap>] (from client routeros-02 port 0 cli 10.15.30.1)
! What if freeradius goes away?
[root@netsvr-01 /var/log/radius] $ systemctl stop radiusd
[root@netsvr-01 /var/log/radius] $ systemctl status radiusd
radiusd.service - FreeRADIUS high performance RADIUS server.
Loaded: loaded (/usr/lib/systemd/system/radiusd.service; enabled; vendor preset: disabled)
Active: inactive (dead) since Tue 2020-05-19 00:04:37 BST; 5s ago
Process: 1682 ExecStart=/usr/sbin/radiusd -d /etc/raddb (code=exited, status=0/SUCCESS)
Process: 1523 ExecStartPre=/usr/sbin/radiusd -C (code=exited, status=0/SUCCESS)
Process: 1514 ExecStartPre=/bin/chown -R radiusd.radiusd /var/run/radiusd (code=exited, status=0/SUCCESS)
Main PID: 1684 (code=exited, status=0/SUCCESS)
May 18 22:37:42 netsvr-01 systemd[1]: Starting FreeRADIUS high performance RADIUS server....
May 18 22:37:43 netsvr-01 systemd[1]: Started FreeRADIUS high performance RADIUS server..
May 19 00:04:37 netsvr-01 systemd[1]: Stopping FreeRADIUS high performance RADIUS server....
May 19 00:04:37 netsvr-01 systemd[1]: Stopped FreeRADIUS high performance RADIUS server..
$ ssh [email protected]
[email protected]'s password:
Permission denied, please try again.
$ ssh [email protected]
[email protected]'s password:
MMM MMM KKK TTTTTTTTTTT KKK
MMMM MMMM KKK TTTTTTTTTTT KKK
MMM MMMM MMM III KKK KKK RRRRRR OOOOOO TTT III KKK KKK
MMM MM MMM III KKKKK RRR RRR OOO OOO TTT III KKKKK
MMM MMM III KKK KKK RRRRRR OOO OOO TTT III KKK KKK
MMM MMM III KKK KKK RRR RRR OOOOOO TTT III KKK KKK
MikroTik RouterOS 6.45.8 (c) 1999-2020 http://www.mikrotik.com/
[?] Gives the list of available commands
command [?] Gives help on the command and list of arguments
[Tab] Completes the command/word. If the input is ambiguous,
a second [Tab] gives possible options
/ Move up to base level
.. Move up one level
/command Use command at the base level
----------------------------------------
|
| This banner was generated by Ansible
|
----------------------------------------
|
| You are logged into routeros-02
|
----------------------------------------
may/18/2020 23:01:53 system,error,critical login failure for user jeff from 10.15.30.1 via ssh
may/18/2020 23:05:06 system,error,critical login failure for user yetiops from 10.15.30.1 via ssh
All looking good!
Parent playbook
The parent playbook (i.e. the playbook that brings all the roles together) is below: -
---
- hosts: mikrotik
gather_facts: no
tasks:
- import_role:
name: system
- import_role:
name: interfaces
- import_role:
name: firewall
- import_role:
name: routing
- import_role:
name: snmp
- import_role:
name: nat
- import_role:
name: aaa
All configuration changes are saved immediately, so there is no need to run a task to save the final configuration at the end.
Safe mode
RouterOS has has a feature known as “safe mode”. This allows you to apply configuration, and if it detects that your console/SSH session has timed out or reset, it will roll back all of the configuration applied within “safe mode”.
This isn’t as comprehensive as JunOS’s configuration commits, but it does mean that you can apply changes without losing access to the device.
There are cases where you may expect to lose access briefly (e.g. changing of IP addressing or routing) that safe mode does not account for these. However as a basic automatic rollback measure, it is a nice option to have.
To enter MikroTik’s safe mode, use the key combination CTRL+x. You will then be presented with this prompt: -
[ansible@routeros-02] >
[Safe Mode taken]
[ansible@routeros-02] <SAFE>
When you are finished, press CTRL+x to leave safe mode, “committing” the changes you made.
One point to note is that if you do not exit safe mode, any changes you make will eventually be rolled back (due to SSH or TCP timeouts). I have experienced this myself, rolling back important spanning tree changes that ended up causing a loop, long after I’d left the office that day!
Role Order
The role order is very similar to IOS and JunOS. The main difference here from IOS is that we are using stateful firewalling rather than access lists. The justification for the role order is detailed within Part 3 - Cisco IOS. To summarize: -
system- Sets up logging, hostnames and banners- This ensure we have logging ready for if any of the other roles fail (that the Ansible debug output cannot help with)
interfaces- This is a prerequisite for most of the following tasksfirewall- Apply before routing so that the device is not open to the world when publicly routable- This has to come after the interfaces, as some of the filters and lists are dependent on interfaces existing
routing- Routing is required for NAT and AAA to functionsnmp- No dependency on any service, so this can go anywherenat- Apply this after routing, otherwise the internal router has no default route to reach external destinations anywayaaa- It depends upon routing, and if configured incorrectly it can break login sessions
Running aaa last means that all other configuration is complete. If login sessions do break due to the aaa configuration, you have a fully configured router that is potentially reachable via other means (e.g. Out-of-Band, Admin accounts)
Artifacts
The final directory structure looks like the below: -
$ tree -L 2
.
├── ansible.cfg
├── ansible.log
├── group_vars
│ └── mikrotik
├── host_vars
│ ├── routeros-01.yaml
│ └── routeros-02.yaml
├── inventory
├── mikrotik.yaml
└── roles
├── aaa
├── firewall
├── interfaces
├── nat
├── routing
├── snmp
└── system
The final contents of our group_vars are: -
ansible_user: ansible
ansible_connection: network_cli
ansible_network_os: routeros
ansible_ssh_pass: !vault |
$ANSIBLE_VAULT;1.1;AES256
383###REDACTED###############################################################566
363###REDACTED###############################################################366
343###REDACTED###############################################################565
326###REDACTED###############################################################362
3431
log_host: 10.100.104.254
tacacs:
ipv4: 192.0.2.1
secret: supersecret
snmp:
location: Yeti Home
contact: The Hairy One
user: yetiops
group: yetiops_group
auth_key: !vault |
$ANSIBLE_VAULT;1.1;AES256
383###REDACTED###############################################################566
363###REDACTED###############################################################366
343###REDACTED###############################################################565
326###REDACTED###############################################################362
3431
priv_key: !vault |
$ANSIBLE_VAULT;1.1;AES256
386###REDACTED###############################################################764
613###REDACTED###############################################################630
646###REDACTED###############################################################331
376###REDACTED###############################################################137
3563
The final contents of our host_vars are: -
routeros-01.yaml
router_id: 192.0.2.104
rtr_role: edge
bgp:
local_as: 65104
redist:
ospf: true
ospfv3: true
neighbors:
ipv4:
- peer: 10.100.104.254
remote_as: 65430
ebgp: true
name: netsvr-01-v4
acl: bgp-ipv4-peers
- peer: 192.0.2.204
remote_as: 65104
ibgp: true
update_source: loopback0
default_originate: true
name: routeros-02-v4
ipv6:
- peer: "2001:db8:104::ffff"
remote_as: 65430
ebgp: true
name: netsvr-01-v6
acl: bgp-ipv6-peers
- peer: "2001:db8:904:beef::2"
remote_as: 65104
ibgp: true
update_source: loopback0
name: routeros-02-v6
vlans:
- name: netsvr-01
vlan_id: 104
interface: ether2
- name: routeros-02
vlan_id: 204
interface: ether2
interfaces:
- routeros_if: "ether1"
desc: "Management"
enabled: true
ipv4: "10.15.30.53/24"
- routeros_if: "ether2"
desc: "VLAN Bridge"
enabled: true
- routeros_if: "vlan104"
desc: "To netsvr"
enabled: true
ipv4: "10.100.104.253/24"
ipv6: "2001:db8:104::f/64"
acl:
ipv4:
- bgp-ipv4-peers
- syslog
- aaa
ipv6:
- bgp-ipv6-peers
ospf:
area: "0.0.0.0"
passive: true
ospfv3:
area: "0.0.0.0"
passive: true
- routeros_if: "vlan204"
desc: "To routeros-02"
enabled: true
ipv4: "10.100.204.254/24"
ipv6: "2001:db8:204::a/64"
ospf:
area: "0.0.0.0"
ospfv3:
area: "0.0.0.0"
nat:
role: inside
- routeros_if: "ether3"
desc: "To the Internet"
enabled: true
ipv4: "dhcp"
nat:
role: outside
- routeros_if: "loopback0"
desc: "Loopback"
enabled: true
ipv4: "192.0.2.104/32"
ipv6: "2001:db8:904:beef::1/128"
ospf:
area: "0.0.0.0"
passive: true
ospfv3:
area: "0.0.0.0"
passive: true
radius:
secret: !vault |
$ANSIBLE_VAULT;1.1;AES256
356###REDACTED###############################################################33
623###REDACTED###############################################################63
623###REDACTED###############################################################65
633###REDACTED###############################################################61
333###REDACTED###########################################36
routeros-02.yaml
router_id: 192.0.2.204
rtr_role: internal
bgp:
local_as: 65104
neighbors:
ipv4:
- peer: 192.0.2.104
remote_as: 65104
ibgp: true
update_source: loopback0
name: routeros-01-v4
ipv6:
- peer: "2001:db8:904:beef::1"
remote_as: 65104
ibgp: true
update_source: loopback0
name: routeros-01-v6
vlans:
- name: routeros-01
vlan_id: 204
interface: ether2
interfaces:
- routeros_if: "ether1"
desc: "Management"
enabled: true
- routeros_if: "ether2"
desc: "VLAN Bridge"
enabled: true
subint:
vlans:
- 204
- routeros_if: "vlan204"
desc: "To routeros-01"
enabled: true
ipv4: "10.100.204.253/24"
ipv6: "2001:db8:204::f/64"
ospf:
area: "0.0.0.0"
ospfv3:
area: "0.0.0.0"
- routeros_if: "loopback0"
desc: "Loopback"
enabled: true
ipv4: "192.0.2.204/32"
ipv6: "2001:db8:904:beef::2/128"
ospf:
area: "0.0.0.0"
passive: true
ospfv3:
area: "0.0.0.0"
passive: true
radius:
secret: !vault |
$ANSIBLE_VAULT;1.1;AES256
356###REDACTED###############################################################33
623###REDACTED###############################################################63
623###REDACTED###############################################################65
633###REDACTED###############################################################61
333###REDACTED###########################################36
Despite the lack of modules, having this few variables to configure a fully functioning router is still incredibly useful, as well as being able to make idempotent updates of BGP peers, OSPF networks and more.
If you integrate these tasks with something like Netbox to supply your variables, you can build a huge estate all from a single source of truth, with little to no manual configuration involved.
Running the playbooks
Below is an Asciinema output of my terminal when running the playbooks, so you can see them being applied: -
Unlike the other labs, no task is marked as changed. This is because Ansible is applying raw RouterOS commands, with no real feedback mechanism to say whether the changes were successful or not. In this scenario, Ansible defaults to the change being “OK”, unless the task times out or receives a failed exit code.
Native modules versus routeros_command
Unlike the other vendors we have covered so far, we only have one module we can use for configuration, routeros_command. The only other Ansible native modules we have used are set_fact and routeros_facts to either create variables, or to retrieve data from the routers where it is available.
Whether other modules will be created for RouterOS is an unknown. Given that the syntax for removing or updating configuration differs from adding configuration, the Ansible modules would be more complex than those for IOS or JunOS (although not impossible).
Thoughts compared to IOS, JunOS and EOS
I knew going into this part of the series that the lack of any native modules would increase the complexity of the roles. Having supported MikroTik routers earlier in my career, as well as running them in my own network, I know the quirks and useful features that were available.
For anyone new to RouterOS, you’ll need to become familiar with the syntax before using configuring them with Ansible, otherwise you would struggle to extend the roles or create your own.
The experience of using Ansible for this compared to IOS, JunOS and EOS is vastly different. With IOS, JunOS and EOS, you can build Jinja2 templates if a module does not exist. This option is not available with RouterOS, meaning you need to know what you expect each command to do and what order to apply them to create repeatable tasks.
I still believe that using Ansible for RouterOS has many advantages over managing your network estate manually. You gain consistency in the changes being made, and the ability to manage devices en masse, both of which are huge benefits.
Summary
Overall, managing RouterOS with Ansible is more of a challenge than it is with other vendors. However we still meet the goals of repeatable and consistent configuration changes. This alone for me outweighs the need to effectively create your own modules to achieve idempotence.
I am hoping that at some point in the future, using Jinja2 templates is an option. This would allow the use of filters and conditional logic within a template. We could then define eBGP and iBGP peers within the same task, or IPv4 and IPv6, and much more.
The final configs from the routers are in my Network Automation with Ansible repository. The next part of this series will cover configuring VyOS routers using Ansible.
devops sysadmin ansible config management networking
technical sysadmin config management networking
18039 Words
2020-05-21 19:05 +0000