Neutron ML2 Port Security

A new NFV feature, ML2 port security is introduced in Openstack Kilo release.

Before this feature, when a neutron port is attached to a VM instance, several anti-spoofing iptables rules are applied to the port, which block some NFV use cases. For example, if one VM wants to host some DHCP/TFTP server services for other VMs, it does not work since those anti-spoofing rules block related traffic. To workaround it, the cloud infra has to disable security group function globally for whole cloud.

ML2 port security is introduced to address this use case, it can disable all anti-spoofing rules just for single port, while still keep security group function working for all other ports.

Let's try it out.

ML2 port security is a ML2 extension driver, needs to be enabled in /etc/neutron/plugins/ml2/ml2_conf.ini:

extension_drivers = port_security  

Restart neutron-server:

systemctl restart neutron-server  

Note: At the time of writing, we need to enable port_security extension driver before creating any neutron network, otherwise we will hit this bug.

How things work before

Now let's create a neutron network:

[root@overcloud-controller-0 ~]# neutron net-show test-port-security
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| id                        | dcb6c275-4fb6-4996-9c4e-344e9893c1c0 |
| mtu                       | 0                                    |
| name                      | test-port-security                   |
| port_security_enabled     | True                                 |
| provider:network_type     | vxlan                                |
| provider:physical_network |                                      |
| provider:segmentation_id  | 28                                   |
| router:external           | False                                |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   | 67f53d37-9d43-461c-b072-0f174074e57e |
| tenant_id                 | c5cb88bd612949a5afaed8acf79350ef     |
+---------------------------+--------------------------------------+

We can see there's a new attribute port_security_enabled, defaults to True.
Then we create a subnet:

neutron subnet-create --name test-port-security test-port-security 192.168.1.0/24  

Launch a VM in this network:

nova boot  --flavor m1.small --image cirros.qcow2 \  
 --nic net-id=dcb6c275-4fb6-4996-9c4e-344e9893c1c0\
  vm-port-security

Check VM status when it's up and running:

[root@overcloud-controller-0 ~]# nova show vm-port-security
+--------------------------------------+----------------------------------------------------------+
| Property                             | Value                                                    |
+--------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                                   |
| OS-EXT-AZ:availability_zone          | nova                                                     |
| OS-EXT-SRV-ATTR:host                 | overcloud-compute-1.localdomain                          |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | overcloud-compute-1.localdomain                          |
| OS-EXT-SRV-ATTR:instance_name        | instance-00000001                                        |
| OS-EXT-STS:power_state               | 1                                                        |
| OS-EXT-STS:task_state                | -                                                        |
| OS-EXT-STS:vm_state                  | active                                                   |
| OS-SRV-USG:launched_at               | 2015-12-02T11:35:41.000000                               |
| OS-SRV-USG:terminated_at             | -                                                        |
| accessIPv4                           |                                                          |
| accessIPv6                           |                                                          |
| config_drive                         |                                                          |
| created                              | 2015-12-02T11:34:01Z                                     |
| flavor                               | m1.small (2)                                             |
| hostId                               | b5eb9f770962f19d69fc795e90741863b6cde7af6afd9416f252e3b9 |
| id                                   | ccf8a0aa-f488-438c-96f7-8fab7e245b9e                     |
| image                                | cirros.qcow2 (02718418-8966-4003-9990-a7bf3d6215fe)      |
| key_name                             | -                                                        |
| metadata                             | {}                                                       |
| name                                 | vm-port-security                                         |
| os-extended-volumes:volumes_attached | []                                                       |
| progress                             | 0                                                        |
| security_groups                      | default                                                  |
| status                               | ACTIVE                                                   |
| tenant_id                            | c5cb88bd612949a5afaed8acf79350ef                         |
| test-port-security network           | 192.168.1.3                                              |
| updated                              | 2015-12-02T11:35:42Z                                     |
| user_id                              | a657b7aa0a474dfc84e30aacef7b92bf                         |
+--------------------------------------+----------------------------------------------------------+

We can see VM is launched on overcloud-compute-1, with IP 192.168.1.3.

Check this port's details:

[root@overcloud-controller-0 ~]# neutron port-list| grep 192.168.1.3
| bbf4189d-e56b-44bf-adf5-bce08912c6fd |          | fa:16:3e:af:b7:f0 | {"subnet_id": "67f53d37-9d43-461c-b072-0f174074e57e", "ip_address": "192.168.1.3"} |

[root@overcloud-controller-0 ~]# neutron port-show bbf4189d-e56b-44bf-adf5-bce08912c6fd | grep 'mac\|port_security'
| mac_address           | fa:16:3e:af:b7:f0                                                                  |
| port_security_enabled | True

We can see this port's port_security_enabled is True(takes from the flag from its network), and with mac fa:16:3e:af:b7:f0.

Now let's go to overcloud-compute-1 node to check iptables rules for this port. First we need to know what's tap device name of this port uses:

[root@overcloud-compute-1 ~]# virsh domiflist instance-00000001
Interface  Type       Source     Model       MAC  
-------------------------------------------------------
tapbbf4189d-e5 bridge     qbrbbf4189d-e5 virtio      fa:16:3e:af:b7:f0  

Check all iptables rules related to this port:

[root@overcloud-compute-1 ~]# iptables-save | grep bbf4189d
:neutron-openvswi-ibbf4189d-e - [0:0]
:neutron-openvswi-obbf4189d-e - [0:0]
:neutron-openvswi-sbbf4189d-e - [0:0]
-A neutron-openvswi-FORWARD -m physdev --physdev-out tapbbf4189d-e5 --physdev-is-bridged -m comment --comment "Direct traffic from the VM interface to the security group chain." -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-in tapbbf4189d-e5 --physdev-is-bridged -m comment --comment "Direct traffic from the VM interface to the security group chain." -j neutron-openvswi-sg-chain
-A neutron-openvswi-INPUT -m physdev --physdev-in tapbbf4189d-e5 --physdev-is-bridged -m comment --comment "Direct incoming traffic from VM to the security group chain." -j neutron-openvswi-obbf4189d-e
-A neutron-openvswi-ibbf4189d-e -m state --state INVALID -m comment --comment "Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack." -j DROP
-A neutron-openvswi-ibbf4189d-e -m state --state RELATED,ESTABLISHED -m comment --comment "Direct packets associated with a known session to the RETURN chain." -j RETURN
-A neutron-openvswi-ibbf4189d-e -s 192.168.1.2/32 -p udp -m udp --sport 67 --dport 68 -j RETURN
-A neutron-openvswi-ibbf4189d-e -m set --match-set NETIPv469ca31c5-4567-4432-b src -j RETURN
-A neutron-openvswi-ibbf4189d-e -m comment --comment "Send unmatched traffic to the fallback chain." -j neutron-openvswi-sg-fallback
-A neutron-openvswi-obbf4189d-e -p udp -m udp --sport 68 --dport 67 -m comment --comment "Allow DHCP client traffic." -j RETURN
-A neutron-openvswi-obbf4189d-e -j neutron-openvswi-sbbf4189d-e
-A neutron-openvswi-obbf4189d-e -p udp -m udp --sport 67 --dport 68 -m comment --comment "Prevent DHCP Spoofing by VM." -j DROP
-A neutron-openvswi-obbf4189d-e -m state --state INVALID -m comment --comment "Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack." -j DROP
-A neutron-openvswi-obbf4189d-e -m state --state RELATED,ESTABLISHED -m comment --comment "Direct packets associated with a known session to the RETURN chain." -j RETURN
-A neutron-openvswi-obbf4189d-e -j RETURN
-A neutron-openvswi-obbf4189d-e -m comment --comment "Send unmatched traffic to the fallback chain." -j neutron-openvswi-sg-fallback
-A neutron-openvswi-sbbf4189d-e -s 192.168.1.3/32 -m mac --mac-source FA:16:3E:AF:B7:F0 -m comment --comment "Allow traffic from defined IP/MAC pairs." -j RETURN
-A neutron-openvswi-sbbf4189d-e -m comment --comment "Drop traffic without an IP/MAC allow rule." -j DROP
-A neutron-openvswi-sg-chain -m physdev --physdev-out tapbbf4189d-e5 --physdev-is-bridged -m comment --comment "Jump to the VM specific chain." -j neutron-openvswi-ibbf4189d-e
-A neutron-openvswi-sg-chain -m physdev --physdev-in tapbbf4189d-e5 --physdev-is-bridged -m comment --comment "Jump to the VM specific chain." -j neutron-openvswi-obbf4189d-e

We can see a lot of anti-spoofing rules configured, for example:

-A neutron-openvswi-obbf4189d-e -p udp -m udp --sport 67 --dport 68 -m comment --comment "Prevent DHCP Spoofing by VM." -j DROP

This rule blocks any DHCP offer getting out from the VM, thus DHCP server running inside the VM can NOT work.

How to disable port security

There are 2 ways to disable port security, per network and per port.

Disable port security per port

We can disable the port security on the fly even when the port is already attached to a VM. Let's try this on the VM we created above.

To disable port security for this port, we need remove the security group for it first:

neutron port-update --no-security-groups  bbf4189d-e56b-44bf-adf5-bce08912c6fd  

Then we can now disable the port security:

[root@overcloud-controller-0 ~]# neutron port-update bbf4189d-e56b-44bf-adf5-bce08912c6fd --port-security-enabled=False
Updated port: bbf4189d-e56b-44bf-adf5-bce08912c6fd

[root@overcloud-controller-0 ~]# neutron port-show bbf4189d-e56b-44bf-adf5-bce08912c6fd | grep 'port_security'
| port_security_enabled | False

We can see now the port security is disable for this port. Let's go to compute node to check the iptables rules of this port:

[root@overcloud-compute-1 ~]# iptables-save | grep bbf4189d
-A neutron-openvswi-FORWARD -m physdev --physdev-out tapbbf4189d-e5 --physdev-is-bridged -m comment --comment "Accept all packets when port security is disabled." -j ACCEPT
-A neutron-openvswi-FORWARD -m physdev --physdev-in tapbbf4189d-e5 --physdev-is-bridged -m comment --comment "Accept all packets when port security is disabled." -j ACCEPT
-A neutron-openvswi-INPUT -m physdev --physdev-in tapbbf4189d-e5 --physdev-is-bridged -m comment --comment "Accept all packets when port security is disabled." -j ACCEPT

Ok, now all anti-spoofing rules are gone, only "ACCEPT ALL" rules in place.

Disable port security per network

When we create a network, port security is enable by default, all ports created in this network will have port security enabled by default.

This can be changed by specifying port_security_enabled to False when creating the network:

[root@overcloud-controller-0 ~]# neutron net-create test-port-security-disable -- --port_security_enabled=False
Created a new network:  
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| id                        | f035c5d8-40eb-4bda-8897-66ff3d9a0392 |
| mtu                       | 0                                    |
| name                      | test-port-security-disable           |
| port_security_enabled     | False                                |
| provider:network_type     | vxlan                                |
| provider:physical_network |                                      |
| provider:segmentation_id  | 8                                    |
| router:external           | False                                |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tenant_id                 | c5cb88bd612949a5afaed8acf79350ef     |
+---------------------------+--------------------------------------+

Then let's create a subnet:

neutron subnet-create test-port-security-disable 172.28.0.0/24  

Now let's launch a VM in this network:

nova boot  --flavor m1.small --image cirros.qcow2 --nic net-id=f035c5d8-40eb-4bda-8897-66ff3d9a0392 vm-port-security-disabled  

We will see the VM gets ERROR state because we hit this bug:

From compute node, we could see following trace log:

[root@overcloud-compute-0 ~]# grep 92e83f34-c17d-4b37-815c-e93bd67c6eee /var/log/nova/nova-compute.log | grep " Security"
2015-12-06 20:08:37.852 17015 TRACE nova.compute.manager [instance: 92e83f34-c17d-4b37-815c-e93bd67c6eee] SecurityGroupCannotBeApplied: Network requires port_security_enabled and subnet associated in order to apply security groups.  

Nova always tries to add default security group to port of instance even if port has port security disabled, in this case, security group applying fails, further causing instance launching fails.

Heat support

port_security_enabled attribute support for OS::Neutron::Net and OS::Neutron::Port in Heat are added in Liberty, not in Kilo.