Using VLANs with Virtualbox

After a number of hours wasted trying to figure out why my VLANs stopped working, it's probably a good idea to make note of what I found.

I have a lot of different Virtualbox VMs created that I use for testing different things. A while back I created a VM image to act as a PPPoE server (Debian squeeze guest running on a lenny host). For reasons that made sense to me at the time, I created VLANs on the guest operating system rather than on the host system (and attaching virtual network adaptors to the guest VM). I hadn't used this VM in several months until today; when I fired up the VM for some PPPoE testing. Unsurprisingly, the VM didn't have any network connectivity on boot. I figured it would probably take some finagling to get things working exactly the way they were working before. However the topology was correct, the switch set up properly, and still no PPPoE.

After several hours of screwing around with different VirtualBox versions, different combinations of host/guest VLAN configurations, etc. I identified the following behaviors:

1. On a clean reboot of the host, the VLANs on the virtual machine work fine.
2. As soon as I add a VLAN to the ethernet interface on the host, connectivity to the guest fails.
3. If I remove the VLAN from the host, guest connectivity is restored.

The wheels started spinning and I remembered that at some point in the past I had a bridge interface configured on my host machine. I had recently removed the bridge in order to simplify my configuration. Some quick testing confirmed that when my host machine is configured with a virtual bridge interface, VLANs on the host can happily coexist with VLANs on the guest. My network configuration necessary to set this up is as follows.

1. Add a bridge interface br0 to the host.
2. Add ethernet adaptor eth0 to bridge br0.
3. Add the host VLAN interfaces to bridge br0.
   
4. Configure appropriate IP addressing for the host to br0.
5. Attach the VirtualBox VM network interface to eth0.

In this configuration, both the host and the guest are able to create tagged VLAN interfaces without conflicting.

Did your Kerberos authenticated NFS mounts all just break?

Here's why. A recent update to the krb5 packages disabled weak ciphers, DES in particular. I'm all for stronger security, but when it breaks my system, I get a bit crabby. I was the dummy here, I saw there message during the update process about possible NFS breakage due to the weak ciphers being disabled. I ignored the messages because I was in a hurry and I figured it would be a two minute fix. An hour and a half later I now have it fixed. Here is a link to the bug report that helped me. The fix is to re-enable the weak ciphers in your /etc/krb5.conf filein the libdefaults section.

allow_weak_crypto = true

OpenLDAP in Ubuntu Karmic Koala (9.10)

I've decided to try Ubuntu again because she's prettier than her big sister Debian and I had the naive hope that she might have finally gotten some professional help and a big dose of the crazy pills she needed. I was wrong, once you get past the sexy exterior she's still the same old pile of crazy that she was before.

I've been trying to set up a bunch of services and the documentation is generally useless since it's usually incomplete or so wrong that I wonder if the documentation team is a bunch of chimps flinging poo. So there will be a number of posts here to fill in some of the blanks. Ubuntu doc team, if you read this, please fix the docs rather than just cut and paste the manual from the last release.

First up, LDAP! FWIW, here is the original doc.

Getting it running

OpenLDAP in Ubuntu Karmic Koala (9.10) is not installed as described in the Ubuntu community documentation. Among its many detestable qualities, you are not prompted for the initial directory configuration during the dpkg run, and the way LDAP is setup is different from my (admittedly sketchy) recollection of how it was previously set up.

There is some documentation in forums on getting LDAP working properly, some of it good, some of it not. The post that got OpenLDAP running (properly?) for me is here. At the very least it was enough to allow me to get access to the database with PHPLDAPAdmin.

Creating SSL/TLS Certificates for LDAP

The official SSL Certificate document is surprisingly good. I followed it to set up a CA and create certificates for LDAP with pretty good results. There was only important thing that was not mentioned in the doc:

• Take care to make sure that the correct permissions are set on private keys. These keys should not be readable by any user other than absolutely necessary for security.
  

The section of the LDAP documentation that describes setting up certs does the trick, but there are a couple things I recommend changing.

• The document instructs you to add the openldap user to the ssl-cert group. I don't like this because it means that an attacker that compromises openldap can now read any private SSL keys also owned by ssl-cert (Apache?, OpenVPN?, could be bad). As an alternative, I used ACLs to give the openldap user access to the one specific file that it needs access to.
• The Common Name field on your LDAP certificate should match the hostname of the server that any LDAP clients connect to; otherwise the LDAP clients will freak out.
• Any LDAP clients need to be given a copy of the CA cert and instructed to use it. The following line should be added to the /etc/ldap/ldap.conf file and changed as appropriate to point at the CA certificate.

    TLS_CACERT /usr/local/etc/openldap/cacert.pem

• If you are having difficulty testing to see if your SSL LDAP is working properly, try the following command, substituting your LDAP information.

  ldapsearch -x -ZZ -d -1 -b 'dc=example,dc=com' \
  -D 'cn=user,dc=example,dc=com' -h ldap.example.com -W

• The section that discusses adding additional schemas to OpenLDAP is accurate, but not very clear. Step 3 uses the slapcat command to convert schema files to LDIF format. Unless you want to import the misc schema only, the schema files that you actually want are the ones in the ldif_output directory. Change the first line to the format

  dn: cn=SCHEMA_NAME,dc=schema,dc=config

  and the third line to

  cn: SCHEMA_NAME

  Remove the lines at the bottom as instructed, then import the schema file as instructed (using the correct file name of course).
  

Howto get your windows boot option back after grub2/Squeeze nukes it!

I hate bootloaders, they either work fine, or they're a royal pain. And just when I was really starting to understand grub, they go ahead and release grub2. I mistakenly upgraded to grub2 on my desktop a while ago, and learned that grub2 has a tendency to forget about Windows. This was irritating, but not the end of the world, I just didn't use windows for almost a year.

More recently, Debian Squeeze didn't give me much of a choice about installing grub2, and promptly disappeared Windows on my notebook. This got me doing a little more digging to figure out why a "new and improved" version of grub would actually be less capable. Could this be some stallmanesque scheme to force me into the freedom orgy?

Nope, something far less insidious. Apparently there is now an extra package that grub2 requires to find operating systems besides linux. That package is os-prober. So if you've pwned your windows install like I did, just do an 'apt-get install os-prober', then run 'grub-update'. Your Windows option will magically reappear!

Cisco ICMP6 strange behavior on neighbor solicitations.

I noticed some strange behavior while I was playing with IPv6 on my Cisco lab. I have a Debian server hanging off one of the router interfaces acting as a name server. I found that I could ping and resolve against the name server from the router that is directly attached. However, if I try to ping or resolve from a router further away I get no luck. After I corrected the prefix length on the name servers route to the lab network (off by one bit) I still had no luck; even though I could ping another Debian box on the same subnet without a problem. A little investigation with tcpdump yielded this:

11:23:19.360378 IP6 fe80::215:63ff:fe56:cf1d > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has 2001:4978:34a::1, length 32
11:23:22.315487 IP6 2001:4978:34a::ff > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has 2001:4978:34a::1, length 32
11:23:22.315645 IP6 2001:4978:34a::1 > 2001:4978:34a::ff: ICMP6, neighbor advertisement, tgt is 2001:4978:34a::1, length 32

What we have here are two NDP Neighbor Solicitation messages. Both messages come from the edge router on the Cisco lab. The first message was sent when I pinged from inside the lab network; the second was sent when I pinged from the edge router directly. The difference is that when the edge router needed the name server MAC address for the routed packet it used its link local address. When it needed it for its own packet it used the global unicast packet on the interface. Why does this make a difference? Because the default INPUT policy on the name server ip6tables is DROP, and I had not thought to explicitly allow the link local subnet, causing the first neighbor solicitation to be dropped.

Repeating the experiment, this time pinging from the edge router first, both routers' pings succeed, since by the time the inside routers packets come through there is already a neighbor/ARP entry.

I'm not sure why Cisco would do things this way, maybe there is a logical reason. Maybe it is a bug and no one does goofy things like I do so it has gone unnoticed. Maybe it has been corrected in a more recent version of firmware (the edge router runs 1841 Software (C1841-ADVIPSERVICESK9-M), Version 12.4(12)).

Converting Debian from a single disk to RAID1

I've needed this often enough and stumbled through it; here's a link to a guide for converting a Debian install to a software RAID1 install.

http://www.ctunion.com/node/178

I used it for tips after I was already halfway through, so I don't know how accurate the guide is as a whole. The section dealing with creating a new initrd was close enough for me, but they refer to mkinitrd instead of mkinitramfs that is included in the squeeze initramfs-tools package.

SNMP v3 Configuration for non-SNMP-magi

Initial SNMP v3 Configuration

This works on Debian (and presumably Ubuntu). YMMV for other distros.

Start off with the following configuration in /etc/snmp/snmpd.conf

# create user 'initial' with auth pass 'setuppass'. This is used for the initial setup, then removed.
createUser initial MD5 setupauth DES setuppriv
# add 'initial' to group 'MyRWGroup' using the USM security model.
group   MyRWGroup       usm             initial
# create a view 'viewall' that includes the entire OID tree.
view    viewall included        .1
# tie it all together. allow read and write access to view 'viewall' by users in group 'MyRWGroup'
access  MyRWGroup       ""      usm             authNoPriv              exact   viewall viewall none

Once this is done, restart snmpd and you should be able to test connectivity by walking the OID tree

snmpwalk -v3 -l authNoPriv -u initial -A setupauth localhost .

Assuming all is well so far, remove the 'createUser' line from the configuration file and restart snmpd. This prevents snmpd from recreating the 'initial' user every time is starts. You can now use the snmpusm command to create a new user 'cacti'

snmpusm -v3 -l authNoPriv -u initial -A setupauth localhost create cacti initial

Now use snmpvacm to create a group 'MyROGroup' and 'cacti'. Then create a new view 'allseeing' that can see the entire tree (this is actually the same as what is in the configuration file). Finally, create an access entry allowing 'MyROGroup' group members using USM (3) to connect via authNoPriv (2) using an exact context match (1). The 'allseeing' view is specified for reads, and 'none' is specified for writes and notifies.

snmpvacm -v3 -l authNoPriv -u initial -A setupauth localhost createSec2Group 3 cacti MyROGroup
snmpvacm -v3 -l authNoPriv -u initial -A setupauth localhost createView allseeing .1 80
snmpvacm -v3 -l authNoPriv -u initial -A setupauth localhost createAccess MyROGroup 3 2 1 allseeing none none

At this point you should be able to walk as cacti

snmpwalk -v3 -l authNoPriv -u cacti -A setupauth localhost .

You can also change cacti's Authentication password, but not as cacti. The reason for this is that 'cacti' currently only has read access to the tree. The initial user currently has the necessary write access to the tree. The Privacy password is also changed

snmpusm -v3 -l authNoPriv -u cacti -A setupauth localhost passwd -Ca setupauth authpass
Error in packet.
Reason: noAccess
snmpusm -v3 -l authNoPriv -u initial -A setupauth localhost passwd -Ca setupauth authpass cacti
SNMPv3 Key(s) successfully changed.
snmpusm -v3 -l authNoPriv -u initial -x DES -A setupauth localhost passwd -Cx setupauth privpass cacti
SNMPv3 Key(s) successfully changed.

Now we want to get rid of the 'initial' user. Before we can delete him we should create an administrative user that can make changes to the tree.

snmpusm -v3 -l authNoPriv -u initial -x DES -A setupauth localhost create admin initial
snmpusm -v3 -l authNoPriv -u initial -x DES -A setupauth localhost passwd setupauth secretpass admin
snmpvacm -v3 -l authNoPriv -u initial -A setupauth localhost createSec2Group 3 admin rwgrp
snmpvacm -v3 -l authNoPriv -u initial -A setupauth localhost createAccess rwgrp 3 2 1 allseeing allseeing none
snmpusm -v3 -l authNoPriv -u admin -a MD5 -n "" -A secretpass localhost delete initial

Some Notes

* When I tried creating access for rwgrp with 'priv' permissions I got timeouts (Timeout: No Response from localhost) from the daemon when trying to operate as admin. Not sure why, and the Internet was completely unhelpful in figuring out why.
* Once the above setup is complete, you can remove all of the initial configuration from the snmpd.conf file and restart the daemon. All the use information is stored in /var/lib/snmp/snmpd.conf, leaving a clean-looking /etc/snmp/snmpd.conf file.
* This is probably not the most secure implementation ever, you may want to do some more research and lock down the agent further.

Useful Resources
snmpvacm man page
snmpusm man page