Ubuntu 14.04 Trusty with DNS server and NetworkManager (disabling dnsmasq)

I’ve badly been looking for a way to both run an internal network server on eth0 and connecting to any kind of internet device, be it on wlan0 or a USB dongle.

That way, I can be online and browsing documentation, downloading new stuff etc. on the server but not depend on a static configuration but still use Network Manager for its intended purpose.

In /etc/NetworkManager/NetworkManager.conf, uncomment the dnsmasq option like so, because Network Manager’s dnsmasq blocks listening to these ports if you want to run your own DNS server:

# dns=dnsmasq

But that’s not all! We need /etc/resolv.conf to be updated with the external DNS providers that Network Manager discovers. This can be achieved by removing resolvconf which automatically alters /etc/resolv.conf.

apt-get remove resolvconf

There, done! Now add static configurations in /etc/networks/interfaces.d/ or /etc/networks/interfaces and let Network Manager handle your WLAN interfaces.

Ubuntu, Kickstart, preseeding, and Wireless (WLAN)

I have been trying to get an automated network install running. The starting point is a Kickstart (Kickseed) setup that works fine on LAN. Problem is: Computers do not do PXE netbooting from WLAN, and secondly that the debian installer was not configuring the network correctly. But the steps are very simple.

  1. Get mini.iso, basically the same kernel and initrd that the netbooter is running
  2. Put it on a usb drive to boot the computer (laptop) from: sudo dd if=/path/to/mini.iso of=/dev/[usb device] bs=4096
  3. Make sure that your wireless access point is not protected, just make an open one, we call it UBUNTU-INSTALL
  4. Test that you can connect to it and that the repository is working
  5. Now, boot from the USB drive and press TAB to add boot options: ks=http://1.2.3.4/ks.cfg ksdevice=wlan0 netcfg/wireless_essid=UBUNTU-INSTALL
  6. In order to add your changes to mini.iso, you need to unpack the whole filesystem and edit txt.cfg which is a file that syslinux is using to display the menu.

Changing mini.iso

Tools like UCK do not work for the mini.iso file because it’s not using stuff like Casper. So you need to create a custom boot image the hard way:

$ mkdir iso
$ sudo mount -o loop -t iso9660 mini.iso iso
$ cp -R iso custom
$ sudo chmod 600 custom/txt.cfg
$ sudo nano custom/txt.cfg
$ sudo mkisofs -r -V "Custom Ubuntu Netboot image" -cache-inodes -J -l -b isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -o custom.iso custom-iso
$ sudo umount /media/USB-DRIVE # Remember to unmount before writing the usb
$ sudo dd if=custom.iso of=/dev/[YOU USB DRIVE] bs=4096

Limiting choices in a ModelAdmin list_filter

So, you need to limit choices, because the list of related choices is too long? Then do NOT use limit_choices_to parameter on the ForeignKey! Why? Because you risk having options missing in your forms, deleting relations unknowingly.

For instance, consider that you have the following case, expressed in pseudo code:

ModelClass.related_instance = models.ForeignKey(MyRelatedModel, limit_choices_to = {'active': True}, blank=True, null=True)
object = <ModelClass instance>
object.related_instance = <MyRelatedModel instance>
object.related_instance.active = False

If you edit object, you will see an admin form where <MyRelatedModel instance> is not selectable! Hence, you will risk editing and loosing data without knowing it.

So your best bet is to stop using limit_choices_to and instead define a filter. Luckily, it’s quite easy. It’s not beautiful, though, since admin.filters.RelatedFieldListFilter does not offer a method to overwrite.

Here is an example:

class RelatedFilter(admin.filters.RelatedFieldListFilter):
    def __init__(self, *args, **kwargs):
        super(RelatedFilter, self).__init__(*args, **kwargs)
        self.lookup_choices = [
            (x.id, x) for x in
                models.RelatedModel.objects.filter(status='active')
        ]
 
class MyModelAdmin(admin.ModelAdmin):
 
    list_filter = (('related_field', RelatedFilter), 'other_related_field')

Testing and tracking down Warnings in Django

Warnings are often suppressed through many layers of control mechanisms in Python and Django. However, you should really be aware of these and clean them up once in a while!

In other cases, the Warnings reflect an inconsistent state, such as if you are mixing naive and timezone-aware variables. It can be a real pain to track down RuntimeWarning because it doesn’t leave a nice stack trace. The scenario could be that you have assigned a naive DateTime to a field, and the db layer is complaining at runtime.

However, you can run python in a mode where Warning-types raise exceptions with full stack traces. Use the following syntax:

python -W error:"":RuntimeWarning:django.db.models.fields:0 manage.py runserver
  • ‘error’ means that python should raise an exception – this is what we want!
  • ‘RuntimeWarning’ means to trigger when such is raised – you can also simply put ‘Warning’ to trigger for all Warning types.
  • ‘django.db.models.fields’ means activate for this module (you need the full path). Notice that simply putting ‘django’ does *not* activate for all of django, you need to put the specific module or simply ‘””‘ to enable for all of Python (which might render some interesting Warning goodies that you have never seen!).
  • ’0′ means any line (you probably don’t need)

You can read more in Python Documentation chapter: Warning Control.

Django: Log and block brute force attempts

Here is a very simple mechanism for wrapping a decorator around your views to protect them against brute force attempts. For instance, if you have a secret file download available only with the right secret (/view/id/secret-hash/), you expose your view to simple brute force attempts.

Simply put, this decorator will log a 404 response object or Http404 exception, count pr. IP and return status=400 and send you an email whenever a new block is put into place.

Model

class IllegalLookup(LogModifications):
    """Log and block illegal lookups"""
    created = models.DateTimeField(
        verbose_name=_(u'created'),
        auto_now_add = True,
    )
    modified = models.DateTimeField(
        verbose_name=_(u'created'),
        auto_now = True,
    )
    ip_address = models.CharField(
        max_length=16,
        null=True,
        blank=True,
        verbose_name=_(u'IP address'),
    )
    path = models.CharField(
        max_length=255,
        null=True,
        blank=True,
        verbose_name=_(u'path'),
        help_text=_(u'First attempted path is always logged'),
    )
    count = models.PositiveIntegerField(
        default=1,
    )
 
    @classmethod
    def log_lookup(cls, ip_address, path):
        try:
            now = timezone.now()
            expired = now - timedelta(minutes=settings.BLOCK_EXPIRY)
            lookup = cls.objects.get(ip_address=ip_address,
                modified__gte=expired)
            lookup.count += 1
            lookup.save()
        except cls.DoesNotExist:
            # Delete old entries first
            cls.objects.filter(ip_address=ip_address).delete()
            lookup = cls.objects.create(ip_address=ip_address,
                path=path)
            lookup.save()
 
    @classmethod
    def is_blocked(cls, ip_address):
        try:
            now = timezone.now()
            expired = now - timedelta(minutes=settings.BLOCK_EXPIRY)
            lookup = cls.objects.get(ip_address=ip_address,
                modified__gte=expired)
            if lookup.count == settings.BLOCK_ATTEMPTS:
                mail_admins("IP blocked", "{0} is now blocked, IllegalLookup id: {1}".format(ip_address, lookup.id))
            if lookup.count > settings.BLOCK_ATTEMPTS:
                return True
        except cls.DoesNotExist:
            pass
        return False

Decorator

def log_and_block(func):
 
    def _log_and_block(request, *args, **kwargs):
        remote_ip = request.META.get('REMOTE_ADDR', None)
        if IllegalLookup.is_blocked(remote_ip):
            return HttpResponse('%s is blocked' % remote_ip,
                status=400)
 
        is_404 = False
        is_exception = False
        try:
            return_object = func(request, *args, **kwargs)
            if return_object.status_code == 404:
                is_404 = True
        except Http404:
            is_404 = True
            is_exception = True
 
        if is_404:
            if remote_ip:
                IllegalLookup.log_lookup(remote_ip,
                    request.META.get('PATH_INFO', ''))
 
            if is_exception:
                raise
 
        return return_object
 
    return _log_and_block

Usage

Now, simply wrap the decorator around your view:

@log_and_block
def my_view(request, id, secret_hash):
    object = get_object_or_404(models.MyModel, id=id, secret_hash=secret_hash)

101% objective… always