API documentation

The following documentation is based on the source code of version 0.7 of the linux-utils package.

Available modules

linux_utils

Linux system administration tools for Python.

The linux_utils module contains generic functions to be used by the other modules in the package.

linux_utils.coerce_context(value)

Coerce the given value to an execution context.

Parameters:value – The value to coerce (an execution context created by executor.contexts or None). Returns:An execution context created by executor.contexts. Raises:ValueError when value isn’t None but also isn’t a valid execution context.

This function is used throughout the modules in the linux_utils package to abstract away the details of external command execution using dependency injection:

• If no execution context is provided (value is None) executor.contexts.LocalContext is used by default.
• Callers can provide executor.contexts.RemoteContext in which case the linux_utils package seamlessly adapts to command execution on a remote system over ssh.

linux_utils.coerce_device_file(expression)

Coerce a device identifier to a device file.

Parameters:expression – The device identifier (a string). Returns:The pathname of the device file (a string). Raises:ValueError when an unsupported device identifier is encountered.

If you pass in a LABEL="..." or UUID=... expression (as found in e.g. /etc/fstab) you will get back a pathname starting with /dev/disk/by-label or /dev/disk/by-uuid:

>>> from linux_utils import coerce_device_file
>>> print(coerce_device_file('LABEL="Linux Boot"'))
/dev/disk/by-label/Linux\x20Boot
>>> print(coerce_device_file('UUID=7801a1c2-7ad7-4c0b-9fbb-2a47ae802f71'))
/dev/disk/by-uuid/7801a1c2-7ad7-4c0b-9fbb-2a47ae802f71

If expression is already a pathname it will pass through untouched:

>>> coerce_device_file('/dev/mapper/backups')
'/dev/mapper/backups'

Unsupported device identifiers raise an exception:

>>> coerce_device_file('PARTUUID=e6c021cc-d0d8-400c-8f5c-b10adeff65fe')
Traceback (most recent call last):
  File "linux_utils/__init__.py", line 90, in coerce_device_file
    raise ValueError(msg % expression)
ValueError: Unsupported device identifier! ('PARTUUID=e6c021cc-d0d8-400c-8f5c-b10adeff65fe')

linux_utils.coerce_size(value)

Coerce a human readable data size to the number of bytes.

Parameters:value – The value to coerce (a number or string). Returns:The number of bytes (a number). Raises:ValueError when value isn’t a number or a string supported by parse_size().

linux_utils.atomic

Atomic filesystem operations for Linux in Python.

The most useful functions in this module are make_dirs(), touch(), write_contents() and write_file().

The copy_stat() and get_temporary_file() functions were originally part of the logic in write_file() but have since been extracted to improve the readability and reusability of the code.

linux_utils.atomic.copy_stat(filename, reference=None, mode=None, uid=None, gid=None)

The Python equivalent of chmod --reference && chown --reference.

Parameters:

• filename – The pathname of the file whose permissions and ownership should be modified (a string).
• reference – The pathname of the file to use as reference (a string or None).
• mode – The permissions to set when reference isn’t given or doesn’t exist (a number or None).
• uid – The user id to set when reference isn’t given or doesn’t exist (a number or None).
• gid – The group id to set when reference isn’t given or doesn’t exist (a number or None).

linux_utils.atomic.get_temporary_file(filename)

Generate a non-obtrusive temporary filename.

Parameters:filename – The filename on which the name of the temporary file should be based (a string). Returns:The filename of a temporary file (a string).

This function tries to generate the most non-obtrusive temporary filenames:

1. The temporary file will be located in the same directory as the file to replace, because this is the only location somewhat guaranteed to support “rename into place” semantics (see write_file()).
2. The temporary file will be hidden from directory listings and common filename patterns because it has a leading dot.
3. The temporary file will have a different extension then the file to replace (in case of filename patterns that do match dotfiles).
4. The temporary filename has a decent chance of not conflicting with temporary filenames generated by concurrent processes.

linux_utils.atomic.make_dirs(directory, mode=511)

Create a directory if it doesn’t already exist (keeping concurrency in mind).

Parameters:directory – The pathname of a directory (a string). Returns:True if the directory was created, False if it already existed. Raises:Any exceptions raised by os.makedirs().

This function is a wrapper for os.makedirs() that swallows OSError in the case of EEXIST.

linux_utils.atomic.touch(filename)

The equivalent of the touch program in Python.

Parameters:filename – The pathname of the file to touch (a string).

This function uses make_dirs() to automatically create missing directory components in filename.

linux_utils.atomic.write_contents(filename, contents, encoding='UTF-8', mode=None)

Atomically create or update a file’s contents.

Parameters:

• filename – The pathname of the file (a string).
• contents – The (new) contents of the file (a byte string or a Unicode string).
• encoding – The text encoding used to encode contents when it is a Unicode string.
• mode – The permissions to use when the file doesn’t exist yet (a number like accepted by os.chmod() or None).

linux_utils.atomic.write_file(*args, **kwds)

Atomically create or update a file (avoiding partial reads).

Parameters:

• filename – The pathname of the file (a string).
• mode – The permissions to use when the file doesn’t exist yet (a number like accepted by os.chmod() or None).

Returns:

A writable file object whose contents will be used to create or atomically replace filename.

linux_utils.cli

Command line interface for linux_utils.luks.

The linux_utils.cli module implements command line interfaces for the cryptdisks_start() and cryptdisks_stop() functions.

linux_utils.cli.cryptdisks_start_cli()

Usage: cryptdisks-start-fallback NAME

Reads /etc/crypttab and unlocks the encrypted filesystem with the given NAME.

This program emulates the functionality of Debian’s cryptdisks_start program, but it only supports LUKS encryption and a small subset of the available encryption options.

linux_utils.cli.cryptdisks_stop_cli()

Usage: cryptdisks-stop-fallback NAME

Reads /etc/crypttab and locks the encrypted filesystem with the given NAME.

This program emulates the functionality of Debian’s cryptdisks_stop program, but it only supports LUKS encryption and a small subset of the available encryption options.

linux_utils.crypttab

Parsing of /etc/crypttab configuration files.

The cryptsetup program is used to manage LUKS based full disk encryption and Debian provides some niceties around cryptsetup to make it easier to use, specifically:

• The /etc/crypttab configuration file contains static information about encrypted filesystems and enables unlocking of encrypted filesystems when the system is booted. Refer to the crypttab man page for more information.
• The cryptdisks_start and cryptdisks_stop commands can be used to manually unlock encrypted filesystems that are configured in /etc/crypttab with the noauto option (meaning the device is ignored during boot).

linux_utils.crypttab.parse_crypttab(filename='/etc/crypttab', context=None)

Parse the Debian Linux configuration file /etc/crypttab.

Parameters:

• filename – The absolute pathname of the file to parse (a string, defaults to /etc/crypttab).
• context – See coerce_context() for details.

Returns:

A generator of EncryptedFileSystemEntry objects.

Here’s an example:

>>> from linux_utils.crypttab import parse_crypttab
>>> print(next(parse_crypttab()))
EncryptedFileSystemEntry(
    configuration_file='/etc/crypttab',
    line_number=3,
    target='ssd',
    source='UUID=31678141-3931-4683-a4d2-09eadec81d01',
    source_device='/dev/disk/by-uuid/31678141-3931-4683-a4d2-09eadec81d01',
    key_file='none',
    options=['luks', 'discard'],
)

Changed in version 0.6: It is not an error when filename doesn’t exist, because of my experience that /etc/crypttab doesn’t exist in default Debian and Ubuntu installations (unless that system was specifically set up with root disk encryption using the installation wizard). This used to raise an exception, but this was changed in release 0.6.

class linux_utils.crypttab.EncryptedFileSystemEntry(**kw)

An entry parsed from /etc/crypttab.

Each entry in the crypttab file has four fields, these are mapped to the following properties:

1. target
2. source
3. key_file
4. options

Refer to the crypttab man page for more information about these fields. The computed properties is_available, is_unlocked and source_device are based on the parsed values of the four fields above.

Here’s an overview of the EncryptedFileSystemEntry class:

Superclass:	TabFileEntry
Properties:	is_available, is_unlocked, key_file, options, source, source_device, target and target_device

is_available

True if source_device exists, False otherwise.

is_unlocked

True if target_device exists, False otherwise.

key_file

The file to use as a key for decrypting the data of the source device (a string or None).

When the key file field in /etc/crypttab is set to none the value of this property will be None, this makes checking whether an encrypted filesystem has a key file configured much more Pythonic.

options

The encryption options for the filesystem (a list of strings).

Note

The options property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

source

The block special device or file that contains the encrypted data (a string).

The value of this property may be a UUID=... expression instead of the pathname of a block special device or file.

source_device

The block special device or file that contains the encrypted data (a string).

The value of this property is computed by passing source to coerce_device_file().

Note

The source_device property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

target

The mapped device name (a string).

target_device

The absolute pathname of the device file corresponding to target (a string).

Note

The target_device property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

linux_utils.fstab

Parsing of /etc/fstab configuration files.

linux_utils.fstab.find_mounted_filesystems(filename='/proc/mounts', context=None)

Get information about mounted filesystem from /proc/mounts.

Parameters:

• filename – The absolute pathname of the file to parse (a string, defaults to /proc/mounts).
• context – See coerce_context() for details.

Returns:

A generator of FileSystemEntry objects.

This function is a trivial wrapper for parse_fstab() that instructs it to parse /proc/mounts instead of /etc/fstab. Here’s an example:

>>> from humanfriendly import format_table
>>> from linux_utils.fstab import find_mounted_filesystems
>>> print(format_table(
...    data=[
...        (entry.mount_point, entry.device_file, entry.vfs_type)
...        for entry in find_mounted_filesystems()
...        if entry.vfs_type not in (
...            # While writing this example I was actually surprised to
...            # see how many `virtual filesystems' a modern Linux system
...            # has mounted by default (based on Ubuntu 16.04).
...            'autofs', 'cgroup', 'debugfs', 'devpts', 'devtmpfs', 'efivarfs',
...            'fuse.gvfsd-fuse', 'fusectl', 'hugetlbfs', 'mqueue', 'proc',
...            'pstore', 'securityfs', 'sysfs', 'tmpfs',
...        )
...    ],
...    column_names=["Mount point", "Device", "Type"],
... ))
---------------------------------------------------
| Mount point  | Device                    | Type |
---------------------------------------------------
| /            | /dev/mapper/internal-root | ext4 |
| /boot        | /dev/sda5                 | ext4 |
| /boot/efi    | /dev/sda1                 | vfat |
| /mnt/backups | /dev/mapper/backups       | ext4 |
---------------------------------------------------

linux_utils.fstab.parse_fstab(filename='/etc/fstab', context=None)

Parse the Linux configuration file /etc/fstab.

Parameters:

• filename – The absolute pathname of the file to parse (a string, defaults to /etc/fstab).
• context – See coerce_context() for details.

Returns:

A generator of FileSystemEntry objects.

Here’s an example:

>>> from linux_utils.fstab import parse_fstab
>>> next(e for e in parse_fstab() if e.mount_point == '/')
FileSystemEntry(
  check_order=1,
  configuration_file='/etc/fstab',
  device='UUID=147f7d18-e0c9-499c-8791-401642581b90',
  device_file='/dev/disk/by-uuid/147f7d18-e0c9-499c-8791-401642581b90',
  dump_frequency=0,
  line_number=11,
  mount_point='/',
  options=['defaults', 'errors=remount-ro', 'discard', 'relatime', 'data=ordered'],
  vfs_type='ext4',
)

Note that some miscellaneous FileSystemEntry properties were omitted from the example above to make it more concise.

class linux_utils.fstab.FileSystemEntry(**kw)

An entry parsed from /etc/fstab.

Each entry in the fstab file has six fields, these are mapped to the following properties:

1. device
2. mount_point
3. vfs_type
4. options
5. dump_frequency
6. check_order

Refer to the fstab man page for more information about the meaning of each of these fields. The values of the following properties are computed based on the six fields above:

• device_file
• nfs_directory
• nfs_server

Here’s an overview of the FileSystemEntry class:

Superclass:	TabFileEntry
Properties:	check_order, device, device_file, dump_frequency, mount_point, nfs_directory, nfs_server, options and vfs_type

check_order

The order in which the filesystem should be checked at boot time (an integer number, defaults to 0).

Note

The check_order property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

device

The block special device or remote filesystem to be mounted (a string).

The value of this property may be a UUID=... expression.

device_file

The block special device to be mounted (a string).

The value of this property is computed by passing device to coerce_device_file().

Note

The device_file property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

dump_frequency

The dump frequency for the filesystem (an integer number, defaults to 0).

Note

The dump_frequency property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

mount_point

The mount point for the filesystem (a string).

Each occurrence of the escape sequence \040 is replaced by a space.

Note

The mount_point property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

nfs_directory

The directory on the NFS server (a string or None).

When vfs_type is nfs or nfs4 and device is of the form <server>:<directory> the value of nfs_directory will be the part after the colon (:).

Note

The nfs_directory property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

nfs_server

The host name or IP address of the NFS server (a string or None).

When vfs_type is nfs or nfs4 and device is of the form <server>:<directory> the value of nfs_server will be the part before the colon (:).

Note

The nfs_server property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

options

The mount options for the filesystem (a list of strings).

Note

The options property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

vfs_type

The type of filesystem (a string like ‘ext4’ or ‘xfs’).

linux_utils.luks

Python API for cryptsetup to control LUKS full disk encryption.

The functions in this module serve two distinct purposes:

Low level Python API for cryptsetup

The following functions and class provide a low level Python API for the basic functionality of cryptsetup:

• create_image_file()
• generate_key_file()
• create_encrypted_filesystem()
• unlock_filesystem()
• lock_filesystem()
• TemporaryKeyFile()

This functionality make it easier for me to write test suites for Python projects involving full disk encryption, for example crypto-drive-manager and rsync-system-backup.

Python implementation of cryptdisks_start and cryptdisks_stop

The command line programs cryptdisks_start and cryptdisks_stop are easy to use wrappers for cryptsetup that parse /etc/crypttab to find the information they need.

The nice thing about /etc/crypttab is that it provides a central place to configure the names of encrypted filesystems, so that you can refer to a symbolic name instead of having to constantly repeat all of the necessary information (the target name, source device, key file and encryption options).

A not so nice thing about cryptdisks_start and cryptdisks_stop is that these programs (and the whole /etc/crypttab convention) appear to be specific to the Debian ecosystem.

The functions cryptdisks_start() and cryptdisks_stop() emulate the behavior of the command line programs when needed so that Linux distributions that don’t offer these programs can still be supported by projects like crypto-drive-manager and rsync-system-backup.

linux_utils.luks.DEFAULT_KEY_SIZE = 2048

The default size (in bytes) of key files generated by generate_key_file() (a number).

linux_utils.luks.create_image_file(filename, size, context=None)

Create an image file filled with bytes containing zero (\0).

Parameters:

• filename – The pathname of the image file (a string).
• size – How large the image file should be (see coerce_size()).
• context – See coerce_context() for details.

Raises:

ValueError when size is invalid, ExternalCommandFailed when the command fails.

linux_utils.luks.generate_key_file(filename, size=2048, context=None)

Generate a file with random contents that can be used as a key file.

Parameters:

• filename – The pathname of the key file (a string).
• size – How large the key file should be (see coerce_size(), defaults to DEFAULT_KEY_SIZE).
• context – See coerce_context() for details.

Raises:

ExternalCommandFailed when the command fails.

linux_utils.luks.create_encrypted_filesystem(device_file, key_file=None, context=None)

Create an encrypted LUKS filesystem.

Parameters:

• device_file – The pathname of the block special device or file (a string).
• key_file – The pathname of the key file used to encrypt the filesystem (a string or None).
• context – See coerce_context() for details.

Raises:

ExternalCommandFailed when the command fails.

If no key_file is given the operator is prompted to choose a password.

linux_utils.luks.unlock_filesystem(device_file, target, key_file=None, options=None, context=None)

Unlock an encrypted LUKS filesystem.

Parameters:

• device_file – The pathname of the block special device or file (a string).
• target – The mapped device name (a string).
• key_file – The pathname of the key file used to encrypt the filesystem (a string or None).
• options – An iterable of strings with encryption options or None (in which case the default options are used). Currently ‘discard’, ‘readonly’ and ‘tries’ are the only supported options (other options are silently ignored).
• context – See coerce_context() for details.

Raises:

ExternalCommandFailed when the command fails.

If no key_file is given the operator is prompted to enter a password.

linux_utils.luks.lock_filesystem(target, context=None)

Lock a currently unlocked LUKS filesystem.

Parameters:

• target – The mapped device name (a string).
• context – See coerce_context() for details.

Raises:

ExternalCommandFailed when the command fails.

linux_utils.luks.cryptdisks_start(target, context=None)

Execute cryptdisks_start or emulate its functionality.

Parameters:

• target – The mapped device name (a string).
• context – See coerce_context() for details.

Raises:

ExternalCommandFailed when a command fails, ValueError when no entry in /etc/crypttab matches target.

linux_utils.luks.cryptdisks_stop(target, context=None)

Execute cryptdisks_stop or emulate its functionality.

Parameters:

• target – The mapped device name (a string).
• context – See coerce_context() for details.

Raises:

ExternalCommandFailed when a command fails, ValueError when no entry in /etc/crypttab matches target.

class linux_utils.luks.TemporaryKeyFile(filename, size=2048, context=None)

Context manager that makes it easier to work with temporary key files.

__init__(filename, size=2048, context=None)

Initialize a TemporaryKeyFile object.

Refer to generate_key_file() for details about argument handling.

__enter__()

Generate the temporary key file.

__exit__(exc_type=None, exc_value=None, traceback=None)

Delete the temporary key file.

linux_utils.network

Python API for Linux networking tools.

The functions in this module make it possible to inspect the current network configuration of a Linux system, which can provide hints about the physical location of the system.

linux_utils.network.determine_network_location(context=None, **gateways)

Determine the physical location of the current system.

This works by matching the MAC address of the current gateway against a set of known MAC addresses, which provides a simple but robust way to identify the current network. Because networks tend to have a physical location, identifying the current network tells us our physical location.

Parameters:

• gateways – One or more keyword arguments with lists of strings containing MAC addresses of known networks.
• context – See coerce_context() for details.

Returns:

The name of the matched MAC address (a string) or None when the MAC address of the current gateway is unknown.

Here’s an example involving two networks and a physical location with multiple gateways:

>>> determine_network_location(
...    home=['84:9C:A6:76:23:8E'],
...    office=['00:15:C5:5F:92:79', 'B6:25:B2:19:28:61'],
... )
'home'

This is used to tweak my desktop environment based on the physical location of my laptop, for example at home my external monitor is to the right of my laptop whereas at work it’s the other way around, so the xrandr commands to be run differ between the two locations.

linux_utils.network.find_gateway_ip(context=None)

Find the IP address of the current gateway using the ip route show command.

Parameters:context – See coerce_context() for details. Returns:The IP address of the gateway (a string) or None.

linux_utils.network.find_gateway_mac(context=None)

Find the MAC address of the current gateway using find_gateway_ip() and find_mac_address().

Parameters:context – See coerce_context() for details. Returns:The MAC address of the gateway (a string) or None.

linux_utils.network.find_mac_address(ip_address, context=None)

Determine the MAC address of an IP address using the arp -n command.

Parameters:

• ip_address – The IP address we’re interested in (a string).
• context – See coerce_context() for details.

Returns:

The MAC address of the IP address (a string) or None.

linux_utils.network.have_internet_connection(endpoint='8.8.8.8', context=None)

Check if an internet connection is available using ping.

Parameters:

• endpoint – The public IP address to ping (a string).
• context – See coerce_context() for details.

Returns:

True if an internet connection is available, False otherwise.

This works by pinging 8.8.8.8 which is one of Google public DNS servers. This IP address was chosen because it is documented that Google uses anycast to keep this IP address available at all times.

linux_utils.tabfile

Generic parsing of Linux configuration files like /etc/fstab and /etc/crypttab.

linux_utils.tabfile.parse_tab_file(filename, context=None, encoding='UTF-8')

Parse a Linux configuration file like /etc/fstab or /etc/crypttab.

Parameters:

• filename – The absolute pathname of the file to parse (a string).
• context – See coerce_context() for details.
• encoding – The name of the text encoding of the file (a string).

Returns:

A generator of TabFileEntry objects.

This function strips comments (the character # until the end of the line) and splits each line into tokens separated by whitespace.

class linux_utils.tabfile.TabFileEntry(**kw)

Container for the results of parse_tab_file().

You can set the values of the configuration_file, context, line_number and tokens properties by passing keyword arguments to the class initializer.

Here’s an overview of the TabFileEntry class:

Superclass:	PropertyManager
Properties:	configuration_file, context, line_number and tokens

context

The execution context from which the configuration file was retrieved.

Note

The context property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

configuration_file

The name of the configuration file from which this entry was parsed (a string).

Note

The configuration_file property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

line_number

The line number from which this entry was parsed (an integer).

Note

The line_number property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

tokens

The tokens split on whitespace (a nonempty list of strings).

Note

The tokens property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().