API documentation¶
The following documentation is based on the source code of version 0.7 of the linux-utils package.
Available modules
linux_utils
¶
Linux system administration tools for Python.
The linux_utils
module contains generic functions
to be used by the other modules in the package.
-
linux_utils.
coerce_context
(value)[source]¶ Coerce the given value to an execution context.
Parameters: value – The value to coerce (an execution context created by executor.contexts
orNone
).Returns: An execution context created by executor.contexts
.Raises: ValueError
when value isn’tNone
but also isn’t a valid execution context.This function is used throughout the modules in the
linux_utils
package to abstract away the details of external command execution using dependency injection:- If no execution context is provided (value is
None
)executor.contexts.LocalContext
is used by default. - Callers can provide
executor.contexts.RemoteContext
in which case thelinux_utils
package seamlessly adapts to command execution on a remote system over ssh.
- If no execution context is provided (value is
-
linux_utils.
coerce_device_file
(expression)[source]¶ Coerce a device identifier to a device file.
Parameters: expression – The device identifier (a string). Returns: The pathname of the device file (a string). Raises: ValueError
when an unsupported device identifier is encountered.If you pass in a
LABEL="..."
orUUID=...
expression (as found in e.g./etc/fstab
) you will get back a pathname starting with/dev/disk/by-label
or/dev/disk/by-uuid
:>>> from linux_utils import coerce_device_file >>> print(coerce_device_file('LABEL="Linux Boot"')) /dev/disk/by-label/Linux\x20Boot >>> print(coerce_device_file('UUID=7801a1c2-7ad7-4c0b-9fbb-2a47ae802f71')) /dev/disk/by-uuid/7801a1c2-7ad7-4c0b-9fbb-2a47ae802f71
If expression is already a pathname it will pass through untouched:
>>> coerce_device_file('/dev/mapper/backups') '/dev/mapper/backups'
Unsupported device identifiers raise an exception:
>>> coerce_device_file('PARTUUID=e6c021cc-d0d8-400c-8f5c-b10adeff65fe') Traceback (most recent call last): File "linux_utils/__init__.py", line 90, in coerce_device_file raise ValueError(msg % expression) ValueError: Unsupported device identifier! ('PARTUUID=e6c021cc-d0d8-400c-8f5c-b10adeff65fe')
-
linux_utils.
coerce_size
(value)[source]¶ Coerce a human readable data size to the number of bytes.
Parameters: value – The value to coerce (a number or string). Returns: The number of bytes (a number). Raises: ValueError
when value isn’t a number or a string supported byparse_size()
.
linux_utils.atomic
¶
Atomic filesystem operations for Linux in Python.
The most useful functions in this module are make_dirs()
,
touch()
, write_contents()
and write_file()
.
The copy_stat()
and get_temporary_file()
functions were
originally part of the logic in write_file()
but have since been
extracted to improve the readability and reusability of the code.
-
linux_utils.atomic.
copy_stat
(filename, reference=None, mode=None, uid=None, gid=None)[source]¶ The Python equivalent of
chmod --reference && chown --reference
.Parameters: - filename – The pathname of the file whose permissions and ownership should be modified (a string).
- reference – The pathname of the file to use as
reference (a string or
None
). - mode – The permissions to set when reference isn’t given or doesn’t
exist (a number or
None
). - uid – The user id to set when reference isn’t given or doesn’t
exist (a number or
None
). - gid – The group id to set when reference isn’t given or doesn’t
exist (a number or
None
).
-
linux_utils.atomic.
get_temporary_file
(filename)[source]¶ Generate a non-obtrusive temporary filename.
Parameters: filename – The filename on which the name of the temporary file should be based (a string). Returns: The filename of a temporary file (a string). This function tries to generate the most non-obtrusive temporary filenames:
- The temporary file will be located in the same directory as the file to
replace, because this is the only location somewhat guaranteed to
support “rename into place” semantics (see
write_file()
). - The temporary file will be hidden from directory listings and common filename patterns because it has a leading dot.
- The temporary file will have a different extension then the file to replace (in case of filename patterns that do match dotfiles).
- The temporary filename has a decent chance of not conflicting with temporary filenames generated by concurrent processes.
- The temporary file will be located in the same directory as the file to
replace, because this is the only location somewhat guaranteed to
support “rename into place” semantics (see
-
linux_utils.atomic.
make_dirs
(directory, mode=511)[source]¶ Create a directory if it doesn’t already exist (keeping concurrency in mind).
Parameters: directory – The pathname of a directory (a string). Returns: True
if the directory was created,False
if it already existed.Raises: Any exceptions raised by os.makedirs()
.This function is a wrapper for
os.makedirs()
that swallowsOSError
in the case ofEEXIST
.
-
linux_utils.atomic.
touch
(filename)[source]¶ The equivalent of the touch program in Python.
Parameters: filename – The pathname of the file to touch (a string). This function uses
make_dirs()
to automatically create missing directory components in filename.
-
linux_utils.atomic.
write_contents
(filename, contents, encoding='UTF-8', mode=None)[source]¶ Atomically create or update a file’s contents.
Parameters: - filename – The pathname of the file (a string).
- contents – The (new) contents of the file (a byte string or a Unicode string).
- encoding – The text encoding used to encode contents when it is a Unicode string.
- mode – The permissions to use when the file doesn’t exist yet (a
number like accepted by
os.chmod()
orNone
).
-
linux_utils.atomic.
write_file
(*args, **kwds)[source]¶ Atomically create or update a file (avoiding partial reads).
Parameters: - filename – The pathname of the file (a string).
- mode – The permissions to use when the file doesn’t exist yet (a
number like accepted by
os.chmod()
orNone
).
Returns: A writable file object whose contents will be used to create or atomically replace filename.
linux_utils.cli
¶
Command line interface for linux_utils.luks
.
The linux_utils.cli
module implements command line interfaces for the
cryptdisks_start()
and cryptdisks_stop()
functions.
-
linux_utils.cli.
cryptdisks_start_cli
()[source]¶ Usage: cryptdisks-start-fallback NAME
Reads /etc/crypttab and unlocks the encrypted filesystem with the given NAME.
This program emulates the functionality of Debian’s cryptdisks_start program, but it only supports LUKS encryption and a small subset of the available encryption options.
-
linux_utils.cli.
cryptdisks_stop_cli
()[source]¶ Usage: cryptdisks-stop-fallback NAME
Reads /etc/crypttab and locks the encrypted filesystem with the given NAME.
This program emulates the functionality of Debian’s cryptdisks_stop program, but it only supports LUKS encryption and a small subset of the available encryption options.
linux_utils.crypttab
¶
Parsing of /etc/crypttab
configuration files.
The cryptsetup program is used to manage LUKS based full disk encryption and Debian provides some niceties around cryptsetup to make it easier to use, specifically:
- The
/etc/crypttab
configuration file contains static information about encrypted filesystems and enables unlocking of encrypted filesystems when the system is booted. Refer to the crypttab man page for more information. - The cryptdisks_start and cryptdisks_stop commands can be used to manually
unlock encrypted filesystems that are configured in
/etc/crypttab
with thenoauto
option (meaning the device is ignored during boot).
-
linux_utils.crypttab.
parse_crypttab
(filename='/etc/crypttab', context=None)[source]¶ Parse the Debian Linux configuration file
/etc/crypttab
.Parameters: - filename – The absolute pathname of the file to parse (a string,
defaults to
/etc/crypttab
). - context – See
coerce_context()
for details.
Returns: A generator of
EncryptedFileSystemEntry
objects.Here’s an example:
>>> from linux_utils.crypttab import parse_crypttab >>> print(next(parse_crypttab())) EncryptedFileSystemEntry( configuration_file='/etc/crypttab', line_number=3, target='ssd', source='UUID=31678141-3931-4683-a4d2-09eadec81d01', source_device='/dev/disk/by-uuid/31678141-3931-4683-a4d2-09eadec81d01', key_file='none', options=['luks', 'discard'], )
Changed in version 0.6: It is not an error when filename doesn’t exist, because of my experience that
/etc/crypttab
doesn’t exist in default Debian and Ubuntu installations (unless that system was specifically set up with root disk encryption using the installation wizard). This used to raise an exception, but this was changed in release 0.6.- filename – The absolute pathname of the file to parse (a string,
defaults to
-
class
linux_utils.crypttab.
EncryptedFileSystemEntry
(**kw)[source]¶ An entry parsed from
/etc/crypttab
.Each entry in the crypttab file has four fields, these are mapped to the following properties:
Refer to the crypttab man page for more information about these fields. The computed properties
is_available
,is_unlocked
andsource_device
are based on the parsed values of the four fields above.Here’s an overview of the
EncryptedFileSystemEntry
class:Superclass: TabFileEntry
Properties: is_available
,is_unlocked
,key_file
,options
,source
,source_device
,target
andtarget_device
-
is_available
¶ True
ifsource_device
exists,False
otherwise.
-
is_unlocked
¶ True
iftarget_device
exists,False
otherwise.
-
key_file
¶ The file to use as a key for decrypting the data of the source device (a string or
None
).When the key file field in
/etc/crypttab
is set tonone
the value of this property will beNone
, this makes checking whether an encrypted filesystem has a key file configured much more Pythonic.
-
options
[source]¶ The encryption options for the filesystem (a list of strings).
Note
The
options
property is alazy_property
. This property’s value is computed once (the first time it is accessed) and the result is cached.
-
source
¶ The block special device or file that contains the encrypted data (a string).
The value of this property may be a
UUID=...
expression instead of the pathname of a block special device or file.
-
source_device
[source]¶ The block special device or file that contains the encrypted data (a string).
The value of this property is computed by passing
source
tocoerce_device_file()
.Note
The
source_device
property is alazy_property
. This property’s value is computed once (the first time it is accessed) and the result is cached.
-
target
¶ The mapped device name (a string).
-
target_device
[source]¶ The absolute pathname of the device file corresponding to
target
(a string).Note
The
target_device
property is alazy_property
. This property’s value is computed once (the first time it is accessed) and the result is cached.
-
linux_utils.fstab
¶
Parsing of /etc/fstab configuration files.
-
linux_utils.fstab.
find_mounted_filesystems
(filename='/proc/mounts', context=None)[source]¶ Get information about mounted filesystem from
/proc/mounts
.Parameters: - filename – The absolute pathname of the file to parse (a string,
defaults to
/proc/mounts
). - context – See
coerce_context()
for details.
Returns: A generator of
FileSystemEntry
objects.This function is a trivial wrapper for
parse_fstab()
that instructs it to parse/proc/mounts
instead of /etc/fstab. Here’s an example:>>> from humanfriendly import format_table >>> from linux_utils.fstab import find_mounted_filesystems >>> print(format_table( ... data=[ ... (entry.mount_point, entry.device_file, entry.vfs_type) ... for entry in find_mounted_filesystems() ... if entry.vfs_type not in ( ... # While writing this example I was actually surprised to ... # see how many `virtual filesystems' a modern Linux system ... # has mounted by default (based on Ubuntu 16.04). ... 'autofs', 'cgroup', 'debugfs', 'devpts', 'devtmpfs', 'efivarfs', ... 'fuse.gvfsd-fuse', 'fusectl', 'hugetlbfs', 'mqueue', 'proc', ... 'pstore', 'securityfs', 'sysfs', 'tmpfs', ... ) ... ], ... column_names=["Mount point", "Device", "Type"], ... )) --------------------------------------------------- | Mount point | Device | Type | --------------------------------------------------- | / | /dev/mapper/internal-root | ext4 | | /boot | /dev/sda5 | ext4 | | /boot/efi | /dev/sda1 | vfat | | /mnt/backups | /dev/mapper/backups | ext4 | ---------------------------------------------------
- filename – The absolute pathname of the file to parse (a string,
defaults to
-
linux_utils.fstab.
parse_fstab
(filename='/etc/fstab', context=None)[source]¶ Parse the Linux configuration file /etc/fstab.
Parameters: - filename – The absolute pathname of the file to parse (a string, defaults to /etc/fstab).
- context – See
coerce_context()
for details.
Returns: A generator of
FileSystemEntry
objects.Here’s an example:
>>> from linux_utils.fstab import parse_fstab >>> next(e for e in parse_fstab() if e.mount_point == '/') FileSystemEntry( check_order=1, configuration_file='/etc/fstab', device='UUID=147f7d18-e0c9-499c-8791-401642581b90', device_file='/dev/disk/by-uuid/147f7d18-e0c9-499c-8791-401642581b90', dump_frequency=0, line_number=11, mount_point='/', options=['defaults', 'errors=remount-ro', 'discard', 'relatime', 'data=ordered'], vfs_type='ext4', )
Note that some miscellaneous
FileSystemEntry
properties were omitted from the example above to make it more concise.
-
class
linux_utils.fstab.
FileSystemEntry
(**kw)[source]¶ An entry parsed from /etc/fstab.
Each entry in the fstab file has six fields, these are mapped to the following properties:
Refer to the fstab man page for more information about the meaning of each of these fields. The values of the following properties are computed based on the six fields above:
Here’s an overview of the
FileSystemEntry
class:Superclass: TabFileEntry
Properties: check_order
,device
,device_file
,dump_frequency
,mount_point
,nfs_directory
,nfs_server
,options
andvfs_type
-
check_order
[source]¶ The order in which the filesystem should be checked at boot time (an integer number, defaults to 0).
Note
The
check_order
property is alazy_property
. This property’s value is computed once (the first time it is accessed) and the result is cached.
-
device
¶ The block special device or remote filesystem to be mounted (a string).
The value of this property may be a
UUID=...
expression.
-
device_file
[source]¶ The block special device to be mounted (a string).
The value of this property is computed by passing
device
tocoerce_device_file()
.Note
The
device_file
property is alazy_property
. This property’s value is computed once (the first time it is accessed) and the result is cached.
-
dump_frequency
[source]¶ The dump frequency for the filesystem (an integer number, defaults to 0).
Note
The
dump_frequency
property is alazy_property
. This property’s value is computed once (the first time it is accessed) and the result is cached.
-
mount_point
[source]¶ The mount point for the filesystem (a string).
Each occurrence of the escape sequence
\040
is replaced by a space.Note
The
mount_point
property is alazy_property
. This property’s value is computed once (the first time it is accessed) and the result is cached.
-
nfs_directory
[source]¶ The directory on the NFS server (a string or
None
).When
vfs_type
isnfs
ornfs4
anddevice
is of the form<server>:<directory>
the value ofnfs_directory
will be the part after the colon (:
).Note
The
nfs_directory
property is alazy_property
. This property’s value is computed once (the first time it is accessed) and the result is cached.
-
nfs_server
[source]¶ The host name or IP address of the NFS server (a string or
None
).When
vfs_type
isnfs
ornfs4
anddevice
is of the form<server>:<directory>
the value ofnfs_server
will be the part before the colon (:
).Note
The
nfs_server
property is alazy_property
. This property’s value is computed once (the first time it is accessed) and the result is cached.
-
options
[source]¶ The mount options for the filesystem (a list of strings).
Note
The
options
property is alazy_property
. This property’s value is computed once (the first time it is accessed) and the result is cached.
-
vfs_type
¶ The type of filesystem (a string like ‘ext4’ or ‘xfs’).
-
linux_utils.luks
¶
Python API for cryptsetup to control LUKS full disk encryption.
The functions in this module serve two distinct purposes:
- Low level Python API for cryptsetup
The following functions and class provide a low level Python API for the basic functionality of cryptsetup:
create_image_file()
generate_key_file()
create_encrypted_filesystem()
unlock_filesystem()
lock_filesystem()
TemporaryKeyFile()
This functionality make it easier for me to write test suites for Python projects involving full disk encryption, for example crypto-drive-manager and rsync-system-backup.
- Python implementation of cryptdisks_start and cryptdisks_stop
The command line programs cryptdisks_start and cryptdisks_stop are easy to use wrappers for cryptsetup that parse /etc/crypttab to find the information they need.
The nice thing about /etc/crypttab is that it provides a central place to configure the names of encrypted filesystems, so that you can refer to a symbolic name instead of having to constantly repeat all of the necessary information (the target name, source device, key file and encryption options).
A not so nice thing about cryptdisks_start and cryptdisks_stop is that these programs (and the whole /etc/crypttab convention) appear to be specific to the Debian ecosystem.
The functions
cryptdisks_start()
andcryptdisks_stop()
emulate the behavior of the command line programs when needed so that Linux distributions that don’t offer these programs can still be supported by projects like crypto-drive-manager and rsync-system-backup.
-
linux_utils.luks.
DEFAULT_KEY_SIZE
= 2048¶ The default size (in bytes) of key files generated by
generate_key_file()
(a number).
-
linux_utils.luks.
create_image_file
(filename, size, context=None)[source]¶ Create an image file filled with bytes containing zero (
\0
).Parameters: - filename – The pathname of the image file (a string).
- size – How large the image file should be (see
coerce_size()
). - context – See
coerce_context()
for details.
Raises: ValueError
when size is invalid,ExternalCommandFailed
when the command fails.
-
linux_utils.luks.
generate_key_file
(filename, size=2048, context=None)[source]¶ Generate a file with random contents that can be used as a key file.
Parameters: - filename – The pathname of the key file (a string).
- size – How large the key file should be (see
coerce_size()
, defaults toDEFAULT_KEY_SIZE
). - context – See
coerce_context()
for details.
Raises: ExternalCommandFailed
when the command fails.
-
linux_utils.luks.
create_encrypted_filesystem
(device_file, key_file=None, context=None)[source]¶ Create an encrypted LUKS filesystem.
Parameters: - device_file – The pathname of the block special device or file (a string).
- key_file – The pathname of the key file used to encrypt the
filesystem (a string or
None
). - context – See
coerce_context()
for details.
Raises: ExternalCommandFailed
when the command fails.If no key_file is given the operator is prompted to choose a password.
-
linux_utils.luks.
unlock_filesystem
(device_file, target, key_file=None, options=None, context=None)[source]¶ Unlock an encrypted LUKS filesystem.
Parameters: - device_file – The pathname of the block special device or file (a string).
- target – The mapped device name (a string).
- key_file – The pathname of the key file used to encrypt the
filesystem (a string or
None
). - options – An iterable of strings with encryption options or
None
(in which case the default options are used). Currently ‘discard’, ‘readonly’ and ‘tries’ are the only supported options (other options are silently ignored). - context – See
coerce_context()
for details.
Raises: ExternalCommandFailed
when the command fails.If no key_file is given the operator is prompted to enter a password.
-
linux_utils.luks.
lock_filesystem
(target, context=None)[source]¶ Lock a currently unlocked LUKS filesystem.
Parameters: - target – The mapped device name (a string).
- context – See
coerce_context()
for details.
Raises: ExternalCommandFailed
when the command fails.
-
linux_utils.luks.
cryptdisks_start
(target, context=None)[source]¶ Execute cryptdisks_start or emulate its functionality.
Parameters: - target – The mapped device name (a string).
- context – See
coerce_context()
for details.
Raises: ExternalCommandFailed
when a command fails,ValueError
when no entry in /etc/crypttab matches target.
-
linux_utils.luks.
cryptdisks_stop
(target, context=None)[source]¶ Execute cryptdisks_stop or emulate its functionality.
Parameters: - target – The mapped device name (a string).
- context – See
coerce_context()
for details.
Raises: ExternalCommandFailed
when a command fails,ValueError
when no entry in /etc/crypttab matches target.
-
class
linux_utils.luks.
TemporaryKeyFile
(filename, size=2048, context=None)[source]¶ Context manager that makes it easier to work with temporary key files.
-
__init__
(filename, size=2048, context=None)[source]¶ Initialize a
TemporaryKeyFile
object.Refer to
generate_key_file()
for details about argument handling.
-
linux_utils.network
¶
Python API for Linux networking tools.
The functions in this module make it possible to inspect the current network configuration of a Linux system, which can provide hints about the physical location of the system.
-
linux_utils.network.
determine_network_location
(context=None, **gateways)[source]¶ Determine the physical location of the current system.
This works by matching the MAC address of the current gateway against a set of known MAC addresses, which provides a simple but robust way to identify the current network. Because networks tend to have a physical location, identifying the current network tells us our physical location.
Parameters: - gateways – One or more keyword arguments with lists of strings containing MAC addresses of known networks.
- context – See
coerce_context()
for details.
Returns: The name of the matched MAC address (a string) or
None
when the MAC address of the current gateway is unknown.Here’s an example involving two networks and a physical location with multiple gateways:
>>> determine_network_location( ... home=['84:9C:A6:76:23:8E'], ... office=['00:15:C5:5F:92:79', 'B6:25:B2:19:28:61'], ... ) 'home'
This is used to tweak my desktop environment based on the physical location of my laptop, for example at home my external monitor is to the right of my laptop whereas at work it’s the other way around, so the xrandr commands to be run differ between the two locations.
-
linux_utils.network.
find_gateway_ip
(context=None)[source]¶ Find the IP address of the current gateway using the
ip route show
command.Parameters: context – See coerce_context()
for details.Returns: The IP address of the gateway (a string) or None
.
-
linux_utils.network.
find_gateway_mac
(context=None)[source]¶ Find the MAC address of the current gateway using
find_gateway_ip()
andfind_mac_address()
.Parameters: context – See coerce_context()
for details.Returns: The MAC address of the gateway (a string) or None
.
-
linux_utils.network.
find_mac_address
(ip_address, context=None)[source]¶ Determine the MAC address of an IP address using the
arp -n
command.Parameters: - ip_address – The IP address we’re interested in (a string).
- context – See
coerce_context()
for details.
Returns: The MAC address of the IP address (a string) or
None
.
-
linux_utils.network.
have_internet_connection
(endpoint='8.8.8.8', context=None)[source]¶ Check if an internet connection is available using ping.
Parameters: - endpoint – The public IP address to ping (a string).
- context – See
coerce_context()
for details.
Returns: True
if an internet connection is available,False
otherwise.This works by pinging 8.8.8.8 which is one of Google public DNS servers. This IP address was chosen because it is documented that Google uses anycast to keep this IP address available at all times.
linux_utils.tabfile
¶
Generic parsing of Linux configuration files like /etc/fstab
and /etc/crypttab
.
-
linux_utils.tabfile.
parse_tab_file
(filename, context=None, encoding='UTF-8')[source]¶ Parse a Linux configuration file like
/etc/fstab
or/etc/crypttab
.Parameters: - filename – The absolute pathname of the file to parse (a string).
- context – See
coerce_context()
for details. - encoding – The name of the text encoding of the file (a string).
Returns: A generator of
TabFileEntry
objects.This function strips comments (the character
#
until the end of the line) and splits each line into tokens separated by whitespace.
-
class
linux_utils.tabfile.
TabFileEntry
(**kw)[source]¶ Container for the results of
parse_tab_file()
.You can set the values of the
configuration_file
,context
,line_number
andtokens
properties by passing keyword arguments to the class initializer.Here’s an overview of the
TabFileEntry
class:Superclass: PropertyManager
Properties: configuration_file
,context
,line_number
andtokens
-
context
[source]¶ The execution context from which the configuration file was retrieved.
Note
The
context
property is amutable_property
. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can usedel
ordelattr()
.
-
configuration_file
[source]¶ The name of the configuration file from which this entry was parsed (a string).
Note
The
configuration_file
property is amutable_property
. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can usedel
ordelattr()
.
-
line_number
[source]¶ The line number from which this entry was parsed (an integer).
Note
The
line_number
property is amutable_property
. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can usedel
ordelattr()
.
-