Application Developer Guide For python-hwloc

Copyright 2016-2017 Guy Streeter

This copyrighted material is made available to anyone wishing to use, modify, copy, or redistribute it subject to the terms and conditions of the GNU General Public License v.2.

This material is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

Version information

Introduction

python-hwloc is a set of Python bindings for hwloc, the Portable Hardware Locality software package. The python-hwloc package allows access to the hwloc library from Python applications. Information on hwloc is available at https://www.open-mpi.org/projects/hwloc. The source code for python-hwloc is maintained at https://gitlab.com/guystreeter/python-hwloc.

This document describes the implemented interface between Python and the hwloc library. The first section discusses obtaining and installing the python-hwloc package. The next sections describe the Python classes representing the C structures used in hwloc, with documentation of the methods and properties of each class. Some examples are inter-mixed with the documentation. The source code contains a tests folder providing further usage examples.

Please Note that this package is written for Linux, and has only been tested on recent Fedora® and Red Hat® systems. It should work on any recent Linux platform for which hwloc has been built.

Installation

RPM package repositories for Fedora and for EPEL for Centos® 7 are available at https://copr.fedorainfracloud.org/coprs/streeter/python-hwloc. For other linux platforms, the source code can be down-loaded from https://gitlab.com/guystreeter/python-hwloc and the setup.py file can be used to build and install it. Building python-hwloc requires Cython, and the development files for the hwloc, numactl (or libnuma), and libibverbs packages.

python-hwloc also requires python-libnuma, available from the same RPM repository. The source for python-libnuma is available at https://gitlab.com/guystreeter/python-libnuma.

This document can be recreated in the source tree using the command make doc. This requires the pandoc and latex applications.

Differences from the C library

Wherever the C library uses structures, this package implements classes that allow access to the structure members as properties. If a structure member is a structure or structure pointer, the associated object property is an object. The object instances hold pointers to the hwloc library structures. Except where they are explicitly created, (bitmap allocation for instance), these library structures remain in place until the topology is destroyed. A reference to the topology object should be held as long as any constituent object instance is needed.

Generally, library functions that reference a structure are implemented as methods in the class that represents that structure. Some of the classes support the use of various Python operators, such as conversion to string or comparison.

The Python code handles reference counting, and the freeing of allocated structures when required. Assignment to a new variable name does not copy the library structures, it just make a new reference to the same object instance.

All of the library functions with names ending in snprintf have been changed to an asprintf implementation, and the caller is not responsible for providing a string buffer for the result. The returned value is a native Python string.

Classes

The major classes used are Topology, Obj, and Bitmap. These correspond to the hwloc_topology, hwloc_obj, and hwloc_bitmap structures in the hwloc C code. They may for example be used as follows:

#!/usr/bin/python3

import hwloc

topo = hwloc.Topology()
topo.load()

robj = topo.root_obj

bmap = robj.cpuset

print('root object cpuset is', str(bmap))

Note that unless otherwise specified, properties are read-only

ArgError

ArgError is derived from the base Exception class. It is raised by class methods when one of the calling arguments is invalid. The exception object can be coerced to a string for error messages. For example

#!/usr/bin/python3

from hwloc import Bitmap, ArgError

bmap = Bitmap()
try:
    bmap.sscanf('qwerty')
except ArgError as err:
    print(str(err))

Bitmap

The Bitmap class represents the hwloc_bitmap structure, which is opaque to the application. The class has methods which can operate on the object instance, as well as object properties. It also has some class methods.

Important Note About Bitmap Object Lifetimes Bitmap instances remember automatically if they were created by explicit allocation (or returned as a result of a function call), or if they are references to bitmaps in other objects. An allocated bitmap will be freed when the reference count of the instance goes to zero.
Bitmaps that are references to other object properties are freed when that object instance goes away. A variable that is a reference to a bitmap property of another object will behave unpredictably when the underlying hwloc library structure is freed

Creating an instance of a Bitmap using the constructor (which must not have any parameters) is equivalent to calling the alloc class method without a parameter. These two statements are equivalent:

bmap = Bitmap()

bmap = Bitmap.alloc()

Class methods

The class methods are: * alloc(values=None)

alloc can be called with a parameter. If the parameter is an int, that bit is set in the resulting Bitmap. If the parameter is a str, the resulting Bitmap is set from the sscanf() result of the supplied string.

returns a newly-allocated bitmap from a Linux kernel cpumap file

returns a new bitmap from a list of set bit indexes

Operators

These Python operators can be used on bitmap objects: * | (arithmetic 'or')

creates a new bitmap object using hwloc_bitmap_or

results in a new bitmap object created with hwloc_bitmap_and

creates a new bimap object from hwloc_bitmap_xor

results in a new bitmap object created with hwloc_bitmap_not

<integer> in bitmap is True if hwloc_bitmap_isset is true for the bitmap and the bit index indicated by the integer.
bitmap1 in bitmap2 is True if hwloc_bitmap_isincluded(bitmap1, bitmap2) returns true. That is, if all the bits set in bitmap1 are set in bitmap2

Additionally, a Bitmap object can be converted to a string or list. str(bitmap) returns the result of hwloc_bitmap_asprintf for the bitmap. tuple(bitmap) will create tuple consisting of the index numbers of the bits set in bitmap.

A Bitmap object may be used as a boolean. It is true is hwloc_bitmap_iszero returns zero fir the bitmap.

A Bitmap object may also be used where an iterator is expected. for bit in bitmap will iterate through the indices of the set bits. The length of a Bitmap object is the number of set bits (hwloc_bitmap_weight).

Bitmap objects may be tested for bitwise equality using the == operator. Note however that bitmap1 is bitmap2 is not a test of equality. It is only True when both variable names refer to the same bitmap object.

Assignment of a bitmap to another variable only creates another reference to the same object. In order to create a new copy of a bitmap object, you must use the dup method. The Python copy module's copy() function will also make a duplicate.

Note that unlike the copy method supported by many Python classes, the Bitmap class has a copy function that follows the hwloc_bitmap_copy function. It copies the other bitmap to this bitmap.

Properties

Most of the bitmap functions that require no arguments other than the bitmap itself are implemented as properties. A property <name> is equivalent to the hwloc_bitmap_<name> function. * iszero * isfull * first * all_set_bits -> tuple of int * last

Note that some functions like zero, fill, and weight are presented as methods rather than properties, to indicate that they may be expensive operations.

Methods

A few of the Bitmap methods have a slightly different signature that the C library equivalents. They are:

Without an argument, this behaves like hwloc_bitmap_alloc. If a value is supplied as an integer, hwloc_bitmap_set is called on the newly-allocated bitmap for the integer supplied. If the value is a string, it is passed to hwloc_bitmap_sscanf for the new bitmap.

There is no separate hwloc_bitmap_from_ulong and hwloc_bitmap_from_ith_ulong. Note that the order of arguments is different from the library routine, so that the idx parameter can be defaulted to 0.

Like from_ulong, ulong takes an optional index, and there is no separate hwloc_bitmap_to_ulong equivalent.

Same as above but for hwloc_bitmap_set_ith_ulong.

The rest of the methods have the same argument in order as the hwloc_bitmap_* library routines (except of course the bitmap argument is not supplied). If the method returns a value, the type is specified.

Obj

The Obj class represents the hwloc_obj structure, and has properties corresponding to the structure members. It also has methods matching the library functions that are called with a hwloc_obj structure as an argument.

Important Note About Obj Structure References. Only the top-level Topology instance holds a reference to the hwloc_topology structure and all of the associated hwloc_obj structures, and all of their member structures and their member structures. You must keep a reference to the Topology instance as long as you are accessing anything the C structure points to. When the reference count for a Topology instance goes to zero, hwloc_topology_destroy is called for it, and any member objects still around will have invalid references.

An Obj object can be converted to a string. The resulting string is essentially all the infos, then hwloc_obj_type_snprintf, followed by hwloc_obj_attr_snprintf. This is mostly useful for debugging.

The test for equality between Obj instances is a test to see if they are both physically the same C structure (the structure pointer value is the same).

Properties

If the value of the property is not an integer, it's type is listed.

userdata is read-write. Note: only integer values are supported

The result of calling hwloc_obj_type_string for this object's type

Class Methods

Methods

ObjMemory

The memory property of an Obj class object is an object of the ObjMemory class, corresponding the the hwloc_obj_memory structure. It has these properties:

ObjMemoryPageType

Corresponding to the hwloc_obj_memory_page_type structure, this has two integer properties:

ObjAttr

The attr property of an Obj object is an object of ObjAttr class. Corresponding o the hwloc_obj_attr union, it will return an object instance for one of the following properties:

Note that like the C union, the result of accessing these through the wrong attribute class is undefined.

CacheAttr

Like the hwloc_cache_attr structure, this class has these integer properties:

GroupAttr

This matches the hwloc_group_attr structure, and has this integer property:

PCIDevAttr

Corresponding to the hwloc_pcidev_attr structure, this class has these integer properties:

BridgeAttr

This class represents the hwloc_bridge_attr structure.

BridgeAttrUpstream

BridgeAttrDownstream

BridgeAttrDownstreamPCI

BridgeAttr example

for obj in topology.bridges:
    assert obj.type == hwloc.OBJ_BRIDGE
    if obj.attr.bridge.upstream_type == hwloc.OBJ_BRIDGE_HOST:
        assert obj.attr.bridge.downstream_type == hwloc.OBJ_BRIDGE_PCI
        print(' Found host->PCI bridge for domain %04x bus %02x-%02x' % (
            obj.attr.bridge.downstream.pci.domain,
            obj.attr.bridge.downstream.pci.secondary_bus,
            obj.attr.bridge.downstream.pci.subordinate_bus))

Distances

ObjInfo

ObjInfo represents the hwloc_obj_info structure. It can be converted to a str type, resulting in "<name>:<value>". It has these properties:

TopologySupport

The support property of a Topology object is an object of the TopologySupport class, similar to the hwloc_topology_support structure. This class has 3 properties:

All of these objects have properties that are booleans describing what support is available. See the documentation for hwloc_topology_support in the hwloc package for the meanings of these booleans.

TopologyDiscoverySupport

This corresponds to the hwloc_topology_discovery_support structure. It has this boolean property:

TopologyCpubindSupport

This corresponds to the hwloc_topology_cpubind_support structure, and has these boolean properties:

TopologyMembindSupport

This corresponds to the hwloc_topology_membind_support structure, It has the following boolean properties:

TopologyDiff

Corresponding to the hwloc_topology_diff union, it has these properties. Referencing the member structure through the wrong class type has unknown results, except that type can always be referenced as though the class was TopologyDiffGeneric.

TopologyDiffGeneric

TopologyDiffObjAttrUint64

TopologyDiffObjAttrString

TopologyDiffTooComplex

TopologyDiffObjAttrU

TopologyDiffObjAttr

Topology

Note You must hold a reference to your Topology object as long as you want to reference any constituent part of it

General Methods

Note there is no destroy() method. The topology is destroyed when the Topology instance is no longer referenced.

Properties

Methods for Configuring Topology Detection

The two functions above can take a type value or a string recognized by hwloc_obj_type_sscanf()

Methods for exporting Topologies to XML

cb is a Python function taking (reserved, Topology, Obj) arguments

cb is a Python function taking (Topology, Obj, name, buf) arguments. name and buf are str types

See tests/hwloc_object_userdata.py in the source-code for an example.

Topology Information

CPU Binding

Memory Binding

An example of using the memory methods:

#!/usr/bin/python3
LEN = 1048576

import hwloc, sys, ctypes

topology = hwloc.Topology()
topology.load()

buffer_ = topology.alloc(LEN)
assert buffer_
print('buffer 0x{:X} length {:d}'.format(buffer_, LEN))

if topology.support.membind.get_area_memlocation:
    set_ = topology.get_area_memlocation(buffer_, LEN, hwloc.MEMBIND_BYNODESET)
    print('address 0x{:X} length {:d} allocated in nodeset'.format(buffer_, LEN), str(set_))

# touch the memory
buf = ctypes.cast(buffer_, ctypes.POINTER(ctypes.c_ubyte))
buf[0] = 0
buf[LEN-1] = 0 

topology.free(buffer_, LEN)

Modifying A Loaded Topology

Building Custom Topologies

Object Type Helpers

Retrieve Objects

Object/String Conversion

Basic Traversal Helpers

Finding Objects Inside a CPU set

Finding a single Object covering at least a CPU set

Example usage:

#!/usr/bin/env python3
import hwloc

topo = hwloc.Topology()

set1 = hwloc.Bitmap.alloc('00008f18')

topo.set_synthetic('nodes:8 cores:2 1')
topo.load()

obj = topo.get_next_obj_covering_cpuset_by_type(set1, hwloc.OBJ_NODE, None)
assert obj == topo.get_obj_by_depth(1, 1)

topo = hwloc.Topology()
topo.set_synthetic('nodes:2 socket:5 cores:3 4')
topo.load()

set1.sscanf('0ff08000')

depth = topo.get_type_depth(hwloc.OBJ_SOCKET)
assert depth == 2

for index, obj in enumerate(topo.objs_covering_cpuset_by_depth(set1, depth),
                            start=1):
    assert obj == topo.get_obj_by_depth(depth, index)

Cache-specific Finding Helpers

inding objects, miscellaneous helpers

An example of get_obj_below_array_by_type:

#!/usr/bin/env python3
import hwloc

topo = hwloc.Topology()

topo.set_synthetic('node:3 pack:3 core:3 pu:3')

topo.load()

# find the first thread
obj = topo.get_obj_below_array_by_type((hwloc.OBJ_NODE, 0),
                                       (hwloc.OBJ_SOCKET, 0),
                                       (hwloc.OBJ_CORE, 0),
                                       (hwloc.OBJ_PU, 0))
assert obj == topo.get_obj_by_depth(4, 0)

# find the last core
obj = topo.get_obj_below_array_by_type((hwloc.OBJ_NODE, 2),
                                       (hwloc.OBJ_SOCKET, 2),
                                       (hwloc.OBJ_CORE, 2))
assert obj == topo.get_obj_by_depth(3, 26)

Distributing items over a topology

distribute and distributev are deprecated

Cpuset Helpers

Nodeset Helpers

Converting between CPU sets and node sets

Manipulating Distances

Finding I/O objects

See the BridgeAttr section for an example.

Topology differences

Linux-only helpers

OpenGL display specific functions

Helpers for manipulating Linux libnuma unsigned long masks

Intel Xeon Phi (MIC) Specific Functions

Module-Level Functions

compare_types() can take a type constant or a string recognized by hwloc_obj_type_sscanf() * get_api_version()->int*version_string()->str*cpuset_from_glibc_sched_affinity(list of set bit indexes)->Bitmap`

Constants