Devices

This page describes what the operator publishes and what a claim on it delivers: the devices, their attributes and taints, the two device classes, and the objects of the pairing API. The operator is a Dynamic Resource Allocation (DRA) driver named bluetooth.liken.sh, and it publishes beside liken ’s own driver on the same node.

The slice

The operator writes one ResourceSlice per node, named <node>-bluetooth.liken.sh, beside liken’s own <node>-liken.sh:

$ kubectl get resourceslice liken-1-bluetooth.liken.sh -o yaml
spec:
  driver: bluetooth.liken.sh
  nodeName: liken-1
  devices:
    - name: a0-ab-51-33-b7-12
      attributes:
        address: {string: "A0:AB:51:33:B7:12"}
        connected: {bool: true}
        name: {string: "DualSense Wireless Controller"}
        classOfDevice: {int: 9480}
        majorClass: {string: peripheral}
        minorClass: {string: gamepad}
        addressType: {string: public}
        icon: {string: input-gaming}
        input: {bool: true}
        joystick: {bool: true}
        accelerometer: {bool: true}
        touchpad: {bool: true}
    - name: 04-4a-69-66-92-27-media
      attributes:
        address: {string: "04:4A:69:66:92:27"}
        kind: {string: mediaBus}
        sound.liken.sh/supportsSound: {bool: true}

The slice holds two kinds of device: one for each paired controller, and one media bus for the adapter itself.

The controller list follows the paired set, whether or not each controller is connected. A paired controller that is switched off still publishes, so a pod can claim it and start when somebody turns it on. A controller leaves the slice only when it is unpaired, which is a kubectl delete peripheral.

The media bus publishes as soon as bluetoothd names the adapter, so the slice exists on a machine with a radio and nothing paired to it. The whole slice is deleted only while no adapter has answered and nothing is paired.

The device name is the controller’s MAC in lowercase with dashes, because a DRA device name must be a DNS label. The media bus takes the adapter’s own MAC in the same form, with a -media suffix. The MAC is the only identity on the machine that survives a reboot: the HID instance suffix in sysfs counts up from zero each boot, and the hci0:N handle changes on every reconnect, so a claim against either would allocate different hardware after a reboot.

The attributes

This section covers the paired controllers. The media bus carries its own three attributes, listed in its section.

A selector reads these as device.attributes["bluetooth.liken.sh"].<name>.

Two attributes are on every controller this operator publishes, in every state, the departed-adapter republish included:

Every other attribute is present only when BlueZ reports the fact. The identity facts publish in two layers: the raw code, and the names and flags unpacked from it, so a selector never does bit arithmetic:

The profile flags come from the service UUIDs the device advertised when it paired. Each one is true when the profile is advertised and absent otherwise, and a UUID outside this vocabulary publishes nothing:

The input classes come from the evdev nodes the controller registered, not from Bluetooth. Each one is true when any of the controller’s nodes carries the class and absent otherwise.

The names are udev’s own ID_INPUT_* properties in lower case. The operator applies the rules of systemd’s input_id builtin to the bitmaps the kernel reports for each node, so a controller carries the classes udevadm info would print for it on any Linux machine. The inputs parameter uses the same words to say which of them a claim receives.

The operator reads a controller’s capabilities the first time it connects and keeps them in the bond’s Secret, so the classes stay published while the controller sleeps. A bond that has never connected carries input and no class, because the operator has not read a node for it yet.

The split between always and absent is a contract. The operator omits an attribute it has no value for, rather than publishing it empty. A selector’s read of an absent attribute does not evaluate to false; it fails, and the failure can abort the allocation instead of skipping the device. So a selector on anything past address and connected must guard the read:

has(device.attributes["bluetooth.liken.sh"].name) &&
device.attributes["bluetooth.liken.sh"].name.startsWith("DualSense")

A selector that reads only address or connected needs no guard, because the bluetooth-input class already limits the candidates to this driver’s paired input devices, and both attributes are always on them. Outside that class, the media bus carries address and no connected, so connected takes a guard like any other attribute.

The taints

Three taints go on a controller, and they answer three different questions. The first two say the controller cannot serve a claim. The third says the claim it already serves holds the wrong nodes:

The media bus takes one taint, and only when the adapter has departed:

The effect differs from the controllers’ NoExecute on purpose. The pod that holds the bus is the machine’s one sound server, so an eviction would end its other playback too, and that playback does not need the radio. NoSchedule parks the next claim and leaves the running holder alone.

Tolerate /disconnected only. The NoExecute taint evicts a claim holder after its tolerationSeconds, so tolerating it sets how long a radio may be silent before the pod ends. The NoSchedule taint must stay untolerated, because it parks a claim on a controller that has never connected as Unschedulable. Tolerate both and the scheduler allocates a controller the operator has no node for, NodePrepareResources fails, and the pod churns between ContainerCreating and eviction until somebody switches the controller on.

No consumer tolerates /node-moved. The pod holds device nodes that belong to another controller now, so its eviction is the repair: the kubelet unprepares the claim, and the container that replaces the evicted one is prepared with the nodes the operator delivers now.

The media bus

The media bus is one device per adapter: the claimable permission to connect to this pod’s bluetoothd over its private D-Bus. A Bluetooth speaker creates no kernel device. Its audio exists only while a sound server holds this bus and keeps a media endpoint registered, and BlueZ advertises no A2DP until an endpoint registers.

The audio operator claims the bus through sound.liken.sh/supportsSound, the attribute liken also stamps on each sound card it publishes. That operator’s class names the attribute and no driver, so one claim collects every device on a node that can serve a sound server. This operator ships no class for the bus and runs no sound server itself.

sound.liken.sh/supportsSound is the one qualified name in this driver’s attributes. It uses a domain neither driver owns, so a selector reads it as device.attributes["sound.liken.sh"].supportsSound, where every other attribute here reads under bluetooth.liken.sh.

The bus never carries input, so the bluetooth-input class, which guards on that attribute, never matches it.

The device is exclusive, which is resource.k8s.io/v1’s default: one radio serves one sound server, because two media endpoints registered on one bluetoothd have no contract over the streams.

A claim on the bus delivers a read-only mount of /var/run/bluetooth.liken.sh/dbus at the same path inside the container, and one environment variable that names the socket in it. No device node, no privilege, and no other host path.

DBUS_SYSTEM_BUS_ADDRESS=unix:path=/var/run/bluetooth.liken.sh/dbus/system_bus_socket

The device classes

The operator takes two DeviceClasses , and they split by owner:

bluetooth-adapter ships with the deploy base, because the operator’s own claim template names it and the pod cannot start without it. bluetooth-input is yours to create, because a class a workload claims through is cluster policy, and Install the operator gives its YAML. A class of your own can also carry a default inputs block . It selects the input attribute rather than the whole driver, because the driver publishes more than input devices: a paired speaker publishes as its bond record, no workload should hold one, and the media bus belongs to the machine’s sound server.

The claim

A ResourceClaim against bluetooth-input alone allocates any paired input device. To name one, add a selector on its address:

spec:
  devices:
    requests:
      - name: controller
        exactly:
          deviceClassName: bluetooth-input
          selectors:
            - cel:
                expression: |
                  device.attributes["bluetooth.liken.sh"].address == "A0:AB:51:33:B7:12"
          tolerations:
            - key: bluetooth.liken.sh/disconnected
              operator: Exists
              effect: NoExecute
              tolerationSeconds: 30

Pair a controller and give it to a pod gives the whole flow, with the pod that takes the claim. In a Deployment, claim through a ResourceClaimTemplate rather than a standing ResourceClaim, because a standing claim keeps its allocation across an eviction.

The inputs parameter

A controller publishes every capability it has, and a claim states which classes of input the container receives. The two are separate because a consumer rarely wants everything. A DualSense at rest reports about 2400 motion events a second on its accelerometer node, and a pod that reads its buttons would drop every one of them. With inputs, a class nobody asked for is delivered to nobody, and its events never leave the kernel.

spec:
  devices:
    requests:
      - name: controller
        exactly:
          deviceClassName: bluetooth-input
    config:
      - requests: [controller]
        opaque:
          driver: bluetooth.liken.sh
          parameters:
            inputs: [joystick]

A claim with no configuration block, or with a block that has no inputs key, receives every class. An empty list is refused when the claim is prepared, and the pod stays in ContainerCreating with the refusal in its events. A name outside the table below is refused the same way, and the message lists the names.

The classes are udev’s ID_INPUT_* properties in lower case, and the rules that assign them are a port of systemd’s input_id builtin. udevadm info on any input device prints the same words.

A node can carry several classes, the way udev sets several properties on one node: a remote with a gyroscopic cursor is key and mouse at once. The operator meets the demand per node. A node no prepared claim asks for is never read. A node that carries a class the claim asked for and one it did not is narrowed to the event types of the classes it asked for, so that remote claimed with inputs: [key] delivers its keys and none of its pointer motion.

Two claims on one controller each receive what they asked for. The operator delivers the union of what the prepared claims demand, and narrows again when one of them ends.

A default belongs on a DeviceClass. A class of your own may carry an inputs block, and every claim through that class receives it. A claim’s own block wins over the class’s.

apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
  name: gamepad
spec:
  selectors:
    - cel:
        expression: |
          device.driver == "bluetooth.liken.sh" &&
          has(device.attributes["bluetooth.liken.sh"].joystick)
  config:
    - opaque:
        driver: bluetooth.liken.sh
        parameters:
          inputs: [joystick]

The selector and the inputs block are two separate statements: the selector picks which devices the claim may allocate, and inputs decides which events reach the container once one is allocated.

The bluetooth-input class that Install the operator gives carries no inputs block, so a claim through it receives every class. A class can carry an axes block the same way.

The axes parameter

A stick reports position noise while it rests. A DualSense’s right stick moves one step on ABS_RX and ABS_RY about 270 times a second per axis with nobody touching it, so its node emits about a thousand events a second, and every reader of the node pays for each one. The kernel’s answer to a noisy axis is the axis’s fuzz, and axes is where a claim sets it.

spec:
  devices:
    requests:
      - name: controller
        exactly:
          deviceClassName: gamepad
    config:
      - requests: [controller]
        opaque:
          driver: bluetooth.liken.sh
          parameters:
            inputs: [joystick]
            axes:
              ABS_RX: {fuzz: 4}
              ABS_RY: {fuzz: 4}

The input core drops a position change smaller than half the axis’s fuzz and smooths one smaller than the fuzz before any handler sees it, and it reports a position within the flat of the axis centre as the centre.

This is what systemd’s hwdb does for known devices: its EVDEV_ABS_ entries in 60-evdev.hwdb set the same two fields with the same EVIOCSABS ioctl. The operator carries no hardware database, and the claim holder knows the pad, so the claim states them instead.

The keys are the kernel’s own ABS_* code names, from linux/input-event-codes.h. Each one takes fuzz, flat, or both, and no other field:

A code name outside the kernel’s list, a field beside those two, or a negative value is refused when the claim is prepared. The pod stays in ContainerCreating, and the message names what is accepted. ABS_MT_SLOT is refused too, because the kernel does not change the number of contacts a device reserved.

An axis takes the largest fuzz and the largest flat any prepared claim on the controller states, so each claim receives at least the smoothing it asked for. An axis no remaining claim states goes back to the values the device reported when it first connected, which the operator keeps in the bond’s Secret.

The operator writes the values to the controller’s real node when the claim is prepared, and again each time the controller reconnects, because a reconnect is a new kernel device that carries none of them.

What a claim delivers

What a claim delivers depends on the device it allocated. Both kinds arrive the same way, through the Container Device Interface (CDI) at container creation, and neither delivers any privilege.

A claim on a controller delivers device nodes, and nothing else: /dev/input/event*, one for each evdev node the controller registers. No host mount, no environment variable. The container’s user must be able to open the nodes. Every node is delivered whatever the claim asked for, and the inputs parameter decides which events arrive on each. A claim on the media bus delivers the mount and the variable that The media bus lists, and no device node.

The legacy /dev/input/jsN interface stays out. liken’s kernel may not enable CONFIG_INPUT_JOYDEV at all, and joydev publishes a DualSense’s motion sensors as a wrong second jsN device.

The nodes a claim delivers are not the kernel’s own nodes for the controller. The operator creates one virtual input device for each node the controller registers, with the kernel’s uinput interface, and moves the controller’s events into it whenever the controller is on the air. A virtual node keeps its number for as long as the operator holds it open, so a controller that sleeps and returns as a different eventN changes nothing a pod holds. A pod that tolerates /disconnected and starts while the controller sleeps gets a node that reads nothing until the next press.

The operator reads a controller’s capabilities from its real node the first time it connects, and stores them in that bond’s Secret. On a later start it creates the virtual devices from the stored snapshot, before anything connects. So a controller has to connect once after it is paired, and its claims stop parking from then on.

Events go one way. A gamepad’s rumble is a write into the real node, and the relay does not carry it back.

A Bluetooth LE remote that sleeps sends no key for the press that wakes it. So when a remote connects after a sleep, the operator writes one press and one release of KEY_UNKNOWN (code 240) into its virtual node, before the remote’s own events. A consumer that binds nothing to KEY_UNKNOWN reads it as a press with no meaning, and a sleeping screen wakes on it. The press arrives only under all of these conditions:

• The Peripheral’s Connected condition held Asleep: the link timed out on a device that sleeps between sessions.
• The link was down for at least 10 seconds. A remote that lost its link while it was awake connects again within seconds, and gets no press.
• The node reports a keyboard key, a code below BTN_MISC. A gamepad’s node gets no press.
• A claim on the controller receives EV_KEY from that node.

Each sleep gives one press, on the first node that meets these conditions. Each virtual node with a keyboard key declares KEY_UNKNOWN, although the controller’s own node may not. The stored capabilities, the input classes, and the slice do not include it.

Lifecycle

• A disconnected controller is tainted, never deleted. The device stays in the slice with both taints, and a return clears them. Deleting a device a claim still names would strand the claim: the kubelet retries NodePrepareResources against a device in no slice, with no bound on the retry.
• A departed adapter taints everything. When the radio itself is unplugged, the operator republishes the last paired set fully tainted and connected: false, so no allocation is stranded. The media bus republishes on the same pass with its own NoSchedule taint. The slice is deleted only while no adapter has answered and nothing is paired, which is the window before bluetoothd starts. Unpairing the last controller empties the paired set, not the slice: the media bus stays in it.
• The operator’s pod can restart under a live claim. The prepared CDI files survive on the host, so a running consumer keeps the device node it was given. The replacement pod creates each virtual device again, from the snapshot in the bond’s Secret, and the kernel numbers it from the free minors, which now include the numbers the controllers’ own nodes hold. A consumer whose nodes are not the ones the operator delivers after that takes the /node-moved taint, and its eviction is what repairs it. The bus socket’s directory is a host path for a related reason: a claim prepared against it names the same directory after the restart, where an emptyDir’s host path is under /var/lib/kubelet/pods/, keyed by the pod’s UID, and changes with the replacement pod.

The pairing API

Three CustomResourceDefinitions, group bluetooth.liken.sh/v1alpha1, each with its own reference page: an Adapter is one radio, a Peripheral is one bonded device, and a PairingRequest is one pairing window. A person creates a PairingRequest, edits a Peripheral’s spec, and deletes a Peripheral to unpair; the operator creates and reconciles everything else.

Where the bonds are stored

One Secret for each bond, in the operator’s namespace, named bluetooth-bond-<device> after the controller’s MAC. Each Secret holds the two files BlueZ keeps for the bond, byte for byte, and a bluetooth.liken.sh/adapter label naming the radio the bond is keyed to. The bonds follow the radio: a dongle carried to another machine takes its bonds with it, because the pod that claims it there lists the same Secrets.

The keys are in the cluster datastore. Whether they are encrypted at rest is a property of the cluster, not of this operator. Without encryption at rest the keys are base64 in the datastore and its backups.