9.3. Controlling Authentication
This page covers how PMIx authenticates connections on a system, and the controls a system administrator has over it.
9.3.1. How PMIx authenticates connections
When a process connects to a PMIx server, a security (psec)
component establishes the process’s user and group. Two exist:
native- asks the kernel. Always built, and the default.munge- uses the MUNGE daemon.
Either serves processes on the server’s own node only: a server takes no connection from another node.
What each one checks is described in Accepting Connections. The host environment then makes its own checks before accepting the connection.
The settings that count are the server’s. A server accepts only the components it has loaded, and passes its list to the processes it starts. A process cannot use a method the server does not offer. So apply the controls below where the host environment’s daemons read them.
You control which methods are available in four ways, from the most permanent to the least.
9.3.1.1. 1. Do not build the component
munge is built only when you ask for it, with
--with-munge[=DIR].
To leave out a component that would otherwise build, name it in
--enable-mca-no-build:
shell$ ./configure --enable-mca-no-build=psec-munge ...
Keep at least one component. A server with none fails to start; it never accepts connections without authentication.
See CLI Options for optional support libraries and Installation options.
9.3.1.2. 2. Restrict the components in a parameter file
The psec MCA parameter lists the components that may be used, or,
with a leading ^, those that may not:
# use only munge
psec = munge
# use anything except munge
psec = ^munge
Put the setting in a parameter file. A parameter file belongs to one PMIx installation: each PMIx library reads the files under its own installation prefix when it initializes, and no other. Put the setting in the installation the host environment’s server runs from - its settings are the ones that count. A system with more than one PMIx installation, or applications that bring their own library (in a container, say), has a set of files for each:
$prefix/etc/pmix-mca-params.conf- the installation’s default. A user’s$HOME/.pmix/mca-params.confand the user’s environment can override it.$prefix/etc/pmix-mca-params-override.conf- settings here take precedence over the environment and over every other parameter file. Use this file for settings that users, and the host environment, should not change.
A process reads these files because its environment does not say
otherwise. The files themselves are named by MCA parameters
(mca_base_param_files, mca_base_override_param_file), so a
process whose environment sets those - or sets
mca_base_param_files to none - reads other files, or none. A
parameter file is therefore the policy for the processes started
without such settings: the host’s servers, when the administrator
starts them or controls their environment. To make a component
unavailable to every process of an installation, build the
installation without it, or remove its plugin (below).
A process running as root reads a parameter file only if root owns it and no other user or group can write to it.
See The Modular Component Architecture (MCA) for the file format and Selecting which PMIx components are used at run time for the include and exclude syntax.
9.3.1.3. 3. Remove the component’s plugin
If PMIx was configured with --enable-mca-dso, each component is a
separate plugin file under $prefix/lib/pmix. Deleting a plugin
makes that component unavailable - for example, a component that was
built by mistake:
shell$ rm $prefix/lib/pmix/pmix_mca_psec_munge.so
Components built into the library itself (the default) cannot be removed this way. Use the parameter file instead.
pmix_info lists the components an installation can load:
shell$ pmix_info | grep psec
9.3.1.4. 4. Tune the connection checks
These MCA parameters adjust the checks PMIx makes on connections:
Parameter |
Default |
Effect |
|---|---|---|
|
|
When |
|
none |
Network interfaces the server may or may not listen on. Keep loopback included. |
|
5 |
Seconds a server allows for a new connection’s opening message. |
|
0 |
How many connections may still be sending their request, and,
separately, how many tools may wait for the host’s decision. 0
means an eighth of the process’s file-descriptor limit, for each;
-1 means no limit. A user other than the server’s, root and the
host’s |
|
16777216 |
Most bytes the server allocates, between them, for connection requests still arriving. Each request’s buffer starts at 4 KB and doubles as its bytes arrive, up to the length it names; a request whose buffer cannot grow within the limit is closed. 0 means no limit. |
|
32 |
Largest message accepted, in megabytes. 0 means no limit. |
|
none |
Servers a client or tool will talk to on its host besides one
running as its own user or as root, when no server started it:
|
To see every parameter and its current value:
shell$ pmix_info --param psec all
shell$ pmix_info --param ptl all
Some policies are set by the host environment rather than by MCA parameters, because they depend on how the host is deployed:
Whether tools of other users may connect. By default a server refuses a tool whose user is not the server’s own, root, or the user the host named as the owner of its local socket (
PMIX_USERID). A host can allow them (PMIX_SERVER_ALLOW_FOREIGN_TOOLS), and must then also widen the server’s local socket to them (PMIX_SOCKET_MODE, withPMIX_GRPIDfor a group): by default only the server’s user can connect to it.Processes on other nodes never connect. A server takes no TCP connection at all by default, and when it takes clients and tools of earlier releases it listens for them on loopback only. A host that asks for connections from other nodes (
PMIX_SERVER_REMOTE_CONNECTIONSset to true) fails its init. A client or tool refuses to dial a server at an address that is not its own host’s, and a server refuses a TCP peer at such an address.Clients are always checked against the user and group the host registered them with. No setting changes that.
Consult your host environment’s documentation for how it exposes these choices.
9.3.2. Temporary files
A PMIx server writes rendezvous files, session directories and its
local socket under its temporary directory ($TMPDIR, or /tmp).
Tools find servers through those files. The local socket is in a
directory of its own, pmix-XXXXXX. A server that is killed can leave
its files behind; PMIx reclaims a rendezvous file whose server is gone,
but session directories and socket directories accumulate. Clean out old
pmix* entries as you would other stale temporary files.
Keep the temporary directory’s path short: the local socket’s path is
built under it, and is limited to about 100 characters. A server whose
path is too long fails to start. The default $TMPDIR on macOS is too
long - start the host environment with a shorter one.
See Session Directories.