Problem

qmail-queue fails with error 64 on some FreeBSD systems.

For example:

# /var/qmail/bin/mailsubj testing root <<EOF
just a test
EOF
qmail-inject: fatal: qq trouble creating files in queue (#4.3.0)
#

If you have installed qmail correctly, the problem maybe that the sysctl variable security.bsd.hardlink_check_gid is set:

$ sysctl security.bsd.hardlink_check_gid
security.bsd.hardlink_check_gid: 1
$

Solution / Workaround

If you don’t have a specific reason for setting hardlink_check_gid, then clear it and make sure it isn’t set in /etc/sysctl.conf.

# sysctl security.bsd.hardlink_check_gid=0
#

If you cannot clear hardlink_check_gid, then you can set the set-group-id bit on /var/qmail/bin/qmail-queue:

# chmod g+s /var/qmail/bin/qmail-queue
#

Explanation

Messages are injected into the qmail queue with a specific sequence of operations to ensure reliability. One of the steps in the sequence is a call to link(2) by the setuid process qmail-queue. The link(2) call fails on FreeBSD systems where the security.bsd.hardlink_check_gid sysctl variable is set to 1.

The purpose of hardlink_check_gid is:

$ sysctl -d security.bsd.hardlink_check_gid
security.bsd.hardlink_check_gid: Unprivileged processes cannot create hard links to files owned by other groups

This means that a process whose effective gid and supplemental groups do not match the group of a file cannot create a hard link to that file. This is significant in the case of qmail-queue because of the following sequence of events:

1. qmail-queue is invoked by a user to place a message into the queue. qmail-queue is setuid qmailq and the permissions on the queue directory (/var/qmail/queue/pid) are qmailq:qmail:700. The newly created file is owned by qmailq because of the effective user id of the qmail-queue process, and the group of the newly created file is qmail because of the group ownership of the pid directory.
2. qmail-queue then tries to create a hard link to this file. This fails because the effective gid of the process is not the same as the group ownership of the file. The effective gid of the process is whoever invoked qmail-queue.

So the workarounds work because either the restriction is lifted, or the effective gid of the process wanting to create the hard link is changed to qmail.