FreeBSD daemon(8) Operations: Supervise Foreground Programs Predictably

FreeBSD’s daemon(8) utility detaches a program from its controlling terminal and can supervise a child process, restart it after termination, write PID files, and redirect output. It is useful for a foreground-capable program that lacks its own daemon mode or for a tightly scoped operational wrapper. It is not a general replacement for a proper rc.d service definition, dependency ordering, health checks, or an application-specific supervisor.

The distinction between the supervisor and the child is crucial. With restart enabled, a child PID can disappear and be replaced while the daemon process remains. A PID file for the child is therefore not interchangeable with a PID file for the supervisor. Confusing the two can make stop commands signal the worker, trigger a restart, and leave the service apparently impossible to stop.

Confirm whether daemon is the right layer

First determine whether the program already supports foreground mode and whether the package supplies an rc.d script:

service -e
service -l
ps auxww
grep -R "foreground" /usr/local/etc/rc.d /usr/local/share/doc 2>/dev/null

Inspect the package documentation and its service script rather than inferring flags from another release. If a maintained rc.d script exists, use it. If you need a persistent boot-time service, create or adapt an rc.d integration with explicit enablement, stop behavior, dependencies, and logging. daemon(8) can be used inside a carefully designed service script, but a bare command in a shell startup file is not equivalent to a managed service.

Use the application’s foreground mode when available. Programs that daemonize themselves may double-fork, detach, or rewrite PID files, defeating daemon’s process tracking. Do not wrap a daemonizing application unless the upstream documentation explicitly supports that arrangement. A foreground process should remain attached to the child instance so daemon can observe its exit and apply the configured restart policy.

Start with a controlled command line

The following is a template for an application that remains in the foreground and accepts a documented foreground flag:

daemon -f -P /var/run/sample.supervisor.pid -p /var/run/sample.child.pid -o /var/log/sample.log -H -R 10 /usr/local/sbin/sample --foreground

Replace the command, flag, user, paths, and restart delay with values supported by the installed daemon manual and application. The -P file identifies the supervisor; -p identifies the child. -R configures a restart delay and supervises the program. -o appends child output to a log file, while -H allows the output file to be reopened on SIGHUP for log rotation. -f closes or redirects standard descriptors according to the output options.

Before using the example, create parent directories with ownership and permissions appropriate to the service. daemon’s PID-file owner depends on which user invokes it; its -u option changes child privileges, not necessarily the supervisor’s ownership. Avoid making a log world-readable by default. The manual documents the output-file mode and the exact interaction among -f, -o, and syslog flags.

Choose restart delay and count intentionally. A short fixed delay can create a rapid restart loop when configuration is invalid or a dependency is unavailable. A bounded retry count may be appropriate for transient failures, while a long-lived service may require an external alerting policy and an operator-reviewed restart strategy. daemon does not decide whether a restarted program is healthy, ready, or making progress.

Do not use both a daemon’s own restart feature and an unbounded external restart loop unless the combined behavior is designed. Two supervisors can multiply restart rates, obscure the source of a process, and make stop operations race. Pick a single owner for restart policy, then document how it detects a dead child and how an operator disables retries during incident response.

Distinguish process identity and stop behavior

After startup, compare both PID files and the process tree:

cat /var/run/sample.supervisor.pid
cat /var/run/sample.child.pid
ps -axo pid,ppid,stat,command | grep '[s]ample'

The child may restart under a new PID while the supervisor’s PID remains stable. A PID file is a point-in-time reference, not proof that a process is healthy or even that the file still refers to the expected executable. Confirm command line, parent, start time, and service identity before sending a signal.

Stopping should be integrated with the service’s management path. If an rc.d script launches daemon, its stop action must signal the supervisor or use the correct pidfile, then wait for the child to exit according to a timeout policy. Sending a termination signal only to the child can cause daemon to restart it. Do not manually remove PID files to force startup; a locked PID file is a useful concurrency safeguard and should be cleared by the process lifecycle.

Test shutdown and restart in a maintenance window. Confirm the application handles TERM, flushes state, releases sockets and files, and exits before the service timeout. If it ignores TERM, design escalation deliberately. A forced kill can leave application state inconsistent, and a supervisor may immediately create a replacement unless it has been stopped first.

Route output and rotate logs

daemon can send child output to syslog, a file, or both according to its options. Choose one consistent destination and verify it before deploying. Appending to a file is convenient for a small service but requires rotation and disk-capacity monitoring. Syslog provides centralized routing where configured, but facility, priority, and tag must be explicit enough to distinguish the program.

When using an output file with the -H option, a SIGHUP tells daemon to close and reopen the output file. Coordinate this with newsyslog or the site’s rotation tool, then test that new output lands in the new inode after rotation. Without reopen behavior, a process can continue writing to a rotated file that no longer has the expected pathname.

Do not redirect both daemon and the application to overlapping log paths. Duplicate output makes incident timelines confusing and can expose secrets if the application prints credentials. Test normal startup, child crash, repeated restart, shutdown, and log rotation with harmless input, then confirm each event appears once in the intended log.

Keep stdout and stderr behavior explicit. Some applications log normal operation to stdout and errors to stderr, while others use only a logging library. The -m output mask can select streams; consult daemon(8) before changing it. Empty log files do not prove the service is quiet if the program logs to syslog or another file.

When rc.d should own lifecycle

For a long-running host service, an rc.d script normally provides the FreeBSD-native boot and administration interface. It can define a service name, enable variable, command, configuration, PID file, and dependency ordering, and it can expose expected service actions. The rc.subr framework and service command help administrators consistently query, start, stop, and restart services.

daemon may be appropriate as an implementation detail when the application must remain foregrounded and the service script uses daemon’s supervision features. The script still needs to identify the correct PID file and avoid daemonizing twice. It should define when startup is considered complete, where logs go, how restart loops are bounded, and what exit status indicates a failed launch.

For a one-shot administrative command, daemon may be unnecessary. For a user session process, a terminal multiplexer or job-control approach may better match the need. For a complex production service, consider package-supported service integration or a purpose-built supervisor with health checks and dependency semantics. Select the manager that owns lifecycle rather than layering tools without an explicit responsibility model.

Diagnose failure patterns

The PID file exists but the process is absent. Check whether startup failed after file creation, whether the file is stale, and whether the PID was reused. Compare the process table and daemon logs before removing the file.

The child keeps restarting. Capture its exit status and logs, inspect configuration, permissions, required sockets and dependencies, and verify whether -r or -R is enabled. Temporarily stop the supervisor using the service’s control path before running the command in the foreground for diagnosis.

Stop appears ineffective. Check whether the signal reached the supervisor or only the child, whether the child restarts, whether the application handles TERM, and whether an rc.d script is targeting the right PID file. Avoid killall by executable name on a shared system.

Logs stop after rotation. Confirm -H was supplied with an output file, the intended process received SIGHUP, and the new file has appropriate ownership. Inspect open descriptors with fstat if needed and verify a new message is written after rotation.

Two instances run. Review PID-file paths, lock behavior, rc.d invocations, manual shell launches, and external monitoring restarts. Ensure a single-instance guarantee exists before enabling restart-on-failure.

Acceptance and evidence

A supervised service is accepted when it starts once through the intended service manager, runs the expected foreground child, writes distinct supervisor and child identities if both are configured, restarts according to a reviewed policy, and stops without an orphan or immediate respawn. Verify normal logs and a controlled rotation event. Test both failed startup and child termination without using production traffic.

Record the FreeBSD release, daemon options, application version and foreground mode, PID-file semantics, stop command, restart delay/count, log destination, signal behavior, and test results. daemon(8) supplies process detachment and restart mechanics; it does not provide readiness, dependency health, alerting, or application recovery. Keep those responsibilities visible in the surrounding service design.

Related:

Sources: