The nosh toolset is a suite of system-level utilities for initializing and running a BSD or Linux system, for managing dæmons, for managing terminals, and for managing logging.
Note: This doco has been updated in preparation for the upcoming version 1.41 release, based upon the 1.41 development version that has been available for some time. Development is now in a feature freeze; and testing on FreeBSD, NetBSD, and Debian is ongoing. Version 1.41 is already running the WWW site mentioned below.
For an in depth look at the design and architecture of the toolset, see the introduction in the The nosh Guide. It being a member of the daemontools family, service management with the nosh toolset intentionally began by building upon the daemontools way of doing things; and for those familiar with daemontools there is a section in the Guide about familiar interfaces and commands.
The toolset brings a uniform approach to FreeBSD (and its derivatives such as GhostBSD, DragonFly BSD, and erstwhile PC-BSD/TrueOS), NetBSD, OpenBSD, and Linux-based operating systems such as Debian and Arch. This uniformity ranges from the same commands being available across systems through to the same keyboard map and font file format used for user-space virtual terminals across all systems. There are, conversely, a range of shim tools for when the finger memory is ingrained.
The toolset is available both as a source code archive and as a suite of packages built for a selection of systems. It runs on architectures and platforms ranging from NetBSD on a Raspberry Pi (as one of the servers for the author's WWW site, in fact) to Debian Linux on AMD64 desktop systems.
Conceptually, the toolset covers much the same ground as the Unix Service Access Facility (SAF) with its Service Access Controller (SAC) and its port monitors listen and ttymon, the AIX System Resource Controller (SRC), and the Solaris Service Management Facility (SMF).
It is originally intended for use on BSDs, and to fill the gaps where BSD users don't have the option of launchd, systemd, or upstart, and want more than what other daemontools family systems provide.
The BSD user sees:
BSD licensing: It's licensed under the MIT Expat Licence, the ISC Licence, and the FreeBSD Licence (in the belief that they are all in essence identical).
no more waiting for rc.delay:
The service management brings things up a lot more quickly.
skills transferrability from the systemd world:
There are whole suites of shim layers to make things familiar for someone coming to BSD from a systemd or upstart Linux background.
Want to run systemctl start wibble or initctl status wibble?
You still can, and the command will do the equivalent nosh service management thing.
skills portability from the OpenBSD world:
Want to run rcctl enable wibble?
You still can, and the command will do the equivalent nosh service management thing.
properly designed dæmon management:
The replacement service command does not have the pitfalls of the BSD service command.
It is part of the daemontools family design that dæmons are always started from a known parent process with a known initial execution state, a clean environment that is fully adjustable on a per-dæmon basis, and a known set of open file descriptors.
transparency and interoperability:
Service configuration is expressed with ordinary files, directories, and symbolic links in the filesystem; so is service status.
nosh service bundles and their configurations can be inspected and modified with the likes of the ls, rm, find, and ln commands, with no need for a running Desktop Bus, and even without the management programs running at all.
Status and control information for running dæmons is in a well-known format that has been stable since the 1990s and that not only can be manipulated with other management toolsets but can also be manipulated from ad hoc tools incorporating the likes of Peter Ruibal's and Andrés J. Díaz's supervise library for Python or Voxer's JavaScript daemontools library for Node.
easy ports of Linux packages:
Have a package that comes with a service unit?
It can be run through convert-systemd-units to produce a native nosh service bundle that will run on a BSD.
Have a package that comes with a service unit and a socket unit?
Such a pair can be converted, too.
notions from UNIXes: It's a fully fledged service and system management package. Like the AIX SRC, system management is separated from service management. Like the Solaris SMF, there are targets/milestones. Terminal login is arranged as proper, fully fledged, services that are integrated with the service management and not bolted on to the side of it.
console-free operation: The system might be a smart telephone, or a server in a datacentre, or even a watch. It's not necessarily convenient to operate it using the system console. It might not even have a physical serial port socket. So kernel (and other) logs go to log directories; the terminal management system provides user-space virtual terminals that can be accessed from an SSH session in a pinch; and mechanisms are not in general designed to demand the use of the console.
polish:
This doesn't just hand over the raw toolset and leave the rest to you.
This package runs real BSD machines right now, and has been for some time.
One of the early goals of the package was to write service bundles that did what the 157 scripts in FreeBSD's /etc/rc.d do.
See the roadmap for the detailed status of what is left of this now largely complete effort; such that one has been able to run working desktop and low-end server machines using out-of-the-box services, for over a decade.
no kernel upgrade treadmill: nosh runs on the FreeBSD/NetBSD kernel as-is. It requires no kernel extensions or reworking.
no more waiting for Godot:
The launchd on BSD train is never coming and it's long past time to realize that.
It's now 2025, and one can do proper service and system management on BSD right now, with tools that have been succesfully running FreeBSD systems for well over a decade at this point.
(To emphasize how long this train has never been coming, earlier versions of this blurb said "It's now 2015" and at the time the toolset had been running FreeBSD systems for a couple of years.)
And you don't have to learn XML, let alone put it into process #1.
better virtual terminals: If the system does need a console, user-space virtual terminals provide better than the terminal emulators that are built into an operating system's kernel, without requiring X11 or Wayland or some other GUI. There are actual italics and boldface, not fakery using CGA colours; characters from multiple Unicode planes, not 512 code points at a time; and even the ability for TUI applications to recognize the "application" keys on "multimedia"/"office" keyboards. You really could search in your TUI text editor using your keyboard's "search" key, as user-space virtual terminals give it a terminal control sequence.
better login experience:
getty was a thing of the past back in 1988, on Unix.
But the BSDs still use it, "glass teletype", line speed negotiation on things that have no line speeds, "cooked mode" line discipline, and all.
Now you can catch up with the 1980s, not only with proper services, but also with a login user interface experience that recognizes that you might have these new cursor-addressable terminals with colour and arrow keys and more than a 7-bit character set that the 1980s have brought along.
No more fumbling with there being just "erase" and "kill" that only operate from the end of the line, when you make a typing mistake logging in.
login-envuidgid provides TUI dialogue boxes, that PAM was actually designed to enable, that understand cursor and editing keys.
It can even use MouseText, that new and cool 1980s idea from the Apple IIe that allows TUIs to draw GUI-like dialogue boxes.
However, it can also be used by Linux users, Debian Hurd users, and others. Conversely, the Linux user sees:
It's not systemd:
This is a point in its own right, for some people.
A rather more sensible point of view is that it's part of maintaining heterogeneity in Linux.
systemd unit files do not lock one in to systemd forevermore, because one can run them through convert-systemd-units and get nosh service bundles.
And nosh is part of a family of toolsets where mix-and-match composability is a design feature.
But one doesn't have to un-learn all systemd administration habits:
Like the BSD user, one can still go on typing systemctl start wibble if one wants.
Or service wibble start; as there is a bunch of System 5 command shims, too.
Or even initctl start wibble; as there is also a bunch of upstart command shims.
But it is in the daemontools family:
The fundamental concepts and architecture are well-known; other people's toolsets will interoperate, mix-and-match fashion; and there are widespread third party resources, from tutorials to run program collections, that can be adapted to it with ease.
Know about running daemontools in a Docker container? The principles are much the same. (Indeed it is simpler. A service bundle for SSH is one of the several hundred service bundles pre-supplied, so you don't have to make one.)
Know about running Ruby under runit?
The ideas are all the same, and the change to the run script is a simple substitution of envdir for chpst -e, although that's by no means mandatory.
Know about running node under runit or running node under daemontools? The same principles apply; in the latter case the same command names even apply, although one can do the logging in a better way (see the nosh Guide pages on logging).
Know about running Tomcat under daemontools?
Again, it's just a simple change from multilog t to cyclog (since this is the common use case where none of the advanced features of multilog are used).
Of course, it's not even that if one has multilog present, taken out of daemontools or daemontools encore; since one can mix and match.
separation of policy and mechanism:
service-manager is "mechanism", providing the raw mechanics of supervising dæmon processes.
system-control and service-dt-scanner (a.k.a. svscan) are "policy", determining when, where, and why services are started and stopped.
separation of service management and system management:
The overall state of the system — emergency mode, rescue mode, "multi-user" mode, halted, and so forth — is the domain of the system-manager.
The states of individual services — stopped, running, starting, stopping, failed — are the domain of the service-manager.
avoidance of continually regenerated ephemera: Things such as mount@, fsck@, and ttylogin@ services are not stored in non-persistent storage and regenerated on the fly over and over again. Taking a leaf out of the BSD books, they are stored persistently and re-generated when the source information actually changes. Conversion is performed by a system of semi-automated conversion tools.
properly designed dæmon management: As for BSD users (q.v.).
console-free operation: As for BSD users (q.v.).
transparency and interoperability: As for BSD users (q.v.).
no kernel upgrade treadmill: There are no requirements that one be running the absolute latest kernel and keep on upgrading. nosh doesn't require Desktop Bus in the kernel.
polish:
One of the stumbling blocks for daemontools adoption has historically been rounding up run programs.
Several people have published collections of run scripts, such as Gerrit Pape's famous collection of just under 80.
There are (as of version 1.14) some 230 pre-supplied service bundle directories in the nosh-bundles package, ranging from GNU cron to mongodb, from OSSEC HIDS to RabbitMQ server, from exim4 to dnscache, from bcron to OpenStack.
no central logging bottleneck: The daemontools family way is for individual services to have entirely separate log streams processed by entirely separate log services, which have no access to one another's files and directories and have no need of being the superuser at any point. A service that logs a vitally important line once per week won't have its output of the last few weeks entirely flooded out of the log by a prolific service generating thousands of lines of low importance stuff per minute.
no mandatory logging tool:
There's a whole range of ways to log just to local log files alone, from dumblog (supplied with freedt) through cyclog (supplied with nosh) to s6-log (supplied with s6).
Other people have written further tools if one really does want to log over TCP or UDP to a remote server.
The nosh Guide has pointers in its "Logging" section.
better virtual terminals: As for BSD users (q.v.).
better login experience: As for BSD users (q.v.).
Desktop Bus if you really must:
Although the toolset does not employ Desktop Bus itself, Desktop Bus integrated with some form of service manager is necessary for various graphical desktops.
So the toolset comes with pre-prepared service bundles for both system-wide and per-user Desktop Bus dæmons, and a dbus-daemon-launch-helper that allows Desktop Bus to launch services the nosh way.
reduced code surface in critical system process #1:
system-manager does not link to encryption libraries, Perl regular expression parsing libraries, or compression libraries.
BSD mechanisms and BSD-style tooling:
In order to make things more uniform across platforms, the toolkit includes BSD-style versions of several utilities (e.g. ifconfig, unvis, procstat, and ps) for non-BSD systems.
Furthermore, the toolset enables various BSD mechanisms (e.g. /etc/login.conf and ~/.login_conf for controlling initial login session setup irrespective of login shell, and vis-encoded output that can be reliably processed by awk and Miller) on non-BSD systems.
The features break down into dæmon management, system management, provision of socket-based services, pseudo-terminal and virtual terminal management, and log management.
Its fundamental dæmon management mechanism is that of the daemontools family, which is explained at that hyperlink for the unfamiliar. It comprises several of the augmentations from that family, and further augmentations that are new and unique to nosh. Augmenting that mechanism, or built on top of it, are:
targets/milestones: The extensible system target mechanism is a set of otherwise ordinary service bundles, that "want" or "conflict" with whole sets of other services.
configurable service restart:
To the main run script are added start, stop, and restart scripts.
restart controls service restart: whether and under what circumstances it happens.
It can also be used for logging/notifying a system operator of service failures, restoring a sane service state after an abort, and even simply inserting configurable delays so that continually failing services don't restart too frequently.
service interdependencies and startup/shutdown orderings:
The service and supervise directories form two subdirectories of what is known as a service bundle, that also contains service interdependency, ordering, and installation information in other subdirectories.
more chain-loading tools:
There are some extra chain-loading tools for manipulating process state, including such things as prependpath and appendpath.
a choice of alternative service management policies:
The /service directory can be eliminated in favour of a decentralized system that allows one to (for example) place "boot" service bundles in /etc/service-bundles, "system" service bundles in /var/service-bundles, and "add-on" service bundles anywhere that one likes.
logging as "first class" services: "Log services" are now merely one case of a general-purpose mechanism. Although one can still employ a one-log-service-per-main-service mechanism, one can also alternatively arrange fan-in of multiple services to a single logging service, and pipelines of three or more services.
Several notions are retained, including:
the filesystem as the database:
Everything involved in services is done with files and directories in the filesystem.
Service interdependences, for example, are expressed with ordinary symbolic links.
(Unlike systemd, nosh doesn't attempt to be "clever" with the contents of symbolic links, either.
They are used with the conventional filesystem semantics.)
the filesystem as the control/status API:
The service control/status API is the familiar supervise directory.
limited and controlled parsing: Parsing the same things over and over, from text to machine-readable forms, is avoided; parsing is done in its proper place, at configuration time not at run time; and parsers are avoided in security-sensitive areas. There are no parsers in process #1, for example.
For compatibility:
The daemontools /service mechanism can still be used.
However, unlike some other daemontools-style service scanners, the monitoring process does not use polling every few seconds.
It relies upon the NOTE_WRITE capability of the kqueue() system function to watch without polling for directory modifications.
A mechanism for converting systemd unit files (within certain limits that should cover the majority of units, albeit by no means every possible unit) to service bundles is provided.
Several commands have daemontools aliases: svstat as an alias for service-status, svscan for service-dt-scanner, and svc for service-control, for examples.
The bundle mechanism falls back to a daemontools-compatible directory structure, and the daemontools-encore augmentations to the supervise control/status mechanism fall back to a reduced-functionality Bernstein-daemontools-compatible API.
The preset command can employ systemd preset information, FreeBSD/TrueOS enable/disable information from /etc/rc.conf and /etc/rc.conf.local, and on/off information from /etc/ttys.
Other features:
A utility is provided for reading the daemontools-encore status API for a service and returning a message and an exit code suitable for incorporating into Nagios as plug-in that checks daemontools/nosh dæmon states. The utility returns critical errors (to Nagios) about unexpectedly down dæmons, and warnings about dæmons that aren't staying up for longer than a second or so.
The socket mechanism is compatible with UCSPI services, as well as with services that use systemd's file descriptor passing protocol.
Taking a leaf out of systemd's (and inetd's) book, the package augments Bernstein's UCSPI-TCP mechanism.
There are separate chain-loading programs for listening on sockets (tcp-socket-listen, local-stream-socket-listen, and local-datagram-socket-listen) and for accepting client connections (tcp-socket-accept and local-stream-socket-accept) to then spawn services in response, with configurable concurrency limits.
One can use them in combination or individually.
Thus listening on a socket and accepting client connections can be separated.
Because of this, the package also works with services that expect the "listening" file descriptor rather than the "accepted" file descriptor, which Bernstein's UCSPI-TCP does not handle.
There is also a ucspi-socket-rules-check chain loading program for implementing socket access control for both UCSPI-TCP and UCSPI-UNIX.
The target mechanism is similar to that of systemd (and, to a lesser extent, Solaris SMF's "milestones").
The procedures of enabling, disabling, starting, and stopping services are intentionally fairly similar to those of systemd, and the standard system targets are also intentionally similar to targets in systemd.
The targets themselves have conventional well-understood names such as local-fs, poweroff, rescue, and basic.
There's an emergency target that is started if the system manager is handed the -b option; which gives an emergency superuser login prompt without attempting to mount filesystems.
There's a rescue target that is started if the system manager is handed the -s option; which gives an emergency superuser login prompt after attempting to mount filesystems.
The multi-user and server targets are similar to the similarly named SMF milestones.
There are no run levels. run-levels have been obsolete in Unix since 1990.
The service manager can be used subordinate to another system management mechanism, such as systemd.
But the package also includes a system manager proper, separate from the service manager, that is intended to be used as process #1 of a BSD or Linux system.
It handles system state including bringing the system up into its normal running state; responding to reboot, halt, and poweroff requests; bringing the system up in "rescue" and "emergency" modes; and fielding things like power notifications and a secure attention key on the system console device.
It also does some of the mandatory system initialization tasks that process #1 is quietly expected to do, such as mounting "API" filesystems, creating "API" device files, and applying bodges to the system clock as early as possible.
The nosh system manager has no dealings in terminals, which are no longer the worry of process #1. Like with the Solaris SMF, terminal login sessions are simply more services, managed by the service manager. Terminal management features include:
With the chain-loading vc-get-terminal, open-controlling-terminal, vc-reset-terminal, and login-envuidgid utilities, one can replace the getty and login programs entirely, for both virtual terminals and any null-modem real terminals.
This is standard out of the box.
In the pre-built service bundles there are ttylogin@ service bundles for providing TUI login services on kernel virtual terminals (BSD and Linux) and user-space virtual terminals, using these tools in such chains.
As mentioned, one can still arrange to enable/disable ttylogin@ service bundles determined by on/off/onifconsole information in the the BSD (and pre-System 5) /etc/ttys, and still control things that way if desired.
The pre-built packages for virtual terminals indeed do this.
One can run user-space virtual terminals, to augment or even to replace the operating system kernel's virtual terminals. This is a combination of:
A console-terminal-emulator utility, that does the actual emulation work.
It aims to be drop-in compatible with the netbsd6 (NetBSD kernel virtual terminal in mis-named "vt100" mode), linux (Linux kernel virtual terminal), and teken (FreeBSD version 9 and later kernel virtual terminal) terminal types.
Several co&oouml;perating realizer utilities, that realize a virtual terminal via real, physical, display device and HID hardware.
A console-multiplexor utility, that multiplexes one or more virtual terminals onto another one, allowing one to switch between them using keyboard hot-keys.
In combination user-space virtual terminals look like this. For some background on the design, see my white paper in user-space virtual consoles from a decade ago. The design has changed moderately since then, but the ideas and reasoning are the same. See also the white paper on how parts of this system can be integrated with BRLTTY to provide user-space virtual terminals for BRLTTY users that come up before TTY login.
By replacing vc-get-terminal and login with pty-get-terminal and pty-run respectively one can create a tool that can run arbitrary commands attached to the slave side of a nonce pseudo-terminal.
As a bonus feature, the source package contains ptybandage and ptyrun execlineb scripts doing exactly this that provide workalikes for Dan Bernstein's never-completed 1999 ptyget utilities.
These are not the only uses of these tools.
For examples:
login-banner can also be used to write out a parameterized notice to a service's log file.
console-terminal-emulator can be attached to a "call-out" terminal device in order to log on to another system using (null-modem) serial connections.
The nosh package includes cyclog, a simple logging dæmon that covers the commonest simple use cases and that is modelled after the program of the same name that used to come with daemontools.
The log file sets that it writes are automatically rotated, timestamped, size-capped both as to total size and as to individual file size, and strictly size capped (in that there's no way, as there is with logrotate-managed logging, for a log file set to even temporarily exceed its size cap).
There are also tools for importing logs into nosh, turning them into streams of log data that can be pushed through to a log service.
syslog-read is a UCSPI tool that can be attached to a local or a UDP datagram socket.
klog-read is a UCSPI tool that can be attached to a FIFO or a local or TCP stream socket.
For extensive details, see their manual pages and the "Logging" section of the nosh Guide.