OpenBSD Ports - Porting Guide [Handbook Index]

• Overview
• Porting Checklist
• Handling Complex Situations
  • Know the Software
  • Figure Out Important Options
  • The Ideal Case: MULTI_PACKAGES and PSEUDO_FLAVORS
  • Interdependencies Between Subpackages
  • True FLAVORS and PKGNAMES
• Updating Ports
  • Update Checklist
  • Committing Port Updates
• OpenBSD Porting Policy
• Security Recommendations
• Generic Porting Hints
• Other Helpful Hints
• Additional Information

Overview

The most important thing to do is to communicate. Ask people on ports@openbsd.org if they are working on the same port. Tell the original software authors about it, including problems you may find. If licensing information appears incorrect, tell them. If you had to jump through hoops to make the port build, tell them what can be fixed. If they are only developing on Linux and feel like ignoring the rest of the Unix world, try to make them change their view.

Communication makes the difference between a successful port and a port that will slowly be abandoned by everyone.

First, look at the porting information on this page. Test, then re-test, and finally test again!

OpenBSD fully supports updates. This means that some issues must be taken into account.

Submit the port. Create a gzipped tarball of the port directory. You can then either place it on a public HTTP server, sending its URL to ports@openbsd.org, or send the port MIME encoded to the same address.

Porting some new software takes time. Maintaining it over time is harder. It is quite okay to port software and let other people handle it afterwards. It is also okay to help other people update and maintain other ports, as long as you communicate to avoid doing the same things twice.

In OpenBSD culture, MAINTAINERship is not a status item, but a responsibility. We have CVS and comments to give credit to the person who did the work. A port MAINTAINER is something else: a person who assumes responsibility for the working of the port, and is willing to spend some time ensuring it works as best as can be.

Porting Checklist

1. If you want to be a maintainer, subscribe to ports@openbsd.org.
   • This is where all ports discussions take place.
   • Reading this list is important since many announcements are made there.
   • You will find a lot of porting-savvy people there. They can often give you good advice or test ports for you.
   
   
2. Being a maintainer means more than just submitting ports. It also means trying to keep them up to date, and answering questions about them.
   
   
3. Check out a copy of the ports tree from CVS. You can find instructions on how to do this at the AnonCVS page. As a porter, you should keep your base OS, ports tree, and installed packages up to date.
   
   
4. From the names of the first level subdirectories of /usr/ports/, pick a primary category for your port. Create a new directory below /usr/ports/<category>/ or /usr/ports/mystuff/<category>/ and create the basic infrastructure there. Copy the template Makefile from /usr/ports/infrastructure/templates/Makefile.template. Fill in CATEGORIES with the primary category you have chosen.
   
   
5. Add the fetch portions of the Makefile.
   • Fill in EXTRACT_SUFX if it's anything besides .tar.gz. Other examples are .tar.Z, or .tgz.
   • Fill in DISTNAME with the name of the file minus the extract suffix. If you have foo-1.0.tar.gz, DISTNAME is foo-1.0.
   • Fill in SITES with a list of URLs to the locations where the distfile is kept, https://ftp.openbsd.org/pub/OpenBSD/distfiles/ for example. Don't forget the trailing slash. Try to have at least three distinct sites as well. Place the most easily accessible first as they are traversed in order.
   • Keep in mind that fetch references the file as ${SITES}${DISTNAME}${EXTRACT_SUFX}. All three are used. Don't set DISTNAME to point to the file directly.
   • You can check to see if you have filled these values in correctly by typing make makesum.

   For more complex ports, you have more options and tools available to you:

   • You also have the variable PATCHFILES available. This is a list of vendor (not OpenBSD) patches to the port. Common uses are things like security or reliability fixes which are occasionally distributed as a patch rather than a full release.
   • If your ports are available over large public mirrors such as GNU, Sourceforge, or CPAN, we have already provided a list of sites for your use in /usr/ports/infrastructure/db/network.conf. Set SITES to ${SITE_GNU}, or ${SITE_SOURCEFORGE}, etc. To simplify this process, use the construct ${SITE_FOO:=subdir/} to append the distribution subdirectory.
   • For ports with source distributed from GitHub there are several possibilities.
     • In some cases, for example icinga or darktable, a distribution tar file has been prepared and is available in a /releases/ directory. Where available, use these in preference to other options as they usually include the proper build infrastructure (configure scripts, etc) which are often missing from "tagged" releases. Specify as normal, using something like SITES=https://github.com/acct/project/releases/download/relname/
     • In other cases, files have been tagged but they rely on github's on-the-fly archive creation. These can be specified using the following bsd.port.mk(5) variables: GH_ACCOUNT, GH_PROJECT, GH_TAGNAME.
     • Occasionally there is no alternative but to have a port reference only a commit id. In this case, GH_COMMIT should be used instead of GH_TAGNAME.
   • Ports normally correspond to given versions of software. Once they are retrieved, files are checksummed and compared to the recorded checksum(s) in distinfo. So, to avoid confusion, DISTFILES and PATCHFILES should have clearly visible version numbers: don't retrieve foo-latest.tar.gz if it is a link to foo-1.0.5.tar.gz. File names should not contain just the version number (1.0.5.tar.gz); files can be renamed during fetching if necessary, see bsd.port.mk(5). If something is only available in an unversioned file, gently ask the original program author to make such distinctions clear. In the meantime, if you must use a file like this, set DIST_SUBDIR to what a reasonable DISTNAME would be, like foo-1.0.5, such that different, identically named versions of the distfile do not clash on the distribution file mirrors or build machines.
   • If a given port needs more than about five DISTFILES and PATCHFILES to work, use DIST_SUBDIR to avoid cluttering DISTDIR (/usr/ports/distfiles by default) too much. In this case, DIST_SUBDIR must not include version numbers. When the port is updated to a later version, some distfiles may not change, but will be refetched if DIST_SUBDIR is changed. Even if all distfiles change, it is easier for the user to track cruft.
   • All DISTFILES and PATCHFILES don't necessarily come from the same set of SITES. Supplementary sites can be defined using suffixes: DISTFILES.alt = foo-1.0.5.tar.gz will come from SITES.alt.
   • Some ports don't always need to retrieve all files in all circumstances. For instance, some ports may have some compilation options, and associated files which are only required in such a case. Or they may need some files for some architectures only. In such a case, those supplementary optional files must be mentioned in the SUPDISTFILES variable. Targets such as makesum or fetch-all will fetch those supplementary files that the casual user doesn't need.
   
   
6. Create a checksum in distinfo by typing make makesum.
7. All files in DISTFILES* are usually processed during make extract. EXTRACT_ONLY may be used to limit extraction to a subset of files (possibly empty). The customary use of this variable is to customize extraction: for instance, if some DISTFILES need some special treatment, they will be removed from EXTRACT_ONLY and handled manually at post-extract stage. For historic reasons, make extract does set up the working directory first along with extracting files. Thus, providing a pre-extract or do-extract target is highly unusual (and fairly suspicious behavior, indicative of a high degree of obfuscation in the port).
8. Patches that need specific treatment should be mentioned in DISTFILES, and removed from EXTRACT_ONLY, for historic reasons.
   
9. Extract the port with make extract. Pay attention to where the base of the sources are. Usually, it's /usr/ports/pobj/${PKGNAME}${FLAVOR_EXT}/${DISTNAME}. You may need to modify the Makefile's WRKDIST variable if it is different.
   
   
10. Read the installation documentation and note what you have to do to build the port and any special options that might be needed.
    
    
11. Now is also a good time to figure out what kind of licensing restrictions apply to your port. Many are freely redistributable, but quite a few are not. We need two questions answered to distribute ports properly. These are the PERMIT_* values in the Makefile.

    PERMIT_PACKAGE tells us whether we can put the package on the mirrors.
    PERMIT_DISTFILES tells us whether we can mirror the distfiles.

    If both are permitted, simply set PERMIT_PACKAGE=Yes. Otherwise, set each variable to either Yes or a comment string indicating the reason distribution is not allowed. Pay attention to any special conditions you may need to fulfill later on. For example, some ports require installing a copy of the license. We recommend you place the license in /usr/local/share/doc/<name>/.

    In addition to the PERMIT_* values, put a license marker like # License above them as a comment, this way we know why the PERMIT_* values are set the way they are. For GPL, specify which version is applicable e.g. "GPLv2 only", "GPLv2+" (meaning "v2 or newer"), "GPLv3 only", etc.

12. Add configuration options to Makefile and/or create the configuration script.
    • If GNU configure is used, run ./configure --help to see what options are available. In particular, look for optional dependencies which may not be present on your system, but might be picked up on another system or in a bulk build.
    • Anything that you may want to override can be changed by adding the --option flags to the CONFIGURE_ARGS parameter in the Makefile.
    • Use CONFIGURE_ARGS+= to append to the variable. CONFIGURE_ARGS= will overwrite it.
    
    
13. Try building the port with make build.
    • If you're lucky, the port will go all the way through without errors.
    • If it exits with an error, you will need to generate patches for your port. Figure out what needs to be changed and make a patch for it.
    • the sequence to make a patch for a file is usually:
      • cd `make show=WRKSRC` ; cp foo/bar.c{,.orig.port}
      • edit foo/bar.c to fix the error.
      • cd - ; make update-patches
      • this will create patches/patch-foo_bar_c with your modifications.
    • The easiest way to reset the port and test your patches is make clean patch. This will delete the work directory, re-extract, and patch your port.
    
    
14. Begin a cycle of make build, edit source code, generate a patch using make update-patches and make clean patch.
    • Patches go in the directory patches/ and should be named patch-* with * being something meaningful. The preferred naming is patch-FILENAME, where FILENAME is the name of the file it is patching. (make update-patches does this automatically for you.)
    • Applying PATCHFILES is the first half of the make patch stage. It can be invoked separately as make distpatch, which is a convenient target for porters. Ignore this if you haven't set it.
    • Only patch one source file per patch file, please.
    • Use make update-patches to generate patches.
    • All patches MUST be relative to ${WRKDIST}.
    • Check that patches DON'T contain tags that cvs will replace (update-patches(1) will warn you about it). If they do, your patches won't apply after you check them in. You can check in your changes with -kk to avoid this.
    • Write a short explanation at the beginning of the patch file about its purpose (not mandatory, but for example often done for security patches).
    • Where your patches make sense for use outside of the ports tree, please feed them back to the author of that software.
    
    
15. Try setting SEPARATE_BUILD.
    • If the port can build with object files outside its source tree, this is cleaner (many programs using CONFIGURE_STYLE=gnu can).
    • This can also spare you some effort, as you will possibly be able to restart the cycle at configure most of the time.
    
    
16. Peruse the output (if any) and tweak any options in the Makefile. To repeat issue the command make clean configure.

    Configuration files will usually be installed somewhere under /usr/local and handled by pkg_add(1) @sample annotations. After a package has been installed the contents of pkg/MESSAGE will be displayed if it exists.

    The OpenBSD file locations are:

    user executables:			/usr/local/bin
    system admin executables:		/usr/local/sbin
    program executables:			/usr/local/libexec
    libraries:				/usr/local/lib
    architecture dependent data:		/usr/local/lib/<name>
    installed include files:		/usr/local/include or
    					/usr/local/include/<name>
    single-machine data:			/etc or /etc/<name>
    local state:				/var/run
    games score files:			/var/games
    GNU info files:				/usr/local/info
    man pages:				/usr/local/man/...
    read-only architecture-independent:	/usr/local/share/<name>
    misc documentation:			/usr/local/share/doc/<name>
    examples files:				/usr/local/share/examples/<name>
    

17. Begin a cycle of makes until the port is ready. Patch (see above) clean, and make until the port is generated. Get rid of all warnings if possible, especially security-related warnings.
    
    
18. Add COMMENT in Makefile. COMMENT is a SHORT one-line description of the port (max. 60 characters). Do NOT include the package name (or version number of the software) in the comment. Do NOT start with an uppercase letter unless semantically significant, and do NOT end with a period. DON'T EVER START WITH AN INDEFINITE ARTICLE SUCH AS "a" or "an" - remove the article altogether. The main usage of COMMENT is the one-liner from pkg_info(1):

    py3-cairo-1.24.0    cairo bindings for Python
    

    
    
    
19. Put a longer description of the port into pkg/DESCR. One to a few paragraphs concisely explaining what the port does is sufficient. Lines should be no longer than 80 characters. This can be done by first editing the DESCR file and then running it through fmt.
    
    
20. If the application needs to create a user or a group, choose the lowest free id from /usr/ports/infrastructure/db/user.list for your port to use and don't forget to include that file in your commit.
    
    
21. Install the application with make fake. Libraries should never be stripped. Executables are stripped by default, unless you set DEBUG_PACKAGES; this is governed by ${INSTALL_STRIP}. ${INSTALL_PROGRAM} honors this automatically and is preferable to unconditional stripping (e.g., by an install-strip target or by running strip from the Makefile). You can use objdump --syms to determine if a binary is stripped or not. Stripped files have no symbols in the SYMBOL TABLE.
    
    
22. Check port for security holes again. This is especially important for network-facing programs. See our security recommendations for that.
    
    
23. Make sure your /etc/mtree directory is up to date. (The next step uses the mtree lists to remove existing directories from generated packing-lists). Remember that the OpenBSD (U)pdate does not touch /etc... For automatic updating of /etc, sysmerge(8) may help.
    
    
24. Create pkg/PLIST. After the installation with make fake is complete, use the developer's command make update-plist, which creates or updates the file PLIST in the pkg directory. This file is a candidate packing list.

    Peruse PLIST and verify that everything was installed and that it was installed in the proper locations. Anything not installed can be added to a port Makefile post-install rule. Note that PLIST annotations are documented in the pkg_create(1) manual, and that update-plist(1) is also documented.

25. It is possible some directories do not need to be in the PLIST as they've been installed by a dependency; if you added to LIB_DEPENDS or RUN_DEPENDS, run make update-plist to remove these.
    
    
26. Test the packaging with make package, test installation of the resulting package with make install, and test its removal with make uninstall. When dealing with multi-packages, it may instead be convenient to use pkg_add(1) and pkg_delete(1) directly, setting TRUSTED_PKG_PATH to /usr/ports/packages/%a/all/ in the environment.
    
    
27. Verify dependencies. Peruse your logs to verify the port did detect what is mentioned in *DEPENDS, and nothing more. Check names, particularly in the make configure stage, for hidden dependencies (stuff that exists elsewhere in the ports tree and might be detected if the user installs some other ports first).
    
    
28. Verify shared library dependencies. Run make port-lib-depends-check and add every LIB_DEPENDS or WANTLIB annotation that is needed until it runs cleanly. You may want to read the update guidelines to understand why this is so important.
    
    
29. Check for regression tests, and whether they run cleanly. Set NO_TEST=Yes if a port has no test infrastructure. If dependencies are required to run the tests, but not to build the port, list them in TEST_DEPENDS. Please note: do not set NO_TEST if a port has an empty regression test infrastructure.
    
    
30. Test that make clean succeeds. Sometimes the make fake stage creates files in the build directory which will cause this to fail.
    
    
31. Run the /usr/ports/infrastructure/bin/portcheck utility in your port directory and take care of problems it finds, if any.
    
    
32. Mail ports@openbsd.org with a description, the homepage (if any), and a short note asking for comments and testing. Make sure to attach the port/patch to this email (or include a URL where it can be found) and send it out.

    Try to get others to test it on a variety of platforms for you.

    • The AMD64 systems are good because they are fast, and because sizeof(int) != sizeof(long) on this platform.
    • sparc64 systems are good because they are fairly common, can be quite fast, and their byte order is the opposite of i386; if you developed on sparc64, of course, you'd want it tested on i386.
    
    
33. Incorporate any useful feedback you get. Test it again on your platform. Get those who gave you feedback to test it again from your new port.
    
    
34. Get the port committed to CVS.

    If you are not a developer with CVS access, then you will have to find an OpenBSD developer to do the following steps for you (ask on ports@openbsd.org).

    Before you import anything, get at least one "OK" from another ports developer (the more the better).

    If using @newuser or @newgroup in the PLIST files, check that no users were added to /usr/ports/infrastructure/db/user.list since the port was created.

    For new ports we use cvs import, rather than adding new files individually. For example, to import a new lang/kaffe1 port, first do a dry-run import:

    $ cd /usr/ports/lang/kaffe1
    $ cvs -ndcvs.openbsd.org:/cvs import ports/lang/kaffe1 username username_yyyymmdd
    

    Where username is your account username, and yyyymmdd is the current date. If this succeeds, then you can remove -n to import for real. Your editor will be invoked so that you can enter a commit message. In the commit message, at least state what you are importing and which developers provided OKs. Make sure the import path is correct; it always starts with ports/.

    Alternatively, you can use ports/infrastructure/bin/portimport to import new ports.

35. Last but not least, add a one-line entry for the new port in its parent directory's Makefile, e.g., for ports/lang/kaffe1, add it to ports/lang/Makefile. Don't forget to commit any changes made to /usr/ports/infrastructure/db/user.list.
    
    
36. Maintain the port! As time goes by, problems may arise, or new versions of the software may be released. You should strive to keep your port up to date. In other words - iterate, test, test, iterate...
    
    
37. When updating a port, remember to handle dependencies! You shouldn't break any port that depends on yours. The sqlports package includes show-reverse-deps, making it trivial to know the full tree of ports that depend directly or indirectly on a given port. In case of problems, communicate with the maintainers of such ports. Likewise, be alert for dependency updates, and check that the maintainer did their job.

Handling Complex Situations

Know the Software

Identify options

The first step is usually to identify build options. You will often have to read the configuration log, see what stuff your port auto-detects. Read the configure script options. Read the port documentation for extra stuff.

Make options work

Recompile your port with various options. Install extra dependencies. Make sure your port detects them correctly. Add supplementary patches to ensure compilation. Test the result, and verify extra stuff does work.

Identify missing software

Some dependencies won't be fulfilled because the missing software has not yet been ported. It is highly recommended to explicitly disable those options. Failure to do that breaks bulk builds all the time: people port new software and import it, and soon after, old ports stop building because they detect the dependency, try to use it, and fail to build or package.

Check run-time dependencies versus build-dependencies

Update your packing-list with make update-plist. Use make port-lib-depends-check to see what libraries your software needs (that will end up in LIB_DEPENDS or WANTLIB, usually). Identify various files and binaries in the dependencies that have to be present for the port to work.

By this point, you should have a fair understanding of your port's working.

Figure Out Important Options

Considering all possible options, you should be left with a much smaller set of options for your port, mostly depending on what packages are needed to run the software. For now, do not worry about build dependencies. Remember that the OpenBSD ports system is focused on the end user, and the end user will want to install binary packages, so it doesn't matter if you need a huge piece of software to build your port if it doesn't show up as a library or runtime dependency.

The Ideal Case: MULTI_PACKAGES and PSEUDO_FLAVORS

• which new files show up when you activate each option
• which libraries/runtime files are needed when you activate each option

In this case, try setting the MULTI_PACKAGES variable to a list of -sub packages, add corresponding COMMENTS, and look at your packaging. Basically, MULTI_PACKAGES only affects the packaging: if you have MULTI_PACKAGES=-s1 -s2 all stuff relevant to the package will exist in two variants: COMMENT-s1 for the first package, COMMENT-s2 for the second package, PLIST-s1, PLIST-s2, DESCR-s1, DESCR-s2. You need to write those COMMENT-s1 and COMMENT-s2 in the Makefile, split your PLIST into two parts, and create DESCR-s1/DESCR-s2.

It is a good idea to start with the minimal framework work required: just copy the existing description and comments, because you will have to fiddle with MULTI_PACKAGES and stuff before you polish this.

Once you've separated the files properly, you will need to check dependencies: LIB_DEPENDS, WANTLIB, and RUN_DEPENDS will be split for each subpackage. It is usually time to check that your multi-packaging "works," and that those nasty dependencies you don't want to force on the user are indeed relegated to a specific subpackage.

Assuming everything works, you're mostly done. Just pick reasonable names for the various packages, and fill in the comments and descriptions. The end-user will be able to just install the package(s) they want.

But wait. What about the build, you say? Well, having a lot of dependencies during build is not a problem. Most packages are built by the OpenBSD team using special build runs (known as bulk builds) where a developer just builds all possible packages on a dedicated machine (or several, for slow architectures). Since everything will get built, having big dependencies is not an issue. Building the same thing several times, is an issue, though, which is why MULTI_PACKAGES are the best way to handle options (when possible): one build, one set of packages to test, better quality overall...

If you also want to help people who build packages themselves, you may consider adding PSEUDO_FLAVORS. A pseudo-flavor is a way to tweak an option (say, disable the graphical interface) that's very similar to actual flavors. In fact, the biggest difference is a functional difference: a pseudo flavor should only affect the set of packages that get built, but it is never allowed to modify the actual package contents.

For instance, assuming you separated the graphical interface into a separate subpackage (MULTI_PACKAGES=-main -x11), you could create a pseudo flavor no_x11 that avoids building the -x11 subpackage. The crucial point is that this flavor should NOT affect the -main package in any way.

You would end up with a Makefile that looks something like this:

COMMENT-main =	foo core application
COMMENT-x11 =	foo graphical interface

V =		1.0
DISTNAME =	foo-$V
CATEGORIES =	app

MULTI_PACKAGES =	-main -x11
PSEUDO_FLAVORS =	no_x11
FLAVOR ?=

WANTLIB = c m crypto ssl
WANTLIB-x11 = ${WANTLIB} X11 Xt

RUN_DEPENDS-x11 =	${BASE_PKGPATH},-main>=$V

CONFIGURE_STYLE =	gnu

.include <bsd.port.arch.mk>

.if !${BUILD_PACKAGES:M-x11}
CONFIGURE_ARGS += --disable-x11
.endif

.include <bsd.port.mk>

Interdependencies Between Subpackages

The infrastructure takes special care of library inter-dependencies: it can detect which WANTLIB come from external dependencies, and which come from inter-dependencies. While external LIB_DEPENDS and WANTLIB are checked at the start of build, LIB_DEPENDS and WANTLIB that refer to one of the subpackages currently being built will only be checked at packaging time (and thus packages may be created in a specific order to satisfy interdependencies).

The infrastructure provides specific variables to help in writing inter-dependencies: BUILD_PKGPATH contains the PKGPATH used during building the current packages, taking flavors and pseudo-flavors into account. It is highly recommended to use this variable instead of rolling your own: failure to do so will often trigger rebuilds in interesting flavors situations. For instance:

...
FLAVORS = a b
FLAVOR ?=
MULTI_PACKAGES = -p1 -p2
WANTLIB-p1 = foo
LIB_DEPENDS-p1 = some/pkgpath,-p2
...

LIB_DEPENDS-p1 = ${BUILD_PKGPATH},-p2

True FLAVORS and PKGNAMES

PKGNAME = foo-1.0
FLAVORS = f1 f2 f3

There are sometimes mixed situations, where some packages do depend on the FLAVOR, and some don't. For instance, some ports include a large set of documentation that does not depend on the FLAVOR, and some actual programs that depend on the FLAVOR. In such cases, you can specify the FULLPKGNAME for the documentation subpackage explicitly. Something like this:

CATEGORIES = app
COMMENT-core = foo application
COMMENT-doc = foo documentation
V = 1.0
DISTNAME = foo-1.0
PKGNAME-core = foo-$V
FULLPKGNAME-doc = foo-doc-$V
FLAVORS = crypto

MULTI_PACKAGES = -core -doc
WANTLIB-core = c m

.if ${FLAVOR:L:Mcrypto}
WANTLIB-core += ssl crypto
CONFIGURE_ARGS += --enable-crypto
.endif

By default, packages with the same stem do conflict, and update paths will look at candidates with the same stem. The right package will be the one coming from the exact same PKGPATH, or matching @pkgpath annotation in the packing-list.

Usually, MULTI_PACKAGES should not conflict, so they must have different names (and the infrastructure has no way to build those names). On the other hand, flavors should conflict, and thus have the same name. The flavor information should end at the end of the package name, except for pseudo-flavors, which do not change the way a package is built.

As far as dependencies go, by default, specifying a PKGPATH will just create a stem-* dependency, meaning any package with the right stem will match the dependency. By default, any flavor will match. If only specific flavors are desired, you must include them in your specification, e.g., stem-*-flavor. If some flavors are unwanted, you can remove them from matching packages, e.g., stem-*-!flavor.

Since OpenBSD 4.9, asking for at least some version of a dependency can be directly tacked to the PKGPATH, e.g., dir/foo>=2.0.

Updating Ports

There are basically two update models of which maintainers must be aware.

• Some users follow current on a loose basis. They update their packages every few days/every few weeks. Either all packages, or only some selected packages. The update mechanism should work for them: it can force them to update some specific packages, or to install some new stuff, but it should not fail silently. Micro-updates must be tested. These users will most often be able to cope with small changes, like program name changes, or configuration adjustments.
• Some users update every six months with a new release. They also will download stable updates (security and critical updates). During six months, a large part of the ports tree changes. These users do not care about individual software changes. They just want a system that works. Changing binary names is completely unfriendly to these users. Tweaking hundreds of configuration files will be a major pain for them. Maintainers can help by providing smooth update paths, and major hints whenever something important changes.

You should note that part of the update process, especially the macroscopic changes for users who update every six months, is not yet automated. The global update mechanism is still a work in progress, and pkg_add will be able to cope with more issues in the future. As of now, you should focus on making sure the system updates correctly, one port at a time, and that one port's update takes other ports into account, as far as conflicts and other issues are concerned.

Update Checklist

• Make sure the software authors are aware of changes that are required to make it work on OpenBSD. Some changes only relate to ports policies and these don't usually make sense to feed back, but other than this, having changes included upstream makes it easier for the author to test on OpenBSD (they will usually want to do this outside of the ports framework), and reduces the need to merge conflicting patches later yourself. When updating, check that the patches still make sense. Perhaps the same problem was fixed in a different way, in which case a patch may still apply but should be removed.
• Make sure the software authors understand version numbering. Sometimes the names of the distfiles do not contain version numbers, or reference a branch - that is, they're not a fixed point in the code, but a moving target. If this happens, communicate with the authors and try to get permanent URLs.
• Document work-arounds. This will help you remember to check whether they are still necessary when you try to update a port.
• Document dependencies, especially the stuff you don't use. Some ports can use external software that may not be available at the time of porting. Make sure you do not pick it up, and document it, so that you can update your port when this software becomes available later.
• If the port uses libtool, copy its log from the file ${WRKBUILD}/shared_libs.log verbatim as a basis for your SHARED_LIBS setup. This will help during updates.
• PLIST_REPOSITORY is set by default and will build a database of packing-lists as packages are built on your system. This is useful to find out about forgotten package name bumps, and also to check for conflicts with older packages.
• Make sure dependencies stay as loose as they can be. By default, RUN_ and LIB_DEPENDS will be satisfied by any version of a package. Do not insist on a given version unless you have to. Use minimal versions whenever possible.

• Each port update needs a package name bump. Otherwise, the update mechanism for binary packages won't work. Anything that affects the binary package implies a bump. This includes HOMEPAGE, MAINTAINER or description changes, changes to patches or build flags. If the upstream version has not changed, the package name bump is done by incrementing REVISION if already present, otherwise adding REVISION = 0 towards the top of the Makefile. If MULTI_PACKAGES are used, increment REVISION to set the default revision for all subpackages, or increment the individual REVISION-subpkg as necessary (this overrides REVISION). If in doubt, use make show=PKGNAMES to check before and after making changes.
• Version numbers always go forward. If something unexpected happens and you have to revert an update, or if the upstream numbering changes completely so that the version number appears to go backwards, you must use EPOCH to make sure pkg_add(1) sees the package as a newer package. This will add v${EPOCH} to FULLPKGNAME to form a full package-name conforming to packages-specs(7).
• If the package never built with a particular version, no bump is needed.
• Changes to make sure a port does not pick/does pick an external dependency require a bump.
• Changes in existing dependencies generally do not affect a port version number. The package system uses a signature mechanism such that a binary package is fully identified by its package names, plus the dependencies against which it was built, plus the version numbers of all shared libraries it contains. So, if your port depends on a library, a change in that library's version number will trigger an update for packages built from your port.
• The version comparison ordering can be summarized as: 0.1 < 0.1p0 < 0.2 < 0.1v0 < 0.1p0v0 < 0.2v0 < 0.1v1. Also additional words including beta, rc and pre are handled. See packages-specs(7) for more detailed information, and use pkg_check-version(1) to check the ordering of version numbers.

• Run make patch to have an initial copy of the port before the update.

• Edit the port's Makefile to grab the new version, run make makesum and make patch as a starting basis.
• Once you've fixed patches that failed, run a full diff between the old and the new version to figure out exactly what changed. Take notes, be prepared to revisit things later.
• Do whatever porting work is necessary to get the new version to run under OpenBSD: adjust dependencies, change package contents, etc.
• Double check shared library versions. For libtool-based ports, you have the shared_libs.log to check whether the software authors did significant changes. Note well that this is not enough. Many software authors do not really understand shared library issues. You have to read the full diff between the old and the new version, and bump library versions accordingly. When in doubt, bump the major version.
• Be aware of conflicts with already-built packages. For simple updates, nothing needs to be done. If you split a package into several subpackages, make sure the subpackages have proper conflict annotations: they should usually conflict with the "old" single package.
• Help the user. If some specific steps must be taken beyond pkg_add -u, make sure they are visible in the package. When your packaging policy changes, document it in the package description. For instance, if you move the documentation into a separate package for space constraints, the main package description should mention that the documentation is now available as a separate package.

Committing Port Updates

1. Get at least one "OK" from another ports developer.
   
   
2. Use cvs add to add any new files.
   
   
3. Use cvs rm to remove files that are no longer needed (e.g. upstreamed patches).
   
   
4. Check the output of cvs -n up -d. New files should be marked A, deleted files should be marked D, and changed files should be marked M. Look for files marked ? - did you mean to cvs add them?
   
   
5. If all is well, commit the new/deleted/changed files using cvs commit. When invoking cvs commit, you can either list the files individually, or if you provide no filenames, CVS will recursively commit (be careful with this feature). You will be asked to enter a commit message, in which you should state which port is being updated, and who provided an OK.

OpenBSD Porting Policy

• OpenBSD does NOT use /usr/local/etc/rc.d. /usr/local is often shared between several machines, thanks to NFS. For this reason, configuration files that are specific to a given machine can't be stored under /usr/local. /etc is the central repository for per machine configuration files. Moreover, OpenBSD policy is to never update files under /etc automatically. Ports that need some specific boot setup should advise the administrator about what to do instead of blindly installing files.
• OpenBSD does NOT compress man pages.
• OpenBSD does NOT require -lcrypt, -ldl, or -lrt. The functions provided by these libraries are part of libc. The crypt() function does not support DES, only bcrypt.
• OpenBSD has a separate namespace for users and groups created by ports. See /usr/ports/infrastructure/db/user.list for details.
• OpenBSD is strongly security-oriented. You should read and understand this page's security section.
• The goal is to get all ported applications to support OpenBSD. To achieve this goal, feed patches to support running on OpenBSD back to the application maintainer. (If you are not the port maintainer, check with them first. There may be a reason why they have deliberately not done this).

Security Recommendations

• Do not use alpha or beta code when preparing a port. Use the latest regular or patch release.
• Any software to be installed as a server should be scanned for buffer overflows, especially unsafe use of strcat/strcpy/strcmp/sprintf. In general, sprintf should be replaced with snprintf.
• Never use filenames instead of true security. There are numerous race conditions where you don't have proper control. For instance, an attacker who already has user privileges on your machines may replace files in /tmp with symbolic links to more strategic files, such as /etc/master.passwd.
• For instance, both fopen and freopen create a new file or open an existing file for writing. An attacker may create a symbolic link from /etc/master.passwd to /tmp/addrpool_dump. The instant you open it, your password file is hosed. Yes, even with an unlink right before. You only narrow the window of opportunity. Use open with O_CREAT|O_EXCL and fdopen instead.
• Another very common problem is the mktemp function. Heed the warnings of the bsd linker about its uses. These must be fixed. This is not quite as simple as s/mktemp/mkstemp/g. Refer to mktemp(3) for more information. Correct code using mkstemp includes the source to ed or mail. A rare instance of code that uses mktemp correctly can be found in the rsync port.
• Just because you can read it doesn't mean you should. A well-known hole of this nature was the startx problem. As a setuid program, you could launch startx with any file as a script. If the file was not a valid shell script, a syntax error message would follow, along with the first line of the offending file, without any further permission check. Pretty handy to grab the first line of a shadow passwd file, considering these often start with root entry. Do not open your file, and then do an fstat on the open descriptor to check if you should have been able to open it (or the attacker will play with /dev/rst0 and rewind your tape) -- open it with the correct uid/gid/grouplist set.
• Don't use anything that forks a shell in setuid programs before dropping your privileges. This includes popen and system. Use fork, pipe and execve instead.
• Pass open descriptors instead of filenames to other programs to avoid race conditions. Even if a program does not accept file descriptors, you can still use /dev/fd/0.
• Access rights are attached to file descriptors. If you need setuid rights to open a file, open that file, then drop your privileges. You can still access the open descriptor, but you have less to worry about. This is double-edged: even after dropping privileges, you should still watch out for those descriptors.
• Avoid root setuid as much as you can. Basically, root can do anything, but root rights are very rarely needed, except maybe to create socket ports with a number under 1024. You must know the appropriate magic for writing daemons to achieve that. It could be argued that you have no business writing setuid programs if you don't know how to do that.
• Use setgid instead of setuid. Apart from those specific files needed by setgid programs, most files are not group-writable. Hence, a security problem in a setgid program won't compromise your system as much: with only group rights, the worst an intruder will be able to do is hack a games score table or some such. See the xkobo port for an instance of such a change.
• Don't trust group-writable files. Even though they are audited, setgid programs are not perceived as important potential security holes. Hence stuff they can tamper with shouldn't be considered sensitive information.
• Don't trust your environment! This involves simple things such as your PATH (never use system with an unqualified name, avoid execvp), but also more subtle items such as your locale, timezone, termcap, and so on. Be aware of transitivity: even though you're taking full precautions, programs you call directly won't necessarily. Never use system in privileged programs, build your command line, a controlled environment, and call execve directly. The perlsec man page is a good tutorial on such problems.
• Never use setuid shell-scripts. These are inherently insecure. Wrap them into some C code that ensures a proper environment. On the other hand, OpenBSD features secure perl scripts.
• Beware the dynamic loader. If you are running setuid, it will only use trusted libraries that were scanned with ldconfig. Setgid is not enough.
• Dynamic libraries are tricky. C++ code sets a similar problem. Basically, libraries may take some action based upon your environment before your main program even gets to check its setuid status. OpenBSD issetugid addresses this problem, from the library writer point of view. Don't try to port libraries unless you understand this issue thoroughly.
• On OpenBSD, rand(3), random(3), and the drand48(3) family return non-deterministic random numbers by default. Any specified seed value is ignored by the associated seed function, and arc4random is used instead. If the deterministic (i.e: repeatable) behavior must be preserved, use the OpenBSD extensions: srand_deterministic, srandom_deterministic, srand48_deterministic, seed48_deterministic and lcong48_deterministic.

Generic Porting Hints

• __OpenBSD__ should be used sparingly, if at all. Constructs that look like

  #if defined(__NetBSD__) || defined(__FreeBSD__)
  

  are often inappropriate. Don't add blindly __OpenBSD__ to it. Instead, try to figure out what's going on, and what actual feature is needed. Manual pages are often useful, as they include historic comments, stating when a particular feature was incorporated into BSD. Checking the numeric value of BSD against known releases is often the right way. See the NetBSD pkgsrc guide for more information.
• Defining BSD is a bad idea. Try to include sys/param.h. This not only defines BSD, it also gives it a proper value. The right code fragment should look like:

  #if (defined(__unix__) || defined(unix)) && !defined(USG)
  #include <sys/param.h>
  #endif
  

• Test for features, not for specific OSes. In the long run, it is much better to test whether tcgetattr works than whether you're running under BSD 4.3 or later, or SystemVR4. These kind of tests just confuse the issue. The way to go about it is, for instance, to test for one particular system, set up a slew of HAVE_TCGETATTR defines, then proceed to the next system. This technique separates features tests from specific OSes. In a hurry, another porter can just add the whole set of -DHAVE_XXX defines to the Makefile. One may also write or add to a configure script to check for that feature and add it automatically. As an example not to follow, check nethack 3.2.2 source: it assumes loads of things based on the system type. Most of these assumptions are obsolete and no longer reflect reality: POSIX functions are more useful than older BSD versus SystemV differences, to the point that some traditional bsd functions are now only supported through compatibility libraries.
• Avoid include files that include other includes that... First, because this is inefficient. Your project will end up including a file that includes everything. Also, because it makes some problems difficult to fix. It becomes harder to not include one particular file at a given point.
• Avoid bizarre macro tricks. Undefining a macro that was defined by a header file is a bad idea. Defining macros to get some specific behavior from an include file is also a bad idea: compilation modes should be global. If you want POSIX behavior, say so, and #define POSIX_C_SOURCE throughout the whole project, not when you feel like it.
• Don't ever write system function prototypes. All modern systems, OpenBSD included, have a standard location for these prototypes. Likely places include unistd.h, fcntl.h or termios.h. The man page frequently states where the prototype can be found. You might need another slew of HAVE_XXX macros to procure the right file. Don't worry about including the same file twice; include files have guards that prevent all kinds of nastiness. If some broken system needs you to write the prototype, don't impose on all other systems. Roll-your-own prototypes will break because they may use types that are equivalent on your system, but are not portable to other systems (unsigned long instead of size_t), or get some const status wrong. Also, some compilers, such as OpenBSD's own gcc, are able to do a better job with some very frequent functions such as strlen if you include the right header file.
• Carefully check the build log for any compiler warnings.
  • implicit declaration of function foo() indicates that a function prototype is missing. This means that the compiler will assume the return type to be an integer. Where the function actually returns a pointer, on 64-bit platforms it will be truncated, usually causing a segmentation fault.
• Don't use the name of a standard system function for anything else. If you want to write your own function, give it its own name, and call that function everywhere. If you wish to revert to the default system function, you just need to comment your code out, and define your own name to the system function. Don't do it the other way around. Code should look like this:

  /* prototype part */
  #ifdef USE_OWN_GCVT
  char *foo_gcvt(double number, size_t ndigit, char *buf);
  #else
  /* include correct file */
  #include <stdlib.h>
  /* use system function */
  #define foo_gcvt  gcvt
  #endif
  
  /* definition part */
  #ifdef USE_OWN_GCVT
  char *foo_gcvt(double number, size_t ndigit, char *buf)
     {
     /* proper definition */
     }
  
  /* typical use */
  s = foo_gcvt(n, 15, b);
  

Other Helpful Hints

• Recent versions of bsd.port.mk set the installer's path. Specifically, they set /usr/bin and /bin to be searched before /usr/local/bin and /usr/X11R6/bin.
• It is OK to rely on a feature that appeared in a recent version of bsd.port.mk, as people are supposed to update their whole ports tree, including bsd.port.mk.
• Prefer using update-plist to generate and update packing-lists instead of doing things manually. You can comment unwanted lines out with @comment. update-plist can detect most file types and copy most extra annotations correctly. It is not infallible; review the changes it makes. Check they make sense, and check if any new example configuration files should be followed by @sample annotations.
• In OpenBSD, curses.h/libcurses/libtermlib is the "new curses." Change:
  ncurses.h ==> curses.h
  "old (BSD) curses" is available by defining _USE_OLD_CURSES_ before including curses.h (usually in a Makefile) and linking with -locurses.
• In OpenBSD, terminal discipline has been upgraded from the older BSD sgtty to the newer POSIX tcgetattr family. Avoid the older style in new code. Some code may define tcgetattr to be a synonym for the older sgtty, but this is at best a stopgap measure on OpenBSD. The xterm source code is a very good example of what not to do. Try to get your system functionality right: you want a type that remembers the state of your terminal (possible typedef), you want a function that extracts the current state, and a function that sets the new state. Functions that modify this state are more difficult, as they tend to vary depending upon the system. Also, don't forget that you will have to handle cases where you are not connected to a terminal, and that you need to handle signals: not only termination, but also background (SIGTSTP). You should always leave the terminal in a sane state. Do your tests under an older shell, such as sh, which does not reset the terminal in any way at program's termination.
• The newer termcap/terminfo and curses, as included with OpenBSD, include standard sequences for controlling color terminals. You should use these if possible, reverting to standard ANSI color sequences on other systems. However, some terminal descriptions have not been updated yet, and you may need to be able to specify these sequences manually. This is the way vim handles it. This is not strictly necessary. Except for privileged programs, it is generally possible to override a termcap definition through the TERMCAP variable and get it to work properly.
• Signal semantics are tricky, and vary from one system to another. Use sigaction to ensure a specific semantics, along with other system calls referenced in the corresponding man page.

Additional Information

• The man page bsd.port.mk(5). This documents the ports infrastructure makefile that is included at the end of each individual port makefile. There are still a few comments at the start of the file itself, but most of the useful information is now documented.
• The man pages packages-specs(7) and library-specs(7). These explain the exact syntax of package and library dependencies, respectively.
• The man page port-modules(5). This documents the different port modules which are used throughout the ports infrastructure, e.g. cpan, java, python, and so on.