This pullrequest removes redundant codepaths in
xbps-src-do{build,configure,install}.sh and joins the code in the
run_step function. This causes slightly different behavior to
do_install:
Do install will chdir to wrksrc only _before_ the first step. The
current behavior is that pre_install will run without a chdir, do_ and
post_ is runned with a chdir. This is a subtle but breaking change and
may cause some templates to break at install phase.
Previously -q was passed to sub processes. When I changed
-q's meaning, this made everything quiet. -Q is now passed
for those instances, and -q is passed for quieting.
This still isn't perfect. When the common/xbps-src/shutils/chroot.sh
function chroot_init() is called, the value for $XBPS_FFLAGS, which is
defined in common/build-profiles/bootstrap.sh, is empty.
Put the immediate value into the generated /etc/xbps/xbps-src.conf
file until someone finds out where passing the value of $XBPS_FFLAGS
throughout the scripts is missing.
Introduce an environment variable `FCC` for fortran, just as CC,
CXX etc. are defined for the other compilers.
It is set to `${XBPS_CROSS_TRIPLET}-gfortran` when cross compiling, or
to just `gfortran` when building for the native architecture.
Use just "$FCC" now when specifying the fortran compiler in a template.
Add another stage 'check' between 'build' and 'install'. It is be enabled using the
variable XBPS_CHECK_PKGS=yes and disabled if unset, set to "0" or "no" in your
local etc/conf.
A new xbps-src option `-q` for `quick` will disable XBPS_CHECK_PKGS by overriding it to 0.
If enabled, `common/xbps-src/shutils/xbps-src-docheck.sh` checks for an existing
`do_check()` function in the package's template and, if it exists, calls it.
A new template variable `checkdepends` may be present and list packages required
to run the `do_check()` function. Example: `checkdepends="bc unittest-cpp"`.
It seems that some (many?) projects rely on the environment defining
compilers, tools and flags by specific XYZ_host and XYZ_target variables.
When cross compiling, define these environment variables.
This enables e.g. qt5/qtwebengine build to succeed. Specificially ninja
relies on these variables when cross compiling.
The `etc/virtual` file declares the default package to be built for
virtual dependencies declared as "virtual?foo" in $depends.
Before this change, the run-time dependency was added as is to the final
binary package but no pkg providing this virtual pkg was built.
With this file we declare the *default* pkg to be built.
NOTE: "virtual?foo" is only applicable to *run* time dependencies, i.e
only those declared in $depends.
- XBPS_TARGET_ARCH var renamed to XBPS_TARGET_MACHINE (gets rid of an extra var).
- Renamed cross profiles to match XBPS_TARGET_MACHINE.
- Added symlinks to keep compatibility with old profiles.
localedef is only available with glibc, so that defer locale creation
once the chroot is ready.
This fixes installing bootstrap glibc pkgs on musl hosts.
Some autotool files (configure.ac) expect host build flags to be
defined as BUILD_CFLAGS, BUILD_CXXFLAGS, BUILD_CPPFLAGS and
BUILD_LDFLAGS. Especially these files tend to default to
LDFLAGS for BUILD_LDFLAGS, which makes host utilities built
with these flags fail with buildpie=yes because of differing
compiler and linker options.
One example is libatasmart where this patch fixes the build.
+ Also update gcc-multilib to gcc-4.9.3
Trying to build `gcc-multilib` failed because `glibc-32bit`
has no separate `srcpkg/glibc-32bit/template`, but is a
result of building `glibc` for target arch `i686`.
For gcc-4.9.3 to build a patch is required that gets us rid
of strange errors when including /usr/include/bits/stdio2.h
This expects a variable argument list with package names that will be
returned to stdout topologically sorted.
$ ./xbps-src sort-dependencies libarchive-devel liblzma-devel libxbps zlib-devel bzip2-devel
bzip2
xz
zlib
libarchive
xbps
Such packages should set the `restricted' var to allow building a binary package.
Note that such packages do not allow redistribution of sources and binaries,
so that it's up to the user if (s)he wants to pkg it locally.
If XBPS_CHROOT_CMD == uchroot with overlayfs (-O), the temporary masterdir
will be removed completely after synchronizing remote repositories, and
later on, when the build starts the repos are unsynchronized.
So do the sync on the real masterdir before entering the chroot.
When sourcing templates to get $checksum there are templates
calling msg_error(). This is not relevant for this function,
thus define a local dummy msg_error() to ignore these cases.
This should result in a considerable performance improvement, because the
expensive configuration phase of all dependencies is skipped.
Hopefully this does not introduce build issues :-)
Available implementations at common/chroot-style/*.sh. Each .sh
script there implements a chroot style to be able to chroot and bind
mount with multiple utilities. The current supported list:
- uunshare (uses xbps-uunshare(8))
- uchroot (uses xbps-uchroot(8))
- proot (uses proot, see http://proot.me)
The XBPS_CHROOT_CMD can be set in etc/conf to use a specific implementation,
and XBPS_CHROOT_CMD_ARGS to pass in additional arguments to the cmd.
This reverts commit d803775b3d.
This breaks all pkgs that exec make directly without ${makejobs}
(which should be built as -j1) and makes installation also parallel,
which we do not want.
We'll probably resurrect this in a certain future.
This ensures that packages with custom build functions also use our
provided MAKEFLAGS. Without this commit it was necessary to always
append the "${makejobs}" variable manually to make. For compatibility
reasons that is still possible but should probably be removed in the
future by making "makejobs" a local variable.
... otherwise deps specified by the cross-vpkg-dummy pkg aren't
taken into account. Assume that version provided by the vpkg is
good enough for building.
That means that version comparators are not supported anymore in
hostmakedepends and makedepends.
This will ensure that a pkg is always built with the same build dependencies
everywhere, if the srcpkgs tree is uptodate.
The build.sh script is now responsible to handle all the logic to build
a source package and its subpackages, as well as all its required
build dependencies. Thanks to this and subshells, dependencies are now
built into its own child process, creating a process tree that can go
nested as long as your system allows forking and has enough memory :-)
This fixes some issues that have been while building pkgs that have lots
of nested dependencies.
Rather than appending to the final LDFLAGS, add it to the head,
this way packages that use LDFLAGS="-lfoo" will still work with --as-needed.
This fixes hsetroot build, at least.
Rather than setting them in the build profile, just set them in
etc/defaults.conf (overridable via etc/conf) to simplify the code.
- Also enable _FORTIFY_SOURCE=2 in CPPFLAGS.
- Also enable -z,relro in LDFLAGS.
As suggested by @chneukirchen.
This temporary masterdir contains hardlinks of files stored in a real
masterdir, and once the target has finished will be removed.
Could be useful to create fully up to date and immutable real masterdirs
that are used as base for the temporary masterdirs.
XBPS_STATEDIR is created in XBPS_BUILDDIR/.xbps-${sourcepkg}-${version}
and contains logs and state files for all phases, as well as wrappers.
We can now inspect the output from all hooks.
common/build-profiles contains the compilation flags used in native
builds, and are defined per architecture:
- x86_64 (glibc)
- x86_64-musl (musl)
If XBPS_CFLAGS or XBPS_CXXFLAGS are set in `etc/conf`, the settings
from the build profile will be overrided.
- This should work for almost 90% current packages, as reported by OpenBSD/ports.
- Packages that use install -o <user> -g <group> will get fixed progressively.
- xbps-create(8) by default sets pkg files as 0:0, if any pkg needs special
permissions those can be set via INSTALL scripts (for now).
...otherwise the following would happen:
- foo depends on musl
- musl is provided by cross-vpkg-dummy
- musl is then detected as resolved and it's not built.
The `etc/defaults.virtual` file contains the default replacement
for our use cases; at this moment we only use it to use `eudev`
rather than `systemd` for udev.
If `etc/virtual` exists it overrides the settings of `etc/defaults.virtual`.
Add the helpers vopt_if, vopt_with and vopt_enable that simplify common
option-based operations.
Instead of a bunch of
if [ "$build_option_foo" ]; then
configure_args+=" --with-foo"
makedepends+=" foo-devel"
else
configure_args+=" --without-foo"
fi
one can use
configure_args="... $(vopt_with foo)
makedepends="... $(vopt_if foo foo-devel)"
instead.
We're adding these functions to common/xbps-src/shutils/common.sh but
that might not be the ideal place. I would've preferred
common/helpers/options.sh, but helpers are only available in the
individual phases, not when the template itself gets parsed.
- There's no need to install rdeps into the masterdir, just base-files.
- There's no need to exit if XBPS_TARGET_PKG == sourcepkg, just return.
- After building base-chroot, install bootstrap binary packages.
The prepare-32bit hook has been moved to the pre-pkg stage and now
those hooks are executed as part of the install phase. That means that
pre-pkg hooks are now independently run for the subpkgs and the sourcepkg
after the install-destdir phase and pre/do/install hooks.
As bonus, finally correct rdeps can now be collected for the 32bit pkgs
and works for all them, without the need to sort the order of subpkgs!
10 Non development packages are processed first lexically.
20 Development packages are processed after 10.
This resolves the issue of 32bit pkg creation with development pkgs
that depend on other non-development subpkgs, and result in wrong
dependencies being propagated to the 32bit development pkgs.
- broken var simply exits (without error) and skips building a pkg.
- nocross var now simply exits without an error, to make buildbot continue building pkgs.
The syntax "pkg?vpkg" can be used in hostmakedepends, makedepends and depends;
but the "virtual" keyword can be declared specifically for runtime dependencies
(depends).
Run-time dependencies declared via "${depends}" are now also part of the
build process, but those are not installed to the master directory; rather
those dependencies are only checked if a binary package exists in a
local repository to satisfy the required version.
Thanks to @dominikh for idea and suggestions.
Permanent global pkg build options can be defined via 'XBPS_PKG_OPTIONS' in etc/conf.
Permament per-pkg build options can be defined via 'XBPS_PKG_OPTIONS_<pkgname>' in etc/conf.
Close GH #452.
Jürgen Buchmüller
Michael Gehring
Enno Boland
Duncan Overbruck
Toyam Cox
Florian Wagner
Enno Boland
Duncaen
Hadrian Węgrzynowski
Alessio Sergi
Juan RP
Christian Neukirchen
Juan RP
Sören Tempel
Eivind Uggedal
Enno Boland
Dominik Honnef