Commit Graph

56 Commits

Author SHA1 Message Date
Duncaen 1f6ab168de xbps-src: simplify dbulk-dump output
* List dependencies of subpackages as part of the source packages
  dependencies, knowing the subpackage they are coming from does
  not matter to a build scheduler as long as xbps-src requires
  those to exist in the same way to main depends= have to exist.
* Resolve virtual? packages through existing means in xbps-src to
  avoid duplicating and possibly diverting from xbps-src's
  behaviour in tools that use dbulk-dump.
* Filter out dependencies on the template itself from sub packages,
  this matches the xbps-src behaviour and there is no need to
  have to duplicate logic for this into the scheduling tool.
* Error out on archs= restrictions as we do already on broken=
  and nocross=, there shouldn't be the need to discover archs=
  restrictions when attempting to build the package.
2021-03-12 18:33:11 -05:00
Piotr Wójcik 6cd3bd1806 xbps-src: break out of following cyclic dependencies 2020-04-15 20:34:06 +02:00
Piotr Wójcik af6e1a7e79 xbps-src: build depends of subpackages
resolves #20508
2020-04-15 11:17:54 +02:00
Juan RP 12f676e6f2
xbps-src: install_pkg_deps: restore previous behaviour.
Don't msg_error if a dependency failed to build, maybe
due to broken or nocross.
2020-01-05 10:48:13 +01:00
Juan RP 75eca1b03e xbps-src: base-chroot-v2 (reduce artifacts/deps)
This patchset contains multiple changes to xbps-src and
its required package "base-chroot" for building packages
via chroot.

- moved xbps.d(5) conf files to `etc/xbps.d`.
- renamed xbps.d(5) repository files to `etc/xbps.d/repos-{local,remote}*`.
- do not set `--repository` to any xbps command that supports it,
  xbps-src now simply populates `rootdir/etc/xbps.d` with correct
  settings (taking care of CHROOT_READY/IN_CHROOT).
- Unless `-C` is set (to preserve builddir/destdir/autodeps), when
  entering to the chroot (if CHROOT_READY is set), xbps-src will
  clean up the masterdir and then perform a system update to always
  use a constant set of packages for that exact date.
- Improved some normal/error msgs.
- Includes support for `xbps>=0.58`.
- common/hooks: switch to bsdtar.
- base-chroot:
  - base-chroot-musl is gone, now unified for glibc/musl.
  - deps removed: gettext, mpfr, readline, texinfo, which, xz.
  - deps changed: tar -> bsdtar.

Effectively this reduces dependencies in `base-chroot`, makes
it unified for musl and glibc, switches xbps-src to use `bsdtar`
rather than GNU `tar` and `xz`, gets rid of useless host dependencies
like GNU gettext, texinfo, etc.

I've been testing these changes for 1 month or so already,
I was able to build from scratch `base-system` for both native
and multiple targets, i.e `./xbps-src -a target -Nt pkg base-system`
2020-01-05 10:44:56 +01:00
Piotr Wójcik 85f5e06fda xbps-src: don't install checkdepends for cross build 2020-01-02 23:15:41 +01:00
Juan RP f7412cdf8d
xbps-src: don't set -i to xbps-install(1).
As reported by @pullmoll via https://github.com/void-linux/xbps/issues/190
the culprit is this assignment, rather than in the xbps code.

Tested with local and remote repos, no regressions found.
2019-12-15 18:53:04 +01:00
John 46ac0aeb23 common/build_dependencies.sh: Disallow subpackages in makedepends 2019-10-26 00:44:22 +02:00
Duncaen 581ee5b7b2 xbps-src: add back support for 32bit dependencies 2019-10-21 18:37:53 +02:00
John 1c16711b3f common/build_dependencies.sh: don't ignore subpkgs in host while cross
compiling, this falls (obviously) apart if it is missing.
See e.g. https://build.voidlinux.org/builders/aarch64-musl_builder/builds/22678/steps/shell_3/logs/stdio
2019-10-15 21:34:20 +03:00
Juan RP e4984d01ea xbps-src: multiple performance improvements
- use xbps-checkvers(1) to resolve dependencies.
- all dependencies are installed at once for the host and target.
- the show-build-deps target is now much faster.
- the update-bulk/show-repo-updates targets are now much faster.
- the update-sys/show-sys-updates targets are now much faster.
- the bootstrap target now works on musl hosts.
- simplified some loops.
- use cut(1) rather than awk(1) where applicable.
- multiple random changes to improve performance.

Based on work started by @Duncaen on #12433

Close #12433
Close #11282
2019-07-10 20:28:46 +02:00
Jürgen Buchmüller 17ba00b095 Revert "xbps-src: multiple performance improvements."
This reverts commit 2a4e178e35.
2019-07-05 12:08:45 +02:00
Juan RP 2a4e178e35 xbps-src: multiple performance improvements.
- use xbps-checkvers(1) to resolve dependencies.
- all dependencies are installed at once for the host and target.
- the `show-build-deps` target is now much faster.
- the `update-bulk/show-repo-updates` targets are now much faster.
- the `update-sys/show-sys-updates` targets are now much faster.
- the `bootstrap` target now works on musl hosts.
- simplified some loops.
- use cut(1) rather than awk(1) where applicable.
- multiple random changes to improve performance.

Based on work started by @Duncaen on https://github.com/void-linux/void-packages/pull/12433

Close https://github.com/void-linux/void-packages/pull/12433
Close https://github.com/void-linux/void-packages/pull/11282
2019-07-05 08:53:51 +02:00
Juan RP b52aae3114
xbps-src: do not ignore EEXIST rval from xbps-install(1).
Due to -I it cannot return EEXIST, and this error value shall
be treated as a real error.

Does not change the current behaviour but will do if we drop -I.
2019-06-26 12:52:59 +02:00
Juan RP e2f098b972
xbps-src: print detected file conflicts. 2019-06-21 18:46:14 +02:00
Juan RP 2127bd2e00
xbps-src: require xbps>=0.55.
Build dependencies are now installed with -I, --ignore-file-conflicts.
A new option introduced in xbps-0.55 that just prints
conflicting files but does not error out.

Switch to xbps-fetch and xbps-digest.
2019-06-21 17:21:38 +02:00
Juan RP d778cbdc70
Revert "xbps-src: require xbps>=0.54."
This reverts commit 198acb8e39.

I'm reverting this until I fix the issue properly.
2019-06-21 13:04:43 +02:00
Juan RP 198acb8e39
xbps-src: require xbps>=0.54.
if xbps-install(1) returns EEXIST, treat it as an error,
do not ignore it.

xbps-install(1) now returns only EEXIST for file conflicts
in transaction, so that it should not be ignored.
2019-06-21 11:47:15 +02:00
John edad3f9f9f
xbps-src/build-dependencies.sh: check for instead of 2019-04-17 16:44:19 +02:00
John 9ae512bbbb
xbps-src: only detect host build loops when not cross compiling 2019-04-17 16:39:23 +02:00
Juan RP 35a8e8c6b6 xbps-src: detect two types of build loops and abort()
- depends on itself
- dep depends on targetpkg
2019-04-15 10:54:04 -03:00
maxice8 5850b188b4 shutils/build_dependencies.sh: also show build_helper that are set. 2019-01-27 09:15:17 +01:00
Jürgen Buchmüller 63db848413 build_dependencies: print $build_style, if any
To make it more obvious what is going to happen next print the
build_style, if set, when emitting the "building ..." messages.
2018-01-16 05:31:32 +01:00
Michael Gehring 9ffc64e20c common: drop 0/no logic from XBPS_{CHECK_PKGS,BUILD_ONLY_ONE_PKG}
use the common convention as described in etc/defaults.conf
2017-11-20 21:12:54 +01:00
Toyam Cox bfbfbf2cab xbps-src: use -1 to trigger errors on missing deps 2017-09-04 16:34:18 -04:00
Jürgen Buchmüller c31dd888f6 xbps-src: implement a 'check' stage
Add another stage 'check' between 'build' and 'install'. It is be enabled using the
variable XBPS_CHECK_PKGS=yes and disabled if unset, set to "0" or "no" in your
local etc/conf.

A new xbps-src option `-q` for `quick` will disable XBPS_CHECK_PKGS by overriding it to 0.
If enabled, `common/xbps-src/shutils/xbps-src-docheck.sh` checks for an existing
`do_check()` function in the package's template and, if it exists, calls it.

A new template variable `checkdepends` may be present and list packages required
to run the `do_check()` function. Example: `checkdepends="bc unittest-cpp"`.
2017-03-08 11:33:48 +01:00
Juan RP d2bac19750 xbps-src: re-introduce etc/virtual.
The `etc/virtual` file declares the default package to be built for
virtual dependencies declared as "virtual?foo" in $depends.

Before this change, the run-time dependency was added as is to the final
binary package but no pkg providing this virtual pkg was built.

With this file we declare the *default* pkg to be built.

NOTE: "virtual?foo" is only applicable to *run* time dependencies, i.e
only those declared in $depends.
2016-04-27 20:16:59 +02:00
Juan RP d3bc0de1e7 xbps-src: get rid of etc/{defaults.,}virtual.
It wasn't too useful after all and this slowed down the dependency
process marginally.

We don't need this, we can live without this.
2016-03-22 10:39:45 +01:00
Juan RP d407e18e19 xbps-src: xbps-install(1) can return EAGAIN (11) with pkg conflicts. 2015-10-29 08:48:38 +01:00
Juan RP d8030e5651 Revert "xbps-src: install dependencies with -U (--unpack-only)."
This reverts commit 8d2226deba.

This breaks all bins that need ca-certificates (curl, wget, git, etc).
2015-06-07 17:02:20 +02:00
Juan RP 8d2226deba xbps-src: install dependencies with -U (--unpack-only).
This should result in a considerable performance improvement, because the
expensive configuration phase of all dependencies is skipped.

Hopefully this does not introduce build issues :-)
2015-06-07 12:06:55 +02:00
Juan RP 94f2e004b4 xbps-src: remove autodeps while building hostdeps too. 2015-05-23 09:52:19 +02:00
Juan RP 4f67c26652 xbps-src: force autodeps cleanup while building dependencies.
Before that change, "./xbps-src -C pkg foo" would not remove dependencies
installed by previous builds required by foo.
2015-05-02 03:40:52 +02:00
Juan RP 11836f56c1 xbps-src: a more meaningful msg when dep cannot be resolved. 2015-04-30 09:09:03 +02:00
Juan RP c6999a235d xbps-src: if build dependency cannot be resolved, exit with error. 2015-04-30 08:26:12 +02:00
Juan RP f7cd626a1c xbps-src/build_dependencies: create XBPS_STATEDIR.
This fixes cross compilation of pkgs that depend on itself.
2015-03-28 04:49:18 +01:00
Juan RP 7cffdb2d28 xbps-src/build_dependencies: handle deps provided by vpkgs correctly.
... otherwise deps specified by the cross-vpkg-dummy pkg aren't
taken into account. Assume that version provided by the vpkg is
good enough for building.
2015-03-28 03:22:07 +01:00
Juan RP 1c5f4690e6 xbps-src: always require current srcpkgs version in build dependencies.
That means that version comparators are not supported anymore in
hostmakedepends and makedepends.

This will ensure that a pkg is always built with the same build dependencies
everywhere, if the srcpkgs tree is uptodate.
2015-03-27 10:59:58 +01:00
Juan RP 03ef5b6804 xbps-src: make -N -a <target> work.
If required cross pkgs are not available in xbps repos, those are now built
automagically.
2015-03-23 12:44:37 +01:00
Juan RP 0337b86f43 xbps-src: return 0 on success, 1 on error and 2 on nocross/broken (ignore). 2015-03-19 10:08:02 +01:00
Juan RP 849d22e46d xbps-src: refactor build logic into its own script (build.sh).
The build.sh script is now responsible to handle all the logic to build
a source package and its subpackages, as well as all its required
build dependencies. Thanks to this and subshells, dependencies are now
built into its own child process, creating a process tree that can go
nested as long as your system allows forking and has enough memory :-)

This fixes some issues that have been while building pkgs that have lots
of nested dependencies.
2015-03-14 07:52:29 +01:00
Juan RP d2f127d7ab xbps-src: create log files in $XBPS_STATEDIR while installing bdeps. 2015-03-03 10:45:34 +01:00
Juan RP 952848ebc3 xbps-src: make sure a target is always passed in to xbps-uhelper. 2014-09-25 12:44:36 +02:00
Juan RP 1ce278f035 xbps-src: drop fakeroot; xbps-create(8) is our friend.
- This should work for almost 90% current packages, as reported by OpenBSD/ports.
- Packages that use install -o <user> -g <group>  will get fixed progressively.
- xbps-create(8) by default sets pkg files as 0:0, if any pkg needs special
  permissions those can be set via INSTALL scripts (for now).
2014-09-24 19:51:17 +02:00
Juan RP 8063adde71 xbps-src: for runtime deps (depends) do not resolve vpkgs.
...otherwise the following would happen:

	- foo depends on musl
	- musl is provided by cross-vpkg-dummy
	- musl is then detected as resolved and it's not built.
2014-09-21 11:44:45 +02:00
Juan RP 75b6468fa0 Introduce a way to override dependencies for all pkgs thru a file.
The `etc/defaults.virtual` file contains the default replacement
for our use cases; at this moment we only use it to use `eudev`
rather than `systemd` for udev.

If `etc/virtual` exists it overrides the settings of `etc/defaults.virtual`.
2014-09-06 12:43:24 +02:00
Juan RP 13c2ca3bd2 xbps-src: reorganize code in more sections. 2014-09-05 11:10:07 +02:00
Juan RP 9d50c2fb05 xbps-src: use "virtual?" keyword to declare virtual runtime dependencies.
The syntax "pkg?vpkg" can be used in hostmakedepends, makedepends and depends;
but the "virtual" keyword can be declared specifically for runtime dependencies
(depends).
2014-08-17 19:21:25 +02:00
Juan RP 0ac14b23ad xbps-src: remove a local var introduced in previous. 2014-08-05 11:52:12 +02:00
Juan RP 1e1136cfac xbps-src: do not skip build deps that are also runtime deps. 2014-08-05 11:51:25 +02:00