Commit Graph

1175 Commits

Author SHA1 Message Date
Juan RP 75eca1b03e xbps-src: base-chroot-v2 (reduce artifacts/deps)
This patchset contains multiple changes to xbps-src and
its required package "base-chroot" for building packages
via chroot.

- moved xbps.d(5) conf files to `etc/xbps.d`.
- renamed xbps.d(5) repository files to `etc/xbps.d/repos-{local,remote}*`.
- do not set `--repository` to any xbps command that supports it,
  xbps-src now simply populates `rootdir/etc/xbps.d` with correct
  settings (taking care of CHROOT_READY/IN_CHROOT).
- Unless `-C` is set (to preserve builddir/destdir/autodeps), when
  entering to the chroot (if CHROOT_READY is set), xbps-src will
  clean up the masterdir and then perform a system update to always
  use a constant set of packages for that exact date.
- Improved some normal/error msgs.
- Includes support for `xbps>=0.58`.
- common/hooks: switch to bsdtar.
- base-chroot:
  - base-chroot-musl is gone, now unified for glibc/musl.
  - deps removed: gettext, mpfr, readline, texinfo, which, xz.
  - deps changed: tar -> bsdtar.

Effectively this reduces dependencies in `base-chroot`, makes
it unified for musl and glibc, switches xbps-src to use `bsdtar`
rather than GNU `tar` and `xz`, gets rid of useless host dependencies
like GNU gettext, texinfo, etc.

I've been testing these changes for 1 month or so already,
I was able to build from scratch `base-system` for both native
and multiple targets, i.e `./xbps-src -a target -Nt pkg base-system`
2020-01-05 10:44:56 +01:00
Stefano Ragni da3b4d552a xbps-src: fix spelling 2019-11-08 19:18:30 +01:00
Juan RP b94cc0ba26
xbps-src/zap: missing \n 2019-10-21 18:56:24 +02:00
Piraty 80e4a39926 xbps-src: (zap) correctly empty the masterdir
Instead of relying on a potentially outdated list of files and dirs to
remove, just remove+recreate the whole masterdir.

This previously led to leftover .xbps* files, which broke subsequent
operations

Fixes #14476
2019-10-21 18:55:19 +02:00
q66 ad34a8553c xbps-src: generalize 32-bit masterdirs on 64-bit hosts
Previously the check was hardcoded for i686 on x86_64. This can
however also work for ppc64 (BE) running a ppc masterdir as well
as aarch64 running armv* masterdirs.
2019-08-11 14:05:13 +02:00
Doan Tran Cong Danh 06cbdef6eb xbps-src: redirect various error msg to stderr
We couldn't use msg_red or msg_error,
since they won't be available until line #680
2019-07-24 15:13:54 +02:00
Juan RP 818d2d59cd
xbps-src: relax check for writable hostdir.
buildbot does not like it.
2019-07-10 20:30:18 +02:00
Juan RP e4984d01ea xbps-src: multiple performance improvements
- use xbps-checkvers(1) to resolve dependencies.
- all dependencies are installed at once for the host and target.
- the show-build-deps target is now much faster.
- the update-bulk/show-repo-updates targets are now much faster.
- the update-sys/show-sys-updates targets are now much faster.
- the bootstrap target now works on musl hosts.
- simplified some loops.
- use cut(1) rather than awk(1) where applicable.
- multiple random changes to improve performance.

Based on work started by @Duncaen on #12433

Close #12433
Close #11282
2019-07-10 20:28:46 +02:00
Jürgen Buchmüller 17ba00b095 Revert "xbps-src: multiple performance improvements."
This reverts commit 2a4e178e35.
2019-07-05 12:08:45 +02:00
Jürgen Buchmüller 7c4b177043 Revert "xbps-src: fix reconfiguration of bootstrap pkgs."
This reverts commit ba4c05aa00.
2019-07-05 12:08:42 +02:00
Jürgen Buchmüller a5968491c7 Revert "xbps-src: silence `show-avail`"
This reverts commit 5c4d340c5f.
2019-07-05 12:08:40 +02:00
Juan RP 5c4d340c5f xbps-src: silence `show-avail` 2019-07-05 08:53:52 +02:00
Juan RP ba4c05aa00 xbps-src: fix reconfiguration of bootstrap pkgs. 2019-07-05 08:53:51 +02:00
Juan RP 2a4e178e35 xbps-src: multiple performance improvements.
- use xbps-checkvers(1) to resolve dependencies.
- all dependencies are installed at once for the host and target.
- the `show-build-deps` target is now much faster.
- the `update-bulk/show-repo-updates` targets are now much faster.
- the `update-sys/show-sys-updates` targets are now much faster.
- the `bootstrap` target now works on musl hosts.
- simplified some loops.
- use cut(1) rather than awk(1) where applicable.
- multiple random changes to improve performance.

Based on work started by @Duncaen on https://github.com/void-linux/void-packages/pull/12433

Close https://github.com/void-linux/void-packages/pull/12433
Close https://github.com/void-linux/void-packages/pull/11282
2019-07-05 08:53:51 +02:00
Juan RP 67cd850ebf
xbps-src: introduce XBPS_REPO_COMPTYPE for etc/conf.
This sets the repository data compression format, as explained
in xbps-rindex(1). By default set to `gzip'.
2019-06-25 10:52:37 +02:00
Juan RP 2127bd2e00
xbps-src: require xbps>=0.55.
Build dependencies are now installed with -I, --ignore-file-conflicts.
A new option introduced in xbps-0.55 that just prints
conflicting files but does not error out.

Switch to xbps-fetch and xbps-digest.
2019-06-21 17:21:38 +02:00
Juan RP d778cbdc70
Revert "xbps-src: require xbps>=0.54."
This reverts commit 198acb8e39.

I'm reverting this until I fix the issue properly.
2019-06-21 13:04:43 +02:00
Juan RP 198acb8e39
xbps-src: require xbps>=0.54.
if xbps-install(1) returns EEXIST, treat it as an error,
do not ignore it.

xbps-install(1) now returns only EEXIST for file conflicts
in transaction, so that it should not be ignored.
2019-06-21 11:47:15 +02:00
John c614dba6cf xbps-src: set NOCOLOR early 2019-06-19 16:26:57 +00:00
Juan RP 343448a186
xbps-src: take into account XBPS_ALT_REPOSITORY if set.
otherwise it won't be used while installing bootstrap pkgs.
2019-06-19 16:12:49 +02:00
Juan RP fab6617758
xbps-src: remove builddir properly in the clean target.
I've seen some go packages changing the permissions to
some directories, hence force them.
2019-06-19 11:49:05 +02:00
Peter Bui 7e1ffb6d71 export XBPS_BINPKG_EXISTS rather than mark it as readonly
srcpkg_get_version and srcpkg_get_pkgver require unsetting
XBPS_BINPKG_EXISTS to work properly (e.g. without it, some packages have
a trailing -).  Unfortunately, bash does not allow unsetting readonly
variables, which XBPS_BINPKG_EXISTS is exported as in xbps-src.

To resolve this problem, simply export XBPS_BINPKG_EXISTS as a normal
variable rather than marking it as readonly.  This addresses #11740.
2019-06-04 15:44:49 +02:00
yopito c15d85987f xbps-src: recompute XBPS_MASTERDIR and XBPS_HOSTDIR in chroot stage 2019-06-04 15:43:48 +02:00
human 2a854ad697 xbps-src: enter chroot before bootstrap-update
fixes https://github.com/void-linux/void-packages/issues/11848
2019-06-04 11:51:19 +02:00
human 1c9aa6911e xbps-src: fix typo in help text 2019-06-04 11:51:19 +02:00
maxice8 fa40215823 xbps-src: replace dirname(1) usage 2019-04-14 12:46:51 -03:00
Juan RP 1ee6a3174f xbps-src: reduce basename(1) usage.
This saves a few exec()s.
2019-04-14 01:47:16 -03:00
maxice8 fadca94b98 common: add chroot-style 'ethereal' 2019-04-05 05:49:34 -03:00
maxice8 2536f0c67a xbps-src: remove support for noarch on show-pkg-var 2019-03-14 19:22:06 -03:00
maxice8 0c31f1001e xbps-src: remove support for only_for_archs on show-pkg-var 2019-03-14 19:20:07 -03:00
maxice8 93a0a93ea9 xbps-src; export XBPS_GIT_CMD, defaulting to chroot-git and fallback on
git.
2019-03-07 17:26:45 -08:00
maxice8 548c3dab25 xbps-src: drop git requirement
All the code should try chroot-git then git. The 2 instances of git
usage do it, the only code left is in common/travis.
2019-03-07 17:26:45 -08:00
maxice8 6f0342c65e xbps-src: add explanation of patch phase and fix ordering 2019-02-27 22:56:19 -03:00
maxice8 4abed9c425 xbps-src: add support for patch phase 2019-02-27 22:56:19 -03:00
Duncaen f0ef8e8792 xbps-src: add clean-repocache command 2019-02-14 05:42:03 -02:00
Duncaen 2e108ebfdf xbps-src: unbreak bootstrap-update by reverting to previous method 2019-02-05 13:01:05 +01:00
Juan RP 9bf179825f xbps-src: fix 'clean' and 'remove-autodeps' targets again.
Broke them in f35aa4cc19
2019-02-02 19:03:58 +01:00
Juan RP f35aa4cc19
xbps-src: perform chroot setup via binary-boostrap or bootstrap-update.
This will improve performance somewhat on buildbot because the bootstrap pkgs
will only be configured once, and not each time due to `xbps-src -t`.

Also remove redundant checks.
2019-02-02 13:35:18 -02:00
Cameron Nemo 2b569f1ef8 xbps-src: add show host/makedepends subcommand 2019-01-26 21:41:10 -02:00
maxice8 78333e638f
xbps-src: export XBPS_BUILDHELPERDIR 2019-01-25 20:44:41 -02:00
maxice8 de9ca04298 xbps-src: make XBPS_FETCH_CMD readonly 2019-01-21 16:17:02 -02:00
maxice8 55032f42c4 xbps-src: don't export XBPS_GCC_* 2019-01-18 05:42:18 -02:00
Toyam Cox f6c93590a2 xbps-src: tell dxpb provides, 32bit, and debug 2018-09-30 10:31:47 -04:00
Michael Aldridge a4dd50388a xbps-src: Always enable the gocache
[ci_skip]
2018-09-01 22:05:07 -07:00
Jürgen Buchmüller 4ca83f7f90 xbps-src: update GCC version to 8.2.0 2018-08-15 06:50:23 +02:00
Duncaen 5334409c12 Revert "xbps-src: ensure 'changelog' is available"
This reverts commit 1441834458.
2018-05-04 16:46:41 +02:00
Toyam Cox 1441834458 xbps-src: ensure 'changelog' is available 2018-05-04 09:27:48 -04:00
Michael Gehring e3f56ecb01 xbps-src: support caching go builds
Piggyback on the existing XBPS_CCACHE logic to enable/disable. Stored in
${XBPS_HOSTDIR}/gocache.
2018-03-19 08:29:05 +01:00
Michael Gehring 70102792c1 Revert "xbps-src: temporarily turn on debug output for xbps-rindex"
This reverts commit 1a6579043d.
2018-03-04 22:19:23 +00:00
Michael Gehring 1a6579043d xbps-src: temporarily turn on debug output for xbps-rindex 2018-03-04 15:26:15 +01:00
Michael Gehring c84f58bb8d xbps-src: support $NO_COLOR 2018-02-28 12:54:54 +01:00
Jürgen Buchmüller 909c890a67 xbps-src: enable CCACHE_COMPRESS to save space
Using CCACHE_COMPRESS is transparent. It compresses manifests and object
files in the cache but can read both, compressed and uncompress files.
2017-12-07 16:39:18 +01:00
Michael Gehring 1a5b47bcc3 xbps-src: always enable checks for the check target 2017-11-20 13:02:30 +00:00
Michael Gehring 565de75a06 xbps-src: flip logic of -Q 2017-11-20 13:02:30 +00:00
Jürgen Buchmüller f06915551b xbps-src: export XBPS_GCC_VERSION* 2017-11-05 12:05:23 +01:00
Jürgen Buchmüller 1c2290eec1 gcc: update to 7.2.0 2017-11-02 15:57:31 +01:00
Toyam Cox 332a6aaf23 xbps-src: make show-avail work as documented 2017-09-23 21:44:25 -04:00
Toyam Cox bfbfbf2cab xbps-src: use -1 to trigger errors on missing deps 2017-09-04 16:34:18 -04:00
Toyam Cox 52a560fa39 xbps-src: add show-pkg-var mode for dxpb's usage 2017-08-29 13:23:54 -04:00
Toyam Cox f7e3ea1a7e xbps-src: change -q and -Q flag meanings 2017-08-29 13:23:54 -04:00
Toyam Cox debdd551f4 xbps-src: add quiet mode 2017-08-29 13:23:54 -04:00
Toyam Cox fc7015fd7e xbps-src: respect XBPS_ARCH as environment variable 2017-08-29 13:23:54 -04:00
Toyam Cox 04f489e839 xbps-src: expand usage of ignore-problems in reading pkgs 2017-08-29 13:23:54 -04:00
Toyam Cox 5a7e73b89a xbps-src: add informative run mode 2017-08-29 13:23:54 -04:00
Michael Gehring 133f31a5ef xbps-src: s/repoache/repocache/ 2017-07-27 13:56:25 +02:00
Michael Gehring 31db228f9b xbps-src: allow update-check for broken packages 2017-07-01 22:31:17 +00:00
Enno Boland 33c1040159 Merge branch 'master' into check 2017-06-05 14:58:57 +02:00
Duncaen 102fc6f99f xbps-src: portability fixes 2017-05-24 20:02:46 +02:00
Jürgen Buchmüller 5a11957d14 Merge branch 'master' into check 2017-05-02 09:18:52 +02:00
Jürgen Buchmüller 272114cd05 xbps-src: trying to fix FFLAGS construction
This still isn't perfect. When the common/xbps-src/shutils/chroot.sh
function chroot_init() is called, the value for $XBPS_FFLAGS, which is
defined in common/build-profiles/bootstrap.sh, is empty.

Put the immediate value into the generated /etc/xbps/xbps-src.conf
file until someone finds out where passing the value of $XBPS_FFLAGS
throughout the scripts is missing.
2017-03-08 17:36:22 +01:00
Jürgen Buchmüller c31dd888f6 xbps-src: implement a 'check' stage
Add another stage 'check' between 'build' and 'install'. It is be enabled using the
variable XBPS_CHECK_PKGS=yes and disabled if unset, set to "0" or "no" in your
local etc/conf.

A new xbps-src option `-q` for `quick` will disable XBPS_CHECK_PKGS by overriding it to 0.
If enabled, `common/xbps-src/shutils/xbps-src-docheck.sh` checks for an existing
`do_check()` function in the package's template and, if it exists, calls it.

A new template variable `checkdepends` may be present and list packages required
to run the `do_check()` function. Example: `checkdepends="bc unittest-cpp"`.
2017-03-08 11:33:48 +01:00
Jürgen Buchmüller cb7df7b544 xbps-src: update XBPS_GCC_VERSION* 2017-02-06 03:38:06 +01:00
ojab 2083d0ff8e Export XBPS_PKG_COMPTYPE so setting it in configuration actually works (#5459) 2017-01-04 19:29:56 +01:00
Enno Boland a0e3cb9a42 gcc: update to 6.2.1. 2016-11-27 18:37:02 +01:00
Juergen Buchmueller fd9ff4fedf xbps-src: Define XBPS_GCC_VERSION
To simplify testing for the currently used gcc version introduce
global environment variables to avoid running `gcc --version`.
2016-10-25 23:19:58 +02:00
Toyam Cox 7c9d1f61b0 xbps-src: allow show of broken pkgs 2016-10-20 19:49:57 -04:00
Juan RP a5e248c730 xbps-src: require automake, libtool and pkg-config in the host for bootstrap.
This fixes bootstrapping of libarchive. Close #4516.
2016-07-28 08:02:37 +02:00
Enno Boland 05ff430223 xbps-src: Preprocess arguments using getopt in order to allow options before and after XBPS_TARGET. 2016-07-04 16:38:22 +02:00
Juan RP af11e6c891 xbps-src: set XBPS_CHROOT_CMD=uchroot with -t; improve desc for -a. 2016-04-27 17:03:24 +02:00
Juan RP 561ab61608 xbps-src: make i686 masterdirs (glibc) work on x86_64/musl hosts. 2016-04-13 08:07:02 +02:00
Enno Boland 096aac3603 xbps-src: add consistency check
`xbps-src consistency-check` iterates over all templates and looks for
potentially broken dependencies.
2016-03-22 00:36:30 +01:00
Enno Boland 16050ad3f2 hooks/pre-pkg: fixed mtime support for packages.
overwrites the mtime of all files in a package with the commit-date
of the package.
2016-02-19 16:21:56 +01:00
Steve Prybylski f18def49cd xbps-src: export XBPS_ALLOW_RESTRICTED. 2015-12-27 02:19:02 -05:00
Juan RP e69b2491b7 xbps-src: fix -t by appending to XBPS_CHROOT_CMD_ARGS instead. 2015-12-12 10:13:22 +01:00
Juan RP 0a58fd0c01 Revert "xbps-src: fix uchroot chroot-style to not overwrite XBPS_CHROOT_CMD_ARGS."
This reverts commit ba4bba2031.
2015-12-12 10:12:33 +01:00
Juan RP ba4bba2031 xbps-src: fix uchroot chroot-style to not overwrite XBPS_CHROOT_CMD_ARGS. 2015-12-12 10:03:41 +01:00
Juan RP c9dede4e23 xbps-src: force glibc locale generation on musl hosts.
localedef is only available with glibc, so that defer locale creation
once the chroot is ready.

This fixes installing bootstrap glibc pkgs on musl hosts.
2015-12-12 07:35:30 +01:00
Jürgen Buchmüller 7e72db86c6 xerces-c: set DISABLE_CCACHE=yes for i686
Re-enable the shared CCACHE_DIR which was not the root
cause of the problems.
2015-11-19 16:11:43 +01:00
Jürgen Buchmüller 9bd521e9bd xerces-c: no need to disable PIE for i686
It seems that sharing a common CCACHE_DIR for all compilers
is problematic between at least x86_64 and i686 in some cases.
Revert to using per architecture ccache directories.
2015-11-19 15:18:35 +01:00
Juan RP 60110c0c2f xbps-src: the 'show-var' target now sources native build-profile too. 2015-11-14 11:51:45 +01:00
Juan RP 2d9b01bce5 xbps-src: make 'show-var' source environment/setup.
This makes XXX_SITE vars work.
2015-11-14 10:25:39 +01:00
Juan RP cacdf5e724 xbps-src: use host git(1) to detect current branch.
This now guesses correctly the current git branch when the target
masterdir does not match the host architecture.
2015-11-13 10:54:14 +01:00
Juan RP eae488d3c4 xbps-src: typo fix for zap. 2015-11-02 20:32:12 +01:00
Juan RP 5f030fe2f1 xbps-src: force reconfiguration of bootstrap pkgs on chroot setup.
Right now ca-certificates internally requires the `openssl` utility
to generate /etc/ssl/certs. Certainly we need to perform this task
only once and iff we are within the chroot, because the host might
not have it in PATH.

This should fix the travis issue seen since xbps-0.48 was released.
2015-11-02 20:25:16 +01:00
Juan RP 8e7b21a472 xbps-src: force reconfiguration of ca-certificates. 2015-10-31 08:19:09 +01:00
Juan RP 0cd87b3cdd xbps-src: perform bootstrapping in two phases.
- binary-bootstrap unpacks all pkgs and configure base-files.
- after accessing the chroot for the first time all pkgs are configured.

This should fix the issue of ca-certificates not generating the certs.
2015-10-31 07:59:06 +01:00
Juan RP 3a64a0f770 xbps-src: remove ca-certificates workaround; will be fixed correctly. 2015-10-31 07:51:56 +01:00
Juan RP 7e92158bbd xbps-src: only reconfigure ca-certificates if /etc/ssl/certs does not exist. 2015-10-30 15:54:13 +01:00
Juan RP c7611a9292 xbps-src: document ca-certificates stupid kludge. 2015-10-30 15:36:43 +01:00
Juan RP 0d55d32ba0 xbps-src: force reconfiguration of ca-certificates only in the chroot. 2015-10-30 15:33:57 +01:00