[Bug 232254] ports-mgmt/pkg: Unable to sign repos in -current

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

Bug ID: 232254
Summary: ports-mgmt/pkg: Unable to sign repos in -current
Product: Ports & Packages
Version: Latest
Hardware: amd64
OS: Any
Status: New
Severity: Affects Some People
Priority: ---
Component: Individual Port(s)
Assignee: ***@FreeBSD.org
Reporter: ***@FreeBSD.org
Assignee: ***@FreeBSD.org
Flags: maintainer-feedback?(***@FreeBSD.org)

After the base update of openssl, pkg-static and pkg rebuilt against this
version of openssl are unable to sign repos with a given key:

access("/root/ssl/pkg.key",R_OK) = 0 (0x0)
open("/root/ssl/pkg.key",O_RDONLY,0666) = 5 (0x5)
close(5) = 0 (0x0)
write(1,"\n",1) = 1 (0x1)
write(2,"pkg-static: ",12) = 12 (0xc)
write(2,"can't load key from /root/ssl/pk"...,37) = 37 (0x25)
write(2,"\n",1) = 1 (0x1)
write(4,"\M-}7zXZ\0\0\^D\M-f\M-V\M-4F\^B"...,76) = 76 (0x4c)
close(4) = 0 (0x0)
unlink("/tmp/foo/meta") = 0 (0x0)
ioctl(1,TIOCGETA,0x7fffffffd718) = 0 (0x0)
write(1,"\rPacking files for repository: "...,35) = 35 (0x23)
ioctl(1,TIOCGETA,0x7fffffffd718) = 0 (0x0)
write(1,"\n",1) = 1 (0x1)
close(3) = 0 (0x0)
exit(0x41)
process exit, rval = 65



***@bob.nyi:/usr/local/poudriere/data/packages/12-amd64-cluster-default # pkg
info pkg
pkg-1.10.5_4
Name : pkg
Version : 1.10.5_4
Installed on : Sun Oct 14 17:13:09 2018 UTC
Origin : ports-mgmt/pkg
Architecture : FreeBSD:12:amd64
Prefix : /usr/local
Categories : ports-mgmt
Licenses : BSD2CLAUSE
Maintainer : ***@FreeBSD.org
WWW : https://wiki.freebsd.org/pkgng
Comment : Package manager
Options :
DOCS : on
Shared Libs provided:
libpkg.so.4
Annotations :
FreeBSD_version: 1200085
Flat size : 12.7MiB
Description :
Package management tool

WWW: https://wiki.freebsd.org/pkgng
***@bob.nyi:/usr/local/poudriere/data/packages/12-amd64-cluster-default #
/usr/local/sbin/pkg-static repo /tmp/foo /root/ssl/pkg.key
Creating repository in /tmp/foo: 100%
Packing files for repository: 0%
pkg-static: can't load key from /root/ssl/pkg.key
Packing files for repository: 100%

[b***@freebsd.org]

Bugzilla Automation <***@FreeBSD.org> has asked freebsd-pkg mailing list
<***@FreeBSD.org> for maintainer-feedback:
Bug 232254: ports-mgmt/pkg: Unable to sign repos in -current
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254



--- Description ---
After the base update of openssl, pkg-static and pkg rebuilt against this
version of openssl are unable to sign repos with a given key:

access("/root/ssl/pkg.key",R_OK) = 0 (0x0)
open("/root/ssl/pkg.key",O_RDONLY,0666) = 5 (0x5)
close(5) = 0 (0x0)
write(1,"\n",1) = 1 (0x1)
write(2,"pkg-static: ",12) = 12 (0xc)
write(2,"can't load key from /root/ssl/pk"...,37) = 37 (0x25)
write(2,"\n",1) = 1 (0x1)
write(4,"\M-}7zXZ\0\0\^D\M-f\M-V\M-4F\^B"...,76) = 76 (0x4c)
close(4) = 0 (0x0)
unlink("/tmp/foo/meta") = 0 (0x0)
ioctl(1,TIOCGETA,0x7fffffffd718) = 0 (0x0)
write(1,"\rPacking files for repository: "...,35) = 35 (0x23)
ioctl(1,TIOCGETA,0x7fffffffd718) = 0 (0x0)
write(1,"\n",1) = 1 (0x1)
close(3) = 0 (0x0)
exit(0x41)
process exit, rval = 65



***@bob.nyi:/usr/local/poudriere/data/packages/12-amd64-cluster-default # pkg
info pkg
pkg-1.10.5_4
Name : pkg
Version : 1.10.5_4
Installed on : Sun Oct 14 17:13:09 2018 UTC
Origin : ports-mgmt/pkg
Architecture : FreeBSD:12:amd64
Prefix : /usr/local
Categories : ports-mgmt
Licenses : BSD2CLAUSE
Maintainer : ***@FreeBSD.org
WWW : https://wiki.freebsd.org/pkgng
Comment : Package manager
Options :
DOCS : on
Shared Libs provided:
libpkg.so.4
Annotations :
FreeBSD_version: 1200085
Flat size : 12.7MiB
Description :
Package management tool

WWW: https://wiki.freebsd.org/pkgng
***@bob.nyi:/usr/local/poudriere/data/packages/12-amd64-cluster-default #
/usr/local/sbin/pkg-static repo /tmp/foo /root/ssl/pkg.key
Creating repository in /tmp/foo: 100%
Packing files for repository: 0%
pkg-static: can't load key from /root/ssl/pkg.key
Packing files for repository: 100%

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

Roger Pau Monné <***@freebsd.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Keywords| |regression

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

--- Comment #1 from Glen Barber <***@FreeBSD.org> ---
Please see the Github freebsd/pkg pull request that resolves this:
https://github.com/freebsd/pkg/pull/1716

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

--- Comment #2 from Glen Barber <***@FreeBSD.org> ---
Updated pull request for 1.10.x specifically:
https://github.com/freebsd/pkg/pull/1717

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

Sean Bruno <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Summary|ports-mgmt/pkg: Unable to |ports-mgmt/pkg: pkg-stiatc
|sign repos in -current |unable to sign repos in
| |-current

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

Sean Bruno <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Summary|ports-mgmt/pkg: pkg-stiatc |ports-mgmt/pkg: pkg-static
|unable to sign repos in |unable to sign repos in
|-current |-current

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

***@utanet.at changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@utanet.at

--- Comment #3 from ***@utanet.at ---
Should fixed with ports r482214.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

--- Comment #4 from Roger Pau Monné <***@freebsd.org> ---
Is there anyway that we could prevent this from happening in the future? Like
not updating the front facing package repository if there are critical errors
detected in the building phase?

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

--- Comment #5 from Sean Bruno <***@FreeBSD.org> ---
(In reply to Roger Pau Monné from comment #4)
This was a side effect of the base openssl upgrade. I'm unsure how the ports
team would have detected this without doing the full upgrade and trying to
build the repository.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

--- Comment #6 from Roger Pau Monné <***@freebsd.org> ---
(In reply to Sean Bruno from comment #5)
I have to admit I know nothing about the package building infrastructure, but
if I understand correctly what happened here is a failure to sign the index in
the builders, which I would expect should have caused the update of the front
facing repository to fail, leaving it in the state it was previously.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

--- Comment #7 from Sean Bruno <***@FreeBSD.org> ---
(In reply to Roger Pau Monné from comment #6)
This would have happened if the package builders were updated to the openssl
update revision, not just the poudriere jails on the package builders AFAIK. I
only ran into this in the freebsd cluster when we attempted to use -current on
the host that was building our repositories *and* I updated pkg to the version
build in a jail that was at the same revision.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

--- Comment #8 from Roger Pau Monné <***@freebsd.org> ---
(In reply to Sean Bruno from comment #7)
As said, I'm afraid I don't really understand how all this infrastructure
works, so my reply might be completely wrong.

I would expect the builders to pick the svn updates and build a new set of
packages, together with the index and all the needed metadata, and once this is
done everything is pushed to the front facing repository for people to consume.
In this case there was an error during index generation, which should have
halted this process and instead kept the previously working set of packages and
metadata in the public repository for clients to consume?

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

--- Comment #9 from Roger Pau Monné <***@freebsd.org> ---
Could the pkg binary in the mirrors be updated:

http://pkg.freebsd.org/FreeBSD:12:amd64/latest/Latest/

This is a build from 11/10 which contains the bug and makes pkg-static
completely unusable.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

--- Comment #10 from Roger Pau Monné <***@freebsd.org> ---
The long-standing lack of a working pkg-static binary in the package repository
has forced Xen to drop the Freebsd tests from the CI:

https://lists.xenproject.org/archives/html/xen-devel/2018-10/msg01833.html

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

Glen Barber <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@FreeBSD.org
Severity|Affects Some People |Affects Many People

--- Comment #11 from Glen Barber <***@FreeBSD.org> ---
It is unclear to me why the timestamp of the latest/Latest/pkg.txz package is
seemingly stale.

http://pkg0.nyi.freebsd.org/FreeBSD:12:amd64/latest/Latest/pkg.txz has a
timestamp of 2018-Oct-11 01:41.

Can portmgr force a rebuild of this single package to bump it to pkg-1.10.5_5
to get the pkg-static fix?

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

--- Comment #12 from Antoine Brodin <***@FreeBSD.org> ---
(In reply to Glen Barber from comment #11)
We can't do it easily this way, the jail / packages were upgraded to
13.0-CURRENT.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

--- Comment #13 from Glen Barber <***@FreeBSD.org> ---
(In reply to Antoine Brodin from comment #12)

Post by b***@freebsd.org
(In reply to Glen Barber from comment #11)
We can't do it easily this way, the jail / packages were upgraded to
13.0-CURRENT.

Where does the pkg-static binary in the jail come from? Is it installed by the
broken version on the mirrors? Or is it baked into the build jails?

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232254

--- Comment #14 from Antoine Brodin <***@FreeBSD.org> ---
(In reply to Glen Barber from comment #13)
The pkg-static binary in the head jails was built on the head jails the last
time when pkg version or jail version was bumped.