[Bug 225104] ports-mgmt/pkg: "pkg update" fails to update package database saying that kernel too old for some package

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

Mathieu Arnold <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Assignee|freebsd-ports-***@FreeBSD. |***@FreeBSD.org
|org |

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

Baptiste Daroussin <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Assignee|***@FreeBSD.org |freebsd-ports-***@FreeBSD.
| |org

--- Comment #2 from Baptiste Daroussin <***@FreeBSD.org> ---
What is uname -UK telling you? (both of you?)

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

Mark Linimon <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Assignee|freebsd-ports-***@FreeBSD. |***@FreeBSD.org
|org |

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

Yngve Fiskum <***@hotmail.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@hotmail.com

--- Comment #8 from Yngve Fiskum <***@hotmail.com> ---
FreeNAS with Plex jail

***@Plex:/ # pkg update -f
Updating FreeBSD repository catalogue...
[Plex] Fetching meta.txz: 100% 944 B 0.9kB/s 00:01
[Plex] Fetching packagesite.txz: 100% 6 MiB 6.2MB/s 00:01
Processing entries: 0%
pkg: Newer FreeBSD version for package gnome-desktop:
- package: 1101001
- running kernel: 1100122
pkg: repository FreeBSD contains packages for wrong OS version:
FreeBSD:11:amd64
Processing entries: 100%
Unable to update repository FreeBSD
Error updating repositories!
***@Plex:/ # uname -UK
1101505 1100122

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

--- Comment #9 from Nils Beyer <***@renzel.net> ---
Okay, after digging through the pkg source code, I found a possible workaround.
Add following line to "/usr/local/etc/pkg.conf":
-------------------------------------------------------------------------
OSVERSION = 1200056;
-------------------------------------------------------------------------

for the 12 users.


Respective:
-------------------------------------------------------------------------
OSVERSION = 1101001
-------------------------------------------------------------------------

for the 11 users.


After that, the repository updates should be working again...

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

--- Comment #10 from Guido Falsi <***@FreeBSD.org> ---
(In reply to Nils Beyer from comment #9)

Post by b***@freebsd.org
Okay, after digging through the pkg source code, I found a possible
-------------------------------------------------------------------------
OSVERSION = 1200056;
-------------------------------------------------------------------------
for the 12 users.
-------------------------------------------------------------------------
OSVERSION = 1101001
-------------------------------------------------------------------------
for the 11 users.
After that, the repository updates should be working again...

Can also be passed through the command line:

pkg -o OSVERSION=1200056 upgrade

Which reduces the risk of forgetting the line in the configuration file.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

***@ximalas.info changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@ximalas.inf
| |o

--- Comment #11 from ***@ximalas.info ---
(In reply to Nils Beyer from comment #9)
The advice in comment #9 is needed when using synth to upgrade the
"LiveSystem".

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

--- Comment #12 from Nils Beyer <***@renzel.net> ---
pkg verifies "osversion" using the data in the ELF ".note.tag" section of the
file specified by the environment variable "ABI_FILE". If "ABI_FILE" is not
defined it chooses "/bin/sh" as the reference file:
-----------------------------------------------------------------------------
#objdump --section=.note.tag -s /bin/sh

/bin/sh: file format elf64-x86-64-freebsd

Contents of section .note.tag:
400218 08000000 04000000 01000000 46726565 ............Free
400228 42534400 b84f1200 08000000 04000000 BSD..O..........
400238 02000000 46726565 42534400 00000000 ....FreeBSD.....

#echo "ibase=16; 00124FB8" | bc
1200056

#uname -KU
1200058 1200058
-----------------------------------------------------------------------------

Because I'm doing incremental builds (using -DNO_CLEAN) of world/kernel, the
version information in my "/bin/sh" and all other binaries is not increased
because "host-osreldate.h" in ${OBJTOP} is not increased for some reason.
"getosreldate()" returns the correct value though...

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

Roger Pau Monné <***@freebsd.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@freebsd.org

--- Comment #13 from Roger Pau Monné <***@freebsd.org> ---
I certainly preferred previous pkg behaviour, which would let you install
anything (and it might fail to load because of missing/wrong shared libraries).
Is there any option to restore previous behaviour? If not, can we please get
one?

Thanks, Roger.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

Shawn Webb <***@hardenedbsd.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@hardenedbsd.org

--- Comment #14 from Shawn Webb <***@hardenedbsd.org> ---
I myself have been bit with this on both vanilla FreeBSD and HardenedBSD. We at
HardenedBSD have at least two or three users getting hit with this on a weekly
basis.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

--- Comment #15 from Nils Beyer <***@renzel.net> ---
(In reply to Roger Pau Monné from comment #13)

well, I can understand that some packages - especially kernel modules packages
- need a minimum kernel version for functioning properly.

But why "timidity++-tcltk" or "qt5-x11extras" (in my yesterday case) are
needing a minimum kernel version, that's a mystery.

And relying on data in ELF sections in binaries where the version of the
building host and not the version of the currently running host is saved,
instead of using getosreldate() or sysctls is another question IMHO...

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

--- Comment #16 from Roger Pau Monné <***@freebsd.org> ---
(In reply to Nils Beyer from comment #15)
Also keep in mind that's there's no base library ABI stability in HEAD, so it's
not just kernel modules, but everything linked to any base library is only
guaranteed to work against the version it has been build against. Everything
else is just luck.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

--- Comment #17 from Nils Beyer <***@renzel.net> ---
(In reply to Roger Pau Monné from comment #16)

true; so, there really have to be two checks:

current kernel version vs. package kernel version requirement

AND

current base system version vs. package base system version requirement

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

***@ximalas.info changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@ximalas.inf
| |o

--- Comment #18 from ***@ximalas.info ---
(In reply to Trond.Endrestol from comment #11)
The right place is the LiveSystem-environment file or equivalent.

(In reply to Nils Beyer from comment #12)
Regarding comment #12, I first thought it prudent to delete the binaries from
/usr/obj/$ARCH/usr/src/bin/sh prior to running make buildworld, until the
matter is fully resolved. I.e. delete sh, sh.debug, and sh.full. I was proved
wrong. Clearing /usr/obj and rebuilding world led to a sh binary with the right
osversion, see below. Which file(s) should we forcefully remove to ensure the
right value of osversion for sh?

# objdump --section=.note.tag -s sh
sh: file format elf64-x86-64-freebsd
Contents of section .note.tag:
400218 08000000 04000000 01000000 46726565 ............Free
400228 42534400 c5ce1000 08000000 04000000 BSD.............
400238 02000000 46726565 42534400 00000000 ....FreeBSD.....

C5CE1000 (LE) => 0010CEC5 (BE) = 1101509

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

--- Comment #19 from Nils Beyer <***@renzel.net> ---
(In reply to Trond.Endrestol from comment #18)

As far as I understand; the responsible file would be:

${OBJTOP}/host-osreldate.h

But that doesn't matter because the "osversion" tag within the ELF files is
always lagging behind.

Example: you're running osversion=1101508 and building a new world/kernel with
osversion=1101509. All of your ELF files are tagged with what is in

/usr/include/osreldate.h

at the moment when you build world. So, after finishing building and installing
world you have all ELF files tagged with 1101508 although you're running
1101509. Furthermore, "host-osreldate.h" is never updated (here on my machine
using NO_CLEAN), even if I build the same world with the up-to-date installed
world; it just says "`host-osreldate.h' is up to date." which is not true.

Perhaps it would be better for "libpkg" to rely on what "sysctl" or
"getosreldate()" are saying.

But that's just my simple-minded user's point of view...

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

--- Comment #20 from Nils Beyer <***@renzel.net> ---
(In reply to Roger Pau Monné from comment #13)

my solution to get the old behaviour back would be to add following line to
"/usr/local/etc/pkg.conf":
--------------------------------------------------------------------------
OSVERSION = 9999999;
--------------------------------------------------------------------------

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

--- Comment #21 from ***@ximalas.info ---
(In reply to Nils Beyer from comment #19)
I see the same behaviour on stable/11 and there's no ${OBJTOP}/host-osreldate.h
in sight, only:

# locate osreldate.h
/usr/include/osreldate.h
/usr/obj/usr/src/include/osreldate.h
/usr/obj/usr/src/lib32/usr/include/osreldate.h
/usr/obj/usr/src/tmp/usr/include/osreldate.h
/usr/obj/usr/src/world32/usr/src/include/osreldate.h

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

--- Comment #22 from Nils Beyer <***@renzel.net> ---
(In reply to Trond.Endrestol from comment #21)

you're right; I don't have that file on my older system, too. So this file does
not matter.

I'm out off ideas here, sorry; you'll have to clean "/usr/obj" before building
world or have to use the "OSVERSION" workaround in "/usr/local/etc/pkg.conf"...

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

--- Comment #23 from ***@ximalas.info ---
(In reply to Nils Beyer from comment #22)
Relying on /usr/include/osreldate.h during make buildworld must be due to a
short circuit somewhere. Maybe I'm wrong. Using
/usr/obj/usr/src/include/osreldate.h might be problematic for the preprocessor
when handling the other header files.

(In reply to Nils Beyer from comment #20)
I like your idea of setting OSVERSION to its maximum value in pkg.conf (and
LiveSystem-environment for users of synth).

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

Bryan Drewery <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@FreeBSD.org

--- Comment #24 from Bryan Drewery <***@FreeBSD.org> ---
(In reply to Roger Pau Monné from comment #13)

Post by b***@freebsd.org
I certainly preferred previous pkg behaviour, which would let you install
anything (and it might fail to load because of missing/wrong shared
libraries). Is there any option to restore previous behaviour? If not, can
we please get one?
Thanks, Roger.

The previous behavior was called "russian roulette". Every time you run
"pkg upgrade" you risk nuking the usability of all packages depending on
what forward-incompatible change they were built against. Some recent
examples are linking against libdl and ino64.

The workaround people are using should not be used - the proper thing with
pkg on CURRENT has *always been* that you should upgrade the system before
upgrading packages. It is a different process than building ports directly
on your system. Building locally ensures ABI compatibility every time, but
with pkg you are getting packages from a future CURRENT system. FreeBSD
does not have forward-compatibility in general, only backwards.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

--- Comment #25 from Bryan Drewery <***@FreeBSD.org> ---
(In reply to Nils Beyer from comment #19)

Post by b***@freebsd.org
(In reply to Trond.Endrestol from comment #18)
${OBJTOP}/host-osreldate.h

This file should be used for absolutely nothing. It is a private build
metadata file.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

--- Comment #26 from Nils Beyer <***@renzel.net> ---
(In reply to Bryan Drewery from comment #24)

[...] the proper thing with pkg on CURRENT has *always been* that you should upgrade the system before upgrading packages.

that's what I have done (and always have done); please see my comment #3 - that
doesn't explain why I'm getting while bootstrapping "pkg":
------------------------------------------------------------------------
pkg-static: Newer FreeBSD version for package pkg:
- package: 1200055
- running kernel: 1200054
------------------------------------------------------------------------

although "uname -UK" tells otherwise:
------------------------------------------------------------------------
#uname -UK
1200056 1200056
------------------------------------------------------------------------


I think the string "running kernel" in pkg's output is misleading as the
"libpkg" logic doesn't check the kernel version but the tagged version in
"/bin/sh"...

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

--- Comment #27 from commit-***@freebsd.org ---
A commit references this bug:

Author: brd
Date: Thu Feb 15 00:14:50 UTC 2018
New revision: 461849
URL: https://svnweb.freebsd.org/changeset/ports/461849

Log:
Update pkg to 1.10.5

PR: 225104
Reviewed by: bdrewery

Changes:
head/ports-mgmt/pkg/Makefile
head/ports-mgmt/pkg/distinfo

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

--- Comment #28 from Roger Pau Monné <***@freebsd.org> ---
(In reply to Bryan Drewery from comment #24)
I agree, but I still want to know what's the recommended way to circumvent this
limitation. Is adding "OSVERSION = 9999999;" to pkg.conf the right way?

I know things might blow up, but worst case I will just update base if
required.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

Brad Davis <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@FreeBSD.org

--- Comment #29 from Brad Davis <***@FreeBSD.org> ---
(In reply to Roger Pau Monné from comment #28)
Hi Roger,

I would recommend using:

pkg -o OSVERSION=whatever install/upgrade/etc

until 1.10.5 is on the mirrors.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225104

Brad Davis <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |FIXED
Status|New |Closed