Dependencies, their tracking and what we need to improve.

[Dennis Veatch]

On Sunday, July 17, 2011 3:15:29 PM Auke Kok wrote:
> On 07/17/2011 03:14 AM, Duncan Gibson wrote:
> > Auke wrote:
...
> >> It sounds rather harsh, I can understand that. But given the amount
> >> of core code I wrote for lunar over the years, I think I can safely
> >> say that I've learned enough from it to realize that it would be more
> >> efficient to design something from scratch.
> > 
> > Based on your experience then, and with the benefit of hindsight, what
> > are the pros and cons of the current Lunar package management system?
> > 
> > - Is the basic information/structure of the module control files,
> > 
> >    DETAILS, DEPENDS, etc. sufficient?
> 
> no - the biggest gap is that "BUILD" needs to be split up in "BUILD" and
> "INSTALL" scripts. That would enable things like fakeroot, installwatch
> much more easy. It would also enable building-as-non-root.

I can grasp a split of BUILD making some things more easy. But I will ask a 
dumb question; what is the big deal of building as non-root? Excluding of 
course the potential to set $SOURCE_DIRECTORY/$BUILD_DIRECTORY to / or some 
such.

> 
> DEPENDS/CONFLICTS are inherently the same thing, so they should be in
> one file, actually, why are we doing separate files again? None of this
> should be in separate files to begin with IMO

I dunno why, someone a long time ago suggested CONFLICTS as a quick fix. In any 
case there is probably a more elegant way than a simple c&p of a CONFLICTS 
into DEPENDS. Care to give an example?

> 
> > - Are there things missing?
> 
> definately. subpackages for one.

You mean a split like other distros and you end up with -devel, -manual, -this 
and -that type thing?

> 
...
> 
> > - Is bash the Achilles' Heel of the whole system?
> 
> no.
> 
> but it will never be performant, or secure.
> 
> > If you were to build a new package management system from scratch for
> > the new Sofaris :-) distribution, how would you do it differently?
> 
> no lame names, like "sofaris", for one ;)
> 
....
> 
> I would completely not use any install method like installwatch,
> checkinstall, or paco. It's just not needed anymore... chroot building
> fixes that problem, and it fixes the other problem too. (*)

Ratler made a little patch for gcc to get around that and it looks like this;

sedit 's;^\(build_install_headers_dir=\).*;\1install-headers-cp;' 
gcc/config.build &&

I have been using it for a while now and not found any issues. Sure its a hack 
to get around the installwatch problem of being basically; broken.

> 
> > Whether the package management system is written in bash, C/C++, Perl
> > Python, uses a database for dependencies, or whatever, is irrelevant.
> 
> Not really. At one point the amount of packages needing to be handled
> becomes too big, exactly the problem with lunar.
> 
> That's why redoing lunar in python/perl doesn't make sense: You'd be an
> idiot not going for C/C++ in the first place. So, we stick with bash.

OK. So C/C++ to replace many of the functions used in the lunar 
functions/plugins with bash retained for DETAILS, DEPENDS, etc. So would that 
be it or would a database still be needed/useful?

> 
> > Sure, some give better support for certain operations than others, and
> > may be faster at run-time, but none of the languages themselves address
> > the issues above. They could also introduce new problems: would you
> > want to automatically re-lin pyLunar when lunar update rebuilds Python?
> 
> yes, you most definately will want to do that.
> 
> > For whatever they're worth, my suggestions would be:
> > (a) improve the cpu detection/handling for the gcc plugins;
> 
> what's wrong with -O2 -march=native?
> 
> really, this seems like a non-issue. Just make that the entire
> distro-default. Anyone can then change it for themselves.

I'd say on the next ISO spin to make that the default.

> 
> > (b) streamline the ISO build system so that we can release new ISOs
> > 
> >      as each major version of gcc/udev/whatever hits the moonbase.
> >      Maybe not as full-release ISOs as such, but for use "as-is".
> > 
> > I don't see how we can code around the "smart" configuration systems
> > without building each module in an almost empty chroot environment.
> 
> what's wrong with building in a "clean" environment? seems like an ideal
> situation to me...
> 
> > We could maybe switch to using the slightly more recent installwatch
> > from the checkinstall sources, but that doesn't solve everything.
> > I wonder how the other source based distros work round these issues.
> 
> I checked it out, and, it doesn't function properly: You'll get all
> sorts of broken compiles if you use that codebase.
> 
> > Enough waffle from me.
> 
> Auke

Out of all this I still have to ask this silly question; How does any of this 
address the "clever" scripts that do not have on/off switches?

Or is the real answer to those a less automated build?

-- 
Dennis `stumbles` Veatch
Lunar Linux Developer
http://www.lunar-linux.org/