Dependencies, their tracking and what we need to improve.

[Auke Kok]

On 07/15/2011 04:36 AM, Dennis Veatch wrote:
> On Friday, July 15, 2011 1:19:44 PM wookietreiber wrote:
>> On Fri, Jul 15, 2011 at 06:45:21AM -0400, Dennis Veatch wrote:
>>> On Thursday, July 14, 2011 11:50:19 PM Auke Kok wrote:
>>>> On 07/14/2011 07:20 AM, Jean-Michel Bruenn wrote:
>>>>> Hey
>>>>>
>>>>>> if we had a way to build packages in a completely pristine
>>>>>> environment without non-preferred dependencies, then this
>>>>>> would not be a problem.
>>>>>>
>>>>>> the cost of building in a chroot with only dependencies are
>>>>>> that it is slow. and you might as well run an rpm-based
>>>>>> system almost, since in order to obtain the flexibility that
>>>>>> lunar grants, we'd have to do a LOT
>>>>>> more dependency and option mapping than we do now.
>>>>>>
>>>>>> Solving that problem... wouldn't be worth doing in the current
>>>>>> codebase.
>>>>>
>>>>> The interesting question behind all this is: Do we want an
>>>>> improvement or not? Is it worth, trying to improve this or not?
>>>>> If it's not worth or possible with the current codebase, what
>>>>> steps could we do to make it worth it/possible. Do we have any
>>>>> alternatives? Apart from switching to rpm's of course :)
>>>>
>>>> we could even switch to rpm... I'm not kidding.
>>>>
>>>> the problem isn't the packaging format. the problem is the level of
>>>> freedom that you want. no current build system will give it to you,
>>>> so,
>>>> switching package formats doesn't fill the gap.
>>>>
>>>>> Also, "if we had a way .. in a completly pristine environment"
>>>>> Let's
>>>>> assume, we do it like this. I wrote a howto for a Chrooted
>>>>> Lunar,
>>>>> which is working quite nice. I've been using that thing first as
>>>>> zone in opensolaris, later as 32bit chroot for lunar and now
>>>>> just for testing. I might be able to make it a lot smaller
>>>>> (~500 MB packed, 1.8 G unpacked) for the above purpose, and i
>>>>> might make it install automatically using "lchroot" or
>>>>> something. So let's assume further, we take this chroot, for
>>>>> the purpose of doing dependency tracking, even if that sounds a
>>>>> bit.. masochistic. What possible improvements could we do, to
>>>>> make it "faster" (as you said it would be slow) and what
>>>>> alternatives do we have?
>>>>
>>>> stop coding it in bash, start over in C.
>>>
>>> I wouldn't be opposed to C or perl, python or whatever.
>>>
>>> What I do not see any of those being able to solve is the root of the
>>> problem or nearly so; the lack of an on/off switch. As I mentioned,
>>> there are many instances of configure and cmake that will look for an
>>> application but give you no way to enable/disable. How would you
>>> address that with C?
>>
>> doesn't depend on the language
>>
>> what I think could solve the problem is to combine CONFIGURE and DEPENDS or
>> at least use logic in any way in depends -- both solve hell'a more
>> problems, e.g.
>>
>> with-x? then
>> 	with-x-screensaver?
>> else
>> 	with-direct-fb?
>> ...
>>
>> currently this kind of configuration is impossible to do right ...
>>
>> regards
>> wookietreiber
>
> No that really doesn't solve the no on/off switch problem. Suppose you have
> grantlee installed and for whatever reason you do not want kdelibs to use it.
> The kdelibs cmake does not have a dis-abler and while grantlee is installed,
> cmake will search for and include its libs/includes during its configure and
> kdelibs will be built with grantlee. The only way I see to allow for a user to
> truly say no to grantlee is to lrm grantlee before kdelibs runs cmake.
>
> So for those configure and cmake scripts that do not provide an on/off switch,
> what do we insert into the optional_depends for grantlee to allow the user the
> ability to truly say no?
>
> I do not see a level of logic we can insert or rewrite how optional_depends
> are used when we are dependent on configure, cmake, waf -configure, etc to
> provide us with the proper/appropriate methods to enable/disable an optional
> dependency.
>
> Maybe there is a way and I am just blind.
>

the only real solution is to build in an environment where you control 
100% what the configure script "sees".

there are several ways of doing that

- chroot or VM - guaranteed 100% works exactly as expected

- tricks (like my pkg-config plugin) that fool the system into thinking 
disabled dependencies are not there - this will break in the end.

I'm not too convinced to make lunar more complex with really nasty hacks 
anymore... I've spent enough time on it to realize that apart from 
keeping lunar going as is, it's just not worth my time to "refine" the 
current codebase.

changing lunar to build stuff in chroots isn't something small either, 
and because chroot building is complex (you need to redo the chroot for 
every compile), the current bash codebase is just too slow to do this 
efficiently.

do I know how to solve it? sure. but it will be the end of lunar, and 
like starting from scratch....

Auke