Problems with the new versioning system

[elaine]

Florin, et al,

I'm replying on -dev because it's a long email and a complex
subject, I can cc or forward later to the main list if/when
appropriate.


A point on naming: versioning 'system' would suggest  that there is
a design and supporting 'stuff' to make versioning work (easily).

There isn't. Versioning is at best a feature. I added it with 1
essential design criteria, that it should not break existing 
functionality, while accomplishing it's objective.



Many key applications install themselves with full version 
information, perl, emacs and gcc would all be good examples. They
build private arch-version specific libraries allowing multiple
versions to be simultaneously installed.

The reason these work is they were developed by people thinking 
about heterogenous networked environments, and they can all be 
installed in multi-architecture/version flavors on a central
network filestore (AFS installations have hooks for this) where
different architectures all pull both their common and arch-specific
code from one location and everything just worked.

If all programs were written in this manner, writing workable
/versioned modules would be a simple matter.

Unfortunately it's not, or at least not always simple. Many 
software builds now seem to install a mix of libraries which
may be accessible thru ldconfig and private libraries known by
their install paths, and it's not always clear which is which
(and some sometimes seem to be both).

Here is an outline of the steps needed to get gnome2/2.4.0 working.

1. ***key*** Backup of /opt/lunar/gnome/2 so that I could roll-back to 
prior state and test bugs in the build.

2. Development of a perl script to roll back /var/state/lunar/packages*
files to match the prior-state gnome(2.2). --- necessary because 
mismatched versions in the packages files cause script breakage.

3. Isolation of the packages which needed to be rewritten in /version
notation. I started with the list which Nick provided for gnome 2.4
build and added a few more which were pulled in as dependencies and
caused breakage if not /versioned. *NOTE* this step is predictably
imperfect, my gnome configuration was only what had been installed
by 'lin gnome2' when it was version 2.2, with not much optional deps
selected. different optional deps and/or added gnome 'stuff' will 
probably add some things to this list.

4. Moving some things out of /usr, into /opt. About 3 of the gnome2
modules used ./configure --prefix=/usr. Nick indicated that this had
been neccessary only because the first time through he'd been unable
to get them working in /opt.


5. Fixing dependencies. Given the list from step 3 I used a shell cmd:

for i in `cat /root/all`; do 
  echo -n "s:($i)[ \t]:\$1/";
  echo -n `basename */$i/[0-9]*`; 
  echo "  :;" ; 
done  > ~/edit.pl

Making the following perl:

s:(linc)[ \t]:$1/1.1.1  :;
s:(libIDL)[ \t]:$1/0.8.2  :;
s:(ORBit2)[ \t]:$1/2.8.1  :;
s:(libbonobo)[ \t]:$1/2.4.0  :;
s:(GConf2)[ \t]:$1/2.4.0.1  :;
...

6. Once all that was done (several iterations of 1,2 a couple 
of 3-5) a clean build.


To Florin's comments below:

Due to the nature of the beast versioning is not going to be
simple and I can't think of anything that's going to materially
smooth the process, however I think the scripting I used to
manage gnome will translate fine to KDE, perhaps we should 
turn it into a moonbase version management tool but I hope 
we can improve on the design before doing that.

On Sat, 20 Sep 2003 01:13:53 -0700 (PDT) (unchecked - local sync NTPstrat4)
Florin Braescu <braescu_f at yahoo.com> did inscribe thusly:


>   To create a new version of a module is not
> difficult. We may just create a new subdirectory in
> the moonbase/$MODULE directory, named it with the new
> version and copy the stuff of the moonbase/$MODULE
> directory in the new subdirectory. After that we begin
> to tweak the files to reflect the new changes.
> 
>   One first problem may consist in establishing the
> location for installing the new version module. For
> kde3 we use /opt/lunar/kde/3, so i have used for the
> new version the /opt/lunar/kde/3.1.4 location. For big
> modules like GNOME, KDE, XFCE, etc it's easy to use
> such /opt/lunar/$MODULE/$VERSION location. It has the
> advantage to have the stuff located in a delimited
> place. It may be a problem for usual modules, if you
> want to keep the old version too (the files will be
> overriden by the new version).
> 
>   A second problem consist in overriding the common
> files between different installed versions of the same
> module. KDE use a /etc/profile.d/kde3.rc file to
> specify the location of the modules. To have different
> versions installed on the system we must create
> versions for such files too.

s/must/may/

As most .rc scripts don't badly break prior operation I 
would just let them overwrite. 

Please remember versioning has 2 (main) intentions:

1. allowing relatively long-term coexistence of (mostly)
programming tool versions.

2. enabling more orderly transition (and perhaps safer rollback)
between versions of the more complex and error-prone modules.

The latter are unlikely to have to co-exist for extended periods,
careful planning and backup of course may be needed to preserve
all details such as .rc scripts which aren't 


>   A third problem is running the new executables.
> Linux use the PATH variable to search the first
> executable with the asked name, and launch it.
> In our case there could be many executables with the
> same name, placed in distinct directories and is
> difficult to select them. I can launch the startkde
> program for version 3.1.4, but the KDE system will
> launch after that the needed programs using the PATH
> variable, so it will use the a previous version of
> them if i do not modify the PATH value as needed.
> 
Yes, it's assumed that if you want to switch between versions 
you may need to a. modify the PATH (not *always necessary,
e.g. *most, not all* applications will build correctly by setting
CC=/path/to/my/C/compiler.

Additionally you may need to either set LD_LIBRARY_PATH or edit 
/etc/ld.so.conf and run 'ldconfig' which is more invasive and 
(for now) requires root privilege.


>   A fourth problem regards dependencies. KDE is a
> integrated package of many modules. It's not difficult
> to create and use a new version of it (after 
> fixing the previuos problems). But related to it there
> are many apps which may work in a large range of KDE
> versions (usually we can see them in kde-apps,
> kde-utils, games, etc directories in moonbase). Their
> associated DEPENDS files contain such lines:
>       depends  "kdelibs3"
> 
>   When we uninstall the kdelibs3 (the previous version
> of KDE) building such a module will want to install
> the depends modules again. So we must find a way to
> point to the new versions. We can use
>      depends  "kdelibs3/3.1.4" 
> but the problem remains the same and it would be a
> huge task to modify all the dependent modules in
> moonbase when we will create a new version for KDE
> (even considering scripts for that). At some point
> some such modules may not work with the new versions.
> For the modules which can be used in a large range or
> versions we need to specify somehow a range of
> depends, something like 3*, 3.1.3-3.1.4 or similar.
> 
>   Another related problem is what version is used for
> building such modules. We can use a global variable
> like 
>    KDE_CURRENT_VERSION=3.1.4 
> in a global config file (or deduce it somehow from the
> installed moonbase) and use something like that
>    depends "kdelibs3/$KDE_CURRENT_VERSION"
> 

Yup, this is somewhat the crux of a module like gnome/kde,
see above.


>   Such modules are installed now in /opt/lunar/kde/3
> directory. I belive we must use /opt/lunar/kde/apps,
> /opt/lunar/kde/utils, /opt/lunar/kde/games,
> etc directories for them for now on, to make the
> install location independent of the version in use.
> 

I think this is going to depend on the application. I can't 
say that this is the right or wrong approach, it's probably
a judgement call. The best choice is probably the one that 
minimizes future pain?


elaine