concurrent builds

[Duncan Gibson]

> I wanted to present a use-case for concurrent builds with lin.
>
> If all modules could be built with 'make -j CORES' this wouldn't be
> necessary, but unfortunately this is not the case. Some modules aren't
> PSAFE (they can't build concurrently), others don't use make. So with
> non-concurrent builds it is not always possible to achieve optimal
> resource usage and the minimum amount of time to install modules (with
> modern CPU architectures in mind, that have many cores).

I'm no expert on the internals of the dependency tracking so what comes
next might be completely wrong.

I have the feeling that for your idea to work, every possible dependency
in the moonbase would need to be explicitly specified somewhere, whether
it's a build time dependency (eg it uses another tool during installation)
or it's a run-time dependency (eg it needs another tool/library to run).
Both depends and optional_depends would need to be tracked.

This would make the maintenance of each module in the moonbase much more
time-consuming because it means that potentially every configure option
needs to be checked, and added/updated/deleted with each version bump.

As it stands, many modules have configuration files that only pull in
features from other modules if those modules are already installed. In
some cases the other module is definitely required at build or run time,
and so the module needs a "depends other_module". In others the module
may build/run without another module, but it relates to a common task
so the user might want to consider installing it, ie an "optional_depends".

Requiring explicit specification of all possible dependencies would make
the system brittle, because all modules' dependencies would need to be
consistent with each other, or else you could build out of order.

It isn't clear to me that the potential benefits of being able to build
modules in parallel to speed up installation outweigh the extra mainenance
required to keep all of the dependencies up-to-date and consistent.

Just my 2c.

D.