[Lunar-bugs] [task] Source verification in wrong order when using mirrors

[Lunar bug reports list]

 Project:      lunar-linux
 ID:           
 Version:      <none>
 Component:    theedge (core tools)
 Category:     tasks
 Priority:     normal
 Assigned to:  sofar
 Reported by:  hardkrash
 Updated by:   tchan
 Status:       fixed

You do realize that your "fix" is only in the lunar branch ?  In other
words your "fix" is not propagated to the "theedge" branch.  So devs
that only use theedge won't see your fix.

tchan



Previous comments:
------------------------------------------------------------------------

Mon, 06/06/2005 - 18:13 : hardkrash

When downloading source and the checksum fails it causes lunar to fail
to install that module.

This stinks,  if other mirrors exist then we should try them as well
till we get a valid source or the mirror list is exhausted, if the user
decides to delete the source.

my thought is to check the checksum just after the file is downloaded
and then decide if it needs to be removed and then the next mirror
tried.


------------------------------------------------------------------------

Tue, 06/07/2005 - 03:33 : sofar

there are 3 parts to this problem

1) there's a huge performance loss in case of large downloads. To
verify every download means to do a sha1/md5/whichever method on the
temporarily downloaded file. Since we really must verify prior to
installation as well (the user could have manually downloaded a corrupt
or trojaned file), this does not really make sense

2) the *REAL* problem is that http mirrors give 404 error pages with an
200 OK http return code, therefore making it impossible to see if the
download was really succesfull or that we were downloading something
completely different. Wget doesn't support strict file downloading and
to me this is a serious flaw in wget

3) the download code has no support for immediate verification of
downloads. This probably means some nasty changes to the download code
that I'd rather not do. I am much more for fixing the problem at the
root, which is (2).

If someone likes to take up a nice C hacking project and add strict
downloading to wget I'd be very grateful.

------------------------------------------------------------------------

Tue, 06/07/2005 - 09:37 : sofar

I've implemented a better fix for this problem: the testpack() function
is called after every "succesfull download" on each file. This is the
perfect location to verify that a tarball is really a tarball but not
to perform integrity checking (We can't rely on this since the
sha1sum/md5sum *may* be absent).

The previous testpack function used 'file' to determine content. This
meant that 404 html documents were NOT checked at all since they didn't
fit the test criteria:

if (contents look like tarballs) then
   check
fi

this has been changed to:

if (name describes tarball) then
   check
fi

this is a very strict method but insures that all files that end in
'.tar.bz2' and all derivatives are really tarballs etc. 

This closes this bug completely IMO. I don't think this will bother us
anymore ;^)

-- 
View: http://lunar-linux.org/?q=node/view/874
Edit: http://lunar-linux.org/?q=project/comments/add/874