md5sums verification results over 12 caches - proposal

Auke Kok sofar at lunar-linux.org
Mon Aug 18 22:47:11 GMT 2003


May I have your attention please?


While gathering md5sums over 12 different /var/spool/lunar/ caches we
got 17981 md5sums over 4914 different files (including old versions too
BTW)

of those files, we have 103 files with conflicting md5sums, of which a
lot are -cvs modules etc... still there are some bizarre results...
Attached is a file with the problem cases. The first column of the file
is the number of unique md5sums and the second number is the number of
md5sums gathered for that file.

not all caches have all files, if we automate adding md5sums to moonbase
and want to add a threshold (minimally x caches required before we
accept a certain md5sum) we get this number

threshold	number of files
3		3219
4		2234
5		1772
6		998
7		396
8		244
9		141
10+		78

I propose we 1) massively add automated md5sums to moonbase based on
these results from out caches, where we only use those md5sums which are
verified by 5 different caches. 

All other files remaining will need to do the following:

* 2) if no conflicting md5sums, check if the author has md5sums on the
website available and add them

* 3) if we have conflicting md5sums, gather all files from the different
caches and investigate on a per-file basis


In order to speed things up we can start with 1) and leave 2) and 3) to
the team effort.

sofar



-- 
Auke Kok <sofar at lunar-linux.org>
-------------- next part --------------
      3       3 /var/spool/lunar/abs-guide-1.6.tar.bz2
      5       5 /var/spool/lunar/abs-guide-1.7.tar.bz2
      2       2 /var/spool/lunar/abs-guide-1.9.tar.bz2
      5       5 /var/spool/lunar/adonthell-0.3.3.tar.gz
      3       4 /var/spool/lunar/alsa-xmms-0.9.tar.gz
      3       8 /var/spool/lunar/aspell-0.50.3-missing-include.patch.bz2
      2       5 /var/spool/lunar/battalion1.4b.tar.bz2
      3       3 /var/spool/lunar/bitstream-vera-1.0.tar.gz
      4       6 /var/spool/lunar/black-box-1.4.1.tar.gz
      4       4 /var/spool/lunar/blender-2.26.tar.bz2
      4       4 /var/spool/lunar/blender-cvs-2.26.tar.bz2
      2       2 /var/spool/lunar/blender-cvs-2.27.tar.bz2
      2       2 /var/spool/lunar/blender-cvs-2.28.tar.bz2
      3       3 /var/spool/lunar/bumprace-1.43.tar.gz
      3       4 /var/spool/lunar/castle-combat-0.6.0.tar.gz
      2       5 /var/spool/lunar/Chaos
      2       2 /var/spool/lunar/crafty-18.15.tar.gz
      3       5 /var/spool/lunar/cutter-1.02.tgz
      3       7 /var/spool/lunar/default-1.7.tar.bz2
      3       5 /var/spool/lunar/drac.tar.Z
      3       8 /var/spool/lunar/edb-1.0.3.tar.gz
      3       3 /var/spool/lunar/endeavour-2.1.19.tar.bz2
      3       3 /var/spool/lunar/fasm-1.41.tar.gz
      2       2 /var/spool/lunar/flash_linux.tar.gz
      3       8 /var/spool/lunar/font-arial-cp1250.tar.bz2
      3       9 /var/spool/lunar/font-arial-iso-8859-1.tar.bz2
      3       9 /var/spool/lunar/font-arial-iso-8859-2.tar.bz2
      4       7 /var/spool/lunar/foomatic-rip
      6       6 /var/spool/lunar/gaim-cvs-beta.tar.bz2
      4       4 /var/spool/lunar/galeon_cvs-beta.tar.bz2
      2       2 /var/spool/lunar/gcc-3.2-glibc-2.3-compat.diff
      5       5 /var/spool/lunar/giftcurs-cvs-beta.tar.bz2
      5       5 /var/spool/lunar/gift-cvs-beta.tar.bz2
      5       5 /var/spool/lunar/giftfast-cvs-beta.tar.bz2
      2       2 /var/spool/lunar/giftgnutella-cvs-beta.tar.bz2
      2       2 /var/spool/lunar/giftopenft-cvs-beta.tar.bz2
      5       5 /var/spool/lunar/giftoxic-cvs-beta.tar.bz2
      5       5 /var/spool/lunar/giftui-cvs-beta.tar.bz2
      4       5 /var/spool/lunar/GKrellM-Skins.tar.gz
      3       5 /var/spool/lunar/glaxium_0.4b.tgz
      2       2 /var/spool/lunar/ieee1394-svn-beta.tar.bz2
      4       6 /var/spool/lunar/ImageMagick-5.5.7-10.tar.bz2
      7       9 /var/spool/lunar/install_flash_player_6_linux.tar.gz
      4       5 /var/spool/lunar/ircii-current.tar.bz2
      3       3 /var/spool/lunar/kde-i18n-fr-3.0.1.tar.bz2
      3       6 /var/spool/lunar/kile-1.5.tar.gz
      2       6 /var/spool/lunar/lbreakout2-2.4.1.tar.gz
      2       5 /var/spool/lunar/lcms-1.09.tar.gz
      3       5 /var/spool/lunar/libpcap-0.7.1.tar.gz
      5       5 /var/spool/lunar/libxfce4mcs-beta.tar.bz2
      5       5 /var/spool/lunar/libxfce4util-beta.tar.bz2
      5       5 /var/spool/lunar/libxfcegui4-beta.tar.bz2
      5       5 /var/spool/lunar/Linux-HOWTOs.tar.gz
      3       3 /var/spool/lunar/linux_logo-4.02.tar.gz
      5       5 /var/spool/lunar/Linux-mini-HOWTOs.tar.gz
      9       9 /var/spool/lunar/lunar.tar.bz2
      5       5 /var/spool/lunar/mknbi-1.2.tar.gz
      3       8 /var/spool/lunar/mozilla-1.4-lunar.tar.bz2
      5       5 /var/spool/lunar/mozilla-source.tar.bz2
      5       5 /var/spool/lunar/openal-cvs-beta.tar.bz2
      5       5 /var/spool/lunar/pan-cvs-cvs.tar.bz2
      5       7 /var/spool/lunar/patch.3.3.11.1
      3       3 /var/spool/lunar/penguin-command-1.6.2.tar.gz
      3       7 /var/spool/lunar/ptail.pl
      4       7 /var/spool/lunar/remember-0.1.14.gz
      6       6 /var/spool/lunar/tidy_docs.tgz
      6       6 /var/spool/lunar/tidy_src.tgz
      2       5 /var/spool/lunar/tkrat-2.0.3.tar.gz
      5       5 /var/spool/lunar/vim-cvs-cvs.tar.bz2
      3       5 /var/spool/lunar/webmin-1.100.tar.gz
      5       6 /var/spool/lunar/whois-current.tgz
      3       9 /var/spool/lunar/win32codecs.tar.bz2
      4       4 /var/spool/lunar/Wine-cvs-beta.tar.bz2
      4       4 /var/spool/lunar/wxAll-2.4.0.tar.gz
      3       3 /var/spool/lunar/xfce4-battery-plugin-cvs-beta.tar.gz
      3       3 /var/spool/lunar/xfce4_cvs-beta.tar.bz2
      5       5 /var/spool/lunar/xfce4-cvs-beta.tar.bz2
      3       3 /var/spool/lunar/xfce4_cvs_gtk2_engine-beta.tar.bz2
      3       3 /var/spool/lunar/xfce4_cvs_gui_lib-beta.tar.bz2
      3       3 /var/spool/lunar/xfce4_cvs_mcs_lib-beta.tar.bz2
      3       3 /var/spool/lunar/xfce4_cvs_mcs_manager-beta.tar.bz2
      3       3 /var/spool/lunar/xfce4_cvs_mcs_plugins-beta.tar.bz2
      2       2 /var/spool/lunar/xfce4_cvs_util_lib-beta.tar.bz2
      3       3 /var/spool/lunar/xfce4_cvs_utils-beta.tar.bz2
      3       3 /var/spool/lunar/xfce4_cvs_xfdeskmenu-beta.tar.bz2
      3       3 /var/spool/lunar/xfce4_cvs_xfdesktop-beta.tar.bz2
      4       4 /var/spool/lunar/xfce4-extras-cvs-beta.tar.bz2
      5       5 /var/spool/lunar/xfce4-iconbox-beta.tar.bz2
      2       2 /var/spool/lunar/xfce4-minicmd-plugin-cvs-beta.tar.gz
      3       3 /var/spool/lunar/xfce4-session-beta.tar.bz2
      2       2 /var/spool/lunar/xfce4-showdesktop-plugin-cvs-beta.tar.gz
      3       3 /var/spool/lunar/xfce4-systemload-plugin-cvs-beta.tar.gz
      5       5 /var/spool/lunar/xfce-mcs-manager-beta.tar.bz2
      5       5 /var/spool/lunar/xfce-mcs-plugins-beta.tar.bz2
      5       5 /var/spool/lunar/xfce-utils-beta.tar.bz2
      5       5 /var/spool/lunar/xfdesktop-cvs-beta.tar.bz2
      5       5 /var/spool/lunar/xffm4-cvs-beta.tar.bz2
      3       3 /var/spool/lunar/xffm_cvs-beta.tar.bz2
      5       5 /var/spool/lunar/xfgtk2-engine-cvs-beta.tar.bz2
      5       5 /var/spool/lunar/xfprint-cvs-beta.tar.bz2
      3       3 /var/spool/lunar/xfwm4_cvs-beta.tar.bz2
      5       5 /var/spool/lunar/xfwm4-cvs-beta.tar.bz2
      2       2 /var/spool/lunar/zsnes-cvs-beta.tar.bz2


More information about the lunar-dev mailing list