From c9d84479d239542f0727f756d2abfc24cdb0d878 Mon Sep 17 00:00:00 2001 From: Silvan Calarco Date: Mon, 12 Mar 2012 15:18:41 +0100 Subject: [PATCH] Initial commit --- COPYING | 341 +++++++++ ChangeLog | 57 ++ Makefile | 75 ++ VERSION | 1 + bin/openmamba-dw-checklist | 216 ++++++ bin/openmamba-newrelease | 25 + bin/openmamba-newrep | 176 +++++ bin/openmamba-repository | 712 ++++++++++++++++++ etc-frontend/cron.hourly/05-ntpd | 2 + .../cron.hourly/10-openmamba-downloads | 77 ++ .../20-openmamba-developer-messages | 28 + etc-frontend/cron.weekly/10-openmamba-backup | 23 + etc/cron.daily/10-openmamba-mirror-daily | 43 ++ .../20-openmamba-autobuild-makedist | 265 +++++++ .../30-openmamba-autobuild-cleanold | 46 ++ etc/cron.daily/40-openmamba-dw-checklist | 11 + etc/cron.daily/50-openmamba-fix-permissions | 19 + etc/cron.hourly/10-openmamba-sync-back | 52 ++ etc/cron.hourly/20-openmamba-apt | 17 + etc/cron.hourly/30-openmamba-distromatic | 74 ++ etc/cron.hourly/40-openmamba-rsync | 78 ++ etc/cron.hourly/50-openmamba-ftpsync | 62 ++ .../60-openmamba-autobuild-autodist | 35 + etc/cron.weekly/10-openmamba-mlarchives | 13 + etc/cron.weekly/20-gencontents | 35 + etc/sysconfig/openmamba-central | 48 ++ 26 files changed, 2531 insertions(+) create mode 100644 COPYING create mode 100644 ChangeLog create mode 100644 Makefile create mode 100644 VERSION create mode 100755 bin/openmamba-dw-checklist create mode 100755 bin/openmamba-newrelease create mode 100755 bin/openmamba-newrep create mode 100755 bin/openmamba-repository create mode 100755 etc-frontend/cron.hourly/05-ntpd create mode 100755 etc-frontend/cron.hourly/10-openmamba-downloads create mode 100755 etc-frontend/cron.hourly/20-openmamba-developer-messages create mode 100755 etc-frontend/cron.weekly/10-openmamba-backup create mode 100755 etc/cron.daily/10-openmamba-mirror-daily create mode 100755 etc/cron.daily/20-openmamba-autobuild-makedist create mode 100755 etc/cron.daily/30-openmamba-autobuild-cleanold create mode 100755 etc/cron.daily/40-openmamba-dw-checklist create mode 100755 etc/cron.daily/50-openmamba-fix-permissions create mode 100755 etc/cron.hourly/10-openmamba-sync-back create mode 100755 etc/cron.hourly/20-openmamba-apt create mode 100755 etc/cron.hourly/30-openmamba-distromatic create mode 100755 etc/cron.hourly/40-openmamba-rsync create mode 100755 etc/cron.hourly/50-openmamba-ftpsync create mode 100755 etc/cron.hourly/60-openmamba-autobuild-autodist create mode 100755 etc/cron.weekly/10-openmamba-mlarchives create mode 100755 etc/cron.weekly/20-gencontents create mode 100644 etc/sysconfig/openmamba-central diff --git a/COPYING b/COPYING new file mode 100644 index 0000000..c3ec62d --- /dev/null +++ b/COPYING @@ -0,0 +1,341 @@ + + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc. + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Library General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) 19yy + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) 19yy name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + , 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Library General +Public License instead of this License. diff --git a/ChangeLog b/ChangeLog new file mode 100644 index 0000000..20e4cb1 --- /dev/null +++ b/ChangeLog @@ -0,0 +1,57 @@ +Changes in version 0.8 20100110 +- openmamba-dw-checklist: integrated use of distdb configuation from autodist +- openmamba-newrelease: add platforms installdvd and bootusb, dropped livestudio +- openmamba-repository: + * added list option + * added distromatic option to populate specs repository directory +- cron.daily/20-openmamba-autobuild-makedist: + * update to new targets + * support for creation of torrent files + * parametric support for languages +- cron.daily/30-openmamba-autobuild-cleanold: + * parametric support for languages + * support for torrent files +- cron.daily/50-openmamba-fix-permissions: added +- openmamba-autobuild-autodist cron moved from daily to hourly cron +- cron.hourly/10-openmamba-sync-back + cron.hourly/30-openmamba-distromatic + cron.hourly/40-openmamba-rsync: minor fixes +- cron.weekly/20-gencontents: generate contentslist inside nonflat dir +--------------------------------------------------------------------------------- + +Changes in version 0.7 +- update to current scripts release +- added script openmamba-newrelease +- frontend: added scripts 05-ntpd and 20-openmamba-developer-messages +--------------------------------------------------------------------------------- + +Changes in version 0.6 +- update to current scripts release +- added script and cron for dw-checklist support +--------------------------------------------------------------------------------- + +Changes in version 0.5 +- cron scripts: lots of updates +- added frontend cron scripts +--------------------------------------------------------------------------------- + +Changes in version 0.4 +- openmamba-import renamed to openmamba-repository with many new features +- cron scripts: lots of updates; added rsync daily cron +--------------------------------------------------------------------------------- + +Changes in version 0.3 +- openmamba-newrep: added script for new repository creation +- cron.weekly/10-openmamba-makedist: added cron script +- small fixes to cron files +--------------------------------------------------------------------------------- + +Changes in version 0.2 +- openmamba-import: added support for local source repositories +- cron files: many small fixes +--------------------------------------------------------------------------------- + +Changes in version 0.1 +- first packaging +--------------------------------------------------------------------------------- + diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..4a36069 --- /dev/null +++ b/Makefile @@ -0,0 +1,75 @@ +# Makefile for openmamba-headserver-tools +# Copyright (C) 2007-2008 by Silvan Calarco + +PACKAGE = openmamba-headserver-tools + +DISTROID = openmamba + +RPM_VENDOR = openmamba +RPM_DISTRIBUTION = openmamba +RPM_DISTROEXT = mamba + +SYSCONFIGFILE = openmamba-central + +include VERSION + +DISTROid = $(shell echo $(DISTROID) | tr A-Z a-z) +DISTROICON = $(DISTROid) + +DESTDIR = + +prefix = /usr +exec_prefix = ${prefix} +sysconfdir = /etc +sbindir = ${exec_prefix}/sbin +bindir = ${exec_prefix}/bin +datadir = ${prefix}/share +initrddir = ${sysconfdir}/rc.d/init.d +mandir = ${prefix}/usr/share/man +sysconfigdir= ${sysconfdir}/sysconfig + +INSTALL = /usr/bin/install +INSTALL_PROGRAM = ${INSTALL} -m 755 +INSTALL_DATA = ${INSTALL} -m 644 +INSTALL_DIR = ${INSTALL} -d -m 755 +INSTALL_SCRIPT = ${INSTALL_PROGRAM} + +distdir = $(PACKAGE)-$(VERSION) +dist_archive = $(distdir).tar.bz2 + +all: + +install-sysconfig: + @$(INSTALL_DIR) $(DESTDIR)$(sysconfigdir) $(DESTDIR)$(sysconfdir)/{cron.hourly,cron.daily,cron.weekly,sysconfig} + $(INSTALL_PROGRAM) etc/cron.hourly/* $(DESTDIR)$(sysconfdir)/cron.hourly/ + $(INSTALL_PROGRAM) etc/cron.daily/* $(DESTDIR)$(sysconfdir)/cron.daily/ + $(INSTALL_PROGRAM) etc/cron.weekly/* $(DESTDIR)$(sysconfdir)/cron.weekly/ + $(INSTALL_PROGRAM) etc-frontend/cron.hourly/* $(DESTDIR)$(sysconfdir)/cron.hourly/ + $(INSTALL_PROGRAM) etc-frontend/cron.weekly/* $(DESTDIR)$(sysconfdir)/cron.weekly/ + $(INSTALL_DATA) -m600 etc/sysconfig/* $(DESTDIR)$(sysconfdir)/sysconfig/ + +install-usertools: $(pck_usertools) + @$(INSTALL_DIR) $(DESTDIR)$(sbindir) + $(INSTALL_SCRIPT) bin/* $(DESTDIR)$(sbindir) + +install: install-sysconfig install-usertools + +dist: clean + @case `sed 15q ChangeLog` in \ + *"$(VERSION)"*) : ;; \ + *) \ + echo "ChangeLog not updated; not releasing" 1>&2;\ + exit 1;; \ + esac + @rm -f $(dist_archive);\ + tar cf - --exclude=$(dist_archive) -C .. $(distdir) | \ + bzip2 -9 -c > $(dist_archive) + @echo "file \`$(dist_archive)' created" + +dist-rpm: dist + @rpm_sourcedir=`rpm --eval=%{_sourcedir}`;\ + mv -f $(dist_archive) $$rpm_sourcedir;\ + echo "and saved as \`$$rpm_sourcedir/$(dist_archive)'" + +clean: + @rm -f $(dist_archive) diff --git a/VERSION b/VERSION new file mode 100644 index 0000000..7055233 --- /dev/null +++ b/VERSION @@ -0,0 +1 @@ +VERSION = 0.8 diff --git a/bin/openmamba-dw-checklist b/bin/openmamba-dw-checklist new file mode 100755 index 0000000..93bb88b --- /dev/null +++ b/bin/openmamba-dw-checklist @@ -0,0 +1,216 @@ +#!/bin/bash +# +# Distrowatch packages update checker +# Copyright (c) 2004-2009 by Silvan Calarco +# + +. /etc/sysconfig/openmamba-central +DISTROMATIC_PREFIX=/distribution/distromatic.html? +DISTROMATIC_REPOSITORY=devel +DISTDB=/etc/autodist/distdb +DISTDBDIR=/etc/autodist/distdb.d + +[ -r $DISTDB ] && { + . $DISTDB + [ -d $DISTDBDIR ] && \ + for f in `ls $DISTDBDIR/*.db`; do + . $f + done +} + +[ -r $PKGLIST_FILE ] || { + echo "Error: file $PKGLIST_FILE cannot be read; aborting." >&2 + exit 1 +} + +[ -r $ALIASES_DB ] || { + echo "Error: file $ALIASES_DB cannot be read; aborting." >&2 + exit 1 +} + +function usage() +{ + echo "openmamba-dw-checklist [-h|-m][-r repository]" + echo + echo " -h: generate distromatic HTML output" + echo " -m: show missing packages only" + echo " -r repository: specify the repository (default: devel)" + echo " -o repository: specify the repository for output data (default: devel-autodist)" + echo +} +echo "openmamba-dw-checklist - distrowatch packages update status file creator" >&2 + +get_job_vector() { + local JNAME=$1 + + # resolve JOB_NAME from distdb + # note: if JOB_NAME contains a "-" it can't be a distdb JOB, so skip it + if [ "${JNAME/-/}" = "${JNAME}" ]; then + local jobtmpfile=`tempfile` + # hack to get an array variable named as $j assigned to the JOB array + echo "echo \${$JNAME[*]}" > $jobtmpfile + JOB=(`. $jobtmpfile`) + rm -f $jobtmpfile + else + JOB=() + fi + + if [ ${#JOB[*]} -eq 0 ]; then + # create a default job with given JOB_NAME + JOB=($JNAME "" "") + elif [ ${#JOB[*]} -eq 1 ]; then + # no variables defined, add an empty job + JOB=(${JOB[*]} "" "") + fi + JOB_PKGS=(${JOB[0]//,/ }) + JOB_VARNAMES=(${JOB[1]//,/ }) +} + +function version_compare() +{ + local A B + A=$1 + B=$2 + + if [ ${A/[a-zA-Z_]*} -gt ${B/[a-zA-Z_]*} ]; then + return 1 + elif [ ${A/[a-zA-Z_]*} -lt ${B/[a-zA-Z_]*} ]; then + return 2 + else + if [[ "$A" > "$B" ]]; then + return 1 + elif [[ "$A" < "$B" ]]; then + return 2 + fi + fi + return 0 + +} + +function version_find_bigger() +{ + local VER1 VER2 FPOS CUTVER1 CUTVER2 + + VER1=$1 + VER2=$2 + FPOS=1 + while true; do + CUTVER1=`echo $VER1. | cut -d. -f $FPOS` + CUTVER2=`echo $VER2. | cut -d. -f $FPOS` + if [ "$CUTVER1" -a ! "$CUTVER2" ]; then + return 1 + elif [ "$CUTVER2" -a ! "$CUTVER1" ]; then + return 2 + elif [ ! "$CUTVER1" -a "$CUTVER2" ]; then + return 0 + else + version_compare $CUTVER1 $CUTVER2 + case $? in + 1) return 1 ;; + 2) return 2 ;; + esac + fi + FPOS=`expr $FPOS + 1` + done + return 0 +} + +while [ "$1" ]; do + case $1 in + -h) distromatic_html=1 ;; + -m) if [ "$distromatic_html" ]; then + echo "Error: options -h and -m cannot be used together." + usage + exit 1 + else + missing_only=1 + fi ;; + -r) [ "$2" ] || { + echo "Error: option -r requires repository name as parameter" + usage + exit 1 + } + DISTROMATIC_REPOSITORY=$2 + shift + ;; + *) echo "Error: invalid option $1."; usage; exit 1 ;; + esac + shift +done + +CACHE_DIR=${local_ftp}/$DISTROMATIC_REPOSITORY/autoupdate/ +PKGLIST_FILE=${local_ftp}/$DISTROMATIC_REPOSITORY/srcpkglist +BUILDLIST_FILE=$CACHE_DIR/distrowatch.in +CONFIG_DIR=$CACHE_DIR +DISTROWATCH_DB=$CACHE_DIR/distrowatch.db +ALIASES_DB=$CACHE_DIR/aliases +MANUALVER_DB=$CACHE_DIR/manualver + +> $DISTROWATCH_DB +> $DISTROWATCH_DB.missing +> $BUILDLIST_FILE + +lynx -width 300 -dump http://distrowatch.com/packages.php | +while read line; do + [ "`echo $line | grep "Package Version Note"`" ] && start_print=1 + [ "`echo $line | grep "____________________"`" ] && unset start_print + [ "$start_print" ] && { + set -- $line + pkg="${1/\[*\]/}" + ver="${2/\[*\]/}" + alias=`grep "^$pkg " $ALIASES_DB` + echo "$pkg $ver ${alias/* /}" >> $DISTROWATCH_DB + } +done + +while read pkg ver alias; do + unset pkgline + unset found_manual + unset found_alias + pkgline=`grep "^$pkg " $MANUALVER_DB` && found_manual=1 + if [ ! "$found_manual" ]; then + if [ "$alias" ]; then + get_job_vector $alias + lastjob=${#JOB_PKGS[*]} + pkgline=`grep "^${JOB_PKGS[$lastjob-1]} " $PKGLIST_FILE` && found_alias=1 + else + get_job_vector $pkg + lastjob=${#JOB_PKGS[*]} + pkgline=`grep "^${JOB_PKGS[$lastjob-1]} " $PKGLIST_FILE` + fi + fi + if [ "$pkgline" ]; then + set -- $pkgline + pkgname=$1 + pkgver=$2 + version_find_bigger $pkgver ${ver/-/.} + vercmp=$? + [ "$found_manual" ] && pkgname=$3 + if [ ! "$missing_only" ]; then + if [ "$distromatic_html" = "1" ]; then + [ "$found_alias" -o "$found_manual" ] && nameadd="$pkg" || unset nameadd + unset veradd + [ ${vercmp} = 2 ] && veradd="$ver" + [ ${vercmp} = 1 ] && veradd="$ver" +# [ "${ver/-/}" != "$pkgver" ] && veradd="$ver" || unset veradd + [ "$veradd" -o "$nameadd" ] && { + [ "$veradd" -a "$nameadd" ] && \ + verappend="($nameadd;$veradd)" || + verappend="(${nameadd}${veradd})" + } || unset verappend + echo "${JOB_PKGS[0]} $pkgver $verappend
" + [ $vercmp = 2 ] && { + if [ "$found_alias" ]; then + echo "$alias +$ver 0" >> $BUILDLIST_FILE + else + echo "$pkg +$ver 0" >> $BUILDLIST_FILE + fi + } + else + echo "$pkg $pkgver ($ver)" + fi + fi + else + echo "$pkg ($ver)" >> $DISTROWATCH_DB.missing + fi +done < $DISTROWATCH_DB diff --git a/bin/openmamba-newrelease b/bin/openmamba-newrelease new file mode 100755 index 0000000..3ad10fe --- /dev/null +++ b/bin/openmamba-newrelease @@ -0,0 +1,25 @@ +#!/bin/bash +echo "openmamba media release script" +echo "Copyright (c) 2008 by Silvan Calarco " +echo +[ "$1" ] || { + echo "Error: release not specified; aborting" + echo "Usage:" + echo "openmamba-newrelease [release]" + echo + echo "Example:" + echo "openmamba-newrelease 1.0pre5" + exit 1 +} + +sudo makedist openmamba-milestone1 PLATFORM=livecd FORCE=1 LANGUAGE=en +sudo makedist openmamba-milestone1 PLATFORM=livecd FORCE=1 LANGUAGE=it +sudo makedist openmamba-milestone1 PLATFORM=livecd SUBPLATFORM=livestudio-root FORCE=1 LANGUAGE=en +sudo makedist openmamba-milestone1 PLATFORM=livecd SUBPLATFORM=livestudio-root FORCE=1 LANGUAGE=it +sudo makedist openmamba-milestone1 PLATFORM=installdvd FORCE=1 LANGUAGE=en +sudo makedist openmamba-milestone1 PLATFORM=installdvd FORCE=1 LANGUAGE=it +sudo makedist openmamba-milestone1 PLATFORM=bootusb LANGUAGE=en +sudo makedist openmamba-milestone1 PLATFORM=bootusb LANGUAGE=it + +FORCE_OVERWRITE=1 RELEASE=$1 /etc/cron.daily/20-openmamba-autobuild-makedist +exit 0 diff --git a/bin/openmamba-newrep b/bin/openmamba-newrep new file mode 100755 index 0000000..94ceff6 --- /dev/null +++ b/bin/openmamba-newrep @@ -0,0 +1,176 @@ +#!/bin/bash +# +# openmamba new repository creation script +# Copyright (c) 2007-2008 by Silvan Calarco +# + +. /etc/sysconfig/openmamba-central + +CONFMODE=$1 +REPNAME=$2 +REPDIR=$local_ftp/$REPNAME +DISTROMATICDIR=$local_ftp/distromatic/$REPNAME +WEBDAV_DIR=/var/www/davdb + +[ "$CONFMODE" = "frontend" -o "$CONFMODE" = "backend" -a "$REPNAME" ] || { + echo "openmamba new repository creation tool" + echo "Copyright (c) 2007-2008 by Silvan Calarco" + echo + echo "Usage:" + echo "$0 {backend|frontend} name [-t type]" + echo + echo " backend this is the backend server" + echo " frontend this is the frontend server" + echo " name repository name (e.g. devel-silvan)" + echo " type the type of repository, default is 'people', choices are:" + echo " people group" + echo + exit 1 +} + +echo "openmamba new repository creation tool" +echo "Copyright (c) 2007-2008 by Silvan Calarco" +echo + +if [ "$CONFMODE" = "backend" ]; then + + for r in ${local_reps[*]}; do + [ "$r" = "$REPNAME" ] && REPTYPE=local + done + + for r in ${remote_reps[*]}; do + [ "$r" = "$REPNAME" ] && REPTYPE=remote + done + + [ "$REPTYPE" ] || { + echo "Error: please add repository to /etc/sysconfig/openmamba-central first; aborting." + exit 1 + } + + + [ -e $REPDIR ] && { + echo "Error: a file or directory called $REPDIR already exits; aborting." + exit 1 + } + + [ -e $DISTROMATICDIR ] && { + echo "Error: a file or directory called $DISTROMATICDIR already exits; aborting." + exit 1 + } + + mkdir -p $REPDIR/{base,SRPMS.base,home,old} || { + echo "Error: cannot create directories in $REPDIR; aborting." + exit 1 + } + + mkdir -p $REPDIR/nonflat/RPMS + ln -s ../SRPMS.base $REPDIR/nonflat/SRPMS + + for a in ${archs[*]}; do + ln -s ../../RPMS.$a $REPDIR/nonflat/RPMS/$a + mkdir -p $REPDIR/RPMS.$a || { + echo "Error: cannot create directories in $REPDIR; aborting." + exit 1 + } + done + + mkdir -p $DISTROMATICDIR || { + echo "Error: cannot create directory $DISTROMATICDIR; aborting." + exit 1 + } + + if [ "$REPTYPE" = "local" ]; then + useradd ftp$REPNAME -d $REPDIR -s /bin/true || { + echo "Error: cannot create ftp$REPNAME user; aborting." + exit 1 + } + chown -R ftp$REPNAME.ftp $REPDIR/{SRPMS.base,RPMS.*,nonflat,old} + fi + + cat > $REPDIR/.htaccess << _EOF +AuthType Basic +AuthUserFile /var/www/davdb/$REPUSER +AuthName "Authentication required" +_EOF + + echo "Repository has been created on the backend, now you need to do the following things:" + echo "+ configure distromatic for the $REPNAME repository" + echo "+ launch 'openmamba-newrep frontend' on the frontend server" + echo "+ add HTML code for the repository publication of the website" + echo + +elif [ "$CONFMODE" = "frontend" ]; then + REPUSER="${REPNAME/*-/}" + TEMPLATEDIR=$local_ftp/template + + [ "`getent passwd $REPUSER`" ] && { + echo "Error: user $REPUSER already exits; aborting." + exit 1 + } + + [ -e $REPDIR ] && { + echo "Error: a file or directory called $REPDIR already exits; aborting." + exit 1 + } + + useradd $REPUSER -d $REPDIR -s /bin/true || { + echo "Error: cannot create ftp$REPNAME user; aborting." + exit 1 + } + REPUSER_PASSWD=`mkpasswd -s 0 -l 9` + echo "$REPUSER_PASSWD" | passwd $REPUSER --stdin + + mkdir -p $REPDIR/{base,SRPMS.base,home,old} || { + echo "Error: cannot create directories in $REPDIR; aborting." + exit 1 + } + + cp -a $TEMPLATEDIR/home/* $REPDIR/home/ + mkdir -p $REPDIR/nonflat/RPMS + ln -s ../SRPMS.base $REPDIR/nonflat/SRPMS + + for a in ${archs[*]}; do + ln -s ../../RPMS.$a $REPDIR/nonflat/RPMS/$a + mkdir -p $REPDIR/RPMS.$a || { + echo "Error: cannot create directories in $REPDIR; aborting." + exit 1 + } + done + + chown -R $REPUSER.nobody $REPDIR/{SRPMS.base,RPMS.*,nonflat,old} + chmod -R 775 $REPDIR/{SRPMS.base,RPMS.*,nonflat,old} + ln -s $REPDIR/home/ /var/www/www.openmamba.org/people/$REPUSER + + htpasswd -nb $REPUSER $REPUSER_PASSWD > $WEBDAV_DIR/$REPUSER + + echo "Repository has been created on the frontend" + echo "* ftp address: ftpaccounts.openmamba.org" + echo "+ ftp user: $REPUSER" + echo "+ ftp password: $REPUSER_PASSWD" + echo + echo "You may configure autospec by copying the following lines at the" + echo "bottom of file .autospec in your home directory:" + echo + echo "#==================================================#" + echo "# devel-$REPUSER repository #" + echo "#==================================================#" + echo "ftp_rw_server[x]=\"ftp://ftpaccounts.openmamba.org\"" + echo "ftp_rw_port[x]=" + echo "ftp_rw_passive_mode[x]=on" + echo "ftp_rw_user[x]=$REPUSER" + echo "ftp_rw_passwd[x]=\"$REPUSER_PASSWD\"" + echo "ftp_rw_rpms_dir[x]=\"/RPMS.@arch@\"" + echo "ftp_rw_srpms_dir[x]=\"/SRPMS.base\"" + echo "arch_list[x]=\"${archs[*]}\"" + echo "arch_noarch_upload[x]=\"${arch_list[8]}\"" + echo "ftpdir_rw_old[x]=\"\"" + echo "ftpurl_ro_rpms[x]=\"ftp://ftpaccounts.openmamba.org/pub/openmamba/devel-$REPUSER/RPMS.@arch@\"" + echo "ftpurl_ro_srpms[x]=\"ftp://ftpaccounts.openmamba.org/pub/openmamba/devel-$REPUSER/SRPMS.base\"" + echo "#==================================================#" + echo + echo "Warning: you have to replace the occurencies of 'x' within brackets '[x]'" + echo " with the first free server number in your current autospec configuration." + echo +fi + +exit 0 diff --git a/bin/openmamba-repository b/bin/openmamba-repository new file mode 100755 index 0000000..eda44eb --- /dev/null +++ b/bin/openmamba-repository @@ -0,0 +1,712 @@ +#!/bin/bash +# +# openmamba inter-repository import script from remote ftp to local +# Copyright (c) 2007-2009 by Silvan Calarco +# +. /etc/sysconfig/openmamba-central + +me=${0##*/} + +function usage() { + + echo "openmamba inter-repository import script" + echo "Copyright (c) 2007-2008 by Silvan Calarco" + echo + echo "Usage:" + echo "$me list" + echo "$me archive REPOSITORY PKGS ..." + echo "$me query REPOSITORY PKGS ..." + echo "$me verify REPOSITORY [PKGS ...]" + echo "$me inspect REPOSITORY {PKGS ...} [-d REPOSITORY]" + echo "$me setwarning REPOSITORY {PKG ...} -t \"TEXT\"" + echo "$me diff REPOSITORY [PKGS ...] [-d REPOSITORY]" + echo "$me import REPOSITORY [PKGS ...] [-d REPOSITORY] [-s]" + echo "$me release REPOSITORY [PKGS ...] [-d REPOSITORY] [-s]" + echo "$me distromatic REPOSITORY" + echo + echo " -d use given repository as destination (default: devel)" + echo " -s simulate operations to see if it would work" + echo " -t warning text" +} + +# get_pkg_buildinfo - uses distromatic generated build file for +# getting information on the repository +# +# $1: repositoy name +# $2: pkg name +# $3: architecture +function get_pkg_buildinfo() { + + local pkg i + + [ $1 ] && rep=$1 || exit 1 + [ $2 ] && pkg=$2 + [ $3 ] && buildarch="$3" || buildarch="i586" + + DISTROMATIC_BUILD_FILE=$local_distromatic/$rep/builds-$buildarch + + [ -e $DISTROMATIC_BUILD_FILE ] || { + echo "Error: missing distromatic generated file $DISTROMATIC_BUILD_FILE; aborting." + exit 1; + } + + pkg_header=(); + pkg_builds=(); + pkg_obsoletes=(); + + . $DISTROMATIC_BUILD_FILE + + for i in ${pkg_list[*]}; do + if [ "$i" == "${pkg_header[0]}" ]; then + pkg_name=${pkg_header[0]}; + # Note: pkg_arch reported in builds file is just last arch source was + # built for, so we use repository arch instead + pkg_arch=${pkg_header[1]}; + [ "$pkg_arch" = "noarch" ] || pkg_arch=$buildarch + pkg_version=${pkg_header[2]}; + pkg_release=${pkg_header[3]}; + pkg_group=${pkg_header[4]}; + pkg_license=${pkg_header[5]}; + pkg_size=${pkg_header[6]}; + pkg_buildtime=${pkg_header[7]}; + pkg_altrep=${pkg_header[8]}; + return 0 + fi + done + + unset pkg_name pkg_arch pkg_version pkg_release \ + pkg_group pkg_license pkg_size pkg_buildtime pkg_altrep + + return 0 +} + +function import_file() { + [ $1 ] || exit 1 + + local f import_mode + f=$1 + import_mode=$2 + if [ "$import_mode" = "backup" ]; then + curl_delete_add="-Q '-DELE $f'" + else + curl_delete_add="" + fi + + [ "$simulate" != "1" ] && + if [ "$ORIG_MODE" = "remote" ]; then + if [ $ORIG_URL_LOCAL_ARCH -a ! -f $ORIG_URL_LOCAL_ARCH/$f ]; then + echo "Warning: package missing in local mirror; setting copy from remote repository." + fi + + if [ "$DEST_MODE" = "local" ]; then + # remote -> local + if [ $ORIG_URL_LOCAL_ARCH -a -f $ORIG_URL_LOCAL_ARCH/$f ]; then + echo -n "(L) " + # if file exists in a local mirror use it by preference + cp $ORIG_URL_LOCAL_ARCH/$f $DEST_URL_ARCH/ || { + echo "Error: cannot move file $ORIG_URL_LOCAL_ARCH/$f to $DEST_URL_ARCH/$f; aborting." + exit 1 + } + eval curl -s -u$ftpuser:$ftppass $ORIG_URL_ARCH $curl_delete_add >/dev/null && { + rm -f $ORIG_URL_LOCAL_ARCH/$f + } || { + echo "Warning: cannot delete remote file $ORIG_URL_ARCH/$f; you'll have to delete it." + echo curl -s -u$ftpuser:$ftppass $ORIG_URL_ARCH $curl_delete_add + } + else + echo -n "(R) " + curl -s -u$ftpuser:$ftppass \ + --get $ORIG_URL_ARCH/$f \ + -o $DEST_URL_ARCH/$f $curl_delete_add || { + echo "Error: cannot get file $ORIG_URL_ARCH/$f; aborting." + echo curl -s -u$ftpuser:$ftppass \ + --get $ORIG_URL_ARCH/$f \ + -o $DEST_URL_ARCH/$f $curl_delete_add + exit 1 + } + fi + else + # remote -> remote + echo "Error: remote to remote file import is not implemented yet; aborting." + exit 1 + fi + else + if [ "$DEST_MODE" = "local" ]; then + # local -> local + cp $ORIG_URL_ARCH/$f $DEST_URL_ARCH/ || { + echo "Error: cannot copy file $ORIG_URL_ARCH/$f to $DEST_URL_ARCH/$f; aborting." + exit 1 + } + if [ "$import_mode" = "backup" ]; then + rm -f $ORIG_URL_ARCH/$f || { + echo "Error: cannot remove file $ORIG_URL_ARCH/$f; aborting." + exit 1 + } + fi + else + # local -> remote + echo "Error: local to remote file import is not implemented yet; aborting." + exit 1 + fi + fi + +} + +function backup_local_file() { + [ $1 ] || return + + [ -e $LOCAL_BACKUP ] || mkdir -p $LOCAL_BACKUP + movefiles=$1 + + #`find $DEST_URL_ARCH -maxdepth 1 -regex ".*/${pkgname}-[^-]*-[^-]*"` + for m in $movefiles; do + echo "backing up $m" + if [ "$simulate" != "1" ]; then + mv $m $LOCAL_BACKUP/ || { + echo "Error: can't move $m to $LOCAL_BACKUP; aborting." + exit 1 + } + fi + done + +} + +function backup_package() { + local rep reg i + + [ "$1" ] || return + [ "$2" ] && rep=$2 || rep=$destrepository + [ "$3" ] && reg=$3 || reg=$DESTREGFILE + + archive_pkg=$1 + + get_pkg_buildinfo $rep $archive_pkg + + if [ "$pkg_name" ]; then + PKG_FILENAME="$pkg_name-$pkg_version-$pkg_release.src.rpm" + + [ -f $DEST_URL_LOCAL/SRPMS.base/$PKG_FILENAME ] || { + echo "Error: package $PKG_FILENAME does not exist in local repository; aborting." + exit 1 + } + + backup_local_file $DEST_URL_LOCAL/SRPMS.base/$PKG_FILENAME + if [ "$simulate" != "1" -a "$DEST_MODE" = "remote" ]; then + curl -s -u$ftpuser:$ftppass $DEST_URL/SRPMS.base/ -Q "-DELE $PKG_FILENAME" >/dev/null || { + echo "Warning: cannot delete remote file $DEST_URL/SRPMS.base/$PKG_FILENAME; you'll have to delete it." + } + fi + + for i in ${pkg_builds[*]}; do + PKG_FILENAME="$i-$pkg_version-$pkg_release.$pkg_arch.rpm" + [ -f $DEST_URL_LOCAL/RPMS.i586/$PKG_FILENAME ] || { + echo "Error: package $PKG_FILENAME does not exist in local repository; aborting." + exit 1 + } + + backup_local_file $DEST_URL_LOCAL/RPMS.i586/$PKG_FILENAME + if [ "$simulate" != "1" -a "$DEST_MODE" = "remote" ]; then + curl -s -u$ftpuser:$ftppass $DEST_URL/RPMS.i586/ -Q "-DELE $PKG_FILENAME" >/dev/null || { + echo "Warning: cannot delete remote file $DEST_URL/RPMS.i586/$PKG_FILENAME; you'll have to delete it." + } + fi + done + + # write register + [ "$simulate" != "1" ] && { + echo "`date +%Y%m%d%H%M` Package $pkg_name ($pkg_version-$pkg_release) archived" >> $reg +# | \ +# tee -a $ORIGREGFILE $DESTREGFILE >/dev/null +# echo "`date +%Y%m%d%H%M` \"\" \"package $pkg_name ($pkg_version-$pkg_release) archived from $rep\"" >> $reg + } + else + echo "Warning: package $archive_pkg does not exists in $rep; skipping." + fi +} + +function import_package() { + [ $1 ] || exit 1 + + local import_pkg import_mode + + import_pkg=$1 + import_mode=$2 + + # check release in dest repository + get_pkg_buildinfo $destrepository $import_pkg + + [ "$pkg_version" ] && { + destpkgname="$pkg_name" + destpkgversion="$pkg_version-$pkg_release" + destpkgarch="$pkg_arch" + } || destpkgversion="none" + + get_pkg_buildinfo $origrepository $import_pkg + + [ "$pkg_version" ] && { + origpkgname="$pkg_name" + origpkgversion="$pkg_version-$pkg_release" + origpkgarch="$pkg_arch" + } || origpkgversion="none" + + [ $origpkgname ] || { + echo "Error: package $import_pkg does not exist; aborting." + exit 1 + } + + DEST_URL_ARCH=$DEST_URL/SRPMS.base/ + ORIG_URL_ARCH=$ORIG_URL/SRPMS.base/ + ORIG_URL_LOCAL_ARCH=$ORIG_URL_LOCAL/SRPMS.base/ + + PKG_FILENAME="$origpkgname-$origpkgversion.src.rpm" + + [ "$ORIG_MODE" = "remote" ] && + ORIG_FILELIST=`curl -s -l -u$ftpuser:$ftppass --url $ORIG_URL_ARCH/` || + ORIG_FILELIST=`ls $ORIG_URL_ARCH` + + check_existence=0; + + for i in $ORIG_FILELIST; do + [ "$i" = "$PKG_FILENAME" ] && check_existence=1; + done + + if [ $check_existence = 1 ]; then + + [ "$destpkgversion" = "$origpkgversion" ] && { + echo "Warning: same version of $origpkgname exists in destination" + } + + echo -n "Import $PKG_FILENAME ($origpkgversion -> $destpkgversion) [y/N]?" + read ans + + [ "$ans" = "y" -o "$ans" = "Y" ] && { + + echo -n "Importing $PKG_FILENAME " + import_file $PKG_FILENAME $import_mode + + for i in ${pkg_builds[*]}; do + PKG_FILENAME="$i-$origpkgversion.$origpkgarch.rpm" + DEST_URL_ARCH=$DEST_URL/RPMS.i586/ + ORIG_URL_ARCH=$ORIG_URL/RPMS.i586/ + ORIG_URL_LOCAL_ARCH=$ORIG_URL_LOCAL/RPMS.i586/ + + echo -n "$PKG_FILENAME " + import_file $PKG_FILENAME $import_mode + done + echo + + # write register + [ "$simulate" != "1" ] && { + echo "`date +%Y%m%d%H%M` Package $import_pkg ($origpkgversion -> $destpkgversion) imported from $origrepository to $destrepository" | \ + tee -a $ORIGREGFILE $DESTREGFILE >/dev/null + } + + if [ "$import_mode" = "backup" ]; then + # backup stuff + [ "$destpkgversion" != "none" -a \ + "$destpkgversion" != "$origpkgversion" ] && { + backup_package $import_pkg $destrepository $DESTREGFILE + } + # remove distromatic extra files associated with this package + [ -e $local_distromatic/$rep/warnings/$import_pkg.in ] && { + rm -f $local_distromatic/$rep/warnings/$import_pkg.in || + echo "Warning: cannot remove file $local_distromatic/$rep/warnings/$import_pkg.in" + } + fi + + #for i in ${pkg_obsoletes}; do + # PKG_FILENAME="$i-$pkg_version-$pkg_release.$namearch.rpm" + # DEST_URL_ARCH=$DEST_URL/RPMS.i586/$PKG_FILENAME + # [ -e $DEST_URL_ARCH ] && echo "Warning: obsoleted package $i exists" + # backup_package $i $destrepository $DESTREGFILE + # #echo rm $DEST_URL_ARCH + #done + + } # ans = y + + else # check_existence != 1 + echo "Warning: $import_pkg reported by distromatic does no longer exist" + fi +} + +function extract_diffinfo() { + PKG=$1 + REP=$2 + TMP=$3 + local i + + get_pkg_buildinfo $REP $PKG + if [ "$pkg_name" ]; then + PKG_FILENAME="$local_ftp/$REP/SRPMS.base/$pkg_name-$pkg_version-$pkg_release.src.rpm" + [ -e "$PKG_FILENAME" ] || { + echo "Error: package $PKG_FILENAME missing in $origrepository; skipping" + return 1 + } + rpm -qp $PKG_FILENAME --requires > $TMP/buildrequires + + autospec -q -x $PKG_FILENAME -F \*.spec --destdir $TMP >/dev/null || { + echo "Error: could not extract specfile from $PKG_FILENAME; skipping package" + return 1 + } + [ -e "$TMP_SPEC_DIR/$pkg_name.spec" ] || { + SPEC_FOUND="`ls $TMP_SPEC_DIR/*.spec`" + mv $SPEC_FOUND $TMP_SPEC_DIR/$pkg_name.spec + echo "Warning: specfile name should be $pkg_name.spec instead of ${SPEC_FOUND/*\//} in $REP repository" + } + > $TMP/requires + > $TMP/provides + for i in ${pkg_builds[*]}; do + PKG_FILENAME="$local_ftp/$REP/RPMS.i586/$i-$pkg_version-$pkg_release.$pkg_arch.rpm" + [ -e "$PKG_FILENAME" ] || { + echo "Error: package $PKG_FILENAME missing in $origrepository; skipping" + return 1 + } + rpm -qp $PKG_FILENAME --requires >> $TMP/requires + rpm -qp $PKG_FILENAME --provides >> $TMP/provides + rpm -qlp $PKG_FILENAME >> $TMP/files + done + else + #echo "Warning: can't find package $PKG in $REP repository" + return 1 + fi + return 0 +} + +[ $1 ] || { usage; exit 1; } + +origrepository= +destrepository=devel +packages= +command= +simulate=0 + +while [ "$1" ]; do + case $1 in + -d) + destrepository=$2; shift ;; + -s) + simulate=1 ;; + -t) + shift + warningtext="$@" + break ;; + *) + if [ "$command" ]; then + case "$command" in + "import"|"release"|"query"|"verify"|"archive"|"diff"|"inspect"|"setwarning"|"distromatic") + [ "$origrepository" ] && + packages="$packages $1" || + origrepository=$1 + ;; + *) usage + echo "Error: invalid option $1; aborting." + exit 1 + ;; + esac + else + case "$1" in + "import"|"release"|"query"|"verify"|"archive"|"list"|"diff"|"inspect"|"setwarning"|"distromatic") command=$1 ;; + *) + usage + echo "Errror: $1 is not a valid command; aborting." + exit 1 + ;; + esac + fi + ;; + esac + shift +done + +[ "$command" = "" ] && { usage; exit 1; } + +[ "$command" = "list" ] && echo "Local repositories:" +for a in ${local_reps[*]}; do + [ "$a" = "$destrepository" ] && DEST_MODE=local; + [ "$a" = "$origrepository" ] && ORIG_MODE=local; + [ "$command" = "list" ] && echo "$a" +done +[ "$do_list" ] && echo "Remote repositories:" +for a in ${remote_reps[*]}; do + [ "$a" = "$destrepository" ] && DEST_MODE=remote; + [ "$a" = "$origrepository" ] && ORIG_MODE=remote; + [ "$command" = "list" ] && echo "$a" +done +[ "$command" = "list" ] && exit 0; + +[ "$origrepository" ] || { usage; exit 1; } + +[ "$DEST_MODE" ] || { echo "Error: $destrepository is not a valid repository; aborting."; exit 1; } +[ "$ORIG_MODE" ] || { echo "Error: $origrepository is not a valid repository; aborting."; exit 1; } +[ "$DEST_MODE" = "remote" ] && { echo "Waring: destination is a remote repository; this is an EXPERIMENTAL feature."; } + +[ "$command" = "query" ] && { + [ "$packages" ] || { usage; exit 1; } + for i in $packages; do + get_pkg_buildinfo $origrepository $i + if [ ! "$pkg_name" ]; then + echo "$i: package not found in $origrepository repository" + else + echo "Name: $pkg_name" + echo "BuildArch: $pkg_arch" + echo "Version: $pkg_version" + echo "Release: $pkg_release" + echo "Group: $pkg_group" + echo "License: $pkg_license" + echo "Size: $pkg_size" + echo "Builds: ${pkg_builds[*]}" + echo "Obsoletes: ${pkg_obsoletes[*]}" + echo + fi + + done + + exit 0; +} + +[ "$command" = "verify" ] && { + [ "$packages" ] || { + get_pkg_buildinfo $origrepository + packages=${pkg_list[*]} + } + for i in $packages; do + get_pkg_buildinfo $origrepository $i + if [ ! "$pkg_name" ]; then + echo "$i: package not found in $origrepository repository" + else + PKG_FILENAME="$i-$pkg_version-$pkg_release.src.rpm" + rpm2cpio $local_ftp/${origrepository}/SRPMS.base/$PKG_FILENAME &>/dev/null || { + echo "Warning: source package $PKG_FILENAME is empty or corrupted." + } + + for l in ${pkg_builds[*]}; do + PKG_FILENAME="$l-$pkg_version-$pkg_release.$pkg_arch.rpm" + rpm2cpio $local_ftp${origrepository}/RPMS.i586/$PKG_FILENAME &>/dev/null || { + echo "Warning: package $PKG_FILENAME is empty or corrupted." + } + done + fi + + done + + exit 0; +} + +[ "$command" = "diff" ] && { + [ "$packages" ] || { + get_pkg_buildinfo $origrepository + packages=${pkg_list[*]} + } + TMP_SPEC_DIR=`mktemp -d` + for i in $packages; do + echo + echo "*******************************************************************" + echo "$i package check:" + echo "*******************************************************************" + + extract_diffinfo $i $origrepository $TMP_SPEC_DIR || continue + [ -e $TMP_SPEC_DIR/$i.spec ] || { + echo "Error: could not extract specfile for $i in $origrepository repository; skipping" + continue + } + mv $TMP_SPEC_DIR/$i.spec $TMP_SPEC_DIR/$i.spec.origrep + mv $TMP_SPEC_DIR/files $TMP_SPEC_DIR/files.origrep + sort -u $TMP_SPEC_DIR/buildrequires > $TMP_SPEC_DIR/buildrequires.origrep + sort -u $TMP_SPEC_DIR/requires > $TMP_SPEC_DIR/requires.origrep + sort -u $TMP_SPEC_DIR/provides > $TMP_SPEC_DIR/provides.origrep + + extract_diffinfo $i $destrepository $TMP_SPEC_DIR || { + echo "Looks like a new package; inspecting data:" + echo "" + echo "SPECFILE:" + echo "=========" + cat $TMP_SPEC_DIR/$i.spec.origrep + echo "" + echo "REQUIRES:" + echo "=========" + cat $TMP_SPEC_DIR/requires.origrep + echo "" + echo "PROVIDES:" + echo "=========" + cat $TMP_SPEC_DIR/provides.origrep + echo "" + echo "FILES:" + echo "======" + cat $TMP_SPEC_DIR/files.origrep + continue + } + [ -e $TMP_SPEC_DIR/$i.spec ] || { + echo "Error: could not extract specfile for $i in $destrepository repository; skipping" + } + mv $TMP_SPEC_DIR/$i.spec $TMP_SPEC_DIR/$i.spec.destrep + mv $TMP_SPEC_DIR/files $TMP_SPEC_DIR/files.destrep + sort -u $TMP_SPEC_DIR/buildrequires > $TMP_SPEC_DIR/buildrequires.destrep + sort -u $TMP_SPEC_DIR/requires > $TMP_SPEC_DIR/requires.destrep + sort -u $TMP_SPEC_DIR/provides > $TMP_SPEC_DIR/provides.destrep + + echo "Showing differences between package version in $origrepository and $destrepository:" + echo "" + echo "SPECFILE:" + echo "=========" + diff -u $TMP_SPEC_DIR/$i.spec.destrep $TMP_SPEC_DIR/$i.spec.origrep + echo "" + echo "BUILDREQUIRES:" + echo "==============" + diff -u $TMP_SPEC_DIR/buildrequires.destrep $TMP_SPEC_DIR/buildrequires.origrep + echo "" + echo "REQUIRES:" + echo "=========" + diff -u $TMP_SPEC_DIR/requires.destrep $TMP_SPEC_DIR/requires.origrep + echo "" + echo "PROVIDES:" + echo "=========" + diff -u $TMP_SPEC_DIR/provides.destrep $TMP_SPEC_DIR/provides.origrep + echo "" + echo "FILES:" + echo "======" + diff -u $TMP_SPEC_DIR/files.destrep $TMP_SPEC_DIR/files.origrep + echo + done + rm -rf $TMP_SPEC_DIR + exit 0; +} + +[ "$command" = "setwarning" ] && { + [ "$packages" ] || { usage; exit 1; } + TMP_SPEC_DIR=`mktemp -d` + for i in $packages; do + extract_diffinfo $i $origrepository $TMP_SPEC_DIR + [ -e $TMP_SPEC_DIR/$i.spec ] || { + echo "Error: could not extract specfile for $i in $origrepository repository; aborting." + exit 1 + } + echo "$warningtext" > $local_ftp/distromatic/$origrepository/warnings/$i.in + done + [ "$TMP_SPEC_DIR" != "/" ] && rm -rf $TMP_SPEC_DIR + exit 0 +} + +[ "$command" = "inspect" ] && { + [ "$packages" ] || { usage; exit 1; } + TMP_SPEC_DIR=`mktemp -d` + for i in $packages; do + extract_diffinfo $i $origrepository $TMP_SPEC_DIR + [ -e $TMP_SPEC_DIR/$i.spec ] || { + echo "Error: could not extract specfile for $i in $origrepository repository; aborting." + exit 1 + } + echo "$i: details of package in $origrepository repository" + echo "" + echo "SPECFILE:" + echo "=========" + cat $TMP_SPEC_DIR/$i.spec + echo "" + echo "REQUIRES:" + echo "=========" + cat $TMP_SPEC_DIR/requires + echo "" + echo "PROVIDES:" + echo "=========" + cat $TMP_SPEC_DIR/provides + done + [ "$TMP_SPEC_DIR" != "/" ] && rm -rf $TMP_SPEC_DIR + exit 0; +} + +[ "$command" = "distromatic" ] && { + SRCPKGLIST="$local_ftp/$origrepository/srcpkglist" + [ -r $SRCPKGLIST ] || { + echo "Error: srcpkglist file missing for $origrepository repository; aborting." + exit 1 + } + [ -d $local_ftp/$origrepository/specs ] || mkdir $local_ftp/$origrepository/specs + [ -d $local_ftp/$origrepository/patches ] || mkdir $local_ftp/$origrepository/patches + + while read line; do + set -- $line + [ -e $local_ftp/$origrepository/SRPMS.base/$1-$2-$6.src.rpm ] && { + [ $local_ftp/$origrepository/SRPMS.base/$1-$2-$6.src.rpm -nt \ + $local_ftp/$origrepository/specs/$1.spec ] && { +# echo $local_ftp/$origrepository/SRPMS.base/$1-$2-$6.src.rpm + autospec -x $local_ftp/$origrepository/SRPMS.base/$1-$2-$6.src.rpm -F '*.spec' \ + --destdir $local_ftp/$origrepository/specs/ -q >/dev/null + touch $local_ftp/$origrepository/specs/$1.spec + grep -i "^Patch[0-9]*:" $local_ftp/$origrepository/specs/$1.spec &>/dev/null && { + autospec -x $local_ftp/$origrepository/SRPMS.base/$1-$2-$6.src.rpm -F '*.patch' \ + --destdir $local_ftp/$origrepository/patches/ -q >/dev/null + } + } + } + done < $SRCPKGLIST + exit 0 +} + +[ "$simulate" = "1" ] && echo "Simulation mode enabled." +# +# import and other active commands +# +DEST_URL=$local_ftp/$destrepository +DEST_URL_LOCAL=$local_ftp/$destrepository +LOCAL_BACKUP=$DEST_URL/old + +# operation files are always in the local copy of the repository +ORIGREGFILE=$local_ftp/$origrepository/operations.log.html +DESTREGFILE=$local_ftp/$destrepository/operations.log.html + +# ORIG_URL_LOCAL is set if a local copy of the repository exists +# and will be preferred for file transfer optimizations +if [ "$ORIG_MODE" = "remote" ]; then + ORIG_URL=$dest_ftp/pub/openmamba/$origrepository +else + ORIG_URL=$local_ftp/$origrepository +fi +ORIG_URL_LOCAL=$local_ftp/$origrepository + +[ "$command" = "archive" ] && { + DEST_URL=$ORIG_URL + DEST_URL_LOCAL=$ORIG_URL_LOCAL + DEST_MODE=$ORIG_MODE + LOCAL_BACKUP=$DEST_URL_LOCAL/old/archived + + [ "$packages" ] || { usage; exit 1; } + + for i in $packages; do + backup_package $i $origrepository $ORIGREGFILE + done + + exit 0 +} + +[ "$command" = "import" -o "$command" = "release" ] && { + LOCAL_BACKUP=$DEST_URL/old/import-$origrepository + + [ "$origrepository" = "$destrepository" ] && { + echo "Error: source and destination repository cannot be the same; aborting."; exit 1; } + + if [ "$command" = "import" ]; then + echo "Importing $1: $origrepository ($ORIG_MODE) => $destrepository ($DEST_MODE)" + backup_mode=backup + else + echo "Releasing $1: $origrepository ($ORIG_MODE) => $destrepository ($DEST_MODE)" + backup_mode=release + fi + + if [ ! "$packages" ]; then + get_pkg_buildinfo $origrepository + + for i in ${pkg_list[*]}; do + import_package $i $backup_mode + done + else + for i in ${packages[*]}; do + import_package $i $backup_mode + done + fi + + exit 0 + +} + +usage +echo "Error: $command is not a valid command; aborting." +exit 1 diff --git a/etc-frontend/cron.hourly/05-ntpd b/etc-frontend/cron.hourly/05-ntpd new file mode 100755 index 0000000..fd767af --- /dev/null +++ b/etc-frontend/cron.hourly/05-ntpd @@ -0,0 +1,2 @@ +#!/bin/bash +ntpdate -sbup8 `cat /etc/ntp/step-tickers` diff --git a/etc-frontend/cron.hourly/10-openmamba-downloads b/etc-frontend/cron.hourly/10-openmamba-downloads new file mode 100755 index 0000000..e4501e5 --- /dev/null +++ b/etc-frontend/cron.hourly/10-openmamba-downloads @@ -0,0 +1,77 @@ +#!/bin/bash +. /etc/sysconfig/openmamba-central + +SOURCEDIR=/var/makedist +SNAPSHOT_MEDIADIR=/var/ftp/pub/openmamba/devel/media +MILESTONE1_MEDIADIR=/var/ftp/pub/openmamba/milestone1/media +CACHEDIR=/var/cache/openmamba-downloads +DESTBASEURL=/pub/openmamba/devel/media +SNAPDATE=`date +%Y%m%d` +GREPFILE=$CACHEDIR/current.dat + + +# do_all_langs_downloads() +# +# Parameters; +# +# $1: target +# $2: suffix +function do_all_langs_downloads() { + local l f count count_curr + + tmpfile=`mktemp` + + echo "" >> $tmpfile + + mv $tmpfile $3/$1/downloads.inc + chmod 644 $3/$1/downloads.inc +} + +for f in /var/log/httpd/www.openmamba.org-access_log.*.gz; do + FILEDATE=`stat $f --printf=%Y` + [ -e $CACHEDIR/$FILEDATE.dat ] || { + echo "Creating new cache file from $f" + gunzip -c $f | \ + grep "GET /pub/openmamba/.*/media/.*/openmamba-.* 200 .*+$" > $CACHEDIR/$FILEDATE.dat + } +done + +#[ ! -e $CACHEDIR -o \ +# $CACHEDIR /var/log/httpd/www.openmamba.org-access_log.1.gz +#[ -e $CACHEDIR/ +# create a temp file with all successfull media downloads +grep "GET /pub/openmamba/.*/media/.*/openmamba-.* 200 .*+$" /var/log/httpd/www.openmamba.org-access_log > $GREPFILE + +do_all_langs_downloads livecd milestone1 $MILESTONE1_MEDIADIR milestone1 0 0 +#732 207 +do_all_langs_downloads livecd snapshot $SNAPSHOT_MEDIADIR devel 0 0 +# 732 207 +do_all_langs_downloads livestudio milestone1 $MILESTONE1_MEDIADIR milestone1 0 0 +# 97 10 +do_all_langs_downloads livestudio snapshot $SNAPSHOT_MEDIADIR devel 0 0 +# 69 49 +do_all_langs_downloads livegames snapshot $SNAPSHOT_MEDIADIR devel 0 0 +# 69 20 +do_all_langs_downloads bootusb milestone1 $MILESTONE1_MEDIADIR milestone1 0 0 +do_all_langs_downloads bootusb snapshot $SNAPSHOT_MEDIADIR devel 0 0 +do_all_langs_downloads flash-1GB snapshot $SNAPSHOT_MEDIADIR devel 0 0 +# 581 457 +do_all_langs_downloads bootcd snapshot $SNAPSHOT_MEDIADIR devel 0 0 +# 187 66 +do_all_langs_downloads installdvd milestone1 $MILESTONE1_MEDIADIR milestone1 0 0 + +#rm -f $GREPFILE diff --git a/etc-frontend/cron.hourly/20-openmamba-developer-messages b/etc-frontend/cron.hourly/20-openmamba-developer-messages new file mode 100755 index 0000000..1c0e3ee --- /dev/null +++ b/etc-frontend/cron.hourly/20-openmamba-developer-messages @@ -0,0 +1,28 @@ +#!/bin/bash +. /etc/sysconfig/openmamba-central + +TEMPFILE=`mktemp` +DEST_FILE=$local_ftp/devel/developers-status.inc +> $TEMPFILE + +for a in ${local_reps[*]} ${remote_reps[*]}; do + [ -e $local_ftp/$a/home/developer-status.inc ] && { + DATE_UNIX=`date -r $local_ftp/$a/home/developer-status.inc +%s` + DATE_HUMAN=`date -r $local_ftp/$a/home/developer-status.inc +%c` + TEXT=`cat $local_ftp/$a/home/developer-status.inc` + [ "$TEXT" ] || continue + TEXT=`echo $TEXT|sed "s|<[^<]*>||g"` + echo -n "${DATE_UNIX} " >> $TEMPFILE + echo -n "${a/*-} ($a)" >> $TEMPFILE + echo -n "$TEXT ($DATE_HUMAN)" >> $TEMPFILE + echo "" >> $TEMPFILE + } +done +TEMPFILE2=`mktemp` +echo "" > $TEMPFILE2 +cat $TEMPFILE | sort -nrk1 | sed "s|^[0-9]* ||" >> $TEMPFILE2 +echo "
" >> $TEMPFILE2 +mv $TEMPFILE2 $DEST_FILE +chmod 644 $DEST_FILE +rm -f $TEMPFILE +exit 0 diff --git a/etc-frontend/cron.weekly/10-openmamba-backup b/etc-frontend/cron.weekly/10-openmamba-backup new file mode 100755 index 0000000..6aabd5c --- /dev/null +++ b/etc-frontend/cron.weekly/10-openmamba-backup @@ -0,0 +1,23 @@ +#!/bin/bash +SNAPDATE=`date +%Y%m%d` +TMPDIR=`mktemp -d` +MYSQL_USER= +MYSQL_PASSWORD= +echo "Creating www sites backup archives..." +tar cjf $TMPDIR/openmamba-www-backup-$SNAPDATE.tar.bz2 /var/www/www.openmamba.org/ +tar cjf $TMPDIR/openmamba-wiki-backup-$SNAPDATE.tar.bz2 /var/www/wiki.openmamba.org/ +tar cjf $TMPDIR/openmamba-forum-backup-$SNAPDATE.tar.bz2 /var/www/forum.openmamba.org/ +tar cjf $TMPDIR/openmamba-bugs-backup-$SNAPDATE.tar.bz2 /var/www/bugs.openmamba.org/ + +echo "Creating openmamba database dump..." +mysqldump -u $MYSQL_USER --password="$MYSQL_PASSWORD" openmamba > $TMPDIR/openmamba-mysql.dump + +echo "Creating /etc backup..." +tar cjf $TMPDIR/openmamba-etc-backup-$SNAPDATE.tar.bz2 /etc + +echo "Creating final global archive..." +cd $TMPDIR +tar cjvf /root/openmamba-server-backup-$SNAPDATE.tar.bz2 * || exit 1 +cd .. + +[ "$TMPDIR" ] && rm -rf $TMPDIR diff --git a/etc/cron.daily/10-openmamba-mirror-daily b/etc/cron.daily/10-openmamba-mirror-daily new file mode 100755 index 0000000..feaa724 --- /dev/null +++ b/etc/cron.daily/10-openmamba-mirror-daily @@ -0,0 +1,43 @@ +#!/bin/bash +# +# openmamba daily sync script +# (c) 2006-2008 by Silvan Calarco +# +. /etc/sysconfig/openmamba-central +RSYNC_OPT="-av --no-o --delete --delete-after" + +echo "Creating and syncing pkglist" +for a in ${local_reps[*]}; do + distromatic -q -t $a --genpkglist > $local_ftp/$a/pkglist + [ "$ftpsync" ] && { + curl -s -T $local_ftp/$a/pkglist -u$ftpuser:$ftppass --url $dest_ftp/pub/openmamba/$a/ + } +done + +if [ "$ftpsync" ]; then + echo "Syncing openmamba devel utils dir" + ftpsync -q -p $local_ftp/devel/utils \ + $dest_ftp/pub/openmamba/devel/utils ftpuser=$ftpuser ftppasswd=$ftppass +fi + +echo "Syncing openmamba db dir" +if [ "$ftpsync" ]; then + ftpsync -q -p $local_ftp/db \ + $dest_ftp/pub/openmamba/db ftpuser=$ftpuser ftppasswd=$ftppass +else + rsync \ + $RSYNC_OPT \ + $local_ftp/db $dest_rsync \ + --password-file=/etc/rsync.password --bwlimit=400 +fi + +echo "Syncing openmamba lists" +if [ "$ftpsync" ]; then + ftpsync -q -p $local_ftp/lists \ + $dest_ftp/pub/openmamba/lists ftpuser=$ftpuser ftppasswd=$ftppass +else + rsync \ + $RSYNC_OPT \ + $local_ftp/lists $dest_rsync \ + --password-file=/etc/rsync.password --bwlimit=400 +fi diff --git a/etc/cron.daily/20-openmamba-autobuild-makedist b/etc/cron.daily/20-openmamba-autobuild-makedist new file mode 100755 index 0000000..d6baa2c --- /dev/null +++ b/etc/cron.daily/20-openmamba-autobuild-makedist @@ -0,0 +1,265 @@ +#!/bin/bash +# +# Makedist automatic build script +# Copyright (c) 2007-2009 by Silvan Calarco +# +# Usage: +# +# To overwrite already existing images +# FORCE_OVERWRITE=1 ./20-openmamba-makedist +# +# To commit a new release (relies on snapshot images): +# RELEASE=1.0pre3 ./20-openmamba-makedist +# +. /etc/sysconfig/makedist +. /usr/share/makedist/functions.inc.sh + +DISTRONAME=openmamba +MAKEDIST_TARGET=openmamba +arch=i586 +SOURCEDIR=/var/makedist + +[ "$TODAYDATE" ] || TODAYDATE=`date +%Y%m%d` +[ "$WEEKDAY" ] || WEEKDAY=`date +%u` + +# DAY: 1=monday ... 7=sunday +if [ "$RELEASE" ]; then + DISTVERSION=milestone1 + DESTBASEDIR=/var/ftp/pub/openmamba/$DISTVERSION/media + ALLMEDIAS=(livecd/livecd-root livecd/livestudio-root installdvd/livecd-root bootusb) + ALLMEDIASDAY=(8 8 8 8) +# DOWNLOADSFILEPREFIX=$DESTBASEDIR/downloads + DESTLATESTFILENAME=latest-$RELEASE.inc.js + SOURCEIMGFILEPREFIX="openmamba-$DISTVERSION" +else + DISTVERSION=devel + DESTBASEDIR=/var/ftp/pub/openmamba/$DISTVERSION/media + ALLMEDIAS=(livecd/livecd-root livedvd/livedvd-root livecd/livegames-root livecd/testcd-root flash-1GB bootcd bootusb) + ALLMEDIASDAY=(5 2 6 4 7 7 5) +# DOWNLOADSFILEPREFIX=$DESTBASEDIR/downloads.inc + DESTLATESTFILENAME=latest.inc.js + SOURCEIMGFILEPREFIX="openmamba" +fi +DESTIMGFILEPREFIX="openmamba" +ALLMEDIASLAST=`expr ${#ALLMEDIAS[*]} - 1` +DESTBASEURL=/pub/openmamba/$DISTVERSION/media + +# do_makedist() +# +# Parameters; +# +# $1: dist filename +# $2: dest filename +# $3: language +# $4: platform +# $5: subplatform +# $6 product_name +function do_makedist() { + local DIST_CREATED=0 + SOURCEIMG=$SOURCEDIR/$1 + DESTNAME=$2 + DESTDIR=$DESTBASEDIR/$6/$3 + DESTIMG=$DESTDIR/$DESTNAME + DESTMD5=$DESTDIR/$DESTNAME.MD5.txt + DESTTORRENT=$DESTDIR/$DESTNAME.torrent + [ "$RELEASE" ] && \ + DESTLATEST=$DESTDIR/latest-is-$RELEASETAG.txt || \ + DESTLATEST=$DESTDIR/latest-is-$SNAPDATE.txt + + if [ ! -f $DESTIMG -o "$FORCE_OVERWRITE" = "1" ]; then + [ ! "$SIMULATE" ] && { + makedist openmamba PLATFORM=$4 SUBPLATFORM=$5 FORCE=1 LANGUAGE=$3 || { + echo "Error building openmamba $6 platform; aborting." + exit 1 + } + DIST_CREATED=1 + } + if [ "$RELEASE" ]; then + [ -e $DESTDIR ] || mkdir -p $DESTDIR + cp $SOURCEIMG $DESTIMG + cp $SOURCEDIR/.$1.inc.html $DESTDIR/buildinfo-$RELEASE.inc.html + [ "$RELEASE/pre" != "$RELEASE" ] && ln -sf buildinfo-$RELEASE.inc.html $DESTDIR/buildinfo-pre.inc.html + cd $DESTDIR + md5sum $DESTNAME > $DESTMD5 + maketorrent-console --title "openmamba $RELEASE $6 $3" --tracker_name "mambaSoft BitTorrent tracker" \ + http://www.mambasoft.it:8080/announce $DESTNAME --target $DESTTORRENT + rm -f $DESTDIR/latest-is-*.txt + > $DESTLATEST + else + [ -e $DESTDIR ] || mkdir -p $DESTDIR + cp $SOURCEIMG $DESTIMG + cp $SOURCEDIR/.$1.inc.html $DESTDIR/buildinfo.inc.html + cd $DESTDIR + md5sum $DESTNAME > $DESTMD5 + maketorrent-console \ + --comment "openmamba snapshot $SNAPDATE $6 $3 from http://www.openmamba.org" \ + --tracker_name "mambaSoft BitTorrent tracker" \ + --language $3 \ + http://www.mambasoft.it:8080/announce \ + $DESTNAME --target $DESTTORRENT + rm -f $DESTDIR/latest-is-*.txt + > $DESTLATEST + fi + fi + DESTSIZE=`du $DESTIMG -h | gawk '{ print $1 }'` + DESTMD5=`cat $DESTMD5 | gawk '{ print $1 }'` + + cat > $DESTDIR/$DESTLATESTFILENAME << _EOF +nameArray['$6-$RELEASETAG-$3']='$DESTNAME'; +pathArray['$6-$RELEASETAG-$3']='/download.html?file=$DESTBASEURL/$6/$3/'; +sizeArray['$6-$RELEASETAG-$3']='$DESTSIZE'; +md5Array['$6-$RELEASETAG-$3']='$DESTMD5'; +_EOF + + return $DIST_CREATED +} + +# do_all_langs_makedist() +# +# Parameters; +# +# $1: dist filename +# $2: dest filename +# $3: platform +# $4: subplatform +# $5: html output language +# $6: product name +function do_makedist_and_html() { + + local DIST_LINE DIST_CREATED DIST_DWD + + local tempfile2=`tempfile 2>/dev/null` || tempfile=/tmp/makedist_tmpfile2 + + PRODNAME=$6 + [ $PRODNAME ] || PRODNAME=$PLATFORM + + cat > $tempfile2 << _EOF +
+
+_EOF + + case $5 in + it) echo -n "Lingua: " >> $tempfile2 ;; + *) echo -n "Language: " >> $tempfile2 ;; + esac + + for l in $LANGS; do + [ "$l" = "$5" ] && { + checked="checked" + } || checked="" + case $l in + it) langname="italiano" ;; + es) langname="español" ;; + en) langname="english" ;; + *) langname="unknown" ;; + esac + echo "$langname" >> $tempfile2 + done + + cat >> $tempfile2 << _EOF +
+ +_EOF + + if [ "$RELEASE" ]; then + DESTHTML=$DESTBASEDIR/$PRODNAME/index.inc.$5.$RELEASE.html + ln -sf index.inc.$5.$RELEASE.html $DESTBASEDIR/$PRODNAME/index.inc.$5.pre.html + else + DESTHTML=$DESTBASEDIR/$PRODNAME/index.inc.$5.html + fi + mv $tempfile2 $DESTHTML + chmod +r $DESTHTML +} + +LANGUAGE=_SPLIT_ + +tempfile=`tempfile 2>/dev/null` || tempfile=/tmp/makedist_tmpfile + +for l in $LANGS; do + echo "" > $tempfile.$l +done + +for i in `seq 0 $ALLMEDIASLAST`; do + + PLATFORM=${ALLMEDIAS[i]/\/*/} + SUBPLATFORM=${ALLMEDIAS[i]/*\//} + unset PRODUCT_NAME + unset PRODUCT_NAME_OVERRIDE + + [ $SUBPLATFORM ] && . /var/makedist/targets/$MAKEDIST_TARGET/platforms/$SUBPLATFORM/settings.inc + . /var/makedist/targets/$MAKEDIST_TARGET/platforms/$PLATFORM/settings.inc + [ "$SUBPLATFORM" ] && \ + media_name=`PLATFORM=$SUBPLATFORM media_name` || + media_name=`media_name` + media_suffix=${media_name/*_SPLIT_\./} + + [ $PRODUCT_NAME ] || PRODUCT_NAME=$PLATFORM + [ "$PRODUCT_NAME_OVERRIDE" ] && PRODUCT_NAME=$PRODUCT_NAME_OVERRIDE + + #echo "PLATFORM=$PLATFORM SUBPLATFORM=$SUBPLATFORM MEDIA_SUFFIX=$media_suffix" + + if [ "$RELEASE" ]; then + echo "Releasing $PRODUCT_NAME media (${PLATFORM}/${SUBPLATFORM})" + SIMULATE=1 + elif [ "$WEEKDAY" = "${ALLMEDIASDAY[i]}" -o \ + "$FORCEBUILD" = "$PRODUCT_NAME" ]; then + echo "Building $PRODUCT_NAME media" + unset SIMULATE + else + SIMULATE=1 + fi + + if [ -e $DESTBASEDIR/$PRODUCT_NAME/en ]; then + unset latest + latest=`ls $DESTBASEDIR/$PRODUCT_NAME/en/latest-is-*.txt 2>/dev/null| sed "s/.*latest-is-\(.*\)\.txt/\1/"` + [ "$latest" -a "$SIMULATE" ] && SNAPDATE=$latest || SNAPDATE=$TODAYDATE + [ "$RELEASE" ] && RELEASETAG=milestone1-$RELEASE || RELEASETAG=snapshot-$SNAPDATE + + for l in $LANGS; do + # build specific language version + [ "$latest" -o ! "$SIMULATE" -o "$RELEASE" ] && + do_makedist_and_html \ + $media_suffix $RELEASETAG.$media_suffix \ + $PLATFORM "$SUBPLATFORM" $l $PRODUCT_NAME + done + +# # build english version +# [ "$latest" -o ! "$SIMULATE" -o "$RELEASE" ] && +# do_makedist_and_html \ +# $media_suffix $RELEASETAG.$media_suffix \ +# $PLATFORM "$SUBPLATFORM" en $PRODUCT_NAME + else + echo "Warning: missing directory $DESTBASEDIR/$PRODUCT_NAME/en; skipping" + fi +done + +for l in $LANGS; do + echo "
" >> $tempfile.$l + if [ "$RELEASE" ]; then + cat $tempfile.$l > $DESTBASEDIR/downloads-$RELEASE.inc.$l + [ "${RELEASE/pre}" != "$RELEASE" ] && ln -sf downloads-$RELEASE.inc.$l $DESTBASEDIR/downloads-pre.inc.$l + else + cat $tempfile.$l > $DESTBASEDIR/downloads.inc.$l + fi + rm -f $tempfile.$l +done + +exit 0 diff --git a/etc/cron.daily/30-openmamba-autobuild-cleanold b/etc/cron.daily/30-openmamba-autobuild-cleanold new file mode 100755 index 0000000..a202d32 --- /dev/null +++ b/etc/cron.daily/30-openmamba-autobuild-cleanold @@ -0,0 +1,46 @@ +#!/bin/bash +# +# Makedist automatic build script +# Copyright (c) 2007-2009 by Silvan Calarco +# +. /etc/sysconfig/makedist +SOURCEDIR=/var/makedist +DESTBASEDIR=/var/ftp/pub/openmamba/devel/media +DESTBASEURL=/pub/openmamba/devel/media +[ "$TODAYDATE" ] || TODAYDATE=`date +%Y%m%d` +GLOBDOWNLOADSFILE=$DESTBASEDIR/downloads.inc +ARCH=i586 + +tmpfile=`mktemp` + +[ $tmpfile ] || { + echo "Error: cannot create temporary files; aborting." + exit 1 +} + +for l in $LANGS; do + for d in `find $DESTBASEDIR/*/$l -maxdepth 0 -type d`; do + > $tmpfile + for f in `find $d -maxdepth 1 -name \*.iso -o -name \*.cpio.gz`; do + echo `stat -c %Z $f` `basename $f` >> $tmpfile + done + + filecount=0 + for f in `cat $tmpfile | sort -r | awk '{ print $2 }' `; do + if [ $filecount = 0 ]; then + [ "$VERBOSE" -o "$SIMULATE" ] && echo "KEEP $f" + else + [ "$VERBOSE" -o "$SIMULATE" ] && echo "MOVE $f" + [ "$SIMULATE" ] && continue + [ -e $d/old ] || mkdir -p $d/old + mv $d/$f $d/old/ + [ -e $d/$f.MD5.txt ] && mv $d/$f.MD5.txt $d/old/ + [ -e $d/$f.torrent ] && mv $d/$f.torrent $d/old/ + fi + filecount=`expr $filecount + 1` + done +# echo +# ls --sort=time $d/*.iso $d/*.cpio.gz + #find $DESTBASEDIR -name *.iso + done +done diff --git a/etc/cron.daily/40-openmamba-dw-checklist b/etc/cron.daily/40-openmamba-dw-checklist new file mode 100755 index 0000000..15fd4d4 --- /dev/null +++ b/etc/cron.daily/40-openmamba-dw-checklist @@ -0,0 +1,11 @@ +#!/bin/bash +# +# openmamba-dw-checklist daily cron script +# (c) 2008-2009 by Silvan Calarco +# +. /etc/sysconfig/openmamba-central +REPOSITORY=devel-kde4 +openmamba-dw-checklist -h -r $REPOSITORY > $local_ftp/distromatic/$REPOSITORY/_popular.html + +REPOSITORY=milestone1-1.1 +openmamba-dw-checklist -h -r $REPOSITORY > $local_ftp/distromatic/$REPOSITORY/_popular.html diff --git a/etc/cron.daily/50-openmamba-fix-permissions b/etc/cron.daily/50-openmamba-fix-permissions new file mode 100755 index 0000000..593b742 --- /dev/null +++ b/etc/cron.daily/50-openmamba-fix-permissions @@ -0,0 +1,19 @@ +#!/bin/bash +# +# Fixes permissions on repositories +# Copyright (c) 2009-2010 by Silvan Calarco +# +. /usr/share/makedist/functions.inc.sh +. /etc/sysconfig/openmamba-central + +for rep in ${local_reps[*]}; do + getent passwd ftp$rep >/dev/null || { + echo "Warning: missing user ftp$rep for $rep repository; skipping." + continue + } + find /var/ftp/pub/openmamba/$rep/RPMS.i586 /var/ftp/pub/openmamba/$rep/SRPMS.base \ + -uid 0 -exec chown ftp$rep:users {} \; + # >/dev/null +done + +exit 0 diff --git a/etc/cron.hourly/10-openmamba-sync-back b/etc/cron.hourly/10-openmamba-sync-back new file mode 100755 index 0000000..c1eee6a --- /dev/null +++ b/etc/cron.hourly/10-openmamba-sync-back @@ -0,0 +1,52 @@ +#!/bin/bash +# +# openmamba repository sync-back script +# (c) 2006-2008 by Silvan Calarco +# +. /etc/sysconfig/openmamba-central + +[ "$dest_rsync" ] && { + RSYNC_OPT="-av --no-o --delete --delete-excluded" + + [ "`pidof rsync`" ] && { + echo "Error: an rsync process is already running; exiting for safety." + exit 1 + } + + for rep in ${remote_reps[*]}; do + echo "==> Syncing back $rep remote repository" + for a in ${archs[*]}; do + rsync \ + $RSYNC_OPT \ + $dest_rsync/$rep/RPMS.$a $local_ftp/$rep \ + --password-file=/etc/rsync.password --bwlimit=500 \ + --exclude */old > /dev/null + done + rsync \ + $RSYNC_OPT \ + $dest_rsync/$rep/SRPMS.base $local_ftp/$rep \ + --password-file=/etc/rsync.password --bwlimit=500 \ + --exclude */old >/dev/null + rsync \ + $RSYNC_OPT \ + $dest_rsync/$rep/home $local_ftp/$rep \ + --password-file=/etc/rsync.password --bwlimit=500 \ + --exclude */old > /dev/null +# rsync \ +# $RSYNC_OPT \ +# $dest_rsync/$rep/autoupdate $local_ftp/$rep \ +# --password-file=/etc/rsync.password --bwlimit=500 \ +# --exclude */old > /dev/null + done + exit 0 +} + +[ "$ftpsync" -a "$dest_ftp" ] && { + for a in ${remote_reps[*]}; do + echo "==> Syncing back remote $a repository" + ftpsync -q -g $local_ftp/$a/RPMS.i586 \ + $dest_ftp/pub/openmamba/$a/RPMS.i586 ftpuser=$ftpuser ftppasswd=$ftppass + ftpsync -q -g $local_ftp/$a/SRPMS.base \ + $dest_ftp/pub/openmamba/$a/SRPMS.base ftpuser=$ftpuser ftppasswd=$ftppass + done +} \ No newline at end of file diff --git a/etc/cron.hourly/20-openmamba-apt b/etc/cron.hourly/20-openmamba-apt new file mode 100755 index 0000000..94f834d --- /dev/null +++ b/etc/cron.hourly/20-openmamba-apt @@ -0,0 +1,17 @@ +#!/bin/sh +# +# openmamba repository script for apt +# (c) 2006-2007 by Silvan Calarco +# + +# generate apt database +. /etc/sysconfig/openmamba-central + +for a in ${local_reps[*]} ${remote_reps[*]}; do + [ "`find $local_ftp/$a/SRPMS.base/ -cnewer $local_ftp/$a/base`" -o \ + ! -e $local_ftp/$a/base/release ] && { + echo "==> Generating APT database for $a repository" + genbasedir --flat $local_ftp/$a/ ${archs[*]} base >/dev/null + } +done +exit 0 diff --git a/etc/cron.hourly/30-openmamba-distromatic b/etc/cron.hourly/30-openmamba-distromatic new file mode 100755 index 0000000..e7b998c --- /dev/null +++ b/etc/cron.hourly/30-openmamba-distromatic @@ -0,0 +1,74 @@ +#!/bin/sh +# +. /etc/sysconfig/openmamba-central + +function distromatic_genhtml() +{ + REP=$1 + shift + TMPDIR=/tmp/distromatic-tmp-$REP + TMPFILE=`mktemp` + LOGFILE=$local_ftp/$REP/distromatic.log + [ -e $TMPDIR ] && rm -rf $TMPDIR + mkdir -p $TMPDIR + + [ ! $local_ftp/$REP/SRPMS.base -nt $local_ftp/$REP/distromatic.log 2>/dev/null -a \ + ! $local_ftp/$REP/RPMS.i586 -nt $local_ftp/$REP/distromatic.log 2>/dev/null -a \ + ! $local_distromatic/$REP/warnings -nt $local_ftp/$REP/distromatic.log 2>/dev/null -a \ + -e $local_distromatic/$REP/deps-i586 -a ! "$FORCE" ] && return + + echo "==> Executing distromatic on $REP repository" + (cd $local_distromatic/$REP/ + +# for a in *; do +# cp -a $a $TMPDIR/ +# done + echo "\ +# distromatic log for the $REP repository (`date`) +# ==================================================================== +" > $LOGFILE + distromatic -q -t $REP --genhtml --gendatatables --gensrcpkglist 1>$local_ftp/$REP/srcpkglist 2>$TMPFILE + cat $TMPFILE | grep -v "Warning: skipping unhandled requirement" >> $LOGFILE + + /usr/sbin/openmamba-repository distromatic $REP + + if [ "${REP:0:6}" = "devel-" ]; then + + echo "==> Checking packages integrity" + echo "\ + +# ============================================================================ +# INTEGRITY VERIFICATION for the $REP openmamba repository (`date`) +# ============================================================================ +" >> $LOGFILE + /usr/sbin/openmamba-repository verify $REP >> $LOGFILE + + echo "==> Checking package contents" + echo "\ + +# ============================================================================ +# PACKAGES CHECK for the $REP openmamba repository (`date`) +# ============================================================================ +" >> $LOGFILE + /usr/sbin/openmamba-repository diff $REP >> $LOGFILE + fi + +# for a in *; do# +# # set date t#o past for files which are identical +# diff $a $TMPDIR/$a &>/dev/null || touch -d "Jan 01 2007 00:00" $a +# done + ) +# rm -rf $TMPDIR +} + +JOB=$1 + +if [ "$JOB" ]; then + FORCE=1 + distromatic_genhtml $JOB +else + for a in ${local_reps[*]} ${remote_reps[*]}; do + distromatic_genhtml $a + done +fi +exit 0 diff --git a/etc/cron.hourly/40-openmamba-rsync b/etc/cron.hourly/40-openmamba-rsync new file mode 100755 index 0000000..e7923cc --- /dev/null +++ b/etc/cron.hourly/40-openmamba-rsync @@ -0,0 +1,78 @@ +#!/bin/bash +# +# openmamba rsync script +# +. /etc/sysconfig/openmamba-central + +#BWLIMIT="--bwlimit=400" +TIMEOUT="--timeout=120" +[ "$1" ] && JOB=$1 || unset JOB + +[ "$dest_rsync" ] || exit 0 + +[ "`pidof rsync`" -a ! "$FORCE" -a ! "$JOB" ] && { + echo "Error: an rsync process is already running; exiting for safety." + exit 1 +} + +# use ssh method if dest_ssh is defined +[ "$dest_ssh" ] && dest_rsync=$dest_ssh + +RSYNC_OPT="-av -P --no-o --delete-after \ + $BWLIMIT $TIMEOUT" + +[ "$PROGRESS" = "1" ] && RSYNC_OPT="$RSYNC_OPT --progress" +#echo rsync $RSYNC_OPT /var/ftp/pub/openmamba/ $RSYNC_DEST --password-file=/etc/rsync.password + +if [ ! "$JOB" -o "$JOB" = "distromatic" ]; then + echo "==> Syncing distromatic directory" + eval rsync \ + $RSYNC_OPT \ + $local_ftp/distromatic $dest_rsync \ + --exclude distromatic/devel* \ + --exclude distromatic/milestone* +fi + +for rep in ${local_reps[*]}; do + if [ ! "$JOB" -o "$JOB" = "$rep" ]; then + echo "==> Syncing $rep local repository" + eval rsync \ + $RSYNC_OPT --exclude base --exclude media --exclude "*/developers-status.inc" \ + $local_ftp/$rep $dest_rsync \ + --exclude "*/old" + if [ $? -eq 0 -a -d $local_ftp/$rep/base ]; then + echo "==> Syncing apt for $rep local repository" + eval rsync \ + $RSYNC_OPT $local_ftp/$rep/base $dest_rsync/$rep + echo "==> Syncing distromatic directory for $rep local repository" + eval rsync \ + $RSYNC_OPT \ + $local_ftp/distromatic/$rep $dest_rsync/distromatic + fi + if [ -d $local_ftp/$rep/media ]; then + echo "==> Syncing media directory for $rep local repository" + eval rsync \ + $RSYNC_OPT $local_ftp/$rep/media \ + --exclude "*/old" --exclude "*/downloads.inc"\ + $dest_rsync/$rep + fi + fi +done + +for rep in ${remote_reps[*]}; do + if [ ! "$JOB" -o "$JOB" = "$rep" ]; then + echo "==> Syncing $rep remote repository" + eval rsync \ + $RSYNC_OPT \ + $local_ftp/$rep $dest_rsync \ + --exclude $rep/SRPMS.base \ + --exclude "$rep/RPMS.*" \ + --exclude $rep/home \ + --exclude $rep/autoupdate \ + --exclude $rep/old + echo "==> Syncing distromatic directory for $rep local repository" + eval rsync \ + $RSYNC_OPT \ + $local_ftp/distromatic/$rep $dest_rsync/distromatic + fi +done diff --git a/etc/cron.hourly/50-openmamba-ftpsync b/etc/cron.hourly/50-openmamba-ftpsync new file mode 100755 index 0000000..fffed7b --- /dev/null +++ b/etc/cron.hourly/50-openmamba-ftpsync @@ -0,0 +1,62 @@ +#!/bin/bash +# +# openmamba repository mirror script +# (c) 2006-2007 by Silvan Calarco +# +. /etc/sysconfig/openmamba-central + +[ "$ftpsync" ] || exit 0 +[ "$dest_ftp" ] || exit 0 + +ftp_sync() { + err=9 + start=`date +%s` + elapsed=0 + while [ $err -eq 9 ] && [ $elapsed -lt 300 ]; do + ftpsync -q -p $1 $2 ftpuser=$ftpuser ftppasswd=$ftppass timeout=20 + err=$? + now=`date +%s` + elapsed=`expr $now - $start` + [ $err -gt 0 ] && echo " Error: ftpsync exited with error $err" + done + return 0 +} + +for rep in ${remote_reps[*]}; do + echo "==> Working on $rep remote repository" + + echo " Syncyng APT database" + ftp_sync $local_ftp/$rep/base/ $dest_ftp/pub/openmamba/$rep/base + + echo " Syncing log files" + for a in $local_ftp/$rep/*log; do + curl -s -T $a -u$ftpuser:$ftppass --url $dest_ftp/pub/openmamba/$rep/ + done +done + +for rep in ${local_reps[*]}; do + echo "==> Working on $rep local repository" + for a in ${archs[*]}; do + err=1 + start=`date +%s` + elapsed=0 + echo " Syncing RPMs for arch $a" + ftp_sync $local_ftp/$rep/RPMS.$a/ $dest_ftp/pub/openmamba/$rep/RPMS.$a + done + echo " Syncing APT database" + ftp_sync $local_ftp/$rep/base/ $dest_ftp/pub/openmamba/$rep/base + + echo " Syncing SRPMs repository" + ftp_sync $local_ftp/$rep/SRPMS.base/ $dest_ftp/pub/openmamba/$rep/SRPMS.base + + echo " Syncing logs" + for a in $local_ftp/$rep/*log; do + curl -s -T $a -u$ftpuser:$ftppass --url $dest_ftp/pub/openmamba/$rep/ + done +done + +echo "==> Syncing distromatic" +ftpsync -q -p $local_distromatic $dest_ftp/pub/openmamba/distromatic ftpuser=$ftpuser ftppasswd=$ftppass timeout=20 + +echo "==> Syncing media directory for devel repository" +ftp_sync $local_ftp/devel/media/ $dest_ftp/pub/openmamba/devel/media timeout=20 diff --git a/etc/cron.hourly/60-openmamba-autobuild-autodist b/etc/cron.hourly/60-openmamba-autobuild-autodist new file mode 100755 index 0000000..68a3704 --- /dev/null +++ b/etc/cron.hourly/60-openmamba-autobuild-autodist @@ -0,0 +1,35 @@ +#!/bin/bash +# +# openmamba-dw-checklist daily cron script +# (c) 2008-2009 by Silvan Calarco +# +. /etc/sysconfig/openmamba-central + +PIDFILE=/var/run/autodist/autodist.pid +HOUR=`date +%H` + +function pid_check() { + + [ -e $PIDFILE ] && PIDCHECK=`cat $PIDFILE` + + [ "$PIDCHECK" -a -e /proc/$PIDCHECK ] && { + echo "Error: an autodist process is already running; exiting for safety." + exit 0 + + } +} + +pid_check +REPOSITORY=devel-autodist +openmamba-dw-checklist -h -r $REPOSITORY > $local_ftp/distromatic/$REPOSITORY/_popular.html + +pid_check +distromatic -q -t devel-autodist --gensrcpkglist > $local_ftp/$REPOSITORY/srcpkglist + +pid_check +su -l autodist -c "autodist -a --server devel-autodist" > $local_ftp/$REPOSITORY/autodist.log +if [ $? -eq 0 ]; then + cp $local_ftp/$REPOSITORY/autodist.log $local_ftp/$REPOSITORY/autodist-last.log +else + cp $local_ftp/$REPOSITORY/autodist.log $local_ftp/$REPOSITORY/autodist-last-error.log +fi diff --git a/etc/cron.weekly/10-openmamba-mlarchives b/etc/cron.weekly/10-openmamba-mlarchives new file mode 100755 index 0000000..b6e1846 --- /dev/null +++ b/etc/cron.weekly/10-openmamba-mlarchives @@ -0,0 +1,13 @@ +#!/bin/bash +# +# openmamba mailing list archives sync weekly cron +# Copyright (c) 2008 by Silvan Calarco +# +. /etc/sysconfig/openmamba-central + +MAILING_LISTS="openmamba-devel openmamba-devel-it openmamba-notizie openmamba-news openmamba-users openmamba-users-it" + +echo "Syncing mailing lists archives" +for l in $MAILING_LISTS; do + rsync -av /var/mailman/archives/private/$l $local_ftp/lists/archives/ +done diff --git a/etc/cron.weekly/20-gencontents b/etc/cron.weekly/20-gencontents new file mode 100755 index 0000000..7abde59 --- /dev/null +++ b/etc/cron.weekly/20-gencontents @@ -0,0 +1,35 @@ +#!/bin/bash + +. /etc/sysconfig/openmamba-central + +echo "Creating contentslist file" + +CONTENTSLISTFILE=$local_ftp/contentslist-i586 +CONTENTS_FILELIST="" + +for rep in ${local_reps[*]} ${remote_reps[*]}; do + [ -e $local_ftp/$rep/contentslist-i586 ] && + CONTENTS_FILELIST="$CONTENTS_FILELIST \ + $local_ftp/$rep/contentslist-i586" +done + +[ "$CONTENTS_FILELIST" ] && { + sort $CONTENTS_FILELIST > $local_ftp/contentslist-i586 + bzip2 -f $CONTENTSLISTFILE +} + +for c in $CONTENTS_FILELIST; do + bzip2 -f $c +# mv $c.bz2 `dirname $c`/.. +done + +echo "Syncing contentslist file" +curl -T $CONTENTSLISTFILE.bz2 \ + -u$ftpuser:$ftppass --url $dest_ftp/pub/openmamba/ + +for rep in ${local_reps[*]} ${remote_reps[*]}; do + [ -e $local_ftp/$rep/contentslist-i586.bz2 ] && + curl -T $local_ftp/$rep/contentslist-i586.bz2 \ + -u$ftpuser:$ftppass \ + --url $dest_ftp/pub/openmamba/$rep/ +done diff --git a/etc/sysconfig/openmamba-central b/etc/sysconfig/openmamba-central new file mode 100644 index 0000000..f09435f --- /dev/null +++ b/etc/sysconfig/openmamba-central @@ -0,0 +1,48 @@ +# +# openmamba ftp destination repository address and login information +# leave dest_ftp blank to disable sync +# +dest_ftp=ftp://ftpaccounts.openmamba.org +ftpuser= +ftppass= + +# +# openmamba rsync destination repository addess +# leave dest_rsync blank to disable sync +# +dest_ssh_port=22 +dest_ssh_key=/var/ssl/private/tao-topix2-rsync.key +dest_rsync="--password-file=/etc/rsync.password rsync://rsyncpriv@topix2.openmamba.org/openmamba-upload" +dest_ssh="-e 'ssh -o ForwardX11=no -i $dest_ssh_key -p $dest_ssh_port' root@topix2:/var/ftp/pub/openmamba" + +# perform ftp sync (obsolete) +#ftpsync=1 + +# +# base directory for local ftp repository +# +local_ftp=/var/ftp/pub/openmamba/ + +# +# distromatic base HTML directory +# +local_distromatic=$local_ftp/distromatic/ + +# +# list of available architectures +# +archs=(i586 arm ppc x86_64) + +# +# media targets for different release levels +# +stable_medias=(livecd installdvd livestudio bootusb) +prerelease_medias=(livecd installdvd livestudio bootusb) +snapshot_medias=(livecd livestudio libgames bootusb flash-1GB bootcd) + +# +# list of available local and remote repositories +# +local_reps=(devel devel-silvan devel-makedist devel-embedded devel-games devel-autodist devel-future devel-kde4 devel-kernel devel-xorg milestone1 milestone1-updates milestone1-games milestone1-1.1 milestone1-makedist milestone2) +remote_reps=(devel-contrib devel-stefano devel-aleph0 devel-tiziano devel-skiver devel-fabiog devel-mambaclimb devel-gil devel-bonzo devel-distservers \ + devel-the_packagizers devel-stativa devel-ercolinux devel-ciccio devel-Z10Bel devel-iw2oaz devel-zorro milestone1-java milestone1-contrib devel-java)