Path: tut!sunic!uupsi!rpi!uwm.edu!zaphod.mps.ohio-state.edu!brutus.cs.uiuc.edu!psuvax1!psuvm!cunyvm!ndsuvm1!plains!overby From: overby@plains.UUCP (Glen Overby) Newsgroups: comp.os.minix Subject: Upgrading Your Minix System: A Tutorial Summary: A tutorial on how to upgrade Minix from the upgrade kits available on the nets. Keywords: info answers Message-ID: Date: 27 Apr 90 03:38:55 GMT Reply-To: overby@plains.nodak.edu (Glen Overby) Followup-To: comp.os.minix Organization: North Dakota State University, Fargo ND, USA Lines: 663 [UpgradeTutorial 4/26/90 Glen Overby ] Upgrading Your Minix System Glen Overby Minix's author, Andy Tanenbaum, has graciously given the network community upgrades to Minix by posting "diff" listings between the previously released version and his working copy. In fact, the network community gets MUCH more frequent upgrades than Prentice-Hall sells, and are rather openly used as guineapigs for his next formal release. The availability and timeliness that Andy sends to the net make it quite attractive way to keep your Minix system up-to-date. This tutorial is an attempt to explain what has to be done in order to use the updates you see on the net. The upgrade process can take anywhere from an evening to an en- tire weekend, depending on how much CPU power and disk space you have available, as well as how major of an upgrade you are doing. Unfortunately, many people have difficulties with the upgrade process. This tutorial was written to help newcomers along the way. I would like to note right now that I do not (and will not) pro- vide individual hand-holding -- you are on your own. I do wel- come rational comments and suggestions for improving this docu- ment, but don't expect me to address your every pet peeve. WHAT IS AN UPGRADE KIT? An upgrade kit typically consists of several context diffs, pos- sibly some complete files, and a CRC list of the full Minix source directory tree. The combination of these will allow you to upgrade your current Minix sources to the current version. A "diff" file is a listing of differences on a line-by-line basis between two files. The diff(1) program will find lines that have been added, deleted or changed between two files (like an old and a new version). Regular diffs list only the lines changed or ad- ded, while context diffs contain not only the line(s) added or changed, but several lines around them. The few extra lines of context help out greatly in finding the proper place to drop the new changes into. Context diffs also typically contain the names and last modification date of the compared files. You should study the manual pages for the programs "diff" and "cdiff" which are used to find the differences between files, as well as the "fix" and "patch" programs ("fix" is a rather simple program which can apply regular diffs, and patch is an elaborate program that can apply context diffs quite well, even when things don't match up quite right). Being familiar with their use and the input and output will help you make effective use of the patch files that others post on the net to fix bugs and add features to Minix. If you ever make some changes to Minix which you wish to distri- bute, context diffs are the preferred format. OBTAINING AN UPGRADE KIT There are two sources of upgrade kits. The original source of official patches for new versions of Minix comes from Minix's au- thor, Andy Tannenbaum. Andy typically makes several patch levels between Minix releases, which must be incrementally applied to your original sources to arrive at a full release point. The more patches you have to apply to files, the more chances there are for mistakes which will keep you from coming out with a 100% correct update. To help avoid the problems with this, there are generally a few people who will collect all the changes made for a full release and make a canonical upgrade kit which re- quires applying one diff to each file or directory. This tutorial is oriented towards using the diff files that Andy himself sends out, but the ideas and techniques will apply to the others, too. Andy uses a separate diff file for each changed source file, while others doing kits which encompass several releases will have an entire directory or the entire source tree in one diff file. Your first task is to obtain the upgrade kit(s) needed to bring your system up to the current level (1.5.8 as of 4/24/90). If you saved Andy's upgrade kits as he posted them, you should be all set. Otherwise, get a copy of the Minix Information Sheet (it's posted monthly and is available from several archive sites; instructions for obtaining it by mail are in Appendix A) and select an archive site to obtain the files from, preferably one close to you network wise. If you're working from Andy's postings, you will have CRCs of the raw distribution files which you should immediately check. If they do not match, make sure that your files do not have any Usenet or mail headers on them, or any extraneous characters at the end. HOW THE PACKAGE IS PUT TOGETHER An upgrade package is created by running "cdif" for each file changed in the new version, giving a "filename.cdif" file. The cdiff files are gathered together into several "shar" files, which are compressed with the "compress" program and encoded with "uuencode". You must undo all of these steps to get text cdiff files. A program to do most of this work for you, called "unpak" is in Appendix B. The reason for all of this is first to gather all the cdiffs from one directory into a single "bundle", which may be extracted with the Unix shell (which everyone gets with their Minix system) or the "unshar" program (unshar is generally a bit speeder). Encod- ing with "uuencode" is done so that the cdiffs can be transferred through IBM (Bitnet) sites with no problems; if this is not done, the ASCII to EBCDIC translations will severely damage certain un- common characters, such as curly braces ("{" and "}"), tilde ("~") and so on. It makes no difference whether the file being encoded is ASCII or binary, so we take advantage of that by using "compress" to reduce it's size. The 35% (3:4) expansion done by uuencode is more or less canceled out by being able to get 40-60% compression with compress. The details of how these three pro- grams (shar, compress and uuencode) are left "as an exercise to the reader". You will also find "CRC" listings. A CRC, or Cyclic Redundancy Check, is an error detecting code that is used to check that all your files were upgraded properly. For more information on what CRCs are, see the source for "crc.c" and any good Computer Net- works book. Note that others who collect several levels of changes from Andy normally collect their diffs on a system or directory basis; it's all a matter of preference. BEFORE YOU START Before you start applying the diffs, there are a few files which you must copy into the /usr/src tree from other areas. All text files from /etc go into /usr/src/etc group, message, passwd, rc, setup_root, setup_usr, termcap, ttys, ttytype All text files from /usr/lib go into /usr/src/Lib crtso.s, dictionary, end.s, fortune.dat, head.s, help.more Temporarily move /usr/include to /usr/src/include (and move it back after you are done applying the cdiffs). now you should be all set. APPLYING THE DIFFS After splitting the uuencoded, compressed shar files up into an empty directory, you should have a couple dozen subdirectories named after a directory in the Minix source tree, each containing new files (or files that have had enough changes that their cdiffs would be larger than the source file itself), context diffs against old files, and a CRC for this directory (there is generally a global CRC list distributed; you can use either). Now to apply the diffs to your source directory. Before you start on this, make certain you have a backup copy for all your sources in case you mess things up and need to start over. You need to run "patch" for each cdif file in each directory, with it's input coming from the cdif file. It should be obvious right now that this is a VERY tedious process, and is ripe for automa- tion. See the script "doup" in Appendix B that will automate this process. When this script finally finishes, you will have a fully patched system. Take a look at the file "PROBLEMS" in the directory you ran it from in case it ran into some problems. You will also have not only your new source files in /usr/src, but the origi- nals (with a "~" appended to their name; these may be removed later after you have verified that the upgrade was done properly and don't need them), and possibly some files ending with a "#" which are left by Patch when it encounters problems. PROBLEMS There are many little problems that can come up when upgrading. They come from various sources, such as not starting with correct files (such as from a previous upgrade which did not work proper- ly), manual error, and there are some problems nobody has really tracked down. Many files in the "commands" directory require large amounts of space on /tmp, on the order of 200-300K. Most people don't have RAM disks with that much extra space, and must mount either another hard disk partition or a floppy disk on /tmp. There is no option on patch to tell it to write it's temporary files in another place. Early indications that you are having problems will come from the "patch" program. It will start talking about "fuzz" and some- times reject patches. Check that you are starting with a good set of files, and that your patches have the correct CRCs. Sometimes patch runs out of memory. The only solution for this is to use a larger machine to apply the diff for the file or files that have this problem. In fact, if you have access to a large, fast Unix machine with lots of disk space, it's much easier to apply all the patches there than under Minix, but it is still quite doable on Minix. [Yes, I know this section is lacking in detail; the problems I have had in upgrading can be attributed largely to starting with a bad file, or have defied explanation]. CHECKING CRCs Once you have completed applying the diffs, you need to make sure that everything went OK. Even a lack of adverse diagnostics from the patch program and upgrade script does not mean that every- thing went OK. You can manually check the CRCs, or use the pro- gram "crccheck" in Appendix B do do it for you. BUILDING A LIBRARY Now that you have a properly upgraded Minix source tree, you need to compile it. The best place to start is with the library, since everything else relies on it. Cd to each directory under /usr/src/lib, and type "make all" in each directory. After several hours of compiling, you will have an assembly file for each source file. I then move (or link) all the assembly (.s) files into a directory called "obj" where I can work with them as a group. There are a few files in IBM which should not be moved: crtso.s, end.s, head.s you also need the run-time library for the C compiler, which is in your current C library (/usr/lib/libc.a). A list of these files are in Appendix C for your convenience. Next comes the part which many people have trouble with: creating a library order. The C library must be ordered such that there are no backwards references; that is, anything that a library function relies on must come LATER in the library, or it will not be found. See the scripts "mkdorder" and "libbuild" in Appendix B to do the ordering for you. A copy of the ordering I obtained with these scripts is also included in Appendix D, but you should be able to recreate it. A brief explanation of the scripts is in order. Two utilities are required to create a library order: lorder and tsort. Lorder creates a dependency list, that is, a list of what functions are required by what other functions. Tsort takes the output of lorder and does a "topological sort" to create an ordering with no backwards references. One small problem comes up in this process: the command line to a process in Minix can only be 2048 bytes long, and the filenames in your lib/obj directory are far larger than this. A work- around is to create an unordered library by running "ar" with a few files at a time, and then feeding this one file to tsort. Run the "mkorder" script from your "lib/obj" directory, with one command option of an output file for the order to be placed in when everything is done. Next run "libbuild" with two parame- ters: the file to place your library in (don't clobber /usr/lib/libc.a at this point -- at least keep a copy of your original library around in case things don't work properly) and the order file created in the above step. After much disk churning, you should have a properly ordered li- brary. There are currently no circular references in the C li- brary sources, so you should not need to "tweak" the library ord- ering at all. If all this worked properly, you can copy your new library to /usr/lib/libc.a, again without clobbering your original library. Compiling the rest of the directories should be very straightfor- ward, just type "make all" in each directory. APPENDIX A Obtaining the Minix Information Sheet There are several sites which archive all messages sent to the list, plus anything else that they see fit. Information about many archive sites, as well as other common questions about Minix can be found in the "Minix Information Sheet". A copy of it can be had by anonymous FTP to plains.nodak.edu [134.129.111.64] in pub/Minix/doc/Info_Sheet or by sending electronic mail to archive-server@plains.nodak.edu (aka "plains" on uucp and bitnet) saying: send Minix/doc/Info_Sheet a report on what machines Minix has been tested on is available in the same directory in the file "Compatability". Glen Overby info-minix List Maintainer uunet!plains!minix (UUCP) minix@plains (Bitnet) APPENDIX B Scripts for the Upgrade Process The following scripts are contained within a shar file: unpak - unpack the upgrade kits) doup - apply several directories of patches crccheck - check CRCs mkorder - create a library ordering libbuild - build a library #! /bin/sh # This is a shell archive. Remove anything before this line, then unpack # it by saving it into a file and typing "sh file". To overwrite existing # files, type "sh file -c". You can also feed this as standard input via # unshar, or by typing "sh 'unpak' <<'END_OF_FILE' X# X# Unpak.sh - unpack Minix distribution X# X# Synopsis: X# unpak files X# X# Will uudecode, uncompress and unshar Minix distribution files from X# Dr. Tanenbaum. Each file is unshared into a directory based on the X# filename, minus sequence number, such that "kernel_00" and "kernel_01" X# will both unpack into the directory "kernel". X# X# Author: X# 4/24/90 Glen Overby X# X# uue to shar for i in $* do X echo $i X uudecode <$i X j=`basename $i .uue` X uncompress $j.Z X b=`echo $j | cut -f1 -d_` X mkdir $b 2>/dev/null X (cd $b ; unshar ../$j) done END_OF_FILE if test 563 -ne `wc -c <'unpak'`; then echo shar: fi chmod +x 'unpak' # end of 'unpak' fi if test -f 'doup' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file else echo shar: Extracting sed "s/^X//" >'doup' <<'END_OF_FILE' X# X# doup - Apply Minix upgrade patches X# X# Synopsis: X# doup directories X# X# for each file in the listed directories, either copy the file to the X# source directory (for new files), run "patch" on cdiff (context diff) X# files, or log an error message. X# X# Author: X# 4/24/90 Glen Overby X# X src=/usr/src here=`pwd` X for new in $* do X echo "Processing directory $new" X X case $new in X Lib | etc | fs | include | kernel | mm | test | tools ) X d=$new ;; X IBM | ansi | posix | other | string ) X d=lib/$new ;; X bawk | de | dis88 | ibm | ic | indent | kermit | m4 | make | mined | X nroff | patch | sh | zmodem ) X d=commands/$new ;; X command ) X d=commands ;; X minix | other | sys ) X d=include/$new ;; X * ) X echo "Don't know what to do with $new" X echo "dir $new" >>$here/PROBLEMS X continue ;; X esac X X echo "Patching $d" X cd $src/$d X X for f in $here/$new/* X do X echo "File $f" X case $f in X *.c | *.h | */Makefile* ) X echo "New file $f" X o=`basename $f` X mv $o $o~ X cp $f . X ;; X *.cdif ) X echo "Patching $f" X patch <$f X ;; X *.hdr | *.crc ) X echo "ignoring" $f X ;; X * ) X echo "Don't know what to do with $f" X echo $f >> $here/PROBLEMS X ;; X esac X done done END_OF_FILE if test 1209 -ne `wc -c <'doup'`; then echo shar: fi # end of 'doup' fi if test -f 'crccheck' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file else echo shar: Extracting sed "s/^X//" >'crccheck' <<'END_OF_FILE' X#!/bin/ksh X# X# Hacked-up crc checking program. Really arcane, but it should give you X# the idea of what to do. Probably won't run on Minix (ran on SunOS). X# X# Author: X# 4/24/90 Glen Overby X# X d=`basename $PWD` X echo $d X for i in $* do X echo $i X crc $i/*[chx] $i/[Mm]akefile | sort >$i/$i.CRC X sed -e 's//home/top/ast/minix/1.5.2/lib///' <$i/$i.crc |X sort | diff $i/$i.CRC - done END_OF_FILE if test 418 -ne `wc -c <'crccheck'`; then echo shar: fi chmod +x 'crccheck' # end of 'crccheck' fi if test -f 'mkorder' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file else echo shar: Extracting sed "s/^X//" >'mkorder' <<'END_OF_FILE' X# X# mkdorder - Order the C library X# X# Synopsis: X# mkorder Order-File X# X# Mkorder will build a library order for all the *.s files in the current X# directory, using lorder and tsort. The final output of tsort is copied X# to 'Order-File'. X# X# Steps: X# * Build an un-ordered library from all the sources X# * run lorder on the un-ordered library X# * sort (tsort) the output from lorder to create a library ordering X# * mung (if necessary) the sorted output to remove recursive problems X# * build C library X# X# Bugs/MisFeatures: X# uses the 'libbuild' program, and assumes it is one directory up. X# X# Author: X# 4/24/90 Glen Overby X# X rm -f all.a ls >/tmp/$$.all X../libbuild all.a /tmp/$$.all lorder all.a >/tmp/$$.lorder tsort /tmp/$$.tsort cp /tmp/$$.tsort $1 X rm -f all.a /tmp/$$.* echo "now run 'libbuild libc.a ORDER'" END_OF_FILE if test 873 -ne `wc -c <'mkorder'`; then echo shar: fi # end of 'mkorder' fi if test -f 'libbuild' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file else echo shar: Extracting sed "s/^X//" >'libbuild' <<'END_OF_FILE' X# X# libbuild - Build a library from a file containing a list of files to X# be placed in that library X# X# Synopsis: X# libbuild library list-of-files-file X# X# Since Minix limits it's command parameters to 2048 bytes or less, it is X# not possible to build a library with a single "ar" command. This script X# circumvents that problem by running several "ar" commands (20 files X# per command). X# X# Author: X# 4/24/90 Glen Overby X# X rm -f /tmp/lib* $1 split -20 $2 /tmp/lib for i in /tmp/lib* do X ar q $1 `cat $i` done END_OF_FILE if test 542 -ne `wc -c <'libbuild'`; then echo shar: fi # end of 'libbuild' fi echo shar: End of shell archive. exit 0 APPENDIX C C Compiler Run-Time Library Files The following files make up the ACK C Compiler Run-Time library. _dup.s adi.s and.s blm.s cii.s cmi4.s cms.s cmu4.s com.s csa2.s csb2.s cuu.s dvi4.s dvu4.s exg.s fakfp.s fat.s getutil.s gto.s iaar.s ilar.s inn.s ior.s isar.s lar2.s lfr6.s lfr8.s loi.s mli4.s mon.s nop.s perf.s rck.s ret6.s ret8.s retarea.s return.s rmi4.s rmu4.s sar2.s sbi.s set.s sti.s stop.s strhp.s trp.s unknown.s vars.s xor.s APPENDIX D Sample C Library Ordering execlp.s popen.s system.s curses.s nlist.s termcap.s ttyname.s closedir.s fclose.s opendir.s setbuf.s fdopen.s fopen.s putenv.s regexp.s malloc.s exec.s brk.s ctime.s fseek.s perf.s seekdir.s strtol.s assert.s getopt.s lock.s fprintf.s sprintf.s doprintf.s getcwd.s printk.s strtoul.s cuserid.s getlogin.s getgrent.s getpwent.s atoi.s atol.s scanf.s lrand.s sleep.s readdir.s freopen.s ftell.s getdents.s peekpoke.s portio.s rewinddir.s telldir.s getpass.s strerror.s puts.s fputs.s fwrite.s vsprintf.s fputc.s cleanup.s fflush.s perror.s prints.s stderr.s uniqport.s utime.s fgets.s fread.s gets.s fgetc.s pathconf.s abort.s tmpnam.s mktemp.s fpathconf.s iaar.s ilar.s isar.s sincos.s gtty.s stty.s ioctl.s sysconf.s cii.s csa2.s csb2.s cuu.s fakfp.s strhp.s unknown.s adi.s sbi.s fat.s mon.s stop.s trp.s _exit.s alarm.s chown.s close.s dup.s dup2.s exit.s fcntl.s fork.s fstat.s getegid.s geteuid.s getgid.s getpid.s getppid.s getuid.s kill.s link.s mkdir.s mkfifo.s mknod.s mount.s open.s pause.s pipe.s read.s rename.s setgid.s setuid.s stat.s sync.s syslib.s time.s times.s umask.s wait.s write.s access.s chdir.s chmod.s chroot.s creat.s rmdir.s umount.s unlink.s isatty.s lseek.s ptrace.s signal.s stime.s call.s vectab.s catchsig.s rand.s lfr6.s lfr8.s ret6.s ret8.s index.s rindex.s regsub.s strxfrm.s strstr.s ctermid.s strcoll.s bzero.s lsearch.s bcopy.s memmove.s bcmp.s bsearch.s crypt.s ffs.s getenv.s itoa.s memccpy.s memchr.s memcmp.s memcpy.s memset.s qsort.s stb.s strcat.s strcmp.s strcpy.s strcspn.s strlen.s strncat.s strncmp.s strncpy.s strpbrk.s strrchr.s strspn.s strtok.s swab.s ungetc.s abs.s amoeba.s strchr.s xor.s vars.s sti.s setjmp.s set.s sendrec.s sar2.s rmu4.s rmi4.s return.s retarea.s rck.s printdat.s nop.s mli4.s message.s loi.s lar2.s ior.s inn.s gto.s getutil.s exg.s errno.s dvu4.s dvi4.s ctype.s com.s cmu4.s cms.s cmi4.s brksize.s blm.s and.s _dup.s GLOSSARY compress diff context diff patch uuencode THANKS Special thanks to the following people for their good comments on how to improve this tutorial: Lars Fredrickson -- Glen Overby uunet!plains!overby (UUCP) overby@plains (Bitnet)