Running phase: unpackPhase unpacking source archive /nix/store/wbaw39kxnpmwqz1pqh7qjsscwgwd713k-beta-nettee-0.3.4.tar.gz source root is nettee-0.3.4 setting SOURCE_DATE_EPOCH to timestamp 1235595798 of file "nettee-0.3.4/nettee" Running phase: patchPhase Running phase: updateAutotoolsGnuConfigScriptsPhase Running phase: configurePhase no configure script, doing nothing Running phase: buildPhase nettee (NETwork TEE) Version 0.3.4 25 FEB 2009 David Mathog License: GPL 2 [ nettee is derived from dolly 0.58C by Felix Rauch ] nettee is a network version of the Unix "tee" program. See the man page nettee.1 for more info. To compile, use gcc -Wall -pedantic -std=c99 -D_LARGEFILE64_SOURCE -D_POSIX_SOURCE -D_XOPEN_SOURCE -o nettee nettee.c rb.c nio.c Returns status is "EXIT_SUCCESS" if there are no errors and "EXIT_FAILURE" if any node fails in a manner that cannot be handled. With version 0.2.0 a helper script "topology_info" is distributed. It extracts data from a topology.txt file that describes the local cluster's distribution tree. This method is used by some of the other scripts here to set up a distribution tree. The topology.txt file is placed in a directory readable on all nodes. A different topology file may be specified with the shell variable NTOPOLOGY. The output of topology_info may be fed into cut or extract to pull out the relevant information. Examples: % topology_info slave03 slave03.cluster internal slave04 % topology_info slave20 slave20.cluster tip none To point to a different file. The pdist*.sh scripts are example wrappers that use nettee. For instance: pdist_file.sh % pdist_file.sh pdist_store.sh /tmp/foobar.txt >/tmp/cmdfifo indicates that there is about a .007s delay before the first node executes and .009s delay before the last node executes. The test nodes were synched with ntp to the master node which issued the above command. This is a more efficient way to use nettee to distribute small files. Ie: echo "nettee -next \$NEXTNODE -out wherever" >>/tmp/cmdfifo nettee -root -in smallfile -next $FIRSTNODEINCHAIN only suffers about .01 s delay in setting up the chain. Conversely using pdist_file.sh to distribute small files (one at a time) is slow due to the rsh setup overhead. execinput, accudate, and extract, are referred to in the nettee documentation and/or used by these scripts. These are available in source code here: ftp://saf.bio.caltech.edu/pub/software/linux_or_unix_tools/ Instructions for using nettee with SystemImager are in the files SI_METHOD1.TXT and SI_METHOD2.TXT. WARNING: On AMD Athlon processors (the 32 bit ones, not the Athlon64) the "athcool" program, if active, will degrade throughput significantly wherever nettee must read from the net and then write to disk. When transferring large files on these machines it is best to first do "athcool off", then transfer, then "athcool on". Please send comments to the email address above. Files in distribution: beowulf.master Cluster imaging script for use with System Imager and nettee LICENSE GPL 2 license nettee.1 nettee man page nettee.c nettee source code nettee_cmd.3 nettee -process interface man page nettee_cmd.html nettee -process interface html documentation nettee_man.html nettee html documentation nio.c nettee cmd library source nio.h nettee cmd library header pdist_copy.sh example script for copying files with nettee pdist_file.sh example script for copying files with nettee ppdist_gunzip_detar.sh helper script for pdist_file.sh dist_shell.sh example script for distributing commands with nettee pdist_store.sh helper script for pdist_file.sh rb.c ring buffer library source rb.h ring buffer header README.TXT This file SI_METHOD1.TXT Instructions for using nettee with system imager SI_METHOD2.TXT Another way of using nettee with system imager topology_info example topology script topology.txt example data for topology script Change log: 0.3.4 added -branch. Fixed a bug in error reporting from child nodes, and another triggered where multiple hostslists were used and the first failed to connect to a child node. 0.3.2 fixed two bugs (one in nio.c, one in nio.h) which broke the -process method. 0.3.1 changed all shutdown() to close(). Added -burst, -sync, and time stamp options. 0.2.0 Numerous changes. Added ring buffers, -process option, more continue on options, -flow (push or pull) and others. 0.1.8 Added -connf flag and failovers for -next. Modified formatting of messages slightly. 0.1.7 Minor change to get a clean compilation on solaris. 0.1.6 Added nettee.spec file from dag wieers. RPM page for nettee for OS's that are maintained there are: http://dag.wieers.com/packages/nettee/ Modified error handling on bad count. Previously if node K+1 returned a bad byte count node K would return the bad byte count, and the error condition would be propagated upward in that manner. In this version node K returns its own byte count but sets an error flag that propagates upward instead. Also the CONWF, COLWF, etc error announcements now include the name of the preceding node if that node generated the error. The general idea being to allow the distribution chain to continue functioning when certain types of errors occur on individual nodes. Theoretically these error messages could be piped into something like "socket" or "logger" and then logged to a central server. Without that they would need to be stored some place like /var/run/nettee_messages.txt on each node and processed in a post mortem. If -v 1 is used and no errors occur this would generate no traffic, so it shouldn't interfere with throughput if everything is working. Added changes to handle compilation warnings noticed by Tru Huynh on an X86_64 machine. Fixed a bug in beowulf.master which sometimes resulted in the end node not being able to set its HOSTNAME. nettee.c: In function 'open_and_connect_child_socks': nettee.c:899:8: warning: variable 'stestcount' set but not used [8;;https://gcc.gnu.org/onlinedocs/gcc-14.2.0/gcc/Warning-Options.html#index-Wno-unused-but-set-variable-Wunused-but-set-variable8;;] 899 | int stestcount, swaitcount; | ^~~~~~~~~~ nettee.c: In function 'process_command_line_args': nettee.c:3036:14: error: implicit declaration of function 'gethostname'; did you mean 'gethostbyname'? [8;;https://gcc.gnu.org/onlinedocs/gcc-14.2.0/gcc/Warning-Options.html#index-Wimplicit-function-declaration-Wimplicit-function-declaration8;;] 3036 | (void) gethostname(nt->myhostname, NAMESIZE-1); | ^~~~~~~~~~~ | gethostbyname nio.c: In function 'ncmd_control_child': nio.c:149:11: warning: variable 'readable' set but not used [8;;https://gcc.gnu.org/onlinedocs/gcc-14.2.0/gcc/Warning-Options.html#index-Wno-unused-but-set-variable-Wunused-but-set-variable8;;] 149 | int readable; /* number of open input streams which may be read */ | ^~~~~~~~