Report forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to Rob Browning <osiris@cs.utexas.edu>:
New bug report received and forwarded. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   Rob Browning <osiris@cs.utexas.edu>  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at submit) by bugs.debian.org; 16 Oct 1996 01:44:31 +0000 Received: (qmail 19803 invoked from smtpd); 16 Oct 1996 01:44:25 -0000 Received: from slip-76-15.ots.utexas.edu (HELO raven.ots.utexas.edu) (osiris@128.83.254.127) by master.debian.org with SMTP; 16 Oct 1996 01:44:21 -0000 Received: (from osiris@localhost) by raven.ots.utexas.edu (8.7.6/8.7.3) id UAA08081; Tue, 15 Oct 1996 20:33:46 -0500 Sender: osiris@cs.utexas.edu To: submit@bugs.debian.org Subject: Afio fails to restore from remote backup file From: Rob Browning Date: 15 Oct 1996 20:33:46 -0500 Message-ID: <87enizofph.fsf@raven.ots.utexas.edu> Lines: 23 X-Mailer: Red Gnus v0.49/Emacs 19.34 Package: afio Version: 2.4.2-1 Try this simple test (assumes you have localhost in your .rhosts file): %find /usr/include/asm-i386 | afio -o localhost:/tmp/foo.afio %afio -t localhost:/tmp/foo.afio asm-i386 asm-i386/sigcontext.h asm-i386/unistd.h asm-i386/bitops.h [...] afio: "localhost:/tmp/foobar.afio": Premature input EOF Not good. If I get the time I'll look at the code and see if I can figure it out. Thanks -- Rob   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to Dirk.Eddelbuettel@qed.econ.queensu.ca:
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   Dirk.Eddelbuettel@qed.econ.queensu.ca  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792) by bugs.debian.org; 16 Oct 1996 02:52:41 +0000 Received: (qmail 8916 invoked from smtpd); 16 Oct 1996 02:52:37 -0000 Received: from knot.queensu.ca (HELO post.QueensU.CA) (root@130.15.126.54) by master.debian.org with SMTP; 16 Oct 1996 02:52:34 -0000 Received: from miles.econ.queensu.ca (free2-slip224.tele.QueensU.CA [130.15.126.224]) by post.QueensU.CA (8.6.12/8.6.10+ASH) with SMTP id WAA11969; Tue, 15 Oct 1996 22:41:49 -0400 Received: by miles.econ.queensu.ca id m0vDLvQ-00014oC (Debian /\oo/\ Smail3.1.29.1 #29.37); Tue, 15 Oct 96 22:41 EDT Message-Id: Date: Tue, 15 Oct 96 22:41 EDT To: Rob Browning , 4792@bugs.debian.org Subject: Re: Bug#4792: Afio fails to restore from remote backup file In-Reply-To: <87enizofph.fsf@raven.ots.utexas.edu> References: <87enizofph.fsf@raven.ots.utexas.edu> X-NoJunk: Please do NOT send junk email, spams or commercial ads to me. From: Dirk.Eddelbuettel@qed.econ.queensu.ca Rob Browning writes: Rob> Rob> Package: afio Rob> Version: 2.4.2-1 Rob> Rob> Try this simple test (assumes you have localhost in your .rhosts file): Rob> Rob> %find /usr/include/asm-i386 | afio -o localhost:/tmp/foo.afio Rob> %afio -t localhost:/tmp/foo.afio Rob> asm-i386 Rob> asm-i386/sigcontext.h Rob> asm-i386/unistd.h Rob> asm-i386/bitops.h Rob> [...] Rob> afio: "localhost:/tmp/foobar.afio": Premature input EOF Rob> Rob> Not good. Very, very odd. Not only does it works for me here from bird, my laptop, to miles, my desktop, as seen from this log: edd@bird:~> find /usr/include/asm-i386/ | afio -o miles:/tmp/new.afio edd@bird:~> afio -t miles:/tmp/new.afio usr/include/asm-i386 usr/include/asm-i386/sigcontext.h usr/include/asm-i386/unistd.h usr/include/asm-i386/bitops.h [...others deleted...] usr/include/asm-i386/math_emu.h usr/include/asm-i386/vm86.h usr/include/asm-i386/unaligned.h usr/include/asm-i386/semaphore.h edd@bird:~> but I also used this recently to back bird up using tob's remote backup setup (using a pipe and dd). Could it be that it works for me as I use a slow link through a null-modem cable? I presume that you use ethernet? Otherwise, I'm clueless. Both my machines are standard Debian-1.1 with the usual libc5 etc. Puzzled, Dirk -- Dirk Eddelb"uttel http://qed.econ.queensu.ca/~edd   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to Rob Browning <osiris@cs.utexas.edu>:
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   Rob Browning <osiris@cs.utexas.edu>  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792) by bugs.debian.org; 16 Oct 1996 03:12:09 +0000 Received: (qmail 12356 invoked from smtpd); 16 Oct 1996 03:12:02 -0000 Received: from slip-76-15.ots.utexas.edu (HELO raven.ots.utexas.edu) (osiris@128.83.254.127) by master.debian.org with SMTP; 16 Oct 1996 03:12:01 -0000 Received: (from osiris@localhost) by raven.ots.utexas.edu (8.7.6/8.7.3) id WAA08696; Tue, 15 Oct 1996 22:01:11 -0500 Sender: osiris@cs.utexas.edu To: Dirk.Eddelbuettel@qed.econ.queensu.ca Cc: 4792@bugs.debian.org Subject: Re: Bug#4792: Afio fails to restore from remote backup file References: <87enizofph.fsf@raven.ots.utexas.edu> From: Rob Browning Date: 15 Oct 1996 22:01:11 -0500 In-Reply-To: Dirk.Eddelbuettel@qed.econ.queensu.ca's message of Tue, 15 Oct 96 22:41 EDT Message-ID: <87aftnobns.fsf@raven.ots.utexas.edu> Lines: 30 X-Mailer: Red Gnus v0.49/Emacs 19.34 Dirk.Eddelbuettel@qed.econ.queensu.ca writes: > but I also used this recently to back bird up using tob's remote backup setup > (using a pipe and dd). Could it be that it works for me as I use a slow link > through a null-modem cable? I presume that you use ethernet? Otherwise, I'm > clueless. Both my machines are standard Debian-1.1 with the usual libc5 etc. > > Puzzled, Dirk What about using localhost? Does it work for you then, when you're not connected to another net. The loopback interface should be fast enough to cause trouble if that's the issue. Also, what's the speed of the machines involved, perhaps it's processor speed related. I'm suspecting it might be that afio's not checking it's calls to close. If you check "man 2 close" it mentions that if you don't you might lose data. I'm guessing that afio might be quitting before all the data is written on stdout across the rsh connection. I haven't investigated enough yet to know, and I can't figure out how to flush the data. fflush works, but only on FILE*'s. I was trying to find something that worked on fd's. It's repeatable here on at least two machines, one's Debian, the other's not. BTW, am I suppoesed to be cc'ing this to you, or do you get it whenever it's sent to bugs? My default follup includes you... -- Rob   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to Rob Browning <osiris@cs.utexas.edu>:
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   Rob Browning <osiris@cs.utexas.edu>  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792) by bugs.debian.org; 16 Oct 1996 03:27:10 +0000 Received: (qmail 20926 invoked from smtpd); 16 Oct 1996 03:27:04 -0000 Received: from slip-76-15.ots.utexas.edu (HELO raven.ots.utexas.edu) (osiris@128.83.254.127) by master.debian.org with SMTP; 16 Oct 1996 03:27:03 -0000 Received: (from osiris@localhost) by raven.ots.utexas.edu (8.7.6/8.7.3) id WAA08829; Tue, 15 Oct 1996 22:15:40 -0500 Sender: osiris@cs.utexas.edu To: Dirk.Eddelbuettel@qed.econ.queensu.ca, 4792@bugs.debian.org Subject: Re: Bug#4792: Afio fails to restore from remote backup file References: <87enizofph.fsf@raven.ots.utexas.edu> From: Rob Browning Date: 15 Oct 1996 22:15:39 -0500 In-Reply-To: Dirk.Eddelbuettel@qed.econ.queensu.ca's message of Tue, 15 Oct 96 22:41 EDT Message-ID: <877moroazo.fsf@raven.ots.utexas.edu> Lines: 7 X-Mailer: Red Gnus v0.49/Emacs 19.34 I noticed that you just sent the info out to debian-devel. Does the bug system not do that anyway? I thought it did... -- Rob   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to Dirk.Eddelbuettel@qed.econ.queensu.ca:
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   Dirk.Eddelbuettel@qed.econ.queensu.ca  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792) by bugs.debian.org; 16 Oct 1996 03:51:57 +0000 Received: (qmail 25836 invoked from smtpd); 16 Oct 1996 03:51:54 -0000 Received: from knot.queensu.ca (HELO post.QueensU.CA) (root@130.15.126.54) by master.debian.org with SMTP; 16 Oct 1996 03:51:50 -0000 Received: from miles.econ.queensu.ca (free2-slip220.tele.QueensU.CA [130.15.126.220]) by post.QueensU.CA (8.6.12/8.6.10+ASH) with SMTP id XAA16453; Tue, 15 Oct 1996 23:41:15 -0400 Received: by miles.econ.queensu.ca id m0vDMqv-00017qC (Debian /\oo/\ Smail3.1.29.1 #29.37); Tue, 15 Oct 96 23:41 EDT Message-Id: Date: Tue, 15 Oct 96 23:41 EDT To: Rob Browning Cc: Debian Developer , 4792@bugs.debian.org Subject: Re: Bug#4792: Afio fails to restore from remote backup file In-Reply-To: <87aftnobns.fsf@raven.ots.utexas.edu> References: <87enizofph.fsf@raven.ots.utexas.edu> <87aftnobns.fsf@raven.ots.utexas.edu> X-NoJunk: Please do NOT send junk email, spams or commercial ads to me. From: Dirk.Eddelbuettel@qed.econ.queensu.ca Rob> What about using localhost? Does it work for you then, when you're Rob> not connected to another net. The loopback interface should be fast Rob> enough to cause trouble if that's the issue. Also, what's the speed Rob> of the machines involved, perhaps it's processor speed related. No, it doesn't work with localhost, I get the same "Premature input EOF" as you do. But is that really an issue? I mean, localhost is not a remote host after all, and it's meant to work remotely. Rob> I'm suspecting it might be that afio's not checking it's calls to Rob> close. If you check "man 2 close" it mentions that if you don't you Rob> might lose data. I'm guessing that afio might be quitting before all Rob> the data is written on stdout across the rsh connection. I haven't Rob> investigated enough yet to know, and I can't figure out how to flush Rob> the data. fflush works, but only on FILE*'s. I was trying to find Rob> something that worked on fd's. Do you want me to forward this to the upstream maintainer? -- Dirk Eddelb"uttel http://qed.econ.queensu.ca/~edd   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to Rob Browning <osiris@cs.utexas.edu>:
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   Rob Browning <osiris@cs.utexas.edu>  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792) by bugs.debian.org; 16 Oct 1996 04:08:26 +0000 Received: (qmail 30918 invoked from smtpd); 16 Oct 1996 04:08:22 -0000 Received: from slip-76-15.ots.utexas.edu (HELO raven.ots.utexas.edu) (osiris@128.83.254.127) by master.debian.org with SMTP; 16 Oct 1996 04:08:04 -0000 Received: (from osiris@localhost) by raven.ots.utexas.edu (8.7.6/8.7.3) id WAA09212; Tue, 15 Oct 1996 22:56:53 -0500 Sender: osiris@cs.utexas.edu To: Dirk.Eddelbuettel@qed.econ.queensu.ca Cc: Debian Developer , 4792@bugs.debian.org Subject: Re: Bug#4792: Afio fails to restore from remote backup file References: <87enizofph.fsf@raven.ots.utexas.edu> <87aftnobns.fsf@raven.ots.utexas.edu> From: Rob Browning Date: 15 Oct 1996 22:56:53 -0500 In-Reply-To: Dirk.Eddelbuettel@qed.econ.queensu.ca's message of Tue, 15 Oct 96 23:41 EDT Message-ID: <874tjvo92y.fsf@raven.ots.utexas.edu> Lines: 18 X-Mailer: Red Gnus v0.49/Emacs 19.34 Dirk.Eddelbuettel@qed.econ.queensu.ca writes: > No, it doesn't work with localhost, I get the same "Premature input EOF" as > you do. But is that really an issue? I mean, localhost is not a remote host > after all, and it's meant to work remotely. Ah, but on one of the machine pairs where it fails, localhost is not involved. It fails across ethernet. > Do you want me to forward this to the upstream maintainer? Hmm, I was just speculating, and after annotating the calls to close, I discovered that close never fails, so I no longer have a working theory. I think that the actual failure should probably be reported upstream though. -- Rob   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to Rob Browning <osiris@cs.utexas.edu>:
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   Rob Browning <osiris@cs.utexas.edu>  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792) by bugs.debian.org; 16 Oct 1996 05:34:22 +0000 Received: (qmail 23675 invoked from smtpd); 16 Oct 1996 05:34:18 -0000 Received: from slip-76-15.ots.utexas.edu (HELO raven.ots.utexas.edu) (osiris@128.83.254.127) by master.debian.org with SMTP; 16 Oct 1996 05:34:18 -0000 Received: (from osiris@localhost) by raven.ots.utexas.edu (8.7.6/8.7.3) id AAA10393; Wed, 16 Oct 1996 00:23:49 -0500 Sender: osiris@cs.utexas.edu To: 4792@bugs.debian.org Subject: Re: Bug#4792: Afio fails to restore from remote backup file References: <87enizofph.fsf@raven.ots.utexas.edu> From: Rob Browning Date: 16 Oct 1996 00:23:48 -0500 In-Reply-To: Dirk.Eddelbuettel@qed.econ.queensu.ca's message of Tue, 15 Oct 96 22:41 EDT Message-ID: <873ezfo523.fsf@raven.ots.utexas.edu> Lines: 18 X-Mailer: Red Gnus v0.49/Emacs 19.34 OK, after tinkering, I found that the sledgehammer approach of putting a usleep(1) just after the one write call in writeall in afio.c eliminates the problem. Maybe what's happening is either: 1) the server afio finishes writing the archive to the rsh pipe, but the data doesn't get read by the client afio before the server closes the connection causing the client's read to fail. 2) the client is not being patient enough in it's reads. (I think all the files are created blocking, though, so maybe this shouldn't be an issue.) Plausible? -- Rob   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to Rob Browning <osiris@cs.utexas.edu>:
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   Rob Browning <osiris@cs.utexas.edu>  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792) by bugs.debian.org; 16 Oct 1996 06:06:42 +0000 Received: (qmail 31987 invoked from smtpd); 16 Oct 1996 06:06:37 -0000 Received: from slip-76-15.ots.utexas.edu (HELO raven.ots.utexas.edu) (osiris@128.83.254.127) by master.debian.org with SMTP; 16 Oct 1996 06:06:35 -0000 Received: (from osiris@localhost) by raven.ots.utexas.edu (8.7.6/8.7.3) id AAA10818; Wed, 16 Oct 1996 00:56:05 -0500 Sender: osiris@cs.utexas.edu To: 4792@bugs.debian.org Subject: Re: Bug#4792: Afio fails to restore from remote backup file References: <87enizofph.fsf@raven.ots.utexas.edu> <873ezfo523.fsf@raven.ots.utexas.edu> From: Rob Browning Date: 16 Oct 1996 00:56:05 -0500 In-Reply-To: Rob Browning's message of 16 Oct 1996 00:23:48 -0500 Message-ID: <8720ezo3ka.fsf@raven.ots.utexas.edu> Lines: 13 X-Mailer: Red Gnus v0.49/Emacs 19.34 Rob Browning writes: > OK, after tinkering, I found that the sledgehammer approach of putting > a usleep(1) just after the one write call in writeall in afio.c > eliminates the problem. Actually even a single usleep of 100 just before the final exit will work too. I wonder what rsh does if a program quits after it's final write, but before rsh has finished reading that data. -- Rob   Reply sent to Dirk.Eddelbuettel@qed.econ.queensu.ca:
You have marked bug as forwarded.   Dirk.Eddelbuettel@qed.econ.queensu.ca  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792-forwarded) by bugs.debian.org; 16 Oct 1996 15:35:48 +0000 Received: (qmail 30915 invoked from smtpd); 16 Oct 1996 15:35:42 -0000 Received: from knot.queensu.ca (HELO post.QueensU.CA) (root@130.15.126.54) by master.debian.org with SMTP; 16 Oct 1996 15:35:39 -0000 Received: from miles.econ.queensu.ca (free1-slip207.tele.QueensU.CA [130.15.126.207]) by post.QueensU.CA (8.6.12/8.6.10+ASH) with SMTP id LAA01670; Wed, 16 Oct 1996 11:24:54 -0400 Received: by miles.econ.queensu.ca id m0vDXpx-00015qC (Debian /\oo/\ Smail3.1.29.1 #29.37); Wed, 16 Oct 96 11:24 EDT Message-Id: Date: Wed, 16 Oct 96 11:24 EDT To: Koen Holtman Cc: Rob Browning , 4792-forwarded@bugs.debian.org Subject: Re: Bug#4792: Afio fails to restore from remote backup file In-Reply-To: <87enizofph.fsf@raven.ots.utexas.edu> References: <87enizofph.fsf@raven.ots.utexas.edu> X-NoJunk: Please do NOT send junk email, spams or commercial ads to me. From: Dirk.Eddelbuettel@qed.econ.queensu.ca Koen, This email marks Rob's bug report on tob as forwarded to you, the upstream maintainer. When you reply to Rob and/or me, coould you therefore please preserve the Cc: 4792-forwarded@bugs.debian.org which will archive your replies along with the initial mails by Rob and me. Thanks, Dirk -- Dirk Eddelb"uttel http://qed.econ.queensu.ca/~edd   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to Rob Browning <osiris@cs.utexas.edu>:
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   Rob Browning <osiris@cs.utexas.edu>  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792) by bugs.debian.org; 16 Oct 1996 16:17:23 +0000 Received: (qmail 7313 invoked from smtpd); 16 Oct 1996 16:17:19 -0000 Received: from slip-76-15.ots.utexas.edu (HELO raven.ots.utexas.edu) (osiris@128.83.254.127) by master.debian.org with SMTP; 16 Oct 1996 16:17:00 -0000 Received: (from osiris@localhost) by raven.ots.utexas.edu (8.7.6/8.7.3) id LAA13672; Wed, 16 Oct 1996 11:06:25 -0500 Sender: osiris@cs.utexas.edu To: 4792@bugs.debian.org Subject: Re: Bug#4792: Afio fails to restore from remote backup file References: <87enizofph.fsf@raven.ots.utexas.edu> <873ezfo523.fsf@raven.ots.utexas.edu> <8720ezo3ka.fsf@raven.ots.utexas.edu> From: Rob Browning Date: 16 Oct 1996 11:06:24 -0500 In-Reply-To: Rob Browning's message of 16 Oct 1996 00:56:05 -0500 Message-ID: <87ybh6nbb3.fsf@raven.ots.utexas.edu> Lines: 12 X-Mailer: Red Gnus v0.49/Emacs 19.34 Rob Browning writes: > Actually even a single usleep of 100 just before the final exit will > work too. I was premature. It has to be a much larger value to work consistently, and I suspect that it's just masking a more fundamental problem. I wouldn't be suprised if the needed timeout was dependent on machine load. -- Rob   Received: (at 4792-forwarded) by bugs.debian.org; 16 Oct 1996 16:05:18 +0000 Received: (qmail 6889 invoked from smtpd); 16 Oct 1996 16:05:12 -0000 Received: from slip-76-15.ots.utexas.edu (HELO raven.ots.utexas.edu) (osiris@128.83.254.127) by master.debian.org with SMTP; 16 Oct 1996 16:05:11 -0000 Received: (from osiris@localhost) by raven.ots.utexas.edu (8.7.6/8.7.3) id KAA13444; Wed, 16 Oct 1996 10:54:32 -0500 Sender: osiris@cs.utexas.edu To: Dirk.Eddelbuettel@qed.econ.queensu.ca, 4792-forwarded@bugs.debian.org Subject: Re: Bug#4792: Afio fails to restore from remote backup file References: <87enizofph.fsf@raven.ots.utexas.edu> From: Rob Browning Date: 16 Oct 1996 10:54:31 -0500 In-Reply-To: Dirk.Eddelbuettel@qed.econ.queensu.ca's message of Wed, 16 Oct 96 11:24 EDT Message-ID: <87zq1mnbuw.fsf@raven.ots.utexas.edu> Lines: 8 X-Mailer: Red Gnus v0.49/Emacs 19.34 Dirk.Eddelbuettel@qed.econ.queensu.ca writes: > This email marks Rob's bug report on tob as forwarded to you, the upstream Minor correction. This bug report is not for tob, it's for afio. -- Rob   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to kai@khms.westfalen.de (Kai Henningsen):
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   kai@khms.westfalen.de (Kai Henningsen)  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792) by bugs.debian.org; 17 Oct 1996 01:30:26 +0000 Received: (qmail 30921 invoked from smtpd); 17 Oct 1996 01:30:16 -0000 Received: from muenster.westfalen.de (root@193.174.5.2) by master.debian.org with SMTP; 17 Oct 1996 01:30:15 -0000 Received: by muenster.westfalen.de (/\oo/\ Smail3.1.29.1 #29.3) id ; Wed, 16 Oct 96 22:02 MET DST Received: by khms.westfalen.de (CrossPoint v3.1 R/C435); 16 Oct 1996 21:53:34 +0200 Date: 16 Oct 1996 21:06:00 +0200 From: kai@khms.westfalen.de (Kai Henningsen) To: 4792@bugs.debian.org Message-ID: <6IxomJQUcsB@khms.westfalen.de> In-Reply-To: <8720ezo3ka.fsf@raven.ots.utexas.edu> Subject: Re: Bug#4792: Afio fails to restore from remote backup file X-Mailer: CrossPoint v3.1 R/C435 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Organization: Organisation? Me?! Are you kidding? X-No-Junk-Mail: I do not want to get *any* junk mail. Comment: Unsolicited commercial mail will incur an US$100 handling fee per received mail. osiris@cs.utexas.edu (Rob Browning) wrote on 16.10.96 in <8720ezo3ka.fsf@raven.ots.utexas.edu>: > Rob Browning writes: > > > OK, after tinkering, I found that the sledgehammer approach of putting > > a usleep(1) just after the one write call in writeall in afio.c > > eliminates the problem. > > Actually even a single usleep of 100 just before the final exit will > work too. I wonder what rsh does if a program quits after it's final > write, but before rsh has finished reading that data. I dimly recall there was a kernel bug in some early 2.0.x kernels that had similar symptoms. You might check if the same happens with 2.0.22. MfG Kai   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to Dirk.Eddelbuettel@qed.econ.queensu.ca:
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   Dirk.Eddelbuettel@qed.econ.queensu.ca  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792) by bugs.debian.org; 17 Oct 1996 03:36:08 +0000 Received: (qmail 279 invoked from smtpd); 17 Oct 1996 03:36:04 -0000 Received: from knot.queensu.ca (HELO post.QueensU.CA) (root@130.15.126.54) by master.debian.org with SMTP; 17 Oct 1996 03:36:01 -0000 Received: from miles.econ.queensu.ca (free1-slip207.tele.QueensU.CA [130.15.126.207]) by post.QueensU.CA (8.6.12/8.6.10+ASH) with SMTP id XAA27981; Wed, 16 Oct 1996 23:25:13 -0400 Received: by miles.econ.queensu.ca id m0vDj4x-00014YC (Debian /\oo/\ Smail3.1.29.1 #29.37); Wed, 16 Oct 96 23:25 EDT Message-Id: Date: Wed, 16 Oct 96 23:25 EDT To: kai@khms.westfalen.de (Kai Henningsen), 4792@bugs.debian.org Subject: Re: Bug#4792: Afio fails to restore from remote backup file In-Reply-To: <6IxomJQUcsB@khms.westfalen.de> References: <8720ezo3ka.fsf@raven.ots.utexas.edu> <6IxomJQUcsB@khms.westfalen.de> X-NoJunk: Please do NOT send junk email, spams or commercial ads to me. From: Dirk.Eddelbuettel@qed.econ.queensu.ca Kai> I dimly recall there was a kernel bug in some early 2.0.x kernels Kai> that had similar symptoms. You might check if the same happens with Kai> 2.0.22. Well, as it happen edd@miles:~> uname -a Linux miles 2.0.22 #1 Wed Oct 9 01:56:19 EDT 1996 i586 and it failed on my system as seen in the email sent yesterday. -- Dirk Eddelb"uttel http://qed.econ.queensu.ca/~edd   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to Rob Browning <osiris@cs.utexas.edu>:
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   Rob Browning <osiris@cs.utexas.edu>  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792) by bugs.debian.org; 17 Oct 1996 03:38:43 +0000 Received: (qmail 422 invoked from smtpd); 17 Oct 1996 03:38:36 -0000 Received: from slip-51-8.ots.utexas.edu (HELO raven.ots.utexas.edu) (osiris@128.83.251.8) by master.debian.org with SMTP; 17 Oct 1996 03:38:35 -0000 Received: (from osiris@localhost) by raven.ots.utexas.edu (8.7.6/8.7.3) id WAA02823; Wed, 16 Oct 1996 22:27:40 -0500 Sender: osiris@cs.utexas.edu To: kai@khms.westfalen.de (Kai Henningsen) Cc: 4792@bugs.debian.org Subject: Re: Bug#4792: Afio fails to restore from remote backup file References: <6IxomJQUcsB@khms.westfalen.de> From: Rob Browning Date: 16 Oct 1996 22:27:39 -0500 In-Reply-To: kai@khms.westfalen.de's message of 16 Oct 1996 21:06:00 +0200 Message-ID: <87n2xmi82c.fsf@raven.ots.utexas.edu> Lines: 11 X-Mailer: Red Gnus v0.49/Emacs 19.34 kai@khms.westfalen.de (Kai Henningsen) writes: > I dimly recall there was a kernel bug in some early 2.0.x kernels that had > similar symptoms. You might check if the same happens with 2.0.22. Nope, I was running 2.0.21 and then upgraded to 2.0.22 to see if that was the problem, but it fails under both kernels. I've found that the failure seems to be more pronounced on faster machine(s). -- Rob   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to Rob Browning <osiris@cs.utexas.edu>:
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   Rob Browning <osiris@cs.utexas.edu>  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792) by bugs.debian.org; 17 Oct 1996 22:12:42 +0000 Received: (qmail 3866 invoked from smtpd); 17 Oct 1996 22:12:37 -0000 Received: from slip-51-8.ots.utexas.edu (HELO raven.ots.utexas.edu) (osiris@128.83.251.8) by master.debian.org with SMTP; 17 Oct 1996 22:12:36 -0000 Received: (from osiris@localhost) by raven.ots.utexas.edu (8.7.6/8.7.3) id RAA09525; Thu, 17 Oct 1996 17:01:05 -0500 Sender: osiris@cs.utexas.edu To: Koen Holtman Cc: 4792@bugs.debian.org, Dirk.Eddelbuettel@qed.econ.queensu.ca Subject: Re: Bug#4792: Afio premature EOF solution (and patches) References: <87enizofph.fsf@raven.ots.utexas.edu> From: Rob Browning Date: 17 Oct 1996 17:01:04 -0500 In-Reply-To: Dirk.Eddelbuettel@qed.econ.queensu.ca's message of Wed, 16 Oct 96 11:24 EDT Message-ID: <87ybh5cktb.fsf_-_@raven.ots.utexas.edu> Lines: 145 X-Mailer: Red Gnus v0.49/Emacs 19.34 OK, I determined that the problem was that for some reason when you say something like afio -t host:file On fast machines, this would often fail with a premature end of file. This would happen even though the server afio (over the rsh connection) had actually finished writing all the data to the pipe, and had successfully exited. According to the system docs, the kernel should make sure that all the pending data gets delivered, but the client afio wasn't getting it. I never could figure out why this was happening, because AFAICT everyone was playing nice and calling close on their file descriptors, and calling exit, not _exit. I began to suspect that either there was a kernel problem (doubtful since other programs don't have this behavior) or, more likely an rsh problem. My guess was that rsh was not transferring all the data before quitting when the server quit. Maybe it's calling shutdown on the pipe rather than close... Anyway, all that's interesting, but doesn't fix the problem. To fix the problem, I just augmented the protocol so that when afio launches another copy of itself (as an archive server) across an rsh connection, the server doesn't quit until it receives the string "" from the client. So the client afio reads the archive from the server, and then sends the "" message. This appears to have eliminated the problem. Here are the context diffs to afio.c. Feel free to mangle them in whatever way you see fit, and let me know if there's a problem with them, and I'll try to fix it: --- ../afio-2.4.2.dist/afio.c Sun Jan 21 13:39:35 1996 +++ afio.c Thu Oct 17 16:55:00 1996 @@ -152,6 +152,11 @@ unsigned long bytepos; /* position of first byte of current file */ STATIC char *controlscript=NULL; /* script to pipe control files to */ + +STATIC int remote_fd = -1; +STATIC char remote_dir = 'x'; +STATIC void clientwait(int fd); + main (ac, av) int ac; reg char **av; @@ -486,6 +491,7 @@ timewait, arvolume, ctime (&timedone)); } nextclos (); + if(fn == copyin) clientwait(STDIN); goodbye (anycorrupt); /* NOTREACHED */ } @@ -1851,10 +1857,40 @@ arfd = -1; } areof = 0; - if (arname && *arname == '!') + + if (arname && *arname == '!') { + if(remote_dir == 'I') { + char message[] = ""; + writeall(remote_fd, message, strlen(message) + 1); + } pipewait (); + } + } + +STATIC void +clientwait (int fd) { + char message[] = ""; + char buff[256] = ""; + int msglen = strlen(message) + 1; + int total = 0; + + while(total < msglen) { + int result = read(fd, (buff + total), msglen - total); + if(result > 0) + total += result; + else { + fatal("archive server", "Didn't get proper from client"); + } + + } + if(strncmp(message, buff, msglen) != 0) { + fatal("archive server", "Didn't get proper from client"); + } +} + + /* * nextopen() * @@ -3134,24 +3170,10 @@ *av++ = "-c"; *av++ = arname + 1; *av = NULL; - if (mode) - { - VOID close (pfd[1]); - VOID close (STDIN); - VOID dup (pfd[0]); - VOID close (pfd[0]); - VOID close (STDOUT); - VOID open ("/dev/null", O_WRONLY); - } - else - { - VOID close (STDIN); - VOID open ("/dev/null", O_RDONLY); - VOID close (pfd[0]); - VOID close (STDOUT); - VOID dup (pfd[1]); - VOID close (pfd[1]); - } + + VOID dup2 (pfd[0], STDIN); + VOID dup2 (pfd[1], STDOUT); + if (ttyf >= 0) VOID close (ttyf); VOID execvp (arg[0], arg); @@ -3177,12 +3199,14 @@ pipechld (mode, pfd); if (mode) { - VOID close (pfd[0]); + remote_dir = 'O'; + remote_fd = pfd[0]; return (pfd[1]); } else { - VOID close (pfd[1]); + remote_dir = 'I'; + remote_fd = pfd[1]; return (pfd[0]); } } -- Rob   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to Dirk.Eddelbuettel@qed.econ.queensu.ca:
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   Dirk.Eddelbuettel@qed.econ.queensu.ca  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792) by bugs.debian.org; 18 Oct 1996 02:14:09 +0000 Received: (qmail 9088 invoked from smtpd); 18 Oct 1996 02:14:03 -0000 Received: from knot.queensu.ca (HELO post.QueensU.CA) (root@130.15.126.54) by master.debian.org with SMTP; 18 Oct 1996 02:14:01 -0000 Received: from miles.econ.queensu.ca (free1-slip213.tele.QueensU.CA [130.15.126.213]) by post.QueensU.CA (8.6.12/8.6.10+ASH) with SMTP id WAA11387; Thu, 17 Oct 1996 22:03:11 -0400 Received: by miles.econ.queensu.ca id m0vE3xy-00019eC (Debian /\oo/\ Smail3.1.29.1 #29.37); Thu, 17 Oct 96 21:43 EDT Message-Id: Date: Thu, 17 Oct 96 21:43 EDT To: Rob Browning , 4792@bugs.debian.org Cc: Koen Holtman Subject: Re: Bug#4792: Afio premature EOF solution (and patches) In-Reply-To: <87ybh5cktb.fsf_-_@raven.ots.utexas.edu> References: <87enizofph.fsf@raven.ots.utexas.edu> <87ybh5cktb.fsf_-_@raven.ots.utexas.edu> X-NoJunk: Please do NOT send junk email, spams or commercial ads to me. From: Dirk.Eddelbuettel@qed.econ.queensu.ca That looks like a nice idea to send an explicit "end of data" signal using the "". But doesn't afio then loose the capability of using cpio(1) archives? -- Dirk Eddelb"uttel http://qed.econ.queensu.ca/~edd   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to Rob Browning <osiris@cs.utexas.edu>:
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   Rob Browning <osiris@cs.utexas.edu>  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792) by bugs.debian.org; 18 Oct 1996 03:04:32 +0000 Received: (qmail 26611 invoked from smtpd); 18 Oct 1996 03:04:27 -0000 Received: from slip-51-8.ots.utexas.edu (HELO raven.ots.utexas.edu) (osiris@128.83.251.8) by master.debian.org with SMTP; 18 Oct 1996 03:04:24 -0000 Received: (from osiris@localhost) by raven.ots.utexas.edu (8.7.6/8.7.3) id VAA10932; Thu, 17 Oct 1996 21:53:26 -0500 Sender: osiris@cs.utexas.edu To: Dirk.Eddelbuettel@qed.econ.queensu.ca Cc: Rob Browning , 4792@bugs.debian.org, Koen Holtman Subject: Re: Bug#4792: Afio premature EOF solution (and patches) References: <87enizofph.fsf@raven.ots.utexas.edu> <87ybh5cktb.fsf_-_@raven.ots.utexas.edu> From: Rob Browning Date: 17 Oct 1996 21:53:26 -0500 In-Reply-To: Dirk.Eddelbuettel@qed.econ.queensu.ca's message of Thu, 17 Oct 96 21:43 EDT Message-ID: <87vic9q8yh.fsf@raven.ots.utexas.edu> Lines: 20 X-Mailer: Red Gnus v0.49/Emacs 19.34 Dirk.Eddelbuettel@qed.econ.queensu.ca writes: > That looks like a nice idea to send an explicit "end of data" signal using > the "". > > But doesn't afio then loose the capability of using cpio(1) archives? Hmm, I don't think so. The is essentially "out of band" data since it goes over the other half of the pipe which was completely unused before. In fact the pre-patch afio just closed that half of the pipe immediately. Here's what happens: afio is reading an archive over a pipe from another afio that's writing it to its stdout. The is sent by the first afio, the reader, when it's finished reading, to the second afio's stdin where it started waiting after it finished sending the archive. -- Rob   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Acknowledgement sent to koen@win.tue.nl (Koen Holtman):
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   koen@win.tue.nl (Koen Holtman)  Sorry, this message was lost when this bug report was restored from a backup.   Received: (at 4792) by bugs.debian.org; 18 Oct 1996 08:24:20 +0000 Received: (qmail 25909 invoked from smtpd); 18 Oct 1996 08:24:18 -0000 Received: from wsooti23.win.tue.nl (koen@131.155.70.147) by master.debian.org with SMTP; 18 Oct 1996 08:24:18 -0000 Received: by wsooti23.win.tue.nl (8.7.1/1.45) id KAA05807; Fri, 18 Oct 1996 10:13:10 +0200 (MET DST) From: koen@win.tue.nl (Koen Holtman) Message-Id: <199610180813.KAA05807@wsooti23.win.tue.nl> Subject: Re: Bug#4792: Afio premature EOF solution (and patches) To: osiris@cs.utexas.edu (Rob Browning) Date: Fri, 18 Oct 1996 10:13:08 +0200 (MET DST) Cc: Dirk.Eddelbuettel@qed.econ.queensu.ca, osiris@cs.utexas.edu, 4792@bugs.debian.org, koen@win.tue.nl In-Reply-To: <87vic9q8yh.fsf@raven.ots.utexas.edu> from "Rob Browning" at Oct 17, 96 09:53:26 pm X-Mailer: ELM [version 2.4 PL23] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8bit Rob Browning: > >Dirk.Eddelbuettel@qed.econ.queensu.ca writes: > >> That looks like a nice idea to send an explicit "end of data" signal using >> the "". >> >> But doesn't afio then loose the capability of using cpio(1) archives? > >Hmm, I don't think so. The is essentially "out of band" data >since it goes over the other half of the pipe which was completely >unused before. In fact the pre-patch afio just closed that half of >the pipe immediately. > >Here's what happens: afio is reading an archive over a pipe from >another afio that's writing it to its stdout. The is sent by >the first afio, the reader, when it's finished reading, to the second >afio's stdin where it started waiting after it finished sending the >archive. I may put the patch in the next public release of afio -- I'll have to think about it. I think the patch masks a deeper problem, but a masking a problem is better than doing nothing. By the way, I don't want to rule out that it is a kernel problem: I've seen it do strange things before when processes were exiting, without there being an rsh involved. The most likely suspect is the shell, though. >-- >Rob > Koen.   Information forwarded to debian-devel@lists.debian.org, Dirk Eddelbuettel <edd@qed.econ.queensu.ca>:
Bug#4792; Package afio.   debian-devel@lists.debian.orgDirk Eddelbuettel <edd@qed.econ.queensu.ca>  Sorry, this message was lost when this bug report was restored from a backup.   Forwarded to Koen Holtman <koen@win.tue.nl>; Maintainer for afio is Dirk Eddelbuettel <edd@debian.org>.

Message received at 4792@bugs.debian.org:


Received: (at 4792) by bugs.debian.org; 18 Oct 1996 14:46:02 +0000
Received: (qmail 938 invoked from smtpd); 18 Oct 1996 14:46:00 -0000
Received: from slip-51-8.ots.utexas.edu (HELO raven.ots.utexas.edu) (osiris@128.83.251.8)
  by master.debian.org with SMTP; 18 Oct 1996 14:45:59 -0000
Received: (from osiris@localhost) by raven.ots.utexas.edu (8.7.6/8.7.3) id JAA14236; Fri, 18 Oct 1996 09:35:03 -0500
Sender: osiris@cs.utexas.edu
To: koen@win.tue.nl (Koen Holtman)
Cc: 4792@bugs.debian.org
Subject: Re: Bug#4792: Afio premature EOF solution (and patches)
References: <199610180813.KAA05807@wsooti23.win.tue.nl>
From: Rob Browning <osiris@cs.utexas.edu>
Date: 18 Oct 1996 09:35:01 -0500
In-Reply-To: koen@win.tue.nl's message of Fri, 18 Oct 1996 10:13:08 +0200 (MET DST)
Message-ID: <87pw2gqr1m.fsf@raven.ots.utexas.edu>
Lines: 45
X-Mailer: Red Gnus v0.49/Emacs 19.34

koen@win.tue.nl (Koen Holtman) writes:

> I may put the <quit> patch in the next public release of afio -- I'll
> have to think about it.  I think the patch masks a deeper problem, but
> a masking a problem is better than doing nothing.

I agree.  I wasn't able to rule out a kernel problem, but I needed
something that would work now.

> The most likely suspect is the shell, though.

I don't know if I mentioned it, but I did some other experiments that
might prove interesting.  I figured out the rsh command that was being
used to launch the remote afio, and I piped that command into a test
program:

  rsh localhost 'afio -I -b 5120 -c 1 /tmp/test.afio' | checker-program

My initial version of checker-program was written in perl, and just
opened stdin and consumed it to EOF, printing the total number of
bytes at the end.  I then used select to add some subsecond delays at
various points to see if I could get the afio problem to appear.  It
never did.

Since I thought it might be something perl was doing under the hood, I
replaced checker-program with a very simple C program that did the
same thing, and used usleep to try a very wide range of delays.  I
never saw the problem.  Additionally, if you use something like "wc",
there's no problem either.

  rsh localhost 'afio -I -b 5120 -c 1 /tmp/test.afio' | wc -c

Finally, I tried modifying the rsh string inside afio so that afio
would insert a copy of "tee" before going out over the net like this:

  rsh localhost 'afio -I -b 5120 -c 1 /tmp/test.afio | tee /tmp/foo'

This also eliminated the problem.

All of this didn't really help tell me anything substantial, but I
figured someone else might make something of it.

Thanks
--
Rob


Acknowledgement sent to Rob Browning <osiris@cs.utexas.edu>:
Extra info received and forwarded to list. Copy sent to Dirk Eddelbuettel <edd@qed.econ.queensu.ca>.   Rob Browning <osiris@cs.utexas.edu>  Sorry, this message was lost when this bug report was restored from a backup.