Contributed by merdely on from the stayin-alive dept.
Recently, Marc Espie (espie@) reminded us on ports@ that FTP is broken. "So a file transfer goes like this: send some commands, open a data connection, and transfer the file... during the transfer, the control connection stays silent... that is, totally silent. And then, we come back up, and there is NO control connection left. Over the lousy five minutes it took us to transfer the file," the network expired the control connection.
To attempt to make ftp(1) work better, Marc committed code to:
implement a `keep-alive' option that sends bytes over an inactive connection. The FTP protocol provides us with a NOOP operation that is perfectly suitable for that, and so far servers are happy with it. Sending the command slowly is an idea I borrowed from spamd. No change for people not using the option, so it can't break normal ftp.
A new "-k" option was added to the ftp command:
-k seconds Sends a byte after each seconds period over the control connec- tion during long transfers, so that incorrectly configured net- work equipment won't agressively drop it. The FTP protocol sup- ports a NOOP command that can be used for that purpose. This as- sumes the FTP server can deal with extra commands coming over the control connection during a transfer. Well-behaved servers queue those commands, and process them after the transfer.
As Marc then announced on ports@:
[The -k option] doesn't solve the other ftp problem, that in many cases you may have an intermediate proxy that does not understand telnet correctly, and so your urgent ABORTs won't get through correctly, and it will take forever for pkg_add to close connections early (we solved this in our ftp-proxy a few years ago).
So, from pkg_add, you can try out to set the env variable FTP_KEEPALIVE if you think you have the problem... one common symptom is to have the installation of openoffice hanging at the end.
(Comments are closed)
By Anonymous Coward (85.97.21.103) on
Comments
By Anonymous Coward (84.56.88.23) on
You aren't serious about that, are you?
Comments
By Anonymous Coward (85.102.220.197) on
>
> You aren't serious about that, are you?
>
I was serious and I would like to know what I was missing.
Comments
By Anonymous Coward (202.81.60.133) on
> >
> > You aren't serious about that, are you?
> >
>
> I was serious and I would like to know what I was missing.
perhaps your firewall blocks outgoing connection to port >1024 which explains why setting FTPMODE to active solved the problem (as I also have same problem).
By Cabal (Cabal) Cabal on http://www.enginuity.org/
Comments
By Marc Espie (213.41.185.88) espie@openbsd.org on
Two things:
- increasing number of issues with big packages and pkg_add. I guess it's due to more people using pkg_add over ftp.
- very lousy network at my current work, due to people sitting on public IP adresses, nat being overused, and aggressively expiring connections.
Those are `long term' fixes. In many cases, it's fairly hard to test and fix issues you don't see.
For instance, this has nothing to do with active ftp. There are at least three distinct issues with ftp in there !
- active vs. passive ftp
- proxy not relaying telnet ABORT correctly
- nat firewalls dropping control connections.
And maybe more.
Anyways, this is just a band-aid. If someone wants to *look* at ftp and actually fix the code (probably rewrite it), they're welcome.
Comments
By Cabal (Cabal) on http://www.enginuity.org/
By Anonymous Coward (208.146.43.174) on
Comments
By sthen (85.158.45.32) on
> not seeing where ftp-proxy deals with the telnet commands as you
> mentioned....
Unless I'm mistaken it's because /usr/libexec/ftp-proxy uses
NAT/RDR - the data connections are passed straight through,
they don't need special handling in the proxy.
Comments
By sthen (85.158.45.32) on
> > not seeing where ftp-proxy deals with the telnet commands as you
> > mentioned....
>
> Unless I'm mistaken it's because /usr/libexec/ftp-proxy uses
> NAT/RDR - the data connections are passed straight through,
> they don't need special handling in the proxy.
ugh, s/libexec/sbin/ of course
By Anonymous Coward (208.146.43.174) on
> > not seeing where ftp-proxy deals with the telnet commands as you
> > mentioned....
>
> Unless I'm mistaken it's because /usr/libexec/ftp-proxy uses
> NAT/RDR - the data connections are passed straight through,
> they don't need special handling in the proxy.
those commands are sent over the control connection not the data.
By Marc Espie (213.41.185.88) espie@openbsd.org on
This was some fix I did in the *previous* ftp-proxy (read cvs log, you'll see this was completely rewritten).
I haven't actually looked at the new one... I have no idea whether it deals correctly with urgent stuff (which is what this is all about). I'm bound to update my soekris one of these days, now that I've got a big enough flash available to get away from flashdist, but I don't fancy reinstalling my gateway...
Comments
By Anonymous Coward (71.216.4.2) on
>
> This was some fix I did in the *previous* ftp-proxy (read cvs log, you'll see this was completely rewritten).
>
> I haven't actually looked at the new one... I have no idea whether it deals correctly with urgent stuff (which is what this is all about). I'm bound to update my soekris one of these days, now that I've got a big enough flash available to get away from flashdist, but I don't fancy reinstalling my gateway...
>
>
ah ok, i see it. thanks.
well it doesn't appear that the new proxy has any of that.