Back to home page

DOS ain't dead

Forum index page

Log in | Register

Search:  
Back to index page
Thread view  Board view
mbbrutman

Homepage

Washington, USA,
21.01.2010, 18:06
 

mTCP DOS TCP/IP apps for small machines (Announce)

Post reply

I have been a DOS user for 26 years now and a semi-serious developer for the last five years now. My project has been a TCP/IP stack optimized for the original DOS machines - the PC, XT, Jr, AT, etc. (It is optimized for both speed and space - my target machine is an 8088 with 256K of RAM.)

In the last year it has gotten to the point where I can offer usable applications, including a DHCP client, IRC client, Telnet, FTP, Ping and Netcat. I am continuing to work on enhancements for these apps and more apps as the creative impulse hits me.

In general, all that you need is a 192K or 256K machine and an Ethernet card with a packet driver. DOS 2 or better will work fine. The applications will run on better machines too, and will even run in DOSBox if you get the HAL9000 build that has Ethernet support. The applications include the parts of the TCP/IP stack that they need, so there is no TSR to deal with. Just load the packet driver, use the DHCP client to get an address, and start running.

The applications can be found here: http://brutman.com/mTCP/

I will disclose that this is not open source (yet). I am willing to talk to people about code in private. It's not all done yet so I'd like to keep some control before releasing my terror on the world. ;-0

If you have comments or requests I try to be responsive.


Thanks,
Mike
Japheth

Homepage

Germany (South),
22.01.2010, 09:47

@ mbbrutman
 

mTCP DOS TCP/IP apps for small machines

Post reply

> I will disclose that this is not open source (yet). I am willing to talk
> to people about code in private. It's not all done yet so I'd like to
> keep some control before releasing my terror on the world. ;-0
>
> If you have comments or requests I try to be responsive.
Thanks! AFAICS one can just download some applications, which include the statically linked TCP code. So the only tests which can be done are to try out these applications? Perhaps there's a possibility to provide at least a static library and a (small) documentation how your TCP API is supposed to be used? No big problem if the code is restricted to the Borland C v3.0 compiler.

---
MS-DOS forever!
mbbrutman

Homepage

Washington, USA,
22.01.2010, 16:56

@ Japheth
 

mTCP DOS TCP/IP apps for small machines

Post reply

> Thanks! AFAICS one can just download some applications, which include the
> statically linked TCP code. So the only tests which can be done are to try
> out these applications? Perhaps there's a possibility to provide at least a
> static library and a (small) documentation how your TCP API is supposed to
> be used? No big problem if the code is restricted to the Borland C v3.0
> compiler.
Japheth,

I am willing to provide a static library and a header file. In the past it has actually been two libraries - the TCP/IP library that the applications link directly against, and a second (smaller) library that made the first library more 'opaque' to hide some implementation details. I thought this was a less error prone approach for new users, but it was not exactly elegant.

If you are willing to take a look I would be more than happy to take your suggestions on improving it. I will review it once more and add some much needed comments. :-)

The development environment is Borland Turbo C++ 3.0 for DOS. I am also interested in exploring Watcom as a development environment and doing a port if people find Borland Turbo C++ 3.0 for DOS too restrictive. (I figured that this forum would be a good place to learn what I need above the other development environments.)


-Mike

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
mbbrutman

Homepage

Washington, USA,
07.02.2010, 16:35

@ mbbrutman
 

mTCP DOS TCP/IP apps for small machines

Post reply

I have to offer my apologies - I keep finding things that I want to fix!

(It is stable. But upon looking at it, I keep finding things that I don't want. Even the thought of open sourcing this is making my code better!)


Mike

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
mbbrutman

Homepage

Washington, USA,
14.03.2010, 03:22

@ Japheth
 

Forget the library .. time to open-source the code?

Post reply

> > I will disclose that this is not open source (yet). I am willing to
> talk
> > to people about code in private. It's not all done yet so I'd like to
> > keep some control before releasing my terror on the world. ;-0
> >
> > If you have comments or requests I try to be responsive.
>
> Thanks! AFAICS one can just download some applications, which include the
> statically linked TCP code. So the only tests which can be done are to try
> out these applications? Perhaps there's a possibility to provide at least a
> static library and a (small) documentation how your TCP API is supposed to
> be used? No big problem if the code is restricted to the Borland C v3.0
> compiler.
It might be time to take a different approach .. I should probably clean the code up and ship source code, not just libraries. I would not want to use a library without source, and other people probably don't want to either.

I would like to open source it but retain my personal copyright. I think that the GPL version three allows for this. I realize that some people might use the code in a way that might be irritating to me, but there is probably more benefit in making it open than not.

What licenses have people here used for their software? Have their been any 'gotchas'? (I am very familiar with open source software, but have never tried to maintain my own project.)


-Mike

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
Rugxulo

Homepage

Usono,
14.03.2010, 03:56

@ mbbrutman
 

Forget the library .. time to open-source the code?

Post reply

> It might be time to take a different approach .. I should probably clean
> the code up and ship source code, not just libraries. I would not want to
> use a library without source, and other people probably don't want to
> either.
There is no easy answer. I don't see any harm in posting sources, esp. if you don't intend to continue development. (For instance, ADOM doesn't want secrets spoiled or too many forks a la Angband, hence why it's still closed.)

> I would like to open source it but retain my personal copyright. I think
> that the GPL version three allows for this.

Not sure what you mean by this. You are always retaining copyright, but with GPL you are letting people use your code in any fashion as long as they don't publicly distribute it without (all total) sources. (GPLv3 combines GPLv2 and LGPL 2.1 into one license, or so I'm told.)

> I realize that some people
> might use the code in a way that might be irritating to me, but there is
> probably more benefit in making it open than not.
In what way? What are you afraid of happening? Both GPLv2 and v3 actually allow others to commercially sell your code. What do you want to happen to it?

> What licenses have people here used for their software? Have their been
> any 'gotchas'? (I am very familiar with open source software, but have
> never tried to maintain my own project.)
FYI, the GPLv3 is heavily shunned by *BSD and its fans (although I don't know why). They stick to GCC 4.2.1 or less (and want Clang or PCC to succeed) due to that. But most main GNU projects themselves are GPLv3 now. Even FreeDOS proper has a heavy GPL bias, esp. for "BASE".

Personally, I think worrying about licenses is a waste of time. But it's your call.

---
Know your limits.h
mbbrutman

Homepage

Washington, USA,
14.03.2010, 15:01

@ Rugxulo
 

Forget the library .. time to open-source the code?

Post reply

> There is no easy answer. I don't see any harm in posting sources, esp. if
> you don't intend to continue development. (For instance, ADOM doesn't want
> secrets spoiled or too many forks a la Angband, hence why it's still
> closed.)
I fully expect to continue development, so having the code forked is a concern for me. But I realize that giving people only binaries to work with might not be satisfying, and certainly doesn't allow anybody to help improve the quality of the code.

> > I would like to open source it but retain my personal copyright. I
> think
> > that the GPL version three allows for this.
>
> Not sure what you mean by this. You are always retaining copyright, but
> with GPL you are letting people use your code in any fashion as long as
> they don't publicly distribute it without (all total) sources. (GPLv3
> combines GPLv2 and LGPL 2.1 into one license, or so I'm told.)
My intent here is to ensure that I can always do what I want with my code, and the original version is recognized as something that came from my brain. :-)

> > I realize that some people
> > might use the code in a way that might be irritating to me, but there
> is
> > probably more benefit in making it open than not.
>
> In what way? What are you afraid of happening? Both GPLv2 and v3 actually
> allow others to commercially sell your code. What do you want to happen to
> it?
GPL2 and GPL3 allow for commercial use, but you still have to distribute the source code. I would prefer that the code was marked "for hobbyist use only" because I really am not thrilled about people making money off of my hobby time, but I doubt there is much I can do about that. (Then again, a new DOS TCP/IP stack is not going to have a lot of commercial value.)


> > What licenses have people here used for their software? Have their
> been
> > any 'gotchas'? (I am very familiar with open source software, but have
> > never tried to maintain my own project.)
>
> FYI, the GPLv3 is heavily shunned by *BSD and its fans (although I don't
> know why). They stick to GCC 4.2.1 or less (and want Clang or PCC to
> succeed) due to that. But most main GNU projects themselves are GPLv3 now.
> Even FreeDOS proper has a heavy GPL bias, esp. for "BASE".
>
> Personally, I think worrying about licenses is a waste of time. But it's
> your call.
I think the license matters quite a bit. BSD style licenses bother me because the commercial users do not have to release their code. That goes against the nature of what I'm trying to do here, which is release the code for hobbyist use.

My employer is going to make me jump through hoops to release this code, which is another concern. (They want to ensure that anybody who is contributing to open source is not exposing their intellectual property. I'm quite sure my code is safe, but I will have to have it reviewed.)



Mike

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
Arjay

27.06.2010, 22:27

@ mbbrutman
 

mTCP new release 27/June/2010 - now ported to Open Watcom

Post reply

I spotted that Mike has released an updated version mTCP today (27/June/2010):

June 2010: Ported code to Open Watcom - stronger, better, faster!

mTCP homepage


Excellent news!! One very minor issue: README.TXT uses Unix line termination!
Rugxulo

Homepage

Usono,
28.06.2010, 06:30

@ Arjay
 

mTCP new release 27/June/2010 - now ported to Open Watcom

Post reply

> Excellent news!! One very minor issue: README.TXT uses Unix line
> termination!
So? ;-) Don't use Notepad, use something else, e.g. Wordpad. (Even Win95 EDIT seems to read it okay but writes out in CRLF, so that's a quick converter. FreeDOS EDIT does the exact same, oddly. Other editors, like TDE, usually preserve line endings for you.)

EDIT: Okay, so MS edlin can't handle it at all (corrupts, crashes). I think this proves that most editors can handle it. If not, here's a (lame!) TP program I whipped up the other day:


program u2d(nixfile,dosfile); { mainly for TP 5.5 or TMT 3.90 }
var nixfile:file of char;dosfile:text;ch,oldch:char;
begin
if paramcount=0 then
  begin writeln('u2d nixfile dosfile'); halt end
else
  begin assign(nixfile,paramstr(1)); assign(dosfile,paramstr(2)) end;
reset(nixfile); rewrite(dosfile);
repeat oldch := ch; read(nixfile,ch);
  if (ch = chr(10)) and (oldch <> chr(13)) then writeln(dosfile)
  else write(dosfile,ch);
until eof(nixfile);
close(nixfile); close(dosfile);
end.
DOS386

28.06.2010, 12:05

@ Rugxulo
 

mTCP new release 27/June/2010 - now ported to Open Watcom

Post reply

> I spotted that Mike has released an updated version mTCP today (27/June/2010):
> June 2010: Ported code to Open Watcom - stronger, better, faster!
:-)

> One very minor issue: README.TXT uses Unix line termination
Try FASM IDE :-)

> Don't use Notepad, use something else, e.g. Wordpad
WtF ???

If not, here's a (lame!) TP program I whipped up the other day

COOL :-)

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***
Arjay

28.06.2010, 14:22

@ DOS386
 

mTCP new release 27/June/2010 - now ported to Open Watcom

Post reply

> > Don't use Notepad, use something else, e.g. Wordpad
 Tsk, I don't know I make a minor complaint after I spotted Mike was using the wrong text termination for DOS and you assume I was using notepad!!! :no:

> WtF ???
 What is this Wordpad anyway? :confused:

> Try FASM IDE :-)
 heh. Can't beat programmers editors. Now hands up if you've ever been really sad and used debug as a text editor.... :ok:

> If not, here's a (lame!) TP program I whipped up the other day
 Thanks. Kudos for releasing yours (I never have), always good to have more source around. Still on this theme here is a similar Q&D TP cludge I wrote many years ago for fixing XML files which had a mix of line terminations:

Var
   TextFile:Text;
   OneLine:String;
   OneChar:Char;

Begin
  Assign(TextFile, paramstr(1));
  Reset(TextFile);
  While NOT EOF(TextFile) do
    Begin
      Read(TextFile, OneChar);
      Case OneChar of
        #10:{do nothing};
        #13:{do nothing};
        '>':Writeln('>');
      else Write(OneChar);
      end;
    End;
  Close(TextFile);
End.
mbbrutman

Homepage

Washington, USA,
01.07.2010, 07:08

@ Arjay
 

mTCP new release 27/June/2010 - now ported to Open Watcom

Post reply

Sorry about the newline chars .. I probably wrote that file using VI under Cygwin. I'll get that fixed for the next release. :-)

I've had a small flurry of updates recently. The biggest news is that I ported everything to Open Watcom from Turbo C++. OW generates much better code and I can run the compiler natively under Windows, which improves my productivity tremendously. (I had been developing on a real 386-40 before. Now that is just for testing.)

For the most part the apps are a little smaller and the speed is a little better. For things like IRC and telnet the speed doesn't matter much, but it is quite measurable/noticeable on FTP.

I'm still planning to open source this sometime. I just want to get it relatively clean and complete first.

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
Laaca

Homepage

Czech republic,
01.07.2010, 13:14

@ mbbrutman
 

mTCP new release 27/June/2010 - now ported to Open Watcom

Post reply

It is great you are continuing to develop your utilities.

If you plan to extend the network set it would be great to create some interactive FTP client. It means no writting commands like PUT or GET but something like Norton commander (just fileselector) to easily manage the files. We have no such native DOS aplication, only FAR manager via HX extender.

---
DOS-u-akbar!
mbbrutman

Homepage

Washington, USA,
01.07.2010, 16:06

@ Laaca
 

mTCP new release 27/June/2010 - now ported to Open Watcom

Post reply

> It is great you are continuing to develop your utilities.
>
> If you plan to extend the network set it would be great to create some
> interactive FTP client. It means no writting commands like PUT or GET but
> something like Norton commander (just fileselector) to easily manage the
> files. We have no such native DOS aplication, only FAR manager via HX
> extender.
If you have looked at my apps you may have noticed that my user interfaces are spartan. Almost to the point of stripped clean. :-) The command line FTP client works fairly well - it transfer hundreds of files per mget/mput, and it can reliably transfer gigabytes per file.

I'd be happy to work with anybody who wants to tackle the user interface. It's not my strongpoint, and I have a bit of work ahead of me just keeping the code functional and performing well. (My next big task is to add support for IP fragments.)


Mike

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
DOS386

01.07.2010, 13:20

@ mbbrutman
 

mTCP new release 27/June/2010 - now ported to Open Watcom

Post reply

> I've had a small flurry of updates recently. The biggest news is that I ported
> everything to Open Watcom from Turbo C++. OW generates much better
Thanks for your work and sorry for my previous lack of feedback :-)

There are a few tiny issues:

[image]

> 8088 processor or better
> Somewhere between 96KB and 160KB available RAM
COOL :-)

> The path and name of the configuration file is specified by the 'MTCPCFG'
> environment variable which you need to set before the application starts
Unnecessary, if no var, just assume "MBTCP.INI" or whatever in current directory.

> Here is an example of a minimum configuration file for mTCP:
> PACKETINT 0x60
Why ? Packet driver INT can be detected easily and safely. So if not given, just autodetect it.

> HOSTNAME PCjr
What's the goal of this ?

> ascii - Set transfer type to 'ASCII'
Unless I'm missing something crucial this "Text" or "ASCII" mode is inherently faulty and should never have been invented :-(

> image - Set transfer type to 'image' or 'binary'
Why not "binary" ? Why isn't binary the default (see shot, 2.) ? The "image" command doesn't work (see shot, 4.).

I'll post more later.

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***
mbbrutman

Homepage

Washington, USA,
01.07.2010, 16:03

@ DOS386
 

mTCP new release 27/June/2010 - now ported to Open Watcom

Post reply

On the file transfer, it looks like you didn't specify binary before starting the transfer. So LFs (0x0a) were converted to CR/LFs (0x0a 0x0d) by the server during the transfer. There isn't much I can do about that without doing something ugly like stuffing a 'BIN' command in the command ahead of the transfer first. I'm going to call that one a user error. :-)

I don't know why the server closed the control connection. It's possible that it timed out, and on the first command that you sent the dropped connection was noted. From the statistics at the end I can see no retransmits. I can see an enormous amount of dropped packets though - you are losing more than 10% of the packets into your machine. Are you running something else on that machine in the background? What kind of machine is it? (My PCjr and other older machines don't drop packets like this unless I have turned on full tracing, and they have the slowest hard drives in the universe.)


On the config file comments:

I don't like defaults. It's not harsh to have the person set an environment variable to get to the configuration file. I could go looking for the packet interrupt too, but in the unlikely event that the person has two cards they would need to specify the packet interrupt anyway. Also, any extra code that I spend on initialization is stuck in memory long after it is needed so I prefer to keep the initialization code to a minimum.

Don't have a PCjr? Nobody is perfect. Set your own hostname then! I think it is used for DHCP - none of my other apps use it (yet).


BIN vs. ASCII existed long before I ever touched a keyboard. FTP was designed for data transfer across heterogeneous machines, and the only thing that was agreed upon back then was that nobody knew what a newline meant. Which is why Telnet specifies the 'CF/LF' pair and allows each side to translate as needed. FTP has a much richer command set and file transfer modes than anybody probably implements anymore. BIN vs. ASCII is just a holdover that they all still honor even if the other ones are not implemented.

The server determines the default, not the client. You need to send 'BIN' first. I can add 'IMAGE' as an alias.

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
Laaca

Homepage

Czech republic,
01.07.2010, 19:16

@ mbbrutman
 

mTCP new release 27/June/2010 - now ported to Open Watcom

Post reply

I have idea about configuration files. It would be great if mTCP could process and load configuration from WATTCP.CFG file.
I know - it is a part of another implementation and your stack has nothing to do with it.
BUT there is many Wattcp/watt32 programs written, all use this file so it could referred as a "semistandard".
Noram user doesn't will to examine whether this or that application uses Watt32 or mTCP. He knows that DOS internet is configured via wattcp.cfg file.

---
DOS-u-akbar!
mbbrutman

Homepage

Washington, USA,
03.07.2010, 16:04

@ Laaca
 

mTCP new release 27/June/2010 - now ported to Open Watcom

Post reply

> I have idea about configuration files. It would be great if mTCP could
> process and load configuration from WATTCP.CFG file.
> I know - it is a part of another implementation and your stack has nothing
> to do with it.
> BUT there is many Wattcp/watt32 programs written, all use this file so it
> could referred as a "semistandard".
> Noram user doesn't will to examine whether this or that application uses
> Watt32 or mTCP. He knows that DOS internet is configured via wattcp.cfg
> file.
That's not a bad idea, but I'd rather be trying to find ways to reduce the amount of initialization code, not adding to it. Any initialization code that I add has to reside in memory for the duration of the program. (I had thought about using overlays at one point just so that I could overlay the initialization code.)

The todo list looks something like this:

- Add wget to make getting updates from the web site easier.
- Add support for IP fragments
- Rework the IRCjr user interface; the registration code needs to be rewritten, multi-channel support and multi-server support would be great.
- Write a small ntp client for machines without clock/calendars.


Mike

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
DOS386

14.07.2010, 03:51

@ mbbrutman
 

mTCP new release 27/June/2010 - now ported to Open Watcom

Post reply

> On the file transfer, it looks like you didn't specify binary before
> starting the transfer. So LFs (0x0a) were converted to CR/LFs (0x0a 0x0d)
> by the server during the transfer.
> The server determines the default, not the client.
Damn stupid server :-(

> I can see an enormous amount of dropped packets though - you
> are losing more than 10% of the packets into your machine.
Bad :-(

> Are you running something else on that machine in the background?
NO. :-(

> What kind of machine is it?
PCI network card ... UDMA hard disk ... RAMDISK

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***
Arjay

20.07.2010, 12:41

@ mbbrutman
 

mTCP interfacing with TINY remote control for DOS?

Post reply

Hi Mike,

> My project has been a TCP/IP stack optimized for the original DOS machines
> - the PC, XT, Jr, AT, etc. (It is optimized for both speed and space
> - my target machine is an 8088 with 256K of RAM.)
I noted that Josh Levine author of Tiny remote control for DOS has the following FAQ response to this question:

Q: Can you make TINY run with no extra TCPIP stack?
I'd actually love to start a project to have TINY talk directly to the network card, probably using a Crynwr packet driver. This would reduce the amount of memory needed on a TINY host and with one less piece of software things would generally be simpler to set up. Let me know if you have a good reason to need this.

Note: TINY is being also actively developed (last release 6/4/10).

Sounds like you guys should have a chat ;)
mbbrutman

Homepage

Washington, USA,
20.07.2010, 16:09

@ Arjay
 

mTCP interfacing with TINY remote control for DOS?

Post reply

I have bumped into Tiny before but I didn't give it too much thought - in his FAQ he points out that getting his code to work with WATTCP might be challenging. Given that mTCP is structured the same way that WATTCP is, he might not think too highly of it either.

The other immediate problem mTCP has is that it doesn't support IP fragmentation, and it looks like Tiny requires it. But that's on the todo list, so it shouldn't be a problem for too much longer.

On the positive side, UDP is a very simple protocol to implement. There would not be much of mTCP to modify/port if we chose to try it. :-)

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
Arjay

20.07.2010, 17:49
(edited by Arjay, 20.07.2010, 18:43)

@ mbbrutman
 

mTCP interfacing with TINY remote control for DOS?

Post reply

> I have bumped into Tiny before but I didn't give it too much thought
I think probably the same here. However revisiting it the java remote control aspsect is interesting, although watchout is obviously insecure (as he notes).

> The other immediate problem mTCP has is that it doesn't support IP
> fragmentation, and it looks like Tiny requires it.
 Ah yes. I did note Josh's comments that without it things would be tooooo slow which is fairly obvious when you think about it. Still anyone who has ever experienced any speed below 2400bps for long periods will know unpleasant but better than nothing ;)

> But that's on the todo list, so it shouldn't be a problem for too much longer.
> On the positive side, UDP is a very simple protocol to implement. There
> would not be much of mTCP to modify/port if we chose to try it. :-)
 Excellent!
Arjay

20.07.2010, 14:07
(edited by Arjay, 20.07.2010, 22:06)

@ mbbrutman
 

mTCP DOS - NC documentation issue (design bug)?

Post reply

Just referenced the mTCP version of NC in this forum reply. To be honest I've been too busy with my own coding/away from the keyboard activities to test mTCP yet.

However I did note the following line in http://brutman.com/mTCP/nc.txt:

"(If you send "GET /" without the quotes and press Enter, you will get a web page back!)"

I don't know if this is the case with the mTCP version of NC but this is wrong you shouldn't get a webpage back on most servers if you do the above. This is an example of a correct basic GET request as per the RFC's:

"GET / HTTP/1.0[CRLF]" where [CRLF] = 0Ah,ODh
or
"GET / HTTP/1.1[LF]" where [CRLF] = 0Ah,ODh

The above should also be case sensitive.

However be aware that some servers are so badly coded (or on purpose) that even this basic test may not work, e.g. "GET / HTTP/1.1" will work but "GET / HTTP/1.0" will NOT work or the server may sit there waiting for a longer url to be requested etc.

Note: 0Ah can often be obtained by [CTRL]+[ENTER]. For anyone interesting in scripting reading this also note many DOS based editors allow you to insert control codes via [CTRL ^P] and followed by [CTRL whatever] where whatever is the appropriate combination required, e.g. [CTRL]+[P] then [CTRL]+[ENTER] or [CTRL]+[P] then [CTRL]+[B] to insert a bell. This is useful to know if for example you wish to feed a text file via redirection to something like netcat.

The vast majority of telnet based servers will tend to respond with a header if you just send a few CRLF or or if that fails a few lineFeeds (CTRL+[ENTER]) or Carriage Returns (ENTER)'s instead, it depends on the server and if the programmer has read the RFC's etc. In the case of HTTP servers, using sending a HEAD request instead of a GET request is also useful.

For anyone needing more info: http://www.google.com/search?q=http+get+telnet

[EDIT]Corrected a technical error that I made whilst quickly writing this reply.
mbbrutman

Homepage

Washington, USA,
20.07.2010, 15:22

@ Arjay
 

mTCP DOS - NC documentation issue (design bug)?

Post reply

Hi Arjay,

I missed that other thread - thanks for pointing it out!

As always, it depends on the server. http://www.cnn.com and http://www.brutman.com will happily give you a default page even though the header is technically wrong.

A proper telnet newline is a carriage return and linefeed pair. Your post implies that only the LF is needed. If so, that's news to me - I thought that HTTP was expecting telnet newlines, not just plain LFs.

(Just in case, I gave the user a command line option for sending the Enter key as a carriage return, or a CR/LF pair when using it interactively. Ctrl-J and Ctrl-Enter can be used for sending just a pure line feed, and Ctrl-M for sending a pure CR if anybody never needs that.)

I'm going to be doing wget shortly, so I'll crisp up the docs then.




> Just referenced the mTCP version of NC
> in this
> forum reply. To be honest I've been too busy with my own
> coding/away from the keyboard activities to test mTCP yet.
>
> However I did note the following line in
> http://brutman.com/mTCP/nc.txt:
>
> "(If you send "GET /" without the quotes and press Enter, you will get a
> web page back!)"
>
> I don't know if this is the case with the mTCP version of NC but this is
> wrong you shouldn't get a webpage back on most servers if you do the above.
> This is an example of a correct basic GET request as per the RFC's:
>
> "GET / HTTP/1.0[LF]" where [LF] = character 0Ah
>
> or
>
> "GET / HTTP/1.1[LF]" where [LF] = character 0Ah
>
> The above should also be case sensitive.
>
> However be aware that some servers are so badly coded (or on purpose) that
> even this basic test may not work, e.g. "GET / HTTP/1.1" will work
> but "GET / HTTP/1.0" will NOT work or the server may sit there
> waiting for a longer url to be requested etc.
>
> Note: 0Ah can often be obtained by [CTRL]+[ENTER]. For anyone interesting
> in scripting reading this also note many DOS based editors allow you to
> insert control codes via [CTRL ^P] and followed by [CTRL whatever] where
> whatever is the approrpriate combination required, e.g. [CTRL]+[P] then
> [CTRL]+[ENTER] or [CTRL]+[P] then [CTRL]+[B] to insert a bell. This is
> useful to know if for example you wish to feed a text file via redirection
> to something like netcat.
>
> The vast majority of telnet based servers will tend to respond with a
> header if you just send a few LineFeeds (CTRL+[ENTER]) or if that fails try
> sending Carriage Returns (ENTER)'s instead or CRLF, it depends on the
> server and if the programmer has read the RFC's etc. In the case of HTTP
> servers, using sending a HEAD request instead of a GET request is also
> useful.
>
> For anyone needing more info:
> http://www.google.com/search?q=http+get+telnet

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
Arjay

20.07.2010, 17:30
(edited by Arjay, 20.07.2010, 17:40)

@ mbbrutman
 

mTCP DOS - NC documentation issue (design bug)?

Post reply

> As always, it depends on the server.
Yup. Btw Slashdot is a good server to test with if you like Futurama :)

> A proper telnet newline is a carriage return and linefeed pair.
 To be honest it's is ages since I read all the RFC's and indeed age since I coded web servers etc. Indeed in someways to be honest I am partly relearning technical (similar theme I was amused to read this recent blog posting by Trixter).

> Your post implies that only the LF is needed. If so, that's news to me
> - I thought that HTTP was expecting telnet newlines, not just plain LFs.
 In a former life I spent more time with web stuff than I care to remember, however I could be wrong (often I am and happy to admit it ;), still I did do a few quick Google's to re-check facts. Anyway I guess we both need to do a bit of RTFM. Too many manuals to read+remember and just not enough time !!!

> (Just in case, I gave the user a command line option for sending the Enter
> key as a carriage return, or a CR/LF pair when using it interactively.
Neat :)

> I'm going to be doing wget shortly, so I'll crisp up the docs then.
 Excellent. What about another cURL/libcURL, DOS port?

EDIT: wput?
I also noted that Daniel Stenberg has created 2 articles (biased obviously :) called:
curl vs wget
curl vs libcurl
mbbrutman

Homepage

Washington, USA,
20.07.2010, 19:32

@ Arjay
 

mTCP DOS - NC documentation issue (design bug)?

Post reply

I am pretty certain on the CR/LF issue. All of the RFCs for things like FTP and HTTP that use 'newline' chars over the network use the TELNET standard, which is CR/LF. There is a lot of confusion because MS/DOS derived OSes use CR/LF while Unix derived OSes just use LF, but the standard clearly is CR/LF when using HTTP. My request is malformed, but not for lack of a proper newline char.

WGET is on my todo list because it will give people an easier way to upgrade mTCP from an existing installed mTCP. Right now my distribution is from a web site, and I can't use anonymous FTP with my particular web host. WGET is generally useful anyway, although I don't think that I want to get into cookie management, security, etc. It will be a very simple WGET.

Curl? Still have never used it. :-)

Also on the todo list is an SNTP client. I had started on it a few months ago, but got bogged down in the process of interpreting timestamps from the server to generate the correct local time.

If/when I get this open sourced I'm hoping that I'll get some help. It's tough trying to keep up on top of all of these applications. The approach I'm taking of integrating the stack with the apps is labor intensive, but I think it works better in the long run than trying to rely on a TSR to provide the stack. I can debug this and fix it, which I can't do with a TSR.

While we're on the topic, I posted another update two days ago. The churn from the port to the Watcom compilers is mostly over. That was a two or three month ordeal, but I'm glad that it's over. Open Watcom makes me more productive, generates better code, and potentially opens development to more people.

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
Arjay

20.07.2010, 21:56

@ mbbrutman
 

mTCP DOS - NC documentation issue (design bug)?

Post reply

> I am pretty certain on the CR/LF issue.
 You were correct; Quoting RFC2616: HTTP/1.1 defines the sequence CR LF as the end-of-line marker for all protocol elements except the entity-body

> There is a lot of confusion
Indeed and the more platforms, OS's you've worked with/used the harder it is to remember what rule goes where at times particularly if you become detached from a particular technology. I've found that myself getting back into DOS.

> My request is malformed, but not for lack of a proper newline char.
 We were both wrong but we got there eventually ;-)

> WGET is on my todo list because it will give people an easier way to
> upgrade mTCP from an existing installed mTCP. Right now my distribution is
> from a web site, and I can't use anonymous FTP with my particular web host.
 wget is excellent when you just want to download a file. curl can be a bit more annoying in that regards at times (depends on the version etc).

> WGET is generally useful anyway, although I don't think that I want to get
> into cookie management, security, etc. It will be a very simple WGET.
 And that is where cURL is excellent for cookie management, form POSTing etc.

> Curl? Still have never used it. :-)
 Your loss :-) at least try it; There is a 2006 DOS port of v7.10.5 by Doug Kaufman available on his site.

> If/when I get this open sourced I'm hoping that I'll get some help.
Hopefully that will help you but to be honest I wouldn't count on it either.

> It's tough trying to keep up on top of all of these applications.
 I can understand hence I've decide a while back to take a long term approach to some of my software preservation projects. Long term as in "decades".

> I'm taking of integrating the stack with the apps is labor intensive,
 A labor of love :)

> While we're on the topic, I posted another update two days ago.
 Great.
DOS386

23.07.2010, 07:27

@ mbbrutman
 

FTP ASCII vs binary "image"

Post reply

> I am pretty certain on the CR/LF issue.
I tested the ftp://ftp.openwatcom.org server and same problems:

1. mTCP defaults to ASCII, non-DOS FTP and Firefox to binary

2. bad performance (mTCP 22 KiB/s, other 82 KiB/s)

3. many lost packets

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***
mbbrutman

Homepage

Washington, USA,
24.07.2010, 01:04

@ DOS386
 

FTP ASCII vs binary "image"

Post reply

> > I am pretty certain on the CR/LF issue.
>
> I tested the ftp://ftp.openwatcom.org server and same problems:
>
> 1. mTCP defaults to ASCII, non-DOS FTP and Firefox to binary
>
> 2. bad performance (mTCP 22 KiB/s, other 82 KiB/s)
>
> 3. many lost packets
[1] Try the command line Windows FTP client with that server. You'll notice that it defaults to an ASCII transfer even on a file that should be sent in binary. This is the correct behavior (as defined by FTP RFCs) for a command line FTP client. If FireFox is getting it as a binary file, then it must be stuffing a 'BIN' command in the command stream first. You'd have to use a packet sniffer to verify that. But remember, the server sets the default transfer mode, not the client. If you find a client that gets BINARY, it is stuffing a BIN command under the covers.

That is something I'm not willing to do. Forcing a command on somebody transparently is wrong, even if you are trying to be helpful.


[2 & 3] What OS are you running Firefox under? You are comparing a full blown OS network stack to a DOS packet driver, and I'm willing to bet that the DOS packet driver is part of the problem. My code doesn't lose packets - it just reports the ones that it didn't get.

Other people are not reporting this, and generally when it happens it is something specific to the packet driver. Have you looked for a different version of the packet driver for your card? I remember you had a PCI cards, and by the PCI era packet drivers weren't being as well tested as they had been before.

Are all FTP servers bad, or only particular ones?



Mike

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
DOS386

24.07.2010, 12:12

@ mbbrutman
 

FTP ASCII vs binary "image"

Post reply

> This is the correct behavior (as defined by FTP RFCs) for a command line FTP client.
> If FireFox is getting it as a binary file, then it must be stuffing a 'BIN' command
> in the command stream first. You'd have to use a packet sniffer to verify that.
> But remember, the server sets the default transfer mode, not the client.
Faulty servers ...

> If you find a client that gets BINARY
Many clients, and the servers are apparently "optimized" for such :-(

> [2 & 3] What OS are you running Firefox under?
Windaube ???

> blown OS network stack to a DOS packet driver, and I'm willing to bet that
> the DOS packet driver is part of the problem. My code doesn't lose packets
> - it just reports the ones that it didn't get.
:-|

> Other people are not reporting this, and generally when it happens it is
> something specific to the packet driver. Have you looked for a different
> version of the packet driver for your card?
No. The internet is not really crowded with DOS drivers AFAIK :-(

> Are all FTP servers bad, or only particular ones?
I tested 2 only so far.

I'll test on other PC (different PCI card) too.

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***
Japheth

Homepage

Germany (South),
24.07.2010, 15:31

@ DOS386
 

FTP ASCII vs binary "image"

Post reply

I also just did a brief test of mTCP's FTP client and ftp.openwatcom.org. I tried to download 2 files, running mTCP's ftp.exe and also the Win32 version of "Far Manager" (FM) - both in MS-DOS 7.1. The results are:

- file devel/gui.pdf. No client did loose packets. mTCP needed 25 sec for this file, while FM did it in about 8 seconds.

- file devel/open-watcom-w32api.0.4.zip. mTCP reports an error that it cannot create the file and aborts - the error is reported regardless if DOSLFN is loaded or not. FM had no problems.

I have a RT8139 card installed and use RTSPKT.COM packet driver.

---
MS-DOS forever!
DOS386

24.07.2010, 15:36

@ Japheth
 

FTP ASCII vs binary "image"

Post reply

> - file devel/open-watcom-w32api.0.4.zip. mTCP reports an error that it
> cannot create the file and aborts - the error is reported regardless if
> DOSLFN is loaded or not. FM had no problems.
You can specify an output filename :-)

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***
mbbrutman

Homepage

Washington, USA,
24.07.2010, 16:23

@ Japheth
 

FTP ASCII vs binary "image"

Post reply

> I also just did a brief test of mTCP's FTP client and ftp.openwatcom.org. I
> tried to download 2 files, running mTCP's ftp.exe and also the Win32
> version of "Far Manager" (FM) - both in MS-DOS 7.1. The results are:
>
> - file devel/gui.pdf. No client did loose packets. mTCP needed 25 sec for
> this file, while FM did it in about 8 seconds.
>
> - file devel/open-watcom-w32api.0.4.zip. mTCP reports an error that it
> cannot create the file and aborts - the error is reported regardless if
> DOSLFN is loaded or not. FM had no problems.
>
> I have a RT8139 card installed and use RTSPKT.COM packet driver.
You mentioned Win32 version of FM, but said it was MS-DOS 7.1. Is it in Windows, or is it running in DOS with some sort of compatibility layer? What networking stack is it using?


Long filenames: mTCP checks the filename format and gives up. Get allows you to special a local filename to save to. I'll look into DOSLFN.


Mike

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
Japheth

Homepage

Germany (South),
24.07.2010, 16:57

@ mbbrutman
 

FTP ASCII vs binary "image"

Post reply

> You mentioned Win32 version of FM, but said it was MS-DOS 7.1. Is it in
> Windows, or is it running in DOS with some sort of compatibility layer?

The latter of course - it uses HX's Win32 emulation. For FAR in DOS, see

http://www.bttr-software.de/forum/forum_entry.php?id=3628

> What networking stack is it using?
HX's WSOCK32.DLL uses WATT32 - slightly modified.

> Long filenames: mTCP checks the filename format and gives up. Get allows
> you to special a local filename to save to. I'll look into DOSLFN.
DOSLFN just implements the LFN API introduced with Windows 95. To support LFNs, use DOS int 21h, AX=716Ch to create/open files and fall back to the standard method if LFN isn't installed ( the DOS int returns ax=7100h then ).

---
MS-DOS forever!
mbbrutman

Homepage

Washington, USA,
25.07.2010, 01:35

@ Japheth
 

FTP ASCII vs binary "image"

Post reply

> > You mentioned Win32 version of FM, but said it was MS-DOS 7.1. Is it in
> > Windows, or is it running in DOS with some sort of compatibility layer?
>
> The latter of course - it uses HX's Win32 emulation. For FAR in DOS, see
>
> http://www.bttr-software.de/forum/forum_entry.php?id=3628
>
> > What networking stack is it using?
>
> HX's WSOCK32.DLL uses WATT32 - slightly modified.
>
> > Long filenames: mTCP checks the filename format and gives up. Get
> allows
> > you to special a local filename to save to. I'll look into DOSLFN.
>
> DOSLFN just implements the LFN API introduced with Windows 95. To support
> LFNs, use DOS int 21h, AX=716Ch to create/open files and fall back to the
> standard method if LFN isn't installed ( the DOS int returns ax=7100h then
> ).
Interesting. I've never done a performance comparison against WATT32 - I need to find out what the differences are. (A quick read of the web page did not shed much light.) Usually my code runs faster than WATTCP based code, so I'm curious as to what is going on. The two most performance sensitive parts of the code are the IP checksum and memcpy ... I avoid memcpy like the plague, but when disk I/O is performed there are many more memcpys being performed.

I'll look at DOS LFN support - that doesn't look too hard to implement.

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
Rugxulo

Homepage

Usono,
25.07.2010, 05:34

@ mbbrutman
 

FTP ASCII vs binary "image"

Post reply

>
> I'll look at DOS LFN support - that doesn't look too hard to implement.
Keep in mind that OpenWatcom 1.9 now has optional LFN support for DOS, so in most simple cases you don't need any explicit changes, just an additional define: __WATCOM_LFN__
DOS386

25.07.2010, 06:16

@ Rugxulo
 

FTP ASCII vs binary "image"

Post reply

> Keep in mind that OpenWatcom 1.9 now has optional LFN support for DOS, so
> in most simple cases you don't need any explicit changes, just an
> additional define: __WATCOM_LFN__
This is the end of DOS support in WATCOM :-(

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***
mbbrutman

Homepage

Washington, USA,
25.07.2010, 17:17

@ DOS386
 

FTP ASCII vs binary "image"

Post reply

> > Keep in mind that OpenWatcom 1.9 now has optional LFN support for DOS,
> so
> > in most simple cases you don't need any explicit changes, just an
> > additional define: __WATCOM_LFN__
>
> This is the end of DOS support in WATCOM :-(
Can you explain what you mean? I haven't seen anything that says Open Watcom is going to drop support for DOS, or any other OS.

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
DOS386

26.07.2010, 02:03

@ mbbrutman
 

FTP ASCII vs binary "image" | NTLFN issues

Post reply

> > This is the end of DOS support in WATCOM :-(
> Can you explain what you mean? I haven't seen anything that says Open
> Watcom is going to drop support for DOS, or any other OS.
See DGJPP. They will pretend DOS support but in fact support NTVDM (with great NTLFN support) only, see DGJPP. This is a fear so far, no final fact, but ... :-| Also they reportedly recently "fixed" WD on NTVDM, on FreeDOS it is and probably will remain unfixed :-(

2 of my > 1'000'000'000 ENOENT bug reports: 7918 and 4079

> You are comparing a full blown OS network stack to a DOS packet driver,
> and I'm willing to bet that the DOS packet driver is part of the problem

Connecting to ftp.openwatcom.org, port 21 ...
220 ProFTPD 1.3.0 Server (ftp.openwatcom.org) [216.55.181.205]
USER anonymous
331 Anonymous login ok, send your complete email address as your password.
PASS user@provider.com
230 Anonymous access granted, restrictions apply.
227 Entering Passive Mode (216,55,181,205,239,144).
TYPE I
200 Type set to I
RETR manuals/current/guitool.pdf
Connecting to 216.55.181.205, port 61328 ...
150 Opening BINARY mode data connection for manuals/current/guitool.pdf (4852090 bytes)
Closing connection...


Arachne automagically uses the BINARY mode and downloads the 4'852'090 Byte's in 50s (96 KiB/s) while mTCP FTP needs the image command and takes 150s to download the file (32 KiB/s). Apparently the Openwatcom server was slightly faster this time. ;-) And the biggest shocker: both files were identical. :surprised: Nevertheless, the "argument" about "full blown OS" is fully blown away - since Arachne is fast too. :-) I don't know how to find out how many packets got lost when using Arachne :-( Also, I didn't test the other PC yet :-(

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***
mbbrutman

Homepage

Washington, USA,
26.07.2010, 03:11

@ DOS386
 

FTP ASCII vs binary "image" | NTLFN issues

Post reply

The default transfer mode is set by the FTP server. Arachne is stuffing a 'BIN' command in the command stream for you - see the 'TYPE I' message. I don't do that in my FTP client because that assumes that the FTP client knows more about the file being transferred than the user, which is not something that a program should do.


I just downloaded that same file (guitool.pdf) twice. The first download took 67.76 seconds, and the second took 67.32 seconds. That is an average rate of 70KB/sec for this machine, which is a 386-40 with an NE2000 Ethernet card. I don't remember the particulars of your machine, but I know it is far faster and it should be doing better. I think that your experience is atypical and that you either have packet driver or hardware problems because you are getting far lower performance than you should, and mTCP is reporting the dropped packets for you.

It's possible that Arachne is doing better in that situation because it's handling the dropped packets better. I'm sure that WATTCP is doing Round Trip Time calculations, where mTCP is just using a fixed timeout value. But still, on a fast PCI bus machine like yours you should be doing far better than you are. So comparing the results of something that probably isn't working right doesn't make a lot of sense.

Please post the details on this machine - CPU and Ethernet card type. And ry it on another machine - unless your network is horrible I'm sure the results will be vastly different. My 386-40 should not be anywhere near close to your PCI bus based machine.

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
DOS386

27.07.2010, 02:53

@ mbbrutman
 

FTP ASCII vs binary "image" | NTLFN issues

Post reply

> Ethernet card. I don't remember the particulars of your machine, but I
> know it is far faster and it should be doing better.
MPXPLAY can play local compressed files with less than 5% CPU hogging and HD write performance is 15 MiB/s raw and 5 MiB/s files (FYI: FAT sucks), so the performance of the PC is not the problem at all (except you use Wista) ;-)

> think that your experience is atypical and that you either have packet
> driver or hardware problems because you are getting far lower performance
> than you should, and mTCP is reporting the dropped packets for you
Looks like a problem of mTCP or packer driver or both together :-(

I had problems with other stuff too (WSTEST, MPXPLAY, OWB, WGET), so Arachne is the one providing best performance and causing least problems.

[image]

New ^^^ version 2010-Jul-1018, retest, performance just marginally better, also upload tested.

> mTCP applications should work on any DOS PC with an Ethernet adapter,
> SLIP or PPP connection. My personal testing includes:
> Generic AMD 386-40 with an NE2000 Ethernet adapter
> IBM PC XT at 4.77Mhz with a 3Com 3C503 Ethernet adapter
> IBM PC XT at 4.77Mhz with a Xircom PE3-10BT parallel-port to Ethernet
> IBM PCjr at 4.77Mhz with a NEC V20 CPU, Xircom PE3-10BT parallel-port to Ethernet
So you have 3 different 8086-based PC's ??? :clap:

> A note about the screen shot - it was taken on a Windows XP system running a
> special build of DOSBox
:-(

> that has networking support. This is the best way to
> get screen captures.
:-(

> Your results on real hardware will be similar.
NO, see above.

Why don't you make the shot on your XT rather than XP ???

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***
mbbrutman

Homepage

Washington, USA,
27.07.2010, 03:31

@ DOS386
 

FTP ASCII vs binary "image" | NTLFN issues

Post reply

Well, on that FTP server you were getting 57KB a second. My 386 is beating that. Something is definitely wrong on your machine. mTCP really doesn't know or care too much about the packet driver - as far as it is concerned, they are all the same. (Which is the intent.)

Off Topic .. I have quite a few machines here, which makes testing really fun. Most of my machines are older 8088 class machines, so I don't get to use extended memory.

The screen shots are easier on the XP machine running DOSBox. Otherwise, I'd have to get the camera out. :-) I could use a TSR and get an ASCII dump of the CGA text mode screen, but that doesn't really prove anything. Video tape is the best. :-)

---
mTCP - TCP/IP apps for vintage DOS machines!
http://www.brutman.com/mTCP
DOS386

27.07.2010, 03:40

@ mbbrutman
 

FTP ASCII vs binary "image" | NTLFN issues

Post reply

> I could use a TSR and get an ASCII dump of the CGA text mode screen
What is the resolution ???

> but that doesn't really prove anything.
Convert you ASCII into BMP and then into PNG :-)

> Otherwise, I'd have to get the camera out.

I got my shots without :-)

> Video tape is the best
I prefer celluloid film :-)

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***
Back to index page
Thread view  Board view
22632 Postings in 2109 Threads, 402 registered users, 321 users online (0 registered, 321 guests)
DOS ain't dead | Admin contact
RSS Feed
powered by my little forum