View Full Version : The joys of filesharing ..
mbbrutman
July 6th, 2006, 08:23 PM
Before the RIAA slaps a lawsuit on me, let me explain. :-)
Getting TCP/IP to run on an older machine has some serious challenges:
Existing DOS TCP/IP software is out of date and not supported
Memory is tight on a 5150 PC. Some features just can't be supported.
Some source code exists, but then you have to be a master and porting/debugging.
My predicament is worse - I'm on a PCjr, which doesn't take ISA cards.
So a few years ago a friend of mine (and fellow PCjr nutcase) found that Xircom parallel-port to Ethernet adapters work fine on the Jr. You need to do a little tweaking, but with that you can run Telnet and FTP from Trumpet, or NCSA Telnet. Not a bad start. Trumpet isn't supported anymore, and there is no source code available. NCSA Telnet has source code available, but it is a giant monster.
Instead of living with what was out there, I decided to start writing my own .. Using Turbo C++ 3.0 on a 386-40 machine, creating executables that still run on the PCjr. The 386-40 has a real Ethernet card, while the PCjr has a Xircom adapter.
So far I've got ARP and UDP working. No TCP yet .. that's a big mess of code. I'm not stealing code from anywhere - this is all 'cleanroom' done by reading textbooks and googling.
To prove the UDP piece was working correctly, I wrote a 'fileserver' type program in Java that uses a simple protocol over UDP. That program can run on Windows or Linux. The client side runs on the old DOS machine. It basically does Xmodem over UDP.
So after a marathon hacking session, I've got it working. It only sends data to the server, and there are probably some bugs in it, but it does seem to send the data correctly.
On the 386-40 with a real Ethernet card it is pushing 170KB per second out to the file server. (The file is in a ram disk to eliminate any overhead that the file read might be causing.) I used a fairly good sized file for the test - 125KB.
The PCjr suprised me. It clocked in at about 16KB per second, which is less than one tenth of what the 386-40 can do. But when you consider that the PCjr is running at 4.77Mhz, doesn't have an L2 cache like the 386-40 does, and has to use the cruddy parallel-port adapter instead of a real bus attached Ethernet adapter, it's amazing that it did that well.
On the ToDo list:
Cleanup this nasty pile of code I threw together.
Add the ability to receive files as well
Enhance the error checking
Move to a 'sliding window' type system to improve throughput.
In a few months this will be to the point where if you have a suitable machine that runs Java to use as a server, you can put a Xircom or a real Ethernet card on your machine and have a fairly lightweight and fast file transfer program. Beats shuffling floppies ..
Eventually when I get the TCP part done this will turn into a Telnet BBS. Running on the Jr of course. :-)
Guys/Gals - don't just collect the machines. Use them!
Terry Yager
July 6th, 2006, 09:41 PM
Hey, Mike,
You're already waaayy over my head with what you're doing. All I can say iz. Keep On Coding, and lemme know when you've perfected it, BTW, I love your ROM-dump program.
--T
CP/M User
July 7th, 2006, 03:33 AM
Sounds like you need some Memory!
Guess it's possible to make a RAMpack with easy plug in & have
the program utilsing this memory.
Don't know how you're goin' to do that though.
CP/M User.
mbbrutman
July 7th, 2006, 07:06 AM
The machine has 640K of memory, so I'm pretty good. The major problem for TCP/IP is that a maximum sized packet is around 64K. And of course as packets go through the network they can become fragmented. It is the responsibility of the receiving machine to put everything back together again.
Imagine getting a few fragmented packets and trying to put them back together .. you would run out of memory fairly quickly if they were large packets. I have to look this up to be sure, but I don't think that you can tell at the beginning how big the full IP packet was supposed to be - you have to wait until you receive the last packet, which has an indicator that says it is. Which means that you always have to allocate a worst-case buffer to reassemble in.
The solution is that machines that are tight on memory don't do packet reassembly. You impose a restriction - any IP traffic to the machine has to fit within an Ethernet frame, or bust.
I know you program quite a bit - ever tackle networking code before?
Chris2005
July 8th, 2006, 08:29 AM
"Beats shuffling floppies .."
I realize you're not doing this simply to facilitate network file transfers...and in that case I would simply recommend a $4 null-modem adapter!
You might want to check out the linux-86 mailing list. Activity is low, but there is at least 1 goorue on there - Alan Cox. I've gotten mails with his signature and I was like wtf.
As time allows, and I hope that to be soon, I want to learn something about network programming, but more then likely it'll be in a UNIX environment. I need to catch up on alot of things come to think of it...
mbbrutman
July 8th, 2006, 06:16 PM
A $4 null modem adapter isn't going to let the machine transfer at 16KB/sec. Ethernet does ..
As for Alax Cox, he's quite famous in the Linux community. Linux kernel mailing lists have nothing to do with what I'm doing.
NathanAllan
July 9th, 2006, 03:56 AM
A $4 null modem adapter isn't going to let the machine transfer at 16KB/sec. Ethernet does ..
As for Alax Cox, he's quite famous in the Linux community. Linux kernel mailing lists have nothing to do with what I'm doing.
...or token ring. A great feat would be to get token ring and ethernet to talk to each other. That way whenever the PCjr would recieve a packet it would have a much smaller packet to deal with.
http://www.cisco.com/en/US/tech/tk331/tk332/technologies_tech_note09186a00800a6163.shtml
Or you may want to go totally with token, since I know there's not a really good way to make them compatible. The network would be wired up differently, yes, and there's a lot of work involved... so maybe it isn't such a good idea after all.
Sorry, I just really like token ring and always consider it as a network solution when others aren't practical. Something to think about.
Nathan
mbbrutman
July 9th, 2006, 07:30 AM
Ethernet packets can be pretty small too. The minimal sized Ethernet frame is about 64 bytes. The Ethernet header is 14 bytes. The minimal sized IP header in that frame is 20 bytes. So the actual payload might be quite small.
Ethernet and Token ring will never talk together. Something has to pluck the IP packet out of the Ethernet and put it in a Token ring packet, and vise versa. These devices were known as routers, bridges, etc.
You really should look at Token ring and Ethernet has equivalent .. It's a different way of getting the data around, but it is packet based and fairly high speed. Quite unlike point-to-point serial connections. If Ethernet is a bad solution for something, chances are that Token ring is equally bad.
On the PCjr there is a fascinating option called 'Cluster' that I've only explored very briefly. Cluster was a Ethernet-like (but not Ethernet) network from IBM in the 1982 or 1983 timeframe. Cluster is natively supported on PCs, XTs, ATs and Jrs. The problem with Cluster is that it was quickly supplanted by 'PC Network', then Token Ring and Ethernet. Cluster adapters for the PCjr are fairly common, but ISA Cluster adapters are very hard to find.
Since Cluster is just a generic packet based transport (like Ethernet and Token ring), you could run TCP/IP over it. Then on another machine you could have Cluster and Ethernet, and do the bridging there. The most practical thing would be to put it on a low end 486 and try to shoehorn Linux onto it, and then write a Linux device driver for the old Cluster adapter. That way you could use all of the Linux networking code nearly transparently.
Another project for another year .. I'm more likely to graft a simple Ethernet chipset directly only the bus of the machine to speed things up.
Chris2005
July 10th, 2006, 08:01 AM
"As for Alax Cox, he's quite famous in the Linux community. Linux kernel mailing lists have nothing to do with what I'm doing"
From: Alan Cox <alan@lxorguk.ukuu.org.uk>
To: editor@lwn.net, scoop@freshmeat.net, kernel@linuxtoday.com,
kernel@linuxfr.org, arthur@tucows.com, chris@linuxdev.net,
kernel@linuxhq.com, staff@appwatch.com, kernel@linsight.com,
sde@linuxlinks.com, dml@tonnikala.net, editors@newsforge.com,
kenneth.g@linux.nu, linux-kernel-announce@vger.kernel.org
Subject: Linux 8086
Date: Mon, 1 Apr 2002 01:30:03 +0100 (BST)
Do you wish you could run Linux on your old IBM/XT or indeed the original IBM
PC itself ?
We are pleased to announce that Linux 8086 has now reached the point where
it is ready for some wider testing. Harry Kalogirou has the TCP/IP stack
functioning...
mbbrutman
July 10th, 2006, 08:40 AM
I'm writing TCP/IP for DOS so that I can squeeze it into a telnet BBS and directly control the performance by writing the code myself. They are trying to cram an entire OS in. Other than for the OS having some networking, it's not related.
Also, from http://elks.sourceforge.net/:08 May 2006: ELKS revisited and back in action!
ELKS has been in virtual hibernation for two or three years now. The CVS access is broken, the website has been left untended, and 0.1.3-pre1 never really got released! After a new maintainer stepped up to the plate, the breath of life has slowly crept into ELKS once again. Downloads are temporarily hosted by Jody Bruchon on his personal equipment due to an unexplained CVS breakage for ELKS anonymous CVS access. As always, any assistance you can offer will be greatly appreciated!
There is hope, but I wouldn't call this a viable project. It was been dormant for quite a while.
Chris2005
July 10th, 2006, 08:53 AM
the simple point was there's people on their who are knowledgeable with network coding. And the fact that it is pretty inactive (though not entirely, I've gotten a few dozen posts since subscribing earlier this year) is a very good reason to perhaps sign up. They could use the traffic, and I'm sure there's people who'd appreciate an exchange.
mbbrutman
July 10th, 2006, 09:14 AM
Maybe I misunderstand .. I'm interested in Elks as an alternative OS to DOS, but joining the mailing list of a dormant project trying to shrink Linux onto an 8088 class machine just to tap their networking expertise doesn't seem like the right thing to do. They are having a hard enough time keeping the project alive; tangential networking questions are not what they need, even if it does generate some traffic.
Especially since I'm not having any trouble with the networking part. ;-)
Alan Cox is interested in the Linux kernel. Although he posted that announcement years ago, he is not on the ELKS development team, and I wouldn't think of hassling him unless it were on topic.
mbbrutman
July 17th, 2006, 09:25 PM
Some food for thought ..
Xmodem was good in the day, but it wasted a lot of potential bandwidth. For those who don't know or remember xmodem, each time the computer sends a 128 byte data packet to the other machine, it has to wait for the other machine to say 'I got it!' before sending the next packet. Given the round trip time of a 2400 bps modem and the slow machines, there was a lot of dead time on the line.
TCP/IP has a similar problem. If you waited for a separate ack packet for each packet of data sent, you wouldn't make very good use of your dialup, leased line, or ethernet. So TCP/IP implements two bandwidth friendly devices - a sliding window and piggybacking. The sliding window trick allows a machine to send data without waiting for an immediate ack back, up to a certain point. Piggybacking makes better use of full duplex connections.
So I was thinking about my Xmodem over UDP implementation here, and it doesn't use a sliding window or piggybacking. I thought about the sliding window, but then decided against it. Seems like a good idea, so why not do it?
Well, here is the problem. My target machines are old DOS PCs ranging from 4.77Mhz and a floppy drive to maybe 486s. The server machine would be running Java on a more modern machine. If the old DOS machine sends a packet to the faster machine, chances are that the faster machine has processed the packet and sent an ACK even before the old DOS machine can get around to looking for the ACK. Or put another way, the server machines that people would probably use (anything capable of running a small Java program) are so fast that it just doesn't matter.
The sliding window really only matters if you've got a huge round trip latency, and a lot of that latency was caused by the other machine being busy doing other things. But if you throw a 4.77Mhz PCjr against a 1Ghz machine, the round trip latency from the perspective of the PCjr is non-existent. Chances are that the server is going to go into power suspend and sleep for a bit before the PCjr gets ready to send the next packet. ;-)
I might add the sliding window just to prove the point .. any thoughts on this? Does it make sense, or am I smoking something bad?
Terry Yager
July 17th, 2006, 09:32 PM
As usual, you're way over my head, and whatever you're smoking must be some good sh!t...(I can barely get my mind around Modem7 & checksumz...fagiddabout X-, Y-, Z-Modem, & CRCs, etc).
--T
modem7
July 18th, 2006, 02:50 AM
Many years ago I wrote code to do communications through parallel and serial ports, but never Ethernet cards.
However the basic principles of comms are the same.
The design has to allow for the fact that the CPU may be occupied when a data packet arrives.
If your code is directly talking to the Ethernet card, there maybe a hardware buffer on the card that will hold the ACK packet for collection.
If there's no hardware buffer on the card, you may need to configure the card to generate an interrupt on packet arrival, and write an interrupt processor that buffers received packets.
If your code is talking to a packet driver (or some other low-level code), then a receive buffer may already be in place.
Erik
July 18th, 2006, 05:26 AM
I might add the sliding window just to prove the point .. any thoughts on this? Does it make sense, or am I smoking something bad?
Based just on your discussion it seems like it would be a waste of time - unless you were trying to prove the point. :)
mbbrutman
July 18th, 2006, 06:05 AM
I was always a serial port programmer myself. The serial port makes things pretty simple - things arrive in order, even if they do get corrupted.
UDP presents some special challenges because it is only 'best effort':
Packets can be lost without a trace
Duplicate packets can arrive after the originals
The client machine (DOS) uses a packet driver for the network card. The user software controls the buffering. Right now I've allocates space for 10 buffers, but never have more than 3 in use at a time even on my slowest machines. The card itself is buffered and interrupt driven, so the buffer management code has to be interrupt safe.
My thinking is based on two things:
the DOS machine is single tasking, and is doing nothing but this file sending task. It's never going to be busy doing other things for a long period of time.
a modern machine that somebody would use as a server might be doing something, but in general will be far more responsive than the DOS machine.
So far it seems to be true. The only real latency I have is the transmission time on the Ethernet itself, so doing a small sliding window might improve that. The real way to figure out if it is worth doing is to write a test .. I should try to time one packet going over and getting a reply back from the server, with nothing else happening to get the 'bare wire' speed.
mbbrutman
July 18th, 2006, 06:09 AM
Based just on your discussion it seems like it would be a waste of time - unless you were trying to prove the point. :)
Sometimes you think more when you talk at other people .. :-) I'm going to cobble up a quick test to see just how fast I can do a round trip on the Ethernet from the old machine to the new machine, and then decide if that latency is worth hiding.
The 386-40 running this code is only pushing around 150KB/sec. I used data in a RAM disk to cut the diskette drive out of the loop, and that is still pretty slow. (ISA bus speed is much higher, and 10 Mb/s Ethernet should be around 1200KB/sec.) The latency of the round trip might not be an issue for a 4.77Mhz machine, but on something like the 386-40 it might be accumulating.
modem7
July 19th, 2006, 05:22 AM
I used to install/configure the hubs/switches/routers in a building, and so I know a little about network protocols.
I would rarely choose to use UDP. If I was sending outside temperature info to a remote display, UDP is a good choice. When packets get dropped/repeated/unordered, I doubt that anyone looking at the display would notice. The criticality is just not there.
For everything else, choose TCP so that it handles the dropped/repeated/unordered packets.
But I can see that as a programmer, you're giving yourself the challenge of coding your own TCP-like protocol.
------------------------------------------------------------------------------------------------
Some stuff you may already know:
The similarities with serial port programming are there. No matter what layer 1 you use (serial [232/422 etc.], Ethernet, Token Ring, RF, etc.) flow control and adequate buffering is needed to cater for communicating nodes that have differing data processing speeds.
And even on serial networks (eg. X.25), packets sometimes disappeared on the network (eg. caused by a router with bad RAM). And so no matter what layer 1 you use, you can't assume that the packet sent by your hardware will see guaranteed delivery.
mbbrutman
July 19th, 2006, 08:09 AM
The goal of the project is to write a reasonable TCP/IP for small PCs. I normally wouldn't use UDP for anything mission critical, but before I write the TCP part of the stack I want to test what I have. So far I have a basic ARP implementation, IP header, and UDP. I need ICMP, IP header options, and routing before I start TCP.
I'm doing an Xmodem-like protocol over UDP to stress the existing code and give me confidence that I'm building on a good foundation. It also gives me a chance to analyze the performance of the existing code, and makes it useful to me immediately. (I hate shuffling floppies.)
TCP is the goal, but TCP is complicated due to the sliding window and piggybacking. I can't swallow the elephant except in small bytes. :-) And in a lot of ways, what I'm doing with this xmodem protocol will directly apply to TCP - the TCP code has to handshake, detect errors and retry packets just like this Xmodem-like code I'm doing. So I look at this as a practice run for the big target ..
mbbrutman
July 19th, 2006, 08:28 PM
I did some testing based on the ideas I gathered here. All tests are run on the same machine (386-40) and are run for a meaningful length of time. I run each test a few times and take the best number.
Sending a real file one packet at a time with UDP checksums: 166KB/sec
Sending a real file one packet at a time wo/ UDP checksums: 200KB/sec
So turning off UDP checksums improves the speed by 20%.
Now do the same test, but just send packets as fast as you can without checking for reply packets from the server. (This tests how fast you can read a file and shove it down the Ethernet.)
Sending a real file as fast as possible with UDP checksums: 260KB/sec
Sending a real file as fast as possible w UDP checksums: 338KB/sec
Not waiting for the server to ack each individual packet makes a big difference. It's over 50% faster. Here the difference between UDP checksums and no UDP checksums is slightly more pronounced .. I don't understand why, but it's a 30% difference.
And finally, just for grins, see what the affect of file reading is by not sending real data. In this test I just send dummy data to avoid the file read.
With UDP checksums: 492KB/sec
Wo/ UDP checksums: 649KB/sec
Wow, eh? Too bad sending dummy data doesn't accomplish anything. The difference between UDP checksums here and no UDP checksums is 30% again, so I'm going to assume in the first test something was flawed and it should have been a 30% difference there too, not a 20% difference.
Note that a 10Mb/sec Ethernet works out to 1220KB/sec if everything is perfect. File transfer throughput suffers because of gaps between frames, frame headers, IP headers, etc. If the machine was fast enough the file xfer rate would be close to 1000KB/sec, including the TCP/IP overhead.
The only thing I could do differently would be to use a slightly larger packet size to cut down on some overhead. (I'm using a packet size of approximately 1100 bytes now.)
Anyway, this is getting too long. Contrary to what I was thinking, the waiting for the acks for each packet does take a while. So I implemented a simple mechanism to let the 386 'get ahead' of the server a little bit, while still keeping all of the error detection. Here are the new results:
With UDP checksums: 248KB/sec
Wo/ UDP checksums: 310KB/sec
So, compare 248KB/sec vs. 166KB/sec, or 310KB/sec vs. 200KB/sec. It's about 50% faster with no loss of error checking!
Lesson learned .. at least for a machine like the 386-40, the sliding window is worthwhile.
mbbrutman
August 9th, 2006, 08:25 PM
New numbers for the PCjr using a new Ethernet card:
With UDP checksums: 21.4KB/sec
Without UDP checksums: 39.26KB/sec
With the Xircom parallel port Ethernet adapter it was 15KB/sec and 24KB/sec, so this is a nice bump.
Here's a picture of the nasty hack :-)
http://brutman.com/PCjr_WD_small.jpg
80sFreak
August 11th, 2006, 09:57 AM
Cool! :) Now that is almost literally hanging off the side of that PCjr! ;)
mbbrutman
August 11th, 2006, 07:32 PM
Yeah, it's a little exposed. Hardware hacking at it's best. I need to build a proper enclosure because if a stray pen touches anything it'll all go up in smoke!
Got the receive code done, so now I can send and receive. The send code is pretty good by the receive code only just started working, so I have to go back and add all of the error checking, sliding window, and other tweaks. That'll take another two or three weeks. But then I'll be and to send and receive from the DOS machines to the Java program running on a Windows or Linux box. After that gets polished it will be time to start the TCP part.
mbbrutman
September 8th, 2006, 07:43 PM
I did some performance work on my UDP checksum code and got some great advice on exploiting the 8088 from Jim Leonard of '8088 Corruption' fame. Here are the new numbers:
Machine Card Before After
386DX-40 NE2000 clone 335800 356800
PCjr Xircom PE3-10BT 18000 20700
PCjr WD 8003 25400 31200
(All numbers are in bytes per second)
What a difference a little more 8088 assembly language makes! Especially well chosen instructions, and trying to keep everything in registers while in a performance sensitive loop.
carlsson
September 10th, 2006, 11:16 AM
Does boundary cross (e.g. pages of 256 bytes) make a difference in x86 assembler, or is everything running at the same speed as long as you stay within the same 64K segment?
mbbrutman
September 10th, 2006, 11:41 AM
There is no penalty as there are no boundaries really. If you go past your current segment, the pointer just wraps around as it is a 16 bit pointer.
You only start to get boundary penalties when you get into virtual memory. Everything I'm doing on the 8088 is a relatively flat memory model .. there is no virtual memory or paging even possible. The segment registers are just a necessary evil to extend your range past 64K.
I just got my first multi-packet TCP/IP transaction to run. The DOS machine contacted Apache running on a Linux box and sent 'GET /'. Apache in turn spewed forth many small packets, while my code dumped the packets to the screen and sent acks.
I'm still missing a lot of code, but getting multi-packet transactions is awesome .. this is coming together much faster than I thought it would.
mbbrutman
September 20th, 2006, 08:50 PM
The code has improved quite a bit, and I've learned much more. It's stable enough now such that I'm starting to measure performance.
I added some required error checking (the infamous checksum) for incoming data which slowed down receives quite a bit, but there really is no way around that. So now, as far as I can tell, I detect all required errors. Handling them gracefully is a different issue. :-)
I also cheated and found a simple way to force the TCP 'maximum segment size' to 1024, up from the default of 576. That helps things a lot because there is a bit of overhead to make a TCP/IP packet, so bigger packets reduces your overhead. Unless you start losing packets .. then it sucks. The size will be configurable at some point.
Lastly, to measure the speed of the code and the Ethernet card and not my disk drives, I've changed my testing methodology. If I do a 'speed test' the machine sends 4MB of random bytes from memory to a Linux machine instead of reading data from a file. The random crap is still error checked and has to arrive correctly. That simulates an infinitely fast disk. Similarly, on a receive I receive packets but then toss them away instead of writing to disk.
And here are the numbers
JR with WD Ethernet: Send 37.7KB/sec, Receive 36.9KB/sec
386-40 with NE2000: Send 475.0KB/sec, Receive 492.0KB/sec
I'll update tomorrow with numbers from an XT running an original 10MB hard drive, with data being written to and from the hard disk.
Mike Chambers
September 21st, 2006, 10:43 AM
The code has improved quite a bit, and I've learned much more. It's stable enough now such that I'm starting to measure performance.
I added some required error checking (the infamous checksum) for incoming data which slowed down receives quite a bit, but there really is no way around that. So now, as far as I can tell, I detect all required errors. Handling them gracefully is a different issue. :-)
I also cheated and found a simple way to force the TCP 'maximum segment size' to 1024, up from the default of 576. That helps things a lot because there is a bit of overhead to make a TCP/IP packet, so bigger packets reduces your overhead. Unless you start losing packets .. then it sucks. The size will be configurable at some point.
Lastly, to measure the speed of the code and the Ethernet card and not my disk drives, I've changed my testing methodology. If I do a 'speed test' the machine sends 4MB of random bytes from memory to a Linux machine instead of reading data from a file. The random crap is still error checked and has to arrive correctly. That simulates an infinitely fast disk. Similarly, on a receive I receive packets but then toss them away instead of writing to disk.
And here are the numbers
JR with WD Ethernet: Send 37.7KB/sec, Receive 36.9KB/sec
386-40 with NE2000: Send 475.0KB/sec, Receive 492.0KB/sec
I'll update tomorrow with numbers from an XT running an original 10MB hard drive, with data being written to and from the hard disk.
NICE! those are some very good numbers for computers that old.
mbbrutman
September 21st, 2006, 11:30 AM
Here are the updated numbers, with the XT included:
PCjr with WD Ethernet: Send 37.7KB/sec, Receive 36.9KB/sec
PC XT with 3Com Etherlink II: Send 36.3KB/sec, Receive 33.8KB/sec
386-40 with NE2000: Send 475.0KB/sec, Receive 492.0KB/sec
So the Jr is marginally faster than the XT, but mine has a NEC V20 and a different Ethernet card so it's close enough.
On a real world file transfer (885KB) using the XT I got the following:
Receive: 16.43KB/sec, Send: 16.57 KB/sec
Same machine, but using NCSA Telnet/FTP for the file transfer:
Receive: 17KB/sec, Send: 20.58KB/sec
Ugh, it would seem that I got spanked by 11 year old code. But you need to peel the onion a bit to see why.
NCSA's FTP code uses a 10000 or 20000 byte buffer. It doesn't read or write from the disk unless it has a full buffer, so on a machine with a slow disk it makes a big difference. To use such a large buffer they have to use memory copies to get data into packets, which is expensive but they have a nice memory copy loop.
My code uses 1KB buffers, but no memcopies.
Apparently the disk overhead is so bad that it pays to make fewer, bigger operations and suffer through the memcopies than it does to make more frequent operations, but not have the memcopies.
If I do a real FTP client and design it like that, then I should be able to beat NCSA's FTP, which is very good.
Mike
mbbrutman
October 6th, 2006, 07:29 PM
Still working on the TCP/IP code ...
Right now the code is capable of opening a socket connection to another machine, sending and receiving data, and closing the connection cleanly. The machine is rock solid during hours of coding and testing so I'm not corrupting memory or anything bad like that. And the Linux machine on the other side isn't getting mad at my sloppy disconnects anymore. :-)
It still performs fairly well, but it is getting slower ... as I add more features to meet TCP/IP requirements it slows down. Still, I'm getting 22KB/sec when sending a file and 16KB/sec when receiving, and that includes the overhead of writing to disk. And if I increase the size of my disk reads and writes, those numbers will improve. Those numbers are on a PC XT with the original 10MB hard disk and a 3Com Etherlink II. On my 386-40 the numbers are around 300KB/sec.
On the todo list:
- Add Unix style send and recv calls. This will slow things down by introducing a memcpy, but when doing file transfers it will open the door to larger disk reads and writes. I'll still have my current send & recv calls, which are fast because they avoid the extra memcpy but they limit the amount of data sent and received to what fits in a single packet.
- Add a listen call so that the code can be used to implement servers.
- Fix my connection reset detection code .. (or lack thereof)
- Go back and clean up my UDP implementation based on what I've learned from TCP. I haven't touched the UDP code in two months, and I'm sure it needs work.
- Do a simple DNS implementation so that you don't always have to specify raw IP addresses.
- Improve my timeout and retransmission code and my window-size code.
- More performance work.
Do we have any C programmers in the audience? I plan to use what I'm writing here for a telnet BBS, but it would be nice to have other people using some of this.
I've been working on it on and off for close to a year now. Last year when I started it was a struggle just to get a packet of any data over the Ethernet. Now I'm starting to get ideas for what the telnet BBS should look like. :-)
On a related note, is anybody else involved in a long term programming project on vintage hardware? If so, start a new thread here and let us know what your building!
carlsson
October 7th, 2006, 04:13 AM
I've not worked professionally in C, but as an advanced hobbyist. Not on older 8086/88 class systems though. I think I've used Turbo C version 1 or so on a Wang in PC emulation mode. Some of the more modern mechanisms were not implemented IIRC, like a function can not return void (?) or that calloc() has to be replaced by malloc().
vBulletin® v3.8.4, Copyright ©2000-2009, Jelsoft Enterprises Ltd.