DOS drive imaging backup via FTP!!

[Mike Chambers]

was bored again tonight, and wondering what else i could do that that quickbasic TCP code i wrote a while back. i remembered how i've been getting funky CRC errors at times during hard drive writes on my XT clone with the MiniScribe MFM drive in it... so i figured i'd put together a raw drive image backup program that uploads it to an FTP server.

it DOES work just fine, and i even tested the complete uploaded image file by loading it up in QEMU and doing extensive tests on it there and it all worked out great.

here's a couple screen shots from it running on the XT... i will upload the prog very shortly for you guys to download. i'm sure there are plenty of you on this site that's dealing with an old hard drive on it's last legs in at LEAST 1 computer. ;)

http://rubbermallet.org/netghost1.png

http://rubbermallet.org/netghost2.png

[Terry Yager]

Kewl!

--T

[Mike Chambers]

thanks, i was hoping you guys here would be able to make good use of it. right now i'm fixing a couple bugs in it, but i'll have it up for download soon.

[dongfeng]

Wow, perfect! I've wanted a software like this for ages, Laplink takes ages...

Nice one, Mike :D

[rachael24]

Looks good..thanks for the screenshots!

[barythrin]

Just a tip and feature tip and idea which is probably way more coding than of interest but 1. for the password you could just change the font to black during that input and back to white again at some point to avoid the password showing issue.

You could also add compression to the transfer if you wanted to sift through some open source zip or 7z code although this could take significantly longer than useful on a vintage system.

Definitely awesome an program! Great work :-)

- John

[Mike Chambers]

Just a tip and feature tip and idea which is probably way more coding than of interest but 1. for the password you could just change the font to black during that input and back to white again at some point to avoid the password showing issue.

You could also add compression to the transfer if you wanted to sift through some open source zip or 7z code although this could take significantly longer than useful on a vintage system.

Definitely awesome an program! Great work :-)

- John

yup! i was going to do that with the password input, i just haven't bothered yet. what i'm probably going to do soon is just re-design the entire interface with a GUI and mouse support to make it easier for noobs to use.

and yeah, i'm not even going to both with compression at least at this point. the 320 GB drive in the computer i'm using as the FTP server isn't complaining about storing an image of a 20 MB miniscribe lol.

quickbasic + TCP on an XT is slow enough as it is anyway. i generally get between 8 and 10 KB/s transfer rate when using this software after a compile. (about 6 KB/s(!!!) running interpreted inside of QB45.

sorry i haven't posted the program yet, there are a couple little annoyances i'm trying to fix before i upload it. it'll be within a day or two. however, here is proof that the uploaded image works... i told QEMU to use the raw image file it made as it's hda:

http://rubbermallet.org/proof.png

and there it is after booting, loading QB, and opening one of it's sample code files. :biggrin:

[Mike Chambers]

i will also, of course, write up a restore program to pull the data back off an FTP server and write it to a hard drive.

another very cool use for that would be the ability to make a new raw disk image in QEMU, load up whatever you want on it there and then write it to a real HDD.

it would make installing something like windows 3.0 a WHOLE lot faster using floppy images of it rather than actually making disks and installing it on an 8088. hehe.

[Micom 2000]

All I can say is "Wow". Great job Mike. When will we able to get the finished product ?

Lawrence

[Mike Chambers]

ty, i think i made it just in time too. my miniscribe is really starting to act up badly today. maybe the several full drive reads i've done with this program finally pushed it over the edge and it's ready to fail all the way. sometimes it won't boot now. it'll sit there clicking very loudly and making buzzing noises. then if i shut the PC off and back on it generally boots.

i guess i can't complain. being over 22 years old, that drive has definitely delivered faithfully until just recently. i will be posting the program binary and source later tonight.

[Mike Chambers]

alright guys, i know i said a few times i'd have it uploaded. sorry. problem is that my XT's hard drive is more or less completely gone. it usually doesn't even boot. good news is i was able to set it up as a secondary drive with another in good shape as the boot drive. miraculously, i was able to pull all of the important stuff i had on there off and put it on a couple floppies like all .BAS files, and some photos i didn't want to lose. yes, i know most people wouldn't trust a 23 year old computer with important photos... bad idea. i'm gonna not do that in the future.

anyway, on my 286 DR-DOS 7.02 is installing on a nice shiny (and almost brand new) 60 gig IDE drive.... DOS only sees 500 MB, but that's a lot better than the 20 MB MFM and it's soooooooooo much faster. when that's done i'm throwing QB45 on and i'll compile a binary to post.

don't expect it to work right. i have no idea if it will for you or not. that 8088 and old MFM drive is the only system i've tried it on so far. if you have problems please let me know so i can try to take care of them.

[Trixter]

You could also add compression to the transfer if you wanted to sift through some open source zip or 7z code although this could take significantly longer than useful on a vintage system.


The only compression that would be appropriate for a backup program would be something that is specifically tailored for high-speed compression (decompression speed is unimportant since backups are done much more than restores). The only compression that wouldn't noticeably slow down the program would be something like Oberhumer's LZO or Bloom's LZP, but neither of them have quickbasic interfaces IIRC. In fact, since the author is currently limited to quickbasic, I think even simple RLE is not an option.

However, there *are* assembler compression plugin libraries for quickbasic, but since I haven't coded in qb in 20 years I wouldn't know the first place to look. I think phatcode.net could help you in that area, Mike; or if not, maybe just a couple of google searches...

[Trixter]

anyway, on my 286 DR-DOS 7.02 is installing on a nice shiny (and almost brand new) 60 gig IDE drive.... DOS only sees 500 MB, but that's a lot better than the 20 MB MFM and it's soooooooooo much faster.

Don't fall into the trap of coding for the machine you're coding on. For example, if you're coding utilities that you claim to work on an 8088, you should test them there to see if they really do. Your 286 is at bare minimum twice as fast as a typical XT (and could be as much as 4 times as fast, depending on what clock speed it's running at) so be wary of that.

[barythrin]

I haven't seen the raw code or anything but was implying writing compression code himself if he's opening the device or files as binary or however it's doing it. But yeah, I think the time taken to compress the data would outweigh the speed of the program and size of the data to transfer so it wouldn't be a useful feature.

[Mike Chambers]

I haven't seen the raw code or anything but was implying writing compression code himself if he's opening the device or files as binary or however it's doing it. But yeah, I think the time taken to compress the data would outweigh the speed of the program and size of the data to transfer so it wouldn't be a useful feature.

nope, not opening files at all. you wouldn't be able to get a real image that way. i'm using BIOS interrupt calls at 13h to directly talk to the drive controller.

compression wouldn't be a bad idea, but i'll upload the current version shorty for everybody. i can put that in later after i verify all the main stuff works correctly. maybe i'll look up tech info on algorithms...


ugh, can you imagine compressing a whole drive with 100% QB code on an 8088? ::shudders::

now, if it's being done a more recent system... like a 486 or early pentium, it wouldn't be that bad.

[Trixter]

ugh, can you imagine compressing a whole drive with 100% QB code on an 8088? ::shudders::


No, but that's why you don't use QB code for the compression routines and instead link in an ASM library. There are plenty of sites with asm libraries that can be linked into QB 4.5, and surely one or more of them supports RLE or some other compression algorithm tuned for high speed.

I used to back up old systems using Norton Backup v1.0 which would perform realtime compression while it formatted and wrote to disks. It is a marvel of engineering, and since TCP/IP isn't going to be faster on an XT than writing to a floppy drive, it's certainly possible.

[Mike Chambers]

No, but that's why you don't use QB code for the compression routines and instead link in an ASM library. There are plenty of sites with asm libraries that can be linked into QB 4.5, and surely one or more of them supports RLE or some other compression algorithm tuned for high speed.

I used to back up old systems using Norton Backup v1.0 which would perform realtime compression while it formatted and wrote to disks. It is a marvel of engineering, and since TCP/IP isn't going to be faster on an XT than writing to a floppy drive, it's certainly possible.

true. i usually like to write all my own code, but for something like this there's definitely no sense in re-inventing the wheel. especially when i'd be re-inventing it worse than the existing wheel.

speaking of ASM routines, as a future project i want to write a telnet server that works as a TSR. something that would allow you to connect with a client and see/control what's really on the text screen like you're at the computer. i've been looking at using the timer interrupt to latch some code to for this, but you probably know a better way to do it.

it would just read the contents of the video memory starting at B800, tracking the differences between each read and then send that to the client.

if i feel adventurous, maybe some sort of VNC server that can handle the graphics modes as well. i'm not sure how possible that is in DOS (at least with reasonable speed. certainly couldn't use it to play games with.)

[Trixter]

speaking of ASM routines, as a future project i want to write a telnet server that works as a TSR. something that would allow you to connect with a client and see/control what's really on the text screen like you're at the computer. i've been looking at using the timer interrupt to latch some code to for this, but you probably know a better way to do it.

it would just read the contents of the video memory starting at B800, tracking the differences between each read and then send that to the client.


The quick way, which could be nearly as fast as realtime, would be to hook console input/output. That would limit you to the command-line and programs that used the BIOS, but the feedback/response would be nearly as good as the real thing. That would be step one.

Unfortunately, as you've discovered, there are too many programs that write directly to screen memory, so step two would be something like this:


Scan video memory
Compare to the last time you scanned video memory
Gather a list of changes
Send the changes only, using ANSI escape sequences to denote location and color


If you scanned video memory once a second, that shouldn't impact performance too much. You could make the update rate user-tunable.


if i feel adventurous, maybe some sort of VNC server that can handle the graphics modes as well. i'm not sure how possible that is in DOS (at least with reasonable speed. certainly couldn't use it to play games with.)

Anything is possible; whether or not it would be practical to spend a year or more writing it is the real question. I like silly projects as much as the next guy, but I never take on a project that would never be used by anyone.

[Mike Chambers]

yeah, lots of programs don't behave well enough to work by hooking the console lol. i was thinking the same thing about the ANSI codes. i have plenty of experience with ANSI. back in the mid-90's i wrote a multi-node BBS prog. it was pretty nice actually, with all kinds of little frills like multi-user chat mode (kinda like IRC) and split screen chat interface for sysop to caller.

ah the good ol' days. i miss BBSes!! i wish i still had the disk with the BBS code on it. i have no idea where that went.