DOS ain't dead

On Bret Johnson's forum at --

http://bretjohnson.us/forum/viewtopic.php?f=5&t=148

Note his last post dated Tue Jun 01, 2010 6:18 PM, in which he
says that UIDE's external interface is "essentially unusable",
that I am unwilling to do changes to UIDE's interrupt handling
and his users will have to "live with" USBDRIVE not being able
to be removed, when his next USBDRIVE is released.

I do hope folks "Understand BETTER!" than what Bret Johnson is
trying to make his readers believe!

I consider UIDE to be a "hard" SYSTEM driver, one which caches
essential devices like the system hard-disk and CD/DVD drives,
and thus people should NEVER want to unload it! I have been
asked about RDISK unloading, but absolutely NOBODY before Bret
Johnson ever inquired re: unloading UIDE. However, he seems
to want ALL drivers to be unloadable, and he very disdainfully
"puts down" ones that cannot be, even drivers that worked fine
BEFORE his USB showed-up!

Re: modifying UIDE's interrupt handling, I see no reason to do
this either. UIDE was written to "hook" Int 13h and thus get
"last look" at any I-O request BEFORE the actual BIOS, so UIDE
could "intercept" I-O for disks/diskettes and cache it. Bret
Johnson now wants HIS driver to get such a "last look", and so
I am "expected" to change UIDE to meet some new and previously
UNNEEDED interrupt-convention, merely to satisfy HIM! There
are many other drivers NOT using this convention that ALSO run
fine without it, and require NO changes to GO ON running fine!

Re: UIDE's external interface being "essentially unusable", it
in fact DOES work just fine. What Bret Johnson does not like
is that it does not allocate/deallocate cache-unit numbers, so
two different drivers can use the same number and corrupt each
others' data. This might occur if only "trainees" used UIDE!
Just a few friendly E-Mails, among the FEW of us who must deal
with cache-unit numbers, would ELIMINATE any need for allocate
and deallocate logic. UIDE now "reserves" cache-units 48 to
55 for CD/DVD drives, and its comments could ALSO note another
block of cache-units is "reserved" for USB. But Bret Johnson
noted in a PM that no allocate/deallocate code is "LUDICROUS"!

So much for "friendly" E-Mails, which I thus decided to END!!!

In 2006, I did NOT like being made by the "FreeDOS Pundits" to
look like a "bad guy" for creating XMGR (then QHIMEM), instead
of working ONLY with their FD-HIMEM. The above-noted post on
Bret Johnson's forum now tries to make just ME appear like the
bad guy, for REFUSING to "obey" ONLY HIS "agenda"! I did NOT
like 2006 -- at least they "went SILENT", after JEMM386/HIMEMX
sent FDEMM386/FDHIMEM "down the pipes!" -- and I do NOT accept
"LUDICROUS" applied to me now, for "Refusing to OBEY ..."!!

UIDE's design IS NOT the problem here! UIDE works fine, just
"as is", and I have NO reason to make any changes! As I also
did write to Khusraw and to Johnson Lam, UIDE in fact COULD be
made to run with USBDRIVE, "as is" and with NO changes, and in
complete support of ALL items on that "other agenda"!

But, I shall remain SILENT! After "all of the above", I hope
everybody realizes why UIDE with USB now flatly DISGUSTS ME!!!

Now, it is time for BREAKFAST at 11:35 A.M. after I can manage
to stop "Chewing NAILS!", in this case the METAL ones!!!

---
(Account disabled on user's request.)

> I consider UIDE to be a "hard" SYSTEM driver, one which caches
> essential devices like the system hard-disk and CD/DVD drives,
> and thus people should NEVER want to unload it! I have been
> asked about RDISK unloading, but absolutely NOBODY before Bret
> Johnson ever inquired re: unloading UIDE.

The only reason (IMHO) to unload would be to free up precious memory without requiring a reboot. And most users don't have any shortage of RAM, esp. for DOS apps. (However, I still think 15 MB used of 32 MB is too much, which you suggested once; heck, I even tested it, couldn't find any speedup, so I stick to 5 MB on that old clunker.)

>> I consider UIDE to be a "hard" SYSTEM driver, one which caches
>> essential devices like the system hard-disk and CD/DVD drives,
>> and thus people should NEVER want to unload it! ...
>
> The only reason (IMHO) to unload would be to free up precious memory
> without requiring a reboot. And most users don't have any shortage
> of RAM, esp. for DOS apps.

One has to be "Quite DESPERATE!" for memory, if saving the 944 bytes
of upper-memory used by the current UIDE is actually THAT important!

> However, I still think 15 MB used of 32 MB is too much, which you
> suggested once; heck, I even tested it, couldn't find any speedup,
> so I stick to 5 MB on that old clunker.

"The theory" of UIDE's caches is that, if possible, you should set a
cache TWICE the size of your largest data file, PLUS extra memory so
a reasonable amount of DOS directories can remain "in cache" and not
get discarded, when more new data arrives. Keeping DOS directories
"in cache" is the GREATEST advantage of using UIDE, as DOS otherwise
does only SLOW "designed in 1981" single-sector directory handling!

Using a 5-MB cache, if you have no files larger than about 2-MB, you
can copy such files using 4-MB of cache (2 for input, 2 for output),
and you still have about 1-MB left for directories. When DOS wants
to copy another file or load a new program, it loses NO speed, since
the directories were not discarded due to too-much "new" cache data.

Many DOS-only systems (NO Windows!) have only such small files, thus
a 5-MB UIDE cache is still useful for them. However, try copying a
BIG 100-MB file or more with only a 5-MB cache, and you will "see" a
speed loss, which you would NOT see if using a 250-MB cache or more!
The loss is small, and UIDE re-inputs discarded directories quickly.
But for BEST speed, do set UIDE cache sizes using "the theory" above!

---
(Account disabled on user's request.)

I just read the 06/01 post at bret's forum. AFAIU the workaround loading

a) USBDRIVE - and then
b) UIDE via DEVLOAD

does work currently and will also work after his planned additions. The only drawback is that USBDRIVE will be "unremovable" afterwards.

If this is true, then IMO it is a rather acceptable condition. No need to invest further time for improvements.

---
MS-DOS forever!

> Note his last post dated Tue Jun 01, 2010 6:18 PM, in which he
> says that UIDE's external interface is "essentially unusable",

Please don't assume evil faith.

Yeah an unlodable TSR is IMHO the better way to do than DEVICE= , at least until someone proves me wrong here

Why uninstall UIDE? Well, to measure the performance without it maybe ?

FYI, I appreciate the work of both Bret and Jack

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

> > However, I still think 15 MB used of 32 MB is too much, which you
> > suggested once; heck, I even tested it, couldn't find any speedup,
> > so I stick to 5 MB on that old clunker.
>
> "The theory" of UIDE's caches is that, if possible, you should set a
> cache TWICE the size of your largest data file, PLUS extra memory so
> a reasonable amount of DOS directories can remain "in cache" and not
> get discarded, when more new data arrives. Keeping DOS directories
> "in cache" is the GREATEST advantage of using UIDE, as DOS otherwise
> does only SLOW "designed in 1981" single-sector directory handling!

The problem is that backends for GCC 3.4.4 (which I use there) are 3.5 MB unpacked. (Latest DJGPP GCC 4.4.2 is 8.5 MB for CC1.EXE, yikes.) The 2.04 libc itself is 800k. And the assembler and linker (AS, LD) are also pretty big (800k, 600k). The internal Pentium cache is ridiculously small.

(EDIT: Looks like 3.4.4 CC1.EXE is actually 4.2 MB, I forgot I must've recompiled GCC 3.4.4 in blind hopes of speedup a while back.)

Anyways, I just now tried a bit messing with GPC. Seems that the linking stage takes the longest, so copying the relevant libs to RAM disk and using -L. speeds it up more than UIDE /S15 did. Hence, I'll stick to /S5 for now. (Plus I usually need the RAM for other stuff. And this is also why I like TDSK to be able to resize, as by default I use 5 MB for it too.)

> Using a 5-MB cache, if you have no files larger than about 2-MB, you
> can copy such files using 4-MB of cache (2 for input, 2 for output),
> and you still have about 1-MB left for directories. When DOS wants
> to copy another file or load a new program, it loses NO speed, since
> the directories were not discarded due to too-much "new" cache data.

Well, choice of software affects this, and you can indeed blame me for using either a "low RAM" cpu or bloated compiler or whatnot. I know TP55 and TMT are probably faster, but they aren't as good for some (most?) things. Haven't benchmarked FPC on that clunker yet (have to pare it down first, been busy with other things). They always claim faster compilation, so it would be nice to see if that's true.

> Why uninstall UIDE? Well, to measure the performance without it maybe ?
>

Hm this is used very often by users, lol :)
So you can just reboot and choose different menu in config.sys With JEMM quick boot feature it's done in 5 seconds ;)

---
DOS gives me freedom to unlimited HW access.

> So you can just reboot and choose different menu in config.sys
Totally agree. I can't remember when I last unloaded a TSR or device driver.

> with JEMM quick boot feature it's done in 5 seconds ;)
I haven't used JEMM but that sounds like the old Int 19h trick or Lazy Man's reboot as I call it:

copy con lmboot.com
[ALT 205][CTRL^Y][CTRL^Z]

useful at times but also risky.

> I do hope folks "Understand BETTER!" than what Bret Johnson is
> trying to make his readers believe!
>
> I consider UIDE to be a "hard" SYSTEM driver, one which caches
> essential devices like the system hard-disk and CD/DVD drives,
> and thus people should NEVER want to unload it! I have been
> asked about RDISK unloading, but absolutely NOBODY before Bret
> Johnson ever inquired re: unloading UIDE. However, he seems
> to want ALL drivers to be unloadable, and he very disdainfully
> "puts down" ones that cannot be, even drivers that worked fine
> BEFORE his USB showed-up!

FYI: There seems to be a problem with your caps-lock-key. It seems to hang now and then.

> FYI: There seems to be a problem with your caps-lock-key. It seems to
> hang now and then.

Only when I wish it so -- I was taught capital letters are for EMPHASIS!

---
(Account disabled on user's request.)

> > get discarded, when more new data arrives. Keeping DOS directories
> > "in cache" is the GREATEST advantage of using UIDE, as DOS otherwise
> > does only SLOW "designed in 1981" single-sector directory handling!
>
> The problem is that backends for GCC 3.4.4 (which I use there) are 3.5 MB
> unpacked. (Latest DJGPP GCC 4.4.2 is 8.5 MB for CC1.EXE, yikes.) The 2.04
> libc itself is 800k. And the assembler and linker (AS, LD) are also pretty
> big (800k, 600k). The internal Pentium cache is ridiculously small.

But only the working set size matters for cache, not the overall size. If you have a 99.99% hit ratio in 32k, it will work fine

> Anyways, I just now tried a bit messing with GPC. Seems that the linking
> stage takes the longest, so copying the relevant libs to RAM disk and
> using -L. speeds it up more than UIDE /S15 did. Hence, I'll stick to /S5
> for now. (Plus I usually need the RAM for other stuff. And this is also
> why I like TDSK to be able to resize, as by default I use 5 MB for it
> too.)

This is some what strange. While I do directly understand that the ramdisk speeds up, I'm a bit surprised this is very noticable in the linking step.

> > Using a 5-MB cache, if you have no files larger than about 2-MB, you
> > can copy such files using 4-MB of cache (2 for input, 2 for output),
> > and you still have about 1-MB left for directories. When DOS wants
> > to copy another file or load a new program, it loses NO speed, since
> > the directories were not discarded due to too-much "new" cache data.
>
> Well, choice of software affects this, and you can indeed blame me for
> using either a "low RAM" cpu or bloated compiler or whatnot.

You forgot the most important part: choice of computers

> I know TP55 and TMT are probably faster, but they aren't as good for some
> (most?) things.

Depends on what you do. For a quick calculation TP55 might be quite ok. (at least if it supports copro), but with serious programs, you get out of memory fast.

Anyway, TP and VP share the problem that they are unmaintained, something that I avoid for active development.

> Haven't benchmarked FPC on that clunker yet (have to pare it down
> first, been busy with other things). They always claim faster compilation,
> so it would be nice to see if that's true.

Its not so much the compilation as the integrated assembler/linker.

But contrary to other systems, I assume DJGPP might have had a lot of dos-specific work done to mitigate the standard naieve approach of the GNU toolchain, so it might not be so prominent on Dos.

Note that that is fast compared to GCC/GPC, not to the rest.

Lastely there is the 300 pound gorilla, the delphi commandline compiler with wdosx, generating wdosx apps.

> > FYI: There seems to be a problem with your caps-lock-key. It seems to
> > hang now and then.
>
> Only when I wish it so -- I was taught capital letters are for EMPHASIS!

AnD I ThOuGhT YoU OnLy DiD iT tO mAkE iT hArD tO rEaD

Please don't, rants are already fatiguing enough.

Re: my ongoing "issues" about UIDE used with USBDRIVE, there is another huge
problem --

Bret Johnson has noted on his forum that he plans a "workaround" for what he
sees as UIDE's problems, in which USBDRIVE will load first, and UIDE will be
loaded afterward by Eric Auer's DEVLOAD or equivalent. UIDE can then "see"
every USB device I-O request and cache them using its "call the BIOS" logic.
"The BIOS" in this case will be USBDRIVE, which will "intercept" and perform
needed I-O, then UIDE will cache the data (if needed) on exit from USBDRIVE.

The problem: USBDRIVE declares all of its drives as "removable" HARD disks,
when an EDD BIOS "Int 13h AH=048h" call is issued!

DOS is an "old" system, designed when a hard-disk really was HARD, and never
"removable"! If a supposed hard-disk were to return a "media change" code,
as diskette drives do when a new one is inserted, I DARE NOT "trust", across
all DOS kernels, that their internal disk buffers (CONFIG.SYS "BUFFERS=" and
"BUFFERSHIGH=") are going to get re-input! There was never any requirement
for HARD disks in a DOS system to cause this, nor even to present any "media
change" codes to the system!

In the past I have owned Iomega JAZ and Castlewood ORB cartridge disks, both
good ideas, but each failed after only 6 months which was "not good"! When
used with V6.22 MS-DOS, in fact I did notice that the only "reliable" way to
make DOS "forget" all data for an unloaded cartridge was to REBOOT!!

Nor are "media changes" a responsibility for UIDE, either. UIDE is not the
"driver" here -- USBDRIVE is!! UIDE merely provides a "caching service" to
USBDRIVE, no-matter which of the two may or may-not load first! So, it is
a responsibility for USBDRIVE to "tell UIDE" when its cache must be flushed,
which UIDE's "external reset" entry-point does permit.

If all USBDRIVE does is to issue Int 13h I-O commands, but NOT "notify" UIDE
that a media-change requires a cache-flush, then "data corruption" far worse
than Bret Johnson ever imagined WILL occur, using his "workaround"!

And "Puh-LEEZE" do NOT ask me to add such media-change logic -- UIDE's SATA/
IDE routines run HARD disks, NOT funny-bunny REMOVABLE "hard" disks, same as
most DOS kernels still expect! I have no wish to "clutter up" UIDE's HARD
disk logic, nor give-up any of its performance, for "removability" BULLSH*T!

To protect against these problems, and the even more-unresolvable one of DOS
flushing its internal hard-disk "buffers", I have decided to change the UIDE
driver so it will NOT "handle" any BIOS unit of 080h or greater (hard disks)
that are flagged "removable" by an "Int 13h AH=048h" call! Any such units
will now be ignored by UIDE's initialization logic, and I-O for them will be
totally "passed" to the next driver in the Int 13h "chain", i.e. NOT cached!

I WILL NOT permit UIDE to get blamed for DOS or other-driver design FAULTS!!

My Thanks to Khusraw for testing the new driver (he has USB units, I do not)
and it will shortly become the "standard" UIDE.

If USBDRIVE desires to call UIDE for caching, using its "external entry" and
"external reset" call points as intended, "fine and dandy", and USBDRIVE can
deal with "removability" issues any way it desires, as it is the only driver
which SHOULD deal with them! But henceforth, UIDE shall support only FIXED
hard-disks in its SATA and IDE disk routines!

---
(Account disabled on user's request.)

> AnD I ThOuGhT YoU OnLy DiD iT tO mAkE iT hArD tO rEaD
>
> Please don't, rants are already fatiguing enough.

Agreed. So if you will give up this not-very-humorous thread about
English style, that I was taught in High-School and has been part of
my life for nearly 50 years since then, I will spare you my comments
on the Cyrillic alphabet, and perhaps the word "nekulturny" as well.

---
(Account disabled on user's request.)

> > > FYI: There seems to be a problem with your caps-lock-key.
Jack might have being posting via a C64 ;)

> AnD I ThOuGhT YoU OnLy DiD iT tO mAkE iT hArD tO rEaD
?daer ot drah eb ot ti detnaw eh fi sdrawkcab egassem sih nettirw evah dlouw kcaJ yleruS


> Please don't, rants are already fatiguing enough.
I can see both perspectives and would hope that all parties concerned can aim for some middle ground.

> I can see both perspectives and would hope that all parties concerned
> can aim for some middle ground.

Also agreed. In my post below, about "No Removable HARD Disks", I did
in fact eliminate much capitalization before posting it, since Marcov's
post was noted by me.

Why don't we keep BTTR as the best "technical" board, and stop all such
unneeded items. That, in fact, is why I post here, and not elsewhere.

---
(Account disabled on user's request.)

> > with JEMM quick boot feature it's done in 5 seconds ;)
> I haven't used JEMM but that sounds like the old Int 19h trick or Lazy
> Man's reboot as I call it:

Yes, probably. Also QEMM that I used before has this feature. It's good when e.g. tunning maximum low memory and rebooting very often...

> copy con lmboot.com
> [ALT 205][CTRL^Y][CTRL^Z]

Hehe copy con, hardcore programmer's way :)

---
DOS gives me freedom to unlimited HW access.

> > I can see both perspectives and would hope that all parties concerned
> > can aim for some middle ground.
> Also agreed.
Thanks.

I have briefly familiarized myself today with Bret's work - like Jack's work I can see Bret's code is also well structured and well commented. Good stuff!


I have always liked flexible but reliable solutions. So in the hope of helping to find some diplomacy/middle ground to help resolve this situation which isn't benefiting anyone I thought I would add some personal perspective on some the points raised by way of focusing on "flexibility and reliability".


Firstly concerning flexibility I agree with various points of view that have been raised re supporting TSR loading, particularly when I know (having done this myself many years ago) that it is technically possible to have a loadable CONFIG.SYS device driver (EXE format) which can also be run from the command-line by adding very little extra code. So I agree ad-hoc TSR device driver loading is a good thing as is loading devices early in a PC's start up.

In other words in an ideal world it would be good in terms of "flexibility" if you both implemented your work as combined device driver / loadable TSR's.


Normally unloading TSR's/devices is also a very good thing, however I have reservations about the safe and reliable unloading of low level DISK routines.
That is to say I would question if the unload risks outweigh any flexibility.


On a similar note when it comes to reliability I believe that it is always very dangerous to ASSUME anything with the old ASSUME adage being so often true.

On that basis I believe there have been very valid points raised about media change codes / kernel differences. Indeed I remember issues with the Iomega parallel port DOS drivers with a reboot being the best option. I also vaguely remember various oddities with Stacker on removable floppies (don't ask...!).


Certainly I believe Jack is right to be cautious and I greatly respect that approach with regards to disk routines. However the Pareto principle aka 80:20 rule is also a good guide in that it is impossible (and not worthwhile) to aim for 100% perfection in anything. As programmers we will never think of all possible events but it is important to cover the most likely. So in some ways Jack I would say that you are being over cautious in some ways but not without good reason.

So what might be the middle ground here is to take a flexibility/safety compromise in allowing people to do things that could be dangerous but at the same time giving them very large warnings/prompts before action. e.g. "Yes, you can unload this disk routine and yes I would flush my buffers. But be aware that you do so at your own risk. Are you sure you wish to continue?".


> Why don't we keep BTTR as the best "technical" board, and stop all such
> unneeded items. That, in fact, is why I post here, and not elsewhere.

I think it is important to keep BTTR inclusive. Personally I have seen very few non-worthwhile posts on here. We can all learn off each other. The noise ratio here is also low compared to many other forums as we are mostly sensible with our posts most of the time. Sure occasionally we may all write a longer post than we mean, or might be tired or had too many beers etc, on the whole I would say that this is a very good forum. Thanks RR for hosting!

Indeed I have decided that I will post a DOS USB bug report here for Bret to see simply because I do NOT wish to sign up to yet another forum; sorry Bret!

> Indeed I have decided that I will post a DOS USB bug report here for Bret
> to see simply because I do NOT wish to sign up to yet another forum; sorry
> Bret!

On second revision it would appear that the WriteZErr / OvlNotFoundMsg2 error display bug I found in USBUHCI was fixed in a later version (just re-tested and realized I tested with an earlier ZIP that I downloaded but not studied):

This was the bug in v0.11:


USBUHCIL 0.11, (C) 2007-2009, Bret E. Johnson.
DOS Driver for a Universal HCI compatible USB Host Controller.
  LITE version (maximum 16 Devices, no Isochronous Transactions).

Required Data File USBUHCIL.OVL is missing, invalid, or incorrect version.♥x☻└?#
╚■Ç☻Ö►



which seems to have been fixed in later versions:


USBUHCI 0.14, (C) 2007-2010, Bret E. Johnson.
DOS Driver for a Universal HCI compatible USB Host Controller.

Required Data File USBUHCI.OVL is missing, invalid, or incorrect version.


Likewise the #03 bug in the AMIS header was also fixed. However I did note that USBUHCI.A36 from the latest usbsourc.zip states "Changes in version 0.15" in the heading but the version number is coded as 0.14 in the rest of the code. It would appear from the changes.doc that the zip file that I downloaded was version 0.14.

>
> But only the working set size matters for cache, not the overall size. If
> you have a 99.99% hit ratio in 32k, it will work fine

I would bet that most software, even so-called 586-optimized, has lots of cache misses and unpaired instructions. I need to test more (e.g. VP21, which only goes up to 586, oddly enough.)

> > Anyways, I just now tried a bit messing with GPC. Seems that the
> linking stage takes the longest
>
> This is some what strange. While I do directly understand that the ramdisk
> speeds up, I'm a bit surprised this is very noticable in the linking step.

Well, you once told me a horror story about GNU ld, so you can't be that surprised. Maybe I'll swap to 2.16.1 temporarily and use HDPMI32.

> > Well, choice of software affects this, and you can indeed blame me for
> > using either a "low RAM" cpu or bloated compiler or whatnot.
>
> You forgot the most important part: choice of computers

Benchmarking only on newer computers is fine, but it doesn't give you the full picture.

> > I know TP55 and TMT are probably faster, but they aren't as good for
> some
> > (most?) things.
>
> Depends on what you do. For a quick calculation TP55 might be quite ok.
> (at least if it supports copro), but with serious programs, you get out of
> memory fast.

I originally didn't care about TP at all, but since it exists and I can test it, I tried. It works there too, so I'm happy. I don't need much RAM here (Befunge93). Plus I can (maybe) tell Trixter to test it on his 8088!

> Anyway, TP and VP share the problem that they are unmaintained, something
> that I avoid for active development.

More important is bugs (TMT -> EDGETEST.BEF) or lack of OS support to actually run them (Win64)! (VP21 has a leg up there.)

> > Haven't benchmarked FPC on that clunker yet (have to pare it down
> > first, been busy with other things). They always claim faster
> compilation,
> > so it would be nice to see if that's true.
>
> Its not so much the compilation as the integrated assembler/linker.

But there is no integrated linker for DOS FPC (despite being a variant of COFF), and LD 2.19 isn't supported in 2.4.0 (bug, "already fixed").

> But contrary to other systems, I assume DJGPP might have had a lot of
> dos-specific work done to mitigate the standard naieve approach of the GNU
> toolchain, so it might not be so prominent on Dos.
>
> Note that that is fast compared to GCC/GPC, not to the rest.

It only has to be reasonable, not blisteringly fast. I don't think most people try one-pass compilation anymore. It would matter more on a 486 or with a big project.

> Lastely there is the 300 pound gorilla, the delphi commandline compiler
> with wdosx, generating wdosx apps.

I don't have any Delphi, though, so I can't test. Besides, to be honest, WDOSX works well, but it has some compatibility issues.

---
Know your limits.h

> In other words in an ideal world it would be good in terms of
> "flexibility" if you both implemented your work as combined device
> driver / loadable TSR's.

Unneeded for UIDE, given Eric Auer's DEVLOAD that can load it at any time.

> Normally unloading TSR's/devices is also a very good thing, however I
> have > reservations about the safe and reliable unloading of low level
> DISK routines. That is to say I would question if the unload risks
> outweigh any flexibility.

There would be few risks unloading UIDE, as it "hooks" only the Int 13h
vector and can easily "give back" its XMS and upper memory. The problem
is that most users load it in the HMA, and I am not sure if HMA space can
be "given back". Also, when it uses HMA space, UIDE normally takes only
944 bytes of upper/DOS memory for its stack (520 bytes), entry and exit
logic, and its common data. As I noted in this thread to Rugxulo, are
people THAT "desperate" for memory that only 944 bytes really matters??

> Certainly I believe Jack is right to be cautious and I greatly respect
> that approach with regards to disk routines. However the Pareto principle
> aka 80:20 rule
> is also a good guide in that it is impossible (and not worthwhile) to aim
> for 100% perfection in anything. As programmers we will never think of
> all possible events but it is important to cover the most likely. So in
> some ways Jack I would say that you are being over cautious in some ways
> but not without good reason.

Computers are among VERY few "black and white" subjects on earth. They
either work 100%, or they "Fall on their NOSES!", with not so many "minor
bug" situations in the middle. Also, since most of us on this board are
"senior" (not by age, like me, but by KNOWLEDGE), I hope none of you will
be offended when I say there ARE "users out there" who WILL tend to blame
drivers like UIDE for all their problems, since UIDE was not part of the
original DOS system and is not absolutely necessary, despite its speed or
performance. MSDOS.SYS and COMMAND.COM in fact ARE absolutely required,
but my drivers aren't. In their minds, that makes UIDE a whole lot more
"suspect", and so I MUST be cautious. Also, I want UIDE to be "generic"
and run across ALL variants of DOS, and that too requires caution.

"Blessed are the peacemakers", and you will get to Heaven before me, my
friend. However, on the idea of a hard-disk being fixed or removable,
I do not see any "middle ground" that is reliable-ENOUGH for me to permit
in UIDE. DOS hard-disk logic was never designed to support removability
and may in fact IGNORE a media-change code, as its disks ARE supposed to
be "HARD". So "down the pipes!" you go, as you also noted for cartridge
devices/software. On that issue, I also MUST remain cautious, and I am
about to send a 9-Jun-2010 DRIVERS.ZIP to Khusraw and to Johnson Lam for
their final tests.

>> Why don't we keep BTTR as the best "technical" board, and stop all such
>> unneeded items. That, in fact, is why I post here, and not elsewhere.
>
> I think it is important to keep BTTR inclusive. Personally I have seen
> very few non-worthwhile posts on here. We can all learn off each other.

Well, if even Marcov can get me to eliminate a few capital letters, then
I guess you are right, and "A gentleman and a scholar in ANY universe!",
as Captain Kirk once said to the "Pirate Ship" Mr. Spock in my all-time
favorite 1967 Star Trek episode entitled "Mirror, Mirror"!

---
(Account disabled on user's request.)

> I haven't used JEMM but that sounds like the old Int 19h trick or Lazy
> Man's reboot as I call it:
>
> copy con lmboot.com
> [ALT 205][CTRL^Y][CTRL^Z]
>
> useful at times but also risky.

It isn't that simple - and it requires a cooperative DOS which knows what to do when an INT 19h is issued. IIRC some adjustments were necessary for the FD kernel to make it compatible.

Even with a cooperative DOS kernel the fast reboot isn't fool-proofed, probably because some BIOSes will expect certain values in their (extended) data segment inside the INT 19h code.

---
MS-DOS forever!

> It isn't that simple - and it requires a cooperative DOS which knows what
> to do when an INT 19h is issued. IIRC some adjustments were necessary for
> the FD kernel to make it compatible.

Yes it's not such simple. I also remember some old brogram fastboot or so which first needed to dump and store virgin IVT during boot to a files (it used floppy disk to do that) and then it used this file to restore IVT during fast reboot.

---
DOS gives me freedom to unlimited HW access.

> > So you can just reboot and choose different menu in config.sys
> Totally agree. I can't remember when I last unloaded a TSR or device
> driver.
>
> > with JEMM quick boot feature it's done in 5 seconds ;)
> I haven't used JEMM but that sounds like the old Int 19h trick or Lazy
> Man's reboot as I call it:
>
> copy con lmboot.com
> [ALT 205][CTRL^Y][CTRL^Z]
>
> useful at times but also risky.
I use to use reboot routine of keyboard controller (out 64h,feh).

> There would be few risks unloading UIDE, as it "hooks" only the Int 13h
> vector and can easily "give back" its XMS and upper memory. The problem
> is that most users load it in the HMA, and I am not sure if HMA space can
> be "given back".

Not in MSDOS 5/6, but yes in MSDOS 7+ it is easy enough to free a block of HMA (that you "own"!) and give it back to DOS - each allocated block is preceded by a control block (similar to an MCB, let's call them HMCBs). There is no API for giving the block back,[°edit : or maybe there is one, I don't recall and can't check now but it doesn't matter as...] we can achieve that by direct HMCB editing... However that will leave a hole in the HMA which probably won't be reused anyway, there is no way to move system blocks around and defragment the HMA ;)

As for FreeDOS and others, I have no idea how they manage allocation from the HMA. Oh, I seem to remember DRDOS 6 had a working scheme, much in advance of MS-DOS (of course...)

---
Ninho

> Not in MSDOS 5/6, but yes in MSDOS 7+ it is easy enough to free a block of
> HMA (that you "own"!) and give it back to DOS - each allocated block is
> preceded by a control block (similar to an MCB, let's call them HMCBs).
> There is no API for giving the block back,[°edit : or maybe there is one,
> I don't recall and can't check now but it doesn't matter as...] we can
> achieve that by direct HMCB editing... However that will leave a hole in
> the HMA which probably won't be reused anyway, there is no way to move
> system blocks around and defragment the HMA ;)

Ignoring older and more-obsolete versions, both V6.22 and V7.10 MS-DOS have
undocumented "issues" re: using HMA space. V6.22 MS-DOS declares over 26K
of "free HMA", but if I dare use over 19,952 bytes (UIDE /S600 /HL /P), the
system CRASHES rather quickly above a 600-MB cache size. V7.10 also fails
if UIDE takes over 9136 bytes of "free HMA", as I've known for years and as
Lucho can confirm re: his use of DOS "extenders". For V7.10, a difference
of only 16 bytes, re-assembling UIDE with 16 bytes of unused "filler", will
make V7.10 MS-DOS survive or CRASH, as Lucho can also confirm. NOT good,
even ignoring the "Damned LIE!" of over 13K free HMA using V7.10 MS-DOS!

As such situations exist, and as Gates & Co. have "walked away" from MS-DOS
and "DON'T want to talk-about!" such kernel and other never-to be-corrected
[EMM386] ERRORS, I shall again be "conservative" like Arjay noted and say I
am AGAINST "playing games" re: MS-DOS HMA space, specifically "giving back"
HMA once it has been allocated. I have strong reason to expect this might
not work, and might also "Get you DEAD!!", given all of the above.

> As for FreeDOS and others, I have no idea how they manage allocation from
> the HMA. Oh, I seem to remember DRDOS 6 had a working scheme, much in
> advance of MS-DOS (of course...)

Nor do I have any-idea, as I have never reviewed that part of FreeDOS. If
you seriously want to consider "giving back" HMA space, you should speak to
Tom Ehlert, as he wrote the FreeDOS HMA logic, IIRC.

---
(Account disabled on user's request.)

> Ignoring older and more-obsolete versions, both V6.22 and V7.10 MS-DOS have
> undocumented "issues" re: using HMA space. V6.22 MS-DOS declares over 26K
> of "free HMA", but if I dare use over 19,952 bytes (UIDE /S600 /HL /P), the
> system CRASHES rather quickly above a 600-MB cache size. V7.10 also fails
> if UIDE takes over 9136 bytes of "free HMA", as I've known for years and as
> Lucho can confirm re: his use of DOS "extenders". For V7.10, a difference
> of only 16 bytes, re-assembling UIDE with 16 bytes of unused "filler", will
> make V7.10 MS-DOS survive or CRASH, as Lucho can also confirm. NOT good,
> even ignoring the "Damned LIE!" of over 13K free HMA using V7.10 MS-DOS!
>

I don't doubt your or Lucho's findings, nor shall the finding of one more MSDOS bug astound me :=) I've used MSDOS provided HMA since it existed (we're speaking sharing the HMA with the DOS kernel here, that is to say after "DOS=HIGH", not plain HIMEM/XMS HMA, right ?) but never been hit by bugs myself. Just a verification we're on the same wave length, and not to be taken as an insult, you're 100% sure you are using the local open/local close HMA calls systematically before/after jumping to your HMA code. Otherwise depending on the config, DOS "closes" the A20 line for compatibility with very old exepack'd software which could cause strange crashes when you try to jump to your code above 1024k. Just a wild guess.

Regards


--
Ninho

>> Ignoring older and more-obsolete versions, both V6.22 and V7.10 MS-DOS
>> have undocumented "issues" re: using HMA space ...
>
> I don't doubt your or Lucho's findings, nor shall the finding of one more
> MSDOS bug astound me I've used MSDOS provided HMA since it existed
> (we're speaking sharing the HMA with the DOS kernel here, that is to say
> after "DOS=HIGH", not plain HIMEM/XMS HMA, right?) ...

Correct.

> ... but never been hit by bugs myself. Just a verification we're on the
> same wave length, and not to be taken as an insult, you're 100% sure you
> are using the local open/local close HMA calls systematically before/after
> jumping to your HMA code. Otherwise depending on the config, DOS "closes"
> the A20 line for compatibility with very old exepack'd software that could
> cause strange crashes when you try to jump to your code above 1024k. Just
> a wild guess.

UIDE uses exactly ONE run-time "A20 local enable", XMS request-code 05h, in
both its "Entry" logic for BIOS/"external" I-O and its "DevInt" entry logic
for SHCDX33E/MSCDEX calls to its CD/DVD driver. When UIDE finishes an I-O
request or if its BIOS logic "passes" a request, it then does an "A20 local
disable", XMS request-code 06h, before returning to the caller. Both such
calls are issued in UIDE's 944-byte "base" logic that always uses upper- or
640K DOS memory.

I am "rather-well aware" of what happens if one jumps up to the HMA without
issuing those calls correctly -- we in the U.S. would call it "Kissing your
[rear-end] GOODBYE!", i.e. CRASH!! UIDE has used those calls successfully
for 3 years!

---
(Account disabled on user's request.)

> > driver / loadable TSR's.
> Unneeded for UIDE, given Eric Auer's DEVLOAD that can load it at any
> time.
Personally I tend to like avoiding external helper applications if I know I can do something without them. Indeed I agree with DOS386's comments in this earlier LOADSYS | DEVLOAD | DDU discussion re device loaders being a "hack" - a useful hack at that (like PING, Mode-X etc) but if there is a better way of doing something that avoids the need for hacks then surely that is even better? In a "gentlemanly" fashion I am happy to "agree to disagree". I have however also done some reading to better understand/remind myself re drivers:

Firstly my humble apologies to both you and Bret as my subsequent "refresher" reading (mostly my well thumbed copy of PC Intern) and studying of your respective sources has highlighted to me that I made a few fundamental errors in my earlier comments (as I have not dealt with DOS drivers for some years and have thus forgotten various key points over time, e.g character vs block).

I also reminded myself that installable .EXE device drivers are effectively TSR's "acting" as "character" devices and only supported device function 00h (initialization). I am partly paraphrasing the PC Intern's author on these later points rather than quoting these as facts - or to put it another way I have read many things that say you can't do that but then found the opposite.

Concerning .SYS device drivers I also forgot about the "next driver seg/ofs" field values having the annoying default values of FFFFh which is an invalid op-code combination (???? DI) thus preventing a .SYS file from being a .COM file. Doh! Still related to that I was interested to note how UPX (which I spotted you use on XMGR) carefully preserves the 3 first fields of the .SYS header (ofs/seg/attr) whilst still allowing packing including even the last field (device name in case of a character device) which I thought was neat :)

On this area I think that area of UIDE.asm would read better written as:

@       dw       0FFFFh         ; Offset addr to next driver  (filled by DOS)
        dw       0FFFFh         ; Segment addr to next driver (filled by DOS)



Anyway I have reminded myself combined drivers are not a straight forward as I "vaguely" remembered. I also have remembered config.sys install came about in a latter versions of DOS. DOS 5?? (along with himem.sys?) or was it v 4?

As mentioned earlier I believe as much flexibility as possible is always good,
however programmers need to draw lines somewhere, e.g. UIDE is already 386+

So obviously if a .SYS device is a character device driver only using the init function then in my "flexibility" book it makes sense if that device driver is also available as an installable EXE driver (via CONFIG.SYS or AUTOEXEC.BAT) without the need to rely on any additional programs.

However my dusting off of this area and review of UIDE has highlighted that UIDE uses the attribute C800h which as binary 1100100000000000b decodes I believe as follows:

1 - UIDE is a character device
1 - functions 03h-0Ch are supported
0 - function 10h is *NOT* supported
0 - reserved
1 - functions 0Dh-0Eh are supported


Correct? According to what I have read function 00h (drive initialization) only allows function 01h to 0Ch to be called as DOS has not yet finished initializing as that point. In otherwords if I am reading this correctly it means although UIDE is a character driver it can't easily become an EXE based driver as it supports other functions. I also appreciate IOCTL comes in etc.

Obviously there are other ways of loading device drivers from DOS (and doing these things) but I'm just looking/working to understand UIDE as it "stands".


> There would be few risks unloading UIDE, as it "hooks" only the Int 13h
Yes, noted in the source which I have now studied more than I did previously.


> logic, and its common data. As I noted in this thread to Rugxulo, are
> people THAT "desperate" for memory that only 944 bytes really matters??
:) Well I guess it depends on the circumstances. I once had to write a menu system in assembler just because a new network stack on a 80186 network left no memory for the earlier DOS menu shell to be able load an app
However that was with DOS low mem. I can't say I've experienced similar upper memory problems but I can imagine them from what Rugxulo has written.

Although I like flexibility I also appreciate that you can't please everyone.


> Computers are among VERY few "black and white" subjects on earth. They
> either work 100%, or they "Fall on their NOSES!", with not so many "minor
> bug" situations in the middle.

It still amazes me every day when people turn on their computers and they just "work" considering the significant number of people involved in producing their various parts over the years, the number of mistakes/flaws.

Indeed having also read/studied many books/texts such as Fatal Defect - chasing killer computer bugs - in the past I would have tended to have fully agreed with you (particularly around Y2K!!). However over time I have realized it simply is not worth trying to dot every "i" and cross every "t" even when it comes to software reliability and that there is a careful balance between perfection for reliability and perfection for the sake of perfection. I have certainly been guilty of the later in the past myself.

> I hope none of you will be offended when I say there ARE "users out there"
> who WILL tend to blame drivers like UIDE for all their problems,
Well firstly I think it is important to remember that we have all been users at one point. Indeed back in the days before built-in DOS help existed and flashing warnings. Due to not bothering to fully RTFM a large manual (at that time); I discovered first hand just what FDISK could do the "hard way" only a few hours after getting my own PC and in my excitement wanting to split a 50Mb drive into a C: + D:... The "delete existing partition" warning is now somewhat clearer now then it was then Did I blame MS? No I didn't.

So we can all be idiots at times. Even some of the worlds smartest people really lack common sense at times. I think people can only go so far helping to protect others from themselves. In the case of programming I would say as long as you put very large warnings (disclaimers) / hide "dangerous" options away then at least you can say you've done your best to protect "end users" from themselves. So that if they screw something up they really can't have any come back - as you've taken time to warn/prevent self harm etc. That said it is shocking the number of people who just act upon emails saying "delete the file with a bear icon..." :) Still my point is you can only do some much to protect people from themselves. I have thus come to the conclusion that it isn't worth disadvantaging the majority for the sake of a few people who would jump off a cliff regardless of the number of signs that are put up.


> since UIDE was not part of the
> original DOS system and is not absolutely necessary, despite its speed or
> performance.

You write excellent software Jack as does Bret. Sure you've had a difference of approach here. But having studied both of your programs at length now and taken the time to see the comments and thoughts that have clearly gone into both I would urge you guys to put aside your differences and seek out middle ground in terms of improving flexibility - perhaps in a black box kind of way?


> but my drivers aren't. In their minds, that makes UIDE a whole lot more
> "suspect", and so I MUST be cautious.
Re suspect - *You* must be cautious or or users? To be honest I think you need to really put any onus with the end users rather than your good self.


> MSDOS.SYS and COMMAND.COM in fact ARE absolutely required,
With the exception of a few fairly "rare" embedded/custom DOS versions with COMMAND.COM specific hardware code I've never known it to be absolutely required. Indeed there are times when removing COMMAND.COM's from the picture can be very helpful in testing. e.g. Someone can load/install drivers via CONFIG.SYS and then load straight into "an app" instead (e.g. via SHELL= or by renaming it to COMMAND.COM if like me you often don't even bother with CONFIG.SYS either!)

So for people experiencing problems with COMMAND.COM shells (e.g. Freecom...) they can always rename their app to COMMAND.COM and try it. If they need BATch files (yuk) which don't need COMMAND.COM specifics (e.g. DIR) then they can compile them with BAT2EXEC, BAT2EXE, BAT2COM etc and then renaming the resulting program to COMMAND.COM thus allowing them to do many of the fun things DOS batch files allow people to do still without needing COMMAND.COM.

Note: From memory the documentation for those BATch compilers all talk about it not being a good idea to load TSR's via the above programs (for obvious reasons) but to be honest back when I still used these programs I never had a problem loading device/drivers in that way for what I was doing with them.
Not something I would recommend under normal circumstances (unless you know what you are doing) but for testing (or special configs) it can be useful.


> Also, I want UIDE to be "generic" and run across ALL variants of DOS,
> and that too requires caution.
Absolutely, I think DOS variant compatibility is a good thing to aim for.

> "Blessed are the peacemakers",
:)

> and you will get to Heaven before me,
There are several ways of reading this !!!!

>> Unneeded for UIDE ... Eric Auer's DEVLOAD that can load it at any time.
>
> Personally I tend to like avoiding external helper applications if I know
> I can do something without them. Indeed I agree with DOS386's comments
> re: device loaders being a "hack", a useful hack at that ... but if there
> is a better way of doing something ...

I do not view DEVLOAD or other such .SYS file loaders as any sort of "hack"
as they provide a service that DOS designers never imagined. Many FreeDOS
users would argue "heatedly" with you, since without DEVLOAD, they couldn't
put most of UIDE's logic (especially its faster /P binary-search tables) in
the HMA. FreeDOS does not make HMA available until after AUTOEXEC begins.
Also, if DEVLOAD runs so well, why should I and others need to add the same
logic in our .SYS drivers?? "Counter-productive", says I!

> Concerning .SYS device drivers I also forgot about the "next driver
> seg/ofs" field values having the annoying default values of FFFFh which is
> an invalid op-code combination (???? DI) thus preventing a .SYS file from
> being a .COM file. Duh!

The reason is that an FFFFFFFFh value denotes the LAST driver in the DOS
driver list. Re: changing how I document that area, you also noted that
device-drivers are not straight forward -- as one needs much knowledge to
write drivers, little reason for me to change my comments on such a basic
header field.

> So obviously if a .SYS device is a character device driver only using the
> init function then in my "flexibility" book it makes sense if that device
> driver is also available as an installable EXE driver ...

UIDE is not such a driver. It is "character" in its handling of Int 13h
BIOS I-O. But it is also "block", which explains its attribute bits, to
handle CD/DVD requests, which require a "block" driver.

>> logic, and its common data. As I noted in this thread to Rugxulo, are
>> people THAT "desperate" for memory that only 944 bytes really matters??
>
> :) Well I guess it depends on the circumstances ...

I agree, and since there are really few DOS drivers that load in the HMA
and so "leave behind" only 944 bytes of stack, data, and entry/exit code
in upper/DOS memory (that I spent YEARS achieving!) why don't we all "Be
Happy" with UIDE's design and forget "unloading". For 944 lousy bytes,
that is also "counter-productive".

> It still amazes me every day when people turn on their computers and they
> just "work" considering the significant number of people involved in
> producing their various parts over the years, the number of
> mistakes/flaws.

It amazes me even more that today's 4-GB PCs in fact do LESS work than the
16K mainframes (1966-1969) or 64K mini-computers (1969-1987) that I worked
with in the past. When Apollo 13's fuel tanks exploded in 1971, and they
had to use only the LANDER's engine to arc around the moon, their on-board
computer did all the calculations and "Got them HOME!" alive! GUESS just
how much memory their computer used -- "Would you believe" 64K?? Today's
PC computers are the most-successfully OVERSOLD BULLSH*T in man's history!

>> since UIDE was not part of the original DOS system and is not absolutely necessary, despite its speed or performance.
>
> You write excellent software Jack as does Bret. Sure you've had a
> difference of approach here. But having studied both of your programs at
> length now and taken the time to see the comments and thoughts that have
> clearly gone into both I would urge you guys to put aside your differences
> and seek out middle ground in terms of improving flexibility - perhaps in
> a black box kind of way?

I shall "defer" that issue to others who know and understand what I consider
a more "viable" and more SAFE approach to be. The ideas of UIDE supporting
"removable" HARD disks or "re-entrant" Int 13h calls, as it would have to do
for Bret Johnson's "workaround", are in my opinion NOT viable but RISKY, and
they would require a LOT more logic in UIDE for it to remain SAFE!

>> ... but my drivers aren't [absolutely necessary]. In their minds, that
>> makes UIDE a whole lot more "suspect", and so I MUST be cautious.
>
> Re suspect - *You* must be cautious or or users? To be honest I think you
> need to really put any onus with the end users rather than your good self.

No, in fact it must be ME who is cautious, since "unenlightened" users WILL
blame UIDE and me for other problems, and they will do so even BEFORE I get
a chance to explain or to defend myself.

>> MSDOS.SYS and COMMAND.COM in fact ARE absolutely required,
>
> With the exception of a few fairly "rare" embedded/custom DOS versions
> with COMMAND.COM specific hardware code I've never known it to be
> absolutely required ...

A "default" processor in-between executing DOS programs is necessary. If
not COMMAND.COM, some equivalent is still required. So for the "average"
users who do not play-games with alternates, I said COMMAND.COM instead of
being more general.

>> Also, I want UIDE to be "generic" and run across ALL variants of DOS,
>> and that too requires caution.
>
> Absolutely, I think DOS variant compatibility is a good thing to aim for.

Then it should be obvious why UIDE uses such a simple ".SYS only" and "Int
13h only" design. Anything more complex WILL get me into problems that I
do NOT need!

---
(Account disabled on user's request.)

> I do not view DEVLOAD or other such .SYS file loaders as any sort of
> "hack" as they provide a service that DOS designers never imagined.
True, I guess they can be viewed like other DOS enhancements we can all be grateful for like DOS extenders, CD-ROM drivers and even the IBM-PC itself :)

> Many FreeDOS users would argue "heatedly" with you,
Hopefully not! Still to be clear I was NOT not saying such hacks are a totally a bad thing, technical fields are moved forwards by people like the folks who frequent this forum working out how to constantly push things in all directions to the point that often as you know such "hacks" become "standard". With regards to DEVLOAD I guess you are right command-line .SYS device loading can be viewed as a standard option. And if it ain't broke....

> FreeDOS does not make HMA available until after AUTOEXEC begins.
Interesting I wasn't aware of that for several reasons: 1) For years I used other non-FreeDOS kernels. 2) I haven't bothered with HMA for years as most of what I have done with DOS hasn't needed me to even need to worry about it.

> Also, if DEVLOAD runs so well, why should I and others need to add the
> same logic in our .SYS drivers??
Ok, as above I see your viewpoint but was also being the devils advocate here. On a personal note in the past I used DDU/DDL which on the "rare" occasions I needed to load drivers from command-lines they did their job, thus I never sought out or indeed ever became aware of DEVLOAD. If I have/had it anywhere it was probably as part of a larger archive (like Simtel) which I mirrored. Indeed the first time I can remember hearing of DEVLOAD was on here.

> "Counter-productive", says I!
Ok.

> The reason is that an FFFFFFFFh value denotes the LAST driver in the DOS
> driver list.
Indeed like Zbikowski's "Z" on MCB's.

> Re: changing how I document that area,
On this point I was simply suggesting making that part of your source clearer to folks by better showing the 2 values as segment/offset "filled in by DOS".

> you also noted that device-drivers are not straight forward --
Yup, it's been many years since I last compiled one. Well actually that's not now entirely true as I recompiled from source all of Bret's code the other night and was impressed when I got zero complaints from a86/a386.

Anyway somewhere I have a SYS driver I wrote to force a headless PC to switch back to color VGA from mono VGA, and 1 or 2 several others. But nothing as complex as what you/Bret have written and I'm rusty after a 15 years absence.


> > driver is also available as an installable EXE driver ...
> UIDE is not such a driver.
Yup, hence I corrected myself in case anyone else reading what I wrote earlier (which no one else corrected me on) read it in a way where this all it sounded easier to do than it is. Thx for confirming what I was suspecting!

> > :) Well I guess it depends on the circumstances ...
> I agree, and since there are really few DOS drivers that load in the HMA
> and so "leave behind" only 944 bytes of stack, data, and entry/exit code
Well you could keep use the stack to clear bytes and push an exit jump, e.g:


org 100h
mov sp,0110
mov ax,20cd


Just kidding.... ) [and yes I know that 2 line example is very "bad"...]


> in upper/DOS memory (that I spent YEARS achieving!) why don't we all "Be
> Happy" with UIDE's design and forget "unloading".
I find the quickest way to unload is a reboot... (or overwriting memory.... :)

> For 944 lousy bytes, that is also "counter-productive".
:)))


> It amazes me even more that today's 4-GB PCs in fact do LESS work than
> 16K mainframes (1966-1969) or 64K mini-computers (1969-1987)
Probably true of the operators too.... far too many "cut and paste" folks !
Ah yes 1401 et al - still this is a DOS forum, so lets keep MF's to pvt msgs:)

> "Would you believe" 64K??
As we both know 64k is a lot of space.... :) An interesting point of note here is the fact that the first DOS machine only had 16k. I used to own a PC with 256k but sadly loaned it out to someone from whom I never got it back :(

> > a black box kind of way?
> I shall "defer" that issue to others who know and understand what I
> consider a more "viable" and more SAFE approach to be.
Ok

>> To be honest I think you need to really put any onus with the end
>> users rather than your good self.
> No, in fact it must be ME who is cautious, since "unenlightened" users
> WILL blame UIDE and me for other problems, and they will do so even
> BEFORE I get a chance to explain or to defend myself.
I also realize you've been burnt several times in the past but rest assured there are those of us happy to defend people and whilst also seeking the middle ground where appropriate. Sadly though I think it is fair to say "unenlightened" folks as you put it will always aim to burn others for the most silly of reasons. e.g. I got burned in the past by quite a number of folks for taking some PC screen captures with a digital camera. Fairly trivial point but some "unenlightened" folks didn't realize that due to what I was taking a picture of that I didn't/couldn't possibly have any screen capture software installed on the PC in question. Enlightened people understood straight away. Obviously had I had easy access to another PC with a VGA2PAL adapter and capture board I would have wired 2 PC's together to take the screen-shots but people forget that not everyone has the same resources.


> A "default" processor in-between executing DOS programs is necessary.
> If not COMMAND.COM, some equivalent is still required.
Well FreeDOS at least handles a missing shell in quite a neat way allowing you to run a program and then return back to the missing shell prompt before then allowing end users to run another program. Indeed with that in mind I am planning to deliberately allow a (re-written) RJDOS to be fully "exited" in a certain way. Just in case someone (e.g. Rayer) wanted to use feature.

> So for the "average" users who do not play-games with alternates,
> I said COMMAND.COM instead of being more general.
:)

> Then it should be obvious why UIDE uses such a simple ".SYS only"
And indeed it is. Thanks for directly clarifying re the block aspect as well.
We've also avoided discussing the joys of cooked vs raw.... as well :)

> Anything more complex WILL get me into problems that I do NOT need!
Indeed I understand... especially with those damn neighbors of yours... grrr!

Thanks for an interesting discussion which has made me think more about how some of my long term work can hopefully help narrow down some DOS issues.

snippety-snip... Just an detail which you may or may not remember

>> The reason is that an FFFFFFFFh value denotes the LAST driver in the DOS
>> driver list.
> Indeed like Zbikowski's "Z" on MCB's.

Actually, when loading a driver, MS-DOS (others ?) check only the segment part (high word) of this double word for the special value FFFFh. This behaviour (by design?) will allow us to make a dual device driver/ dot com executable, using the first two bytes as a short jump over the rest of the driver header. Of course, the init routine for the 'sys' specific part of the init code should replace FFFF at offset zero before giving back control to the DOS system initialisation routines...

Cheers

--
Ninho

> snippety-snip... Just an detail which you may or may not remember
LOL :) I'll keep this reply short....

> Actually, when loading a driver, MS-DOS (others ?) check only the
> segment part (high word) of this double word for the special value
> FFFFh.
Interesting. So have you produced or seen a combined SYS/COM previously?


> driver/ dot com executable, using the first two bytes as a short jump over
> the rest of the driver header.
Ok, I fully understand. e.g. using EB1Eh, FFFFh instead of FFFFh, FFFFh.

Unfortunately one small point I have noted already is if someone also wants to use UPX that UPX (at least v3.05w) flags up the following error (as you would expect) if the header on a SYS is not set to FFFFh FFFFh:
upx: xmgr3.sys: UnknownExecutableFormatException

So obviously if UPX packing is required (unless they do us an update?) an additional wrapper is also needed for UPX SYS drivers modified in this way.


> Of course, the init routine for the 'sys' specific part of the init
> code should replace FFFF at offset zero before giving back control
> to the DOS system initialisation routines...
Yup. As the code will be short I might even give this a try this evening.

>> Actually, when loading a driver, MS-DOS (others ?) check only the
>> segment part (high word) of this double word for the special
>> value FFFFh.

> Interesting. So have you produced or seen a combined SYS/COM previously?

I've "produced" a lot of the kind, they come out handy. Others may have found and used the same trick independently, though I have not seen any.

> Ok, I fully understand. e.g. using EB1Eh, FFFFh instead of FFFFh, FFFFh.

Correct. And re-patch FFFF in place of EB xx before returning to DOS.

> Unfortunately one small point I have noted already is if someone also
> wants to use UPX that UPX (at least v3.05w) flags up the following error
> (as you would expect) if the header on a SYS is not set to FFFFh FFFFh:
> upx: xmgr3.sys: UnknownExecutableFormatException

Good to know. I don't pack my drivers - they've been generally small enough by themselves :=)

> Yup. As the code will be short I might even give this a try this evening.

Y.W.! Watch out the necessary segment adjustments between .sys and .com formats, to compensate for the DOS PSP in the latter.

---
Ninho

> I've "produced" a lot of the kind, they come out handy. Others may have
> found and used the same trick independently, though I have not seen any.
Ok, well I couldn't wait until this evening and have already started one :)

> Good to know. I don't pack my drivers - they've been generally small
> enough by themselves :=)
heh

> Y.W.! Watch out the necessary segment adjustments between .sys and .com
> formats, to compensate for the DOS PSP in the latter.

Indeed. Anyway for the curious here is my current work in progress as it stands. Unfortunately I need to do some other things (housework!) but hope to finish something later on this evening (and test on a FreeDOS box).



; ***************************************************************************
;                     SYSCOM v0.01 (a work in progress version)
; ***************************************************************************

Org     0000h               ; Device drivers do NOT have a PSP, so no org 100h
assume cs:code, ds:code     ; ...remember that COM files do have a PSP though!


; ***************************************************************************
SysHead:                    ; SYS Device Driver header (18 bytes in total)
  jmp   short COMEntry      ; Offset address of next driver (patched below)
  dw    0FFFFh              ; Segment address of next driver
  dw    08000h              ; Device attributes (1000000000000000b=basic char)
  dw    (Strategy-SysHead)  ; Offset address of strategy routine
  dw    (Interrupt-SysHead) ; Offset address of Interrupt routine
  db    'SYSCOMv1'          ; Device driver name (8 bytes)
; ***************************************************************************

ict_tab dw offset Initialize  ; Function 0
        dw offset dummy       ; Function 1
        dw offset dummy       ; Function 2
        dw offset dummy       ; Function 3
        dw offset dummy       ; Function 4
        dw offset dummy       ; Function 5
        dw offset dummy       ; Function 6
        dw offset dummy       ; Function 7
        dw offset dummy       ; Function 8
        dw offset dummy       ; Function 9
        dw offset dummy       ; Function 10
        dw offset dummy       ; Function 11
        dw offset dummy       ; Function 12
        dw offset dummy       ; Function 13
        dw offset dummy       ; Function 14
        dw offset dummy       ; Function 15
        dw offset dummy       ; Function 16

ptr2db  dw (?), (?)           ; Pointer to Data Block


; ***************************************************************************
COMEntry:
  mov word [SysHead+100h], 0ffffh ; Set Offset address of next driver to FFFFh
  jmp dummy


Strategy:
  mov   cs:ptr2db,   bx
  mov   cs:ptr2db+2, es
  ret


Interrupt:
  ret  ; apparently ret not an iret


Initialize:
  mov   word ptr es:di, offset Initialize
  mov   es:di, cs
  xor   ax, ax
  ret


dummy:
;  mov   ax, 8003h    ; Call not supported
  xor   ax, ax        ; Pretend we've done something!


exit:
  ret                 ; Returns to first 2 bytes of PSP under DOS (Int 20h!)

; ***************************************************************************
;                                END OF SYSCOM.ASM
; ***************************************************************************



Compiled binary as it stands:

00000000:  EB 36 FF FF-00 80 41 00-4C 00 53 59-53 43 4F 4D
00000010:  76 31 4D 00-58 00 58 00-58 00 58 00-58 00 58 00
00000020:  58 00 58 00-58 00 58 00-58 00 58 00-58 00 58 00
00000030:  58 00 58 00-00 00 00 00-C7 06 00 01-FF FF E9 17
00000040:  00 2E 89 1E-34 00 2E 8C-06 36 00 C3-C3 26 C7 05
00000050:  4D 00 26 8C-CF 33 C0 C3-33 C0 C3


[EDIT: Just to add: Yes, I know several optimizations and SYS side "broken"]

....
> I am "rather-well aware" of what happens if one jumps up to the HMA without
> issuing those calls correctly ...

Of course you are. Well, I'm dismayed to hear your code hit buggy behaviour from MSDOS's HMA logic, which I believed to be quite solid. Maybe you'll want to document the cases for the sake of other interested developpers ? I can see you've given some detail in this thread, is it enough to reproduce the bug?

(Not myself going to dive into it soon, the wife's health condition restricting both the time I can devote to non essential activities as well as the interest I would otherwise take in them.)

---
Ninho

Interesting, what driver you going to programm? And what will be the function of COM part? I never tried to write a dos driver but it seems not sooo complicated :)

---
DOS gives me freedom to unlimited HW access.

So, did you find if FreeDOS does mimic MS-DOS's undocumented behaviour i.e. will it accept to load our tricky COM files as drivers ?

FWIW, here's one short source complete from my archives, hopefully demonstrating how I do it (assembles w/ TASM in MASM mode). Apologies : I didn't review, trim or edit the specifics - for lack of time. HTH anyway !






; =============
; == S.R. 09 ==
; =============
;
; ==== Objet :  Rendre accessible la RAM @ E0000-EFFFF sur le systeme AsRock 741
;               et l'initialiser pour UMBPCI (=FFFFFF...)
; ==== Forme :  Double! A la fois Pseudo-Driver-DOS, ET programme .COM

        .586P
        .model  small
        JUMPS

        ASSUME CS:CODE, DS:nothing
code    SEGMENT USE16
        ORG     0

; DEBUG = 1 ; supprimer en regime de croisiere !

;
; =======  Equates :
;
CR      EQU     0Dh
LF      EQU     0Ah
BELL    EQU     07h
EoL     EQU     CR,LF,'$'  ; fin de message ( pour DOS 21/09)

ERRG    EQU     8003h      ; erreur generale (driver)

; =======  Deplacements dans le bloc de parametres de driver:
NumFonc     EQU  02h     ; byte
CodeRet     EQU  03h     ; word ( sortie )
Offs_Fin    EQU  0Eh     ; ptr
ParmAddress EQU  12h     ; ptr
InitErrFlag EQU  17h     ; byte ( sortie ; pour faire aff msg d'err par DOS )

; =======       En-tete (pilote de caracteres, bidon). Doit etre a l'Offs zero
        ORG     0          ; initialement en 0100h si .COM ...

HEADER  LABEL NEAR         ;  En-tete Pilote, ET debut .COM !
        jmp    short  COM_Start  ; saut (relatif) court  necessaire .
        DW      -1
        DW      8000h       ; attributs du "peripherique"
        DW      Offset Strat
        DW      Offset IntRou
        DB      "741UMEM"   ; nom de pilote, bidon, Pour diagnostic !
COM_Flag DB     "$"  ; fin du nom de pilote, OU 00 si appel-COM !

; ========      Debut-COM : Ajuster le CS pour commencer :      ====

COM_Start:      CLD   ; au depart CS=DS=PSP, IP=100h
                MOV     AX, CS
                ADD     AX, 10h    ; compense la taille du PSP
                MOV     Word Ptr[DS:req_header_ptr] +2 +100h, AX
                MOV     Word Ptr [DS:req_header_ptr] +100h, offset COM_Adjusted
                JMP     DWord Ptr [DS:req_header_ptr]+100h ;-> COM_Adjusted

; ========      Donnees: constantes / variables.     ===============

req_header_ptr  DW      2 DUP (?) ; ptr -> bloc de parametres (req header)


; ========      Donnees specifiques a CE pilote

SysConfig = 0C0010010h
MTRR_FIX4K_E0000 = 026Ch
; MTRR_FIX4K_E8000 = 026Dh
PCI_Addr_Port = 0CF8h
SiS741_shadow_RAM = 70h
Enable_RAM_E0000 = 0F008F00h ; shadow enabled, +RW at E0000
SiS741 = 07411039h ; PCI device and vendor ID

; ======== Messages Ascii '$' - pour DOS int 21/09 : EoL=CR LF $

MSG0      DB    'Repair SiS741 upper memory, by S.R. (c) 2009' ,EoL

M_MBREAL  DB    BELL, 'Sorry! Processor must be in REAL mode for 741UMEM to do its magic. Aborting!' ,CR,LF
 DB 'Please try launching it from CONFIG.SYS, BEFORE any memory manager', EOL

M_BADCHIP DB    BELL, 'Error! Wrong chipset or processor.' ,EoL
M_DidntWork DB  BELL, 'Error! Something went wrong, aborting.' ,EoL
M_Fin_OK  DB   'SUCCESS! RAM at segment E0000 available & initialised.',EoL
                         
; ======== Code. =========================================================
                                     
;    'Strategy routine' = acquisition du <request header>
STRAT   PROC FAR
        MOV CS:[req_header_ptr]  , BX
        MOV CS:[req_header_ptr+2], ES
        RET
STRAT   ENDP
;=========================================================================

;   'Interruption routine' pour le pseudo-pilote:
INTROU  PROC FAR
        CLD      ; ----- vers le HAUT, une fois pour toutes.
        PUSHA
           ASSUME  DS: Code
        PUSH    CS
        POP     DS
        LES     DI, DWORD PTR [req_header_ptr]
        MOV     BL, ES: [DI+ NumFonc]
        OR      BL , BL
        JZ      near ptr INIT  ; le code fonction doit etre = zero
        MOV     AX, ERRG       ; err generale signalee a MSDOS, et retour
        JMP     FFIN2

;       Suite adaptation pour la version .COM
;       On arrive ici avec le CS voulu, le meme que le driver:

COM_Adjusted:                                             
;       DS  pointe encore vers le PSP !!
        MOV     COM_Flag + 100h, 0  ; marquer version .COM
        JMP     short INIT1
; =========================================================================

INIT:  ; driver,  fonction unique : atteinte via un JMP
        MOV     AH, 09
        MOV     DX, Offset MSG0 ; message d'accueil
        INT     21h    ; ne touche qu' AX

        LES     DI, DWORD PTR [req_header_ptr]
        lds     si, es:[di + ParmAddress] ; pointe sur ligne device=...
; =========================================================================

INIT1:              ; commune a COM et SYS
        ASSUME  DS: Nothing ; (DS:SI) -> ligne d'appel

        PUSH    CS
        POP     DS
        ASSUME  DS: code
                                   
                                                                 
;            ==============  TESTS  =================       


         IFDEF   Debug
           int   3
         ENDIF


; 1st : check for REAL MODE !

                MOV     DX, offset M_MBREAL
                SMSW    AX
                TEST    AX, 1
                JNZ             MSG_et_FIN  ; sinon, message d'erreur et retour :

; OK, now check proc and chipset

; call CPUID fun 0 :
        MOV             EAX, 0
        CPUID
        CMP             EBX, "htuA"
        JNE             bad
        CMP             EDX, "itne"
        JNE             bad
        CMP             ECX, "DMAc"
        JNE             bad

; call CPUID extended func 80000001 :
        MOV             EAX, 80000001h
        CPUID
        CMP             AH, 07  ;  AMD family number
        JB              bad
        AND             EDX, (1 SHL 12) ;Memory type registers
        JNZ             testNB  ; OK, next check North bridge
bad:
        MOV             DX, offset M_BADCHIP ; wrong processor or north bridge
; fall through -> MSG_ET_FIN

; =========================================================================
MSG_et_FIN  Label NEAR
        PUSH    CS
        POP     DS  ; c'est necessaire. Message adresse' par [DS:DX]
        MOV     AH, 9
        INT     21h

        TEST    [COM_Flag], 0FFh
        JNZ     FFIN

;    retour a l'appelant , version 'COM' (programme) !  - - - - - -
FFIN_COM  LABEL  NEAR
        MOV     AX, 4C00h  ; pas d'erreur a renvoyer.
        INT     21h        ; et c'est tout !

;    retour a MSDOS pour la  version 'SYS' (driver) ====
FFIN    LABEL NEAR
        XOR     AX,AX  ; signale initialisation reussie!
        PUSH    CS
        POP     DS
        LES     DI, DWORD PTR [req_header_ptr]

FFIN2:  OR      AX, 0100h ; fixer le bit <termine'>
        MOV     ES: [DI+ CodeRet],   AX
        MOV     WORD PTR ES: [DI+ Offs_Fin], 0  ; p/ rendre tte la memoire et
        MOV     ES: [DI+ Offs_Fin+2], CS        ; ne PAS s'installer resident
        POPA
        RET     ; FAR! vers MSDOS (config.sys).
; =========================================================================
; OK assume this Athlon (or better) can do the memory trick
; access PCI config space, northbridge (bus 0 dev 0 fun 0)
testNB:
        mov     DX, PCI_Addr_Port ; CF8
        mov     EAX, 0 + (1 SHL 31)
        CLI
        out     DX, EAX
        JNC     loc3
loc3:   MOV     DL, 0FCh ; set DX = PCI_Data_Port
        IN      EAX, DX ; get manufacurer & model
        JC      loc4 ; delay, just in case
loc4:   STI
                MOV     DX, offset M_BADCHIP
        CMP EAX, SiS741
        JNE           MSG_et_FIN

; All correct !

;            ==============  ACTION  ================   

; Program K7 MSRs & SiS741 shadow :
; the meat of this program !

; 1 : K7:  open access to extended MTRR attributes :

        WBINVD  ; before messing with mem mappings

        mov     ECX, SysConfig  ; MSR C001_0010
        RDMSR
        or      EAX, 1 SHL 19
        WRMSR

; 2 : K7: appliquer tout le segment "E" (64 k) sur le bus PCI

        mov     ECX, MTRR_FIX4K_E0000
        mov     EAX, 06060606h  ; IO, write-back !
        mov     EDX, EAX
        WRMSR
        INC     CX ; MTRR_FIX4K_E8000
        WRMSR

;3 : K7: close access to extended MTRR attributes :

        mov     ECX, SysConfig  ; MSR C001_0010
        RDMSR
        and     EAX, NOT (1 SHL 19)
        WRMSR

; 4 : SiS 741 : enable shadow RAM, segment E (R/W)

; access PCI config space, bus 0 dev 0 fun 0

        mov     DX, PCI_Addr_Port ; CF8
        mov     EAX, SiS741_shadow_RAM + (1 SHL 31)
        CLI
        out     DX, EAX
        JNC     loc1
loc1:   MOV     DL, 0FCh ; set DX = PCI_Data_Port
        IN      EAX, DX ; read shadow RAM attributes
        JC      loc2 ; delay, just in case
loc2:
        OR      EAX, Enable_RAM_E0000
        out     DX, EAX ; update SiS shadow
        STI

; Check if it worked : can we access the memory at E0000 ?

        MOV     AX, 0E000h
        MOV     DS, AX
        MOV     ES, AX
        ASSUME DS: nothing, ES: nothing

        MOV      AX, [DS:0]
        MOV      BX, 0-1
        XOR      BX, AX
         CLI     ; in case foreign BIOS would use the region
        MOV      [DS:0], BX
        CMP      BX, [DS:0]
        MOV      [DS:0], AX ; reset it
         STI
        MOV      DX, offset M_DidntWork
        JNE      done
; OK! Initialise segment E - fill it with FF's
        XOR     AX, AX
        MOV     DI, AX
        DEC     AX      ; set to FFFFh
        MOV     CX, 8000h
        REP     STOSW
        MOV     DX, offset M_Fin_OK
done:
;  FIN ! affiche un message... et retour a MSDOS
                JMP     MSG_et_FIN

INTROU   ENDP

; ==========================================================================

; end_of_code     =       $

code    ENDS
        END     HEADER   ; dummy label so as to please TASM


---
Ninho

> Interesting, what driver you going to program?
For now a test "template" character driver which will do little more than display a messages, but one that should be straight forward for others to reuse. I may have another interesting related driver template (something else I am testing) which if I can get it to work I will release at the same time which is why I have also now coded an update to an old program of mine called RJDUMP (basically my equiv of tdump). My update has been to add the dumping of SYS driver headers including combined COM/SYS drivers:

* Note: version shown is a work in progress (ver # my change etc) *

q:\>rjdump int20.exe
RJDUMP v2.00a - (c)Copyright 1996-2010 Richard L. James

Filename                                              'int20.exe'

DOS File Size                                         0022h  ( 34. )

EXE Signature                                         5A4Dh  ( 'MZ' )
Load Image Size                                       0002h  ( 2. )
Relocation Table entry count                          0000h  ( 0. )
Relocation Table address                              001Ch  ( 28. )
Size of header record      (in paragraphs)            0002h  ( 2. )
Minimum Memory Requirement (in paragraphs)            0000h  ( 0. )
Maximum Memory Requirement (in paragraphs)            FFFFh  ( 65535. )
File load checksum                                    0000h  ( 0. )
Overlay Number                                        0000h  ( 0. )

Initial Stack Segment  (SS:SP)                    FFF0:0000
Program Entry Point    (CS:IP)                    FFF0:0100


q:\>rjdump uide.sys
RJDUMP v2.00a - (c)Copyright 1996-2010 Richard L. James

Filename                                              'uide.sys'

DOS File Size                                         2404h  ( 9220. )

DOS SYS Header
OfsAddrOfNextDriver                                   FFFFh  ( 65535. )
SegAddrOfNextDriver                                   FFFFh  ( 65535. )
DeviceAttributes                                      C800h  ( 51200. )
OfsAddrOfStrategy                                     C800h  ( 51200. )
OfsAddrOfInterrupt                                    1640h  ( 5696. )
DeviceName                                            'UDVD1   '

q:\>rjdump syscom.sys
RJDUMP v2.00a - (c)Copyright 1996-2010 Richard L. James

Filename                                              'syscom.sys'

DOS File Size                                         005Bh  ( 91. )

DOS SYS Header
OfsAddrOfNextDriver                                   36EBh  ( 14059. )
SegAddrOfNextDriver                                   FFFFh  ( 65535. )
DeviceAttributes                                      8000h  ( 32768. )
OfsAddrOfStrategy                                     0041h  ( 65. )
OfsAddrOfInterrupt                                    004Ch  ( 76. )
DeviceName                                            'SYSCOMv1'


I will release this version of RJDUMP as freeware with the templates (since releasing it will aid others in this task) however I have removed the Windows EXE dumping code for now (mainly because most of that code was NE related).


> And what will be the function of COM part?
Basically as above as I want to return to what I was working on, which was other programming utilities/libraries which I am using to rewrite RJDOS.

> I never tried to write a dos driver but it seems not sooo complicated :)
Indeed, basic drivers are not. However what Bret/Jack are doing is a lot more complex. Hence I thought I would focus my time releasing a driver dump utility (flags dumping is missing in the above at the moment) and some templates demoing Ninho's idea which will help test/get others involved with drivers. The reason I'm drawing the line there is for now my interests (and also time) are elsewhere but as I happened to be working on similar things I thought I would take a small slight step to one side of them to test this out.

> So, did you find if FreeDOS does mimic MS-DOS's undocumented behaviour i.e.
> will it accept to load our tricky COM files as drivers ?
Not yet as I haven't had a chance to test on my FreeDOS test PC which has recently moved location. I have tried a few quick tests with loadsys etc.

> FWIW, here's one short source complete from my archives, hopefully
> demonstrating how I do it (assembles w/ TASM in MASM mode).
Thanks. I will review in addition to the UIDE/PC Intern examples.

> Apologies : I didn't review, trim or edit the specifics
No worries, I did note you are having to look after a family member. I wish you both well. Coding/life is a careful balance. Indeed I have now increased up my output by moving a PC to somewhere where I can juggle life/coding better.

> if UPX packing is required (unless they do us an update?) an
> additional wrapper is also needed for UPX SYS drivers modified in this
> way.

Concerning a wrapper for UPX'd SYSCOM's - there are 2 further complications: 1) The UPX license doesn't allow the UPX decompressor to be modified. 2) UPX CRC's data - however it does NOT CRC all of the SYS header data and appears to not care about extra bytes being added (I need to do some further testing).

> Concerning a wrapper for UPX'd SYSCOM's - there are 2 further
> complications: 1) The UPX license doesn't allow the UPX decompressor to
> be modified. 2) UPX CRC's data - however it does NOT CRC all of the SYS
> header data and appears to not care about extra bytes being added (I need
> to do some further testing).

"You can use a modified UPX version or modified UPX stub only for
programs that are compatible with the GNU General Public License."

So it's not that dire, they just want you to presumably be able to actually unpack it to its original form (esp. for using newer UPX with better compression).

Unfortunately, since Jack has decided to air our "dirty laundry" publicly on this Forum, I feel forced to make an appropriate response. I normally do not use this Forum, and came across this discussion quite by accident.

Jack and I were approached by Laaca, requesting that we attempt to make UIDE & USBDRIVE compatible with one another. Jack and I exchanged several e-mails, where I suggested things such as making UIDE a TSR (thereby making it removable and installable after USBDRIVE without the aid of external programs such as DEVLOAD), adding support for AMIS (Alternative Multiplext Interrupt Specification,) or at least supporting IISP (IBM Interrupt Sharing Protocol) to allow "automated" support for external programs such as USBDRIVE that are installed after UIDE. These suggestions were summarily rejected by Jack. Jack did offer up some suggestions to "automate" the process as well, none of which would completely alleviate the compatibility problems.

After this, I told Jack that I would like to support UIDE, even if the process is not "automatic", by supporting the external protocol option (/E) of UIDE in the next version of USBDRIVE. I started to investigate the external option, asking Jack some specific questions about exactly how it worked (the formal documentation regarding the interface is, to put it politely, sparse). One of the input parameters required when calling the external interface is a "Cache Unit Number". I asked Jack how the Cache Unit Numbers were determined and managed. This is his response back to me, and my response back to him (verbatim):

******************************

> I do not, and the subject of who gets what cache unit-numbers is one
> I left to users, so they can determine their own "schemes". UIDE
> only has a "hold" on cache units up to 55 (37h), which are for its
> own BIOS units and up to 8 CD/DVD drives. Other cache units are
> "up to you" to specify and manage.

This is a very dangerous road your going down -- this almost guarantees data corruption and crashes. The Cache Unit Number is essentially a Handle, and it is NEVER a good idea to make a program generate its own Handles. If only one external program ever used UIDE, you could get away with something like that. With the possibility of more than one (e.g., separate USBDRIVE and USBCDROM programs), that is, frankly, ludicrous. It would be possible, I suppose, to develop some sort of external protocol to manage the Handle numbers, but that is something UIDE should manage all by itself. At a minimum, you need some form of "Get Handle" and "Release Handle" calls.

******************************

Jack's (paraphrased) response to this was that he was personally insulted, and never wanted to hear from me again.

I am still willing to make the next version of USBDRIVE compatible with the external option of UIDE, but will only do this if Jack fixes the external interface so that it manages and provides its internally-used Cache Unit (Handle) numbers to external programs. Jack's proposed solution of requiring programmers who want to use the external interface to (paraphrasing) "negotiate handle numbers amongst themselves via e-mail" is simply unacceptable.

I did send a final follow up e-mail to Jack, in which I apologized and suggested that he either fix the external interface or eliminate it altogether. I suspect that Jack may have deleted that e-mail without ever reading it. After a few days with no further response from Jack, I posted a follow-up message in my Forum to Laaca summarizing what had transpired between myself and Jack.

I stand by my assertion that the UIDE external interface in its current state is unusable, and either needs to be fixed or eliminated. If it is fixed, I intend to support it in the next version of USBDRIVE. I also stand by my assertion that I did not intend the e-mail as a personal insult, and am very sorry that Jack feels that was what I was trying to do.

P.S. Now that Jack has made changes to UIDE to guarantee that it will NEVER automatically work with removable hard drives as required by USBDRIVE (or other removable interfaces like Firewire, Bluetooth, PCMCIA, network file servers, ?), the only way to make UIDE and USBDRIVE compatible is with UIDE's (currently broken) external interface. The option of loading USBDRIVE first and then installing UIDE with DEVLOAD (or some equivalent) afterwards is no longer viable.

> ... Jack and I exchanged several e-mails, where I suggested things such
> as making UIDE a TSR (thereby making it removable and installable after
> USBDRIVE without the aid of external programs such as DEVLOAD), adding
> support for AMIS (Alternative Multiplext Interrupt Specification,) or at
> least supporting IISP (IBM Interrupt Sharing Protocol) to allow
> "automated" support for external programs such as USBDRIVE that are
> installed after UIDE. These suggestions were summarily rejected ...

Because I felt they were not necessary, and I in fact SAID SO! That is
NOT a "summary rejection", that is me using my BRAIN to avoid unrequired
GARBAGE, which has never-before been needed in UIDE or in other drivers!
UIDE's external-interface logic can handle calls by USBDRIVE without all
the above suggested changes.

> ... I started to investigate the external option, asking Jack some
> specific questions about exactly how it worked (the formal documentation
> regarding the interface is, to put it politely, sparse).

Because no one, before now, had ever voiced any interest in having UIDE do
caching for another driver's I-O. Few people still do any device-drivers
for DOS, thus "formal" documentation of UIDE's external-interface routines
simply did not seem necessary.

> ... I asked Jack how the Cache Unit Numbers were determined and managed.
> This is his response back to me, and my response back to him (verbatim):
>
> ******************************
>
>> ... The subject of who gets what cache unit-numbers is one I left to
>> users, so they can determine their own "schemes". UIDE only has a
>> "hold" on cache units up to 55 (37h), which are for its own BIOS
>> units and up to 8 CD/DVD drives. Other cache units are "up to you"
>> to specify and manage.
>
> This is a very dangerous road your going down -- this almost guarantees
> data corruption and crashes. The Cache Unit Number is essentially a
> Handle, and it is NEVER a good idea to make a program generate its own
> Handles. If only one external program ever used UIDE, you could get away
> with something like that. With the possibility of more than one (e.g.,
> separate USBDRIVE and USBCDROM programs), that is, frankly, ludicrous. It
> would be possible, I suppose, to develop some sort of external protocol to
> manage the Handle numbers, but that is something UIDE should manage all by
> itself. At a minimum, you need some form of "Get Handle" and "Release
> Handle" calls.
>
> ******************************
>
> Jack's (paraphrased) response to this was that he was personally insulted,
> and never wanted to hear from me again.

In fact, what I actually replied is as follows --

> UIDE provides only the needed cache handling and entry-points for
> such handling. With so little interest in its capabilities till
> now, there was NO reason for me to define more, certainly NOT any
> need for me to manage nor "dictate" cache unit-number values, and
> so I had reasons -- I SAY AGAIN, REASONS!! -- for staying silent.
>
> You are free to write a "top-level arbiter" for UIDE cache-units.
>
> Also, I was taught the word "ludicrous" denotes RIDICULOUS to the
> point of being LAUGHABLE.
>
> You and I will now PART COMPANY. Send me NO MORE such insulting
> damned E-Mails, and expect no more comment or assistance from me!

Do any of the rest of YOU enjoy being labelled LUDICROUS??!! Since
I did not, that in my mind was the END of our E-Mails!!

> I am still willing to make the next version of USBDRIVE compatible with
> the external option of UIDE, but will only do this if Jack fixes the
> external interface so that it manages and provides its internally-used
> Cache Unit (Handle) numbers to external programs. Jack's proposed
> solution of requiring programmers who want to use the external interface
> to (paraphrasing) "negotiate handle numbers amongst themselves via e-mail"
> is simply unacceptable.

As noted above: You are free to write a "top-level arbiter" for UIDE
cache-units. Personally, I still feel that only a few "Friendly" E-Mails
between those driver-writers who must manage cache-unit numbers (currently
only 2 people I know of!!) is acceptable.

> I did send a final follow up e-mail to Jack, in which I apologized and
> suggested that he either fix the external interface or eliminate it
> altogether. I suspect that Jack may have deleted that e-mail without ever
> reading it.

I read it and did not feel it was worth any response.

> I stand by my assertion that the UIDE external interface in its current
> state is unusable, and either needs to be fixed or eliminated. If it is
> fixed, I intend to support it in the next version of USBDRIVE. I also
> stand by my assertion that I did not intend the e-mail as a personal
> insult, and am very sorry that Jack feels that was what I was trying to
> do.

Fine words, AFTER what went before. My Father once taught me there are
not 10 Commandments but 11, the last being "Thou shalt not get CAUGHT!!"

Also note that unless the /E switch is specified when loading UIDE, its
external-entry logic IS eliminated (the pointer to it at driver address
00016h will be zeroed!), and for most users who put most of UIDE in HMA
space, the 48 bytes of related logic are in fact DISCARDED! Given all
that, I also do NOT think it necessary to delete that source code, too!

> P.S. Now that Jack has made changes to UIDE to guarantee that it will
> NEVER automatically work with removable hard drives as required by USBDRIVE
> (or other removable interfaces like Firewire, Bluetooth, PCMCIA, network
> file servers, …), the only way to make UIDE and USBDRIVE compatible is with
> UIDE's (currently broken) external interface. The option of loading
> USBDRIVE first and then installing UIDE with DEVLOAD (or some equivalent)
> afterwards is no longer viable.

The other such drives are ALSO dangerous to use with only HARD DISK logic.
And do NOTE that I said "with only HARD DISK logic", as would get used for
BIOS units numbered 080h and up. That class of logic, and the DOS system
itself, may or may-NOT process a "media change" code for such drives. If
not, as I suspect is true for most DOS systems (if not ALL of them!), data
corruption is GUARANTEED to occur!

I have NO REGRETS re: upgrading UIDE to avoid such cases and so stay SAFE!
I will NOT have UIDE users suffer as I did, in trying to handle JAZ or ORB
"removable" HARD-disk cartridges with DOS! Windows/Unix may handle them,
but DOS is an "older" system that DID NOT handle them properly for me, and
as Gates & Co. have now "walked away" from DOS, it probably STILL does not
handle such units any better with only HARD-disk logic! Those at FreeDOS
can call me "cautious" -- I call it STAYING SAFE!!

If such drives call UIDE using its external-interface logic, and so do NOT
use HARD-disk logic, responsibility for noting a media-change is then ONLY
with the device-driver that issues such calls. It, and not the "regular"
HARD-disk logic in DOS or in UIDE, can deal with a media-change however it
wishes. UIDE thus becomes only an I-O subroutine for the calling driver,
need NOT deal with HARD-disk media-changes, and the calling driver is then
responsible for telling UIDE if-and-when a cache flush is necessary.

That is called "Keeping it SIMPLE!", as I intended for UIDE!

Despite any of the above, I DO NOT consider UIDE's external-entry logic to
be "broken" in any way. It works, every time I test it using UIDE's "old
style" CD/DVD logic that in fact DID call it (UIDE now has faster internal
schemes). If others consider it "broken", they are free NOT to use it at
all, with my absolute BLESSING at this point!!!

---
(Account disabled on user's request.)

> Unfortunately, since Jack has decided to air our "dirty laundry" publicly
> on this Forum, I feel forced to make an appropriate response. I normally
> do not use this Forum, and came across this discussion quite by accident.

Probably just to public inform Laaca (and others) why he won't change UIDE, unlikely as an affront to you. But, to be honest, it's not that big a deal, and I haven't heard any huge outcries, so it's really not (and shouldn't) become that heated over such a silly technical thing. So no worries, everybody can get back to work/play/the usual.

I agree. The only reason I felt it was necessary to respond at all was Jack's insistence that I insulted him personally, which I did not do. I called his external interface ludicrous, dangerous, and unusable (which I still believe is true, and why I will not use it in its current state), but I did not call him personally ludicrous.

> I agree. The only reason I felt it was necessary to respond at all was
> Jack's insistence that I insulted him personally, which I did not do. I
> called his external interface ludacris, dangerous, and unusable (which I
> still believe is true, and why I will not use it in its current state), but
> I did not call him personally ludacris.

Regarding the Device Driver vs. TSR "controversy", I would like to throw in my two cents.

TSR's are actually "easier" to write. While CONFIG.SYS is being processed, many of the DOS services that you generally take for granted (even very basic things like writing text to the screen, manipulating files, and redirection/piping) are limited or non-existent. With TSR's, all DOS services are available for use.

I think it is safe to say that anything that can be done with a device driver in CONFIG.SYS can also be done with a TSR (either in AUTOEXEC.BAT or from the command-line), but the reverse most certainly is NOT true. The only drivers that actually need to be in CONFIG.SYS are things that the BIOS or DOS kernel may require early in the boot process (e.g., a memory manager needs to be in CONFIG.SYS or DOS may not be able to use/manage UMB's or HMA).

Even you never take advantage of the opportunity, I think you should always have the option of removing drivers from memory ad-hoc if you want to without needing to reboot. It is true that rebooting isn't nearly as big of a deal in DOS as it is in Windows or Linux, but should still be avoided whenever possible. Also, just because you believe there should never be a reason to uninstall any particular driver doesn't mean the end-user will think the same thing, no matter how "illogical" you think their reasoning may be. FORCING the user to reboot to change something when it isn't really necessary is bad policy, IMHO.

Although they are a very interesting concept, there really is no reason to build "dual" EXE/device-drivers (or COM/device-drivers), except for those that are needed by DOS while booting as discussed above. From a practical perspective, device drivers provide no advantage whatsoever over TSR's, either to the programmer or the end-user. In fact, device drivers have many disadvantages.

If a DOS driver is so old that it is not being updated or maintained any more, than there is not a lot you can do about that. But, all modern DOS programs should IMHO use modern protocols and installation techniques such as AMIS (which inherently includes IISP), which provide a structured environment for TSR's and device drivers to communicate and cooperate with each other regarding items such as interrupt handlers, device headers, and hot-keys.

I must admit that even I was slow to accept AMIS, I think because I was confusing it with the older TesSeRact which I knew from experience was fraught with problems. After Christian Masloch mentioned AMIS to me, I investigated it a little and found that AMIS has addressed most (though not quite all) of the issues that were present in TesSeRact. I personally think all modern TSR's and device drivers should use it, and will be adding it to all of my programs as they are updated. In many cases, AMIS also makes the order of installation (and removal) far less critical to the end-user, which is a good thing.

> ... While CONFIG.SYS is being processed, many of the DOS services that
> you generally take for granted (even very basic things like writing
> text to the screen, manipulating files, and redirection/piping) are
> limited or non-existent ...

Who needs any of THAT except text displays, when all you want to do is
load a DEVICE-driver?? Last I noticed, Int 21h AH=09h will display a
text line on any DOS system, and one can always write an Int 10h loop,
since that is how DOS handles an Int 21h AH=09h anyway. More complex
graphics have NO BUSINESS being in a DEVICE-driver's load routines!

> ... The only drivers that actually need to be in CONFIG.SYS are things
> that the BIOS or DOS kernel may require early in the boot process ...

THANK YOU for "making my case" re: why UIDE always-was and always-will-be
a SYSTEM driver named UIDE.SYS!!!

> ... I think you should always have the option of removing drivers from
> memory ad-hoc if you want to without needing to reboot.

Such an "unload" process requires "tracking down" interrupt vectors and
doing some other things to ensure a TSR in fact CAN be unloaded! This
would add maybe 1000 bytes to UIDE, only to save the lousy 944 bytes it
now takes in upper-memory. Many writers of even smaller drivers than
UIDE might have much WORSE to say about an "everything unloadable" idea
which I think is flatly UNREASONABLE!!

> If a DOS driver is so old that it is not being updated or maintained any
> more, than there is not a lot you can do about that. But, all modern DOS
> programs should IMHO use modern protocols and installation techniques ...

BALONEY, and consider me PROUD in that case to write "old" drivers that
flatly do not NEED and will never HAVE such utter GARBAGE!! Especially
not needed, after my most-recent changes to UIDE ...

I can give you all a much better "order" to follow, rather than any such
"Sponsored by FATHEADED College-Professor DUFII" schemes like the above.

In the U.S. it is an old engineering "joke", not-so-much of a JOKE any
more, called the "K.I.S.S. Principle". The letters K.I.S.S. stand for
Keep It SIMPLE, STUPID!!

Perhaps good advice, for all those who have only a B.S.C.S. degree, and
PROVE IT all the time!!

I always wondered if the degree B.S.C.S. was, in itself, a huge JOKE by
professors in other more-established college disciplines who had become
rather DISGUSTED by "Gurus", "Wizards", or other self-inflated BOOBS in
computers, as the last 2 letters are often noted to mean Chicken S***!!

---
(Account disabled on user's request.)

I realize that you despise me, Jack, but is it even possible for us to disagree in a public forum without the need to resort to name-calling, cussing, and SCREAMING like a two-year old?

> I realize that you despise me, Jack, but is it even possible for us to
> disagree in a public forum without the need to resort to name-calling,
> cussing, and SCREAMING like a two-year old?

If you dislike "cussing", very well, one of two such words in my post is
changed and the second is disguised. Do not ask me to delete that word
as it is a well-known "paraphrase" for a certain U.S. college degree.

If you dislike my use of capital letters, I say the same to you as I did
to another on this forum: I use capital letters for emphasis when in my
opinion such emphasis is necessary. That is a style of writing, and is
not "screaming like a 2-year-old". If you do not like my style, do not
read my posts.

And where, in my post, was there anything specifically directed at you??
Do you find the words "You are ..." anywhere?? If not, then do accept,
same as you expect me to accept, that such comments are about classes of
people and computer "philosophy", and such comments were not directed at
you personally.

As for us despising or not-despising each other, that is not appropriate
material for this forum, as I hope Robert Riebisch will tell you.

---
(Account disabled on user's request.)