DOS ain't dead

Hello Japheth and all,
I see that that jwasm 2.0 production release was posted today on http://www.japheth.de/
I read the "64bit DOS sample" comments on this forum & my question is this: What can be done with this mode?
You call it 64bit real mode, what memory limitations does it have?
Using the sample code "Dos64.asm" as a model, would it be possible to write DOS programs/device drivers/command processors that uses this mode?
Could a 64bit Real Mode DOS be written that would be backwards compatable with 16bit DOS?
Best regards, Stephen William Foyle

> Hello Japheth and all,
> I see that that jwasm 2.0 production release was posted today on
> http://www.japheth.de/
> I read the "64bit DOS sample" comments on this forum & my question is
> this: What can be done with this mode?
> You call it 64bit real mode, what memory limitations does it have?
> Using the sample code "Dos64.asm" as a model, would it be possible to
> write DOS programs/device drivers/command processors that uses this mode?
> Could a 64bit Real Mode DOS be written that would be backwards compatable
> with 16bit DOS?
> Best regards, Stephen William Foyle

In lieu of Japheth's expert advice while we wait for his reply, I'll give my lame n00b opinion:

64-bit mode is incompatible with 16-bit (which most people consider obsolete). DOSEMU 1.4.0 can run on 64-bit Linux (e.g. XUbuntu 8.04.1 is what I tried), but it has to emulate the 16-bit stuff since there is no V86 mode. In other words, it ain't perfect (some bugs), but it more or less runs pretty well. 32-bit DJGPP stuff is able to run almost natively (albeit with BIOS emulation and DOS API translation too). Of course, that used vanilla FreeDOS 1.0 as the actual DOS (although MS or DR probably work too).

Japheth's example is apparently not using the BIOS or any memory managers or device drivers. In other words, it's pretty raw stuff. Calling it "DOS" is a stretch, it just runs "on top" of DOS at the beginning.

Can you access more than 3-4 GB in DOS? Probably, via PAE (see DOS/32A hack). Is it going to happen? Unlikely. Sure, anything's possible, but a lot of stuff might have to be emulated due to hardware limitations.

> Hello Japheth and all,
> I see that that jwasm 2.0 production release was posted today on
> http://www.japheth.de/

Yes. Changes:

- support for 64-bit enabled.
- directives for 64-bit SEH added (.ALLOCSTACK, .PUSHFRAME, ... )
- directive .SAFESEH and cmdline option -safeseh supported.
- cmdline option -Zd implemented for COFF output format.
- cmdline option -Zi implemented for OMF and COFF output formats.
- operators IMAGEREL and SECTIONREL work with BIN format, which makes it
possible to create PE binaries using this format.
- ELF32 format: LD extensions for 8/16-bit relocations supported.

See http://www.japheth.de/JWasm.html for more.

> I read the "64bit DOS sample" comments on this forum & my question is
> this: What can be done with this mode?
> You call it 64bit real mode, what memory limitations does it have?

No, it isn't "64bit real mode", it's the normal 64bit long-mode. There are no memory limitations in this mode.

> Using the sample code "Dos64.asm" as a model, would it be possible to
> write DOS programs/device drivers/command processors that uses this mode?

You can probably write a few useful programs using this sample as a template, but without a possibility to access DOS or BIOS functions it's rather limited.

> Could a 64bit Real Mode DOS be written that would be backwards compatable
> with 16bit DOS?

No. But it's possible to write a 64-bit DOS extender which runs application in 64-bit long mode.

However, as far as access to more than 4 GB is concerned: unlike to what Rugxulo babbles it's pretty likely that a small extension will be implemented in JemmEx ( and maybe also in J.R. Ellis' XMGR ) which supports more that 4 GB of memory. It is likely because it's not much work and the main reason why it isn't done yet for JemmEx is because I don't have access to such a machine yet. The extension will implement 4 MB pages. IIRC it's possible to access up to 1 TB (40-bit addresses) of physical memory with 4 MB pages.

---
MS-DOS forever!

> ... It's pretty likely that a small extension will be implemented in
> JemmEx (and maybe also in J.R. Ellis' XMGR) which supports more than
> 4 GB of memory.

I do plan to update XMGR for over 4-GB of memory, when needed. But I
have no 64-bit system, so I will likely have to "copy" Japheth's logic
from JEMM386/JEMMEX after he does it. Japheth is "The Master" of all
protected-mode logic (I am only a "rank amateur" at such things!), and
I will follow his "lead", as I did in upgrading XMGR/UIDE for JEMM386!

---
(Account disabled on user's request.)

> it's pretty likely that a small extension will be
> implemented in JemmEx ( and maybe also in J.R. Ellis' XMGR ) which
> supports more that 4 GB of memory.

Well, and when we will have such feature in some XMS manager how it would be utilized? Is there any DOS C compiler that can handle it? Even if yes who will/need to make dos apps that needs so much RAM?

---
DOS gives me freedom to unlimited HW access.

> Well, and when we will have such feature in some XMS manager how it would
> be utilized?

Through an API, perhaps?

> Is there any DOS C compiler that can handle it?

I don't know ... who cares?

> Even if yes who will/need to make dos apps that needs so much RAM?

Me, for example. HDPMI can use the memory. Or probably Jack's RDISK might prefer to use memory beyond the 4 GB limit and leave the memory below the limit to the less advanced tools, i.e. DGPJJ?

---
MS-DOS forever!

> When we have such a feature [over 4-GB RAM] in some XMS manager, how
> would it be utilized? ...

I agree with Japheth. All of my drivers will support "over 4-GB RAM".

XMGR will make it available for drivers/programs that can use it. New
XMS request codes [to be defined] shall specify use of "over 4-GB RAM".
UIDE and RDISK will ask the XMS manager (HIMEMX, JEMMEX, XMGR) if "over
4-GB RAM" is available, and if so, they will request it. If not, they
will request normal XMS memory below 4-GB, same as they do now.

With minor changes, UIDE can be upgraded from its current 2-GB limit to
handle caches up to 4-GB, using "over 4-GB RAM". With a fast CPU, its
cache "overhead" should still be manageable, even at a 4-GB cache size.
I was SURPRISED that a 2-GB cache was no-problem, and CPUs keep getting
faster!

With a bit more work, RDISK can be upgraded to handle multiple 2-GB RAM
disks using "over 4-GB RAM". I want to keep each RAM disk at 2-GB, so
older DOS systems having only FAT-16 logic can still make use of RDISK.

All the above changes should have no impact on current programs. They
can still request XMS memory below 4-GB using the currently-defined XMS
command codes. UIDE setting caches up to 4-GB, and RDISK setting more
than one 2-GB RAM disk, also shouldn't affect current programs. These
are "management issues" that only affect UIDE and RDISK.

Japheth and I must decide (A) when "over 4-GB RAM" is required, and (B)
what new XMS logic is needed for it. Microsoft no-longer supports DOS
and will never update their HIMEM driver, thus this task is for Japheth
and me. If his HIMEMX/JEMMEX and my XMGR get the same new logic, they
can go on being used "interchangeably" for XMS work, same as now.

---
(Account disabled on user's request.)

> Through an API, perhaps?

Sorry I don't know details of XMS spec but I didn't know if it is ready to hadle over 4G. Or did you mean that you will write your own API extension?

> > Is there any DOS C compiler that can handle it?
> I don't know ... who cares?

We all know you're ASM |-|4rk0r3 dUd3 but I'm interested in higher level languages. Maybe someone else too.

> Me, for example. HDPMI can use the memory. Or probably Jack's RDISK might
> prefer to use memory beyond the 4 GB limit and leave the memory below the
> limit to the less advanced tools, i.e. DGPJJ?

OK, yes it makes a sense.

---
DOS gives me freedom to unlimited HW access.

Aha, so it have to be defined, thanks for clarification.

---
DOS gives me freedom to unlimited HW access.

> >> Is there any DOS C compiler that can handle it?
> > I don't know ... who cares?
>
> We all know you're ASM |-|4rk0r3 dUd3 but I'm interested in higher level
> languages. Maybe someone else too.

If anyone wants to use these API extensions in HLLs they'll just write their own library for it (or extend the library of their compiler if applicable).

---
l

Jack since you are discussing xmgr is it possible to have xmgr leave some "unmanged extended memory" I have a DOS program Framework III by Ashton Tate that I like to use and it can use "extended memory" but not xms memory
to load larger documents. I is a really nis=ce programe Word Processor/Spredasheet/Database/Terminal with a "Windowed" environment.
Best regards, Stephen William Foyle

> Jack, since you are discussing XMGR is it possible to have XMGR leave
> some "unmanged extended memory"?

No good way to do this. The BIOS uses top-down memory allocation, but
this is not fully reliable. Some XMS managers "limit" how much memory
they use but provide no way to access any remainder, if present. Best
to obey the XMS Specification "intent", that an XMS manager has control
of ALL "extended memory", and so I feel obliged to leave XMGR as it is.

---
(Account disabled on user's request.)

The concept of making memory > 4 GB accessible under DOS is an exciting one! I for one welcome all future support and hope you do it soon! I have a 16 GB system and it will be interesting to finally be able to "see" all this RAM in pure DOS.

4 GB caches sound great... How to do this?

Re: RDSISK and > 4 GB. Will it not be possible to create a FAT32 ramdisk > 4 GB? Or are we strictly limited to 2 GB (4 GB with 64-kb cluster) FAT volumes? I can understand wanting to maintain compatibility with DOS 6.2, but would you consider making a souped-up seperate version of RDISK with > 4 GB support for those who use MS- "DOS" 7.1 and DR-DOS, etc?

Another big THANK YOU for all the work you have done Jack, and for your wonderful skills!

> The concept of making memory > 4 GB accessible under DOS is an exciting
> one! I for one welcome all future support and hope you do it soon! I
> have a 16 GB system ...

Regrettably I am retired on low-income, and I have only a 1-GB AMD 3000+
on an old VIA mainboard. I would need a new system with at least 8-GB,
to do such upgrades. In private E-Mails, Japheth noted his top goal is
over 4-GB for his HDPMI, and a "general use" XMS driver over 4-GB is not
yet in big demand. For both of us, over 4-GB "may be a while, yet"!

> 4 GB caches sound great ... How to do this?

UIDE now offers 2-GB caches. At 64-KB per cache block (due to UltraDMA
64-KB "boundaries"), this is 16 blocks per cache megabyte, or 32K blocks
total. For 4-GB, I need 64K total cache blocks and twice as much cache
memory. A few adjustments needed in UIDE's code, but "not a big deal"!

> Re: RDISK and > 4 GB. Will it not be possible to create a FAT32 ramdisk >
> 4 GB? Or are we strictly limited to 2 GB (4 GB with 64-kb cluster) FAT
> volumes?

I still use V6.22 MS-DOS which is FAT-16, so I have no FAT-32 knowledge.
My "guess" is that RDISK will only need to initialize an empty directory
in its RAMdisk memory for FAT-32, as it does now for FAT-16. All RDISK
does after init is to read or write RAMdisk blocks -- the DOS system has
all the "intelligence" about exactly what file system is in use.

> I can understand wanting to maintain compatibility with DOS 6.2, but
> would you consider making a souped-up seperate version of RDISK with >
> 4 GB support for those who use MS- "DOS" 7.1 and DR-DOS, etc?

I prefer not to have 2 drivers. If properly managed, a 2- or 4-GB RAM
disk is still a LOT of data, and it must be copied from elsewhere up to
XMS memory each time a system boots. I can't imagine users wanting to
do this for MORE than a 4-GB RAMdisk on every system boot -- too slow!!

> Another big THANK YOU for all the work you have done Jack, and for your
> wonderful skills!

You are welcome -- DOS must SURVIVE, and so I try to do the best I can,
in all my drivers. No magic, and I am no "Hoodoo Man", just hard work
[43 years, since 1966!], often with nice results like XMGR/UIDE/RDISK!

---
(Account disabled on user's request.)

> > Jack, since you are discussing XMGR is it possible to have XMGR leave
> > some "unmanged extended memory"?
>
> No good way to do this. The BIOS uses top-down memory allocation, but
> this is not fully reliable. Some XMS managers "limit" how much memory
> they use but provide no way to access any remainder, if present. Best
> to obey the XMS Specification "intent", that an XMS manager has control
> of ALL "extended memory", and so I feel obliged to leave XMGR as it is.

Jack, "The BIOS uses top-down memory allocation" Actually that is exactly what I want, 1989 Framework III wich knows nothing about XMS memory, would make a request to the BIOS for the extended memory then handle whatever it found available itself. So on my 500mb machine if I had a switch in XMGR that would leave the bottom 32mb as unmanaged extended memory it may just work!!
Best regards, Stephen

> > "The BIOS uses top-down memory allocation ..."
>
> Actually, that is exactly what I want ... If I had a switch in XMGR that
> leaves the bottom 32-MB as unmanaged extended memory, it may just work!!

"MAY" just work is what worries me. BIOS writers have been eliminating
old features to make room in BIOS chips for new things, like USB. Also
note that XMGR's current scan-memory logic exactly fits in the space for
a table of 48 XMS "handles", leaving no wasted initialization code. If
anything else is added, I need a new "scheme" for where to locate XMGR's
initialization routines, so they disappear completely after init.

Give me a bit more time to "think about" this item. I enjoy a "Mission
IMPOSSIBLE" task, as were XMGR/UIDE, and some DO say I am a "Hoodoo Man"
who works miracles, i.e. it probably can be done!

---
(Account disabled on user's request.)

> > >> Is there any DOS C compiler that can handle it?
> > > I don't know ... who cares?
> >
> > We all know you're ASM |-|4rk0r3 dUd3 but I'm interested in higher
> level
> > languages. Maybe someone else too.
>
> If anyone wants to use these API extensions in HLLs they'll just write
> their own library for it (or extend the library of their compiler if
> applicable).

DJGPP 2.04 beta (according to CWS of CWSDPMI fame) has better support for 4 GB files (and symlinks, better C99 support, etc). It may have better memory management too, I'm not sure. (I've very rarely noticed better behavior under Windows, for instance.) Actually, wasn't it 2.02 that first got the improved malloc? And CBFalconer's nmalloc was never fully finished (DJ wanted debug hooks a la standard implementation) or integrated due to personal issues on his side, I guess. But it was allegedly much faster at freeing memory, at least.

In short, CWS has strong interest in finishing / releasing 2.04 final one of these days. But it will need to have testers, release manager, etc. since it will probably be the last official DJGPP version. (Of course, somebody could fork it, but ....)