Back to home page

DOS ain't dead

Forum index page

Log in | Register

Back to index page
Thread view  Board view
cm(R)

Homepage E-mail

Düsseldorf, Germany,
08.09.2018, 14:33
 

RxDOS kernel compression (Developers)

Yesterday and today I finally got around to implementing one of these "killer" features of any DOS kernel ;-) which is packing the kernel and adding a depacker that lets it decompress itself at run-time.

As algorithm, I used Joergen Ibsen's BriefLZ, which is very simple and available under the zlib usage conditions. It still manages to compress the kernel to about 50% of its size, which is plenty for me. I haven't compared the compression ratio against that of UPX (as used by FreeDOS's and EDR-DOS's kernel compression), but the BriefLZ usage conditions are permissive and lack UPX's copyleft.

I used the C-source safe depacker as a base, and ported it to 8086-compatible assembly. I optimised it a bit and added the ability to make use of overlapping source and destination buffers. (The compressed source data is moved to the top of useable memory, then decompressed to the destination buffer at linear 0_0600h.) Unlike the solution suggested by BriefLZ, my port supports any block size, including blocks larger than 64 KiB, by implementing some segmentation work.

Unlike my earlier plans, I didn't write a specific utility program, opting instead to use the format of BriefLZ's example program, blzpack. This way, the kernel can be compressed during build time, and included as a compressed payload by iniblz. This, with its included depacker stub, is then the payload to iniload, which handles completely loading the kernel file from the file system.

It's available from https://bitbucket.org/ecm/iniblz/src/default/ (Though when releasing a new RxDOS build I will likely provide pre-built binaries.)

---
l

cm(R)

Homepage E-mail

Düsseldorf, Germany,
08.09.2018, 23:45

@ cm
 

RxDOS kernel compression

> As algorithm, I used Joergen Ibsen's BriefLZ, which is very simple and
> available under the zlib usage conditions. It still manages to compress the
> kernel to about 50% of its size, which is plenty for me. I haven't compared
> the compression ratio against that of UPX (as used by FreeDOS's and
> EDR-DOS's kernel compression), [...]

I tested this now: On a file made from the first 65_000 bytes of some kernel image, UPX compresses to 26_325 bytes (40.50%, with depacker), while BriefLZ compresses to 32_115 bytes (49%, without depacker).

The actual full kernel image is 86_352 bytes, which UPX doesn't accept as a .COM format file, so cannot be tested that way with UPX. BriefLZ compresses it to 43_316 bytes (50%, without depacker).

The depacker INIT0 and INIT1 code is less than 2 KiB large. Additionally, the iniload stage is 2 KiB. So, add 4 KiB of uncompressed handling.

Note: To compress a flat binary as .COM file with UPX, the filename extension must be .com to allow its detection.

---
l

swf(R)

E-mail

Canada,
11.09.2018, 00:22

@ cm
 

RxDOS kernel compression

could you build into
the kernel console full color capabilities as in ansi.sys

Rugxulo(R)

Homepage

Usono,
11.09.2018, 00:31

@ swf
 

RxDOS kernel compression

> could you build into
> the kernel console full color capabilities as in ansi.sys

You mean like the RxANSI tool?

swf(R)

E-mail

Canada,
11.09.2018, 01:39
(edited by Rugxulo, 13.09.2018, 04:49)

@ Rugxulo
 

RxDOS kernel compression

Is this a separate tool like ansi.sys or ansi.com or is it a integral of RxDOS's RxDOS.COM?

Rugxulo(R)

Homepage

Usono,
11.09.2018, 02:08

@ swf
 

RxDOS kernel compression

> Is this a separate tool like ansi.sys or ansi.com or is it a integral of
> RxDOS's RxDOS.COM?

Separate tool, but it can unload. What do you miss/need that it doesn't do? (BTW, read the .DOC file.)

cm(R)

Homepage E-mail

Düsseldorf, Germany,
11.09.2018, 22:11

@ Rugxulo
 

RxANSI and RxDOS kernel

> > Is this a separate tool like ansi.sys or ansi.com or is it a integral of
> > RxDOS's RxDOS.COM?

It is a separate tool, yes.

> Separate tool, but it can unload. What do you miss/need that it doesn't do?
> (BTW, read the
> .DOC
> file.)

I have actually considered the case of embedding RxANSI (my supreme TSR project) into the kernel, but the benefits don't outweigh the costs.

RxDOS.COM will eventually contain things like kernel config (shunned into SYS for the FreeDOS kernel) and file sharing and locking controls ("SHARE"), as well as other kernel-specific tools. For now, it contains an MZ exe header that cannot be loaded as a program [will throw an error]. One step along the way is making iniblz support _IMAGE_EXE, the option to be dual-loadable as either a kernel or a program.

---
l

swf(R)

E-mail

Canada,
12.09.2018, 02:31
(edited by Rugxulo, 12.09.2018, 04:00)

@ cm
 

RxANSI and RxDOS kernel

Where can I download a copy of RxANSI.COM? Did you make a special stripped down version for me back in 2009, that machine died a few years back.

cm(R)

Homepage E-mail

Düsseldorf, Germany,
12.09.2018, 09:09

@ swf
 

RxANSI and RxDOS kernel

> Where can I download a copy of RxANSI.COM? Did you make a special stripped
> down version for me back in 2009, that machine died a few years back.

The latest, 1.10 beta 4+, is available at https://bitbucket.org/ecm/rxansi/downloads/

---
l

swf(R)

E-mail

Canada,
12.09.2018, 18:33
(edited by Rugxulo, 13.09.2018, 04:49)

@ cm
 

RxANSI and RxDOS kernel

Thanks, I found and downloaded it.
Now my original suggestion was could you eventually add the ANSI color control system to your RxDOS.COM so there would be color control from the initial boot up, I know that this is just an esthetic consideration but it may also reduce DOS's memory footprint??

cm(R)

Homepage E-mail

Düsseldorf, Germany,
12.09.2018, 19:46

@ swf
 

RxANSI and RxDOS kernel

> Thanks, I found and downloaded it.
> Now my original suggestion was could you eventually add the ANSI color
> control system to your RxDOS.COM so there would be color control from the
> initial boot up, I know that this is just an esthetic consideration but it
> may also reduce DOS's memory footprint??

http://www.bttr-software.de/forum/forum_entry.php?id=15489

> I have actually considered the case of embedding RxANSI (my supreme TSR project) into the kernel, but the benefits don't outweigh the costs.

---
l

swf(R)

E-mail

Canada,
12.09.2018, 21:01
(edited by Rugxulo, 13.09.2018, 04:50)

@ cm
 

RxANSI and RxDOS kernel

what would the 'costs' be?

cm(R)

Homepage E-mail

Düsseldorf, Germany,
12.09.2018, 21:05

@ swf
 

RxANSI and RxDOS kernel

> what would the 'costs' be?

Writing it. You're welcome to go do it yourself of course.

---
l

Back to index page
Thread view  Board view
15443 Postings in 1400 Threads, 256 registered users, 10 users online (0 registered, 10 guests)
DOS ain't dead | Admin contact
RSS Feed
powered by my little forum