DOS ain't dead

New kernel compression methods - Corrections (Announce)

posted by ecm  , Düsseldorf, Germany, 16.04.2020, 22:06

This part of the lDebug results is wrong. The x method does use that much memory. However, the heatshrink and lzip methods do not. That was due to an error in the mak script used to build the debugger.

> method=x
> size=71680
> ini size=6384
> alloc=343920 bytes = 21495 paragraphs
> 12.471
>
> method=hs
> size=63488
> ini size=5536
> alloc=343904 bytes = 21494 paragraphs
> 7.730
>
> method=lz
> size=53760
> ini size=8336
> alloc=344208 bytes = 21513 paragraphs
> 16.503
> ldebug/tmp$

Here are the fixed results. First, the original lzd port, then all the other methods with the updated lzd that allows overlap and halves the probability table size (localvariables < 16 KiB now).

ldebug/tmp$ hg id
246b9ff45d45+ tip
ldebug/tmp$ fname="ldebug.com"; fname2="debug"; fname3="debug.big"; fnameu="ldebugu.com"; arg="/C=Q"; upcase=0; method=none; ((upcase)) && method="$(echo "$method" | tr 'a-z' 'A-Z')"; mkdir -p "$method"; cp -a ../bin/"$fnameu" "$method"/"$fname"; cp -a "$fname3" "$method/$fname2.$method"; for method in none lz; do ((upcase)) && method="$(echo "$method" | tr 'a-z' 'A-Z')"; echo -e "\nmethod=$method\nsize=$(stat -c %s "$method/$fname")"; echo -e "ini size=$(( $(stat -c %s "$method/$fname") - ( ($(stat -c %s "$method/$fname2.$method") + 15) / 16 * 16 ) ))"; echo -e "alloc=$(exememls "$method/$fname")"; echo "@echo off" > test.bat; for jj in $(seq 0 511); do echo "@$method\\$fname $arg"; done >> test.bat; (export TIMEFORMAT='%3R'; time dosemu -dumb -quiet -K "$PWD" -E "test.bat" 2> /dev/null > /dev/null); done

method=none
size=82944
ini size=4800
alloc=96912 bytes = 6057 paragraphs
2.280

method=lz
size=53760
ini size=8208
alloc=158640 bytes = 9915 paragraphs
15.614
ldebug/tmp$ fname="ldebug.com"; fname2="debug"; fname3="debug.big"; fnameu="ldebugu.com"; arg="/C=Q"; upcase=0; method=none; ((upcase)) && method="$(echo "$method" | tr 'a-z' 'A-Z')"; mkdir -p "$method"; cp -a ../bin/"$fnameu" "$method"/"$fname"; cp -a "$fname3" "$method/$fname2.$method"; for method in none blz lz4 sz exo x hs lz; do ((upcase)) && method="$(echo "$method" | tr 'a-z' 'A-Z')"; echo -e "\nmethod=$method\nsize=$(stat -c %s "$method/$fname")"; echo -e "ini size=$(( $(stat -c %s "$method/$fname") - ( ($(stat -c %s "$method/$fname2.$method") + 15) / 16 * 16 ) ))"; echo -e "alloc=$(exememls "$method/$fname")"; echo "@echo off" > test.bat; for jj in $(seq 0 511); do echo "@$method\\$fname $arg"; done >> test.bat; (export TIMEFORMAT='%3R'; time dosemu -dumb -quiet -K "$PWD" -E "test.bat" 2> /dev/null > /dev/null); done

method=none
size=82944
ini size=4800
alloc=96912 bytes = 6057 paragraphs
2.253

method=blz
size=64512
ini size=5920
alloc=96688 bytes = 6043 paragraphs
5.155

method=lz4
size=62976
ini size=5824
alloc=96704 bytes = 6044 paragraphs
3.403

method=sz
size=70144
ini size=6000
alloc=96864 bytes = 6054 paragraphs
2.907

method=exo
size=54784
ini size=5776
alloc=96736 bytes = 6046 paragraphs
6.188

method=x
size=72192
ini size=6832
alloc=344512 bytes = 21532 paragraphs
12.561

method=hs
size=64000
ini size=5904
alloc=96960 bytes = 6060 paragraphs
7.668

method=lz
size=53248
ini size=7680
alloc=97952 bytes = 6122 paragraphs
15.084
ldebug/tmp$

Here's the RxDOS results in the same order. The mak bug did not affect this build.

rxdos/tmp$ hg id
4e74dca146e1+ tip
rxdos/tmp$ fname="RxDOS.COM"; fname2="RxDOS"; fname3="RxDOS.BIN"; fnameu="RxDOSU.COM"; arg="version"; upcase=1; method=none; ((upcase)) && method="$(echo "$method" | tr 'a-z' 'A-Z')"; mkdir -p "$method"; cp -a ../bin/"$fnameu" "$method"/"$fname"; cp -a "$fname3" "$method/$fname2.$method"; for method in none lz; do ((upcase)) && method="$(echo "$method" | tr 'a-z' 'A-Z')"; echo -e "\nmethod=$method\nsize=$(stat -c %s "$method/$fname")"; echo -e "ini size=$(( $(stat -c %s "$method/$fname") - ( ($(stat -c %s "$method/$fname2.$method") + 15) / 16 * 16 ) ))"; echo -e "alloc=$(exememls "$method/$fname")"; echo "@echo off" > test.bat; for jj in $(seq 0 511); do echo "@$method\\$fname $arg"; done >> test.bat; (export TIMEFORMAT='%3R'; time dosemu -dumb -quiet -K "$PWD" -E "test.bat" 2> /dev/null > /dev/null); done

method=NONE
size=101376
ini size=4512
alloc=98992 bytes = 6187 paragraphs
2.231

method=LZ
size=47104
ini size=8336
alloc=170704 bytes = 10669 paragraphs
15.343
rxdos/tmp$ fname="RxDOS.COM"; fname2="RxDOS"; fname3="RxDOS.BIN"; fnameu="RxDOSU.COM"; arg="version"; upcase=1; method=none; ((upcase)) && method="$(echo "$method" | tr 'a-z' 'A-Z')"; mkdir -p "$method"; cp -a ../bin/"$fnameu" "$method"/"$fname"; cp -a "$fname3" "$method/$fname2.$method"; for method in none blz lz4 sz exo x hs lz; do ((upcase)) && method="$(echo "$method" | tr 'a-z' 'A-Z')"; echo -e "\nmethod=$method\nsize=$(stat -c %s "$method/$fname")"; echo -e "ini size=$(( $(stat -c %s "$method/$fname") - ( ($(stat -c %s "$method/$fname2.$method") + 15) / 16 * 16 ) ))"; echo -e "alloc=$(exememls "$method/$fname")"; echo "@echo off" > test.bat; for jj in $(seq 0 511); do echo "@$method\\$fname $arg"; done >> test.bat; (export TIMEFORMAT='%3R'; time dosemu -dumb -quiet -K "$PWD" -E "test.bat" 2> /dev/null > /dev/null); done

method=NONE
size=101376
ini size=4512
alloc=98992 bytes = 6187 paragraphs
2.251

method=BLZ
size=56320
ini size=6240
alloc=100496 bytes = 6281 paragraphs
4.733

method=LZ4
size=54784
ini size=5712
alloc=99984 bytes = 6249 paragraphs
3.271

method=SZ
size=59904
ini size=5760
alloc=100016 bytes = 6251 paragraphs
3.195

method=EXO
size=47616
ini size=5760
alloc=100016 bytes = 6251 paragraphs
5.601

method=X
size=68608
ini size=6576
alloc=362992 bytes = 22687 paragraphs
24.478

method=HS
size=57856
ini size=6000
alloc=100256 bytes = 6266 paragraphs
6.999

method=LZ
size=46080
ini size=7312
alloc=116288 bytes = 7268 paragraphs
14.264
rxdos/tmp$

---
l

Complete thread:

• New kernel compression methods - ecm, 13.04.2020, 00:05 (Announce)
  • New kernel compression methods - ecm, 13.04.2020, 00:17
  • New kernel compression methods - tkchia, 13.04.2020, 14:23
    • New kernel compression methods - ecm, 13.04.2020, 18:31
      • New kernel compression methods - ecm, 13.04.2020, 18:35
      • New kernel compression methods - tkchia, 14.04.2020, 14:46
        • New kernel compression methods - fdkernpl - ecm, 14.04.2020, 17:05
        • New kernel compression methods - exeExtraBytes jump - ecm, 14.04.2020, 17:06
      • New kernel compression methods - tom, 14.04.2020, 20:06
  • New kernel compression methods - Updated lDebug lzd results - ecm, 16.04.2020, 17:29
  • New kernel compression methods - Corrections - ecm, 16.04.2020, 22:06
    • New kernel compression methods - X compressor layers - ecm, 16.04.2020, 22:22
    • New kernel compression methods - Updated lzd, new lzo - ecm, 25.04.2020, 22:23
      • New kernel compression methods - Updated lzd, new lzo - tom, 26.04.2020, 13:46
        • New kernel compression methods - UPX, 512 runs clarification - ecm, 26.04.2020, 20:17