DOS ain't dead

Rugxulo

Usono,
20.08.2012, 22:46

p7zip 9.20.1 (Users)

For whatever reason, I decided to try building latest p7zip 9.20.1, but this time I used Ozkan's uHexen2 DJGPP cross compiler from atop Linux. Much faster build time by far, but I didn't tweak much. (Also VILE editor helped a lot, e.g. "e !grep -r _WIN32 p7zip_9.20.1" and then ^X-e on each filename I wanted to edit, etc.)

It does (again) fix the kFAT flag internally in .ZIPs so PKUNZIP can hopefully unpack its output. Other than that, I didn't do anything special. I used Khusraw's FSU Pthreads fixed lib. This one does also additionally include 7zr.exe (.7z support only), which he had neglected to include in previous 9.13.

Bug? It still relies on *nix-style forward slashes '/' in dir paths, which is bad, but I couldn't find a solution that actually fixed it (tried and failed, sadly). So that's still annoying. But it's a small price to pay for a working util.

Feel free to kick the tires and test it. I didn't test too too much, but benchmarking doesn't crash, and it compresses its own .ZIP, so I guess that's good. :-)

http://ftp.ibiblio.org/pub/micro/pc-stuff/freedos/files/util/file/7zip/9.20.1/p7z9201.zip (5 MB)

mvojvodic

21.08.2012, 22:43
(edited by mvojvodic, 21.08.2012, 23:44)

@ Rugxulo

p7zip 9.20.1

Post reply

>
> Bug? It still relies on *nix-style forward slashes '/' in dir paths, which
> is bad, but I couldn't find a solution that actually fixed it (tried and
> failed, sadly). So that's still annoying. But it's a small price to pay for
> a working util.
>
> Feel free to kick the tires and test it. I didn't test too too much, but
> benchmarking doesn't crash, and it compresses its own .ZIP, so I guess
> that's good. :-)
>

Tested it under DataLight ROM-DOS 7.10 R4.20.1594SU (last freeware version),
Wengier DOS 7.10, FreeDOS 1.1 (installed from Base CD), and in real DOS mode
of Windows 98 SE (used BootGUI=0 in MSDOS.SYS, so Windows does not start
automatically after DOS and I start Windows by running WIN.COM).

It works perfectly. I did not find any bug testing it for more than four
hours under various conditions.
Forward slashes are no problem (only in DOS Navigator 6.4.0).

Some antivirus packages report many UPX packed exe files as suspicious. This
false alarm is very common, because they wish to act as some sort of police
to protect everything from the pirates. So when they see something packed,
that is malware. The leader in such foolish behaviour is famous Sophos
Antivirus. They flagged the whole SysInternals Suite by Russinovich as
malware. After I packed Sophos using UPX, it moved itself to quarantine!
Sophos flagged your P7ZIP as malware. I do not have Sophos (I never will)
but my friend has it, so I tested P7ZIP using his machine.

Now I can use 7-zip 9.20.1 in DOS without any emulator.

Rugxulo

Usono,
23.08.2012, 00:16

@ mvojvodic

p7zip 9.20.1b "testing"

Post reply

> It works perfectly. I did not find any bug testing it for more than four
> hours under various conditions.
> Forward slashes are no problem (only in DOS Navigator 6.4.0).

I've made another attempt (thanks to some very good hints from Rod P. on news://comp.os.msdos.djgpp). But I also kept the older .ZIP just in case I made it worse somehow.

http://ftp.ibiblio.org/pub/micro/pc-stuff/freedos/files/util/file/7zip/9.20.1/p7z9201b-testing.zip

So this one should (hopefully) work with DOS-style backslashes '\'. My brief tests indicate no obvious errors (besides a few minor nits). So 7zr, 7za, 7zcon all apparently work correctly (though I hate that the sfx is default extract instead of test or list, ugh): "copy /b 7zcon.exe + blah.7z blah.exe" (though 7zdecode.exe is smaller!).

= nit #1). Updating still tries using LFN-only blah.7z.tmp, which is annoying in SFN. I'm too tired right now to fix that (not that I know where it is exactly either). Workaround: "ren blah.7z blah", "7za d -r blah. sources\*.c", "ren blah *.7z". ;-)

= nit #2). Previous build was crashing with -mx8 and -mx9 when compressing .EXEs. I blindly assumed it was a compiler bug uncovered in BCJ*.cpp, so I rebuilt those BCJ*.o (only) with -Os (instead of -O2), and now it works fine.

= nit #3). Doesn't recognize BLAH.7Z [sic], but blah.7z is accepted fine. Dunno.

> Some antivirus packages report many UPX packed exe files as suspicious.

Unpack it and/or use 32LITE. Or nag the vendors. (Maybe turning off heuristics helps??) Annoyingly unavoidable problem. :-(

> Now I can use 7-zip 9.20.1 in DOS without any emulator.

You mean without HX? Yes, HX + 7ZA.EXE still works. Pretty cool alternative. ;-)

Zyzzle

24.08.2012, 22:02

@ Rugxulo

p7zip 9.20.1b "testing"

Post reply

Tested 9.20, and it works just fine for me in all modes, and LZMA -mx8 and -mx9 work very well! However, your compile of 9.20 is 5-10% slower in deflate -mx9 than the old 9.13 compile posted here over a year ago... Also, the your 9.20 is ~200k more bloated than that old 9.13, which I still use, and which was also compiled with pthreads. There seems no speed advantage at all to 9.13 vs 9.20, are there any hidden advantages?

EDITED to ask if it's possible to get a compile of 7zip .ZIP deflate modes ONLY, without any other compression modules?

Anyway, thanks for making 9.20 available for us to test.

Rugxulo

Usono,
25.08.2012, 00:04

@ Zyzzle

p7zip 9.20.1b "testing"

Post reply

Hi,

> Tested 9.20, and it works just fine for me in all modes, and LZMA -mx8 and
> -mx9 work very well! However, your compile of 9.20 is 5-10% slower in
> deflate -mx9 than the old 9.13 compile posted here over a year ago...

What cpu are you on? I only did "-O2 -mtune=i686". Perhaps other settings would be better for you? ("-O3? -fomit-frame-pointer -mtune=??") This "old" G++ 3.4.6 compiler probably isn't as good as newer ones. I haven't tried building "natively" (under DOSEMU) with latest DJGPP's 4.7.1 yet. (That would be preferred by me but is always more difficult.)

> Also, your 9.20 is ~200k more bloated than that old 9.13,

Probably a libc issue, who knows. I could rebuild entirely with "-Os", but it wouldn't be faster, probably slower.

> which I still use,
> and which was also compiled with [FSU] pthreads. There seems no speed advantage
> at all to 9.13 vs 9.20, are there any hidden advantages?

Honestly, I don't know, perhaps bugfixes. I didn't check the changelog, just assumed it was better! :-)

http://www.7-zip.org/history.txt

> EDITED to ask if it's possible to get a compile of 7zip .ZIP deflate modes
> ONLY, without any other compression modules?

No, not really, not that I know of. You can try AdvanceComp's ADVZIP.EXE ("-z4") to repack .ZIPs in place. It may? also allow creating archives from scratch, but I don't know if that's .7z only or what (can't remember).

http://advancemame.sourceforge.net/comp-readme.html

http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/util/file/7zip/advancecomp/

N.B. Those binaries of his are several years old, so if you want fresher (probably more bloated but faster), I can oblige. Just please mention compiler settings preferrred. ;-)

> Anyway, thanks for making 9.20 available for us to test.

I assume this means you used the second one ('\' fixes, aka "9.20.1b-testing")?

Zyzzle

25.08.2012, 00:59

@ Rugxulo

p7zip 9.20.1b "testing"

Post reply

>
> > Also, your 9.20 is ~200k more bloated than that old 9.13,
>
> Probably a libc issue, who knows. I could rebuild entirely with "-Os", but
> it wouldn't be faster, probably slower.

If you've got the time, it would be interesting to test how the size and speed with -Os compares.

I'm testing it on 2 systems, one is an old Intel Atom netbook, the other an i7 2600k, so it appears that -machinetype=i686 would yield best results on both systems. Also, you could try -O3 and I can compare if there are any speed differences.

>
> > which I still use,
> > and which was also compiled with [FSU] pthreads. There seems no speed
> advantage
> > at all to 9.13 vs 9.20, are there any hidden advantages?
>
> Honestly, I don't know, perhaps bugfixes. I didn't check the changelog,
> just assumed it was better! :-)
>
> http://www.7-zip.org/history.txt
>

Lots of "bug fixes" listed since 9.13, but it is not specified what these fixes are or do!

> > EDITED to ask if it's possible to get a compile of 7zip .ZIP deflate
> modes
> > ONLY, without any other compression modules?
>
> No, not really, not that I know of. You can try AdvanceComp's ADVZIP.EXE
> ("-z4") to repack .ZIPs in place. It may? also allow creating archives from
> scratch, but I don't know if that's .7z only or what (can't remember).
>
> http://advancemame.sourceforge.net/comp-readme.html
>
> http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/util/file/7zip/advancecomp/
>
> N.B. Those binaries of his are several years old, so if you want fresher
> (probably more bloated but faster), I can oblige. Just please mention
> compiler settings preferrred. ;-)

Yes, I've used ADVZIP for years, however even its "insane" setting isn't as good as 7zip's -mx9 deflate settings for creating optimized .ZIPS. This may be due to the old age of the code (2005) I'm using, but see below!

I found AdvanceComp 1.16 beta here: http://advancemame.sourceforge.net/beta/
which is the latest version for DOS I can find. It doesn't improve on speed or compression. If you've got newer code that you can compile (with machinetype=i686 and -O2 or -O3 GCC compiler options, I'd love to try them out! Thanks for offering.

>
> > Anyway, thanks for making 9.20 available for us to test.
>
> I assume this means you used the second one ('\' fixes, aka
> "9.20.1b-testing")?

Yes, that second one works perfectly for me with -mx8 and -mx9 and .EXE filters, your first compile did not work with -mx9 and .EXE filters. Good sleuthing on figuring out that compiler error!

Rugxulo

Usono,
25.08.2012, 01:58

@ Zyzzle

p7zip 9.20.1b "testing"

Post reply

> If you've got the time, it would be interesting to test how the size and
> speed with -Os compares.

http://ftp.ibiblio.org/pub/micro/pc-stuff/freedos/files/util/file/7zip/9.20.1/p7z9201b-misc.zip (2.1 MB)

Archive: p7z9201b-misc.zip Zip file size: 2206984 bytes, number of entries: 5 -rwxa-- 6.3 fat 195244 b- defN 12-Aug-24 18:26 7zcon.exe -rwxa-- 6.3 fat 553556 b- defN 12-Aug-24 18:24 o3-i686.exe -rwxa-- 6.3 fat 600620 b- defN 12-Aug-24 18:24 o3i486fp.exe -rwxa-- 6.3 fat 414704 b- defN 12-Aug-24 18:24 os-i386.exe -rwxa-- 6.3 fat 451168 b- defN 12-Aug-24 18:24 osi486fp.exe 5 files, 2215292 bytes uncompressed, 2206472 bytes compressed: 0.4%

N.B. Since I was recompiling anyways, I figured I'd also do the "obvious" and make a "small as possible" sfx module. (Why didn't I think of that before?) Still too big, honestly, but whatever, I guess it supports lots of things that 7zdecode doesn't.

> I'm testing it on 2 systems, one is an old Intel Atom netbook, the other an
> i7 2600k, so it appears that -machinetype=i686 would yield best results on
> both systems. Also, you could try -O3 and I can compare if there are any
> speed differences.

I only tested "p7zip b" (totally silly benchmark, aren't they all?) on this Core i5, but of these four builds above, there isn't much difference. This is probably the compiler's fault, though. The os-i386.exe was a bit slower than others at compressing while o3-i686.exe (686+ only, -march=i686) was a bit faster than others at decompressing. Otherwise, basically the same.

EDIT: I have no idea if native build via DOSEMU works anymore. Probably not, at least not without some tweaks. At least that would let me use newer G++, e.g. 4.5.0 first introduced explicit support for Atom. I never had any Atoms, only heard first ones were "in order" (like 486, see above) and later were "out-of-order". So I have no idea what would help here.

> > > which I still use,
> > > and which was also compiled with [FSU] pthreads. There seems no speed
> > advantage
> > > at all to 9.13 vs 9.20, are there any hidden advantages?

You'll have to ask Khusraw. Maybe I could email him. Anyways, it depends on what compiler version and options he used.

> > Honestly, I don't know, perhaps bugfixes. I didn't check the changelog,
> > just assumed it was better! :-)
> >
> > http://www.7-zip.org/history.txt
>
> Lots of "bug fixes" listed since 9.13, but it is not specified what these
> fixes are or do!

Dunno. If it doesn't bite you, then it doesn't matter, I suppose.

I vaguely remember DOS386 mentioning this one (esp. since he had written his own 3rd-party fix for previous versions), but I don't use passwords, so I never tested it.

"- The console version now doesn't show entered password."

> > > EDITED to ask if it's possible to get a compile of 7zip .ZIP deflate
> > modes
> > > ONLY, without any other compression modules?
> >
> > No, not really, not that I know of. You can try AdvanceComp's ADVZIP.EXE
> > ("-z4") to repack .ZIPs in place. It may? also allow creating archives
> from
> > scratch, but I don't know if that's .7z only or what (can't remember).
> >
> > http://advancemame.sourceforge.net/comp-readme.html
> >
> > N.B. Those binaries of his are several years old
>
> Yes, I've used ADVZIP for years, however even its "insane" setting isn't as
> good as 7zip's -mx9 deflate settings for creating optimized .ZIPS. This may
> be due to the old age of the code (2005) I'm using, but see below!
>
> I found AdvanceComp 1.16 beta here:
> http://advancemame.sourceforge.net/beta/
> which is the latest version for DOS I can find.

I wasn't aware of this, but it's probably? minor.

> It doesn't improve on speed
> or compression. If you've got newer code that you can compile (with
> machinetype=i686 and -O2 or -O3 GCC compiler options, I'd love to try them
> out! Thanks for offering.

I don't have any newer code, but I can cross-compile with (newer) G++ 3.4.6. (Didn't he use older 3.2.3 ??) Actually, I've successfully rebuilt it with DJGPP natively before (though needs GNU Configure + assorted tools, somewhat annoying but not really hard, comparatively!!). So I could try that, but for the moment, I haven't (yet). TODO!

EDIT: In my limited experience, G++ 3.x seems to improve on -O3 for common code, but 4.x doesn't. So I don't think -O3 would help there, but who knows, without testing I can't say for sure.

> > I assume this means you used the second one ('\' fixes, aka
> > "9.20.1b-testing")?
>
> Yes, that second one works perfectly for me with -mx8 and -mx9 and .EXE
> filters, your first compile did not work with -mx9 and .EXE filters. Good
> sleuthing on figuring out that compiler error!

FYI, these "misc" binaries mentioned in this post did not workaround such a thing, so they probably still have that bug. Sorry, but I didn't want to complicate (or confuse) any benchmarking. If you really want, I can recompile again with same switches and have them all "fixed" (just use "-Os" for the BCJ*.cpp files). But honestly, I think random benchmarking non-.EXE files should be sufficient for now.

It's an interesting idea to compile for various machines, but if it doesn't show much benefit, there isn't much point. So I'd rather target only one or two common ones, if possible (so as not to be too confusing). BUT ... we shall see. ;-)

Zyzzle

26.08.2012, 04:09

@ Rugxulo

p7zip 9.20.1b "testing"

Post reply

> > If you've got the time, it would be interesting to test how the size and
> > speed with -Os compares.
>
> http://ftp.ibiblio.org/pub/micro/pc-stuff/freedos/files/util/file/7zip/9.20.1/p7z9201b-misc.zip
> (2.1 MB)
>
> > Archive: p7z9201b-misc.zip > Zip file size: 2206984 bytes, number of entries: 5 > -rwxa-- 6.3 fat 195244 b- defN 12-Aug-24 18:26 7zcon.exe > -rwxa-- 6.3 fat 553556 b- defN 12-Aug-24 18:24 o3-i686.exe > -rwxa-- 6.3 fat 600620 b- defN 12-Aug-24 18:24 o3i486fp.exe > -rwxa-- 6.3 fat 414704 b- defN 12-Aug-24 18:24 os-i386.exe > -rwxa-- 6.3 fat 451168 b- defN 12-Aug-24 18:24 osi486fp.exe > 5 files, 2215292 bytes uncompressed, 2206472 bytes compressed: 0.4% >
>
> N.B. Since I was recompiling anyways, I figured I'd also do the "obvious"
> and make a "small as possible" sfx module. (Why didn't I think of that
> before?) Still too big, honestly, but whatever, I guess it supports lots of
> things that 7zdecode doesn't.

The completely surprising result is that of all those misc versions, the osi486fp.exe works best and fastest on my Atom netbook. it is quite a bit faster than any other version of 7zip posted on this forum so far, by around 3% compression and 10-15% in decompression. Also, that version DOES work fine with .exe filters and -mx8 and -mx9. For my i7, this optimized for size i486 version actually compresses 0.5 to 1% faster than your other versions, and decompresses faster, around 5%. Of course decompression speed of your optimized 7zdec.exe 9.12 beats all these versions by around 50% on my netbook (yes, it decompresses the same archive in 14 sec that osi486fp.exe decompresses in 28 seconds, and o3i686.exe take 33 seconds to extract. This is just one archive, others have tested the same.

Deflate is also improved most by osi486fp.exe in -mx9 mode around 3-5% and in other modes around 7-10%! Wonder why?? This on both my systems. I'll also test on a Core2Duo E8400 chip...

As expected, os-i386.exe is slowest by around 15% on both my systems.

> FYI, these "misc" binaries mentioned in this post did not workaround such a
> thing, so they probably still have that bug. Sorry, but I didn't want to
> complicate (or confuse) any benchmarking. If you really want, I can
> recompile again with same switches and have them all "fixed" (just use
> "-Os" for the BCJ*.cpp files). But honestly, I think random benchmarking
> non-.EXE files should be sufficient for now.

But in my testing osi486.exe works perfectly on all settings, and o3-i686.exe fails on mx8 and -mx9 and exe filters. Very strange.

> It's an interesting idea to compile for various machines, but if it doesn't
> show much benefit, there isn't much point. So I'd rather target only one or
> two common ones, if possible (so as not to be too confusing). BUT ... we
> shall see. ;-)

There seems to be quite a spread over the different compiles of 7zip! More than just 1 or 2% that you might suspect.

Also, there was a vastly increased speed of your AdvanceZIP recompile. For -z -4 method, a typical decrease in time was from 60.4 sec using the 2005 compile to 41.7 sec using your compile. Bravo! What a great difference. This speed optimization holds for both my i7 and Atom systems. Thanks for making these compiles available. There was only around 2% difference in the generic and i686 and Atom versions, however. Times for that same archive ranged from 41.7 sec (fastest = generic) to 42.5 sec (slowest = i686). Wonder why with 7zip there is more variance.

By the way, all my tests were done on a RAMdrive to minimize variation in HD speed affecting the results.

Rugxulo

Usono,
26.08.2012, 04:59

@ Zyzzle

p7zip 9.20.1b "testing"

Post reply

> The completely surprising result is that of all those misc versions, the
> osi486fp.exe works best and fastest on my Atom netbook.

That's not that surprising. 486 optimizations were often referred to in talk of Atom. Though I probably should've compiled one with and one without -fomit-frame-pointer (as it bloats up the code and I have no idea if it helps or hurts here).

> it is quite a bit
> faster than any other version of 7zip posted on this forum so far, by
> around 3% compression and 10-15% in decompression. Also, that version DOES
> work fine with .exe filters and -mx8 and -mx9.

-O2 and -O3 are showing the problem, but -Os isn't. Hence why I rebuilt in 9.20.1b with -Os for the BCJ*.cpp files (only) while keeping -O2 for everything else. (I even changed the makefile default to -Os to be ultra safe so no one else would be confronted by the bug, though I hate doing that. I should've commented why!)

> For my i7, this optimized
> for size i486 version actually compresses 0.5 to 1% faster than your other
> versions, and decompresses faster, around 5%. Of course decompression speed
> of your optimized 7zdec.exe 9.12 beats all these versions by around 50% on
> my netbook (yes, it decompresses the same archive in 14 sec that
> osi486fp.exe decompresses in 28 seconds, and o3i686.exe take 33 seconds to
> extract. This is just one archive, others have tested the same.

9.12 was the one I compiled with a bunch of compilers. 9.22 I only built via DJGPP with FOSS-friendly everything (i.e. no D3X). It's on iBiblio explicitly meant for FreeDOS purists. I can't remember what changed, if anything. (PPMD??)

> Deflate is also improved most by osi486fp.exe in -mx9 mode around 3-5% and
> in other modes around 7-10%! Wonder why?? This on both my systems. I'll
> also test on a Core2Duo E8400 chip...
>
> As expected, os-i386.exe is slowest by around 15% on both my systems.

Very strange. The 486 was very sensitive to alignment, way moreso than the Pentium, hence "-march=i386" and "-march=i486" only differ (last I checked) in that one detail! I didn't know newer cpus reverted back to being so sensitive, seems like a bad decision, oh well.

It's almost ridiculous how much difference a bunch of optimizations can make in binary size!

> > FYI, these "misc" binaries mentioned in this post did not workaround such
> a
> > thing, so they probably still have that bug.
>
> But in my testing osi486.exe works perfectly on all settings, and
> o3-i686.exe fails on mx8 and -mx9 and exe filters. Very strange.

Yes, only -O2 and -O3 show this problem. I wasn't sure if these new attempts would even show the bug, much less present it correctly in benchmarking, so I didn't fiddle with anything.

> There seems to be quite a spread over the different compiles of 7zip! More
> than just 1 or 2% that you might suspect.

Ideally, there wouldn't be 50% binary size increase just to speed up 5%. But speed is probably more important than size (though I still say you can have both, but apparently GCC doesn't care).

> Also, there was a vastly increased speed of your AdvanceZIP recompile. For
> -z -4 method, a typical decrease in time was from 60.4 sec using the 2005
> compile to 41.7 sec using your compile. Bravo! What a great difference.

I'm honestly (somewhat) surprised that newer is better. It isn't always the case, but luckily it is here. I guess their testing has improved (or maybe the new backend is just that much better designed).

But yes, I had tried this years ago, so I knew it was actually faster in this case. (I remember testing on my old P166, heh.)

> This speed optimization holds for both my i7 and Atom systems. Thanks for
> making these compiles available.

Yes, I tested it later (on the huge gcc463s.zip file), and it was 20% faster for me with "advzip -z4" (Core i5), something like 4 mins. vs 5 mins.

> There was only around 2% difference in the
> generic and i686 and Atom versions, however. Times for that same archive
> ranged from 41.7 sec (fastest = generic) to 42.5 sec (slowest = i686).
> Wonder why with 7zip there is more variance.

Dunno, I'd be surprised if "generic" included Atom-friendly stuff, but you never know. But I did notice they were almost the same size, which for GCC usually means not much difference.

> By the way, all my tests were done on a RAMdrive to minimize variation in
> HD speed affecting the results.

Yes, that's fine, me too. :-)

Rugxulo

Usono,
31.08.2012, 23:05
(edited by Rugxulo, 31.08.2012, 23:55)

@ Zyzzle

p7zip 9.20.1b "testing"

Post reply

Well, I moved all the .ZIPs to a /testing/ subdir since I'm not really totally satisfied yet. (N.B. This means all above links are broken, sorry, but it's too raw to rely on linking to them permanently just yet, IMHO.)

http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/util/file/7zip/9.20.1/testing/

I did, however, successfully recompile with latest GCC 4.7.1 (DJGPP 2.04, under DOSEMU ... surprisingly worked!), so if you want to see if these run faster, feel free:

471-generic-O2.zip 2012-Aug-31 15:32:54 1.2M application/zip 471-i486fp-Os.zip 2012-Aug-31 15:46:12 957.3K application/zip 471-atom-O2.zip 2012-Aug-31 16:19:34 1.3M application/zip 471-atomfp-Os.zip 2012-Aug-31 16:31:36 974.9K application/zip 471-corei7-O2.zip 2012-Aug-31 16:44:20 1.3M application/zip

(These are not UPX'd, but a final release will probably be. And note that they packed their own .ZIPs and no -mx9 .EXE filter bug seen, so that's good.)

TODO:
* Currently no docs included
* Currently not in FreeDOS 1.1's preferred .ZIP layout
* seems to act case sensitively on all filenames (bad)
* still has tmpfile-assumes-LFN bug
* still needs more testing, not sure which .EXEs are best (though I have my guesses)

So, for now, that's the current status, if anybody is curious. If Khusraw wants to go ahead and tweak it or give advice, he's welcome, but I haven't heard from him (probably busy like most others).

EDIT: Forgot to build some with Atom and Corei7 optimizations (main advantage of newer G++), doing that now. (Done!)

P.S. If you're curious, it took 10 min. 16 sec. for the full Corei7 build (all three .EXEs) under DOSEMU, so yeah, native DJGPP is a bit slower than otherwise (though of course newer G++ affected that too).

Rugxulo

Usono,
02.09.2012, 22:27

@ Rugxulo

p7zip 9.20.1b "testing"

Post reply

> TODO:
> ...

(sigh) A hacker's job is never done. :-P

* BUG: "7za l 471*.zip" doesn't work without explicit quotes around wildcard spec
= solution: (disabling globbing with empty replacement function is needed as p7zip apparently handles it already, see DJGPP FAQ)

* BUG: -p echos your password to console (still? oops, see DOS386's old fix7zip TSR, if you can find it!)
= solution: -DENV_HAVE_GETPASS (but DJGPP's is limited to 8 chars and yet only cmdline switch "-pabcdefghi" fails if password was "abcdefgh")
+ N.B. not an ideal solution, 8 is way too short, needs patch to use DJGPP's getlongpass() instead

* BUG: (worst, IMHO) "7za a -sfx dumbname somedir" doesn't work except on same drive as 7zcon.sfx (similarly "p7zip l c:\tmp\471*.zip" fails from atop G:\ RAM drive)
= solution??: dunno ... something kludged up in argv[] ?? perhaps drive prefix is being deleted already? (needs further study)

All of these are old bugs that even "old stable" Khusraw's 9.13 had, so I don't guess I should feel too bad. Yeah, ugh, this util really needed a bit of cleaning up for us. Hopefully I can fix it. At worst, we don't have any less than we already had (e.g. older versions, Win32's 7ZA.EXE + HX, Info-Zip, etc).

Zyzzle

04.09.2012, 23:59

@ Rugxulo

p7zip 9.20.1b "testing"

Post reply

Thank you for your continual efforts. The gcc 4.71 builds are a little faster across the board, but a little more bloated also (at least when packed with UPX). The times are best for my Atom with the Atom build (surprise!) and on my i7 with the Generic build (huh?)

On extensive testing your 9.20 builds are inferior in one major way to the old stable 9.13 build. This may be a limitation of the gcc version, or a 2 GB/ 4GB bug, but I can't decompress lzma archvies created with a dictionary of 1024 MB with your 9.20 builds, but they DID decompress just fine (even on my 1 GB Atom) with Krusichav's build of 9.13. Gives a "Can't allocated required memory!" error and bombs out. This error occurs even on my i7 machine which has 32 GB of RAM installed. Some sort of DOS memory allocation error in either DJGPP or gcc?

Rugxulo

Usono,
05.09.2012, 02:12

@ Zyzzle

p7zip 9.20.1b "testing"

Post reply

> Thank you for your continual efforts. The gcc 4.71 builds are a little
> faster across the board, but a little more bloated also (at least when
> packed with UPX).

-Os isn't always smaller after UPX than -O2, strange but true. But again, I don't know what Khusraw's specific compiler and libc and optimization flags were, so I can't honestly compare.

> The times are best for my Atom with the Atom build
> (surprise!) and on my i7 with the Generic build (huh?)

I assume only 1% or 2% difference. They should probably be fairly similar.

> On extensive testing your 9.20 builds are inferior in one major way to the
> old stable 9.13 build. This may be a limitation of the gcc version, or a 2
> GB/ 4GB bug, but I can't decompress lzma archvies created with a dictionary
> of 1024 MB with your 9.20 builds, but they DID decompress just fine (even
> on my 1 GB Atom) with Krusraw's build of 9.13.

Strange that they worked with his build. Anyways, you can try HX + 7ZA or LZMA (utils) or XZ (utils) even to unpack them, I think. Though why you would use 1 GB dictionary is beyond me (is your data at least that big??).

> Gives a "Can't allocated
> required memory!" error and bombs out. This error occurs even on my i7
> machine which has 32 GB of RAM installed. Some sort of DOS memory
> allocation error in either DJGPP or gcc?

Yikes! HPC? Servers? VMs? Games? :-) )

All I can barely guess is it's some limitation or bug. I've encountered some rare errors on my 6 GB machine too. CWSDPMI r7 (via GO32-v2) reports 4 GB free (which is wrong) and thus refuses to swap, assuming it doesn't need to. But it needs to swap for page tables (since so much free RAM is found), which are always in low RAM (CWSDPMI design decision). So some things fail. CWS says DJGPP libc assumes a lot of 2 GB signed stuff, so you may not be able to correctly use more than 2 GB of RAM in most (all?) DJGPP apps except via very very selective use of raw sbrk(), i.e. alloc 1.5 GB x 2. Dunno, never tried that.

I don't remember offhand (would have to check my emails), but eventually CWS just had me binary patch the crt0.S line "shr ecx,8" to use "5" and thus rounds up memory allocations differently, which somehow defeated my particular problem. But I don't know if that'll help you (if only helped me with Seed7 compiled compiler on "chkbig" and "chkset" via "hi chk_all"). You could also try "HDPMI32 -r" first to see if it really is a CWSDPMI r7 bug or not.

Khusraw

Bucharest, Romania,
05.09.2012, 10:30

@ Rugxulo

p7zip 9.20.1b "testing"

Post reply

> -Os isn't always smaller after UPX than -O2, strange but true. But again, I
> don't know what Khusraw's specific compiler and libc and optimization flags
> were, so I can't honestly compare.

It was built with gcc 4.4.2 and libc from djgpp 2.04 beta, -O2 optimization flag.

---
Glory to God for all things

Rugxulo

Usono,
06.09.2012, 23:21

@ Rugxulo

p7zip 9.20.1b "testing"

Post reply

> > TODO:
> > ...
>
> = nit #1). Updating still tries using LFN-only blah.7z.tmp, which is annoying in SFN.
> I'm too tired right now to fix that (not that I know where it is
> exactly either). Workaround: "ren blah.7z blah", "7za d -r blah.
> sources\*.c", "ren blah *.7z".

Actually, it seems to assume .7z type by default, so if you want to "update" ('u' command) or "delete" ('d' command) in a .ZIP or .TAR, you have to also use "-ttar" or "-tzip", then it'll work for them too (as mentioned above, in SFN).

> * BUG: (worst, IMHO) "7za a -sfx dumbname somedir" doesn't work except on
> same drive as 7zcon.sfx (similarly "p7zip l c:\tmp\471*.zip" fails from
> atop G:\ RAM drive)
> = solution??: dunno ... something kludged up in argv[] ?? perhaps drive
> prefix is being deleted already? (needs further study)

Actually, when curdir is on C:, it works okay doing "7za l g:\somedir\myfile.tbz". It's just when sitting atop G: that it for some reason doesn't work reading from C:. (Verified in Khusraw's also.) Apparently a good workaround is "7za l c:c:\somedir\blah.tbz". So I guess the POSIX port is removing the drive name (or maybe only when prefix is "c:") explicitly somewhere.

I was also worried that it wouldn't work without a C:\TMP directory, but renaming mine didn't show any obvious errors in light testing. There is at least one place where it's hardcoded, but I didn't look too closely, so I didn't mess with it (yet).

Just FYI.

favero 21.09.2012, 20:04 @ Rugxulo	p7zip 9.20.1b "testing" Post reply
	Hi Rugxulo, I'm learning how to use 7Zip, and ran into this difficulty: if paths are saved, the 7z file cannot be decompressed. I work under pure FreeDos. I'm using 7ZA.EXE from 2009-01-15, because the more recent one from 2012-08-20 doesn't seem to work at all for me. Regards, Marcos Favero Campinas, Brazil

Rugxulo

Usono,
22.09.2012, 06:16

@ favero

p7zip 9.20.1b "testing"

Post reply

> I'm learning how to use 7Zip, and ran into this difficulty: if paths are
> saved, the 7z file cannot be decompressed.

You mean with or without LFNs enabled? I did notice that it doesn't like truncating long file names (like Info-Zip does, thankfully). Not sure if that's really a bug, just a minor inconvenience.

> I work under pure FreeDos. I'm using 7ZA.EXE from 2009-01-15, because the
> more recent one from 2012-08-20 doesn't seem to work at all for me.

I've not had lots of motivation to mess with it too much lately. Always distracted by other things. :-) I tried to (very roughly) mention certain flaws here as a kinda raw TODO list, but it will probably take a while before I polish it off.

In any case, 2012-08-20 was the very first initial compile, the one with *nix-only slashes, hence I would suggest testing p7z9201-latest.zip (symlink to latest, naturally), which is currently from 2012-09-03. It does unpack subdirs correctly for me (when using 'x' command). But I don't claim it's totally stable or perfect (by far) yet.

P.S. Yes, do feel free to use 7ZA.EXE + HX, nothing wrong with that, I've got that installed too. But I've never (and don't want to!) tried to recompile that Win32 console version. Perhaps Japheth knows a semi-painless way, but I doubt OpenWatcom is up to any of it (without lots of source tweaks). :no:

P.P.S. If you just want to unpack, use 7zdecode (e.g. from LZMA SDK).

Khusraw

Bucharest, Romania,
23.09.2012, 19:44

@ Rugxulo

p7zip 9.20.1b "testing"

Post reply

> P.S. Yes, do feel free to use 7ZA.EXE + HX, nothing wrong with that, I've
> got that installed too. But I've never (and don't want to!) tried to
> recompile that Win32 console version. Perhaps Japheth knows a semi-painless
> way, but I doubt OpenWatcom is up to any of it (without lots of source
> tweaks). :no:

I re-built myself 7za 9.20, which can be downloaded from here. It is a DOS executable, but it needs most dlls from the standard HX DOS-Extender runtime package (HXRT217.ZIP).

---
Glory to God for all things

Rugxulo

Usono,
11.10.2012, 21:56

@ Khusraw

7ZA 9.20 rebuilt by Khusraw

Post reply

> I re-built myself 7za 9.20, which can be downloaded from
> here. It is a DOS executable, but it
> needs most dlls from the standard HX DOS-Extender runtime package
> (HXRT217.ZIP).

(Sorry for late reply.)

Were we supposed to test this before you gave us more info? Did you use OpenWatcom? What flags? Any advantages? I mean, it's cool that you rebuilt it, but you could give us a few more details. :-)

Khusraw

Bucharest, Romania,
12.10.2012, 16:30

@ Rugxulo

7ZA 9.20 rebuilt by Khusraw

Post reply

> Were we supposed to test this before you gave us more info? Did you use
> OpenWatcom? What flags? Any advantages? I mean, it's cool that you rebuilt
> it, but you could give us a few more details. :-)

No, you were not supposed to test it. I built it for those interested to make speed etc. comparisons between various 7-zip archivers available for DOS. I didn't change anything in the makefiles, so you can download the source code package and find there all the details you need. I also built 7.22, but I see nothing special in it.

---
Glory to God for all things

Rugxulo

Usono,
13.10.2012, 05:35

@ Khusraw

7ZA 9.20 rebuilt by Khusraw

Post reply

> No, you were not supposed to test it. I built it for those interested to
> make speed etc. comparisons between various 7-zip archivers available for
> DOS.

DJGPP usually outperforms OpenWatcom, but who knows here, maybe Japheth's fine-tuned assembly HX innards will kick its rear. (But that silly HX regression with numbers makes "7za b" kinda useless, at least for strict comparisons.)

I haven't bothered testing lately, but I suppose one day I could try unpacking a really big file (e.g. GCC463S.ZIP) with both. I'm not sure how else to reliably test. Probably plenty fast enough as long as no blatant errors are present.

> I didn't change anything in the makefiles, so you can download the
> source code package and find there all the details you need.

So it builds "as is" with OpenWatcom 1.9? That's surprising and yet good to know.

> I also built 7.22, but I see nothing special in it.

I've not kept up. Everything "mostly" works fine for my limited uses. I don't archive anything heavy-duty and am just a lowly hobbyist.

A quick check shows that 9.20 ("stable"??) is almost two years old. 9.22 "beta" comes a few months later, then 9.25 "alpha" a few more months later. Then 9.29 "alpha" apparently just came out last month, but it's so fresh that no sources are available just yet. (p7zip is still sticking to 9.20.1 for now, apparently.)

Nothing majorly different for our use cases, I suppose. He does now switch to LZMA2 by default and adds a few other commands and options, e.g. "rn", "-sdel", "-stl".

Khusraw

Bucharest, Romania,
13.10.2012, 09:06

@ Rugxulo

7ZA 9.20 rebuilt by Khusraw

Post reply

> So it builds "as is" with OpenWatcom 1.9? That's surprising and yet good to
> know.

It is not built with OpenWatcom 1.9, it is built with MVCT 2003!

---
Glory to God for all things

Rugxulo

Usono,
25.08.2012, 02:33

@ Zyzzle

AdvanceComp 1.15 -- rebuilt with newer compiler

Post reply

> I found AdvanceComp 1.16 beta here:
> http://advancemame.sourceforge.net/beta/
> which is the latest version for DOS I can find. It doesn't improve on speed
> or compression. If you've got newer code that you can compile (with
> machinetype=i686 and -O2 or -O3 GCC compiler options, I'd love to try them
> out! Thanks for offering.

This is only "stable" (eh?) AdvanceComp 1.15, but here it is anyways:

http://www.ibiblio.org/pub/micro/pc-stuff/freedos/...s/util/file/7zip/advancecomp/471-o2-generic.zip (mtune)
http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/util/file/7zip/advancecomp/471-o2-atom.zip (mtune)
http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/util/file/7zip/advancecomp/471-o2-corei7.zip (march!)

N.B. I didn't UPX these, for whatever reason, nor benchmark them myself. I already know from experience that it will run faster than ye olde 2005 compiles. Other than that, who knows.

roytam

20.03.2013, 07:58

@ Rugxulo

AdvanceComp 1.15 -- rebuilt with newer compiler

Post reply

> > I found AdvanceComp 1.16 beta here:
> > http://advancemame.sourceforge.net/beta/
> > which is the latest version for DOS I can find. It doesn't improve on
> speed
> > or compression. If you've got newer code that you can compile (with
> > machinetype=i686 and -O2 or -O3 GCC compiler options, I'd love to try
> them
> > out! Thanks for offering.
>
> This is only "stable" (eh?) AdvanceComp 1.15, but here it is anyways:
>
> http://www.ibiblio.org/pub/micro/pc-stuff/freedos/...s/util/file/7zip/advancecomp/471-o2-generic.zip
> (mtune)
> http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/util/file/7zip/advancecomp/471-o2-atom.zip
> (mtune)
> http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/util/file/7zip/advancecomp/471-o2-corei7.zip
> (march!)
>
> N.B. I didn't UPX these, for whatever reason, nor benchmark them myself. I
> already know from experience that it will run faster than ye olde 2005
> compiles. Other than that, who knows.

I replaced whole 7-zip with Google's Zopfli DEFLATE-compatible library, and compiled with DJGPP gcc/g++ 4.7.2 (general -O2 build)
http://encode.ru/threads/1691-AdvanceCOMP-with-new-7-zip?p=32687#post32687

Rugxulo

Usono,
21.03.2013, 18:03

@ roytam

AdvanceComp 1.15 -- rebuilt with newer compiler

Post reply

> > > I found AdvanceComp 1.16 beta here:
> > > http://advancemame.sourceforge.net/beta/
> > > which is the latest version for DOS I can find. It doesn't improve on
> > > speed or compression.

I noticed 1.16 final was released earlier this month. I've mirrored that to iBiblio, but his website says he still uses (old) GCC 3.2.3 cross-compiler (2003 ftw!). :no:

In other words, I haven't rebuilt it with newer DJGPP yet, but it shouldn't be hard. Suggestions for me for -mtune -march anyone?? :-)

> I replaced whole 7-zip with Google's Zopfli DEFLATE-compatible library, and
> compiled with DJGPP gcc/g++ 4.7.2 (general -O2 build)
> http://encode.ru/threads/1691-AdvanceCOMP-with-new-7-zip?p=32687#post32687

Does this really help much? Is it worth it? What about comparisons to Kzip or Deflopt? A few example stats would be greatly appreciated. (Or maybe I should just try it myself, ugh.) I'm just slightly skeptical that it will improve much over 7-Zip's Deflate. I mean, if we really wanted better compression, getting an extra few kilobytes from Deflate isn't the right approach, better would be to use a different method (Bzip2? LZMA?) or archiver or ....

And yes, I know, in rare cases AdvanceZip can chomp off a fairly big chunk, e.g. FPC's .ZIP, lemme try old 1.15 on latest 2.6.2 full ... okay, it saved very very little. :crying: Lemme try after unpacking the main .ZIP and trying on all the little .ZIPs separately ... saved almost 3.5 MB! :-P Now testing full's .ZIPs with new 1.16 ... saved only the same amount. :-|

BTW, GCC 4.8.0 is almost finalized, probably next week (and Andris made a RC1 build via DJGPP 2.04 here). Not totally sure if it will improve much, but it should be interesting (if you care to benchmark). :hungry:

roytam

22.03.2013, 01:03

@ Rugxulo

AdvanceComp 1.15 -- rebuilt with newer compiler

Post reply

> > > > I found AdvanceComp 1.16 beta here:
> > > > http://advancemame.sourceforge.net/beta/
> > > > which is the latest version for DOS I can find. It doesn't improve
> on
> > > > speed or compression.
>
> I noticed 1.16 final was released earlier this month. I've mirrored that to
> iBiblio, but his website says he still uses (old) GCC 3.2.3 cross-compiler
> (2003 ftw!). :no:
>
> In other words, I haven't rebuilt it with newer DJGPP yet, but it shouldn't
> be hard. Suggestions for me for -mtune -march anyone?? :-)
>
> > I replaced whole 7-zip with Google's Zopfli DEFLATE-compatible library,
> and
> > compiled with DJGPP gcc/g++ 4.7.2 (general -O2 build)
> >
> http://encode.ru/threads/1691-AdvanceCOMP-with-new-7-zip?p=32687#post32687
>
> Does this really help much? Is it worth it? What about comparisons to Kzip
> or Deflopt? A few example stats would be greatly appreciated. (Or maybe I
> should just try it myself, ugh.) I'm just slightly skeptical that it will
> improve much over 7-Zip's Deflate. I mean, if we really wanted better
> compression, getting an extra few kilobytes from Deflate isn't the right
> approach, better would be to use a different method (Bzip2? LZMA?) or
> archiver or ....
>

because you can't change the file spec which uses deflate/zlib algorithm (for example, PNG images)

> And yes, I know, in rare cases AdvanceZip can chomp off a fairly big chunk,
> e.g. FPC's .ZIP, lemme try old 1.15 on latest 2.6.2 full ... okay, it saved
> very very little. :crying: Lemme try after unpacking the main .ZIP and
> trying on all the little .ZIPs separately ... saved almost 3.5 MB! :-P Now
> testing full's .ZIPs with new 1.16 ... saved only the same amount. :-|
>

because he didn't upgrade the ancient 7-zip to latest version. (I did it, but not all files compress better with new 7-zip deflate algorithm)

> BTW, GCC 4.8.0 is almost finalized, probably next week (and Andris made a
> RC1 build via DJGPP 2.04
> here). Not
> totally sure if it will improve much, but it should be interesting (if you
> care to benchmark). :hungry:

oh great! :-D

Rugxulo Usono, 28.03.2013, 00:24 @ roytam	GCC 4.8.0 (DJGPP 2.04 only) Post reply
	> > BTW, GCC 4.8.0 is almost finalized, probably next week (and Andris made > a > > RC1 build via DJGPP 2.04 > > here). Not > > totally sure if it will improve much, but it should be interesting (if > you > > care to benchmark). > > oh great! It (GCC 4.8.0) has been released, but for unknown reasons, it's only for DJGPP 2.04. See announcement or mirror.

p7zip 9.20.1 (Users)

p7zip 9.20.1

p7zip 9.20.1b "testing"

p7zip 9.20.1b "testing"

p7zip 9.20.1b "testing"

p7zip 9.20.1b "testing"

p7zip 9.20.1b "testing"

p7zip 9.20.1b "testing"

p7zip 9.20.1b "testing"

p7zip 9.20.1b "testing"

p7zip 9.20.1b "testing"

p7zip 9.20.1b "testing"

p7zip 9.20.1b "testing"

p7zip 9.20.1b "testing"

p7zip 9.20.1b "testing"

p7zip 9.20.1b "testing"

p7zip 9.20.1b "testing"

p7zip 9.20.1b "testing"

7ZA 9.20 rebuilt by Khusraw

7ZA 9.20 rebuilt by Khusraw

7ZA 9.20 rebuilt by Khusraw

7ZA 9.20 rebuilt by Khusraw

AdvanceComp 1.15 -- rebuilt with newer compiler

AdvanceComp 1.15 -- rebuilt with newer compiler

AdvanceComp 1.15 -- rebuilt with newer compiler

AdvanceComp 1.15 -- rebuilt with newer compiler

GCC 4.8.0 (DJGPP 2.04 only)