Back to home page

DOS ain't dead

Forum index page

Log in | Register

Back to index page
Thread view  Board view
rr(R)

Homepage E-mail

Berlin, Germany,
12.08.2012, 23:02
 

Testers needed (benchmarking 'b64') (Miscellaneous)

Hi!

As you have probably already guessed from my other posts I'm clearing my HDD. :-P

In early 2007 I had built Bob Trower's small b64 using many different DOS compilers (free, FOSS, commercial). Back then the Digital Mars C/C++ build was as twice as fast as the others on real DOS hardware.
Nowadays I still like Digital Mars C/C++, but also LSI C-86 and Open Watcom C/C++. But currently I don't have access to any real DOS machine and my results on Windows are vague.
That's why I've prepared b64_test.zip containing b64_dm.exe, b64_lsi.exe, and b64_ow.exe. I would like to ask you to report, which of these three builds is the fastest for you in encoding and decoding (on real DOS hardware). Thanks in advance! :flower:

Btw: Original b64 version 0.12R doesn't decode correctly, but this has been fixed in my builds. I've already contacted Bob about that.

Rugxulo(R)

Homepage

Usono,
13.08.2012, 08:36
(edited by Rugxulo, 13.08.2012, 08:49)

@ rr
 

Testers needed (benchmarking 'b64')

All benchmarks tend to be useless or skewed, but they're still better than nothing if you want to compare things. Just be aware than there are ten bazillion factors as to what is relevant or can affect speeds. (I've done some very amateurish benchmarking lately, e.g. very old DJGPP GCCs.)

Well, whatever, lemme test 'em here on this one box, but sadly I can't test older cpus as most of mine died. I do wonder what memory model and base cpu (8086?) you used for these. Did you turn on optimal alignment? Profiling? Max speed or size optimizations?

/me reboots


C:\TMP\B64>runtime b64_lsi -e douay.txt lsi.out
05.33 seconds elapsed

C:\TMP\B64>runtime b64_ow -e douay.txt ow.out
09.34 seconds elapsed

C:\TMP\B64>runtime b64_dm -e douay.txt dm.out
00.77 seconds elapsed

C:\TMP\B64>runtime b64_lsi -d lsi.out lsi.txt
04.78 seconds elapsed

C:\TMP\B64>runtime b64_dm -d dm.out dm.txt
00.88 seconds elapsed

C:\TMP\B64>runtime b64_ow -d ow.out ow.txt
08.41 seconds elapsed

CPU brand is GenuineIntel / CPU is family 6 model 5 revision 2
This CPU calls itself: [Intel(R) Core(TM) i5 CPU         650  @ 3.20GHz_]

rr(R)

Homepage E-mail

Berlin, Germany,
13.08.2012, 20:05

@ Rugxulo
 

Testers needed (benchmarking 'b64')

> Well, whatever, lemme test 'em here on this one box, but sadly I can't test
> older cpus as most of mine died. I do wonder what memory model and base cpu
> (8086?) you used for these. Did you turn on optimal alignment? Profiling?
> Max speed or size optimizations?

8086. Small memory model.
- dmc.exe -ms -o b64.c -ob64_dm.exe
- lcc.exe -O b64.c -o b64_lsi.exe -ltinymain.obj -lintlib
- wcl.exe /q /ox b64.c /fe=b64_ow.exe
- upx.exe --best --lzma --all-filters --8086 bw64_*.exe

Btw: What about your (my) JED release? ;-)

Rugxulo(R)

Homepage

Usono,
14.08.2012, 00:31

@ rr
 

Testers needed (benchmarking 'b64')

> 8086. Small memory model.
> - dmc.exe -ms -o b64.c -ob64_dm.exe
> - lcc.exe -O b64.c -o b64_lsi.exe -ltinymain.obj -lintlib
> - wcl.exe /q /ox b64.c /fe=b64_ow.exe
> - upx.exe --best --lzma --all-filters --8086 bw64_*.exe

Okay, but I'm not sure why OW is so slow. Try /otexan /6. Others I have less experience with, so I don't know optimal settings without looking / trying.

> Btw: What about your (my) JED release? ;-)

I haven't been working on it. Should I? Or do you just want an old (or new?) stock build via latest S-lang, DJGPP, etc.?

rr(R)

Homepage E-mail

Berlin, Germany,
14.08.2012, 22:33

@ Rugxulo
 

Testers needed (benchmarking 'b64')

> > 8086. Small memory model.
> > - dmc.exe -ms -o b64.c -ob64_dm.exe
> > - lcc.exe -O b64.c -o b64_lsi.exe -ltinymain.obj -lintlib
> > - wcl.exe /q /ox b64.c /fe=b64_ow.exe
> > - upx.exe --best --lzma --all-filters --8086 bw64_*.exe
>
> Okay, but I'm not sure why OW is so slow. Try /otexan /6. Others I have
> less experience with, so I don't know optimal settings without looking /
> trying.

"-otexan" gives many of results in Google. ;-)
It's slightly faster here on Windows XP.

"-6" (together with "-otexan") speeds up things a lot.
Does a 8086 still run "-6" code?

I've uploaded b64_ow.zip. If anyone could test this, please?! :-)
b64_ow: wcl.exe -q -otexan b64.c & upx.exe --best --lzma --all-filters --8086
b64_ow6: wcl.exe -q -otexan -6 b64.c & upx.exe --best --lzma --all-filters

> > Btw: What about your (my) JED release? ;-)
>
> I haven't been working on it. Should I? Or do you just want an old (or
> new?) stock build via latest S-lang, DJGPP, etc.?

I just noticed 2007 or 2010 "JED" in my "Projects" folder. You asked me once to upload this stuff so you could grab and release it. Meanwhile you built 0.99-19, but didn't release a binary, which is little useless for DOS users. Therefore I'd like to get JED for DOS released some day.

Rugxulo(R)

Homepage

Usono,
14.08.2012, 23:32

@ rr
 

Testers needed (benchmarking 'b64')

> > Okay, but I'm not sure why OW is so slow. Try /otexan /6. Others I have
> > less experience with, so I don't know optimal settings without looking /
> > trying.
>
> "-otexan" gives many of results in Google. ;-)

Yes, it's a silly mnemonic to remember, at least easier than what -ox really means (IIRC, -obmiler -s)!

> It's slightly faster here on Windows XP.

I'm very surprised (though why would I be?) at the speed differences. OW shouldn't be this slow, and DM shouldn't be this fast. It's crazy.

> "-6" (together with "-otexan") speeds up things a lot.
> Does a 8086 still run "-6" code?

No, I don't think so, I'm fairly sure I once saw it generate (minimally) 186 instructions (e.g. push imm16), but (blindly guessing) probably not much more than that. And it's still only 16-bit code, AFAIK, since "wcl" only does 16-bit while wcl386 only does 32-bit. I think wcl386 /6 still runs on actual "386", though, as I remember reading that somewhere, but who knows if that changed in later versions (doubt it, but ...).

So, honestly, /1 would maybe give the same results as /6 here. It was probably a mistake to suggest /6 to you. (And yes, I realize you want 8086 compatibility, my bad, just curious if it would help any.)

> I've uploaded
> b64_ow.zip. If anyone
> could test this, please?! :-)

I'll test again on this Core i5, but I feel like a chump testing such a new machine. It's the old cpus that need to worry about speed. Get whats-his-face to test on his HP200LX. ;-)

> > > Btw: What about your (my) JED release? ;-)
> >
> > I haven't been working on it. Should I? Or do you just want an old (or
> > new?) stock build via latest S-lang, DJGPP, etc.?
>
> I just noticed 2007 or 2010 "JED" in my "Projects" folder. You asked me
> once to upload this stuff so you could grab and release it.

Yes, I know, I remember that, but that was on my old (now dead) laptop, so I'm fairly (?) certain I don't have that file anymore. IIRC, the main problem (later "fixed" upstream??) was S-Lang incompatibility, believe it or not.

> Meanwhile you
> built 0.99-19, but didn't release a binary, which is little useless for DOS
> users. Therefore I'd like to get JED for DOS released some day.

I need to try rebuilding it again. My last efforts so far were mentioned here on this board, rebuilding it with S-Lang 2.2.3, see this .BAT.

IIRC, one of the nicer things about JED was its mouse support (despite the fact that I personally prefer keyboard). At one time I was interested in comparing all the various DOS editors (with bias towards free/libre), but I got distracted by other things so haven't kept up (not that I've missed much, heh).

Rugxulo(R)

Homepage

Usono,
14.08.2012, 23:53

@ Rugxulo
 

Testers needed (benchmarking 'b64')

> I'll test again on this Core i5

Clean boot (F5), nothing loaded. Tried weakly to avoid the L1/L2/L3 caches by running twice, but it didn't really matter. I expect other cpus will be more responsive than mine.

C:\TMP>runtime b64_ow -e douay.txt one.out
08.74 seconds elapsed
C:\TMP>runtime b64_ow6 -e douay.txt two.out
09.01 seconds elapsed
C:\TMP>runtime b64_ow -e douay.txt one1.out
09.29 seconds elapsed
C:\TMP>runtime b64_ow6 -e douay.txt two2.out
09.89 seconds elapsed
C:\TMP>runtime b64_ow -d one.out one.txt
08.41 seconds elapsed
C:\TMP>runtime b64_ow6 -d two.out two.txt
08.08 seconds elapsed
C:\TMP>runtime b64_ow -d one1.out one1.txt
08.02 seconds elapsed
C:\TMP>runtime b64_ow6 -d two2.out two2.txt
08.02 seconds elapsed

RayeR(R)

Homepage

CZ,
16.08.2012, 00:13
(edited by RayeR, 16.08.2012, 00:39)

@ Rugxulo
 

Testers needed (benchmarking 'b64')

Well I tested it on HP 200LX (186 CPU):
encode io.sys file: 222670B

b64_dm -e : 47,57s
b64_lsi -e : 60,80s
b64_ow -e : 54,16s
b64_ow6 -e : still running (over 20 min), probably hang...
EDIT after reboot I saw it even didn't created the output file so definitely hang at some beginning stage.

So Digital Mars wins.

---
DOS gives me freedom to unlimited HW access.

Rugxulo(R)

Homepage

Usono,
16.08.2012, 16:47

@ RayeR
 

Testers needed (benchmarking 'b64')

> Well I tested it on HP 200LX (186 CPU):
>
> So Digital Mars wins.

I find this quite fascinating. It's so unusual the way various x86 cpus handle things.

BTW, Eric Auer has a tweaked version of a small Mime encoder/decoder (in .C) on his site (mirror), if anyone wants to test it too:

http://ericauer.cosmodata.virtuaserver.com.br/soft/by-others/base64-encode-decode-tuned-mime64e.zip

He's also written a quick C++ hack version, but it's not ready for primetime just yet (basically "works", but ...).

Rugxulo(R)

Homepage

Usono,
19.08.2012, 03:25

@ Rugxulo
 

Testers needed (benchmarking 'b64')

> Okay, but I'm not sure why OW is so slow.

I could be wrong, but I think it's because of BUFSIZ being default of only 512 bytes for 16-bit code.

When I added two calls to setvbuf() using two char [16000] arrays, one for input and one for output, then it sped up. And I compiled with "wcl /otexan". Feel free to benchmark this too, if curious.


--- b64.c       2012-08-13 07:45:52 +0000
+++ b64_me.c    2012-08-18 20:17:32 +0000
@@ -215,6 +215,8 @@
 #include <stdio.h>
 #include <stdlib.h>
 
+static char my_inbuf[16000], my_outbuf[16000];
+
 /*
 ** Translation Table as described in RFC1113
 */
@@ -400,6 +402,7 @@
     }
     else {
         infile = fopen( infilename, "rb" );
+        setvbuf(infile,my_inbuf,_IOFBF,sizeof my_inbuf);
     }
     if( !infile ) {
         perror( infilename );
@@ -411,6 +414,7 @@
         }
         else {
             outfile = fopen( outfilename, "wb" );
+            setvbuf(outfile,my_outbuf,_IOFBF,sizeof my_outbuf);
         }
         if( !outfile ) {
             perror( outfilename );


(clean boot)
C:\TMP\BLAH>runtime b64_me -e douay.txt me.out
03.41 seconds elapsed
C:\TMP\BLAH>runtime b64_ow -e douay.txt rr.out
10.77 seconds elapsed

Japheth(R)

Homepage

Germany (South),
13.08.2012, 12:31

@ rr
 

Testers needed (benchmarking 'b64')


           start       stop          sec
------------------------------------------
b64_dm -e  12:25:02-10 12:25:03-63   1.53
b64_dm -d  12:25:03-63 12:25:06-00   2.37

b64_ow -e  12:25:06-05 12:25:12-70   6.65
b64_ow -d  12:25:12-70 12:25:15-50   2.80

b64_lsi -e 12:25:15-50 12:25:17-48   1.98
b64_lsi -d 12:25:17-48 12:25:19-18   1.70


on an Athlon 64 X2, 4200+
size of test file ~ 6 MB

btw, the result isn't convincing, because the encode/decode process has added a lot of CRs.

---
MS-DOS forever!

rr(R)

Homepage E-mail

Berlin, Germany,
13.08.2012, 19:05

@ Japheth
 

Testers needed (benchmarking 'b64')

> btw, the result isn't convincing, because the encode/decode process has
> added a lot of CRs.

That sounds strange. Doing b64 -e b64.exe b64.b64 + b64 -d b64.b64 b64.ex2 should produce an identical copy of b64.exe in b64.ex2.

Japheth(R)

Homepage

Germany (South),
13.08.2012, 20:56

@ rr
 

Testers needed (benchmarking 'b64')

> That sounds strange. Doing b64 -e b64.exe b64.b64 + b64
> -d b64.b64 b64.ex2 should produce an identical copy of b64.exe in
> b64.ex2.

I used the "redirection method" for my tests, that is,

xxxx -e in_file >out_file

Later I found the "Note:" that it "should" be used only on text files. However, my test file WAS a text file.

Without redirection it works as expected.

---
MS-DOS forever!

Khusraw(R)

E-mail

Bucharest, Romania,
13.08.2012, 15:17
(edited by Khusraw, 13.08.2012, 21:10)

@ rr
 

Testers needed (benchmarking 'b64')

My results with an ~18MB text file on a CoreTM2 Duo T5470 1.60GHz system:

b64_dm -e  07.97 sec
b64_dm -d  03.03 sec

b64_lsi -e 40.02 sec
b64_lsi -d 22.04 sec

b64_ow -e  83.16 sec
b64_ow -d  43.21 sec

Doug(R)

E-mail

13.08.2012, 21:53

@ rr
 

Testers needed (benchmarking 'b64')

Input file: 30mb binary, no disk cache, output file erased after each test, timing granularity 0.01 sec, Intel Pentium 4, 2.4ghz:

               RAM DISK              ATA100 HARD DRIVE
               --------------------  --------------------
  B64_DM -E:   01.65 sec  <-fastest  03.84 sec  <-fastest
  B64_LSI -E:  02.50 sec  <-slowest  11.94 sec
  B64_OW -E:   01.90 sec             15.37 sec  <-slowest

  B64_DM -D:   02.57 sec  <-slowest  04.62 sec  <-fastest
  B64_LSI -D:  02.41 sec             11.52 sec
  B64_OW -D:   02.00 sec  <-fastest  12.06 sec  <-slowest

Results were double (actually *triple*) checked.

And note: The encoded/decoded output files were all identical with one another between the three B64 versions, but when decoded, it was *NOT* identical to the original binary! The test file i used was the MAIN.CVD data file from Clam Antivirus (the biggest file i had on my hard drive):

  MAIN.CVD  30,750,647  (original)
  MAIN.DEC  30,620,636  (after encode & decode)

- Doug B.

rr(R)

Homepage E-mail

Berlin, Germany,
13.08.2012, 22:41

@ Doug
 

Testers needed (benchmarking 'b64')

> And note: The encoded/decoded output files were all identical with one
> another between the three B64 versions, but when decoded, it was *NOT*
> identical to the original binary! The test file i used was the MAIN.CVD
> data file from Clam Antivirus (the biggest file i had on my hard drive):
>
> MAIN.CVD  30,750,647  (original)
> MAIN.DEC  30,620,636  (after encode & decode)
>


Good catch! :-) I can reproduce that here.

It's probably time to revert to version 0.06R completely, which operates fine on main.cvd, although Bob writes: Only cosmetic changes have taken place since its initial release in 2001. This is now 'production stable', even by the author's standards, as evidenced by the removal of the beta tag :)

glennmcc(R)

Homepage E-mail

North Jackson, Ohio (USA),
14.08.2012, 02:05

@ rr
 

Testers needed (benchmarking 'b64')

FYI,

This is the B64 encoder/decoder that I've been using for many years now.

http://archives.scovetta.com/pub/simtelnet/msdos/decode/mime64b.zip

rr(R)

Homepage E-mail

Berlin, Germany,
21.08.2012, 11:46

@ rr
 

Testers needed (benchmarking 'b64')

I stopped wasting my spare time on fixing this issue (;-)): b64 version 0.06 available (build)
Thanks to everyone for testing! :-)

Back to index page
Thread view  Board view
15195 Postings in 1365 Threads, 250 registered users, 13 users online (0 registered, 13 guests)
DOS ain't dead | Admin contact
RSS Feed
powered by my little forum