DOS ain't dead

Japheth released JWasm 2.01 on 12 December 2009.

Website: http://www.japheth.de/JWasm.html
Download: http://www.japheth.de/Download/JWasm/JWasm201b.zip

> 12/12/2009, v2.01:
>
> Bugfixes:
>
> - shift and rotate opcodes accepted any 8-bit register as second
> argument. If it wasn't CL, wrong code was generated.
> - IN/OUT opcodes accepted any 16-bit register as source/target, not
> just DX. If it wasn't DX, wrong code was generated.
> - an unnecessary DS prefix for OUTS string instruction wasn't removed.
> - 16-bit indirect addressing-mode with a scale > 1 wasn't rejected and
> invalid code was generated.
> - if errors occured JWasm still returned error code 0.
> - indirect CALL/JMP needed a size, even if -Zm was set.
> - with -Zm, the offset of a struct member wasn't correct if the struct
> was nested and the member was referenced without the 'dot' operator.
> - in v2.0, error 'Constant value too large' may have occured if the
> SHL operator was used in expressions to define an immediate value
> and the result didn't fit in 32-bits.
> - directive '=': expression wasn't rejected if it contained externals.
> Now error 'Constant expected' is displayed.
> - JWASMR only: detecting the size of an immediate operand often failed,
> which then gave error 'immediate data out of range'.
> - segment/group override used in an argument for the OFFSET operator
> was ignored.
> - format BIN: offsets were always group-relative (OPTION OFFSET:SEGMENT
> was ignored).
> - arithmetic instructions with immediate operand: the signed (=shortest)
> variant of the instruction wasn't generated if the immediate operand
> was a negative WORD value.
> - absolute segments weren't correctly displayed in the listing file.
> - formats MZ: fixups with references to absolute segments weren't
> handled correctly.
> - SSE2 CMPSD/MOVSD: memory operand of type QWORD was rejected.
> - SSSE3 instructions: indirect addressing with ESP didn't emit the
> SIB byte.
> - in v2.0, a struct field with a name which matched an arbitrary type
> was rejected if the type of the struct field was a predefined TYPE
> (example: name1 WORD ? ; [<name1> is both a struct field name
> and an arbitrary type name] ).
> - OMF format: if the LNAME index of a group name was > 255, the link
> between the name and the group definition record became broken.
> - syntax "EXTERN:" and "EXTERNDEF:" (directive + colon, but no type),
> was rejected, but seems to be valid Masm-syntax.
> - for PUSHD, only immediate operands were accepted.
> - in JWasm, using a text macro before it is defined is to cause
> a warning. This worked for text macros defined with EQU only, but
> didn't if the macro was defined with CATSTR or TEXTEQU.
> - for segment registers, the assume search order in v1.96 and v2.0
> didn't match Masm's.
> - macro loop directives (REPEAT, FOR, FORC, WHILE) with no data
> lines caused a GPF.
> - if a text macro contained multiple macro function calls, only the
> first one was evaluated on some conditions.
>
> Other changes:
>
> - equates and assembly time variables: magnitude of values has been
> increased by 1 bit. Now it's 32bit plus a sign bit - similar to Masm > v8.
> - multiple overrides in an expression will give a warning.
> - if an equate was assigned a PROC, the proc-specific info
> wasn't stored,
> which made it impossible to use the equate for the INVOKE directive.
> - instruction sets SSE4.1 and SSE4.2 supported.
> - data types MMWORD and XMMWORD supported.

> Japheth released JWasm 2.01 on 12 December 2009.

>> Bugfixes:
>> - shift and rotate opcodes accepted any 8-bit register as second
>> argument. If it wasn't CL, wrong code was generated.
>> - IN/OUT opcodes accepted any 16-bit register as source/target, not
>> just DX. If it wasn't DX, wrong code was generated.
>> - an unnecessary DS prefix for OUTS string instruction [.....]

Wow!

Out of mere curiosity, what sort of source program sample and test tools/method do you use for checking correctness ? Don't take it as a pique or critique, but I was stunned at the very sight of that bugfix list !


Regards

--
Ninho

[Edit] the question is adressed to Japheth, not the O.P. of course.

[Note: this response was to Ninho but written in a way to help others also]

> but I was stunned at the very sight of that bugfix list !
I'm not. What I am please about is that he's picked up on these things as it shows that his work and own coding skills are maturing further. From what I have seen of his work so far he is a damn good coder. Concerning these bugs, well it is easy to introduce even very basic bugs if changing the base code to accommodate new features, e.g. adding 64 bit or replace previously written established code due to later learnings. Thus many err on the side of caution with "if ain't broke don't fix it" but how would things ever move forward if we all took that approach in life all the time? So it is best to realize there are times when that approach is good and times when it is not.

> Out of mere curiosity, what sort of source program sample and test
> tools/method do you use for checking correctness ?
I expect that he is doing at least the following:

1) Reverse engineering the outputted code to check if it is as expected
2) Compiling known code with other compilers and comparing the binary output

The problem with always using your own debugging/code analysis tools to check your own code is that often your own tools may contain bugs that cause you to miss bugs in the software which you are checking. Obviously this applies to other peoples tool as well, thus often good to do several checks with different software rather than just relying on the output of just one tool.

However even binary comparisons of different compilers output aren't straight due to compiler/linker code signatures etc. An example of what I mean by this is different compilers often use different opcodes which may do the same function (or almost the same function) or use a different order of opcodes, e.g. speed/size optimization. Thus when binaries of the same code produced by different compilers are compared even if they are of the exactly the same filesize they often do not necessarily match. This can also be deliberate as using specific opcodes over others and order of instruction and other techniques are used as a hidden clever way of digital code signatures. Which is why most compiler authors can identify if you have used their tools (or their code), obviously these techniques aren't just used by compilers either!

As what tools does he use, well no doubt he is using his own debugger as well as other internal / experimental tools (e.g. he might have a custom experimental debugger say a v2 that he is testing). Along with a collection of the usual Reverse engineering tools and debuggers available, e.g. IDA pro.

If you want to learn more about code analysis a good place to start is probably Wikibooks: X86 Disassembly Analysis Tools. There are many other references on the net, but I would say a good way to learn is from simply reading posts of experts such as Ilfak Guilfanov and as you already know http://www.GeoffChappell.com/

>> Out of mere curiosity, what sort of source program sample and test
>> tools/method do you use for checking correctness ?

> I expect that he is doing at least the following:

Why on Earth do you feel an urge to try to answer on Japheth's behalf, Arjay ?

I'm asking him if he will disclose the special testing methodology /he/ applied to /his/ assembler. No immediate need for void generalities about debugging. An assembler is not /any/ program, and yes I wrote assemblers - my first one using hash tables was for the IBM 1620, doesn't that date me ?

...snippage...

>> Don't take it as a pique or critique,
> He probably won't.

Again double guessing others, I see

---
Ninho

> Why on Earth do you feel an urge to try to answer on Japheth's behalf,
Firstly apologies if it seemed I came across that way as that was far from my intention. My response was simply intended to be helpful to you / the "readers" that frequent this forum, indeed my own posts are written deliberately to aid others. We can all learn of each other is the way I look at things and I continued to learn a lot from the great work of Japheth.

> I'm asking him if he will disclose the special testing methodology /he/
> applied to /his/ assembler.
Ok, sorry that wasn't clear and again apologies for stepping on any toes here.

> debugging. An assembler is not /any/ program, and yes I wrote assemblers -
Likewise.

> my first one using hash tables was for the IBM 1620, doesn't that date me ?
No, your experience predates my own MVS experience and I'm pleased to know that we share something else in common which we might be able to converse more about (offline from here). I also have the pleasure of conversing with others in that area including folks who predate you there ;) So dare I now point you to http://www.hercules-390.org/ just in case you aren't aware of it! Yes I know "after" the 1620 but may be of interest if you didn't know it existed... but I'd rather not second guess if you know about it or not

> Out of mere curiosity, what sort of source program sample and test
> tools/method do you use for checking correctness ? Don't take it as a
> pique or critique, but I was stunned at the very sight of that bugfix list

There is a collection of self-written small test programs (test suite). A part of it is contained in JWasm's source package. It's "work in progress".

---
MS-DOS forever!

> > Why on Earth do you feel an urge to try to answer on Japheth's behalf,
> Firstly apologies if it seemed I came across that way as that was far from
> my intention. My response was simply intended to be helpful to you / the
> "readers" that frequent this forum

Sheesh, Arjay just gave an opinion and got mauled by the lions ....

I guess I should apologize for announcing it myself instead of letting Japheth do it. I didn't mean to steal his thunder.

> Firstly apologies if it seemed I came across that way as that was far from
> my intention. My response was simply intended to be helpful to you / the
> "readers" that frequent this forum, indeed my own posts are written
> deliberately to aid others. We can all learn of each other is the way I
> look at things and I continued to learn a lot from the great work of
> Japheth.

No need for you to apologise, Arjay, or we both need. It was my mistake in the first place, when writing my question I believed it was in reply to Japheth (rather than Rugxulo). We're also victims of the ambiguity of the English pronoun "you" - whether it means the second person ("thou") or persons, or else refers to one or more of an undefined group...

So I really /should/ have asked somewhat like :

"Praytell, Japheth, what tools and methods usest thou ...?"

Your attempts at being helpful are very welcome and much appreciated, really!

---
Ninho

> > Out of mere curiosity, what sort of source program sample and test
> > tools/method do you use for checking correctness ? Don't take it as a
> > pique or critique, but I was stunned at the very sight of that bugfix
> list
>
> There is a collection of self-written small test programs (test suite). A
> part of it is contained in JWasm's source package. It's "work in
> progress".

Many thanks. I'll have a look at that some time. Undisputably, X86 machine language is particularly encumbered with annoying special cases and the awkward assembler syntax which we inherited from Intel must make code generation even more of a nightmare.

---
Ninho