Back to home page

DOS ain't dead

Forum index page

Log in | Register

Back to index page
Thread view  Board view
Pablo

17.07.2010, 20:36
 

Question: best performance of hard disk reading (Users)

Exists a utility of disk catching with the following functions?
For older computers with small memory (LESS OF 16mb)

1) that it stores to all the FAT and directories (only as READ disk cache to maintain the integrity of this area)

2) that it stores to temporary files: tmp,$$$, or other (as READ/WRITE disk cache).

3) that in the REMAINING memory it stores all the sectors that are possible (only as READ disk cache), beginning from the sector 1 of the files area (command.com, etc, etc...)

I believe that a model like this improvement much the general performance of a older computer.

What utility allows to do that?

Laaca

Homepage

Czech republic,
17.07.2010, 22:39

@ Pablo
 

Question: best performance of hard disk reading

> What utility allows to do that?


Maybe combination of UIDE and LBAcache.

---
DOS-u-akbar!

Rugxulo

Homepage

Usono,
18.07.2010, 02:01
(edited by Rugxulo, 18.07.2010, 04:49)

@ Laaca
 

Question: best performance of hard disk reading

> > What utility allows to do that?
>
> Maybe combination of UIDE and LBAcache.

I'm not entirely clear on what he wants here. Anyways, both UIDE and LBAcache both need a 386 or better to work. They also need XMSv3, I think. (EDIT: No, UIDE can use XMSv2, esp. necessary under default memory managers of ROM-DOS or DR-DOS.) LBAcache can use 0.25 MB increments while UIDE at minimum uses 5 MB cache size. On my old P166, I use UIDE /S5 /N2, and it seems to work fine.

UIDE supports higher amounts of RAM, Ultra DMA, CD/DVD drives, is smaller and faster, doesn't need a separate TICKLE for FDs, and is more frequently updated. Still, Eric's tool is okay too. ;-) You'd really have to benchmark to prove it to yourself, and I don't have any reasonable test results on-hand.

P.S. Set TEMP and TMPDIR to a RAM drive if you want fast temporary file access.

EDIT: Oops, here's what Jack says:

> Johnson has errors on his website, describing the new
> 13-Jul-2010 UIDE. Despite a "Last Update" of 11-Jul-2010,
> the download files ARE now dated 13-Jul-2010, and UIDE will
> now allow a very-fast "memory only" cache for JEMM386 users
> who need absolute protected-mode speed. Uses much memory,
> but some speed-freaks may still want it.

Arjay

18.07.2010, 09:59

@ Rugxulo
 

Question: best performance of hard disk reading

Pablo, you mention old machine what machine specifications did you have in mind? As what when I tend to think of old PC's I'm thinking of 8088, 8086, 80186's based PC's. And what are you trying to do with the PC (or PC's)?
Obviously the problem with older PC's is the further you go back the less memory they tend to have etc. e.g. Some PC's were 64k.

> P.S. Set TEMP and TMPDIR to a RAM drive if you want fast temporary file
> access.
Depending on what you are trying to do, I agree using a RAMDRIVE for various file accesses is a good option in terms of speeding things up. e.g. if you are compiling. Moving COMMAND.COM via comspec=d:\command.com (where d: = ramdrive) will also help if COMMAND.COM is being accessed a lot. Rayer's ROM OS may also be a good option.

Pablo

19.07.2010, 09:24

@ Arjay
 

Question: best performance of hard disk reading

A disk cache is an area of memory that is set to store the data most recently read from the hard disk. But it does not work with the data that not yet read.

Need a disk cache utility to store to the FAT and directories completely (maybe compressed).This is to improve performance, since the system memory is many times faster than the hard disk. Mainly if we consider that the FAT is at the beginning of the disc, and all time heads returning to the FAT doing slow the reading of files especially that are located in the opposite side. I am not have a benchmark to prove it. :-P
[image]

Rugxulo

Homepage

Usono,
19.07.2010, 12:59

@ Pablo
 

Question: best performance of hard disk reading

> A disk cache is an area of memory that is set to store the data most
> recently read from the hard disk. But it does not work with the data that
> not yet read.
>
> Need a disk cache utility to store to the FAT and directories completely
> (maybe compressed).This is to improve performance, since the system memory
> is many times faster than the hard disk. Mainly if we consider that the FAT
> is at the beginning of the disc, and all time heads returning to the FAT
> doing slow the reading of files especially that are located in the opposite
> side. I am not have a benchmark to prove it. :-P

I (maybe incorrectly) thought that some DOSes indeed kept the FAT completely in RAM. I'm a bit fuzzy on the details since I never looked too closely, so maybe I'm thinking of only really small disks. (Obviously not FAT32, but if FAT16 is max. 65535 files, would that fit? Arjay? :-) )

Anyways, this is just another example of why we should have native support of ext2 or HPFS or whatever. (Linux gets 40+ FSes and we get only one, whee! It's a pity they don't help us more. If I weren't so dumb ....)

Arjay

19.07.2010, 16:24
(edited by Arjay, 19.07.2010, 16:40)

@ Rugxulo
 

Question: best performance of hard disk reading

> > A disk cache is an area of memory that is set to store the data most
> > recently read from the hard disk.
> > But it does not work with the data that not yet read.

It depends on the design of the cache and what options have been used by the end user; Read ahead cache buffers have been common on PC's for years in various places not only in hardware such as disk controllers, but also within CD-ROM drivers e.g. MSCDEX's /M option, Smartdrive's /B option etc, there is a dedicated 1986 DOS util called DFACC (DOS File ACCelerator) by S.H.Smith which apparently (dfacc10a.zip) does some of what it appears you were asking asked for in a util. Personally however I wouldn't risk using DFACC with a modern DOS as it is far too old (last tested with version 3.1!). Note: the DFACC.DOC (text) incorrectly says the following: "This is the opposite of a "cache" type program. A cache keeps data AFTER it has been used; DFACC gets data BEFORE it is needed."

I say "incorrectly" as the computer dictionary definition of the word "cache" is "To store data in a faster storage system or a storage system closer to the usage of the data." - in other words the key text book definition phrase covers both BEFORE and AFTER uses. It also doesn't state that cache data has to be accessed "immediately", it's just that most people expect cache data to be used immediately but it doesn't have to be.

> > Need a disk cache utility to store to the FAT and directories completely
> > (maybe compressed).
The reason I asked for a little more info to better understand your needs/thinking was because I was thinking that depending on what you were looking at doing you may not even need to bother with FAT caching anyway.

For example if you were asking because you were planning to code up something to be used on older machines and were mostly thinking about caching because you were planning to use a lot of files; then my thinking would be to take a step back and ask yourself if it is possible to eliminate the need to have the files in the first place. In other words rather than looking for a solution to a problem I was looking to see if we could eliminate the problem instead.

I'll give you an example: Years ago I came across an excellent 128 byte program for displaying a 320x200 PCX file (this was after I'd coded a slightly larger program in TP to do the same). So I already knew when I saw this little program that the header for a PCX file was 128 bytes, the same length as this program (obviously done on purpose). Like most people I suspect who knew this I also immediately realized that the code could be easily combined with a 320x200 PCX file to make a self displaying graphics file. However when I stepped back I also realized that since the program included file handling code that if there was no separate file that the file handling code wasn't needed thus a combined file could be smaller. That is to say if there was no file then no file handling was required. I'm sure the author of the 128 byte program realized this hence 128 bytes in size to make that point even though from memory he never actually stated it in his source.

In a similar mindset if FAT is wasteful as we know in terms of size, rather than necessarily worrying about compressing FAT, etc depending on the particular situation if you can avoid the need to use FAT in the first place. The problem with systems like DOS is people tend to get conditioned, e..g. it is natural to think in terms of files, FAT. Which interestingly is exactly what Rugxulo picked up on, this very fact that often in the DOS world we are still thinking about FAT, files etc.

So a program could for example support a parameter of a pointer to a memory data address instead of expecting a file (or both). This doesn't mean this this is always the right option but my point is there are other alternatives.

> > side. I am not have a benchmark to prove it. :-P
Ever read The Story of Mel in relation to drum memory?

> I'm a bit fuzzy on the details since I never looked too closely,
To be honest I am as well. As although I started coding up some FAT stuff some time back I then realized there was other better ways of doing the same things I wanted to do hence I instead spent the time on things like file packaging, hence instead of writing FAT code I wrote DLL handing code instead.

> so maybe I'm thinking of only really small disks. (Obviously not
> FAT32, but if FAT16 is max. 65535 files,
As I understand it with FAT16 it is 512 in a root directory (seen this many times), 65536 max but also depends on the size of the disk.

> would that fit? Arjay? :-) )
:)

> of ext2 or HPFS or whatever. (Linux gets 40+ FSes and we get only one,
> whee! It's a pity they don't help us more. If I weren't so dumb ....)
Don't put yourself down and yes I totally agree it would be good if there were more FS supported under DOS. It is worth remembering that a CD-ROM driver is an example of DOS being extended in terms of more file systems, it's just that most organizations lost interest in extending DOS. It is also worth noting a number of DOS demos and games not only supported file packages (like WAD's) but some did this by patching the Interrupt 21h file handing routines. Indeed I remember hearing of a demo group having a problem when an EXE virus got "inside" their file system after an infect file was packaged in.

Paul

19.07.2010, 20:03

@ Arjay
 

Small PCX Viewer

> I'll give you an example: Years ago I came across an excellent 128 byte
> program for displaying a 320x200 PCX file (this was after I'd coded a
> slightly larger program in TP to do the same). So I already knew when I
> saw this little program that the header for a PCX file was 128 bytes, the
> same length as this program (obviously done on purpose). Like most people
> I suspect who knew this I also immediately realized that the code could be
> easily combined with a 320x200 PCX file to make a self displaying graphics
> file. However when I stepped back I also realized that since the program
> included file handling code that if there was no separate file that the
> file handling code wasn't needed thus a combined file could be smaller.
> That is to say if there was no file then no file handling was required.
> I'm sure the author of the 128 byte program realized this hence 128 bytes
> in size to make that point even though from memory he never actually stated
> it in his source.

Digressing, but I found a PCX viewer that's only 56 bytes! The author also has a PCX-to-EXE utility. Actually quite relevant to this example, anyway.
http://agrgic.tripod.com/dl/

Arjay

20.07.2010, 11:32

@ Paul
 

Small PCX Viewer / Arminio Grgic's website

> Digressing,
:)

> Actually quite relevant to this example,
Thanks. This example helps show further what can be thrown away abiet at the more extreme end of the spectrum. Also worth noting that file input for the 56byte PCX viewer still occurs but via CONsole redirection.

> http://agrgic.tripod.com/dl/
Arminio Grgic's website is interesting overall and re-reminded of more projects I should release sometime including TDC2BIN which would compliment some of Arminio's DOS image utils.

Happy to talk all things small further under this thread to help keep this one on topic.

RayeR

Homepage

CZ,
22.07.2010, 00:14

@ Arjay
 

Question: best performance of hard disk reading

> As I understand it with FAT16 it is 512 in a root directory (seen this many
> times), 65536 max but also
> depends on
> the size of the disk.

FAT16 can manage up to near 65536 clusters (cluster size is max 32kB [NT hack 64kB]) but there are some reserved values si this number is little bit smaller. Every FAT16 entry is 16bit number (index to next cluster in chain) so the FAT size is up to 128kB. There are usually 2 identical FAT copies on HDD that must be synchronized so it's enough to have one copy in memory. I doubt that DOS woul load entire FAT, it would fit but takes a lot of free mem. Of course cache program can put it in XMS :)
I don't understand the note about 512 root dir entries, it's small enough but larger dirs with thousands of files wil take several kBs. Directories are placed on verious places of HDD compared to FAT which is on stable place (no fragments) and it may take a long time to scan and load entire directory structure to RAM...

---
DOS gives me freedom to unlimited HW access.

marcov

22.07.2010, 05:55

@ Arjay
 

Question: best performance of hard disk reading

> In a similar mindset if FAT is wasteful as we know in terms of size, rather
> than necessarily worrying about compressing FAT, etc depending on the
> particular situation if you can avoid the need to use FAT in the first
> place. The problem with systems like DOS is people tend to get
> conditioned, e..g. it is natural to think in terms of files, FAT. Which
> interestingly is exactly what Rugxulo picked up on, this very fact that
> often in the DOS world we are still thinking about FAT, files etc.

Afaik many bioses scan the partition table, and revert to alternate boot if no recognizable partition/filesystem scheme is found. This started somewhere in P-II,III times for general clones, but maybe earlier for funky big OEM machines.

Before I would use a raw format of the HD, I would first try to check if this wouldn't bring me more trouble. (it could be solved as easily as keeping a partitiontable and some base FAT structures though)

Arjay

22.07.2010, 08:19

@ marcov
 

Question: best performance of hard disk reading

> Afaik many bioses scan the partition table, and revert to alternate boot if
> no recognizable partition/filesystem scheme is found. This started

Hmmm I wasn't thinking re booting but you make a very good point there. So thanks for the info. I have had less experience with HD booting but have seen similar issues with incorrectly structured removable boot media.

> keeping a partitiontable and some base FAT structures though)
Good suggestion of a simple work around.

DOS386

23.07.2010, 07:05

@ Pablo
 

Question: best performance of hard disk reading

> For older computers with small memory (LESS OF 16mb)
> 1) that it stores to all the FAT and directories (only as READ disk cache
> What utility allows to do that?

You will always need the memory for all the mess, still.

Put your stuff into a RAMDISK (if you have 80286 only than you need an 80286-compatible XMS 2.x host (crappy faked-RESET hack) and an 80286-compatible RAMDISK. :-|

---
This is a LOGITECH mouse driver, but some software expect here
the following string:*** This is Copyright 1983 Microsoft ***

Back to index page
Thread view  Board view
22752 Postings in 2119 Threads, 402 registered users (0 online)
DOS ain't dead | Admin contact
RSS Feed
powered by my little forum