Topic: Scribe's speed (1 of 1), Read 21 times, 1 File Attachment
Conf: VEDIT User Applications
From: Scott Lambert
Date: Monday, February 07, 2011 03:44 PM



I have been doing some experiments with Scribe with regards to speed.

It appears that the 100,000+ word english.vdf is a double edged sword. On one hand, it means that Scribe is less likely to flag a correct word. On the other hand it slows Scribe significantly.

For testing, I used a modified mainmenu.vdm that saved the time_tick ( in n-regs 1 & 2) on either side of the call_file command to spelling.vdm for option 1 in the main menu.

I have created a smallish main dictionary mywords.vdf with just under 5000 words, and also created a 2000 word test document ( an except from the thesaurus data file) and ran tests using both english.vdf and mywords.vdf as the main dictionary (languages had no special dictionaries). All words in the test document were in the dictionaries eliminating my response time to the correction submenu from the tests.

Using english.vdf, it took Scribe 123.647 seconds to spell check the 2000 word test document. That is about 16.18 words per second.

Using the mywords.vdf, it took Scribe 31.064 seconds to spell check the same 2000 word test document. That is about 64.38 words per second.

So we are talking almost a 4x speed increase. (you can see the speed increase as Scribe blows thru the file)

However....

If you use a smaller dictionary like mywords.vdf, Scribe pops up the Correction Submenu more often as it is more likely to flag correct words.

Now experts say the aveage adult has about 10,000 to 15,000 words in their vocabulary, so a 100,000+ word dictionary is overkill on an individual basis. However everyone's vocabulary is composed of a different set of words, so for the aveage spell checker a large dictionary would seem a must.

Then you have the issue of different spellings according to region (color vs colour, tires vs tyres).

I believe the solution to the above is to start with a small starting dictionary and do the Learn new Words process outlined in the manual.txt file. Basically you train Scribe to your vocabulary.

Now I have also created a base.vdf (served as the start for mywords.vdf) which only has about 700+ words in it (pronouns, tenses of to be, to do, to have, days of week, months, etc). Very common every day words, we all use. No words that are spell differently by region.

So if Scribe's speed is an issue, you can use the attached mywords.vdf or use base.vdf and the learn new words process outlined in the manual.txt file to create your own mywords.vdf. This may be your best option, aside from developing patience. ;)

I am going to continue building mywords.vdf, however my 10,000 words are probably different from yours. (the attached mywords.vdf contains almost all the words in the thesaurus.vtf file.)

To use mywords.vdf, simply change the main dictionary in a language from english.vdf to mywords.vdf by editing the scribe.ini file. Or create a new language as outlined in the setup.txt file. (backup the scribe.ini file first in either case).

The modified mainmenu.vdm (if you wish to perform your own tests), mywords.vdf, base.vdf, & 2000.txt (the test file) are in the attached zip file.

Note: if you choose to do any of your own speed tests, it is important to exit scribe between tests due to the way Scribe uses the ignore this session buffer to (or try to) increase speed.

Thanks.

Scott

 
SPEED.ZIP (29KB)