Topic: Automatic Unicode handling (1 of 13), Read 23 times
Conf: VEDIT Suggestions
From: Howard Goldstein
Date: Tuesday, March 18, 2008 02:24 PM

It would really be great if VEDIT could recognize a Unicode file when it is opened and automatically make it readable. The buffer contents should be translated back to Unicode when the file is saved. I know that we already have macros to do these translations but making it all automatic would be a lot more convenient. After all -- and I know this will smart a little -- even Notepad can handle Unicode files!

-- Howard

 


Topic: Re: Automatic Unicode handling (2 of 13), Read 18 times
Conf: VEDIT Suggestions
From: Ted Green
Date: Thursday, March 20, 2008 03:47 PM

At 03:23 PM 3/18/2008, you wrote:
>From: "Howard Goldstein"
>
>It would really be great if VEDIT could recognize a Unicode file when it is opened and automatically make it readable. The buffer contents should be translated back to Unicode when the file is saved. I know that we already have macros to do these translations but making it all automatic would be a lot more convenient. After all -- and I know this will smart a little -- even Notepad can handle Unicode files!

I know that making VEDIT natively support Unicode would (will someday) require months of programming effort.
Thank you for the suggestion of auto-running the Unicode conversion macros when a file is opened and closed. I will add an special Unicode internal variable so that the file-close event macro can check if the file is to be converted back to Unicode.

Ted.

 


Topic: Re: Automatic Unicode handling (3 of 13), Read 18 times
Conf: VEDIT Suggestions
From: Ian Binnie
Date: Thursday, March 20, 2008 08:15 PM

On 3/20/2008 3:47:59 PM, Ted Green wrote:
>At 03:23 PM 3/18/2008, you
>wrote:
>>From: "Howard Goldstein"
>>
>I know that making VEDIT
>natively support Unicode would
>(will someday) require months
>of programming effort.

I have been converting a few c/c++ programs to Unicode, and this is time consuming. I shudder at the thought of doing this in assembler.

>Thank you for the suggestion
>of auto-running the Unicode
>conversion macros when a file
>is opened and closed. I will
>add an special Unicode
>internal variable so that the
>file-close event macro can
>check if the file is to be
>converted back to Unicode.

Ted, This is probably a good idea, but I hope you will include some way to bypass the conversion.

I use Vedit to convert some Unicode files to ANSI.
I don't want Vedit to convert them back.

 


Topic: Re: Automatic Unicode handling (4 of 13), Read 17 times
Conf: VEDIT Suggestions
From: Ted Green
Date: Saturday, March 22, 2008 12:26 PM

At 09:42 PM 3/20/2008, you wrote:
>From: Ian Binnie
>
>On 3/20/2008 3:47:59 PM, Ted Green wrote:
>>At 03:23 PM 3/18/2008, you wrote:
>>I know that making VEDIT >natively support Unicode would (will someday) require months of programming effort.
>
>I have been converting a few c/c++ programs to Unicode, and this is time consuming. I shudder at the thought of doing this in assembler.

As I have been rewriting many low level display and windowing routines in VEDIT, I have been making my changes with an eye towards Unicode support. At that same time I will also implement 16-bit color support in VEDIT (currently 8 bit). Therefore a 32-bit display value can consist of a 16-bit character and a 16-bit color attribute.
The toughest part will probably be to create a Unicode version of all the search routines; VEDIT will need to include both 8-bit and 16-bit character search routines.

Ted.

 


Topic: Re: Automatic Unicode handling (6 of 13), Read 17 times
Conf: VEDIT Suggestions
From: Pauli Lindgren
Date: Saturday, March 22, 2008 06:42 PM

On 3/22/2008 12:26:42 PM, Ted Green wrote:
>
>As I have been rewriting many low level
>display and windowing routines in VEDIT,
>I have been making my changes with an
>eye towards Unicode support. At that
>same time I will also implement 16-bit
>color support in VEDIT (currently 8
>bit). Therefore a 32-bit display value
>can consist of a 16-bit character and a
>16-bit color attribute.

I don't quite see how 16-bit color would be better than 8-bit color. You would need 24 bits to display all colors.
On the other hand, you do not need to specify all colors for each character. You only need to specify the LOGICAL color (e.g. "comment color" or "keywords 1"). 8 bits is more than enough for this. The actual color for each logical color should then be specified as 48 bits (24 bits for foreground and 24 bits for background).

--
Pauli

 


Topic: Re: Automatic Unicode handling (7 of 13), Read 17 times
Conf: VEDIT Suggestions
From: Ian Binnie
Date: Saturday, March 22, 2008 07:29 PM

On 3/22/2008 6:42:35 PM, Pauli Lindgren wrote:
>On 3/22/2008 12:26:42 PM, Ted Green
>wrote:
>>
>>As I have been rewriting many low level
>>display and windowing routines in VEDIT,
>>I have been making my changes with an
>>eye towards Unicode support. At that
>>same time I will also implement 16-bit
>>color support in VEDIT (currently 8
>>bit). Therefore a 32-bit display value
>>can consist of a 16-bit character and a
>>16-bit color attribute.
>
>I don't quite see how 16-bit color would
>be better than 8-bit color. You would
>need 24 bits to display all colors.
>On the other hand, you do not need to
>specify all colors for each character.
>You only need to specify the LOGICAL
>color (e.g. "comment color" or "keywords
>1"). 8 bits is more than enough for
>this. The actual color for each logical
>color should then be specified as 48
>bits (24 bits for foreground and 24 bits
>for background).

Pauli,

8 bit colour normally uses a palette to display colours, usually the system palette, and in fact less than 256 colours, as 16 are "reserved".

Vedit in fact only uses the 16 colours, the 256 choices are a combination of 16 foreground and 16 background.

16 bit colour uses 5,6,5 bits to represent the RGB pixel, and can thus use 64k colours, although I suspect Vedit will be actually using 256 foreground and 256 background.

I don't see the need for Vedit to display more colours (although I would welcome more choices in syntax highlighting), but this would allow the user to pick their own colours rather than the current restricted set.

 


Topic: Re: Automatic Unicode handling (8 of 13), Read 18 times
Conf: VEDIT Suggestions
From: Ted Green
Date: Monday, March 24, 2008 11:10 AM

At 07:30 PM 3/22/2008, you wrote:
>8 bit colour normally uses a palette to display colours, usually the system palette, and in fact less than 256 colours, as 16 are "reserved".
>
>Vedit in fact only uses the 16 colours, the 256 choices are a combination of 16 foreground and 16 background.
>
>16 bit colour uses 5,6,5 bits to represent the RGB pixel, and can thus use 64k colours, although I suspect Vedit will be actually using 256 foreground and 256 background.

Yes, with a 16-bit attribute, it would be easiest to support 256 foreground and 256 background colors. This would be a very pleasant improvement over the current 16 colors. To see 256 different colors go to: http://www.w3schools.com/html/html_colors.asp - 2/3 down the page.

There is little reason to support more colors in a "text" based program.

Ted.

 


Topic: Re: Automatic Unicode handling (12 of 13), Read 17 times
Conf: VEDIT Suggestions
From: Pauli Lindgren
Date: Tuesday, March 25, 2008 06:59 AM

On 3/24/2008 11:10:14 AM, Ted Green wrote:
>
>Yes, with a 16-bit attribute,
>it would be easiest to support
>256 foreground and 256
>background colors. This would
>be a very pleasant improvement
>over the current 16 colors. To
>see 256 different colors go
>to:
>http://www.w3schools.com/html/
>html_colors.asp - 2/3 down the
>page.

I hope you are not going to implement a fixed "web safe" palette. The colors should be freely configurable, not limited to 2.5 bits per channel.

It is important to be able to adjust color precisely so that you could for example use different pastel colors for backgrounds of different windows (for example to separate different file types). For that, you need to be able to change the colors with full 8 bits/channel precision. So the palette colors should be configurable.

--
Pauli

 


Topic: Re: Automatic Unicode handling (11 of 13), Read 19 times
Conf: VEDIT Suggestions
From: Pauli Lindgren
Date: Tuesday, March 25, 2008 06:50 AM

On 3/22/2008 7:29:45 PM, Ian Binnie wrote:
>
>Pauli,
>
>8 bit colour normally uses a palette to
>display colours, usually the system
>palette, and in fact less than 256
>colours, as 16 are "reserved".

But the colors can be chosen from the selection of 16 million colors.

>
>Vedit in fact only uses the 16 colours,
>the 256 choices are a combination of 16
>foreground and 16 background.
>
>16 bit colour uses 5,6,5 bits to
>represent the RGB pixel, and can thus
>use 64k colours, although I suspect
>Vedit will be actually using 256
>foreground and 256 background.
>
>I don't see the need for Vedit to
>display more colours (although I would
>welcome more choices in syntax
>highlighting), but this would allow the
>user to pick their own colours rather
>than the current restricted set.

256 colors (8 bits) is definitely enough for Vedit. Vedit currently only uses 12 colors: 4 editing colors plus 8 syntax colors. Even if more syntax colors would be added, it still leaves some 240 colors for the user.

Having user colors would be a nice enhancement. This would make it possible to use colors in macro output windows. (Of course we would need macro commands for that.) In addition, highlight for searched words would be nice, and perhaps multiple different searches (patterns) displayed simultaneously with different colors (as in some HEX editors).

However, if the color system is changed, I would prefer the option to choose the colors from palette of 16 million instead of fixed palette of 256 colors.

For example, when selecting color for "comments", you could use color picker to select any 8+8+8 bit RGB color for it. This could be implemented internally such way that one index in the 8-bit color palette table would be reserved for "comment" color.
Another (more complex) option would be to pick the color from the palette of 256, and with a separate configuration function, choose the actual RGB color for that palette index.

In any case, there should be option to select "transparent" color for any syntax or editing color so that syntax highlight would work correctly with different window background colors.

--
Pauli

P.S. Maybe this discussion should be on different thread since this is not related to Unicode anymore.

 


Topic: Re: Automatic Unicode handling (13 of 13), Read 23 times
Conf: VEDIT Suggestions
From: Ted Green
Date: Tuesday, March 25, 2008 09:54 AM

At 06:53 AM 3/25/2008, you wrote:
>From: "Pauli Lindgren"
>P.S. Maybe this discussion should be on different thread since this is not related to Unicode anymore.

Pauli:

Thank you for your ideas and feedback.
Lets close this thread for now until I am actively working on new color support; then I will open a new thread.

Ted.

 


Topic: Automatic Unicode handling (5 of 13), Read 26 times
Conf: VEDIT Suggestions
From: Pauli Lindgren
Date: Saturday, March 22, 2008 05:59 PM

On 3/18/2008 2:24:56 PM, Howard Goldstein wrote:
>It would really be great if VEDIT could recognize a
>Unicode file when it is opened and automatically make it
>readable. The buffer contents should be translated back to
>Unicode when the file is >saved. I know that we already
>have macros to do these translations but making it all
>automatic would be a lot more convenient. After all -- and
>I know this will smart a little -- even Notepad can
>handle Unicode files!

Definitely not!
Vedit should NOT automatically convert Unicode files!

One of the most important requirements for a text editor is that it should NOT change the file by itself, without user command. You should always be able to load a file and then save it and be sure that nothing has been changed.

There are many editors that claim to support Unicode, but in reality they just automatically convert Unicode files into Windows character set for editing. And this causes lots of problems.

Most of the problems people have with editing Unicode files are caused by editors that do this automatic conversion. Because when you convert Unicode files to Windows character set, any characters that are not included in Windows character set are destroyed. And there is no way to restore them. A text editor should NOT destroy contents of the file.

What we need is native Unicode support. If it takes long time to implement it, then meanwhile we need conversion macro between UTF-8 and Windows character set. Windows API call could do this, but it would be better if we had an option to retain all characters e.g. by using HTML entities.

--
Pauli

 


Topic: Re: Automatic Unicode handling (9 of 13), Read 25 times
Conf: VEDIT Suggestions
From: Ted Green
Date: Monday, March 24, 2008 11:22 AM

At 06:00 PM 3/22/2008, you wrote:
>From: "Pauli Lindgren"
>
>Definitely not!
>Vedit should NOT automatically convert Unicode files!

Pauli:

You make some good points.
Therefore, perhaps the following:

1. A config setting to check for Unicode files.
2. If config is TRUE, and a Unicode file is opened, automatically pop up a dialog
box to convert it - Yes/No. If Yes, the conversion will set a new Internal Value.
3. Upon saving, if the Internal Value is TRUE, convert it back to Unicode.

While Unicode support is months away, I have some of these same issues with the AES encryption that I recently implemented. I have not yet worked on the User Interface portion of it, only have the macro language working.

Ted.

 


Topic: Re: Automatic Unicode handling (10 of 13), Read 23 times
Conf: VEDIT Suggestions
From: Howard Goldstein
Date: Monday, March 24, 2008 06:56 PM

On 3/24/2008 11:22:50 AM, Ted Green wrote:
>At 06:00 PM 3/22/2008, you
>wrote:
>>From: "Pauli Lindgren"
>>
>>Definitely not!
>>Vedit should NOT automatically convert Unicode files!
>
>Pauli:
>
>You make some good points.
>Therefore, perhaps the
>following:
>
>1. A config setting to check
>for Unicode files.
>2. If config is TRUE, and a
>Unicode file is opened,
>automatically pop up a dialog
>box to convert it - Yes/No. If
>Yes, the conversion will set a
>new Internal Value.
>3. Upon saving, if the
>Internal Value is TRUE,
>convert it back to Unicode.
>
As the originator of this topic, I'd like to say that this approach would be perfectly acceptable to me.

-- Howard