Hello Tommi,

On Tuesday November 07 2023 08:05, you wrote to me:

Ok. I've temporarily checked out the commit prior to the "Zero
Conversion" commit, but I can easily go back to normal. Gives me
a chance to learn a little more about git than just to clone and
pull (I'm not a programmer). ;)

How did you do that? I'm not a programmer either, I know only git
clone, stash and pull. :)

'git log' to see the commit IDs
'git checkout [commit id prior to the zero conversion commit]'
'make'

Once the fix was introduced, I just did

'git checkout master'

and it brought me back to the latest.

BTW, The level 4 is back!

I see that! All is well again, and nothing blew up. ;)

Regards,
Nick

... "Take my advice, I don't use it anyway."
--- GoldED+/LNX 1.1.5-b20231106
* Origin: thePharcyde_ distribution system (Wisconsin) (1:154/10)

On 07 Nov 2023, Nicholas Boel said the following...

2) Is this where iconv support would be beneficial, when an incoming message has a CP866 kludge, iconv would translate it to UTF-8 on my end
so that it is readable and writable, and then translate it back to CP866 when the reply is sent? Or would it still stay UTF-8 because I'm forcing it on export?

this is exactly how it works in the IRC client i use. default is UTF-8 internally, and on input from user, but you can set server, channel, nickname level with personalized encoding.

http://epicsol.org/encoding

it's nice that the fido message has a hint, but if it's wrong, or you know a whole ftn is in russian for example, it would be useful to set that whole net or a specific person that is misconfigured to default to what you want instead.

--- Mystic BBS v1.12 A47 2021/12/25 (Windows/32)
* Origin: cold fusion - cfbbs.net - grand rapids, mi (1:120/616)

Hello fusion,

On Tuesday November 07 2023 21:33, you wrote to me:

it's nice that the fido message has a hint, but if it's wrong, or you
know a whole ftn is in russian for example, it would be useful to set
that whole net or a specific person that is misconfigured to default
to what you want instead.

Agreed. I believe that was part of Michiel's recent request to make it configurable and usable (multiple times even) via the 'group' keyword.

Regards,
Nick

... "Take my advice, I don't use it anyway."
--- GoldED+/LNX 1.1.5-b20231106
* Origin: thePharcyde_ distribution system (Wisconsin) (1:154/10)

Hello Nicholas.

07 Nov 23 16:31, you wrote to me:

Git is somewhat complex, but once you got the idea - you may do a
lot of cool things which other code versioning systems cant. Like
reorder commits, move one branch on top of another, etc.

I probably won't ever be reordering or anything super technical, but I googled the easiest way, and 'checkout' seemed to do the trick. Then I just went back to the master branch using the same option.

You just don't need it which is totally fine. :) And I'm just saying that git is very powerful tool if you know how to use it.

If you need some help with that - I'll be glad to do it. It's
just not right echo area for those questions. You may shoot me
netmail too.

I may have to take you up on that offer some day. Thank you! ;)

Sure.

You may convert any charset to UTF-8 actually. And it would be
really cool to have UTF-8 support in GoldEd. I'm still learning
the code. Will try to improve things in that area.

So here's a question or two for you. As of right now I'm using:

xlatimport cp437 (because most incoming messages missing a CHRS kludge falls under this)

xlatexport utf-8 (because that's what I can write with)
xlatlocalset utf-8 (because this is my local setup, 'locale' gives en_US.UTF-8 for everything)

Basically I'm forcing the use of utf-8 when exporting messages, but we have already witnessed that that doesn't work when you write to me
using CP866 and I reply back to you.

1) Is there a way for me to reply to you with the same charset (or
closest translation) that you're using automatically? Or would I have
to change my config file every time I reply to a different CHRS
kludge?

Not automatic, and there is a feature request to do this. I may work on this request when finish charset conversion refactoring.

But! You can do it manually. You may create separate message template and add in the beginning @XLatExport CP866 (or other). It's way inconvenient, but it works. This way I use US-ASCII while my standard export charset is CP866 (I mostly write to Russian echoes). You may switch templates when write new messages or answer to somebody.

So if you have only few desired charsets - it will work for you.

2) Is this where iconv support would be beneficial, when an incoming message has a CP866 kludge, iconv would translate it to UTF-8 on my
end so that it is readable and writable, and then translate it back to CP866 when the reply is sent? Or would it still stay UTF-8 because I'm forcing it on export?

Let me explain how this machinery works and then you'll have less questions if any. :)

For charset translation GoldEd uses three main keywords:
XlatImport - which charset to use if message doesn't have CHRS kludge. XlatLocalSet - your local charset.
XlatExportSet - which craset to use to write! message. This is important.

To convert between each pair you need to have translation tables configured.

Now lets go through some scenario.

XlatImport cp437
XlatLocalSet utf-8
XlatExport utf-8

1. You receive message with CHRS CP866 2.
2. Ignore XlatImport because we have CHRS kludge. Use CP866.
3. Lookup conversion table between CP866 and UTF-8. If it exists - great, use it and convert message to UTF-8 (and you even won't lose any letters because all symbols from CP866 may be converted to UTF-8.

That's for reading part. Now you decide to answer.

Editor will work in UTF-8 and converted message will be displayed fine (there are some issues with UTF-8 there, but let's skip it for now).

1. You write some stuff using English, Russian and let's say Arab letters.
2. When you save it to message base, GoldEd will lookup conversion table from UTF-8 (XLatLocalSet) to UTF-8 (XlatExport). As long as you have it, even if it's really fake, but in this case it works because that's one-to-one conversion, GoldEd will save message perfectly fine in UTF-8.

The problem with this approach is that people which have different local charset (like me) won't see non-english chars.

Now let's imagine that you have XlatExport set to cp437, but XlatLocalSet to utf-8.

When message is being saved, GoldEd would try to find conversion table from utf-8 and cp437. Such table in GoldEd just impossible, because it cannot convert from multibyte charsets to other charsets. So it will just save it as UTF-8 text. I'm not sure which CHRS kludge will be used, but that's very easy to check. :)

Even more. When it converts from one-byte to multibyte charset - it can only use up to three bytes per symbol. Which is enough for most languages, but still is a limitation.

Would iconv improve this situation - yes! You'll be able to convert from multibyte charsets. Sure, some symbols may be lost if they don't exist in target charset. For example if I convert from UTF-8 to KOI8-R, I'd lose German umlauts. :)

Hope that answers your question. If you have more - I'll try to answer them.

I see. And you use that fake conversion table from/to UTF-8,
right? If you do - my next small fix will help you for sure.

We can consider this message my first test using the latest commit. ;)

Hope it works!

It works. Now I see "UTF-8 4".

Vitaliy

... ?u?a??e ?e??y c?po? - ?a? ?u?o??a ?e 6??ae? o?e?a?o?.
--- GoldED+/LNX 1.1.5-b20231030
* Origin: Aurora, Colorado (1:104/117)

Hello Nicholas.

08 Nov 23 05:46, you wrote to fusion:

it's nice that the fido message has a hint, but if it's wrong, or
you know a whole ftn is in russian for example, it would be
useful to set that whole net or a specific person that is
misconfigured to default to what you want instead.

Agreed. I believe that was part of Michiel's recent request to make it configurable and usable (multiple times even) via the 'group' keyword.

XlatImport/XlatExport may be configured in groups already. I didn't get too deep in Michael's request yet, but I'll investigate if that's possible and make sense to do.

Vitaliy

... Xopo?o ?o??a co6a?a - ?py?, ?o ??oxo, ?o??a ?py? - co6a?a.
--- GoldED+/LNX 1.1.5-b20231030
* Origin: Aurora, Colorado (1:104/117)

Hello Nicholas.

07 Nov 23 16:47, you wrote to me:

Ivonv support do work in some cases, but it doesn't cover all the
cases and you still need translation tables, which is odd. My
plan is to use iconv without translation tables.

I think when I tried it last (years ago) it broke more things than it fixed.

It is incompleted work. And it implemented in weird way. And most importantly it wont work with source multibyte charset. But it will after my changes.

I will definitely follow your work and help with testing where I can!

Great. The more testers available with different configurations - the better.

To set correct charset level in kludge I may need to introduce
new configuration parameter, because iconv know nothing about
FidoNet charset levels obviously.

Is the level parameter even needed? I don't think anything actually
uses it for whatever intended purpose it once had. If it wasn't there
to begin with, I would have never saw the change, and you could have continued with your work without distraction. ;)

That parameter is not needed. Conversion code doesn't use it for actual conversion. But it's part of standard and need to be provided. :)

Vitaliy

--- GoldED+/LNX 1.1.5-b20231030
* Origin: Aurora, Colorado (1:104/117)

Hello Vitaliy,

On Wednesday November 08 2023 11:50, you wrote to me:

Would iconv improve this situation - yes! You'll be able to convert
from multibyte charsets. Sure, some symbols may be lost if they don't exist in target charset. For example if I convert from UTF-8 to
KOI8-R, I'd lose German umlauts. :)

Definitely answered any questions I may have had, and also confirmed how I viewed those three xlat settings. You also made me a firm believer that iconv could quite possibly completely replace these character translation tables once and for all! ;)

Regards,
Nick

... "Take my advice, I don't use it anyway."
--- GoldED+/LNX 1.1.5-b20231106
* Origin: thePharcyde_ distribution system (Wisconsin) (1:154/10)

Hello Nicholas.

08 Nov 23 17:40, you wrote to me:

Would iconv improve this situation - yes! You'll be able to
convert from multibyte charsets. Sure, some symbols may be lost
if they don't exist in target charset. For example if I convert
from UTF-8 to KOI8-R, I'd lose German umlauts. :)

Definitely answered any questions I may have had, and also confirmed
how I viewed those three xlat settings. You also made me a firm
believer that iconv could quite possibly completely replace these character translation tables once and for all! ;)

Glad to help. :)

I'm not sure how much time that change take. Don't have too much free time now. I'll post update here.

Vitaliy

--- GoldED+/LNX 1.1.5-b20231030
* Origin: Aurora, Colorado (1:104/117)

Hello Vitaliy,

On Wednesday November 08 2023 18:13, you wrote to me:

I'm not sure how much time that change take. Don't have too much free
time now. I'll post update here.

I can't imagine it's an easy task, but I'll definitely keep my eye out.

Regards,
Nick

... "Take my advice, I don't use it anyway."
--- GoldED+/LNX 1.1.5-b20231106
* Origin: thePharcyde_ distribution system (Wisconsin) (1:154/10)

Legend: (A) Added, (C) Copied, (D) Deleted, (M) Modified, (R) Renamed,
(T) Type changed, (U) Unmerged, (X) Unknown, (B) Pairing Broken ===========================
fix buffer overrun during headers display
Author: Vitaliy Aksyonov <18148062+vitaliy-aksyonov@users.noreply.github.com> Date: 2023-11-12 23:18:12 -0700
Committed by: Vitaliy Aksyonov <18148062+vitaliy-aksyonov@users.noreply.github.com>
Files:
M golded3/gemenu.cpp
===========================
fix import from file or clipboard with tabs.
Author: Vitaliy Aksyonov <18148062+vitaliy-aksyonov@users.noreply.github.com> Date: 2023-11-12 23:00:06 -0700
Committed by: Vitaliy Aksyonov <18148062+vitaliy-aksyonov@users.noreply.github.com>
Files:
M golded.spec
M golded3/geedit2.cpp
M srcdate.h
--- hpt/lnx 1.9.0
* Origin: Moscow, Russia (2:5020/1042.3)

Legend: (A) Added, (C) Copied, (D) Deleted, (M) Modified, (R) Renamed,
(T) Type changed, (U) Unmerged, (X) Unknown, (B) Pairing Broken ===========================
call setlocale() before initscr() (#86)
See section Initialization in man 3 ncurses. https://www.man7.org/linux/man-pages/man3/ncurses.3x.html
Locale shall be initialized before ncurses initialization.
Author: Vitaliy Aksyonov <18148062+vitaliy-aksyonov@users.noreply.github.com> Date: 2024-03-06 13:38:52 -0700
Committed by: GitHub <noreply@github.com>
Files:
M goldlib/gcui/gkbdbase.cpp
M goldlib/gcui/gvidinit.cpp
===========================
fix loading multiple spellchecker dictionaries (#84)
strtok() is used down the stack in schecker.Load() and destroys internal
state.
Only first dictionary was loaded because of this.
Author: Vitaliy Aksyonov <18148062+vitaliy-aksyonov@users.noreply.github.com> Date: 2024-03-06 13:01:07 -0700
Committed by: GitHub <noreply@github.com>
Files:
M golded3/geedit.cpp
===========================
remove space in front of ctzoffset and otzoffset (#85)
Author: Vitaliy Aksyonov <18148062+vitaliy-aksyonov@users.noreply.github.com> Date: 2024-03-06 12:32:22 -0700
Committed by: GitHub <noreply@github.com>
Files:
M golded.spec
M golded3/gemsgs.cpp
M srcdate.h
===========================
add Synchronet msgs.ini support (#83)
resolve #77
Author: Vitaliy Aksyonov <18148062+vitaliy-aksyonov@users.noreply.github.com> Date: 2024-03-06 06:28:26 -0700
Committed by: GitHub <noreply@github.com>
Files:
M cfgs/config/advanced.cfg
M cfgs/config/advanced.rus
M docs/notework.rus
M docs/notework.txt
M golded.spec
M goldlib/gall/CMakeLists.txt
M goldlib/gall/gall.all
A goldlib/gall/giniprsr.cpp
A goldlib/gall/giniprsr.h
M goldlib/gall/gstrall.h
M goldlib/gall/gstrutil.cpp
M goldlib/gcfg/gedacfg.cpp
M goldlib/gcfg/gxsync.cpp
M srcdate.h
M windows/golded.dep
M windows/golded.dsp
M windows/goldlib.dep
M windows/goldlib.dsp
M windows/goldlib.mak
M windows/goldlib.vcproj
M windows/goldlib.vcxproj
--- hpt/lnx 1.9.0
* Origin: Moscow, Russia (2:5020/1042.3)

Legend: (A) Added, (C) Copied, (D) Deleted, (M) Modified, (R) Renamed,
(T) Type changed, (U) Unmerged, (X) Unknown, (B) Pairing Broken ===========================
fix crash when reading removed message in Squish (#87)
Fixed a bug/crash when GoldED opens a message in the current area which
has been deleted outside of GoldED.

This bug leads to enormous memory allocation size and GoldED will either
be killed (OOMKiller) or exit with the Squish msgbase corruption message
in log.

Co-authored-by: Nil Alexandrov <nil.alexandrov@gmail.com>
Author: Vitaliy Aksyonov <18148062+vitaliy-aksyonov@users.noreply.github.com> Date: 2024-03-10 06:08:12 -0600
Committed by: GitHub <noreply@github.com>
Files:
M golded.spec
M goldlib/gmb3/gmosqsh3.cpp
M srcdate.h
--- hpt/lnx 1.9.0
* Origin: Moscow, Russia (2:5020/1042.3)

Legend: (A) Added, (C) Copied, (D) Deleted, (M) Modified, (R) Renamed,
(T) Type changed, (U) Unmerged, (X) Unknown, (B) Pairing Broken ===========================
add uncrustify settings file
Author: Michael Dukelsky <dukelsky@users.noreply.github.com>
Date: 2024-05-31 21:34:48 +0300
Committed by: Michael Dukelsky <dukelsky@users.noreply.github.com>
Files:
A uncrustify70.1.cfg
--- hpt/lnx 1.9.0
* Origin: Moscow, Russia (2:5020/1042.3)