OpenBSD Journal

Citrus is slowly being imported into OpenBSD

Contributed by phessler on from the como-estas dept.

Citrus (a BSD licensed i18n framework project) is slowly being imported into OpenBSD by Marc Espie. wchar, FILE *, mbstate_t, and some related stub functions have been added, triggering some ABI changes, and some libc bumps. Right now, its just headers and stubs being added. Full support of i18n is planned, but progress will occur very carefully.

(Comments are closed)

Comments

  1. By Johan M:son (62.119.71.154) on

    Shouldn't it be "Citrus is slowly being imported into OpenBSD"?

    Comments

    1. By phessler (208.201.244.164) on

      fixed

  2. By Anonymous Coward (82.182.149.44) on

    ... when aberrations like wchar make their way into OpenBSD

    Comments

    1. By uriel (82.182.149.44) lost.goblin@gmail.com on

      And don't get me started with abominations like locale, iconv and gettext.

      Wasn't OpenBSD supposed to stay simple and sane? And follow the True Unix Path rather than the Shitwaris and GNU madness?

      Add UTF-8 support and forget all this insanity.

      Comments

      1. By phessler (208.201.244.164) on

        this *is* utf-8 support.

        if you know of a cleaner way, then go ahead and implement it.

        Comments

        1. By uriel (82.182.149.44) on

          No, this is a mountain of crap.

          UTF-8 support is this: http://plan9.bell-labs.com/sys/doc/utf.html

          Hasn't people learned anything from the locale debacle on linux and Shitwaris?

          I certainly was not hoping to see one of the most painful and nightmarish bits of Linux imported into OpenBSD, I thought only FreeBSD and NetBSD were Linux wanabees.

          Comments

          1. By m0rf (68.104.57.241) on

            > > if you know of a cleaner way, then go ahead and implement it.
            > No, this is a mountain of crap.

            when are you going to stop talking and step up and implement your solution?

            The OpenBSD Developers have come up with a plan and are implementing it, what have you done?

            Comments

            1. By Anonymous Coward (216.107.0.44) on

              he's waiting for the wahmbulance

            2. By uriel (82.182.149.44) on

              >when are you going to stop talking and step up and implement your solution?
              What is the problem again? that OpenBSD is not Loonix or Shitwaris? Well, _that_ is the only reason I use OpenBSD, because it didn't go down the same path as people that never understood Unix.

              As for UTF-8, maybe you should check the ports tree some time, all the necessary bits are there.

              Comments

              1. By Nate (65.94.96.26) on

                You're mistaken, all the parts to integrate into the system cannot come from plan9 sources. plan9's code is not liberal enough to be used. People are telling you to either hack it together yoursef and send a patch, pay someone to do the work the way you'd prefer it or stop whining about what the developers are or aren't doing your way. You cannot complain about what someone else makes on their own time and let's you have it for free, it's called being a prick.

                Comments

                1. By uriel (82.182.149.44) on

                  > You're mistaken, all the parts to integrate into the system cannot come from plan9 sources. plan9's code is not liberal enough to be used.

                  You have not checked carefully enough:

                  http://swtch.com/plan9port/unix/ (MIT License)
                  inferno-utils.tar.gz (MIT License)

                  Most of this is already in ports, and what is missing is on it's way.

                  Comments

                  1. By Nate (65.94.96.26) on

                    There's also: this, perhaps where OpenBSD got part of it's awk.

                    In this, I concede your victory, this code could theoretically be used, but that still does not give Marc a patch.

              2. By Anonymous Coward (194.109.254.76) on

                "loonix" and "shitwaris" ? What are you, a 15-year old?

                Comments

                1. By Anonymous Coward (68.125.80.121) on

                  "loonix" and "shitwaris" ? What are you, a 15-year old?

                  Tell me about it; kids these days can't spell. It should be "Lunix", and "Slowaris". ;-P

      2. By Anonymous Coward (218.214.226.34) on

        So go use MirBSD

      3. By Marc Espie (62.212.102.210) espie@openbsd.org on

        Go on, get started. There are lots of issues UTF-8 doesn't solve, at all. UTF-8 is just an encoding, a clever one, sure, but it can't solve everything, far from it.

        Comments

        1. By uriel (82.182.149.44) on

          What issues were those again? Because I have yet to see anyone mention any valid issues that UTF-8 does not solve.

          Comments

          1. By Anonymous Coward (218.214.226.34) on

            "no valid issues" == "no problems with Western encodings". There are plenty of issues with CJK, Middle-Eastern and Indic languages.

      4. By Jörg Sonnenberger (139.30.252.72) joerg@leaf.dragonflybsd.org on

        Get the facts:
        http://www.usenix.org/publications/library/proceedings/usenix01/freenix01/full_papers/hagino/hagino_html/

        Comments

        1. By Marc Espie (62.212.102.210) espie@openbsd.org on

          I'm not sure the Han unification still is a real problem, with the latest version of Unicode, since the committee have basically recognized the issue. But everything else still stands, in particular the idea that unicode conversion can be not so practical in vast regions of the world.

          I have played enough with japanese applications to know that I will welcome a simplification of input methods, for instance...

          Comments

          1. By Anonymous Coward (210.233.106.4) on

            Unfortunately, it's not quite that easy. There's a lot of historical cruftage hanging around the Japanese (and probably CJK in general) input situation. Even on Linux distributions which have integrated IIIMF, I still see everybody using the tried-and-true combination of cannaserver and kinput2. It's nasty, crufty, unsupported, decaying code, but it's what everybody uses because it works with the applications they're used to.

    2. By Nate (65.94.96.26) on

      Dude, it's part of the POSIX spec, what do you want? OpenBSD to just completely ignore what are considered the modern methods of Unix?

      Comments

      1. By Anonymous Coward (82.182.149.44) on

        PoSix

        "Not only is UNIX dead, it's starting to smell really bad." -- Rob Pike circa 1991

        Comments

        1. By tedu (64.173.147.27) on

          erm, so why are you using openbsd? just go use plan9 and like it.

          openbsd is in many respects one of the more "conservative" unix variants.

          Comments

          1. By uriel (82.182.149.44) on

            "conservative" is quite an stretch of the word when crap like this is adopted.

            I use OpenBSD because it's the only Unix variant that remains minimally true to it's roots and to the original Unix principles and values.

            Including something like Citrus in OpenBSD is a slap in the face of those principles and values.

            Comments

            1. By Hans Insulander (192.195.135.35) on

              Being utterly useless to a majority of the population on the earth is not listed as one of OpenBSDs goals, last time I checked.

              Comments

              1. By Anonymous Coward (195.58.124.197) on

                Adhering to the bloat tradition isn’t either?

                And seriously, the generation of tomorrow will be forced to learn English anyhow!
                Do you think that we shall start writing comment in the source code in multiple languages? Start undeadly.[dk,tw,co.uk,nu] ?

                My point is: it’s more cost effective to do it all in one language!

                http://www.debian.org/doc/manuals/intro-i18n/ch-intro.en.html :
                “Displaying messages by the program in the users' native languages.” – this is where the bloat and incorrectness comes!

                Comments

                1. By Anonymous Coward (195.58.124.197) on

                  Not to do it that is!

                2. By Hans Insulander (192.195.135.35) on

                  Most of the worlds population does not understand english, and the new generation you're talking about wont either.

                  No one wants to translate the source comments, nor would anyone want localized binary names. That would be silly.

                  Why should it not be possible to use your own language when writing mails/documents/whatever in OpenBSD? All these issues were fixed in Windows/Macos and most Unixes years ago, why not fix it in OpenBSD too.
                  It's not like all these languages are going away, or that those "extra" characters are'nt needed.

                  I do however see the practical side of things with bugs and bloat, but that's not reason enough to completely ignore most of the world.

                  Comments

                  1. By uriel (82.182.149.44) on

                    What part of "use UTF-8" you don't understand?

                    gettext and whchar and the rest of that crap are _not_ UTF-8.

              2. By uriel (82.182.149.44) on

                Unix was never designed to be "localized"; still UTF-8 lets you handle any language you might ever like, but how that is done should not be a concern of the OS or it's libraries. Or should we keep symlinks with "localized names" to every command in /bin? Ah, and don't forget options! So in a Spanish locale ls -a should become ls -t(All vs. Todos), and have a symlink from /bin/cp->/bin/wc (Word Count vs. Cuenta Palabras), ah, but then we need to rename /bin/cp to /bin/copiar...

                Half the population of the world can't read or write, and about 90% aren't anywhere near a computer. That doesn't mean that OpenBSD is useless to them, or it's your computer useless to you because it only understands ASM instructions? Or is the OS the phone company runs matter one bit as to what language you talk when you call friends in Japan?

                Again, Unix was never designed to be "localized", and horrible hacks to pretend it can be done are 1) a waste of time 2) create a nightmarish abomination as a result.

                From looking at Linux and all other *NIX that have tried to do it it's rather obvious how stupid and pointless this task is. The only reason that is done is 1) to make nationalistic fundamentalists happy, 2) to make government with pointless procurement rules happy; I don't think OpenBSD should care much for either.

                Comments

                1. By Ido (80.178.196.115) on

                  You're being silly.
                  The whole computer world is English-centric, because most of the fundamental technologies were developed in the US, and any computer user must accept this fact, but it doesn't mean EVERYONE must use only the One True Language.
                  Things like translating command names are just plain stupid, that's not what at the center of discussion, IMO.

                  My English is as good as anyone's, but it's not my mother tongue.
                  I want my OS to support filenames in my mother tongue (which doesn't utilize the Latin alphabet), doubtlessly a complicated issue.
                  Do you know how to achieve this without integrating such support into the OS itself (filesystem code, system utils, etc.)?
                  Please share if you do.

                  Telling everyone to 'just use English and keep it simple' is both ignorant and patronizing.

                  Comments

                  1. By tedu (64.173.147.27) on

                    you can name files anything you want, so long as you don't use the '/' or '\0' bytes. any sequence of the remaining 254 bytes utilizing whatever encoding scheme you fancy is a valid filename.

                  2. By uriel (82.182.149.44) on

                    As tedu pointed out, file names are just byte streams, you can use whatever encoding you like for file names.

                    And if you have a half a clue, you will use UTF-8, which can encode every character ever created, and then some more.

  3. By Bert (216.175.250.42) blambert at thepresidency dot org on

    Jesus, people...

    If you don't like it, fork! I suggest AngloBSD for a name.

    Frankly, this is something that every os that isn't designed to be used solely in non-user interactive settings is eventually going to have to do. Not everyone uses European character sets (hell, even ascii lacks characters that many Europeans use).

    Not every user is going to learn English. It's a fact of life. Deal with it.

    Theo was right.
    Whiners scale really well.

    Comments

    1. By uriel (82.182.149.44) on

      Like most people in this thread, you are missing the point.

      There are two questions:

      1) Should OpenBSD let users work with text and files containing characters that are not part of ASCII? The answer is a resounding YES; and how to do it is with UTF-8, the code is already there, so there is little, if anything, left to be done.

      2) Should OpenBSD buy into the "localization" and "internationalization" hype and follow Linux and other Unixes into the nightmarish hell of locales and whchar? The answer is a resounding NO.

      Comments

      1. By Anonymous Coward (68.104.57.241) on

        it seems you are the one missing the point. It is not up to you in what direction OpenBSD goes, what they do or do not import. It is up to the developers, and if you disagree with them you are free to go use Plan 9, create urielBSD for your pure unix needs, or just go away.

        This is where OpenBSD is going, and until you actually do something beyond bitching on undeadly, you will have no influence on that.

        Comments

        1. By Anonymous Coward (68.125.80.121) on

          It is not up to you in what direction OpenBSD goes

          No, but he/she can influence it.

          Comments

          1. By Anonymous Coward (199.185.136.158) on

            you mean by yupping in here or on misc@ ?
            get real

          2. By djm@ (218.214.226.34) on

            Influence is contributed code and patches, not whining rants on web forums.

            Comments

            1. By Anonymous Coward (68.125.25.70) on

              Influence is contributed code and patches

              That helps, and so does Mindshare.

              I'm glad the person is making noise, even if that is all they can do. I welcome the attention to the possibility of a better alternative.

              I can't think about a better time to be loud about this issue considering that the Hackathon is about to start. I would be surprised if no one there brings it up atleast once.

              Comments

              1. By djm@ (203.217.30.86) on

                We have better things to worry about than rants without patches.

      2. By Lars Hansson (203.65.245.7) lars@unet.net.ph on

        Who died and made you the official OpenBSD policy maker?

    2. By Frank Denis (213.41.131.17) on http://www.00f.net

      It's not just about supporting characters sets, but also collation rules and date formats. And this is really needed. It pisses me off to reinvent the wheel on *BSD systems just because my PHP scripts don't display dates with the french format although I use the fr_FR locale.

  4. By Nate (65.94.96.26) on

    Here's a plan, those of you looking to avoid internationalization and hope for the plan9 approach, contact the people on the misc mailing list and look for more like-minded people.

    After you've gaged how many people there are bring this before the list and ask the developers if there is one amoung them looking to make some money by doing the integrating and find out how much they'd want for it.

    Ask Theo if he'd accept such a thing in if you and your posse paid the sum.

    Do a drive for the funds, pay the developer, have it commited.

    Of course, each step requires sucess at the prior step, but you should get the idea.

    Comments

    1. By uriel (82.182.149.44) on

      Can you read?

      THE CODE TO SUPPORT UTF-8 IS ALREADY IN THE PORTS TREE!
      (and most of the OS dosen't care about it anyway)

      What we are trying here is to save OpenBSD from being infested with one of the worst Linux/PoSix pests, what should we provide? a reverse patch removing all that crap? If so, it will take not much more than ten minutes to generate one.

      Comments

      1. By Marc Espie (62.212.102.210) espie@openbsd.org on

        Considering the number of things I've done with qt, I'm most certainly aware there is UTF-8 code in the tree. I've also gotten frustrated at interesting stuff that just won't compile, or won't do the right thing a number of time. Very often, just because we don't have locale support at all. You can rant all you want, but until you show us proof of the contrary, there are litterally hundreds of specialized applications that don't run, or don't run correctly, on OpenBSD thanks to the lack of i18n support. I don't see you spending any kind of time trying to port them. I don't see you trying to influence the people who write these applications so that they don't use any i18n and rely on UTF-8 instead. That said, I don't really give a fuck that wchar is not a perfect solution. That's what is used today, and having it around means more useful code gets to run. It's not especially big either. As a matter of fact, I was expecting things to get more complicated, and more breakage to occur. So far, adding i18n support proves fairly uneventful.

        Comments

        1. By uriel (82.182.149.44) on

          You seem to have been fortunate enough to not have needed to use that "locale aware" software on Linux or other systems.

          Most packages that use locales are _broken_, I have got quite a few of them to crash or produce random behavior just based on the locale settings, even when the input was plain ASCII!

          Locale handling on Unix is a joke, it has never worked, and it never will, pretending that adding that crap to OpenBSD will be not create a nightmare of broken crud is being delusional.

          Comments

          1. By Anonymous Coward (203.217.30.86) on

            App breakage on Linux isn't evidence that it won't work on OpenBSD. If you break something, we will fix it.

      2. By Damien (213.41.138.23) on

        One of OpenBSD goals is to follow standards and AFAIK wchar is ISO C and POSIX.
        Now if you claim you don't want a unix system that respect such standards i'm supposing you completely miss the point.

        Also what exactly is the problem with it ? You give no argument that demonstrate it is "bad". And if there is one thing that is good with linux distributions then it's certainly the fact they try to implement standards.

Latest Articles

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]