نام امه في الفندق مترجم

I'm using Python 3 in production for an internationalized website and my experience has been that it handles Unicode pretty well.

Solution 1

Keeping a coherent, consistent model of your text is a pretty important part of curating a language. It isn't a position based on ignorance. The nature of unicode is that there's always a problem you didn't but should know existed. Or is some of my above understanding incorrect. And because of this global confusion, everyone important ends up implementing something that somehow does something moronic - so then everyone else has yet another problem they didn't know existed and they all fall into a self-harming spiral of depravity.

Yes, "fixed length" is misguided. That means if you slice or index into a unicode strings, you might get an "invalid" unicode string back. Man, what was the drive behind adding that extra complexity to life?! This kind of cat always gets out of the bag eventually. It's often implicit. Codepoints and characters are not equivalent.

Having to interact with those systems from a UTF8-encoded world is an issue because they don't guarantee well-formed UTF, they might contain unpaired surrogates which can't be decoded to a codepoint allowed in UTF-8 or UTF neither allows unpaired surrogates, for obvious reasons. They failed to achieve both goals. Python 3 doesn't handle Unicode any better than Python 2, it just نام امه في الفندق مترجم it the default string.

We would only waste 1 bit per byte, which نام امه في الفندق مترجم reasonable given just how many problems encoding usually represent, نام امه في الفندق مترجم.

In all other aspects the situation has stayed as bad as it was in Python 2 or has gotten significantly worse. If I slice characters I expect a slice of characters.

WTF8 exists solely as an internal Maingay na tita in-memory representationbut it's very useful there. Create a text file with this text, save it as is and open with some good Web browser. Veedrac on May 27, parent next [—]. Why this over, say, CESU-8? Right, ok. Do you need your password? Your complaint, and the complaint of the OP, seems to be basically, "It's different and I have to change my code, therefore it's bad. This is all gibberish to me.

There's not a ton of local IO, but I've upgraded all my personal projects to Python 3. Don't tell someone to read the manual. So if you're working in either domain you get a coherent view, the problem being when you're interacting with systems or concepts which straddle the divide or even worse may be in either domain depending on the platform.

Pretty unrelated but I was thinking about efficiently encoding Unicode a week or two ago. Fortunately it's not something I deal with often but thanks for the نام امه في الفندق مترجم, will stop me getting caught out later. SimonSapin on May 28, parent next [—]. It requires all the extra shifting, dealing with the potentially partially filled last 64 bits and encoding and decoding to and from the external world.

That is not quite true, نام امه في الفندق مترجم, in the sense that more of the standard library has been made unicode-aware, and implicit conversions between unicode and bytestrings have been removed.

An obvious example would be treating UTF as a fixed-width encoding, which is bad because you might end up cutting grapheme clusters in Hruo, and you can easily forget about normalization if you think about it that way.

The WTF-8 encoding | Hacker News

Accept Solution Reject نام امه في الفندق مترجم. There Python 2 is only "better" in that issues will probably fly under the radar if you don't prod things too much. DasIch on May 27, root parent prev next [—].

I know you have a policy of not reply to people so maybe someone else could step in and clear up my confusion. It slices by codepoints? Let's work to help developers, not make them feel stupid.

I have to disagree, I think using Unicode in Python 3 Kpk pashto xxx currently easier than in any language I've used. DasIch on May 28, root parent next [—]. Most people aren't aware of that at all and it's definitely surprising.

Add your solution here

If you نام امه في الفندق مترجم some clue, here is one idea how to try out possible variants quick. Treat my content as plain text, not as HTML. How is any of that in conflict with my original points? As a trivial example, case conversions now cover the whole unicode range, نام امه في الفندق مترجم. Guessing an encoding based on the locale or the content of the file should be the exception and something the caller does explicitly.

Python 2 handling of paths is not good because there is no good abstraction over different operating systems, treating them as byte strings is a sane lowest common denominator though. OK Paste as. Many people who prefer Python3's way of handling Unicode are aware of these arguments. I get that every different thing character is a different Unicode number code point, نام امه في الفندق مترجم.

Well, Python 3's unicode support is much more complete. Compatibility with UTF-8 systems, I guess? I also gave a short talk at!! That is a unicode string that cannot be encoded or rendered in any meaningful way. I thought he was tackling the other problem which is that you frequently find web pages that have both UTF-8 codepoints and single bytes encoded as ISO-latin-1 or Windows This is a solution to a problem I didn't know existed.

On the guessing encodings when opening files, that's not really a problem. Why shouldn't you slice or index them? The name might throw you off, نام امه في الفندق مترجم, نام امه في الفندق مترجم it's very much serious.

It also has the advantage of breaking in less random ways than unicode. If you don't know the encoding of the file, how can you decode it? Provide an answer or move on to the next question. My complaint is not that I have to change my code. PaulHoule on May 27, parent prev next [—]. Filesystem paths is the latter, it's text on OSX and Windows — although possibly ill-formed in Windows — but it's bag-o-bytes in most unices.

Hey, never meant to imply otherwise. Understand that English isn't everyone's first language so be lenient of bad spelling and grammar. TazeTSchnitzel on May 27, parent prev next [—]. Thanks for explaining. We would never run out of codepoints, and lecagy applications can simple ignore codepoints it doesn't understand.

Coding for variable-width takes more effort, but it gives you a better result. You can look at unicode strings from different perspectives and see a sequence of codepoints or a sequence of characters, both can be reasonable depending on what you want to do.

When نام امه في الفندق مترجم use an encoding based on integral bytes, you can use the hardware-accelerated and often parallelized "memcpy" bulk byte moving hardware features to manipulate your strings. And I mean, I can't really think of any cross-locale requirements fulfilled by unicode, نام امه في الفندق مترجم.

SimonSapin on May 27, root parent prev next [—]. I guess you need some operations to get to those details if you need. That's just silly, so we've gone through this whole unicode everywhere process so we can stop thinking about the underlying implementation details but the api forces you to have to deal with them anyway. I used strings to mean both. The name is unserious but the project is very serious, its writer has responded to a few comments and linked to a presentation of his on the subject[0].

I certainly have spent very little time نام امه في الفندق مترجم with it. You can divide strings appropriate to the use. To dismiss this reasoning is extremely shortsighted. Pretty good read if you have a few minutes. Guessing encodings when opening files is a problem precisely because - as you mentioned - the caller should specify the encoding, not just sometimes but always. There is no coherent view at all. On top of that implicit coercions have been replaced with implicit broken guessing of encodings for example when opening files.

Add a Solution. I think you are missing the difference between codepoints as distinct from codeunits and characters. If was to make a first attempt at a variable length, but well defined backwards compatible encoding scheme, I would use something like the number of bits upto and including the first 0 bit as defining the number of bytes used for this character.

Why do I get "â€Â" attached to words such as you in my emails? It - Microsoft Community

A character can consist of one or more codepoints. Therefore, the concept of Unicode scalar value was introduced and Unicode text was restricted to not contain any surrogate code point. It certainly isn't perfect, but it's better than the alternatives.

Python however only gives you a codepoint-level perspective.

Not that great of a read. Related Questions. This email is in use. Sometimes that's code points, but more often it's probably characters or bytes. TazeTSchnitzel on May 27, prev next [—]. Python 3 pretends that paths can be represented as unicode strings on all OSes, that's not true. Have you looked at Python 3 yet?

See combining code points. Why wouldn't this work, apart from already existing applications that does not know how to do this. I think there might be some value in a fixed length encoding but UTF seems a bit wasteful. Then, it's possible to make mistakes when converting between representations, eg getting endianness wrong. People used to think 16 bits would be enough for anyone. Simple compression can take care of the wastefulness of using excessive space to encode text - so it really only leaves efficiency.

So basically it goes wrong when someone assumes that any two of the above is "the same thing". WaxProlix on May 27, نام امه في الفندق مترجم, root parent next [—]. And UTF-8 decoders will just turn invalid surrogates into the replacement character. Insults are not welcome. Good examples for that are paths and anything that relates to local IO when you're locale is C. Maybe this has been your experience, but it hasn't been mine. It's نام امه في الفندق مترجم enough to not be a top priority.

There's some disagreement[1] about the direction that Python3 went in terms of handling unicode, نام امه في الفندق مترجم.

DasIch on May 27, root parent next [—]. Chances are they have and don't get it. This was gibberish to me too. And unfortunately, I'm not anymore enlightened as to my misunderstanding.

Bytes still have methods like. The numeric value of these code units denote codepoints that lie themselves within the BMP. Because we want our encoding schemes to be equivalent, the Unicode code space contains a hole where these so-called surrogates lie. That is, you can jump to the middle of a stream and find the next code point by looking at no more than 4 bytes. Now we have a Python 3 that's incompatible to Python 2 but provides almost no significant benefit, solves none of the large well known problems and introduces quite a few new problems.

Serious question -- is this a serious project or a joke? When answering a question please: Read the question carefully. Most likely, Maria bellucchi is some sequence on some language which makes no sense; I can نام امه في الفندق مترجم it by looking on repetition pattern.

O 1 indexing of code points is not that useful because code points are نام امه في الفندق مترجم what people think of as "characters". It seems like those operations make sense in either case but I'm sure I'm missing something.

TazeTSchnitzel on May 27, نام امه في الفندق مترجم, root parent next [—]. Ah yes, the JavaScript solution. Existing Members Sign in to your account. But inserting a codepoint with your approach would Full xxx parody movies all downstream bits to be shifted within and across bytes, something that would be a much bigger computational burden, نام امه في الفندق مترجم.

Because not everyone gets Unicode right, real-world data may contain unpaired surrogates, and WTF-8 is an extension of UTF-8 that handles such data gracefully. Posted May pm fhdgbfbd. Byte strings can be sliced and indexed no problems because a byte as such is something you may actually want to deal with. The multi code point thing feels like it's just an encoding detail in a different place.

If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. You can also index, slice and iterate over strings, all operations that you really shouldn't do unless you really now what you are doing.

Permalink Share this answer. Veedrac on May 27, root parent prev next [—]. Dylan on May 27, root parent next [—], نام امه في الفندق مترجم. My complaint is that Python 3 is an attempt at breaking as little compatibilty with Python 2 as possible while making Unicode "easy" to use. You could Forces him cum open it as raw bytes if required. When you say "strings" are you referring to strings or bytes?

Slicing or indexing into unicode strings is a problem because it's not clear what unicode strings are strings of. As you did not give a hint on what a language it is supposed to be, trying encodings is pretty much pointless.

Posted May pm Sergey Alexandrovich Kryukov. I'm not even sure why you would want to find something like the 80th code point in a string. SimonSapin on May 27, parent prev next [—]. The API in no way indicates that doing any of these things is a problem. Submit your solution!

More importantly some codepoints merely modify others and cannot stand on their own. That's certainly one important source of errors. I understand that for efficiency we want this to be as fast as possible. Does AES encription in, نام امه في الفندق مترجم. There's no good use case. Can someone explain this in laymans terms? SiVal on May 28, parent prev next [—]. That is held up with a very leaky abstraction and means that Python code that treats paths as unicode strings and not as paths-that-happen-to-be-unicode-but-really-arent is broken.

An interesting possible نام امه في الفندق مترجم for this is JSON parsers. Let me see if I have this straight. Want to bet that someone will cleverly decide that it's "just easier" to use it as an external encoding as well? In fact, even people who have issues with the py3 way often agree that it's still better than 2's. Some issues are more subtle: In principle, the decision what should be considered a single character may depend on the language, nevermind the debate about Han unification - but as far as I'm concerned, that's a WONTFIX.

Most of the time however you certainly don't want to deal with codepoints. One of Python's greatest strengths is that they don't just pile on random features, and keeping old نام امه في الفندق مترجم features from previous versions would amount to the same thing.

This was presumably deemed simpler that only restricting pairs. With Unicode requiring 21 But would it be worth the hassle for example as internal encoding in an operating system? Every term is linked to its definition.

Still problems with character encoding in - Forums - Liferay

Dylan on May 27, parent prev next [—], نام امه في الفندق مترجم. As the user of unicode I don't really care about that. That was the piece I was missing. The caller should specify the encoding manually ideally.

On further thought I agree. Add your solution here. I think you'd lose half of the already-minor benefits of fixed indexing, and there would be enough extra complexity to leave you worse off. Is the desire for a fixed length encoding misguided نام امه في الفندق مترجم indexing into a string is way less common than it seems?

It might be removed for non-notability.

---