Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrade to new Unicode spec 16.0.0+ #5537

Open
solardiz opened this issue Sep 16, 2024 · 1 comment · May be fixed by #5580
Open

Upgrade to new Unicode spec 16.0.0+ #5537

solardiz opened this issue Sep 16, 2024 · 1 comment · May be fixed by #5580
Assignees
@magnumripper
Labels
enhancement

Comments

@solardiz
Copy link
Member

Our last update seems here:

commit 74e896be8d7dd59c744eda3c6a6bc22ab9d29e15
Author: magnum <[email protected]>
Date:   Tue Aug 25 16:03:26 2020 +0200

    Upgrade Unicode stuff from version 11.0.0 to 13.0.0.

Current is apparently 16.0.0: https://x.com/thorsheim/status/1835612002237972490

Per Thorsheim
@thorsheim
Unicode 16.0.0 adds 5185 new symbols, including 7 complete alphabets from India, Albania and West Africa. Unicoe 16.0.0 is now 154998 symbols in total.

Can't wait to see what @CrackMeIfYouCan will do with this next year! :D

@hashcat @solardiz @yiannistox
@magnumripper
Copy link
Member

Unicode 16.0.0 adds 5185 new symbols

In our case it adds 11,131 new characters, as we missed version 14 and 15. I consider the new characters mostly useless but we have fairly cool scripts that just parses official Unicode Database files and produce character tables, defines classes, adds casing logic and so on.

magnumripper added a commit to magnumripper/john that referenced this issue Nov 14, 2024
@magnumripper
Update our Unicode and legacy codepage support to Unicode 16.0.0
ad67edf 
We missed a couple of releases so this adds 11,131 new characters
compared to our previous version 13.0.0.  These are mostly obscure
alphabets and emojis (I'm just guessing) that are not super likely
to be included in any password.

There were also a few minor changes to definitions of categories such
as punctuation and specials - we always used the Unicode Database's
definition of them and apparently they changed their mind about a few.

Closes openwall#5537
@magnumripper magnumripper linked a pull request Nov 14, 2024 that will close this issue
Open
magnumripper added a commit that referenced this issue Nov 14, 2024
@magnumripper
Update our Unicode and legacy codepage support to Unicode 16.0.0
dc4d519 
We missed a couple of releases so this adds 11,131 new characters
compared to our previous version 13.0.0.  These are mostly obscure
alphabets and emojis (I'm just guessing) that are not super likely
to be included in any password.

There were also a few minor changes to definitions of categories such
as punctuation and specials - we always used the Unicode Database's
definition of them and apparently they changed their mind about a few.

Closes #5537
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@magnumripper magnumripper

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update our Unicode and legacy codepage support to Unicode 16.0.0 magnumripper/john
2 participants
@magnumripper @solardiz