Many keyboard shortcuts stopped working after upgrading to 3.25.0

[jason-wihardja]

I have multiple keyboard shortcuts set for multiple apps. None of them are working after updating to 3.25.0.

A few things about my setup that might be useful for tracing issues:

• MacOS Sequoia
• Karabiner Elements installed
• Mac Mouse Fix installed

Previously this setup worked fine. I can confirm this, because downgrading to 3.24.2 makes everything work again.

[zenangst]

Hey @jason-wihardja

Mind sharing which key combinations stopped working?

[jason-wihardja]

Would it be useful if I upload my entire config file?
config.json

Using this config, even the close tab action within default group stopped working

[zenangst]

Thanks @jason-wihardja,

I'm working on some of the reported issues now.
You supplying a config.json will make this debugging a lot easier, thanks!

[jason-wihardja]

I am using a macropad to support my productivity, so you may find some weird or unusual key combinations, but those are the keys I assigned to my macropad. Normally, my keystroke pipeline looks something like this:

1. My macropad's original signal (for example the letter A/B)
2. Driver maps the signal to some keystrokes (let's assume F15 and F17)
3. Keyboard cowboy captures F15 and F17
4. Do the action

Is there any change to the way you capture keystrokes? For example, instead of capturing the F15/F17 that the driver sends, the software unintentionally captures A/B.

[zenangst]

@jason-wihardja Interesting, so it is all the F-key related shortcuts that have stopped working for you, correct? The other keys still work as intended?

I did some problems with the migration code (moving the config), that was my first guess that something went wrong there. Did manage to fix another bug, so that is good.

Just need to figure out how I can emulate these keystrokes as I don't have a keyboard with those function keys. Doing some additional research around it now.

[jason-wihardja]

Upon further inspection, it's actually inconclusive. For example, in Microsoft Edge group here, previous tab works while next tab doesn't work. History, which is a F-key, also doesn't work

[zenangst]

@jason-wihardja we just released version 3.25.1, mind taking that for a spin to see if things work like it should now.

https://github.com/zenangst/KeyboardCowboy/releases/tag/3.25.1

[jason-wihardja]

Sadly it still doesn't work. For example, in Spotify (using the previously provided config), hitting Home/End would result in increase/decrease volume. On my machine, it behaves like how the Home/End button normally work (go to the top/bottom of the view)

[zenangst]

@jason-wihardja would you mind compiling a list of shortcuts that don't work so that I can write some tests for those specific combinations.

Also, one thing worth trying is to try and re-record the shortcuts, just to verify that there isn't a mismatch between old and new configuration and key mapping and lookup.

[jason-wihardja]

Key	Works
F13
F16
F18
F19
F20
Home
End
Cmd + Ctrl + Shift + Opt + Left Arrow
Cmd + Ctrl + Shift + Opt + Right Arrow

I have also tried re-recording the shortcuts, and it didn't work. There is no difference between the old shortcut and the newly recorded one. The config looks the same.

[jason-wihardja]

While testing for the above keys, I also have Karabiner Event Viewer on all the time. I can verify that there's no difference in the signals being sent from my device and for buttons available in my keyboard (not my macropad), like Home, End, Cmd + Ctrl + Shift + Opt + Left Arrow, Keyboard Cowboy also fails to detect them.

Is there anything significant you change the way this app capture keys? Because version 3.24.2 works flawlessly while version 3.25.0 and 3.25.1 don't. As I apparently rely a lot more on the buttons that don't work than the buttons that work for my daily workflow, I have no other option than to downgrade to 3.24.2 for now.

[zenangst]

@jason-wihardja 3.25.0 did include a complete rewrite of the key lookup tables. Shame that this led to you having to downgrade. As soon as I'm done with the CPU & memory improvements, this will be first in line to get fixed.

[zenangst]

@jason-wihardja just a minor follow-up, so that I'm not chasing a red herring. Would you mind enabling notifications for running workflows in the application preferences, just to verify if this is related to the keyboard shortcut trigger or the command that should forward keys to the system?

[zenangst]

@jason-wihardja Hey Jason, I've got some good news! I finally figured out why Home & End didn't map properly. I have a fix for that. But I still can't seem to reproduce the issue where you use Cmd + Ctrl + Shift + Opt + Left Arrow.

Can you send me the flags that are sent when you press those keys? I'm not sure what Karabiner Event Viewer outputs, but I'm looking for the raw value for the CGEventFlags.

I use Key Codes (https://manytricks.com/keycodes/) to look them up.

Here's what I get when it's working for me.

[zenangst]

@jason-wihardja here's an early release for you to try out!

Keyboard Cowboy 3.25.2.dmg.zip

[jason-wihardja]

@jason-wihardja just a minor follow-up, so that I'm not chasing a red herring. Would you mind enabling notifications for running workflows in the application preferences, just to verify if this is related to the keyboard shortcut trigger or the command that should forward keys to the system?

I tried this option, but no notifications appeared, even for the working ones

[jason-wihardja]

@jason-wihardja Hey Jason, I've got some good news! I finally figured out why Home & End didn't map properly. I have a fix for that. But I still can't seem to reproduce the issue where you use Cmd + Ctrl + Shift + Opt + Left Arrow.

Can you send me the flags that are sent when you press those keys? I'm not sure what Karabiner Event Viewer outputs, but I'm looking for the raw value for the CGEventFlags.

I use Key Codes (manytricks.com/keycodes) to look them up.

Here's what I get when it's working for me.

Here are the keycodes for every key combination I listed on the table above

(the last one is F20, the Key Codes app fails to display the character/key)

Using v3.52.2, I can confirm that Home/End now works. But other non-working keys are still not working.

[zenangst]

Hey @jason-wihardja,

I'm still trying to figure this out. If you don't mind, I'd like to test your configuration in the test suite so I can use it for future regression testing. Is that okay with you?

I tried to match Control+Option+Command+ArrowRight against your configuration, but I can't find any reference to it. Also, it seems a bit odd that F20 doesn't have any modifiers attached to it. Maybe it's a bug in Key Codes, since I can't test it locally (I don't have a keyboard with more than 12 F-keys). In theory, it should share the same modifiers as the rest of the F-keys.

By the way, what are you using Karabiner Elements for? I just want to make sure these two applications aren't actually in conflict with each other.

Table Overview

Key	Works
F13
F16
F18
F19
F20
Home
End
Cmd + Ctrl + Shift + Opt + Left Arrow
Cmd + Ctrl + Shift + Opt + Right Arrow

[zenangst]

@jason-wihardja I just tried your configuration locally, and the Cmd + Ctrl + Shift + Opt + Left Arrow work as intended in Visual Studio Code & Spotify.

[jason-wihardja]

Sure, you may use the config. I'm currently away for a business trip, so I don't have my macropad with me right now. Will test again the key combination once I get back home.

And yes, seems like I included Control+Option+Command+ArrowRight by mistake. It's a key that Karabiner captured.

For context, my karabiner captured different key combinations than the one being mapped in Keyboard Cowboy, in particular the ones that work globally regardless of app. I mainly use Keyboard Cowboy for key combinations that have different actions depending on apps, which is something Karabiner can't really do easily. This has never triggered any conflict before.

[zenangst]

@jason-wihardja could you let me know when you have time to re-test the latest version?
While at work, I checked the standard Apple keyboards and their F-keys only reach F19.
So perhaps it is correct that the additional modifiers should be dropped for that key.

[jason-wihardja]

Ok, so I've gotten back from the trip. I noticed that you've released v3.25.3. Let me re-test them. And what do you mean by additional modifiers should be dropped? I didn't press cmd, shift, or any other modifier when hitting F20. Previously it maps just fine. But let me re-test everything first and report back to you

[jason-wihardja]

So, after I test some more, it seems that for me, the behavior is the same as before. It's still not working. Using Key Codes, I can see that for buttons that are mapped successfully, Key Codes now shows a different keystroke, which is the key I map the button into, but for buttons that are not working, the behavior is still the same. It is as if the Keyboard Cowboy app is not running

[jason-wihardja]

I don't know whether this has something to do with the issue or not, but apparently when I ticked the Log key up events and modifier changes option in Key Codes, apparently my macropad only fire key down and key up event for the entire Control+Option+Command+ArrowRight action (as seen in the first two entries in the screenshot). Meanwhile from the third entry until the end shows how a normal Control+Option+Command+ArrowRight is pressed on a keyboard

[zenangst]

And what do you mean by additional modifiers should be dropped? I didn't press cmd, shift, or any other modifier when hitting F20.

What I meant was, if you compare F13 or F16 with F20, you'll notice that the flags for F1-F19 are different.

Here's the flags for F1-F19:

NX_SECONDARYFNMASK / kCGEventFlagMaskSecondaryFn / NSFunctionKeyMask
	= 8388608 / 0x800000
NX_NONCOALSESCEDMASK / kCGEventFlagMaskNonCoalesced
	= 256 / 0x100

Now, let's look at your F20:

NX_NONCOALSESCEDMASK / kCGEventFlagMaskNonCoalesced
	= 256 / 0x100

I made an assumption that all F-keys should use the function key mask. However, I believe this assumption was incorrect. The keys from F1 to F19 should have the function key mask, while the remaining keys should use the non-coalesced flag.

[CosmicToast]

This is likely related (my bindings also broke with 3.25.0, I only got around to trying to report now).
My situation in particular is that I bind cmd-shift-[…], which works, but not if I use my caps lock (which is rebound by the system to cmd).

For example, I use cmd-shift-k to open iterm with kalker.
As per key codes, the modifiers are different:
capslock as cmd + shift + k: 0x28 keycode, 0x120112 modifiers:

• NX_SHIFTMASK / kCGEventFlagMaskShift / NSShiftKeyMask = 131072 / 0x20000
• NX_DEVICELSHIFTKEYMASK = 2 / 0x2
• NX_COMMANDMASK / kCGEventFlagMaskCommand / NSCommandKeyMask = 1048576 / 0x100000
• NX_DEVICERCMDKEYMASK = 16 / 0x10
• NX_NONCOALSESCEDMASK / kCGEventFlagMaskNonCoalesced = 256 / 0x100

real cmd + shift + k: 0x28 keycode, 0x12010a modifiers:

• NX_SHIFTMASK / kCGEventFlagMaskShift / NSShiftKeyMask = 131072 / 0x20000
• NX_DEVICELSHIFTKEYMASK = 2 / 0x2
• NX_COMMANDMASK / kCGEventFlagMaskCommand / NSCommandKeyMask = 1048576 / 0x100000
• NX_DEVICELCMDKEYMASK = 8 / 0x8
• NX_NONCOALSESCEDMASK / kCGEventFlagMaskNonCoalesced = 256 / 0x100

The difference is that the caps lock cmd variant is reported as RCMD (right side cmd) rather than LCMD (left side cmd).
I'm noticing that, indeed, if I use the right cmd it doesn't go through either.
In 3.25.2, I also noticed that if I define a shortcut with the right cmd (e.g. rcmd, shift, k) then neither the right nor the left cmd can trigger it.

Looks like you internally treat the two as equivalent (despite their different codes), as per ShortcutResolverTests.swift // Verify Command + Control + Shift + Option + Left Arrow.

One approach you could take is instead of checking directly for RCMD or LCMD, instead checking for >0 when applying a 0x18 mask. (i.e. cmd is in use if 0x18 & modifiers > 0), but this is just me guessing based on looking around in the code for a few minutes.
I'm fairly short on time lately, but if you need any help I could try and carve out an hour or two maybe next weekend to help out (at least with this particular instance of the problem, and try to see if there's any other patterns along these lines; though note I have no swift experience so far).

[zenangst]

Thanks a bunch for your response, @CosmicToast!

I'm curious, are you using third-party software to remap your caps lock key to command, or did you do that through System Settings? I'm wondering why it ends up as the right command key, maybe that's by design.

While trying to figure this out, I found a small issue with how keyboard shortcuts are registered in Keyboard Cowboy. It doesn't check if each key is in the correct left or right position. I think this might be part of the problem we're seeing here.

So, if you hold down left-shift + right command, it will be registered as right-shift & right command. Only one of the keys have to be on the right side for all of them to be registered as such.

[zenangst]

@CosmicToast @jason-wihardja found a minor issue in the previous version that I shared (so I deleted that comment). Here is a freshly brewed version.
Keyboard Cowboy 3.25.4.dmg.zip

[CosmicToast]

Running straight from it, the behaviour is a bit different:
When I record the shortcut using the rebound caps, it now registers as left side.
So I can't execute it using the same shortcut (using caps), but I can now execute it using the left side cmd (but not the right side cmd).

I'm curious, are you using third-party software to remap your caps lock key to command, or did you do that through System Settings? I'm wondering why it ends up as the right command key, maybe that's by design.

System settings, probably coincidental (they had to pick one or the other, in the end).

[zenangst]

@CosmicToast

Thanks for testing the new version.

I've started a major overhaul of the key capture process. This way, each key is clearly marked, like the left or right command. I think this will fix the mix-ups.
As mentioned earlier, the flaw in the design that we have right now is that keys will either be reported as being a part of the left or right side, not both.

I set up Karbiner elements so that my caps lock key can either be the escape key (when tapped) or the left control key (when held). I know this might not be the most straightforward way to do it, but it's the way I chose to avoid any confusion.

But this won't solve the problem with the caps lock key being reported as the right command. If you want to use both the left and right sides at the same time, you'll need to change the workflows for different combinations.

I'll keep you posted on any progress that I make.

Thanks for your patience.

[zenangst]

@CosmicToast @jason-wihardja

Here is an update on the current state of things.
I had a long code session with a friend this evening, and we looked into implementing uniqueness for individual modifiers.

What this means is that you can now make commands that involve left, right, or both of the command modifiers (this was not possible before based on the way we determined the modifiers' position).

In order to do this, we had to perform a migration to the new data structure.
This means that some of the bindings (guessing the ones that are not currently working) would have to be re-recorded. All other existing recordings (that don't mix and match left & right modifiers) should be migrated just fine.

In addition to making the transition to the new data structure, a backup will be created of the current configuration and stored in ~/.config/keyboardcowboy/backups/ with the current format:

config.{currentBuild}.json

This should help with rolling back to older versions if we need additional changes to the data structure in the future.

I've successfully migrated my own configuration without hitches (thus far).
Taking it for a spin tomorrow at work to see if there are any additional bugs that need smashing. If I come up empty, would you like to be the first to try out this new version to see if that brings us closer to the finish line?

[jason-wihardja]

Sure. Would love to try that out. Because so far I had no other solutions other than downgrade to 3.24.2. Although I believe that @CosmicToast's problem may be different compared to mine, but who knows. The new data structure hopefully can solve both problems.

[zenangst]

@jason-wihardja @CosmicToast here is another pre-release version that you can test with.
Note the previous things that I mentioned: it will create a backup before migrating to a new data structure. Hopefully, you have your own backups as well (just to be safe).

Keep in mind that you also might have to re-record the keyboard shortcuts that previously didn't work as intended.

Keyboard Cowboy 3.25.4.dmg.zip

[jason-wihardja]

Ok, so I've tried the new one and here's how the re-recorded config looks like
config.json

From this config, the only non-working buttons are:

• F20
• Ctrl + Cmd + Alt + Shift + Left Arrow (the config somehow recorded an extra fn being pressed)
• Ctrl + Cmd + Alt + Shift + Right Arrow (the config also somehow recorded an extra fn being pressed)

[zenangst]

@jason-wihardja Thanks for following up. I'll take another look at your config when life allows and continue to dig into this.

I have yet to do anything about F20, will also take a look at that one.
Are all of the modifiers that you are using located on the left side of the keyboard?

[jason-wihardja]

Yes. everything on the left side.

My workaround for the F20 is to now change the mapping in my macropad to Insert button, which Keyboard Cowboy detects just fine in this version. Let me know when you had done some work on the F20 button. I'll change my macropad back to F20.

[zenangst]

@jason-wihardja just tried to record Ctrl + Cmd + Alt + Shift + Left Arrow, and it does in fact record it with the function key. But that seems correct as all arrow keys should include the function modifier.

Key Codes include the same modifier flags when I test it there.

[jason-wihardja]

I see. Your screenshot seems to be similar to my previous screenshot here

I don't know whether this has something to do with the issue or not, but apparently when I ticked the Log key up events and modifier changes option in Key Codes, apparently my macropad only fire key down and key up event for the entire Control+Option+Command+ArrowRight action (as seen in the first two entries in the screenshot). Meanwhile from the third entry until the end shows how a normal Control+Option+Command+ArrowRight is pressed on a keyboard

So, it seems that hitting that combination results in the fn modifier being included.

[zenangst]

Yeah, and the raw event that I get from the mach port includes the function key.
However, it seems that we didn't properly map this before, we just took the raw key code for arrow keys. Perhaps that is the issue we are seeing.

I'll dig a bit deeper.

Do you trigger the arrow shortcuts mentioned above using your macropad?
If so, can it be configured to include that mask?

[zenangst]

@jason-wihardja I think I'm on to something now, I'm cooking a new version now for you to test. Just give me a minute for it to build.

[zenangst]

Here is the new version, don't forget to re-record the shortcut before testing.

Keyboard Cowboy 3.25.4.dmg.zip

[jason-wihardja]

Do you trigger the arrow shortcuts mentioned above using your macropad?
If so, can it be configured to include that mask?

Yes, I'm using my macropad. But I think I can't really include/exclude individual buttons/modifiers as the way I recorded a macro is through a software where it has a similar record functionality as yours. And since the recorded macro is being saved directly into the device's onboard memory, I don't really have the necessary skill to access or modify it in a manner that's akin to being as raw as editing the json in yours.

Here is the new version, don't forget to re-record the shortcut before testing.

Keyboard Cowboy 3.25.4.dmg.zip

Unfortunately, it still doesn't fix F20 and Ctrl + Cmd + Alt + Shift + Left/Right Arrow. And it broke something else.

Remember the workaround that I mentioned before where I now used Insert button in place for F20 because it's working, previously it was recorded as fn + Insert, now it is recorded as Insert only and it broke that shortcut. However, if I plug in the previous config, where things are still configured as fn + Insert, without re-recording, the shortcut for that button still works. This leads me to highly suspect that the problem may be on the recorder part of your code.

[zenangst]

@jason-wihardja Are you using the macropad or the regular keyboard to record?

The last change that I made was to revert the change where the function gets invoked when recording arrow keys, F-keys etc. Instead, it will record these as a single special key. I thought this would be as this is how we did it before the 3.25.0 release.

[zenangst]

@jason-wihardja, can you let me know which keyboard locale you're using? I'm adding some extra tests to the KeyCodes framework that I'm using to create virtual key tables for keyboard shortcuts.

[jason-wihardja]

The last change that I made was to revert the change where the function gets invoked when recording arrow keys, F-keys etc. Instead, it will record these as a single special key. I thought this would be as this is how we did it before the 3.25.0 release.

Yes, since I often do upgrades/downgrades, this is something I notice as well. However, things work fine in 3.24.x. My keys are recorded without fn key, and it work, whereas in the latest version you uploaded, things seem only work if the button is recorded with fn key included (which the latest build of 3.25.4 fails to do)

Can you let me know which keyboard locale you're using?

I'm using standard US ANSI layout. My macropad, although I configured each of the buttons with weird function keys, it is also registered in the system as US ANSI.

Maybe to explain a little bit of my setup and how the inputs are routed to the system:

1. A 1800 layout keyboard as the main device

   How inputs are routed:
   Keyboard -> Karabiner -> Keyboard Cowboy -> System

2. A 12-button macropad with some buttons assigned to F13-F20, some others assigned to mouse button 6-8

   How inputs are routed for buttons assigned to F13-F20:
   Macropad -> Keyboard Cowboy -> System

   How inputs are routed for buttons assigned to mouse button 6-8:
   Macropad -> Mac Mouse Fix -> Keyboard Cowboy -> System

The shortcuts with a lot of combinations such as Ctrl + Cmd + Alt + Shift + Left/Right Arrow are those produced through buttons assigned to mouse buttons. So, here's how it looks like for one of the buttons:

1. Macropad sends mouse button 7
2. Mac Mouse Fix converts it into Ctrl + Cmd + Alt + Shift + Right Arrow
3. Keyboard Cowboy captures Ctrl + Cmd + Alt + Shift + Right Arrow
4. Converts it to another shortcut, depending on app

My Mac Mouse Fix setup for reference:

My mouse has up to button 5, so the setup up to button 5 are performed through my mouse whereas button 6-8 are pressed through my macropad

[jason-wihardja]

Using the latest version you uploaded, this is the config that has everything working (with some buttons like F20 reassigned to Insert). I wasn't able to record all these through the recorder due to the missing fn modifier I mentioned before, so I edited the file directly.

config.json

Using this new config, the functionality is equivalent to what I was achieving with version 3.24.2 and the config I attached previously here.

[zenangst]

@jason-wihardja here is a new version for test 🤞

Keyboard Cowboy 3.25.4.dmg.zip

[zenangst]

@jason-wihardja I just released 3.25.4, because it included additional other fixes that I wanted to get out there, let me know how it goes.