Sending keyboard input to an inactive window on Wayland

[yue.dongchen]

When learning languages from videos, I use a drawing tablet to take notes on one monitor, and a video in Celluloid (mpv front-end) on another monitor. I have mapped physical buttons to mpris2 media controls (play/pause, seek), but I'm looking to also trigger some keybindings of a lua script that takes extracts from the video, so that I can use the excerpts for later review. If I were on xorg it seems xdotools can send keystrokes to that particular Celluloid window. Is there a way to achieve this on Wayland?

Many thanks

PS I have just heard about the IPC protocol, but I'm not sure whether it's relevant.

 

[yue.dongchen]

I found the command/function names in the lua scripts. If I'm understanding correctly I can issue those commands via json-ipc. Seems promising.

 

[Rahulkumarsree]

When it comes to sending keystrokes or triggering keybindings in a specific application window on Wayland, the process is a bit different compared to Xorg. Wayland has a more secure design that restricts direct access to application windows by other processes.

However, there are alternative approaches you can consider:

• Use the IPC protocol: IPC (Inter-Process Communication) protocols like D-Bus can be used to send messages and commands between different applications. Celluloid, being an MPV front-end, might support D-Bus for communication. You can explore the available D-Bus interfaces and methods exposed by Celluloid to see if there is a way to trigger keybindings or send commands.
• Utilize scripting capabilities: If Celluloid has scripting capabilities, you can write a script that interacts with the application and triggers the desired keybindings or commands. Look into the Celluloid documentation or community resources to see if there are scripting options available.
• Explore application-specific features: Check if Celluloid has any built-in features or settings that allow external triggers or automation. It's possible that the application itself provides functionality to achieve what you're looking for without the need for external tools or scripts.
• Investigate Wayland-specific tools: Wayland has its own set of tools and libraries for interacting with Windows and applications. For example, you can explore the wlr-foreign-toplevel-management library, which allows managing external windows on Wayland. However, this might require more technical knowledge and development skills.

It's important to note that the availability of these options may vary depending on the specific application and its support for Wayland. You may need to do some research or reach out to the Celluloid community to get more information about the possibilities for triggering keybindings or commands on Wayland.

Overall, while Wayland has stricter security measures compared to Xorg, there are still possibilities for achieving your goal through IPC protocols, scripting, or application-specific features.

 

[Insomniac]

ydotool seems similar to xdotool except that it supports Wayland. Maybe that's something to look at.

 

[Rahulkumarsree]

ydotool seems similar to xdotool except that it supports Wayland. Maybe that's something to look at.

Yes, you are correct. ydotool is indeed similar to xdotool, but it has been developed to support the Wayland display server protocol, whereas xdotool is primarily designed for the X Window System.

Wayland is a newer display server protocol that aims to provide a more secure and efficient alternative to the X Window System. It is increasingly being adopted by modern Linux desktop environments. As a result, tools and applications that were originally designed for X11, such as xdotool, may not work seamlessly with Wayland.

To address this limitation, ydotool was developed as a compatible alternative that specifically targets Wayland. It provides similar functionality to xdotool, allowing users to automate input events, simulate keyboard and mouse actions, and manipulate windows.

If you are using a Wayland-based desktop environment and need to automate input or window manipulation, ydotool can be a useful tool to explore. It allows you to script and automate various tasks, making it handy for both personal use and scripting automation workflows.

Keep in mind that the availability and compatibility of ydotool may depend on your specific Linux distribution and the version of Wayland being used. It's recommended to consult the official documentation or resources specific to your distribution to ensure the compatibility and proper installation of ydotool for your Wayland setup.