Installing SikuliX
THIS IS WORK IN PROGRESS
General Aspects#
be aware
While running SikuliX features
... a real, unlocked screen is needed (virtual screens might work)
... the machine is not usable for user interaction being not part of the SikuliX workflow
SikuliX runs on#
- PC's/workstations
- with Windows, macOS or Linux
- and Java 8+ installed.
- all being 64-Bit
Windows: 10+ is recommended. 7+ and server versions should work also, but might need additional stuff. Same goes for special editions, that do not have all features available (e.g. media support).
macOS: 11 or 10.15 are recommended. Latest fully 64-Bit versions 10.11+ should work also. The behavior on macOS 11 with the Apple chip is not yet evaluated.
Linux: Ubuntu 16+ and other Debian-based flavors are recommended. Generally should SikuliX be usable on every Linux version, that fulfills the prerequisites.
More system specific aspects can be found in the OS related topics below.
SikuliX is Java based#
- the usable artifacts are delivered as jar-files
- that can be stored wherever you want on your local file system
- but not in system specific program or application folders (possible problems)
SikuliX storage#
SikuliX stores some stuff, that is needed/created at runtime, in system specific folders in the user space.
Windows: A folder Sikulix is used in the system folder specified by the environment variable %APPDATA%, which usually is something like C:\Users\username\AppData\Roaming.
macOS:
Linux:
Scripting vs. Programming#
Scripting support: SikuliX IDE
SikuliX at the top wants to support easy scripting of visual workflows using popular scripting languages like Python, Ruby and others.
This scripting support is available as the package named SikuliX IDE
Java programming using SikuliX features: SikuliX API
The main SikuliX features:
- searching images and text on a screen or in other images
- using mouse and keyboard to trigger user actions
- reading text from screens or images
- handling applications and their windows
can also be used in Java programs, other Java-aware languages and solutions, that can work with a Java library. This support is available as the package named SikuliX API.
Java#
Beginning with Java 9, when Oracle changed its Java licensing policy, there are only packages OpenJDK freely available, which will normally be used together with SikuliX. In commercial projects, where SikuliX can be used also, the owners have to decide about the Java environment.
In all cases it must be Java 8+ and a 64-Bit version.
Usable versions#
To use the SikuliX features you only need a Java JRE. But since Java 9 this is no longer available as non-commercial package and you have to use the OpenJDK packages, which contain the JRE stuff.
SikuliX will be available for Java 8 in the scripting context (SikuliX IDE), as long as Java 8 is supported.
Generally you can use any Java version 8+.
For special Java aspects about the latest SikuliX version look here.
recommendation
- Use the Java packages available at AdoptOpenJDK
- Use the LTS versions (currently 8 and 11) for scripting
Check your version#
Make this test on a command line:
which should show something like:
Be sure 64-Bit Server is mentioned, so it is the 64-Bit version.
Multiple Java versions#
If for whatever reason you have or need more than one Java version on your machine, be sure to know how to use the intended version in the different usage scenarios:
- starting a Java jar by double-clicking the respective jar-file
- running the java command from a commandline or in sub-processes from scripts or programs
- using the SikuliX jars in some IDE like PyCharm, IDEA, Eclipse, NetBeans, ...
The most common approach is to use the environment variable JAVA_HOME, but on each system variant there are additional, different approaches how to handle the before mentioned cases.
Windows#
The IDE is only available as jar-file, that can be double-clicked to start it.
Currently there is no SikuliX.exe available, but feel free to create your own commandfile, where you can setup your specific runtime environment.
Besides Java there are no prerequisites.
All native libraries are bundled in the jar-file and exported at runtime as needed.
macOS#
The IDE is only available as jar-file, that can be double-clicked to start it.
Currently there is no SikuliX.app available, but feel free to create your own bash-script, where you can setup your specific runtime environment.
Besides Java you have to take care
The native libraries for the OpenCV support are bundled in the jar-file and exported at runtime as needed.
OpenCV image search#
The OpenCV native libraries are needed for SikuliX's image search and other features.
note
Up to SikuliX version 2.0.x: there is nothing to do for you, since the libraries (OpenCV version 3.4.2) are bundled with SikuliX and exported automatically for usage at runtime.
For SikuliX version 2.1.x and higher: you have to create a valid OpenCV installation before you can use SikuliX with all its features.
If this is not done before the first use of SikuliX, in case it will terminate abnormally and present information, what has to be done. , follow these steps, to make a Java OpenJDK ready on your system:
- download the package macOS / x64 tar.gz from jdk.java.net
- unzip/untar (right click and open with standard app)
you get a folder like
jdk-13.0.1.jdk - move this folder to the system folder
/Library/Java/JavaVirtualMachines/ - the above test should now work
... or use any other workflow, that leads to a valid Java 8+ installation and lets you successfully run
java -version.
tip
Recommendation: install Homebrew as package manager, which should not make any problems on recent macOS versions (tested with success on macOS 10.13+).
Get the OpenCV package installation by just running this in a Terminal window:
After successful installation of opencv, try again with SikuliX.
This might work as well with other package managers like MacPorts, Finch, ..., but is not tested.
Tesseract OCR and text#
If you want to use these features, you have to make the Tesseract libraries ready.
If you have tried to use one of the SikuliX OCR features and got an error telling you, that the library libtesseract is missing.
The reason is: The native libraries of the Tesseract package are only bundled with SikuliX for Windows, but have to be installed on macOS and Linux by the SikuliX user himself.
Tesseract for macOS#
The recommendation is to install Homebrew as package manager, which should not make any problems on recent macOS versions.
Get the Tesseract installation by just running this in a Terminal window:
After successful installation of Tesseract run this in a terminal window
Linux#
Java#
You need a Java JDK version 8 or later.
Download the SikuliX IDE jar from the SikuliX Download page and put it in your home folder.
On Linux Systems SikuliX cannot be used out of the box.
First we need OpenCV and Tesseract then wmctrl and xdotool, when using the App class features.
Try to run it in a Terminal Window after navigating to folder:
The parameter -v gives full debug output from the very beginning - hence keep it until everything works.
You might get the following error
Solution enter the follwing:
You might get the following error
look here for possible solutions.
important
Special information on the bundled libJXGrabKey.so
At least on newer Ubuntu versions ldd -r reports unresolved symbols pthread… It seems, that this can be ignored, since JXGrabKey works. If you get problems, that are related to JXGrabKey, you might have to build from the sources and provide the ready built library in the ~/.Sikulix/SikulixLibs folder.
OpenCV support#
SikuliX needs access to the OpenCV Java/JNI bindings (an OpenCV native library named libopencv_javaXYZ.so, where XYZ is a shortcut for the version like 320 or 341).
On Debian systems (like Ubuntu) you should get it with the apt-tool, on other Linux systems you might get it with their package-tool. In any case it is possible, to create it from the sources (see below).
Generally needed:
A link (ln -s ...) as libopencv_java.so in any folder, that is on the library path (e.g. on Ubuntu: /usr/lib), to the real library module libopencv_javaXYZ.so
should work on Ubuntu 18.04 (or other actual Debian systems):
A libopencv_java320.so will be in /usr/lib/jni, to where you have to symbolical link libopencv_java.so. On Ubuntu the best place seems to be /usr/lib, but that is up to you, as long as the containing folder is in the current lib path at runtime.
Build OpenCV library from the sources (might be needed on non-Debian Linux)
Get a source package (recommended: 3.4+) and unzip/untar it to a work folder of your choice.
In a terminal session go into the OpenCV folder and make a folder build at the top level. Finally go into that build folder
All the following steps have to be done being inside this build folder.
There are some general prerequisites for the build process to work, that you have to fulfill either before you start or solve it every time you run into a problem. On Ubuntu, the usual way is to use the apt-tool.
You need: cmake, ant, build-tools, Python (2.7 or 3.x) and a Java JDK
All prerequisites must be available/accessible in the standard environment.
In the build folder run the build setup:
As a result you will get some output, which tells you, wether you will have a chance to get a valid libopencv_javaXYZ.so
Built as dynamic libs: you will have to install the built stuff to your system in a second step.OpenCV modules: the list in the line afterTo be builtMUST containjava. - at least core features2d flann imgcodecs imgproc highgui must be thereMedia I/O: at least ZLib, JPEG and PNG must show a valid entryPython(for build): must show a valid Python runnerJava:antandJNImust show valid entriesInstall to: must make sense
If one or more of the mentioned entries are odd, you have to debug for the reason ;-)
Common problems are (if java is not mentioned in the line To be built), that Python is not available and a valid JDK is not found in the environment.
The usual way to solve the problems is to install the relevant packages.
Each time you fixed something simply run cmake .. again.
If you made any changes to CMakeCache.txt just run cmake .. again.
When you are finally satisfied with the result, just run:
This will install the stuff into your system (will take some minutes to complete). Be sure, that you have a success: no errors are shown.
As a last step you need to find the installed libopencv_javaXYZ.so and as mentioned above create a symbolic link, that is found in the library path.
Tesseract support#
Make sure, to have a version 4.x libtesseract....so is available on your system.
You should first try with your package manager.
To test, what you have on your system, run this:
If anything looks odd, then you have to dive into the gory details.
The following bash snippet will let you launch SikuliX ...
Put your own paths and correct versions to point to where you have unzipped SikuliX and where you keep your other files.
Any paths names or environment variables needed (say a Python path) should be exported inside the script before SikuliX starts.
The line beginning nohup ... launches your process in the background. You can keep using the terminal or launch this script with a .Desktop file. I use "${JDK_08}" to be a specific Java release. If things go wrong comment-out the " > /def/null" part.