Introduction:
Sikuli is an open source automation testing tool which can be used to automate anything you see on the screen. It uses a screenshot of GUI elements from the user to compare and identify/control GUI components on the screen, using which the user can automate the workflow of any application, irrespective of the underlying technology.
Below are some of the commonly used UI automation testing tools:
- Selenium
- HPE Unified Functional Testing (HP – UFT formerly QTP)
- TestComplete
- Sikuli
- QFTest
- Marathon
So… Why Sikuli?
1. Sikuli is particularly useful when there is no easy access to a GUI’s internal source code i.e when the UI components do not have internal id’s and there is no access to it’s source code
2. Sikuli can be used to automate any desktop or web-based application.
3. Sikuli should be one’s choice when someone is automating flash-based application. For more details check http://www.sikuli.org/gaming.html.
4. Sikuli supports reading text over image using OCR (optical character recognition). This is ideal to automate application involves captcha in login screen.
5. Also, one can use sikuli to automate analytics-based application as it provides easy way to capture and validate graphs and ballparks present in the application.
Automation using sikuli:
To start with sikuli, download sikulixsetup.jar file from the link – https://launchpad.net/sikuli/+download
Double click on downloaded setup file (“sikulixsetup-xxx.jar”) and follow the instructions to install.
After the successful installation double click on sikulix.jar file created in installation folder. This will open the sikuli IDE.
Users can capture screenshot of any GUI control present in the screen using “Take screenshot” button in sikuli IDE.
Basic Commands and features of Sikuli:
Generic list of user actions and its syntax to automate any application using sikuli is displayed in left side of the sikuli IDE. Below we have listed few for quick reference.
1. Click
Click is the function used to click on any object present in the application.
2. Type
Type function allows user to enter text in text box present on the screen.
3. Find
Find helps us to locate the exact image on the screen.
4. Hover
Hover command helps user to move the cusor to hover above the click point.
5. Drag and Drop
Drag and drop command performs drag and drop operation from starting click point to target click point.
6. SwitchApp
Switchapp command can be used to switch to the specified application.
Example code to launch and switch to desired application window.
app.open()
sleep(10)
switchApp("app name")
7. AddImagePath
By default sikuli will search for images in current test folder and we can create common image library which can be accessed by any test.
imgPath = list(getImagePath())
print imgPath
8. Similar
Consider this scenario – There is a start date field in application and it displays current date by default. Now we have a test case which involves capturing this start date field with current date and then run the same test every day, but the current date would change everyday. If the date is changed then pixels would change automatically which in turn will cause the image search to fail. To handle this, we could customize the image search for partial match i.e to check for 50 % of pixel match, and here Similar is very useful.
9. Exception handling with try block
Image search failure can be handled with try block.
find("imagename.png")
popup ("found")
except FindFailed:
popup ("notfound")
10. Text
One can fetch the text in any region of the screen using below syntax.
Print F
Now that we’ve looked at the basic commands, let’s see how we can use them to automate a scenario.
Basic Program to understand automation with Sikuli:
Below program validates the presence of “52 week high”, ”52 week low” and “open” labels for any stock in google finance website. Also, sikuli allows user to write custom functions in Jython.
Code Snippet:
wait("searchbar.png",10)
type("https://www.google.com/finance")
type(Key.ENTER)
wait("MarketSummary.png",10)
click("googleSearchBar.png")
type("infy")
type(Key.ENTER)
wait("InfosysLtd.png",10)
type(Key.PAGE_DOWN)
try:
assert exists("52wkHigh.png")
assert exists("52wkLow.png")
assert exists("open.png")
except AssertionError:
print "Unable to find required labels in the screen"
To run this yourself, you can download the attached sample program (which has both source code and all the required screenshots) and run it in your Sikuli IDE.
Sikuli Best Practices
Sikuli uses pixel-based comparison of the provided image against the screen pixels to find the image. Below are some of the best practices that needs to be taken care while capturing the images.
- Windows provides an option to scale the screen for higher resolutions. If there is a difference in scale used while capturing the image and while running the test, the image matching fails. For valid comparison, the images should also be scaled accordingly. To avoid such scaling issue, it is recommended to have the scaling as 100% during capturing images and running tests.
- When the screen resolution is changed, there is no change in pixels rendered. Images captured in one resolution should match in a different resolution. However, the application could have dynamic sizing of controls/images/charts based on screen size which could result in
- Controls having different sizes (causing difference in pixels)
- Controls may or may not fit in the visible screen
This could lead to image captured in one resolution failing in a different resolution. To avoid this complexity, it is recommended to have the tests built and run for few specific resolutions.
- Images captured for automation using Sikuli IDE will be saved in “filename.sikuli” folder. And it is recommended to have a common image library which can be accessed by any test. This reduce the image search effort for modification or deletion.
Conclusion
Overall, Sikuli is a powerful UI test automation tool, which uses image recognition to automate whatever we see on the screen. Also, anyone from non-coding background can jump start with automation using sikuli. Only disadvantage is that automation test will fail for minor changes in the application including changes in color/layout of components. This can be avoided to an extent using pattern match functionality. Given that Sikuli works for any web or desktop application, this can be the one tool you need for all your automation needs.