Sikuli

Table of Contents • What is Capture/Replay tool• Issues to solve• Introducing Sikuli• Sikuli Script• The Structure of Test Folder• Platform Independence• What we need• How Sikuli Works• Using Sikuli IDE• HTMLTestRunner• Advantages• Limitation

What is Capture/Replay tool• The test developer interacts with an application under test, typically

through the graphical user interface (GUI), while some capture tool simultaneously generates an automated test script

Tests are difficult to maintenance as the application changes• ex. screens will get added,

buttons will get removed,column names will get modified

3© Sun Technologies Inc.

Issues to solve• GUIs are difficult to test directly in code• Need of a reliable consistent regression test execution approach• Need of easy-to-read test execution results• Too many repeatable test steps• No suitable tools for specific applications (e.g. mobile app simulators)

Introducing Sikuli• Screenshot driven – it is a visual technology to search and automate GUI

using images Intuitive open-source visual scripting tool Scripts are written in Python• integrates with jUnit and supports unit testing for GUI• ability to import entire libs or only a selected subset of methods (reuse code and

images)

Sikuli Script

• Sikuli Script is a Jython and Java library that automates GUI interaction using image patterns to direct keyboard/mouse events

The Structure of Test Folder• Sikuli source/executable script directory (.sikuli)

• Python source file (.py)• Images files (.png)• While saving a script using

Sikuli IDE, an HTML file is created

System Design

Platform Independence

• Works on any GUI can be displayed on Windows/Linux/Mac• Virtual machines• Remote desktop• Mobile simulators: Android, iPhone• Web: Flash, HTML + Javascript

What we need

• Prerequisites for the lecture• Windows OS• Sun Java 6 JRE 32-bit version

• Download and install Sikuli• site: http://www.sikuli.org/download.html

• Combination of GUI & Traditional coding• Jython coding that can use images as parameters and variables• Screen Capture utilized for image selection instead of object IDs• Can upload image files

How Sikuli Works

Using Sikuli IDE

Predefined Sikuli Global functions IDE Command List – exists, find, findAll, wait, click,

doubleClick, dragDrop, type switchApp("App Name") keyDown(Key.SHIFT+ "a") keyUp(Key.SHIFT+ "a") popup("message") userText=input("msg", "default")

Using Sikuli IDE (2)

•Regions – a rectangular area: Region(x, y, w, h)• Finding Regions:• windowRegion = App.focusedWindow()• buttonRegion = find()

• Acting with Regions• region.highlight, click, type, find, etc..• Extending Regions• region.offset(x,y)

•Tuning the Vision Algorithm

Editor for writing Sikuli scripts in Python

The buttons activate the screenshot mode

HTMLTestRunner• HTMLTestRunner is an extension to the Python standard library's

unittest module• It generates easy to use HTML test reports

Advantages• Allow programmer to make visual references instead of using names or

keywords• More intuitive and natural approach• Useful for specific applications that involve manipulation of GUI

Limitation• Sikuli Script operates only in the visible screen space and thus is not

applicable to invisible GUI elements• ex. elements hidden underneath other windows, in another tab, or scrolled out of

• Dealing with moving, animated objects was hard

Useful Resources• Sikuli Official site:

http://www.sikuli.org/ • Sikuli documentation:

http://doc.sikuli.org/ • Sikuli script help:

http://doc.sikuli.org/sikuli-script-index.html• HTMLTestRunner page: http

://tungwaiyip.info/software/HTMLTestRunner.html

Sikuli

Software

Practical Sikuli: using screenshots for GUI automation and testing

Creating Contextual Help for GUIs Using Screenshotsup.csail.mit.edu/other-pubs/uist2011-sikuli-guide.pdfCreating Contextual Help for GUIs Using Screenshots Tom Yeh, Tsung-Hsiang Chang§,

Exploring the internal state of user interfaces using sikuli

Automation with Sikuli

Waken: Reverse Engineering Usage Information and Interface ... UIST waken.pdf · underlying computer vision algorithms in Sikuli identify GUI elements based on templates from sample

Bob Igo Sikuli Cposc2010

Sikuli Google

Help, It looks Confusingvisual.cs.ucl.ac.uk/pubs/HILC/IUI17_present_finallized.pdf · – Sikuli, Sikuli Slides (Yeh et al 2009). – Koala (Little et al 2007), CoScripter (Leshed

sikuli final

Sikuli for Mobile Testing

Sikuli: Using Screenshots for GUI Automation and Testing

GUI Testing and Automation with Sikuli · Things I Didn't Cover findAll() is useful if you want to test several GUIs at the same time Sikuli provides openApp() and closeApp() to run

Integration with Other Tools - Packt Publishing · PDF fileIntegration with Other Tools 2 Lastly, we will explore how to automate non-web GUI using tools such as AutoIt and Sikuli

Make manual testing to automated by Sikuli bug daybkk2013

The Harbor of the Sea Peoples at Dor Avner Raban The Biblical …dor.huji.ac.il/Download/Article/Raban1987.pdf · 2007-10-30 · the story of Wen-Amon, the Sikuli are mentioned in

Bishop's plea for an end to the slaughterstbernadettes/links_133...in north-eastern DRC, Bishop Melchisédech Sikuli Paluku of Butembo-Beni slammed the indifference of media and politicians

Overview of our - diaspark.com · Tools - Overview Process Tools ... Ranorex, Sikuli, Robotium, Selendroid, ... PowerPoint Presentation Author: Ankit Balani Created Date:

Sikuli: Using GUI Screenshots for Search and Automationfrancky.me/doc/nofd/sikuli-uist2009.pdf · 2018-02-04 · adding screenshots with custom annotations to the index. Screenshot

Tech Next - -AMIT DIXIT -DIPESH BHATEWARA Cross Platform UI testing using Sikuli

Sikuli: Using GUI Screenshots for Search and Automation