Archive

Posts Tagged ‘screen scrapping’

Automated Linux GUI Applications using X11::GuiTest

November 3rd, 2015 No comments

Lately, I wanted to extract from application-embedded medical lexicon to construct a database for easy use of the lexicon data.  The application was ancient (1998) and Windows 32bit.  The application does not work in Windows version beyond Windows XP 32bit.  But fortunately, the application was very runnable in Ubuntu 14.04 over Wine layer.  A snapshot of the application is shown below:

app

There were several restrictions in the application, viz.,  “Select All” does not work, direct copy-paste does not work.  But the traversal of lookup items on the left pane could be fully done using keyboard keys.  So, the process for extracting every item is:

  1. TAB + TAB to go to the left pane containing the lookup items
  2. TAB to move to the right pane.
  3. Ctrl + Shift + End would select the entire text on the right pane.
  4. Ctrl + C copies the selected text to the clipboard
  5. Using “xclip”, it is possible to get the clipboard data into the driver script.
  6. Save the recovered clipboard content to a file.
  7. Move back to the left pane by Shift + TAB.
  8. Down arrow to move to the next item in the list.
  9. Goto Step 2, until exhaustion.

The ideal toolkit for this project is the X11::GuiTest library in Perl (https://metacpan.org/pod/X11::GUITest). The same toolkit is available in software repository, which could be installed using sudo apt-get install libx11-guitest-perl.  The codeblock that I’d used in Perl for getting is done is listed below.

# Start the application 
StartApp('/path/to/app');

# wait unconditionally for 5 seconds to allow the app to come to front.
WaitSeconds(5);

# move the cursor to the scroll list
SendKeys('{TAB 2}');
sleep(2);

for (my $i = 0; $i < 20; $i++ )
{
 # move the cursor to the right pane
 SendKeys('{TAB}');

 # select all
 SendKeys('^(+({END}))');

 # perform copy
 SendKeys('^(c)');
 
 # read the data from clipboard and print that in stdout
 my $clipboard = `xclip -selection clipboard -o`;

 if ( $clipboard ne $clipboard_prev ) { print '-'x 80; print "\n$i\t$clipboard\n" };
 $clipboard_prev = $clipboard;
 
 # move back to scroll list
 SendKeys('+({TAB})');

 # goto next item in the list.
 SendKeys('{DOWN}');
 WaitSeconds(0.7);
}