Automating Windows™ from Java® and WindowTreeDom☂

(So anyway there used to be a page on codeproject containing some of these ideas implemented in C#, but I can’t find it any more. I’m sure it exists.)

So let’s say you’re writing your everything-is-a-service web application and you’ve got RESTful and/or SOAPy calls to your middle tier and everything’s going swimmingly until you discover you need to integrate your app with some 80’s-era RAD Windows application that no-one has the sourcecode to and embodies some vital bit of business intelligence that you need.

This application was written by IBM on the hard drive platters of angels so isn’t going anyway any time soon, but for the purposes of this blog post, let’s call that application “Notepad” [*]

Wouldn’t it be nice if your software could automatically interact with that application, wrap it up in a service-oriented bow and then deliver it’s results to you in a method that some would consider slightly more standardised ?

Of course it would.

So lets have a look at the frontend to Notepad’s data layer, the ‘Save As’ dialog box:

The dialog box is made up of windows and controls, which are visible in the Windows™ window hierarchy.

If you fire up the venerable Spy++ from MSDN, you’ll notice that all the objects on your desktop, including the dialog box, are arranged in a treelike structure.

The Document Object Model (DOM) is also a treelike structure, and has a whole arsenal of utilities that operate on it (XSLT, XPath, that sort of thing).

So let’s represent the window tree as a DOM tree, with each element of the XML tree representing a window in the Windows™ window tree:

...
<window class="Notepad" hwnd="native@0x761476" title="Untitled - Notepad">
  <window class="#32770" hwnd="native@0x2e1528"
    owner="native@0x761476" title="Save As">
    <window class="tooltips_class32" hwnd="native@0x4158a"
      owner="native@0x2e1528" title=""/>
    <window class="tooltips_class32" hwnd="native@0x3159c"
      owner="native@0x2e1528" title=""/>
    <window class="Static" hwnd="native@0x681474" title="Save &amp;in:"/>
    <window class="ComboBox" hwnd="native@0x84141c" title=""/>
    <window class="Static" hwnd="native@0x121534" title=""/>
    <window class="ToolbarWindow32" hwnd="native@0x41580" title=""/>
    <window class="ToolbarWindow32" hwnd="native@0x19154e" title=""/>
    <window class="ListBox" hwnd="native@0x3157c" title=""/>
    <window class="SHELLDLL_DefView" hwnd="native@0x3159a" title="">
      <window class="SysListView32" hwnd="native@0x31596" title="FolderView"/>
    </window>
    <window class="Static" hwnd="native@0x31566" title="File &amp;name:"/>
    <window class="ComboBoxEx32" hwnd="native@0x31574" title="">
      <window class="ComboBox" hwnd="native@0x31578" title="">
        <window class="Edit" hwnd="native@0x31570" title=""/>
      </window>
    </window>
    <window class="Static" hwnd="native@0x3157a" title="Save as &amp;type:"/>
    <window class="ComboBox" hwnd="native@0x31582" title=""/>
    <window class="Button" hwnd="native@0x21568" title="Open as &amp;read-only"/>
    <window class="Button" hwnd="native@0x2156a" title="&amp;Save"/>
    <window class="Button" hwnd="native@0x21564" title="Cancel"/>
    <window class="Button" hwnd="native@0x21562" title="&amp;Help"/>
    <window class="ScrollBar" hwnd="native@0x2155a" title=""/>
    <window class="#32770" hwnd="native@0x2155c" title="">
      <window class="Static" hwnd="native@0x21586" title="&amp;Encoding:"/>
      <window class="ComboBox" hwnd="native@0x2155e" title=""/>
    </window>
    <window class="IME" hwnd="native@0xd1554" owner="native@0x2e1528" title="Default IME">
      <window class="MSCTFIME UI" hwnd="native@0x7b1478"
        owner="native@0xd1554" title="M"/>
    </window>
  </window>
  <window class="Edit" hwnd="native@0x11152c" title=""/>
  <window class="msctls_statusbar32" hwnd="native@0xbe1422" title=""/>
</window>
...

Then to find a particular window, we could code up something similar to the following:

WindowTreeDom wtd = new WindowTreeDom();
Document dom = wtd.getDom();
Element okButtonEl = (Element) XPathAPI.getSingleNode(dom, 
    ".//window[@class='Notepad']/window[@title=\"Save As\"]/window[@title=\"&Save\"]");
User32.SendMessage(WindowTreeDom.getHwnd(okButtonEl), WM_CLICK);

which grabs a handle to the Save button on a Save As dialog for a notepad window, and sends a WM_CLICK message to the control’s message loop.

Which is useful if you want to automatically press the Save button on a Save As dialog on a notepad window without paying half a million dollars for HP LoadRunner.

So here’s the code already:

[rn-sourcecode lang=”java” src=”common-public/src/main/java/com/randomnoun/common/jna/WindowTreeDom.java” errors=”true”]

and here’s a unit test:

[rn-sourcecode lang=”java” src=”common-public/src/test/java/com/randomnoun/common/jna/WindowTreeDomTest.java” errors=”true”]

You’ll need to add JNA and whatever XML toolkit you’re using to your project’s dependencies, which in my pom.xml looks like the following pom.xml fragment:

<dependency>
   <groupId>net.java.dev.jna</groupId>
   <artifactId>jna</artifactId>
   <version>3.2.4</version>
   <type>jar</type>
   <scope>compile</scope>
</dependency>

Hooray.

Update 2013-09-25: This code is in the com.randomnoun.common:common-public maven artifact:

common-public

com.randomnoun.common:common-public

Update 2021-01-29: and github:

common-public

git@github.com:randomnoun/common-public.git