| San Mehat | a430b2b | 2014-09-23 08:30:51 -0700 | [diff] [blame] | 1 | This is VisualNaCro. |
| 2 | |
| 3 | DISCLAIMER: recorder.pl is not yet functional. |
| 4 | |
| 5 | What does it? |
| 6 | |
| 7 | It is a Perl module meant to remote control a VNC server. |
| 8 | |
| 9 | It includes a recorder (written in Perl) to make it easy to |
| 10 | record a macro, which is just a Perl script, and which you can |
| 11 | modify to your heart's content. |
| 12 | |
| 13 | The most important feature, however, is that you can mark a |
| 14 | rectangle which the Perl script will try to find again when you |
| 15 | run it. Thus when you play a game and want to hit a certain button, |
| 16 | you just hit the Ctrl key twice, mark the button, and from then on, |
| 17 | all mouse movements will be repeated relative to that button, even |
| 18 | if the button is somewhere else when you run the script the next |
| 19 | time. |
| 20 | |
| 21 | If you know Tcl Expect, you will recognize this approach. Only this |
| 22 | time, it is not text, but an image which is expected. |
| 23 | |
| 24 | How does it work? |
| 25 | |
| 26 | It acts as a VNC proxy: your Perl script starts its own VNC server. |
| 27 | The script now can intercept inputs and outputs, and act upon them. |
| 28 | In order to write a macro, start |
| 29 | |
| 30 | recorder.pl --script my-macro.pl --timing host:port |
| 31 | |
| 32 | connect with a vncviewer of your choice to <host2>:23, where <host2> |
| 33 | is the computer on which recorder.pl was started (not necessarily the |
| 34 | same as the VNC server!). Now your actions are recorded into |
| 35 | my_macro.pl, and the images you want to grep for will be saved as |
| 36 | my_macro-1.pnm, my_macro-2.pnm, ... |
| 37 | |
| 38 | In order to finish the script, hit Ctrl twice and say "q". |
| 39 | |
| 40 | Why did I do it? |
| 41 | |
| 42 | Because I could ;-) |
| 43 | |
| 44 | No really, I needed a way to write automated tests. While there |
| 45 | exist a lot of OpenSource programs for web testing, I found none |
| 46 | of them easy to use, and for GUI testing I found xautomation. |
| 47 | |
| 48 | Xautomation has this "visual grep" (or "graphical expect") feature: |
| 49 | given an image it tries to find it on the desktop and returns the |
| 50 | coordinates. Unfortunately, there is no easy way to record macros |
| 51 | with it, and it only works on X11. |
| 52 | |
| 53 | As I know VNC pretty well, and there are VNC servers for every OS |
| 54 | and gadget, I thought it might be cool to have this feature to |
| 55 | control a VNC server. |
| 56 | |
| 57 | Actually, it makes it even easier: with plain X11, for example, you |
| 58 | can not know where on the screen the action is if you don't check |
| 59 | the whole screen. This complex problem is beautifully addressed |
| 60 | in Karl Runge's x11vnc. |
| 61 | |
| 62 | My main purpose is to run regression tests on different browsers, |
| 63 | which I can easily do by starting Xvnc and using VisualNaCro. |
| 64 | |
| 65 | How did I do it? |
| 66 | |
| 67 | I wondered long about how to do it. I couldn't take the same approach |
| 68 | as xautomation: I cannot connect to the VNC server thousand times |
| 69 | per second. So I decided to create an interface of LibVNCServer/ |
| 70 | LibVNCClient for use in a script language. |
| 71 | |
| 72 | Fortunately, this task is made very, very easy by SWIG. As Perl |
| 73 | is one of my favorite script languages, I decided to use this. |
| 74 | But SWIG makes it easy to use the very same interface for other |
| 75 | popular languages, so you are welcome to port VisualNaCro to |
| 76 | the language of your choice! |
| 77 | |
| 78 | Isn't it pronounced "Visual Macro"? |
| 79 | |
| 80 | Yes. But I liked the Visual Na Cro play of acronyms. I'm sorry if |
| 81 | you don't find it funny. |
| 82 | |
| 83 | What's the license? |
| 84 | |
| 85 | GPL. It is based on LibVNCServer/LibVNCClient, so it has to be. |
| 86 | If you want to port this package to use vncreflector, which has a |
| 87 | BSD license, go ahead. |
| 88 | |