Add frontend and scheduler for Autotest
The attached tarball includes the new Autotest web frontend for creating
and monitoring jobs and the new scheduler for executing jobs. We've
been working hard to get these complete and stabilized, and although
they still have a long, long way to go in terms of features, we believe
they are now stable and powerful enough to be useful.
The frontend consists of two parts, the server and the client. The
server is implemented in Python using the Django web framework, and is
contained under frontend/ and frontend/afe. The client is implemented
using Google Web Toolkit and is contained under frontend/client. We
tried to use Dojo initially, as was generally agreed on, but it proved
to be too young and poorly documented of a framework, and developing in
it was just too inefficient.
The scheduler is contained entirely in the scheduler/monitor_db Python
file. It looks at the database used by the web frontend and spawns
autoserv processes to run jobs, monitoring them and recording status.
The scheduler was developed by Paul Turner, who will be sending out some
detailed documentation of it soon.
I've also included the DB migrations code for which I recently submitted
a patch to the mailing list. I've included this code because it is
necessary to create the frontend DB, but it will (hopefully) soon be
part of the SVN repository.
Lastly, there is an apache directory containing configuration files for
running the web server through Apache.
I've put instructions for installing a full Autotest server, including
the web frontend, the scheduler, and the existing "tko" results backend,
at http://test.kernel.org/autotest/AutotestServerInstall. I've also
created a brief guide to using the web frontend, with plenty of
screenshots, at http://test.kernel.org/autotest/WebFrontendHowTo.
Please let me know if you find any problems with either of these pages.
Take a look, try it out, send me comments, complaints, and bugs, and I
hope you find it useful!
Steve Howard, and the rest of the Google Autotest team
From: Steve Howard <showard@google.com>
git-svn-id: http://test.kernel.org/svn/autotest/trunk@1242 592f7852-d20e-0410-864c-8624ca9c26a4
diff --git a/frontend/__init__.py b/frontend/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/frontend/__init__.py
diff --git a/frontend/afe/__init__.py b/frontend/afe/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/frontend/afe/__init__.py
diff --git a/frontend/afe/control_file.py b/frontend/afe/control_file.py
new file mode 100644
index 0000000..7b2d7bd
--- /dev/null
+++ b/frontend/afe/control_file.py
@@ -0,0 +1,67 @@
+"""\
+Logic for control file generation.
+"""
+
+__author__ = 'showard@google.com (Steve Howard)'
+
+import os
+import frontend.settings
+
+AUTOTEST_DIR = os.path.abspath(os.path.join(
+ os.path.dirname(frontend.settings.__file__), '..'))
+
+
+KERNEL_INSTALL_TEMPLATE = """\
+def step_init():
+ job.next_step([step_test])
+ testkernel = job.kernel('%(kernel)s')
+ %(kernel_config_line)s
+ testkernel.install()
+ testkernel.boot(args='%(kernel_args)s')
+
+def step_test():
+"""
+
+def kernel_config_line(kernel, platform):
+ if (not kernel.endswith('.rpm') and platform and
+ platform.kernel_config):
+ return "testkernel.config('%s')" % platform.kernel_config
+ return ''
+
+
+def read_control_file(test):
+ control_file = open(os.path.join(AUTOTEST_DIR, test.path))
+ control_contents = control_file.read()
+ control_file.close()
+ return control_contents
+
+
+def get_kernel_stanza(kernel, platform, kernel_args):
+ return KERNEL_INSTALL_TEMPLATE % {
+ 'kernel' : kernel,
+ 'kernel_config_line' : kernel_config_line(kernel, platform),
+ 'kernel_args' : kernel_args}
+
+
+def get_tests_stanza(tests):
+ return ''.join(read_control_file(test) for test in tests)
+
+
+def indent_text(text, indent):
+ lines = [indent + line for line in text.splitlines()]
+ return '\n'.join(lines)
+
+
+def generate_client_control(tests, kernel=None, platform=None):
+ control_file = ''
+ indent = ''
+ if kernel:
+ control_file = get_kernel_stanza(kernel, platform, '')
+ indent = '\t'
+
+ control_file += indent_text(get_tests_stanza(tests), indent)
+ return control_file
+
+
+def generate_server_control(tests):
+ return get_tests_stanza(tests)
diff --git a/frontend/afe/doctests/rpc_test.txt b/frontend/afe/doctests/rpc_test.txt
new file mode 100644
index 0000000..b8b15d4
--- /dev/null
+++ b/frontend/afe/doctests/rpc_test.txt
@@ -0,0 +1,329 @@
+# setup (you can ignore this)
+# ###########################
+
+# a bit of setup to allow overriding rpc_interace with an RPC proxy
+# (to use RPC, we would say
+# import rpc_client_lib
+# rpc_interface = rpc_client_lib.get_proxy('http://hostname:8000/rpc/')
+# )
+>>> if 'rpc_interface' not in globals():
+... from frontend.afe import rpc_interface, rpc_utils, models
+... # set up a user for us to "login" as
+... user = models.User(login='debug_user')
+... user.save()
+... rpc_utils.set_user(user)
+...
+
+# get directory of this test file; we'll need it later
+>>> import frontend.afe.test
+>>> import os
+>>> test_path = os.path.join(os.path.dirname(frontend.afe.test.__file__),
+... 'doctests')
+>>> test_path = os.path.abspath(test_path)
+
+# basic object management
+# #######################
+
+# create a label
+>>> rpc_interface.add_label(name='test_label')
+1L
+
+# we can modify the label by referencing its ID...
+>>> rpc_interface.modify_label(1, kernel_config='/my/kernel/config')
+
+# ...or by referencing it's name
+>>> rpc_interface.modify_label('test_label', platform=True)
+
+# we use get_labels to retrieve object data
+>>> data = rpc_interface.get_labels(name='test_label')
+>>> data == [{'id': 1L,
+... 'name': 'test_label',
+... 'platform': 1,
+... 'kernel_config': '/my/kernel/config'}]
+True
+
+# get_labels return multiple matches as lists of dictionaries
+>>> rpc_interface.add_label(name='label1', platform=False)
+2L
+>>> rpc_interface.add_label(name='label2', platform=True)
+3L
+>>> rpc_interface.add_label(name='label3', platform=False)
+4L
+>>> data = rpc_interface.get_labels(platform=False)
+>>> data == [{'id': 2L, 'name': 'label1', 'platform': 0, 'kernel_config': ''},
+... {'id': 4L, 'name': 'label3', 'platform': 0, 'kernel_config': ''}]
+True
+
+# delete_label takes an ID or a name as well
+>>> rpc_interface.delete_label(3)
+>>> rpc_interface.get_labels(name='label2')
+[]
+>>> rpc_interface.delete_label('test_label')
+>>> rpc_interface.delete_label('label1')
+>>> rpc_interface.delete_label('label3')
+>>> rpc_interface.get_labels()
+[]
+
+# all the add*, modify*, delete*, and get* methods work the same way
+# hosts...
+>>> rpc_interface.add_host(hostname='ipaj1', locked=True)
+1L
+>>> rpc_interface.modify_host('ipaj1', status='Hello')
+>>> data = rpc_interface.get_hosts()
+>>> data == [{'id': 1L,
+... 'hostname': 'ipaj1',
+... 'locked': 1,
+... 'synch_id': None,
+... 'status': 'Hello',
+... 'labels': [],
+... 'platform': None}]
+True
+>>> rpc_interface.delete_host('ipaj1')
+>>> rpc_interface.get_hosts() == []
+True
+
+# tests...
+>>> rpc_interface.add_test(name='sleeptest', test_type='Client',
+... test_class='Kernel', path='sleeptest')
+1L
+>>> rpc_interface.modify_test('sleeptest', path='/my/path')
+>>> data = rpc_interface.get_tests()
+>>> data == [{'id': 1L,
+... 'name': 'sleeptest',
+... 'description': '',
+... 'test_type': 'Client',
+... 'test_class': 'Kernel',
+... 'path': '/my/path'}]
+True
+>>> rpc_interface.delete_test('sleeptest')
+>>> rpc_interface.get_tests() == []
+True
+
+# users...
+>>> rpc_interface.add_user(login='showard')
+2L
+>>> rpc_interface.modify_user('showard', access_level=1)
+>>> data = rpc_interface.get_users(login='showard')
+>>> data == [{'id': 2L,
+... 'login': 'showard',
+... 'access_level': 1}]
+True
+>>> rpc_interface.delete_user('showard')
+>>> rpc_interface.get_users(login='showard') == []
+True
+
+# acl groups...
+# 1 ACL group already exists, named "Everyone" (ID 1)
+>>> rpc_interface.add_acl_group(name='my_group')
+2L
+>>> rpc_interface.modify_acl_group('my_group', description='my new acl group')
+>>> data = rpc_interface.get_acl_groups(name='my_group')
+>>> data == [{'id': 2L,
+... 'name': 'my_group',
+... 'description': 'my new acl group',
+... 'users': [],
+... 'hosts': []}]
+True
+>>> rpc_interface.delete_acl_group('my_group')
+>>> data = rpc_interface.get_acl_groups()
+>>> data == [{'id': 1L,
+... 'name': 'Everyone',
+... 'description': '',
+... 'users': ['debug_user'],
+... 'hosts': []}]
+True
+
+
+# managing many-to-many relationships
+# ###################################
+
+# first, create some hosts and labels to play around with
+>>> rpc_interface.add_host(hostname='host1')
+2L
+>>> rpc_interface.add_host(hostname='host2')
+3L
+>>> rpc_interface.add_label(name='label1')
+5L
+>>> rpc_interface.add_label(name='label2', platform=True)
+6L
+
+# add hosts to labels
+>>> rpc_interface.host_add_labels('host1', ['label1'])
+>>> rpc_interface.host_add_labels('host2', ['label1', 'label2'])
+
+# check labels for hosts
+>>> data = rpc_interface.get_hosts(hostname='host1')
+>>> data[0]['labels']
+['label1']
+>>> data = rpc_interface.get_hosts(hostname='host2')
+>>> data[0]['labels']
+['label1', 'label2']
+>>> data[0]['platform']
+'label2'
+
+# check host lists for labels -- use double underscore to specify fields of
+# related objects
+>>> data = rpc_interface.get_hosts(labels__name='label1')
+>>> [host['hostname'] for host in data]
+['host1', 'host2']
+>>> data = rpc_interface.get_hosts(labels__name='label2')
+>>> [host['hostname'] for host in data]
+['host2']
+
+# remove a host from a label
+>>> rpc_interface.host_remove_labels('host2', ['label2'])
+>>> data = rpc_interface.get_hosts(hostname='host1')
+>>> data[0]['labels']
+['label1']
+>>> rpc_interface.get_hosts(labels__name='label2')
+[]
+
+# ACL group relationships work similarly
+# note that all users and hosts are a member of 'Everyone' by default
+>>> rpc_interface.add_user(login='showard', access_level=0)
+3L
+>>> rpc_interface.add_acl_group(name='my_group')
+3L
+
+>>> rpc_interface.acl_group_add_users('my_group', ['showard'])
+>>> rpc_interface.acl_group_add_hosts('my_group', ['host1'])
+>>> data = rpc_interface.get_acl_groups(name='my_group')
+>>> data[0]['users']
+['showard']
+>>> data[0]['hosts']
+['host1']
+>>> data = rpc_interface.get_acl_groups(users__login='showard')
+>>> [acl_group['name'] for acl_group in data]
+['Everyone', 'my_group']
+>>> data = rpc_interface.get_acl_groups(hosts__hostname='host1')
+>>> [acl_group['name'] for acl_group in data]
+['Everyone', 'my_group']
+
+>>> rpc_interface.acl_group_remove_users('my_group', ['showard'])
+>>> rpc_interface.acl_group_remove_hosts('my_group', ['host1'])
+>>> data = rpc_interface.get_acl_groups(name='my_group')
+>>> data[0]['users'] == data[0]['hosts'] == []
+True
+>>> data = rpc_interface.get_acl_groups(users__login='showard')
+>>> [acl_group['name'] for acl_group in data]
+['Everyone']
+>>> data = rpc_interface.get_acl_groups(hosts__hostname='host1')
+>>> [acl_group['name'] for acl_group in data]
+['Everyone']
+
+# job management
+# ############
+
+# note that job functions require job IDs to identify jobs, since job names are
+# not unique
+
+# add some entries to play with
+>>> rpc_interface.add_label(name='my_label', kernel_config='my_kernel_config')
+7L
+>>> test_control_path = os.path.join(test_path, 'test.control')
+>>> rpc_interface.add_test(name='sleeptest', test_type='Client',
+... test_class='Kernel', path=test_control_path)
+2L
+>>> rpc_interface.add_test(name='my_test', test_type='Client',
+... test_class='Kernel', path=test_control_path)
+3L
+
+# generate a control file
+>>> control_file = rpc_interface.generate_control_file(
+... tests=['sleeptest', 'my_test'],
+... kernel='1.2.3.4',
+... label='my_label')
+>>> control_file
+"def step_init():\n\tjob.next_step([step_test])\n\ttestkernel = job.kernel('1.2.3.4')\n\ttestkernel.config('my_kernel_config')\n\ttestkernel.install()\n\ttestkernel.boot(args='')\n\ndef step_test():\n\tjob.run_test('testname')\n\tjob.run_test('testname')"
+>>> print control_file #doctest: +NORMALIZE_WHITESPACE
+def step_init():
+ job.next_step([step_test])
+ testkernel = job.kernel('1.2.3.4')
+ testkernel.config('my_kernel_config')
+ testkernel.install()
+ testkernel.boot(args='')
+<BLANKLINE>
+def step_test():
+ job.run_test('testname')
+ job.run_test('testname')
+
+# create a job to run on host1, host2, and any two machines in my_label
+>>> rpc_interface.create_job(name='my_job',
+... priority='Low',
+... control_file=control_file,
+... control_type='Client',
+... hosts=['host1', 'host2'],
+... meta_hosts=['my_label', 'my_label'])
+1L
+
+# get job info - this does not include status info for particular hosts
+>>> data = rpc_interface.get_jobs()
+>>> data = data[0]
+>>> data['id'], data['owner'], data['name'], data['priority']
+(1L, 'debug_user', 'my_job', 'Low')
+>>> data['control_file'] == control_file
+True
+>>> data['control_type']
+'Client'
+
+>>> data['created_on'] #doctest: +ELLIPSIS
+'200...'
+
+# get_num_jobs - useful when dealing with large numbers of jobs
+>>> rpc_interface.get_num_jobs(name='my_job')
+1L
+
+# check job status, i.e. status for each host in the job
+# job_status requires ID number, not job name
+>>> data = rpc_interface.job_status(1)
+>>> data == {'host1': {'status': 'Queued', 'meta_count': None},
+... 'host2': {'status': 'Queued', 'meta_count': None},
+... 'my_label': {'status': 'Queued', 'meta_count': 2}}
+True
+
+# get_jobs_summary adds status counts to the rest of the get_jobs info
+>>> data = rpc_interface.get_jobs_summary()
+>>> counts = data[0]['status_counts']
+>>> counts
+{'Queued': 4L}
+
+# abort the job
+>>> rpc_interface.abort_job(1)
+>>> data = rpc_interface.get_jobs_summary(id=1)
+>>> data[0]['status_counts']
+{'Aborted': 4L}
+
+
+# extra querying parameters
+# #########################
+
+# get_* methods can take query_start and query_limit arguments to implement
+# paging and a sort_by argument to specify the sort column
+>>> data = rpc_interface.get_hosts(query_limit=1)
+>>> [host['hostname'] for host in data]
+['host1']
+>>> data = rpc_interface.get_hosts(query_start=1, query_limit=1)
+>>> [host['hostname'] for host in data]
+['host2']
+
+# sort_by = '-hostname' indicates sorting in descending order by hostname
+>>> data = rpc_interface.get_hosts(sort_by='-hostname')
+>>> [host['hostname'] for host in data]
+['host2', 'host1']
+
+
+# advanced args
+# #############
+
+# is_synchronous
+>>> rpc_interface.create_job(name='my_job',
+... priority='Low',
+... control_file=control_file,
+... control_type='Server',
+... is_synchronous=False,
+... hosts=['host1'])
+2L
+
+>>> data = rpc_interface.get_jobs()
+>>> data[0]['synch_type']
+'Asynchronous'
diff --git a/frontend/afe/doctests/test.control b/frontend/afe/doctests/test.control
new file mode 100644
index 0000000..653fafe
--- /dev/null
+++ b/frontend/afe/doctests/test.control
@@ -0,0 +1 @@
+job.run_test('testname')
diff --git a/frontend/afe/enum.py b/frontend/afe/enum.py
new file mode 100644
index 0000000..c26ae81
--- /dev/null
+++ b/frontend/afe/enum.py
@@ -0,0 +1,73 @@
+"""\
+Generic enumeration support.
+"""
+
+__author__ = 'showard@google.com (Steve Howard)'
+
+class Enum(object):
+ """\
+ Utility class to implement Enum-like functionality.
+
+ >>> e = Enum('String one', 'String two')
+ >>> e.STRING_ONE
+ 0
+ >>> e.STRING_TWO
+ 1
+ >>> e.choices()
+ [(0, 'String one'), (1, 'String two')]
+ >>> e.get_value('String one')
+ 0
+ >>> e.get_string(0)
+ 'String one'
+
+ >>> e = Enum('Hello', 'Goodbye', string_values=True)
+ >>> e.HELLO, e.GOODBYE
+ ('Hello', 'Goodbye')
+
+ >>> e = Enum('One', 'Two', start_value=1)
+ >>> e.ONE
+ 1
+ >>> e.TWO
+ 2
+ """
+ def __init__(self, *names, **kwargs):
+ self.string_values = kwargs.get('string_values')
+ start_value = kwargs.get('start_value', 0)
+ self.names = names
+ self.values = []
+ for i, name in enumerate(names):
+ if self.string_values:
+ value = name
+ else:
+ value = i + start_value
+ self.values.append(value)
+ setattr(self, self.get_attr_name(name), value)
+
+
+ @staticmethod
+ def get_attr_name(string):
+ return string.upper().replace(' ', '_')
+
+
+ def choices(self):
+ 'Return choice list suitable for Django model choices.'
+ return zip(self.values, self.names)
+
+
+ def get_value(self, name):
+ """\
+ Convert a string name to it's corresponding value. If a value
+ is passed in, it is returned.
+ """
+ if isinstance(name, int) and not self.string_values:
+ # name is already a value
+ return name
+ return getattr(self, self.get_attr_name(name))
+
+
+ def get_string(self, value):
+ ' Given a value, get the string name for it.'
+ if value not in self.values:
+ raise ValueError('Value %s not in this enum' % value)
+ index = self.values.index(value)
+ return self.names[index]
diff --git a/frontend/afe/fixtures/initial_data.json b/frontend/afe/fixtures/initial_data.json
new file mode 100644
index 0000000..3255791
--- /dev/null
+++ b/frontend/afe/fixtures/initial_data.json
@@ -0,0 +1 @@
+[{"pk": "1", "model": "afe.aclgroup", "fields": {"name": "Everyone", "description": ""}}]
diff --git a/frontend/afe/json_rpc/__init__.py b/frontend/afe/json_rpc/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/frontend/afe/json_rpc/__init__.py
diff --git a/frontend/afe/json_rpc/proxy.py b/frontend/afe/json_rpc/proxy.py
new file mode 100644
index 0000000..9b4ff0e
--- /dev/null
+++ b/frontend/afe/json_rpc/proxy.py
@@ -0,0 +1,54 @@
+
+"""
+ Copyright (c) 2007 Jan-Klaas Kollhof
+
+ This file is part of jsonrpc.
+
+ jsonrpc is free software; you can redistribute it and/or modify
+ it under the terms of the GNU Lesser General Public License as published by
+ the Free Software Foundation; either version 2.1 of the License, or
+ (at your option) any later version.
+
+ This software is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public License
+ along with this software; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+"""
+
+import urllib2
+from frontend.afe.simplejson import decoder, encoder
+json_encoder = encoder.JSONEncoder()
+json_decoder = decoder.JSONDecoder()
+
+class JSONRPCException(Exception):
+ pass
+
+class ServiceProxy(object):
+ def __init__(self, serviceURL, serviceName=None, headers=None):
+ self.__serviceURL = serviceURL
+ self.__serviceName = serviceName
+ self.__headers = headers
+
+ def __getattr__(self, name):
+ if self.__serviceName != None:
+ name = "%s.%s" % (self.__serviceName, name)
+ return ServiceProxy(self.__serviceURL, name, self.__headers)
+
+ def __call__(self, *args, **kwargs):
+ postdata = json_encoder.encode({"method": self.__serviceName,
+ 'params': args + (kwargs,),
+ 'id':'jsonrpc'})
+ request = urllib2.Request(self.__serviceURL, data=postdata,
+ headers=self.__headers)
+ respdata = urllib2.urlopen(request).read()
+ resp = json_decoder.decode(respdata)
+ if resp['error'] != None:
+ error_message = (resp['error']['name'] + ': ' +
+ resp['error']['message'])
+ raise JSONRPCException(error_message)
+ else:
+ return resp['result']
diff --git a/frontend/afe/json_rpc/serviceHandler.py b/frontend/afe/json_rpc/serviceHandler.py
new file mode 100644
index 0000000..1f6036c
--- /dev/null
+++ b/frontend/afe/json_rpc/serviceHandler.py
@@ -0,0 +1,145 @@
+
+"""
+ Copyright (c) 2007 Jan-Klaas Kollhof
+
+ This file is part of jsonrpc.
+
+ jsonrpc is free software; you can redistribute it and/or modify
+ it under the terms of the GNU Lesser General Public License as published by
+ the Free Software Foundation; either version 2.1 of the License, or
+ (at your option) any later version.
+
+ This software is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public License
+ along with this software; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+"""
+
+import traceback
+
+from frontend.afe.simplejson import decoder, encoder
+
+def customConvertJson(value):
+ """\
+ Recursively process JSON values and do type conversions.
+ -change floats to ints
+ -change unicodes to strs
+ """
+ if isinstance(value, float):
+ return int(value)
+ elif isinstance(value, unicode):
+ return str(value)
+ elif isinstance(value, list):
+ return [customConvertJson(item) for item in value]
+ elif isinstance(value, dict):
+ new_dict = {}
+ for key, val in value.iteritems():
+ new_key = customConvertJson(key)
+ new_val = customConvertJson(val)
+ new_dict[new_key] = new_val
+ return new_dict
+ else:
+ return value
+
+json_encoder = encoder.JSONEncoder()
+json_decoder = decoder.JSONDecoder()
+
+
+def ServiceMethod(fn):
+ fn.IsServiceMethod = True
+ return fn
+
+class ServiceException(Exception):
+ pass
+
+class ServiceRequestNotTranslatable(ServiceException):
+ pass
+
+class BadServiceRequest(ServiceException):
+ pass
+
+class ServiceMethodNotFound(ServiceException):
+ pass
+
+class ServiceHandler(object):
+
+ def __init__(self, service):
+ self.service=service
+
+ def handleRequest(self, json):
+ err=None
+ result = None
+ id_=''
+
+ #print 'Request:', json
+
+ try:
+ req = self.translateRequest(json)
+ except ServiceRequestNotTranslatable, e:
+ err = e
+ req={'id':id_}
+
+ if err==None:
+ try:
+ id_ = req['id']
+ methName = req['method']
+ args = req['params']
+ except:
+ err = BadServiceRequest(json)
+
+ if err == None:
+ try:
+ meth = self.findServiceEndpoint(methName)
+ except Exception, e:
+ traceback.print_exc()
+ err = e
+
+ if err == None:
+ try:
+ result = self.invokeServiceEndpoint(meth, args)
+ except Exception, e:
+ traceback.print_exc()
+ err = e
+ resultdata = self.translateResult(result, err, id_)
+
+ return resultdata
+
+ def translateRequest(self, data):
+ try:
+ req = json_decoder.decode(data)
+ except:
+ raise ServiceRequestNotTranslatable(data)
+ req = customConvertJson(req) # -srh
+ return req
+
+ def findServiceEndpoint(self, name):
+ try:
+ meth = getattr(self.service, name)
+# -srh
+# if getattr(meth, "IsServiceMethod"):
+ return meth
+# else:
+# raise ServiceMethodNotFound(name)
+ except AttributeError:
+ raise ServiceMethodNotFound(name)
+
+ def invokeServiceEndpoint(self, meth, args):
+ return meth(*args)
+
+ def translateResult(self, rslt, err, id_):
+ if err != None:
+ err = {"name": err.__class__.__name__, "message":str(err)}
+ rslt = None
+
+ try:
+ data = json_encoder.encode({"result":rslt,"id":id_,"error":err})
+ except TypeError, e:
+ traceback.print_exc()
+ err = {"name": "JSONEncodeException", "message":"Result Object Not Serializable"}
+ data = json_encoder.encode({"result":None, "id":id_,"error":err})
+
+ return data
diff --git a/frontend/afe/management.py b/frontend/afe/management.py
new file mode 100644
index 0000000..c8f8380
--- /dev/null
+++ b/frontend/afe/management.py
@@ -0,0 +1,39 @@
+# use some undocumented Django tricks to execute custom logic after syncdb
+
+from django.dispatch import dispatcher
+from django.db.models import signals
+import frontend.afe.models
+from django.contrib import auth
+
+BASIC_ADMIN = 'Basic admin'
+
+def create_admin_group(app, created_models, verbosity, **kwargs):
+ """\
+ Create a basic admin group with permissions for managing basic autotest
+ objects.
+ """
+ admin_group, created = auth.models.Group.objects.get_or_create(
+ name=BASIC_ADMIN)
+ admin_group.save() # must save before adding permissions
+ PermissionModel = auth.models.Permission
+ have_permissions = list(admin_group.permissions.all())
+ for model_name in ('host', 'label', 'test', 'acl_group'):
+ for permission_type in ('add', 'change', 'delete'):
+ codename = permission_type + '_' + model_name
+ permissions = list(PermissionModel.objects.filter(
+ codename=codename))
+ if len(permissions) == 0:
+ print ' No permission ' + codename
+ continue
+ for permission in permissions:
+ if permission not in have_permissions:
+ print ' Adding permission ' + codename
+ admin_group.permissions.add(permission)
+ if created:
+ print 'Created group "%s"' % BASIC_ADMIN
+ else:
+ print 'Group "%s" already exists' % BASIC_ADMIN
+
+
+dispatcher.connect(create_admin_group, sender=frontend.afe.models,
+ signal=signals.post_syncdb)
diff --git a/frontend/afe/models.py b/frontend/afe/models.py
new file mode 100644
index 0000000..4696a22
--- /dev/null
+++ b/frontend/afe/models.py
@@ -0,0 +1,746 @@
+import datetime
+from django.db import models as dbmodels
+from frontend.afe import enum
+from frontend import settings
+
+
+class ValidationError(Exception):
+ """\
+ Data validation error in adding or updating an object. The associated
+ value is a dictionary mapping field names to error strings.
+ """
+
+
+class AclAccessViolation(Exception):
+ """\
+ Raised when an operation is attempted with proper permissions as
+ dictated by ACLs.
+ """
+
+
+class ModelExtensions(object):
+ """\
+ Mixin with convenience functions for models, built on top of the
+ default Django model functions.
+ """
+ # TODO: at least some of these functions really belong in a custom
+ # Manager class
+
+ field_dict = None
+ # subclasses should override if they want to support smart_get() by name
+ name_field = None
+
+
+ @classmethod
+ def get_field_dict(cls):
+ if cls.field_dict is None:
+ cls.field_dict = {}
+ for field in cls._meta.fields:
+ cls.field_dict[field.name] = field
+ return cls.field_dict
+
+
+ @classmethod
+ def clean_foreign_keys(cls, data):
+ """\
+ Convert foreign key fields in data from <field>_id to just
+ <field>
+ """
+ for field in cls._meta.fields:
+ if field.rel:
+ data[field.name] = data[field.column]
+ del data[field.column]
+
+
+ # TODO(showard) - is there a way to not have to do this?
+ @classmethod
+ def provide_default_values(cls, data):
+ """\
+ Provide default values for fields with default values which have
+ nothing passed in.
+
+ For CharField and TextField fields with "blank=True", if nothing
+ is passed, we fill in an empty string value, even if there's no
+ default set.
+ """
+ new_data = dict(data)
+ field_dict = cls.get_field_dict()
+ for name, obj in field_dict.iteritems():
+ if data.get(name) is not None:
+ continue
+ if obj.default is not dbmodels.fields.NOT_PROVIDED:
+ new_data[name] = obj.default
+ elif (isinstance(obj, dbmodels.CharField) or
+ isinstance(obj, dbmodels.TextField)):
+ new_data[name] = ''
+ return new_data
+
+
+ @classmethod
+ def convert_human_readable_values(cls, data, to_human_readable=False):
+ """\
+ Performs conversions on user-supplied field data, to make it
+ easier for users to pass human-readable data.
+
+ For all fields that have choice sets, convert their values
+ from human-readable strings to enum values, if necessary. This
+ allows users to pass strings instead of the corresponding
+ integer values.
+
+ For all foreign key fields, call smart_get with the supplied
+ data. This allows the user to pass either an ID value or
+ the name of the object as a string.
+
+ If to_human_readable=True, perform the inverse - i.e. convert
+ numeric values to human readable values.
+ """
+ new_data = dict(data)
+ field_dict = cls.get_field_dict()
+ for field_name in data:
+ field_obj = field_dict[field_name]
+ # convert enum values
+ if field_obj.choices:
+ for choice_data in field_obj.choices:
+ # choice_data is (value, name)
+ if to_human_readable:
+ from_val, to_val = choice_data
+ else:
+ to_val, from_val = choice_data
+ if from_val == data[field_name]:
+ new_data[field_name] = to_val
+ break
+ # convert foreign key values
+ elif field_obj.rel:
+ dest_obj = field_obj.rel.to.smart_get(
+ data[field_name])
+ if to_human_readable:
+ new_data[field_name] = (
+ getattr(dest_obj,
+ dest_obj.name_field))
+ else:
+ new_data[field_name] = dest_obj.id
+ return new_data
+
+
+ @classmethod
+ def validate_field_names(cls, data):
+ 'Checks for extraneous fields in data.'
+ errors = {}
+ field_dict = cls.get_field_dict()
+ for field_name in data:
+ if field_name not in field_dict:
+ errors[field_name] = 'No field of this name'
+ return errors
+
+
+ @classmethod
+ def prepare_data_args(cls, data, kwargs):
+ 'Common preparation for add_object and update_object'
+ data = dict(data) # don't modify the default keyword arg
+ data.update(kwargs)
+ # must check for extraneous field names here, while we have the
+ # data in a dict
+ errors = cls.validate_field_names(data)
+ if errors:
+ raise ValidationError(errors)
+ data = cls.convert_human_readable_values(data)
+ return data
+
+
+ def validate_unique(self):
+ """\
+ Validate that unique fields are unique. Django manipulators do
+ this too, but they're a huge pain to use manually. Trust me.
+ """
+ errors = {}
+ cls = type(self)
+ field_dict = self.get_field_dict()
+ for field_name, field_obj in field_dict.iteritems():
+ if not field_obj.unique:
+ continue
+
+ value = getattr(self, field_name)
+ existing_objs = cls.objects.filter(
+ **{field_name : value})
+ num_existing = existing_objs.count()
+
+ if num_existing == 0:
+ continue
+ if num_existing == 1 and existing_objs[0].id == self.id:
+ continue
+ errors[field_name] = (
+ 'This value must be unique (%s)' % (value))
+ return errors
+
+
+ def do_validate(self):
+ errors = self.validate()
+ unique_errors = self.validate_unique()
+ for field_name, error in unique_errors.iteritems():
+ errors.setdefault(field_name, error)
+ if errors:
+ raise ValidationError(errors)
+
+
+ # actually (externally) useful methods follow
+
+ @classmethod
+ def add_object(cls, data={}, **kwargs):
+ """\
+ Returns a new object created with the given data (a dictionary
+ mapping field names to values). Merges any extra keyword args
+ into data.
+ """
+ data = cls.prepare_data_args(data, kwargs)
+ data = cls.provide_default_values(data)
+ obj = cls(**data)
+ obj.do_validate()
+ obj.save()
+ return obj
+
+
+ def update_object(self, data={}, **kwargs):
+ """\
+ Updates the object with the given data (a dictionary mapping
+ field names to values). Merges any extra keyword args into
+ data.
+ """
+ data = self.prepare_data_args(data, kwargs)
+ for field_name, value in data.iteritems():
+ if value is not None:
+ setattr(self, field_name, value)
+ self.do_validate()
+ self.save()
+
+
+ @classmethod
+ def query_objects(cls, filter_data):
+ """\
+ Returns a QuerySet object for querying the given model_class
+ with the given filter_data. Optional special arguments in
+ filter_data include:
+ -query_start: index of first return to return
+ -query_limit: maximum number of results to return
+ -sort_by: name of field to sort on. prefixing a '-' onto this
+ argument changes the sort to descending order.
+ -extra_args: keyword args to pass to query.extra() (see Django
+ DB layer documentation)
+ """
+ query_start = filter_data.pop('query_start', None)
+ query_limit = filter_data.pop('query_limit', None)
+ if query_start and not query_limit:
+ raise ValueError('Cannot pass query_start without '
+ 'query_limit')
+ sort_by = filter_data.pop('sort_by', None)
+ extra_args = filter_data.pop('extra_args', None)
+ query_dict = {}
+ for field, value in filter_data.iteritems():
+ query_dict[field] = value
+ query = cls.objects.filter(**query_dict)
+ if extra_args:
+ query = query.extra(**extra_args)
+ if sort_by:
+ query = query.order_by(sort_by)
+ if query_start is not None and query_limit is not None:
+ query_limit += query_start
+ return query[query_start:query_limit]
+
+
+ @classmethod
+ def list_objects(cls, filter_data):
+ """\
+ Like query_objects, but return a list of dictionaries.
+ """
+ query = cls.query_objects(filter_data)
+ field_dicts = list(query.values())
+ for i in range(len(field_dicts)):
+ cls.clean_foreign_keys(field_dicts[i])
+ field_dicts[i] = cls.convert_human_readable_values(
+ field_dicts[i], to_human_readable=True)
+ return field_dicts
+
+
+ @classmethod
+ def smart_get(cls, *args, **kwargs):
+ """\
+ smart_get(integer) -> get object by ID
+ smart_get(string) -> get object by name_field
+ smart_get(keyword args) -> normal ModelClass.objects.get()
+ """
+ assert bool(args) ^ bool(kwargs)
+ if args:
+ assert len(args) == 1
+ arg = args[0]
+ if isinstance(arg, int) or isinstance(arg, long):
+ return cls.objects.get(id=arg)
+ if isinstance(arg, str) or isinstance(arg, unicode):
+ return cls.objects.get(
+ **{cls.name_field : arg})
+ raise ValueError(
+ 'Invalid positional argument: %s (%s)' % (
+ str(arg), type(arg)))
+ return cls.objects.get(**kwargs)
+
+
+class Label(dbmodels.Model, ModelExtensions):
+ """\
+ Required:
+ name: label name
+
+ Optional:
+ kernel_config: url/path to kernel config to use for jobs run on this
+ label
+ platform: if True, this is a platform label (defaults to False)
+ """
+ name = dbmodels.CharField(maxlength=255, unique=True)
+ kernel_config = dbmodels.CharField(maxlength=255, blank=True)
+ platform = dbmodels.BooleanField(default=False)
+
+ name_field = 'name'
+
+
+ def enqueue_job(self, job):
+ 'Enqueue a job on any host of this label.'
+ queue_entry = HostQueueEntry(meta_host=self, job=job,
+ status=Job.Status.QUEUED,
+ priority=job.priority)
+ queue_entry.save()
+
+
+ def block_auto_assign(self, job):
+ """\
+ Placeholder to allow labels to be used in place of hosts
+ (as meta-hosts).
+ """
+ pass
+
+
+ class Meta:
+ db_table = 'labels'
+
+ class Admin:
+ list_display = ('name', 'kernel_config')
+
+ def __str__(self):
+ return self.name
+
+
+class Host(dbmodels.Model, ModelExtensions):
+ """\
+ Required:
+ hostname
+
+ optional:
+ locked: host is locked and will not be queued
+
+ Internal:
+ synch_id: currently unused
+ status: string describing status of host
+ """
+ Status = enum.Enum('Verifying', 'Running', 'Ready', 'Repairing',
+ 'Repair Failed', 'Dead', string_values=True)
+
+ hostname = dbmodels.CharField(maxlength=255, unique=True)
+ labels = dbmodels.ManyToManyField(Label, blank=True,
+ filter_interface=dbmodels.HORIZONTAL)
+ locked = dbmodels.BooleanField(default=False)
+ synch_id = dbmodels.IntegerField(blank=True, null=True)
+ status = dbmodels.CharField(maxlength=255, default=Status.READY,
+ choices=Status.choices())
+
+ name_field = 'hostname'
+
+
+ def save(self):
+ # is this a new object being saved for the first time?
+ first_time = (self.id is None)
+ super(Host, self).save()
+ if first_time:
+ everyone = AclGroup.objects.get(name='Everyone')
+ everyone.hosts.add(self)
+
+
+ def enqueue_job(self, job):
+ ' Enqueue a job on this host.'
+ queue_entry = HostQueueEntry(host=self, job=job,
+ status=Job.Status.QUEUED,
+ priority=job.priority)
+ # allow recovery of dead hosts from the frontend
+ if not self.active_queue_entry():
+ self.status = Host.Status.READY
+ self.save()
+ queue_entry.save()
+
+
+ def block_auto_assign(self, job):
+ 'Block this host from being assigned to a job.'
+ block = IneligibleHostQueue(job=job, host=self)
+ block.save()
+
+
+ def platform(self):
+ # TODO(showard): slighly hacky?
+ platforms = self.labels.filter(platform=True)
+ if len(platforms) == 0:
+ return None
+ return platforms[0]
+ platform.short_description = 'Platform'
+
+
+ def is_dead(self):
+ return self.status in (Host.Status.DEAD,
+ Host.Status.REPAIR_FAILED)
+
+
+ def active_queue_entry(self):
+ active = list(self.hostqueueentry_set.filter(active=True))
+ if not active:
+ return None
+ assert len(active) == 1, ('More than one active entry for '
+ 'host ' + self.hostname)
+ return active[0]
+
+
+ class Meta:
+ db_table = 'hosts'
+
+ class Admin:
+ # TODO(showard) - showing platform requires a SQL query for
+ # each row (since labels are many-to-many) - should we remove
+ # it?
+ if not settings.FULL_ADMIN:
+ fields = (
+ (None, {'fields' :
+ ('hostname', 'status', 'locked',
+ 'labels')}),
+ )
+ list_display = ('hostname', 'platform', 'locked', 'status')
+ list_filter = ('labels', 'locked')
+ search_fields = ('hostname', 'status')
+
+ def __str__(self):
+ return self.hostname
+
+
+class Test(dbmodels.Model, ModelExtensions):
+ """\
+ Required:
+ name: test name
+ test_type: Client or Server
+ path: path to pass to run_test()
+
+ Optional:
+ test_class: used for categorization of tests
+ description: arbirary text description
+ """
+ Classes = enum.Enum('Kernel', 'Hardware', 'Canned Test Sets',
+ string_values=True)
+ # TODO(showard) - this should be merged with Job.ControlType (but right
+ # now they use opposite values)
+ Types = enum.Enum('Client', 'Server', start_value=1)
+
+ name = dbmodels.CharField(maxlength=255, unique=True)
+ test_class = dbmodels.CharField(maxlength=255,
+ choices=Classes.choices())
+ description = dbmodels.TextField(blank=True)
+ test_type = dbmodels.SmallIntegerField(choices=Types.choices())
+ path = dbmodels.CharField(maxlength=255)
+
+ name_field = 'name'
+
+
+ class Meta:
+ db_table = 'autotests'
+
+ class Admin:
+ fields = (
+ (None, {'fields' :
+ ('name', 'test_class', 'test_type', 'path',
+ 'description')}),
+ )
+ list_display = ('name', 'test_type', 'description')
+ search_fields = ('name',)
+
+ def __str__(self):
+ return self.name
+
+
+class User(dbmodels.Model, ModelExtensions):
+ """\
+ Required:
+ login :user login name
+
+ Optional:
+ access_level: 0=User (default), 1=Admin, 100=Root
+ """
+ ACCESS_ROOT = 100
+ ACCESS_ADMIN = 1
+ ACCESS_USER = 0
+
+ login = dbmodels.CharField(maxlength=255, unique=True)
+ access_level = dbmodels.IntegerField(default=ACCESS_USER, blank=True)
+
+ name_field = 'login'
+
+
+ def save(self):
+ # is this a new object being saved for the first time?
+ first_time = (self.id is None)
+ super(User, self).save()
+ if first_time:
+ everyone = AclGroup.objects.get(name='Everyone')
+ everyone.users.add(self)
+
+
+ def has_access(self, target):
+ if self.access_level >= self.ACCESS_ROOT:
+ return True
+
+ if isinstance(target, int):
+ return self.access_level >= target
+ if isinstance(target, Job):
+ return (target.owner == self.login or
+ self.access_level >= self.ACCESS_ADMIN)
+ if isinstance(target, Host):
+ acl_intersect = [group
+ for group in self.aclgroup_set.all()
+ if group in target.aclgroup_set.all()]
+ return bool(acl_intersect)
+ if isinstance(target, User):
+ return self.access_level >= target.access_level
+ raise ValueError('Invalid target type')
+
+
+ class Meta:
+ db_table = 'users'
+
+ class Admin:
+ list_display = ('login', 'access_level')
+ search_fields = ('login',)
+
+ def __str__(self):
+ return self.login
+
+
+class AclGroup(dbmodels.Model, ModelExtensions):
+ """\
+ Required:
+ name: name of ACL group
+
+ Optional:
+ description: arbitrary description of group
+ """
+ name = dbmodels.CharField(maxlength=255, unique=True)
+ description = dbmodels.CharField(maxlength=255, blank=True)
+ users = dbmodels.ManyToManyField(User,
+ filter_interface=dbmodels.HORIZONTAL)
+ hosts = dbmodels.ManyToManyField(Host,
+ filter_interface=dbmodels.HORIZONTAL)
+
+ name_field = 'name'
+
+
+ class Meta:
+ db_table = 'acl_groups'
+
+ class Admin:
+ list_display = ('name', 'description')
+ search_fields = ('name',)
+
+ def __str__(self):
+ return self.name
+
+# hack to make the column name in the many-to-many DB tables match the one
+# generated by ruby
+AclGroup._meta.object_name = 'acl_group'
+
+
+class JobManager(dbmodels.Manager):
+ 'Custom manager to provide efficient status counts querying.'
+ def get_status_counts(self, job_ids):
+ """\
+ Returns a dictionary mapping the given job IDs to their status
+ count dictionaries.
+ """
+ if not job_ids:
+ return {}
+ id_list = '(%s)' % ','.join(str(job_id) for job_id in job_ids)
+ from django.db import connection
+ cursor = connection.cursor()
+ cursor.execute("""
+ SELECT job_id, status, COUNT(*)
+ FROM host_queue_entries
+ WHERE job_id IN %s
+ GROUP BY job_id, status
+ """ % id_list)
+ all_job_counts = {}
+ for job_id in job_ids:
+ all_job_counts[job_id] = {}
+ for job_id, status, count in cursor.fetchall():
+ all_job_counts[job_id][status] = count
+ return all_job_counts
+
+
+class Job(dbmodels.Model, ModelExtensions):
+ """\
+ owner: username of job owner
+ name: job name (does not have to be unique)
+ priority: Low, Medium, High, Urgent (or 0-3)
+ control_file: contents of control file
+ control_type: Client or Server
+ created_on: date of job creation
+ submitted_on: date of job submission
+ synch_type: Asynchronous or Synchronous (i.e. job must run on all hosts
+ simultaneously; used for server-side control files)
+ synch_count: ???
+ synchronizing: for scheduler use
+ """
+ Priority = enum.Enum('Low', 'Medium', 'High', 'Urgent')
+ ControlType = enum.Enum('Server', 'Client', start_value=1)
+ SynchType = enum.Enum('Asynchronous', 'Synchronous', start_value=1)
+ Status = enum.Enum('Created', 'Queued', 'Pending', 'Running',
+ 'Completed', 'Aborted', 'Failed', string_values=True)
+
+ owner = dbmodels.CharField(maxlength=255)
+ name = dbmodels.CharField(maxlength=255)
+ priority = dbmodels.SmallIntegerField(choices=Priority.choices(),
+ blank=True, # to allow 0
+ default=Priority.MEDIUM)
+ control_file = dbmodels.TextField()
+ control_type = dbmodels.SmallIntegerField(choices=ControlType.choices(),
+ blank=True) # to allow 0
+ created_on = dbmodels.DateTimeField(auto_now_add=True)
+ synch_type = dbmodels.SmallIntegerField(blank=True, null=True,
+ choices=SynchType.choices())
+ synch_count = dbmodels.IntegerField(blank=True, null=True)
+ synchronizing = dbmodels.BooleanField(default=False)
+
+
+ # custom manager
+ objects = JobManager()
+
+
+ def is_server_job(self):
+ return self.control_type == self.ControlType.SERVER
+
+
+ @classmethod
+ def create(cls, owner, name, priority, control_file, control_type,
+ hosts, synch_type):
+ """\
+ Creates a job by taking some information (the listed args)
+ and filling in the rest of the necessary information.
+ """
+ job = cls.add_object(
+ owner=owner, name=name, priority=priority,
+ control_file=control_file, control_type=control_type,
+ synch_type=synch_type)
+
+ if job.synch_type == cls.SynchType.SYNCHRONOUS:
+ job.synch_count = len(hosts)
+ if not job.is_server_job():
+ errors = {'synch_type':
+ 'client jobs cannot be synchronous'}
+ raise ValidationError(errors)
+ else:
+ if len(hosts) == 0:
+ errors = {'hosts':
+ 'asynchronous jobs require at least'
+ + ' one host to run on'}
+ raise ValidationError(errors)
+ job.status = cls.Status.CREATED
+ job.save()
+ return job
+
+
+ def queue(self, hosts):
+ 'Enqueue a job on the given hosts.'
+ self.status = self.Status.QUEUED
+ for host in hosts:
+ host.enqueue_job(self)
+ host.block_auto_assign(self)
+ self.submitted_on = datetime.datetime.now()
+ self.save()
+
+
+ def abort(self):
+ self.status = Job.Status.ABORTED
+ self.save()
+ for queue_entry in self.hostqueueentry_set.all():
+ if queue_entry.active:
+ host = queue_entry.host
+ if host:
+ host.status = Host.Status.DEAD
+ host.save()
+ if not queue_entry.complete:
+ queue_entry.status = Job.Status.ABORTED
+ queue_entry.active = False
+ queue_entry.complete = True
+ queue_entry.save()
+
+
+ def user(self):
+ try:
+ return User.objects.get(login=self.owner)
+ except self.DoesNotExist:
+ return None
+
+
+ class Meta:
+ db_table = 'jobs'
+
+ if settings.FULL_ADMIN:
+ class Admin:
+ list_display = ('id', 'owner', 'name', 'control_type',
+ 'status')
+
+ def __str__(self):
+ return '%s (%s-%s)' % (self.name, self.id, self.owner)
+
+
+class IneligibleHostQueue(dbmodels.Model):
+ job = dbmodels.ForeignKey(Job)
+ host = dbmodels.ForeignKey(Host)
+
+ class Meta:
+ db_table = 'ineligible_host_queues'
+
+ if settings.FULL_ADMIN:
+ class Admin:
+ list_display = ('id', 'job', 'host')
+
+
+class HostQueueEntry(dbmodels.Model):
+ job = dbmodels.ForeignKey(Job)
+ host = dbmodels.ForeignKey(Host, blank=True, null=True)
+ priority = dbmodels.SmallIntegerField()
+ status = dbmodels.CharField(maxlength=255)
+ meta_host = dbmodels.ForeignKey(Label, blank=True, null=True,
+ db_column='meta_host')
+ active = dbmodels.BooleanField(default=False)
+ complete = dbmodels.BooleanField(default=False)
+
+
+ def is_meta_host_entry(self):
+ 'True if this is a entry has a meta_host instead of a host.'
+ return self.host is None and self.meta_host is not None
+
+
+ def save(self):
+ if self.host:
+ user = self.job.user()
+ if user is None or not user.has_access(self.host):
+ raise AclAccessViolation(
+ 'User %s does not have permission to '
+ 'access host %s' % (self.job.owner,
+ self.host.hostname))
+ super(HostQueueEntry, self).save()
+
+
+ class Meta:
+ db_table = 'host_queue_entries'
+
+ if settings.FULL_ADMIN:
+ class Admin:
+ list_display = ('id', 'job', 'host', 'status',
+ 'meta_host')
diff --git a/frontend/afe/rpc_client_lib.py b/frontend/afe/rpc_client_lib.py
new file mode 100644
index 0000000..c970b7d
--- /dev/null
+++ b/frontend/afe/rpc_client_lib.py
@@ -0,0 +1,14 @@
+"""\
+This module provides a get_proxy() function, which should be used to access
+the afe RPC server.
+"""
+
+__author__ = 'showard@google.com (Steve Howard)'
+
+# we currently use xmlrpclib, but we modify it slightly to support keyword args
+# in function calls (with corresponding support in the server)
+
+from json_rpc import proxy
+
+def get_proxy(*args, **kwargs):
+ return proxy.ServiceProxy(*args, **kwargs)
diff --git a/frontend/afe/rpc_handler.py b/frontend/afe/rpc_handler.py
new file mode 100644
index 0000000..ecf730a
--- /dev/null
+++ b/frontend/afe/rpc_handler.py
@@ -0,0 +1,62 @@
+"""\
+RPC request handler Django. Exposed RPC interface functions should be
+defined in rpc_interface.py.
+"""
+
+__author__ = 'showard@google.com (Steve Howard)'
+
+import django.http
+import traceback, pydoc
+
+from frontend.afe.json_rpc import serviceHandler
+from frontend.afe import rpc_interface, rpc_utils, site_rpc_interface
+
+# since site_rpc_interface is later in the list, its methods will override those
+# of rpc_interface
+RPC_INTERFACE_MODULES = (rpc_interface, site_rpc_interface)
+
+class RpcMethodHolder(object):
+ 'Dummy class to hold RPC interface methods as attributes.'
+
+rpc_methods = RpcMethodHolder()
+
+dispatcher = serviceHandler.ServiceHandler(rpc_methods)
+
+# get documentation for rpc_interface we can send back to the user
+html_doc = pydoc.html.document(rpc_interface)
+
+def rpc_handler(request):
+ rpc_utils.set_user(request.afe_user)
+ response = django.http.HttpResponse()
+ if len(request.POST):
+ response.write(dispatcher.handleRequest(request.raw_post_data))
+ else:
+ response.write(html_doc)
+
+ response['Content-length'] = str(len(response.content))
+ return response
+
+
+def allow_keyword_args(f):
+ """\
+ Decorator to allow a function to take keyword args even though the RPC
+ layer doesn't support that. The decorated function assumes its last
+ argument is a dictionary of keyword args and passes them to the original
+ function as keyword args.
+ """
+ def new_fn(*args):
+ assert args
+ keyword_args = args[-1]
+ args = args[:-1]
+ return f(*args, **keyword_args)
+ new_fn.func_name = f.func_name
+ return new_fn
+
+# decorate all functions in rpc_interface to take keyword args
+function_type = type(rpc_handler) # could be any function
+for module in RPC_INTERFACE_MODULES:
+ for name in dir(module):
+ thing = getattr(module, name)
+ if type(thing) is function_type:
+ decorated_function = allow_keyword_args(thing)
+ setattr(rpc_methods, name, decorated_function)
diff --git a/frontend/afe/rpc_interface.py b/frontend/afe/rpc_interface.py
new file mode 100644
index 0000000..a88085c
--- /dev/null
+++ b/frontend/afe/rpc_interface.py
@@ -0,0 +1,353 @@
+"""\
+Functions to expose over the RPC interface.
+
+For all modify* and delete* functions that ask for an 'id' parameter to
+identify the object to operate on, the id may be either
+ * the database row ID
+ * the name of the object (label name, hostname, user login, etc.)
+ * a dictionary containing uniquely identifying field (this option should seldom
+ be used)
+
+When specifying foreign key fields (i.e. adding hosts to a label, or adding
+users to an ACL group), the given value may be either the database row ID or the
+name of the object.
+
+All get* functions return lists of dictionaries. Each dictionary represents one
+object and maps field names to values.
+
+Some examples:
+modify_host(2, hostname='myhost') # modify hostname of host with database ID 2
+modify_host('ipaj2', hostname='myhost') # modify hostname of host 'ipaj2'
+modify_test('sleeptest', test_type='Client', params=', seconds=60')
+delete_acl_group(1) # delete by ID
+delete_acl_group('Everyone') # delete by name
+acl_group_add_users('Everyone', ['mbligh', 'showard'])
+get_jobs(owner='showard', status='Queued')
+
+See doctests/rpc_test.txt for (lots) more examples.
+"""
+
+__author__ = 'showard@google.com (Steve Howard)'
+
+import models, control_file, rpc_utils
+
+# labels
+
+def add_label(name, kernel_config=None, platform=None):
+ return models.Label.add_object(name=name, kernel_config=kernel_config,
+ platform=platform).id
+
+
+def modify_label(id, **data):
+ models.Label.smart_get(id).update_object(data)
+
+
+def delete_label(id):
+ models.Label.smart_get(id).delete()
+
+
+def get_labels(**filter_data):
+ return rpc_utils.prepare_for_serialization(
+ models.Label.list_objects(filter_data))
+
+
+# hosts
+
+def add_host(hostname, status=None, locked=None):
+ return models.Host.add_object(hostname=hostname, status=status,
+ locked=locked).id
+
+
+def modify_host(id, **data):
+ models.Host.smart_get(id).update_object(data)
+
+
+def host_add_labels(id, labels):
+ labels = [models.Label.smart_get(label) for label in labels]
+ models.Host.smart_get(id).labels.add(*labels)
+
+
+def host_remove_labels(id, labels):
+ labels = [models.Label.smart_get(label) for label in labels]
+ models.Host.smart_get(id).labels.remove(*labels)
+
+
+def delete_host(id):
+ models.Host.smart_get(id).delete()
+
+
+def get_hosts(**filter_data):
+ hosts = models.Host.list_objects(filter_data)
+ for host in hosts:
+ host_obj = models.Host.objects.get(id=host['id'])
+ host['labels'] = [label.name
+ for label in host_obj.labels.all()]
+ platform = host_obj.platform()
+ host['platform'] = platform and platform.name or None
+ return rpc_utils.prepare_for_serialization(hosts)
+
+
+
+# tests
+
+def add_test(name, test_type, path, test_class=None, description=None):
+ return models.Test.add_object(name=name, test_type=test_type, path=path,
+ test_class=test_class,
+ description=description).id
+
+
+def modify_test(id, **data):
+ models.Test.smart_get(id).update_object(data)
+
+
+def delete_test(id):
+ models.Test.smart_get(id).delete()
+
+
+def get_tests(**filter_data):
+ return rpc_utils.prepare_for_serialization(
+ models.Test.list_objects(filter_data))
+
+
+# users
+
+def add_user(login, access_level=None):
+ return models.User.add_object(login=login, access_level=access_level).id
+
+
+def modify_user(id, **data):
+ models.User.smart_get(id).update_object(data)
+
+
+def delete_user(id):
+ models.User.smart_get(id).delete()
+
+
+def get_users(**filter_data):
+ return rpc_utils.prepare_for_serialization(
+ models.User.list_objects(filter_data))
+
+
+# acl groups
+
+def add_acl_group(name, description=None):
+ return models.AclGroup.add_object(name=name, description=description).id
+
+
+def modify_acl_group(id, **data):
+ models.AclGroup.smart_get(id).update_object(data)
+
+
+def acl_group_add_users(id, users):
+ users = [models.User.smart_get(user) for user in users]
+ models.AclGroup.smart_get(id).users.add(*users)
+
+
+def acl_group_remove_users(id, users):
+ users = [models.User.smart_get(user) for user in users]
+ models.AclGroup.smart_get(id).users.remove(*users)
+
+
+def acl_group_add_hosts(id, hosts):
+ hosts = [models.Host.smart_get(host) for host in hosts]
+ models.AclGroup.smart_get(id).hosts.add(*hosts)
+
+
+def acl_group_remove_hosts(id, hosts):
+ hosts = [models.Host.smart_get(host) for host in hosts]
+ models.AclGroup.smart_get(id).hosts.remove(*hosts)
+
+
+def delete_acl_group(id):
+ models.AclGroup.smart_get(id).delete()
+
+
+def get_acl_groups(**filter_data):
+ acl_groups = models.AclGroup.list_objects(filter_data)
+ for acl_group in acl_groups:
+ acl_group_obj = models.AclGroup.objects.get(id=acl_group['id'])
+ acl_group['users'] = [user.login
+ for user in acl_group_obj.users.all()]
+ acl_group['hosts'] = [host.hostname
+ for host in acl_group_obj.hosts.all()]
+ return rpc_utils.prepare_for_serialization(acl_groups)
+
+
+# jobs
+
+def generate_control_file(tests, kernel=None, label=None):
+ """\
+ Generates a client-side control file to load a kernel and run a set of
+ tests.
+
+ tests: list of tests to run
+ kernel: kernel to install in generated control file
+ label: name of label to grab kernel config from
+ do_push_kernel: if True, the kernel will be uploaded to GFS if necessary
+ """
+ if not tests:
+ return ''
+
+ is_server, test_objects, label = (
+ rpc_utils.prepare_generate_control_file(tests, kernel, label))
+ if is_server:
+ return control_file.generate_server_control(test_objects)
+ return control_file.generate_client_control(test_objects, kernel,
+ label)
+
+
+def create_job(name, priority, control_file, control_type, is_synchronous=None,
+ hosts=None, meta_hosts=None):
+ """\
+ Create and enqueue a job.
+
+ priority: Low, Medium, High, Urgent
+ control_file: contents of control file
+ control_type: type of control file, Client or Server
+ is_synchronous: boolean indicating if a job is synchronous
+ hosts: list of hosts to run job on
+ meta_hosts: list where each entry is a label name, and for each entry
+ one host will be chosen from that label to run the job
+ on.
+ """
+ owner = rpc_utils.get_user().login
+ # input validation
+ if not hosts and not meta_hosts:
+ raise models.ValidationError({
+ 'arguments' : "You must pass at least one of 'hosts' or "
+ "'meta_hosts'"
+ })
+
+ # convert hostnames & meta hosts to host/label objects
+ host_objects = []
+ for host in hosts or []:
+ this_host = models.Host.smart_get(host)
+ host_objects.append(this_host)
+ for label in meta_hosts or []:
+ this_label = models.Label.smart_get(label)
+ host_objects.append(this_label)
+
+ # default is_synchronous to some appropriate value
+ ControlType = models.Job.ControlType
+ control_type = ControlType.get_value(control_type)
+ if is_synchronous is None:
+ is_synchronous = (control_type == ControlType.SERVER)
+ # convert the synch flag to an actual type
+ if is_synchronous:
+ synch_type = models.Job.SynchType.SYNCHRONOUS
+ else:
+ synch_type = models.Job.SynchType.ASYNCHRONOUS
+
+ job = models.Job.create(owner=owner, name=name, priority=priority,
+ control_file=control_file,
+ control_type=control_type,
+ synch_type=synch_type,
+ hosts=host_objects)
+ job.queue(host_objects)
+ return job.id
+
+
+def abort_job(id):
+ """\
+ Abort the job with the given id number.
+ """
+ job = models.Job.objects.get(id=id)
+ job.abort()
+
+
+def get_jobs(not_yet_run=False, running=False, finished=False, **filter_data):
+ """\
+ Extra filter args for get_jobs:
+ -not_yet_run: Include only jobs that have not yet started running.
+ -running: Include only jobs that have start running but for which not
+ all hosts have completed.
+ -finished: Include only jobs for which all hosts have completed (or
+ aborted).
+ At most one of these fields should be specified.
+ """
+ filter_data['extra_args'] = rpc_utils.extra_job_filters(not_yet_run,
+ running,
+ finished)
+ return rpc_utils.prepare_for_serialization(
+ models.Job.list_objects(filter_data))
+
+
+def get_num_jobs(not_yet_run=False, running=False, finished=False,
+ **filter_data):
+ """\
+ Get the number of jobs matching the given filters. query_start and
+ query_limit parameters are ignored. See get_jobs() for documentation of
+ extra filter parameters.
+ """
+ filter_data.pop('query_start', None)
+ filter_data.pop('query_limit', None)
+ filter_data['extra_args'] = rpc_utils.extra_job_filters(not_yet_run,
+ running,
+ finished)
+ return models.Job.query_objects(filter_data).count()
+
+
+def job_status(id):
+ """\
+ Get status of job with the given id number. Returns a dictionary
+ mapping hostnames to dictionaries with two keys each:
+ * 'status' : the job status for that host
+ * 'meta_count' : For meta host entires, gives a count of how many
+ entries there are for this label (the hostname is
+ then a label name). For real host entries,
+ meta_count is None.
+ """
+ job = models.Job.objects.get(id=id)
+ hosts_status = {}
+ for queue_entry in job.hostqueueentry_set.all():
+ is_meta = queue_entry.is_meta_host_entry()
+ if is_meta:
+ name = queue_entry.meta_host.name
+ hosts_status.setdefault(name, {'meta_count': 0})
+ hosts_status[name]['meta_count'] += 1
+ else:
+ name = queue_entry.host.hostname
+ hosts_status[name] = {'meta_count': None}
+ hosts_status[name]['status'] = queue_entry.status
+ return hosts_status
+
+
+def get_jobs_summary(**filter_data):
+ """\
+ Like get_jobs(), but adds a 'stauts_counts' field, which is a dictionary
+ mapping status strings to the number of hosts currently with that
+ status, i.e. {'Queued' : 4, 'Running' : 2}.
+ """
+ jobs = get_jobs(**filter_data)
+ ids = [job['id'] for job in jobs]
+ all_status_counts = models.Job.objects.get_status_counts(ids)
+ for job in jobs:
+ job['status_counts'] = all_status_counts[job['id']]
+ return rpc_utils.prepare_for_serialization(jobs)
+
+
+# other
+
+def get_static_data():
+ """\
+ Returns a dictionary containing a bunch of data that shouldn't change
+ often. This includes:
+ priorities: list of job priority choices
+ default_priority: default priority value for new jobs
+ users: sorted list of all usernames
+ labels: sorted list of all label names
+ tests: sorted list of all test names
+ user_login: username
+ """
+ result = {}
+ result['priorities'] = models.Job.Priority.choices()
+ default_priority = models.Job.get_field_dict()['priority'].default
+ default_string = models.Job.Priority.get_string(default_priority)
+ result['default_priority'] = default_string
+ result['users'] = [user.login for user in
+ models.User.objects.all().order_by('login')]
+ result['labels'] = [label.name for label in
+ models.Label.objects.all().order_by('name')]
+ result['tests'] = get_tests(sort_by='name')
+ result['user_login'] = rpc_utils.get_user().login
+ return result
diff --git a/frontend/afe/rpc_utils.py b/frontend/afe/rpc_utils.py
new file mode 100644
index 0000000..0bd1acc
--- /dev/null
+++ b/frontend/afe/rpc_utils.py
@@ -0,0 +1,96 @@
+"""\
+Utility functions for rpc_interface.py. We keep them in a separate file so that
+only RPC interface functions go into that file.
+"""
+
+__author__ = 'showard@google.com (Steve Howard)'
+
+import datetime, xmlrpclib, threading
+from frontend.afe import models
+
+def prepare_for_serialization(objects):
+ """\
+ Do necessary type conversions to values in data to allow for RPC
+ serialization.
+ -convert datetimes to strings
+ """
+ new_objects = []
+ for data in objects:
+ new_data = {}
+ for key, value in data.iteritems():
+ if isinstance(value, datetime.datetime):
+ new_data[key] = str(value)
+ else:
+ new_data[key] = value
+ new_objects.append(new_data)
+ return new_objects
+
+
+def extra_job_filters(not_yet_run=False, running=False, finished=False):
+ """\
+ Generate a SQL WHERE clause for job status filtering, and return it in
+ a dict of keyword args to pass to query.extra(). No more than one of
+ the parameters should be passed as True.
+ """
+ assert not ((not_yet_run and running) or
+ (not_yet_run and finished) or
+ (running and finished)), ('Cannot specify more than one '
+ 'filter to this function')
+ if not_yet_run:
+ where = ['id NOT IN (SELECT job_id FROM host_queue_entries '
+ 'WHERE active OR complete)']
+ elif running:
+ where = ['(id IN (SELECT job_id FROM host_queue_entries '
+ 'WHERE active OR complete)) AND '
+ '(id IN (SELECT job_id FROM host_queue_entries '
+ 'WHERE not complete OR active))']
+ elif finished:
+ where = ['id NOT IN (SELECT job_id FROM host_queue_entries '
+ 'WHERE not complete OR active)']
+ else:
+ return None
+ return {'where': where}
+
+
+local_vars = threading.local()
+
+def set_user(user):
+ """\
+ Sets the current request's logged-in user. user should be a
+ afe.models.User object.
+ """
+ local_vars.user = user
+
+
+def get_user():
+ 'Get the currently logged-in user as a afe.models.User object.'
+ return local_vars.user
+
+
+def prepare_generate_control_file(tests, kernel, label):
+ test_objects = [models.Test.smart_get(test) for test in tests]
+ # ensure tests are all the same type
+ test_type = test_objects[0].test_type
+ for test in test_objects[1:]:
+ if test.test_type != test_type:
+ raise models.ValidationError(
+ {'tests' : 'You cannot run both server- and '
+ 'client-side tests together (tests %s '
+ 'and %s differ' % (test_objects[0].name,
+ test.name)})
+
+ is_server = (test_type == models.Test.Types.SERVER)
+ if is_server:
+ if kernel or label:
+ error = 'This field is not supported for server jobs'
+ error_dict = {}
+ if kernel:
+ error_dict['kernel'] = error
+ if label:
+ error_dict['label'] = error
+ raise models.ValidationError(error_dict)
+
+ if label:
+ label = models.Label.smart_get(label)
+
+ return is_server, test_objects, label
diff --git a/frontend/afe/simplejson/__init__.py b/frontend/afe/simplejson/__init__.py
new file mode 100644
index 0000000..181225b
--- /dev/null
+++ b/frontend/afe/simplejson/__init__.py
@@ -0,0 +1,252 @@
+r"""
+A simple, fast, extensible JSON encoder and decoder
+
+JSON (JavaScript Object Notation) <http://json.org> is a subset of
+JavaScript syntax (ECMA-262 3rd edition) used as a lightweight data
+interchange format.
+
+simplejson exposes an API familiar to uses of the standard library
+marshal and pickle modules.
+
+BSD licensed.
+
+Encoding basic Python object hierarchies::
+
+ >>> import simplejson
+ >>> simplejson.dumps(['foo', {'bar': ('baz', None, 1.0, 2)}])
+ '["foo", {"bar": ["baz", null, 1.0, 2]}]'
+ >>> print simplejson.dumps("\"foo\bar")
+ "\"foo\bar"
+ >>> print simplejson.dumps(u'\u1234')
+ "\u1234"
+ >>> print simplejson.dumps('\\')
+ "\\"
+ >>> print simplejson.dumps({"c": 0, "b": 0, "a": 0}, sort_keys=True)
+ {"a": 0, "b": 0, "c": 0}
+ >>> from StringIO import StringIO
+ >>> io = StringIO()
+ >>> simplejson.dump(['streaming API'], io)
+ >>> io.getvalue()
+ '["streaming API"]'
+
+Compact encoding::
+
+ >>> import simplejson
+ >>> simplejson.dumps([1,2,3,{'4': 5, '6': 7}], separators=(',',':'))
+ '[1,2,3,{"4":5,"6":7}]'
+
+Pretty printing::
+
+ >>> import simplejson
+ >>> print simplejson.dumps({'4': 5, '6': 7}, sort_keys=True, indent=4)
+ {
+ "4": 5,
+ "6": 7
+ }
+
+Decoding JSON::
+
+ >>> import simplejson
+ >>> simplejson.loads('["foo", {"bar":["baz", null, 1.0, 2]}]')
+ [u'foo', {u'bar': [u'baz', None, 1.0, 2]}]
+ >>> simplejson.loads('"\\"foo\\bar"')
+ u'"foo\x08ar'
+ >>> from StringIO import StringIO
+ >>> io = StringIO('["streaming API"]')
+ >>> simplejson.load(io)
+ [u'streaming API']
+
+Specializing JSON object decoding::
+
+ >>> import simplejson
+ >>> def as_complex(dct):
+ ... if '__complex__' in dct:
+ ... return complex(dct['real'], dct['imag'])
+ ... return dct
+ ...
+ >>> simplejson.loads('{"__complex__": true, "real": 1, "imag": 2}',
+ ... object_hook=as_complex)
+ (1+2j)
+
+Extending JSONEncoder::
+
+ >>> import simplejson
+ >>> class ComplexEncoder(simplejson.JSONEncoder):
+ ... def default(self, obj):
+ ... if isinstance(obj, complex):
+ ... return [obj.real, obj.imag]
+ ... return simplejson.JSONEncoder.default(self, obj)
+ ...
+ >>> dumps(2 + 1j, cls=ComplexEncoder)
+ '[2.0, 1.0]'
+ >>> ComplexEncoder().encode(2 + 1j)
+ '[2.0, 1.0]'
+ >>> list(ComplexEncoder().iterencode(2 + 1j))
+ ['[', '2.0', ', ', '1.0', ']']
+
+
+Note that the JSON produced by this module's default settings
+is a subset of YAML, so it may be used as a serializer for that as well.
+"""
+__version__ = '1.5'
+__all__ = [
+ 'dump', 'dumps', 'load', 'loads',
+ 'JSONDecoder', 'JSONEncoder',
+]
+
+from decoder import JSONDecoder
+from encoder import JSONEncoder
+
+def dump(obj, fp, skipkeys=False, ensure_ascii=True, check_circular=True,
+ allow_nan=True, cls=None, indent=None, **kw):
+ """
+ Serialize ``obj`` as a JSON formatted stream to ``fp`` (a
+ ``.write()``-supporting file-like object).
+
+ If ``skipkeys`` is ``True`` then ``dict`` keys that are not basic types
+ (``str``, ``unicode``, ``int``, ``long``, ``float``, ``bool``, ``None``)
+ will be skipped instead of raising a ``TypeError``.
+
+ If ``ensure_ascii`` is ``False``, then the some chunks written to ``fp``
+ may be ``unicode`` instances, subject to normal Python ``str`` to
+ ``unicode`` coercion rules. Unless ``fp.write()`` explicitly
+ understands ``unicode`` (as in ``codecs.getwriter()``) this is likely
+ to cause an error.
+
+ If ``check_circular`` is ``False``, then the circular reference check
+ for container types will be skipped and a circular reference will
+ result in an ``OverflowError`` (or worse).
+
+ If ``allow_nan`` is ``False``, then it will be a ``ValueError`` to
+ serialize out of range ``float`` values (``nan``, ``inf``, ``-inf``)
+ in strict compliance of the JSON specification, instead of using the
+ JavaScript equivalents (``NaN``, ``Infinity``, ``-Infinity``).
+
+ If ``indent`` is a non-negative integer, then JSON array elements and object
+ members will be pretty-printed with that indent level. An indent level
+ of 0 will only insert newlines. ``None`` is the most compact representation.
+
+ To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the
+ ``.default()`` method to serialize additional types), specify it with
+ the ``cls`` kwarg.
+ """
+ if cls is None:
+ cls = JSONEncoder
+ iterable = cls(skipkeys=skipkeys, ensure_ascii=ensure_ascii,
+ check_circular=check_circular, allow_nan=allow_nan, indent=indent,
+ **kw).iterencode(obj)
+ # could accelerate with writelines in some versions of Python, at
+ # a debuggability cost
+ for chunk in iterable:
+ fp.write(chunk)
+
+def dumps(obj, skipkeys=False, ensure_ascii=True, check_circular=True,
+ allow_nan=True, cls=None, indent=None, separators=None, **kw):
+ """
+ Serialize ``obj`` to a JSON formatted ``str``.
+
+ If ``skipkeys`` is ``True`` then ``dict`` keys that are not basic types
+ (``str``, ``unicode``, ``int``, ``long``, ``float``, ``bool``, ``None``)
+ will be skipped instead of raising a ``TypeError``.
+
+ If ``ensure_ascii`` is ``False``, then the return value will be a
+ ``unicode`` instance subject to normal Python ``str`` to ``unicode``
+ coercion rules instead of being escaped to an ASCII ``str``.
+
+ If ``check_circular`` is ``False``, then the circular reference check
+ for container types will be skipped and a circular reference will
+ result in an ``OverflowError`` (or worse).
+
+ If ``allow_nan`` is ``False``, then it will be a ``ValueError`` to
+ serialize out of range ``float`` values (``nan``, ``inf``, ``-inf``) in
+ strict compliance of the JSON specification, instead of using the
+ JavaScript equivalents (``NaN``, ``Infinity``, ``-Infinity``).
+
+ If ``indent`` is a non-negative integer, then JSON array elements and
+ object members will be pretty-printed with that indent level. An indent
+ level of 0 will only insert newlines. ``None`` is the most compact
+ representation.
+
+ If ``separators`` is an ``(item_separator, dict_separator)`` tuple
+ then it will be used instead of the default ``(', ', ': ')`` separators.
+ ``(',', ':')`` is the most compact JSON representation.
+
+ To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the
+ ``.default()`` method to serialize additional types), specify it with
+ the ``cls`` kwarg.
+ """
+ if cls is None:
+ cls = JSONEncoder
+ return cls(
+ skipkeys=skipkeys, ensure_ascii=ensure_ascii,
+ check_circular=check_circular, allow_nan=allow_nan, indent=indent,
+ separators=separators,
+ **kw).encode(obj)
+
+def load(fp, encoding=None, cls=None, object_hook=None, **kw):
+ """
+ Deserialize ``fp`` (a ``.read()``-supporting file-like object containing
+ a JSON document) to a Python object.
+
+ If the contents of ``fp`` is encoded with an ASCII based encoding other
+ than utf-8 (e.g. latin-1), then an appropriate ``encoding`` name must
+ be specified. Encodings that are not ASCII based (such as UCS-2) are
+ not allowed, and should be wrapped with
+ ``codecs.getreader(fp)(encoding)``, or simply decoded to a ``unicode``
+ object and passed to ``loads()``
+
+ ``object_hook`` is an optional function that will be called with the
+ result of any object literal decode (a ``dict``). The return value of
+ ``object_hook`` will be used instead of the ``dict``. This feature
+ can be used to implement custom decoders (e.g. JSON-RPC class hinting).
+
+ To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``
+ kwarg.
+ """
+ if cls is None:
+ cls = JSONDecoder
+ if object_hook is not None:
+ kw['object_hook'] = object_hook
+ return cls(encoding=encoding, **kw).decode(fp.read())
+
+def loads(s, encoding=None, cls=None, object_hook=None, **kw):
+ """
+ Deserialize ``s`` (a ``str`` or ``unicode`` instance containing a JSON
+ document) to a Python object.
+
+ If ``s`` is a ``str`` instance and is encoded with an ASCII based encoding
+ other than utf-8 (e.g. latin-1) then an appropriate ``encoding`` name
+ must be specified. Encodings that are not ASCII based (such as UCS-2)
+ are not allowed and should be decoded to ``unicode`` first.
+
+ ``object_hook`` is an optional function that will be called with the
+ result of any object literal decode (a ``dict``). The return value of
+ ``object_hook`` will be used instead of the ``dict``. This feature
+ can be used to implement custom decoders (e.g. JSON-RPC class hinting).
+
+ To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``
+ kwarg.
+ """
+ if cls is None:
+ cls = JSONDecoder
+ if object_hook is not None:
+ kw['object_hook'] = object_hook
+ return cls(encoding=encoding, **kw).decode(s)
+
+def read(s):
+ """
+ json-py API compatibility hook. Use loads(s) instead.
+ """
+ import warnings
+ warnings.warn("simplejson.loads(s) should be used instead of read(s)",
+ DeprecationWarning)
+ return loads(s)
+
+def write(obj):
+ """
+ json-py API compatibility hook. Use dumps(s) instead.
+ """
+ import warnings
+ warnings.warn("simplejson.dumps(s) should be used instead of write(s)",
+ DeprecationWarning)
+ return dumps(obj)
diff --git a/frontend/afe/simplejson/decoder.py b/frontend/afe/simplejson/decoder.py
new file mode 100644
index 0000000..63f70cb
--- /dev/null
+++ b/frontend/afe/simplejson/decoder.py
@@ -0,0 +1,273 @@
+"""
+Implementation of JSONDecoder
+"""
+import re
+
+from scanner import Scanner, pattern
+
+FLAGS = re.VERBOSE | re.MULTILINE | re.DOTALL
+
+def _floatconstants():
+ import struct
+ import sys
+ _BYTES = '7FF80000000000007FF0000000000000'.decode('hex')
+ if sys.byteorder != 'big':
+ _BYTES = _BYTES[:8][::-1] + _BYTES[8:][::-1]
+ nan, inf = struct.unpack('dd', _BYTES)
+ return nan, inf, -inf
+
+NaN, PosInf, NegInf = _floatconstants()
+
+def linecol(doc, pos):
+ lineno = doc.count('\n', 0, pos) + 1
+ if lineno == 1:
+ colno = pos
+ else:
+ colno = pos - doc.rindex('\n', 0, pos)
+ return lineno, colno
+
+def errmsg(msg, doc, pos, end=None):
+ lineno, colno = linecol(doc, pos)
+ if end is None:
+ return '%s: line %d column %d (char %d)' % (msg, lineno, colno, pos)
+ endlineno, endcolno = linecol(doc, end)
+ return '%s: line %d column %d - line %d column %d (char %d - %d)' % (
+ msg, lineno, colno, endlineno, endcolno, pos, end)
+
+_CONSTANTS = {
+ '-Infinity': NegInf,
+ 'Infinity': PosInf,
+ 'NaN': NaN,
+ 'true': True,
+ 'false': False,
+ 'null': None,
+}
+
+def JSONConstant(match, context, c=_CONSTANTS):
+ return c[match.group(0)], None
+pattern('(-?Infinity|NaN|true|false|null)')(JSONConstant)
+
+def JSONNumber(match, context):
+ match = JSONNumber.regex.match(match.string, *match.span())
+ integer, frac, exp = match.groups()
+ if frac or exp:
+ res = float(integer + (frac or '') + (exp or ''))
+ else:
+ res = int(integer)
+ return res, None
+pattern(r'(-?(?:0|[1-9]\d*))(\.\d+)?([eE][-+]?\d+)?')(JSONNumber)
+
+STRINGCHUNK = re.compile(r'(.*?)(["\\])', FLAGS)
+BACKSLASH = {
+ '"': u'"', '\\': u'\\', '/': u'/',
+ 'b': u'\b', 'f': u'\f', 'n': u'\n', 'r': u'\r', 't': u'\t',
+}
+
+DEFAULT_ENCODING = "utf-8"
+
+def scanstring(s, end, encoding=None, _b=BACKSLASH, _m=STRINGCHUNK.match):
+ if encoding is None:
+ encoding = DEFAULT_ENCODING
+ chunks = []
+ _append = chunks.append
+ begin = end - 1
+ while 1:
+ chunk = _m(s, end)
+ if chunk is None:
+ raise ValueError(
+ errmsg("Unterminated string starting at", s, begin))
+ end = chunk.end()
+ content, terminator = chunk.groups()
+ if content:
+ if not isinstance(content, unicode):
+ content = unicode(content, encoding)
+ _append(content)
+ if terminator == '"':
+ break
+ try:
+ esc = s[end]
+ except IndexError:
+ raise ValueError(
+ errmsg("Unterminated string starting at", s, begin))
+ if esc != 'u':
+ try:
+ m = _b[esc]
+ except KeyError:
+ raise ValueError(
+ errmsg("Invalid \\escape: %r" % (esc,), s, end))
+ end += 1
+ else:
+ esc = s[end + 1:end + 5]
+ try:
+ m = unichr(int(esc, 16))
+ if len(esc) != 4 or not esc.isalnum():
+ raise ValueError
+ except ValueError:
+ raise ValueError(errmsg("Invalid \\uXXXX escape", s, end))
+ end += 5
+ _append(m)
+ return u''.join(chunks), end
+
+def JSONString(match, context):
+ encoding = getattr(context, 'encoding', None)
+ return scanstring(match.string, match.end(), encoding)
+pattern(r'"')(JSONString)
+
+WHITESPACE = re.compile(r'\s*', FLAGS)
+
+def JSONObject(match, context, _w=WHITESPACE.match):
+ pairs = {}
+ s = match.string
+ end = _w(s, match.end()).end()
+ nextchar = s[end:end + 1]
+ # trivial empty object
+ if nextchar == '}':
+ return pairs, end + 1
+ if nextchar != '"':
+ raise ValueError(errmsg("Expecting property name", s, end))
+ end += 1
+ encoding = getattr(context, 'encoding', None)
+ iterscan = JSONScanner.iterscan
+ while True:
+ key, end = scanstring(s, end, encoding)
+ end = _w(s, end).end()
+ if s[end:end + 1] != ':':
+ raise ValueError(errmsg("Expecting : delimiter", s, end))
+ end = _w(s, end + 1).end()
+ try:
+ value, end = iterscan(s, idx=end, context=context).next()
+ except StopIteration:
+ raise ValueError(errmsg("Expecting object", s, end))
+ pairs[key] = value
+ end = _w(s, end).end()
+ nextchar = s[end:end + 1]
+ end += 1
+ if nextchar == '}':
+ break
+ if nextchar != ',':
+ raise ValueError(errmsg("Expecting , delimiter", s, end - 1))
+ end = _w(s, end).end()
+ nextchar = s[end:end + 1]
+ end += 1
+ if nextchar != '"':
+ raise ValueError(errmsg("Expecting property name", s, end - 1))
+ object_hook = getattr(context, 'object_hook', None)
+ if object_hook is not None:
+ pairs = object_hook(pairs)
+ return pairs, end
+pattern(r'{')(JSONObject)
+
+def JSONArray(match, context, _w=WHITESPACE.match):
+ values = []
+ s = match.string
+ end = _w(s, match.end()).end()
+ # look-ahead for trivial empty array
+ nextchar = s[end:end + 1]
+ if nextchar == ']':
+ return values, end + 1
+ iterscan = JSONScanner.iterscan
+ while True:
+ try:
+ value, end = iterscan(s, idx=end, context=context).next()
+ except StopIteration:
+ raise ValueError(errmsg("Expecting object", s, end))
+ values.append(value)
+ end = _w(s, end).end()
+ nextchar = s[end:end + 1]
+ end += 1
+ if nextchar == ']':
+ break
+ if nextchar != ',':
+ raise ValueError(errmsg("Expecting , delimiter", s, end))
+ end = _w(s, end).end()
+ return values, end
+pattern(r'\[')(JSONArray)
+
+ANYTHING = [
+ JSONObject,
+ JSONArray,
+ JSONString,
+ JSONConstant,
+ JSONNumber,
+]
+
+JSONScanner = Scanner(ANYTHING)
+
+class JSONDecoder(object):
+ """
+ Simple JSON <http://json.org> decoder
+
+ Performs the following translations in decoding:
+
+ +---------------+-------------------+
+ | JSON | Python |
+ +===============+===================+
+ | object | dict |
+ +---------------+-------------------+
+ | array | list |
+ +---------------+-------------------+
+ | string | unicode |
+ +---------------+-------------------+
+ | number (int) | int, long |
+ +---------------+-------------------+
+ | number (real) | float |
+ +---------------+-------------------+
+ | true | True |
+ +---------------+-------------------+
+ | false | False |
+ +---------------+-------------------+
+ | null | None |
+ +---------------+-------------------+
+
+ It also understands ``NaN``, ``Infinity``, and ``-Infinity`` as
+ their corresponding ``float`` values, which is outside the JSON spec.
+ """
+
+ _scanner = Scanner(ANYTHING)
+ __all__ = ['__init__', 'decode', 'raw_decode']
+
+ def __init__(self, encoding=None, object_hook=None):
+ """
+ ``encoding`` determines the encoding used to interpret any ``str``
+ objects decoded by this instance (utf-8 by default). It has no
+ effect when decoding ``unicode`` objects.
+
+ Note that currently only encodings that are a superset of ASCII work,
+ strings of other encodings should be passed in as ``unicode``.
+
+ ``object_hook``, if specified, will be called with the result
+ of every JSON object decoded and its return value will be used in
+ place of the given ``dict``. This can be used to provide custom
+ deserializations (e.g. to support JSON-RPC class hinting).
+ """
+ self.encoding = encoding
+ self.object_hook = object_hook
+
+ def decode(self, s, _w=WHITESPACE.match):
+ """
+ Return the Python representation of ``s`` (a ``str`` or ``unicode``
+ instance containing a JSON document)
+ """
+ obj, end = self.raw_decode(s, idx=_w(s, 0).end())
+ end = _w(s, end).end()
+ if end != len(s):
+ raise ValueError(errmsg("Extra data", s, end, len(s)))
+ return obj
+
+ def raw_decode(self, s, **kw):
+ """
+ Decode a JSON document from ``s`` (a ``str`` or ``unicode`` beginning
+ with a JSON document) and return a 2-tuple of the Python
+ representation and the index in ``s`` where the document ended.
+
+ This can be used to decode a JSON document from a string that may
+ have extraneous data at the end.
+ """
+ kw.setdefault('context', self)
+ try:
+ obj, end = self._scanner.iterscan(s, **kw).next()
+ except StopIteration:
+ raise ValueError("No JSON object could be decoded")
+ return obj, end
+
+__all__ = ['JSONDecoder']
diff --git a/frontend/afe/simplejson/encoder.py b/frontend/afe/simplejson/encoder.py
new file mode 100644
index 0000000..c83c687
--- /dev/null
+++ b/frontend/afe/simplejson/encoder.py
@@ -0,0 +1,331 @@
+"""
+Implementation of JSONEncoder
+"""
+import re
+
+ESCAPE = re.compile(r'[\x00-\x19\\"\b\f\n\r\t]')
+ESCAPE_ASCII = re.compile(r'([\\"/]|[^\ -~])')
+ESCAPE_DCT = {
+ # escape all forward slashes to prevent </script> attack
+ '/': '\\/',
+ '\\': '\\\\',
+ '"': '\\"',
+ '\b': '\\b',
+ '\f': '\\f',
+ '\n': '\\n',
+ '\r': '\\r',
+ '\t': '\\t',
+}
+for i in range(0x20):
+ ESCAPE_DCT.setdefault(chr(i), '\\u%04x' % (i,))
+
+# assume this produces an infinity on all machines (probably not guaranteed)
+INFINITY = float('1e66666')
+
+def floatstr(o, allow_nan=True):
+ # Check for specials. Note that this type of test is processor- and/or
+ # platform-specific, so do tests which don't depend on the internals.
+
+ if o != o:
+ text = 'NaN'
+ elif o == INFINITY:
+ text = 'Infinity'
+ elif o == -INFINITY:
+ text = '-Infinity'
+ else:
+ return str(o)
+
+ if not allow_nan:
+ raise ValueError("Out of range float values are not JSON compliant: %r"
+ % (o,))
+
+ return text
+
+
+def encode_basestring(s):
+ """
+ Return a JSON representation of a Python string
+ """
+ def replace(match):
+ return ESCAPE_DCT[match.group(0)]
+ return '"' + ESCAPE.sub(replace, s) + '"'
+
+def encode_basestring_ascii(s):
+ def replace(match):
+ s = match.group(0)
+ try:
+ return ESCAPE_DCT[s]
+ except KeyError:
+ return '\\u%04x' % (ord(s),)
+ return '"' + str(ESCAPE_ASCII.sub(replace, s)) + '"'
+
+
+class JSONEncoder(object):
+ """
+ Extensible JSON <http://json.org> encoder for Python data structures.
+
+ Supports the following objects and types by default:
+
+ +-------------------+---------------+
+ | Python | JSON |
+ +===================+===============+
+ | dict | object |
+ +-------------------+---------------+
+ | list, tuple | array |
+ +-------------------+---------------+
+ | str, unicode | string |
+ +-------------------+---------------+
+ | int, long, float | number |
+ +-------------------+---------------+
+ | True | true |
+ +-------------------+---------------+
+ | False | false |
+ +-------------------+---------------+
+ | None | null |
+ +-------------------+---------------+
+
+ To extend this to recognize other objects, subclass and implement a
+ ``.default()`` method with another method that returns a serializable
+ object for ``o`` if possible, otherwise it should call the superclass
+ implementation (to raise ``TypeError``).
+ """
+ __all__ = ['__init__', 'default', 'encode', 'iterencode']
+ item_separator = ', '
+ key_separator = ': '
+ def __init__(self, skipkeys=False, ensure_ascii=True,
+ check_circular=True, allow_nan=True, sort_keys=False,
+ indent=None, separators=None):
+ """
+ Constructor for JSONEncoder, with sensible defaults.
+
+ If skipkeys is False, then it is a TypeError to attempt
+ encoding of keys that are not str, int, long, float or None. If
+ skipkeys is True, such items are simply skipped.
+
+ If ensure_ascii is True, the output is guaranteed to be str
+ objects with all incoming unicode characters escaped. If
+ ensure_ascii is false, the output will be unicode object.
+
+ If check_circular is True, then lists, dicts, and custom encoded
+ objects will be checked for circular references during encoding to
+ prevent an infinite recursion (which would cause an OverflowError).
+ Otherwise, no such check takes place.
+
+ If allow_nan is True, then NaN, Infinity, and -Infinity will be
+ encoded as such. This behavior is not JSON specification compliant,
+ but is consistent with most JavaScript based encoders and decoders.
+ Otherwise, it will be a ValueError to encode such floats.
+
+ If sort_keys is True, then the output of dictionaries will be
+ sorted by key; this is useful for regression tests to ensure
+ that JSON serializations can be compared on a day-to-day basis.
+
+ If indent is a non-negative integer, then JSON array
+ elements and object members will be pretty-printed with that
+ indent level. An indent level of 0 will only insert newlines.
+ None is the most compact representation.
+
+ If specified, separators should be a (item_separator, key_separator)
+ tuple. The default is (', ', ': '). To get the most compact JSON
+ representation you should specify (',', ':') to eliminate whitespace.
+ """
+
+ self.skipkeys = skipkeys
+ self.ensure_ascii = ensure_ascii
+ self.check_circular = check_circular
+ self.allow_nan = allow_nan
+ self.sort_keys = sort_keys
+ self.indent = indent
+ self.current_indent_level = 0
+ if separators is not None:
+ self.item_separator, self.key_separator = separators
+
+ def _newline_indent(self):
+ return '\n' + (' ' * (self.indent * self.current_indent_level))
+
+ def _iterencode_list(self, lst, markers=None):
+ if not lst:
+ yield '[]'
+ return
+ if markers is not None:
+ markerid = id(lst)
+ if markerid in markers:
+ raise ValueError("Circular reference detected")
+ markers[markerid] = lst
+ yield '['
+ if self.indent is not None:
+ self.current_indent_level += 1
+ newline_indent = self._newline_indent()
+ separator = self.item_separator + newline_indent
+ yield newline_indent
+ else:
+ newline_indent = None
+ separator = self.item_separator
+ first = True
+ for value in lst:
+ if first:
+ first = False
+ else:
+ yield separator
+ for chunk in self._iterencode(value, markers):
+ yield chunk
+ if newline_indent is not None:
+ self.current_indent_level -= 1
+ yield self._newline_indent()
+ yield ']'
+ if markers is not None:
+ del markers[markerid]
+
+ def _iterencode_dict(self, dct, markers=None):
+ if not dct:
+ yield '{}'
+ return
+ if markers is not None:
+ markerid = id(dct)
+ if markerid in markers:
+ raise ValueError("Circular reference detected")
+ markers[markerid] = dct
+ yield '{'
+ key_separator = self.key_separator
+ if self.indent is not None:
+ self.current_indent_level += 1
+ newline_indent = self._newline_indent()
+ item_separator = self.item_separator + newline_indent
+ yield newline_indent
+ else:
+ newline_indent = None
+ item_separator = self.item_separator
+ first = True
+ if self.ensure_ascii:
+ encoder = encode_basestring_ascii
+ else:
+ encoder = encode_basestring
+ allow_nan = self.allow_nan
+ if self.sort_keys:
+ keys = dct.keys()
+ keys.sort()
+ items = [(k, dct[k]) for k in keys]
+ else:
+ items = dct.iteritems()
+ for key, value in items:
+ if isinstance(key, basestring):
+ pass
+ # JavaScript is weakly typed for these, so it makes sense to
+ # also allow them. Many encoders seem to do something like this.
+ elif isinstance(key, float):
+ key = floatstr(key, allow_nan)
+ elif isinstance(key, (int, long)):
+ key = str(key)
+ elif key is True:
+ key = 'true'
+ elif key is False:
+ key = 'false'
+ elif key is None:
+ key = 'null'
+ elif self.skipkeys:
+ continue
+ else:
+ raise TypeError("key %r is not a string" % (key,))
+ if first:
+ first = False
+ else:
+ yield item_separator
+ yield encoder(key)
+ yield key_separator
+ for chunk in self._iterencode(value, markers):
+ yield chunk
+ if newline_indent is not None:
+ self.current_indent_level -= 1
+ yield self._newline_indent()
+ yield '}'
+ if markers is not None:
+ del markers[markerid]
+
+ def _iterencode(self, o, markers=None):
+ if isinstance(o, basestring):
+ if self.ensure_ascii:
+ encoder = encode_basestring_ascii
+ else:
+ encoder = encode_basestring
+ yield encoder(o)
+ elif o is None:
+ yield 'null'
+ elif o is True:
+ yield 'true'
+ elif o is False:
+ yield 'false'
+ elif isinstance(o, (int, long)):
+ yield str(o)
+ elif isinstance(o, float):
+ yield floatstr(o, self.allow_nan)
+ elif isinstance(o, (list, tuple)):
+ for chunk in self._iterencode_list(o, markers):
+ yield chunk
+ elif isinstance(o, dict):
+ for chunk in self._iterencode_dict(o, markers):
+ yield chunk
+ else:
+ if markers is not None:
+ markerid = id(o)
+ if markerid in markers:
+ raise ValueError("Circular reference detected")
+ markers[markerid] = o
+ for chunk in self._iterencode_default(o, markers):
+ yield chunk
+ if markers is not None:
+ del markers[markerid]
+
+ def _iterencode_default(self, o, markers=None):
+ newobj = self.default(o)
+ return self._iterencode(newobj, markers)
+
+ def default(self, o):
+ """
+ Implement this method in a subclass such that it returns
+ a serializable object for ``o``, or calls the base implementation
+ (to raise a ``TypeError``).
+
+ For example, to support arbitrary iterators, you could
+ implement default like this::
+
+ def default(self, o):
+ try:
+ iterable = iter(o)
+ except TypeError:
+ pass
+ else:
+ return list(iterable)
+ return JSONEncoder.default(self, o)
+ """
+ raise TypeError("%r is not JSON serializable" % (o,))
+
+ def encode(self, o):
+ """
+ Return a JSON string representation of a Python data structure.
+
+ >>> JSONEncoder().encode({"foo": ["bar", "baz"]})
+ '{"foo":["bar", "baz"]}'
+ """
+ # This doesn't pass the iterator directly to ''.join() because it
+ # sucks at reporting exceptions. It's going to do this internally
+ # anyway because it uses PySequence_Fast or similar.
+ chunks = list(self.iterencode(o))
+ return ''.join(chunks)
+
+ def iterencode(self, o):
+ """
+ Encode the given object and yield each string
+ representation as available.
+
+ For example::
+
+ for chunk in JSONEncoder().iterencode(bigobject):
+ mysocket.write(chunk)
+ """
+ if self.check_circular:
+ markers = {}
+ else:
+ markers = None
+ return self._iterencode(o, markers)
+
+__all__ = ['JSONEncoder']
diff --git a/frontend/afe/simplejson/scanner.py b/frontend/afe/simplejson/scanner.py
new file mode 100644
index 0000000..64f4999
--- /dev/null
+++ b/frontend/afe/simplejson/scanner.py
@@ -0,0 +1,63 @@
+"""
+Iterator based sre token scanner
+"""
+import sre_parse, sre_compile, sre_constants
+from sre_constants import BRANCH, SUBPATTERN
+from re import VERBOSE, MULTILINE, DOTALL
+import re
+
+__all__ = ['Scanner', 'pattern']
+
+FLAGS = (VERBOSE | MULTILINE | DOTALL)
+class Scanner(object):
+ def __init__(self, lexicon, flags=FLAGS):
+ self.actions = [None]
+ # combine phrases into a compound pattern
+ s = sre_parse.Pattern()
+ s.flags = flags
+ p = []
+ for idx, token in enumerate(lexicon):
+ phrase = token.pattern
+ try:
+ subpattern = sre_parse.SubPattern(s,
+ [(SUBPATTERN, (idx + 1, sre_parse.parse(phrase, flags)))])
+ except sre_constants.error:
+ raise
+ p.append(subpattern)
+ self.actions.append(token)
+
+ p = sre_parse.SubPattern(s, [(BRANCH, (None, p))])
+ self.scanner = sre_compile.compile(p)
+
+
+ def iterscan(self, string, idx=0, context=None):
+ """
+ Yield match, end_idx for each match
+ """
+ match = self.scanner.scanner(string, idx).match
+ actions = self.actions
+ lastend = idx
+ end = len(string)
+ while True:
+ m = match()
+ if m is None:
+ break
+ matchbegin, matchend = m.span()
+ if lastend == matchend:
+ break
+ action = actions[m.lastindex]
+ if action is not None:
+ rval, next_pos = action(m, context)
+ if next_pos is not None and next_pos != matchend:
+ # "fast forward" the scanner
+ matchend = next_pos
+ match = self.scanner.scanner(string, matchend).match
+ yield rval, matchend
+ lastend = matchend
+
+def pattern(pattern, flags=FLAGS):
+ def decorator(fn):
+ fn.pattern = pattern
+ fn.regex = re.compile(pattern, flags)
+ return fn
+ return decorator
diff --git a/frontend/afe/site_rpc_interface.py b/frontend/afe/site_rpc_interface.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/frontend/afe/site_rpc_interface.py
diff --git a/frontend/afe/templates/admin/base.html b/frontend/afe/templates/admin/base.html
new file mode 100644
index 0000000..74db94e
--- /dev/null
+++ b/frontend/afe/templates/admin/base.html
@@ -0,0 +1,55 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" lang="{{ LANGUAGE_CODE }}" xml:lang="{{ LANGUAGE_CODE }}" {% if LANGUAGE_BIDI %}dir="rtl"{% endif %}>
+<head>
+<title>{% block title %}{% endblock %}</title>
+<link rel="stylesheet" type="text/css" href="{% block stylesheet %}{% load adminmedia %}{% admin_media_prefix %}css/base.css{% endblock %}" />
+{% if LANGUAGE_BIDI %}<link rel="stylesheet" type="text/css" href="{% block stylesheet_rtl %}{% admin_media_prefix %}css/rtl.css{% endblock %}" />{% endif %}
+{% block extrastyle %}{% endblock %}
+{% block extrahead %}{% endblock %}
+{% block blockbots %}<meta name="robots" content="NONE,NOARCHIVE" />{% endblock %}
+</head>
+{% load i18n %}
+
+<body class="{% if is_popup %}popup {% endif %}{% block bodyclass %}{% endblock %}">
+
+<!-- Container -->
+<div id="container">
+
+ {% if not is_popup %}
+ <!-- Header -->
+ <div id="header">
+ <div id="branding">
+ <h1>Autotest Administration</h1>
+ </div>
+ {% if user.is_authenticated and user.is_staff %}
+ <div id="user-tools">{% trans 'Welcome,' %} <strong>{% if user.first_name %}{{ user.first_name|escape }}{% else %}{{ user.username }}{% endif %}</strong>. <a href="/afe">Go to Frontend</a></div>
+ {% endif %}
+ {% block nav-global %}{% endblock %}
+ </div>
+ <!-- END Header -->
+ {% block breadcrumbs %}<div class="breadcrumbs"><a href="/">{% trans 'Home' %}</a>{% if title %} › {{ title|escape }}{% endif %}</div>{% endblock %}
+ {% endif %}
+
+ {% if messages %}
+ <ul class="messagelist">{% for message in messages %}<li>{{ message|escape }}</li>{% endfor %}</ul>
+ {% endif %}
+
+ <!-- Content -->
+ <div id="content" class="{% block coltype %}colM{% endblock %}">
+ {% block pretitle %}{% endblock %}
+ {% block content_title %}{% if title %}<h1>{{ title|escape }}</h1>{% endif %}{% endblock %}
+ {% block content %}
+ {% block object-tools %}{% endblock %}
+ {{ content }}
+ {% endblock %}
+ {% block sidebar %}{% endblock %}
+ <br class="clear" />
+ </div>
+ <!-- END Content -->
+
+ {% block footer %}<div id="footer"></div>{% endblock %}
+</div>
+<!-- END Container -->
+
+</body>
+</html>
diff --git a/frontend/afe/test.py b/frontend/afe/test.py
new file mode 100644
index 0000000..c66d6b8
--- /dev/null
+++ b/frontend/afe/test.py
@@ -0,0 +1,88 @@
+import os, doctest, glob, sys
+import django.test.utils, django.test.simple
+from frontend import settings, afe
+
+# doctest takes a copy+paste log of a Python interactive session, runs a Python
+# interpreter, and replays all the inputs from the log, checking that the
+# outputs all match the log. This allows us to easily test behavior and
+# document functions at the same time, since the log shows exactly how functions
+# are called and what their outputs look like. See
+# http://www.python.org/doc/2.4.3/lib/module-doctest.html for more details.
+
+# In this file, we run doctest on all files found in the doctests/ directory.
+# We use django.test.utils to run the tests against a fresh test database every
+# time.
+
+app_name = 'afe'
+doctest_dir = 'doctests'
+doctest_paths = [os.path.join(doctest_dir, filename) for filename
+ in os.listdir(os.path.join(app_name, doctest_dir))]
+
+modules = []
+module_names = [os.path.basename(filename)[:-3]
+ for filename in glob.glob(app_name + '/*.py')
+ if '__init__' not in filename and 'test.py' not in filename]
+for module_name in module_names:
+ __import__('frontend.afe', globals(), locals(), [module_name])
+ modules.append(getattr(afe, module_name))
+
+print_after = 'Ran %d tests from %s'
+
+
+def run_tests(module_list, verbosity=1):
+ total_errors = run_pylint()
+ old_db = settings.DATABASE_NAME
+ django.test.utils.setup_test_environment()
+ django.test.utils.create_test_db(verbosity)
+ try:
+ for module in modules:
+ failures, test_count = doctest.testmod(module)
+ print print_after % (test_count, module.__name__)
+ total_errors += failures
+ for path in doctest_paths:
+ failures, test_count = doctest.testfile(path)
+ print print_after % (test_count, path)
+ total_errors += failures
+ finally:
+ django.test.utils.destroy_test_db(old_db)
+ django.test.utils.teardown_test_environment()
+ print
+ if total_errors == 0:
+ print 'OK'
+ else:
+ print 'FAIL: %d errors' % total_errors
+
+
+pylint_opts = {
+ 'models': ['--disable-msg=E0201'],
+ 'rpc_client_lib': ['--disable-msg=E0611'],
+}
+
+pylint_base_opts = ['--disable-msg-cat=warning,refactor,convention',
+ '--reports=no']
+
+pylint_exclude = ['management']
+
+
+def run_pylint():
+ try:
+ import pylint.lint
+ except ImportError:
+ print 'pylint not installed'
+ return
+ original_dir = os.getcwd()
+ os.chdir('..')
+ total_errors = 0
+ for module in modules:
+ module_name = module.__name__
+ module_basename = module_name.split('.')[-1]
+ if module_basename in pylint_exclude:
+ continue
+ print 'Checking ' + module_name
+ opts = (pylint_base_opts + pylint_opts.get(module_basename, []))
+ full_args = opts + [module_name]
+ result = pylint.lint.Run(full_args)
+ errors = result.linter.stats['error']
+ total_errors += errors
+ os.chdir(original_dir)
+ return total_errors
diff --git a/frontend/afe/urls.py b/frontend/afe/urls.py
new file mode 100644
index 0000000..9fe13e2
--- /dev/null
+++ b/frontend/afe/urls.py
@@ -0,0 +1,28 @@
+from django.conf.urls.defaults import *
+import os
+from frontend import settings
+
+pattern_list = [(r'^(?:|noauth/)rpc/', 'frontend.afe.rpc_handler.rpc_handler')]
+
+debug_pattern_list = [
+ (r'^model_doc/', 'frontend.afe.views.model_documentation'),
+ # for GWT hosted mode
+ (r'^(?P<forward_addr>afeclient.*)', 'frontend.afe.views.gwt_forward'),
+ # for GWT compiled files
+ (r'^client/(?P<path>.*)$', 'django.views.static.serve',
+ {'document_root': os.path.join(os.path.dirname(__file__), '..', 'client',
+ 'www')}),
+ # redirect / to compiled client
+ (r'^$', 'django.views.generic.simple.redirect_to',
+ {'url': 'client/afeclient.ClientMain/ClientMain.html'}),
+
+ # redirect /tko to local apache server
+ (r'^(?P<path>tko/.*)$',
+ 'frontend.afe.views.redirect_with_extra_data',
+ {'url': 'http://%(server_name)s/%(path)s?%(getdata)s'})
+]
+
+if settings.DEBUG:
+ pattern_list += debug_pattern_list
+
+urlpatterns = patterns('', *pattern_list)
diff --git a/frontend/afe/views.py b/frontend/afe/views.py
new file mode 100644
index 0000000..e0f74cd
--- /dev/null
+++ b/frontend/afe/views.py
@@ -0,0 +1,32 @@
+import urllib2
+
+from frontend.afe import models
+from django.http import HttpResponse, HttpResponsePermanentRedirect
+
+def model_documentation(request):
+ doc = '<h2>Models</h2>\n'
+ for model_name in ('Label', 'Host', 'Test', 'User', 'AclGroup', 'Job'):
+ model_class = getattr(models, model_name)
+ doc += '<h3>%s</h3>\n' % model_name
+ doc += '<pre>\n%s</pre>\n' % model_class.__doc__
+ return HttpResponse(doc)
+
+
+def redirect_with_extra_data(request, url, **kwargs):
+ kwargs['getdata'] = request.GET.urlencode()
+ kwargs['server_name'] = request.META['SERVER_NAME']
+ return HttpResponsePermanentRedirect(url % kwargs)
+
+
+GWT_SERVER = 'http://localhost:8888/'
+def gwt_forward(request, forward_addr):
+ if len(request.POST) == 0:
+ data = None
+ else:
+ data = request.raw_post_data
+ url_response = urllib2.urlopen(GWT_SERVER + forward_addr, data=data)
+ http_response = HttpResponse(url_response.read())
+ for header, value in url_response.info().items():
+ if header not in ('connection',):
+ http_response[header] = value
+ return http_response
diff --git a/frontend/apache_auth.py b/frontend/apache_auth.py
new file mode 100644
index 0000000..552a6bf
--- /dev/null
+++ b/frontend/apache_auth.py
@@ -0,0 +1,65 @@
+from django.contrib.auth.models import User, Group, check_password
+from django.contrib import auth
+from django import http
+
+from frontend.afe import models, management
+
+DEBUG_USER = 'debug_user'
+
+class SimpleAuthBackend:
+ """
+ Automatically allows any login. This backend is for use when Apache is
+ doing the real authentication. Also ensures logged-in user exists in
+ frontend.afe.models.User database.
+ """
+ def authenticate(self, username=None, password=None):
+ try:
+ user = User.objects.get(username=username)
+ except User.DoesNotExist:
+ # password is meaningless
+ user = User(username=username,
+ password='apache authentication')
+ user.is_staff = True
+ user.save() # need to save before adding groups
+ user.groups.add(Group.objects.get(
+ name=management.BASIC_ADMIN))
+
+ SimpleAuthBackend.check_afe_user(username)
+ return user
+
+
+ @staticmethod
+ def check_afe_user(username):
+ user, _ = models.User.objects.get_or_create(login=username)
+ user.save()
+
+ def get_user(self, user_id):
+ try:
+ return User.objects.get(pk=user_id)
+ except User.DoesNotExist:
+ return None
+
+
+class ApacheAuthMiddleware(object):
+ """
+ Middleware for use when Apache is doing authentication. Looks for
+ REQUEST_USER in requests and logs that user in. If no such header is
+ found, looks for HTTP_AUTHORIZATION header with username to login (this
+ allows CLI to authenticate).
+ """
+
+ def process_request(self, request):
+ # look for a username from Apache
+ user = request.META.get('REMOTE_USER')
+ if user is None:
+ # look for a user in headers. This is insecure but
+ # it's our temporarily solution for CLI auth.
+ user = request.META.get('HTTP_AUTHORIZATION')
+ if user is None:
+ # no user info - assume we're in development mode
+ user = DEBUG_USER
+ user_object = auth.authenticate(username=user,
+ password='')
+ auth.login(request, user_object)
+ request.afe_user = models.User.objects.get(login=user)
+ return None
diff --git a/frontend/client/.classpath b/frontend/client/.classpath
new file mode 100644
index 0000000..1e9bac0
--- /dev/null
+++ b/frontend/client/.classpath
@@ -0,0 +1,9 @@
+<?xml version="1.0" encoding="utf-8" ?>
+<classpath>
+ <classpathentry kind="src" path="src"/>
+ <classpathentry kind="src" path="test"/>
+ <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/>
+ <classpathentry kind="lib" path="/usr/local/lib/gwt/gwt-user.jar"/>
+ <classpathentry kind="var" path="JUNIT_HOME/junit.jar"/>
+ <classpathentry kind="output" path="bin"/>
+</classpath>
diff --git a/frontend/client/.project b/frontend/client/.project
new file mode 100644
index 0000000..0113a02
--- /dev/null
+++ b/frontend/client/.project
@@ -0,0 +1,15 @@
+<?xml version="1.0" encoding="utf-8" ?>
+<projectDescription>
+ <name>AfeClient</name>
+ <comment>AfeClient project</comment>
+ <projects/>
+ <buildSpec>
+ <buildCommand>
+ <name>org.eclipse.jdt.core.javabuilder</name>
+ <arguments/>
+ </buildCommand>
+ </buildSpec>
+ <natures>
+ <nature>org.eclipse.jdt.core.javanature</nature>
+ </natures>
+</projectDescription>
diff --git a/frontend/client/ClientMain-compile b/frontend/client/ClientMain-compile
new file mode 100755
index 0000000..ef8d64c
--- /dev/null
+++ b/frontend/client/ClientMain-compile
@@ -0,0 +1,7 @@
+#!/bin/sh
+APPDIR=`dirname $0`;
+GWTDIR=`$APPDIR/gwt_dir`;
+java \
+ -cp "$APPDIR/src:$APPDIR/bin:$GWTDIR/gwt-user.jar:$GWTDIR/gwt-dev-linux.jar" \
+ -Djava.awt.headless=true \
+ com.google.gwt.dev.GWTCompiler -out "$APPDIR/www" "$@" afeclient.ClientMain
diff --git a/frontend/client/ClientMain-shell b/frontend/client/ClientMain-shell
new file mode 100755
index 0000000..39dae72
--- /dev/null
+++ b/frontend/client/ClientMain-shell
@@ -0,0 +1,4 @@
+#!/bin/sh
+APPDIR=`dirname $0`;
+GWTDIR=`$APPDIR/gwt_dir`;
+java -cp "$APPDIR/src:$APPDIR/bin:$GWTDIR/gwt-user.jar:$GWTDIR/gwt-dev-linux.jar" com.google.gwt.dev.GWTShell -out "$APPDIR/www" "$@" http://localhost:8000/afe/server/afeclient.ClientMain/ClientMain.html;
diff --git a/frontend/client/ClientMain.launch b/frontend/client/ClientMain.launch
new file mode 100644
index 0000000..b5346be
--- /dev/null
+++ b/frontend/client/ClientMain.launch
@@ -0,0 +1,20 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<launchConfiguration type="org.eclipse.jdt.launching.localJavaApplication">
+<listAttribute key="org.eclipse.debug.core.MAPPED_RESOURCE_PATHS">
+<listEntry value="/AfeClient"/>
+</listAttribute>
+<listAttribute key="org.eclipse.debug.core.MAPPED_RESOURCE_TYPES">
+<listEntry value="4"/>
+</listAttribute>
+<booleanAttribute key="org.eclipse.debug.core.appendEnvironmentVariables" value="true"/>
+<listAttribute key="org.eclipse.jdt.launching.CLASSPATH">
+<listEntry value="<?xml version="1.0" encoding="UTF-8"?> <runtimeClasspathEntry containerPath="org.eclipse.jdt.launching.JRE_CONTAINER" javaProject="AfeClient" path="1" type="4"/> "/>
+<listEntry value="<?xml version="1.0" encoding="UTF-8"?> <runtimeClasspathEntry internalArchive="/AfeClient/src" path="3" type="2"/> "/>
+<listEntry value="<?xml version="1.0" encoding="UTF-8"?> <runtimeClasspathEntry id="org.eclipse.jdt.launching.classpathentry.defaultClasspath"> <memento project="AfeClient"/> </runtimeClasspathEntry> "/>
+<listEntry value="<?xml version="1.0" encoding="UTF-8"?> <runtimeClasspathEntry externalArchive="/usr/local/lib/gwt/gwt-dev-linux.jar" path="3" type="2"/> "/>
+</listAttribute>
+<booleanAttribute key="org.eclipse.jdt.launching.DEFAULT_CLASSPATH" value="false"/>
+<stringAttribute key="org.eclipse.jdt.launching.MAIN_TYPE" value="com.google.gwt.dev.GWTShell"/>
+<stringAttribute key="org.eclipse.jdt.launching.PROGRAM_ARGUMENTS" value="-out www http://localhost:8000/afe/server/afeclient.ClientMain/ClientMain.html"/>
+<stringAttribute key="org.eclipse.jdt.launching.PROJECT_ATTR" value="AfeClient"/>
+</launchConfiguration>
diff --git a/frontend/client/generate-javadoc b/frontend/client/generate-javadoc
new file mode 100755
index 0000000..d1cca6b
--- /dev/null
+++ b/frontend/client/generate-javadoc
@@ -0,0 +1,5 @@
+#!/bin/sh
+APPDIR=`dirname $0`;
+GWTDIR=`$APPDIR/gwt_dir`;
+javadoc -classpath "$APPDIR/src:$GWTDIR/gwt-user.jar" -d doc -public \
+ afeclient.client
diff --git a/frontend/client/gwt_dir b/frontend/client/gwt_dir
new file mode 100755
index 0000000..7336500
--- /dev/null
+++ b/frontend/client/gwt_dir
@@ -0,0 +1,2 @@
+#!/bin/sh
+echo /usr/local/lib/gwt
diff --git a/frontend/client/src/afeclient/ClientMain.gwt.xml b/frontend/client/src/afeclient/ClientMain.gwt.xml
new file mode 100644
index 0000000..f7ead6c
--- /dev/null
+++ b/frontend/client/src/afeclient/ClientMain.gwt.xml
@@ -0,0 +1,8 @@
+<module>
+ <inherits name='com.google.gwt.user.User'/>
+ <inherits name='com.google.gwt.json.JSON'/>
+
+ <entry-point class='afeclient.client.ClientMain'/>
+
+ <stylesheet src='afeclient.css'/>
+</module>
diff --git a/frontend/client/src/afeclient/client/ClassFactory.java b/frontend/client/src/afeclient/client/ClassFactory.java
new file mode 100644
index 0000000..f654692
--- /dev/null
+++ b/frontend/client/src/afeclient/client/ClassFactory.java
@@ -0,0 +1,9 @@
+package afeclient.client;
+
+import afeclient.client.CreateJobView.JobCreateListener;
+
+public class ClassFactory {
+ public CreateJobView getCreateJobView(JobCreateListener listener) {
+ return new CreateJobView(listener);
+ }
+}
diff --git a/frontend/client/src/afeclient/client/ClientMain.java b/frontend/client/src/afeclient/client/ClientMain.java
new file mode 100644
index 0000000..d62efe8
--- /dev/null
+++ b/frontend/client/src/afeclient/client/ClientMain.java
@@ -0,0 +1,129 @@
+package afeclient.client;
+
+import com.google.gwt.core.client.EntryPoint;
+import com.google.gwt.user.client.History;
+import com.google.gwt.user.client.HistoryListener;
+import com.google.gwt.user.client.ui.RootPanel;
+import com.google.gwt.user.client.ui.SourcesTabEvents;
+import com.google.gwt.user.client.ui.TabListener;
+import com.google.gwt.user.client.ui.TabPanel;
+
+import afeclient.client.CreateJobView.JobCreateListener;
+import afeclient.client.JobTable.JobTableListener;
+
+public class ClientMain implements EntryPoint, HistoryListener {
+ static final String RPC_URL = "/afe/server/rpc/";
+
+ protected JobListView jobList;
+ protected JobDetailView jobDetail;
+ protected CreateJobView createJob;
+ protected HostListView hostListView;
+
+ protected TabView[] tabViews;
+
+ protected TabPanel mainTabPanel;
+
+ /**
+ * Application entry point.
+ */
+ public void onModuleLoad() {
+ JsonRpcProxy.getProxy().setUrl(RPC_URL);
+
+ jobList = new JobListView(new JobTableListener() {
+ public void onJobClicked(int jobId) {
+ showJob(jobId);
+ }
+ });
+ jobDetail = new JobDetailView();
+ createJob = Utils.factory.getCreateJobView(new JobCreateListener() {
+ public void onJobCreated(int jobId) {
+ showJob(jobId);
+ }
+ });
+ hostListView = new HostListView();
+
+ tabViews = new TabView[4];
+ tabViews[0] = jobList;
+ tabViews[1] = jobDetail;
+ tabViews[2] = createJob;
+ tabViews[3] = hostListView;
+
+ mainTabPanel = new TabPanel();
+ for(int i = 0; i < tabViews.length; i++) {
+ mainTabPanel.add(tabViews[i], tabViews[i].getTitle());
+ }
+ mainTabPanel.addTabListener(new TabListener() {
+ public boolean onBeforeTabSelected(SourcesTabEvents sender,
+ int tabIndex) {
+ // do nothing if the user clicks the selected tab
+ if (mainTabPanel.getTabBar().getSelectedTab() == tabIndex)
+ return false;
+ tabViews[tabIndex].ensureInitialized();
+ tabViews[tabIndex].updateHistory();
+ // let onHistoryChanged() call TabView.display()
+ return true;
+ }
+ public void onTabSelected(SourcesTabEvents sender, int tabIndex) {}
+ });
+
+ // initialize static data, and don't show main UI until that's done
+ StaticDataRepository.getRepository().refresh(
+ new StaticDataRepository.FinishedCallback() {
+ public void onFinished() {
+ finishLoading();
+ }
+ });
+
+ NotifyManager.getInstance().initialize();
+ }
+
+ protected void finishLoading() {
+ final RootPanel tabsRoot = RootPanel.get("tabs");
+ tabsRoot.add(mainTabPanel);
+
+ History.addHistoryListener(this);
+ String initialToken = History.getToken();
+ if (!initialToken.equals("")) {
+ onHistoryChanged(initialToken);
+ }
+
+ // if the history token didn't provide a selected tab, default to the
+ // first tab
+ if (mainTabPanel.getTabBar().getSelectedTab() == -1)
+ mainTabPanel.selectTab(0);
+
+ RootPanel.get("tabs").setStyleName("");
+ }
+
+ protected void showJob(int jobId) {
+ jobDetail.ensureInitialized();
+ jobDetail.setJobID(jobId);
+ mainTabPanel.selectTab(1);
+ }
+
+ public void onHistoryChanged(String historyToken) {
+ if (!historyToken.startsWith(TabView.HISTORY_PREFIX))
+ return;
+
+ // remove prefix
+ historyToken = historyToken.substring(TabView.HISTORY_PREFIX.length());
+ for (int i = 0; i < tabViews.length; i++) {
+ String tabId = tabViews[i].getElementId();
+ if (historyToken.startsWith(tabId)) {
+ tabViews[i].ensureInitialized();
+
+ int prefixLength = tabId.length() + 1;
+ if (historyToken.length() > prefixLength) {
+ String restOfToken = historyToken.substring(prefixLength);
+ tabViews[i].handleHistoryToken(restOfToken);
+ }
+
+ tabViews[i].display();
+ if (mainTabPanel.getTabBar().getSelectedTab() != i)
+ mainTabPanel.selectTab(i);
+
+ return;
+ }
+ }
+ }
+}
diff --git a/frontend/client/src/afeclient/client/CreateJobView.java b/frontend/client/src/afeclient/client/CreateJobView.java
new file mode 100644
index 0000000..64766f3
--- /dev/null
+++ b/frontend/client/src/afeclient/client/CreateJobView.java
@@ -0,0 +1,509 @@
+package afeclient.client;
+
+import com.google.gwt.json.client.JSONArray;
+import com.google.gwt.json.client.JSONBoolean;
+import com.google.gwt.json.client.JSONNumber;
+import com.google.gwt.json.client.JSONObject;
+import com.google.gwt.json.client.JSONString;
+import com.google.gwt.json.client.JSONValue;
+import com.google.gwt.user.client.DOM;
+import com.google.gwt.user.client.Window;
+import com.google.gwt.user.client.ui.Button;
+import com.google.gwt.user.client.ui.ChangeListener;
+import com.google.gwt.user.client.ui.CheckBox;
+import com.google.gwt.user.client.ui.ClickListener;
+import com.google.gwt.user.client.ui.Composite;
+import com.google.gwt.user.client.ui.DisclosureEvent;
+import com.google.gwt.user.client.ui.DisclosureHandler;
+import com.google.gwt.user.client.ui.DisclosurePanel;
+import com.google.gwt.user.client.ui.FlexTable;
+import com.google.gwt.user.client.ui.FocusListener;
+import com.google.gwt.user.client.ui.HorizontalPanel;
+import com.google.gwt.user.client.ui.Hyperlink;
+import com.google.gwt.user.client.ui.KeyboardListener;
+import com.google.gwt.user.client.ui.ListBox;
+import com.google.gwt.user.client.ui.Panel;
+import com.google.gwt.user.client.ui.RadioButton;
+import com.google.gwt.user.client.ui.RootPanel;
+import com.google.gwt.user.client.ui.TextArea;
+import com.google.gwt.user.client.ui.TextBox;
+import com.google.gwt.user.client.ui.VerticalPanel;
+import com.google.gwt.user.client.ui.Widget;
+
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.List;
+
+public class CreateJobView extends TabView {
+ public static final int TEST_COLUMNS = 5;
+
+ // control file types
+ protected static final String CLIENT_TYPE = "Client";
+ protected static final String SERVER_TYPE = "Server";
+
+ protected static final String EDIT_CONTROL_STRING = "Edit control file";
+ protected static final String UNEDIT_CONTROL_STRING= "Revert changes";
+ protected static final String VIEW_CONTROL_STRING = "View control file";
+ protected static final String HIDE_CONTROL_STRING = "Hide control file";
+
+ public interface JobCreateListener {
+ public void onJobCreated(int jobId);
+ }
+
+ protected JsonRpcProxy rpcProxy = JsonRpcProxy.getProxy();
+ protected JobCreateListener listener;
+
+ protected class TestCheckBox extends CheckBox {
+ protected int id;
+ protected String testType;
+
+ public TestCheckBox(JSONObject test) {
+ super(test.get("name").isString().stringValue());
+ id = (int) test.get("id").isNumber().getValue();
+ testType = test.get("test_type").isString().stringValue();
+ String description = test.get("description").isString().stringValue();
+ if (description.equals(""))
+ description = "No description";
+ setTitle(description);
+ }
+
+ public int getId() {
+ return id;
+ }
+
+ public String getTestType() {
+ return testType;
+ }
+ }
+
+ protected class TestPanel extends Composite {
+ protected int numColumns;
+ protected FlexTable table = new FlexTable();
+ protected List testBoxes = new ArrayList(); // List<TestCheckBox>
+ String testType = null;
+
+ public TestPanel(String testType, int columns) {
+ this.testType = testType;
+ numColumns = columns;
+ initWidget(table);
+ }
+
+ public void addTest(TestCheckBox checkBox) {
+ if (!checkBox.getTestType().equals(testType))
+ throw new RuntimeException(
+ "Inconsistent test type for test " + checkBox.getText());
+ int row = testBoxes.size() / numColumns;
+ int col = testBoxes.size() % numColumns;
+ table.setWidget(row, col, checkBox);
+ testBoxes.add(checkBox);
+ }
+
+ public List getChecked() {
+ List result = new ArrayList();
+ for(Iterator i = testBoxes.iterator(); i.hasNext(); ) {
+ TestCheckBox test = (TestCheckBox) i.next();
+ if (test.isChecked())
+ result.add(test);
+ }
+ return result;
+ }
+
+ public void setEnabled(boolean enabled) {
+ for(Iterator i = testBoxes.iterator(); i.hasNext(); ) {
+ ((TestCheckBox) i.next()).setEnabled(enabled);
+ }
+ }
+
+ public void reset() {
+ for(Iterator i = testBoxes.iterator(); i.hasNext(); ) {
+ ((TestCheckBox) i.next()).setChecked(false);
+ }
+ }
+
+ public String getTestType() {
+ return testType;
+ }
+ }
+
+ protected class ControlTypeSelect extends Composite {
+ public static final String RADIO_GROUP = "controlTypeGroup";
+ protected String clientType, serverType;
+ protected RadioButton client, server;
+ protected Panel panel = new HorizontalPanel();
+
+ public ControlTypeSelect() {
+ client = new RadioButton(RADIO_GROUP, CLIENT_TYPE);
+ server = new RadioButton(RADIO_GROUP, SERVER_TYPE);
+ panel.add(client);
+ panel.add(server);
+ client.setChecked(true); // client is default
+ initWidget(panel);
+
+ client.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ onChanged();
+ }
+ });
+ server.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ onChanged();
+ }
+ });
+ }
+
+ public String getControlType() {
+ if (client.isChecked())
+ return client.getText();
+ return server.getText();
+ }
+
+ public void setControlType(String type) {
+ if (client.getText().equals(type))
+ client.setChecked(true);
+ else if (server.getText().equals(type))
+ server.setChecked(true);
+ else
+ throw new IllegalArgumentException("Invalid control type");
+ onChanged();
+ }
+
+ public void setEnabled(boolean enabled) {
+ client.setEnabled(enabled);
+ server.setEnabled(enabled);
+ }
+
+ protected void onChanged() {
+ setRunSynchronous();
+ }
+ }
+
+ protected StaticDataRepository staticData = StaticDataRepository.getRepository();
+
+ protected TextBox jobName = new TextBox();
+ protected ListBox priorityList = new ListBox();
+ protected TextBox kernel = new TextBox();
+ protected TestPanel clientTestsPanel = new TestPanel(CLIENT_TYPE, TEST_COLUMNS),
+ serverTestsPanel = new TestPanel(SERVER_TYPE, TEST_COLUMNS);
+ protected TextArea controlFile = new TextArea();
+ protected DisclosurePanel controlFilePanel = new DisclosurePanel();
+ protected ControlTypeSelect controlTypeSelect;
+ protected CheckBox runSynchronous = new CheckBox("Synchronous");
+ protected Button editControlButton = new Button(EDIT_CONTROL_STRING);
+ protected HostSelector hostSelector;
+ protected Button submitJobButton = new Button("Submit Job");
+
+ protected boolean controlEdited = false;
+ protected boolean kernelPushed = false;
+
+ public CreateJobView(JobCreateListener listener) {
+ this.listener = listener;
+ }
+
+ public String getElementId() {
+ return "create_job";
+ }
+
+ protected void populatePriorities(JSONArray priorities) {
+ for(int i = 0; i < priorities.size(); i++) {
+ JSONArray priorityData = priorities.get(i).isArray();
+ String priority = priorityData.get(1).isString().stringValue();
+ priorityList.addItem(priority);
+ }
+
+ resetPriorityToDefault();
+ }
+
+ protected void resetPriorityToDefault() {
+ JSONValue defaultValue = staticData.getData("default_priority");
+ String defaultPriority = defaultValue.isString().stringValue();
+ for(int i = 0; i < priorityList.getItemCount(); i++) {
+ if (priorityList.getItemText(i).equals(defaultPriority))
+ priorityList.setSelectedIndex(i);
+ }
+ }
+
+ protected void populateTests() {
+ StaticDataRepository staticData = StaticDataRepository.getRepository();
+ JSONArray tests = staticData.getData("tests").isArray();
+
+ for(int i = 0; i < tests.size(); i++) {
+ JSONObject test = tests.get(i).isObject();
+ TestCheckBox checkbox = new TestCheckBox(test);
+ checkbox.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ generateControlFile(false);
+ setInputsEnabled();
+ updateControlType();
+ }
+ });
+ String type = test.get("test_type").isString().stringValue();
+ if (type.equals("Client"))
+ clientTestsPanel.addTest(checkbox);
+ else if (type.equals("Server"))
+ serverTestsPanel.addTest(checkbox);
+ else
+ throw new RuntimeException("Invalid control type: " + type);
+ }
+ }
+
+ protected JSONObject getControlFileParams(boolean pushKernel) {
+ JSONObject params = new JSONObject();
+ JSONArray tests = new JSONArray();
+ List checkedTests = serverTestsPanel.getChecked();
+ String kernelString = "";
+ if (checkedTests.isEmpty()) {
+ checkedTests = clientTestsPanel.getChecked();
+ kernelString = kernel.getText();
+ }
+ if (!kernelString.equals("")) {
+ params.put("kernel", new JSONString(kernelString));
+ params.put("do_push_kernel", JSONBoolean.getInstance(pushKernel));
+ }
+
+ for (int i = 0; i < checkedTests.size(); i++) {
+ TestCheckBox test = (TestCheckBox) checkedTests.get(i);
+ tests.set(i, new JSONNumber(test.getId()));
+ }
+ params.put("tests", tests);
+ return params;
+ }
+
+ protected void generateControlFile(final boolean pushKernel,
+ final SimpleCallback finishedCallback,
+ final SimpleCallback errorCallback) {
+ JSONObject params = getControlFileParams(pushKernel);
+ rpcProxy.rpcCall("generate_control_file", params, new JsonRpcCallback() {
+ public void onSuccess(JSONValue result) {
+ controlFile.setText(result.isString().stringValue());
+ kernelPushed = pushKernel;
+ if (finishedCallback != null)
+ finishedCallback.doCallback();
+ }
+
+ public void onError(JSONObject errorObject) {
+ super.onError(errorObject);
+ if (errorCallback != null)
+ errorCallback.doCallback();
+ }
+ });
+ }
+
+ protected void generateControlFile(boolean pushKernel) {
+ generateControlFile(pushKernel, null, null);
+ }
+
+ protected void setInputsEnabled() {
+ if (!clientTestsPanel.getChecked().isEmpty()) {
+ clientTestsPanel.setEnabled(true);
+ serverTestsPanel.setEnabled(false);
+ kernel.setEnabled(true);
+ }
+ else if (!serverTestsPanel.getChecked().isEmpty()) {
+ clientTestsPanel.setEnabled(false);
+ serverTestsPanel.setEnabled(true);
+ kernel.setEnabled(false);
+ }
+ else {
+ clientTestsPanel.setEnabled(true);
+ serverTestsPanel.setEnabled(true);
+ kernel.setEnabled(true);
+ }
+ }
+
+ protected void disableInputs() {
+ clientTestsPanel.setEnabled(false);
+ serverTestsPanel.setEnabled(false);
+ kernel.setEnabled(false);
+ }
+
+ public void initialize() {
+ StaticDataRepository staticData = StaticDataRepository.getRepository();
+ populatePriorities(staticData.getData("priorities").isArray());
+
+ kernel.addFocusListener(new FocusListener() {
+ public void onFocus(Widget sender) {}
+ public void onLostFocus(Widget sender) {
+ generateControlFile(false);
+ }
+ });
+ kernel.addKeyboardListener(new KeyboardListener() {
+ public void onKeyDown(Widget sender, char keyCode, int modifiers) {}
+ public void onKeyUp(Widget sender, char keyCode, int modifiers) {}
+ public void onKeyPress(Widget sender, char keyCode, int modifiers) {
+ if (keyCode == KEY_ENTER)
+ generateControlFile(false);
+ }
+ });
+
+ populateTests();
+
+ controlFile.setSize("50em", "30em");
+ controlTypeSelect = new ControlTypeSelect();
+ Panel controlOptionsPanel = new HorizontalPanel();
+ controlOptionsPanel.add(controlTypeSelect);
+ controlOptionsPanel.add(runSynchronous);
+ runSynchronous.addStyleName("extra-space-left");
+ Panel controlEditPanel = new VerticalPanel();
+ controlEditPanel.add(controlOptionsPanel);
+ controlEditPanel.add(controlFile);
+
+ Panel controlHeaderPanel = new HorizontalPanel();
+ final Hyperlink viewLink = new SimpleHyperlink(VIEW_CONTROL_STRING);
+ controlHeaderPanel.add(viewLink);
+ controlHeaderPanel.add(editControlButton);
+
+ controlFilePanel.setHeader(controlHeaderPanel);
+ controlFilePanel.add(controlEditPanel);
+
+ editControlButton.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ DOM.eventCancelBubble(DOM.eventGetCurrentEvent(), true);
+
+ if (editControlButton.getText().equals(EDIT_CONTROL_STRING)) {
+ generateControlFile(true);
+ controlFile.setReadOnly(false);
+ disableInputs();
+ editControlButton.setText(UNEDIT_CONTROL_STRING);
+ controlFilePanel.setOpen(true);
+ controlTypeSelect.setEnabled(true);
+ }
+ else {
+ if (controlEdited &&
+ !Window.confirm("Are you sure you want to revert your" +
+ " changes?"))
+ return;
+ generateControlFile(false);
+ controlFile.setReadOnly(true);
+ setInputsEnabled();
+ editControlButton.setText(EDIT_CONTROL_STRING);
+ controlTypeSelect.setEnabled(false);
+ updateControlType();
+ controlEdited = false;
+ }
+ setRunSynchronous();
+ }
+ });
+
+ controlFile.addChangeListener(new ChangeListener() {
+ public void onChange(Widget sender) {
+ controlEdited = true;
+ }
+ });
+
+ controlFilePanel.addEventHandler(new DisclosureHandler() {
+ public void onClose(DisclosureEvent event) {
+ viewLink.setText(VIEW_CONTROL_STRING);
+ }
+
+ public void onOpen(DisclosureEvent event) {
+ viewLink.setText(HIDE_CONTROL_STRING);
+ }
+ });
+
+ hostSelector = new HostSelector();
+
+ submitJobButton.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ submitJob();
+ }
+ });
+
+ reset();
+
+ RootPanel.get("create_job_name").add(jobName);
+ RootPanel.get("create_kernel").add(kernel);
+ RootPanel.get("create_priority").add(priorityList);
+ RootPanel.get("create_client_tests").add(clientTestsPanel);
+ RootPanel.get("create_server_tests").add(serverTestsPanel);
+ RootPanel.get("create_edit_control").add(controlFilePanel);
+ RootPanel.get("create_submit").add(submitJobButton);
+ }
+
+ public void reset() {
+ jobName.setText("");
+ resetPriorityToDefault();
+ kernel.setText("");
+ clientTestsPanel.reset();
+ serverTestsPanel.reset();
+ setInputsEnabled();
+ updateControlType();
+ controlTypeSelect.setEnabled(false);
+ runSynchronous.setEnabled(false);
+ runSynchronous.setChecked(false);
+ controlFile.setText("");
+ controlFile.setReadOnly(true);
+ controlEdited = false;
+ editControlButton.setText(EDIT_CONTROL_STRING);
+ hostSelector.reset();
+ }
+
+ protected void updateControlType() {
+ String type = clientTestsPanel.getTestType(); // the default
+ if (!serverTestsPanel.getChecked().isEmpty())
+ type = serverTestsPanel.getTestType();
+ controlTypeSelect.setControlType(type);
+ }
+
+ protected void setRunSynchronous() {
+ boolean isServer = controlTypeSelect.getControlType().equals("Server");
+ runSynchronous.setChecked(isServer);
+ boolean enabled = isServer &&
+ editControlButton.getText().equals(UNEDIT_CONTROL_STRING);
+ runSynchronous.setEnabled(enabled);
+ }
+
+ protected void submitJob() {
+ // disallow accidentally clicking submit twice
+ submitJobButton.setEnabled(false);
+
+ final SimpleCallback doSubmit = new SimpleCallback() {
+ public void doCallback() {
+ JSONObject args = new JSONObject();
+ args.put("name", new JSONString(jobName.getText()));
+ String priority = priorityList.getItemText(
+ priorityList.getSelectedIndex());
+ args.put("priority", new JSONString(priority));
+ args.put("control_file", new JSONString(controlFile.getText()));
+ args.put("control_type",
+ new JSONString(controlTypeSelect.getControlType()));
+ args.put("is_synchronous",
+ JSONBoolean.getInstance(runSynchronous.isChecked()));
+
+ HostSelector.HostSelection hosts = hostSelector.getSelectedHosts();
+ args.put("hosts", Utils.stringsToJSON(hosts.hosts));
+ args.put("meta_hosts", Utils.stringsToJSON(hosts.metaHosts));
+
+ boolean success =
+ rpcProxy.rpcCall("create_job", args, new JsonRpcCallback() {
+ public void onSuccess(JSONValue result) {
+ int id = (int) result.isNumber().getValue();
+ NotifyManager.getInstance().showMessage(
+ "Job " + Integer.toString(id) + " created");
+ reset();
+ if (listener != null)
+ listener.onJobCreated(id);
+ submitJobButton.setEnabled(true);
+ }
+
+ public void onError(JSONObject errorObject) {
+ super.onError(errorObject);
+ submitJobButton.setEnabled(true);
+ }
+ });
+
+ if (!success)
+ submitJobButton.setEnabled(true);
+ }
+ };
+
+ // ensure kernel is pushed before submitting
+ if (!kernelPushed)
+ generateControlFile(true, doSubmit, new SimpleCallback() {
+ public void doCallback() {
+ submitJobButton.setEnabled(true);
+ }
+ });
+ else
+ doSubmit.doCallback();
+ }
+}
diff --git a/frontend/client/src/afeclient/client/DataTable.java b/frontend/client/src/afeclient/client/DataTable.java
new file mode 100644
index 0000000..582d04f
--- /dev/null
+++ b/frontend/client/src/afeclient/client/DataTable.java
@@ -0,0 +1,188 @@
+package afeclient.client;
+
+
+
+import com.google.gwt.json.client.JSONArray;
+import com.google.gwt.json.client.JSONObject;
+import com.google.gwt.json.client.JSONValue;
+import com.google.gwt.user.client.ui.Composite;
+import com.google.gwt.user.client.ui.FlexTable;
+import com.google.gwt.user.client.ui.Widget;
+
+/**
+ * A table to display data from JSONObjects. Each row displays data from one
+ * JSONObject. A header row with column titles is automatically generated, and
+ * support is included for adding other arbitrary header rows.
+ * <br><br>
+ * Styles:
+ * <ul>
+ * <li>.data-table - the entire table
+ * <li>.data-row-header - the column title row
+ * <li>.data-row-one/.data-row-two - data row styles. These two are alternated.
+ * </ul>
+ */
+public class DataTable extends Composite {
+ public static final String HEADER_STYLE = "data-row-header";
+ public static final String CLICKABLE_STYLE = "data-row-clickable";
+
+ protected FlexTable table;
+
+ protected String[][] columns;
+ protected int headerRow = 0;
+ protected boolean clickable = false;
+
+ /**
+ * @param columns An array specifying the name of each column and the field
+ * to which it corresponds. The array should have the form
+ * {{'field_name1', 'Column Title 1'},
+ * {'field_name2', 'Column Title 2'}, ...}.
+ */
+ public DataTable(String[][] columns) {
+ this.columns = columns;
+ table = new FlexTable();
+ initWidget(table);
+
+ table.setCellSpacing(0);
+ table.setCellPadding(0);
+ table.setStyleName("data-table");
+
+ for (int i = 0; i < columns.length; i++) {
+ table.setText(0, i, columns[i][1]);
+ }
+
+ table.getRowFormatter().setStylePrimaryName(0, HEADER_STYLE);
+ }
+
+ protected void setRowStyle(int row) {
+ table.getRowFormatter().setStyleName(row, "data-row");
+ if ((row & 1) == 0) {
+ table.getRowFormatter().addStyleName(row, "data-row-alternate");
+ }
+ if (clickable) {
+ table.getRowFormatter().addStyleName(row, CLICKABLE_STYLE);
+ }
+ }
+
+ public void setClickable(boolean clickable) {
+ this.clickable = clickable;
+ for(int i = headerRow + 1; i < table.getRowCount(); i++)
+ setRowStyle(i);
+ }
+
+ /**
+ * Clear all data rows from the table. Leaves the header rows intact.
+ */
+ public void clear() {
+ while (getRowCount() > 0) {
+ removeRow(0);
+ }
+ }
+
+ /**
+ * This gets called for every JSONObject that gets added to the table using
+ * addRow(). This allows subclasses to customize objects before they are
+ * added to the table, for example to reformat fields or generate new
+ * fields from the existing data.
+ * @param row The row object about to be added to the table.
+ */
+ protected void preprocessRow(JSONObject row) {}
+
+ protected String getTextForValue(JSONValue value) {
+ if (value.isNumber() != null)
+ return Integer.toString((int) value.isNumber().getValue());
+ else if (value.isString() != null)
+ return value.isString().stringValue();
+ else if (value.isNull() != null)
+ return "";
+ else
+ throw new IllegalArgumentException(value.toString());
+ }
+
+ protected String[] getRowText(JSONObject row) {
+ String[] rowText = new String[columns.length];
+ for (int i = 0; i < columns.length; i++) {
+ String columnKey = columns[i][0];
+ JSONValue columnValue = row.get(columnKey);
+ rowText[i] = getTextForValue(columnValue);
+ }
+ return rowText;
+ }
+
+ /**
+ * Add a row from an array of Strings, one String for each column.
+ * @param rowData Data for each column, in left-to-right column order.
+ */
+ public void addRowFromData(String[] rowData) {
+ int row = table.getRowCount();
+ for(int i = 0; i < columns.length; i++)
+ table.setHTML(row, i, rowData[i]);
+ setRowStyle(row);
+ }
+
+ /**
+ * Add a row from a JSONObject. Columns will be populated by pulling fields
+ * from the objects, as dictated by the columns information passed into the
+ * DataTable constructor.
+ */
+ public void addRow(JSONObject row) {
+ preprocessRow(row);
+ addRowFromData(getRowText(row));
+ }
+
+ /**
+ * Add all objects in a JSONArray.
+ * @param rows An array of JSONObjects
+ * @throws IllegalArgumentException if any other type of JSONValue is in the
+ * array.
+ */
+ public void addRows(JSONArray rows) {
+ for (int i = 0; i < rows.size(); i++) {
+ JSONObject row = rows.get(i).isObject();
+ if (row == null)
+ throw new IllegalArgumentException("rows must be JSONObjects");
+ addRow(row);
+ }
+ }
+
+ /**
+ * Remove a data row from the table.
+ * @param row The index of the row, where the first data row is indexed 0.
+ * Header rows are ignored.
+ */
+ public void removeRow(int row) {
+ int realRow = row + getHeaderRowCount();
+ table.removeRow(realRow);
+ for(int i = realRow; i < table.getRowCount(); i++)
+ setRowStyle(i);
+ }
+
+ /**
+ * Returns the number of data rows in the table. The actual number of
+ * visible table rows is more than this, due to the header rows.
+ */
+ public int getRowCount() {
+ return table.getRowCount() - getHeaderRowCount();
+ }
+
+ /**
+ * Adds a header row to the table. This is an extra row that is added above
+ * the row of column titles and below any other header rows that have been
+ * added. The row consists of a single cell.
+ * @param widget A widget to add to the cell.
+ * @return The row index of the new header row.
+ */
+ public int addHeaderRow(Widget widget) {
+ int row = table.insertRow(headerRow);
+ headerRow++;
+ table.getFlexCellFormatter().setColSpan(row, 0, columns.length);
+ table.setWidget(row, 0, widget);
+ return row;
+ }
+
+ /**
+ * Returns the number of header rows, including the column title row.
+ */
+ public int getHeaderRowCount() {
+ return headerRow + 1;
+ }
+}
diff --git a/frontend/client/src/afeclient/client/DynamicTable.java b/frontend/client/src/afeclient/client/DynamicTable.java
new file mode 100644
index 0000000..6bceca5
--- /dev/null
+++ b/frontend/client/src/afeclient/client/DynamicTable.java
@@ -0,0 +1,534 @@
+package afeclient.client;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.Vector;
+
+import com.google.gwt.user.client.ui.ChangeListener;
+import com.google.gwt.user.client.ui.Composite;
+import com.google.gwt.user.client.ui.HTMLPanel;
+import com.google.gwt.user.client.ui.HorizontalPanel;
+import com.google.gwt.user.client.ui.Image;
+import com.google.gwt.user.client.ui.KeyboardListener;
+import com.google.gwt.user.client.ui.Label;
+import com.google.gwt.user.client.ui.ListBox;
+import com.google.gwt.user.client.ui.SourcesTableEvents;
+import com.google.gwt.user.client.ui.TableListener;
+import com.google.gwt.user.client.ui.TextBox;
+import com.google.gwt.user.client.ui.Widget;
+
+/**
+ * Extended DataTable supporting client-side sorting, searching, filtering, and
+ * pagination.
+ */
+public class DynamicTable extends DataTable {
+ public static final int NO_COLUMN = -1;
+ public static final int ASCENDING = 1, DESCENDING = -1;
+ public static final String ALL_VALUES = "All values";
+ public static final String SORT_UP_IMAGE = "arrow_up.png",
+ SORT_DOWN_IMAGE = "arrow_down.png";
+
+ interface DynamicTableListener {
+ public void onRowClicked(int dataRow, int column);
+ }
+
+ interface Filter {
+ public boolean isActive();
+ public boolean acceptRow(String[] row);
+ public void update();
+ }
+
+ class ColumnFilter implements Filter {
+ public int column;
+ public ListBox select = new ListBox();
+ public boolean isManualChoices = false;
+
+ public ColumnFilter(int column) {
+ this.column = column;
+ select.setStylePrimaryName("filter-box");
+ }
+
+ public String getSelectedValue() {
+ return select.getItemText(select.getSelectedIndex());
+ }
+
+ public boolean isActive() {
+ return !getSelectedValue().equals(ALL_VALUES);
+ }
+
+ public boolean acceptRow(String[] row) {
+ return row[column].equals(getSelectedValue());
+ }
+
+ public void setChoices(String[] choices) {
+ String selectedValue = null;
+ if (select.getSelectedIndex() != -1)
+ selectedValue = getSelectedValue();
+
+ select.clear();
+ select.addItem(ALL_VALUES);
+ for (int i = 0; i < choices.length; i++)
+ select.addItem(choices[i]);
+
+ if (selectedValue != null) {
+ setChoice(selectedValue);
+ }
+ }
+
+ public void update() {
+ if (!isManualChoices)
+ setChoices(gatherChoices(column));
+ }
+
+ public void setChoice(String choice) {
+ for(int i = 0; i < select.getItemCount(); i++) {
+ if(select.getItemText(i).equals(choice)) {
+ select.setSelectedIndex(i);
+ return;
+ }
+ }
+
+ select.addItem(choice);
+ select.setSelectedIndex(select.getItemCount() - 1);
+ }
+ }
+
+ class SearchFilter implements Filter {
+ public int[] searchColumns;
+ public TextBox searchBox = new TextBox();
+
+ public SearchFilter(int[] searchColumns) {
+ this.searchColumns = searchColumns;
+ searchBox.setStylePrimaryName("filter-box");
+ }
+
+ public boolean acceptRow(String[] row) {
+ String query = searchBox.getText();
+ for (int i = 0; i < searchColumns.length; i++) {
+ if (row[searchColumns[i]].indexOf(query) != -1) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ public boolean isActive() {
+ return !searchBox.getText().equals("");
+ }
+
+ public void update() {}
+ }
+
+ class SortIndicator extends Composite {
+ protected Image image = new Image();
+
+ public SortIndicator() {
+ initWidget(image);
+ setVisible(false);
+ }
+
+ public void sortOn(boolean up) {
+ image.setUrl(up ? SORT_UP_IMAGE : SORT_DOWN_IMAGE);
+ setVisible(true);
+ }
+
+ public void sortOff() {
+ setVisible(false);
+ }
+ }
+
+ protected List allData = new ArrayList(); // ArrayList<String[]>
+ protected List filteredData = new ArrayList();
+
+ protected int sortDirection = ASCENDING;
+ protected boolean clientSortable = false;
+ protected SortIndicator[] sortIndicators;
+ protected int sortedOn = NO_COLUMN;
+
+ protected Vector filters = new Vector();
+ protected ColumnFilter[] columnFilters;
+ protected Paginator paginator = null;
+
+ protected DynamicTableListener listener;
+
+ public DynamicTable(String[][] columns) {
+ super(columns);
+ columnFilters = new ColumnFilter[columns.length];
+ }
+
+ // SORTING
+
+ /**
+ * Makes the table client sortable, that is, sortable by the user by
+ * clicking on column headers.
+ */
+ public void makeClientSortable() {
+ this.clientSortable = true;
+ table.getRowFormatter().addStyleName(0, DataTable.HEADER_STYLE + "-sortable");
+
+ sortIndicators = new SortIndicator[columns.length];
+ for(int i = 0; i < columns.length; i++) {
+ sortIndicators[i] = new SortIndicator();
+
+ // we have to use an HTMLPanel here to preserve styles correctly and
+ // not break hover
+ // we add a <span> with a unique ID to hold the sort indicator
+ String name = columns[i][1];
+ String id = HTMLPanel.createUniqueId();
+ HTMLPanel panel = new HTMLPanel(name +
+ " <span id=\"" + id + "\"></span>");
+ panel.add(sortIndicators[i], id);
+ table.setWidget(0, i, panel);
+ }
+
+ table.addTableListener(new TableListener() {
+ public void onCellClicked(SourcesTableEvents sender, int row, int cell) {
+ if (row == headerRow) {
+ sortOnColumn(cell);
+ doAllFilters();
+ }
+ }
+ });
+ }
+
+ protected Comparator getRowComparator() {
+ return new Comparator() {
+ public int compare(Object arg0, Object arg1) {
+ String[] row0 = (String[]) arg0;
+ String[] row1 = (String[]) arg1;
+ return row0[sortedOn].compareTo(row1[sortedOn]) * sortDirection;
+ }
+ };
+ }
+
+ protected void sortData(List data) {
+ if (sortedOn == NO_COLUMN)
+ return;
+ Collections.sort(data, getRowComparator());
+ }
+
+ /**
+ * Set column on which data is sorted. You must call <code>updateData()
+ * </code> after this to display the results.
+ * @param column index of the column to sort on
+ */
+ public void sortOnColumn(int column) {
+ if (column == sortedOn)
+ sortDirection *= -1;
+ else
+ sortDirection = ASCENDING;
+
+ if(clientSortable) {
+ if (sortedOn != NO_COLUMN)
+ sortIndicators[sortedOn].sortOff();
+ sortIndicators[column].sortOn(sortDirection == ASCENDING);
+ }
+ sortedOn = column;
+
+ sortData(allData);
+ }
+
+ // PAGINATION
+
+ /**
+ * Add client-side pagination to this table.
+ * @param rowsPerPage size of each page
+ */
+ public void addPaginator(int rowsPerPage) {
+ paginator = new Paginator(rowsPerPage,
+ new Paginator.PaginatorCallback() {
+ public void doRequest(int start) {
+ refreshDisplay();
+ }
+ });
+
+ addHeaderRow(paginator);
+ updatePaginator();
+ }
+
+ protected void updatePaginator() {
+ if(paginator != null)
+ paginator.setNumTotalResults(filteredData.size());
+ }
+
+ /**
+ * Reset paginator back to first page. You must call
+ * <code>updateData()</code> after this to display the results.
+ */
+ public void resetPaginator() {
+ if(paginator != null)
+ paginator.setStart(0);
+ }
+
+ /**
+ * Get the index of the first row currently displayed.
+ */
+ public int getVisibleStart() {
+ return paginator.getStart();
+ }
+
+ /**
+ * Get the number of rows currently displayed.
+ */
+ public int getVisibleCount() {
+ return getRowCount();
+ }
+
+ // DATA MANIPULATION
+
+ public void addRowFromData(String[] rowData) {
+ int insertPosition = allData.size();
+ if (sortedOn != NO_COLUMN) {
+ insertPosition = Collections.binarySearch(allData, rowData,
+ getRowComparator());
+ if (insertPosition < 0)
+ insertPosition = -insertPosition - 1; // see binarySearch() docs
+ }
+ allData.add(insertPosition, rowData);
+ }
+
+ /**
+ * Remove a row from the table.
+ * @param dataRow the index of the row (indexed into the table's filtered
+ * data, not into the currently visible rows)
+ * @return the column data for the removed row
+ */
+ public String[] removeDataRow(int dataRow) {
+ String[] rowText = (String[]) filteredData.remove(dataRow);
+ allData.remove(rowText);
+ return rowText;
+ }
+
+ public void clear() {
+ super.clear();
+ allData.clear();
+ filteredData.clear();
+ }
+
+ /**
+ * This method should be called after any changes to the table's data. It
+ * recomputes choices for column filters, runs all filters to compute
+ * the filtered data set, and updates the display.
+ */
+ public void updateData() {
+ updateFilters();
+ doAllFilters();
+ }
+
+ /**
+ * Get the number of rows in the currently filtered data set.
+ */
+ public int getFilteredRowCount() {
+ return filteredData.size();
+ }
+
+ /**
+ * Get a row in the currently filtered data set.
+ * @param dataRow the index (into the filtered data set) of the row to
+ * retrieve
+ * @return the column data for the row
+ */
+ public String[] getDataRow(int dataRow) {
+ return (String[]) filteredData.get(dataRow);
+ }
+
+ // DISPLAY
+
+ protected void displayRows(List rows, int start, int end) {
+ super.clear();
+ for (int i = start; i < end; i++) {
+ super.addRowFromData((String[]) rows.get(i));
+ }
+ }
+
+ protected void displayRows(List rows) {
+ displayRows(rows, 0, rows.size());
+ }
+
+ /**
+ * Update the table display.
+ */
+ public void refreshDisplay() {
+ if(paginator != null)
+ displayRows(filteredData, paginator.getStart(), paginator.getEnd());
+ else
+ displayRows(filteredData);
+ }
+
+ // INPUT
+
+ protected int visibleRowToFilteredDataRow(int visibleRow) {
+ if (visibleRow <= headerRow)
+ return -1;
+
+ int start = 0;
+ if (paginator != null)
+ start = paginator.getStart();
+ return visibleRow - getHeaderRowCount() + start;
+ }
+
+ /**
+ * Set a DynamicTableListener. This differs from a normal TableListener
+ * because the row index passed is an index into the filtered data set,
+ * rather than an index into the visible table, which would be relatively
+ * useless.
+ */
+ public void setListener(final DynamicTableListener listener) {
+ table.addTableListener(new TableListener() {
+ public void onCellClicked(SourcesTableEvents sender, int row, int cell) {
+ int dataRow = visibleRowToFilteredDataRow(row);
+ if (dataRow != -1)
+ listener.onRowClicked(dataRow, cell);
+ }
+ });
+ }
+
+ // FILTERING
+
+ protected List getActiveFilters() {
+ List activeFilters = new ArrayList();
+ for (Iterator i = filters.iterator(); i.hasNext(); ) {
+ Filter filter = (Filter) i.next();
+ if (filter.isActive())
+ activeFilters.add(filter);
+ }
+ return activeFilters;
+ }
+
+ protected boolean acceptRow(String[] row, List filters) {
+ for (Iterator i = filters.iterator(); i.hasNext(); ) {
+ Filter filter = (Filter) i.next();
+ if(!filter.acceptRow(row)) {
+ return false;
+ }
+ }
+ return true;
+ }
+
+ protected void doAllFilters() {
+ filteredData.clear();
+ List activeFilters = getActiveFilters();
+ for (Iterator i = allData.iterator(); i.hasNext(); ) {
+ String[] row = (String[]) i.next();
+ if (acceptRow(row, activeFilters))
+ filteredData.add(row);
+ }
+
+ updatePaginator();
+ refreshDisplay();
+ }
+
+ protected int columnNameToIndex(String column) {
+ for(int col = 0; col < columns.length; col++) {
+ if (columns[col][1].equals(column))
+ return col;
+ }
+
+ return -1;
+ }
+
+ /**
+ * Add an incremental search filter. This appears as a text box which
+ * performs a substring search on the given columns.
+ * @param searchColumns the titles of columns to perform the search on.
+ */
+ public void addSearchBox(String[] searchColumns, String label) {
+ int[] searchColumnIndices = new int[searchColumns.length];
+ for(int i = 0; i < searchColumns.length; i++) {
+ searchColumnIndices[i] = columnNameToIndex(searchColumns[i]);
+ }
+ SearchFilter searchFilter = new SearchFilter(searchColumnIndices);
+ filters.add(searchFilter);
+
+ final HorizontalPanel searchPanel = new HorizontalPanel();
+ final Label searchLabel = new Label(label);
+ searchPanel.add(searchLabel);
+ searchPanel.add(searchFilter.searchBox);
+ addHeaderRow(searchPanel);
+ searchFilter.searchBox.addKeyboardListener(new KeyboardListener() {
+ public void onKeyPress(Widget sender, char keyCode, int modifiers) {}
+ public void onKeyDown(Widget sender, char keyCode, int modifiers) {}
+ public void onKeyUp(Widget sender, char keyCode, int modifiers) {
+ doAllFilters();
+ }
+ });
+ }
+
+ protected String[] gatherChoices(int column) {
+ Set choices = new HashSet();
+ for(Iterator i = allData.iterator(); i.hasNext(); ) {
+ String[] row = (String[]) i.next();
+ choices.add(row[column]);
+ }
+
+ if (choices.isEmpty())
+ return new String[0];
+ List sortedChoices = new ArrayList(choices);
+ Collections.sort(sortedChoices);
+ return (String[]) sortedChoices.toArray(new String[1]);
+ }
+
+ /**
+ * Add a column filter. This presents choices for the value of the given
+ * column in a list box, and filters based on the user's choice. This
+ * method allows the caller to specify the choices for the list box.
+ * @param column the title of the column to filter on
+ * @param choices the choices for the filter box (not includes "All values")
+ */
+ public void addColumnFilter(String column, String[] choices) {
+ final ColumnFilter filter = new ColumnFilter(columnNameToIndex(column));
+ if (choices != null) {
+ filter.setChoices(choices);
+ filter.isManualChoices = true;
+ }
+ filter.select.addChangeListener(new ChangeListener() {
+ public void onChange(Widget sender) {
+ doAllFilters();
+ }
+ });
+ filters.add(filter);
+ columnFilters[columnNameToIndex(column)] = filter;
+
+ Label filterLabel = new Label(column + ":");
+ HorizontalPanel filterPanel = new HorizontalPanel();
+ filterPanel.add(filterLabel);
+ filterPanel.add(filter.select);
+
+ addHeaderRow(filterPanel);
+ }
+
+ /**
+ * Add a column filter without specifying choices. Choices will
+ * automatically be determined by gathering all values for the given column
+ * present in the table.
+ */
+ public void addColumnFilter(String column) {
+ addColumnFilter(column, null);
+ }
+
+ /**
+ * Set the selected choice for the column filter of the given column.
+ * @param columnName title of the column for which the filter should be set
+ * @param choice value to select in the filter
+ */
+ public void setColumnFilterChoice(String columnName, String choice) {
+ int column = columnNameToIndex(columnName);
+ ColumnFilter filter = columnFilters[column];
+ if (filter == null)
+ throw new IllegalArgumentException(
+ "No filter on column " + columnName);
+ filter.setChoice(choice);
+ }
+
+ protected void updateFilters() {
+ for(Iterator i = filters.iterator(); i.hasNext(); ) {
+ Filter filter = (Filter) i.next();
+ filter.update();
+ }
+ }
+}
diff --git a/frontend/client/src/afeclient/client/ElementWidget.java b/frontend/client/src/afeclient/client/ElementWidget.java
new file mode 100644
index 0000000..c46fab4
--- /dev/null
+++ b/frontend/client/src/afeclient/client/ElementWidget.java
@@ -0,0 +1,28 @@
+package afeclient.client;
+
+import com.google.gwt.user.client.DOM;
+import com.google.gwt.user.client.Element;
+import com.google.gwt.user.client.ui.Widget;
+
+/**
+ * A simple widget that wraps an HTML element. This allows the element to be
+ * removed from the document and added to a Panel.
+ */
+public class ElementWidget extends Widget {
+ protected Element element;
+
+ /**
+ * @param element the HTML element to wrap
+ */
+ public ElementWidget(Element element) {
+ this.element = element;
+ setElement(element);
+ }
+
+ /**
+ * Remove the wrapped element from the document.
+ */
+ public void removeFromDocument() {
+ DOM.removeChild(DOM.getParent(element), element);
+ }
+}
diff --git a/frontend/client/src/afeclient/client/HostListView.java b/frontend/client/src/afeclient/client/HostListView.java
new file mode 100644
index 0000000..c8dbc3e
--- /dev/null
+++ b/frontend/client/src/afeclient/client/HostListView.java
@@ -0,0 +1,18 @@
+package afeclient.client;
+
+import com.google.gwt.user.client.ui.RootPanel;
+
+public class HostListView extends TabView {
+ protected static final int HOSTS_PER_PAGE = 30;
+
+ public String getElementId() {
+ return "hosts";
+ }
+
+ protected HostTable hostTable = new HostTable(HOSTS_PER_PAGE);
+
+ public void initialize() {
+ hostTable.getHosts();
+ RootPanel.get("hosts_list").add(hostTable);
+ }
+}
diff --git a/frontend/client/src/afeclient/client/HostSelector.java b/frontend/client/src/afeclient/client/HostSelector.java
new file mode 100644
index 0000000..cb56230
--- /dev/null
+++ b/frontend/client/src/afeclient/client/HostSelector.java
@@ -0,0 +1,200 @@
+package afeclient.client;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import com.google.gwt.json.client.JSONArray;
+import com.google.gwt.user.client.ui.Button;
+import com.google.gwt.user.client.ui.ClickListener;
+import com.google.gwt.user.client.ui.HorizontalPanel;
+import com.google.gwt.user.client.ui.ListBox;
+import com.google.gwt.user.client.ui.Panel;
+import com.google.gwt.user.client.ui.RootPanel;
+import com.google.gwt.user.client.ui.TextBox;
+import com.google.gwt.user.client.ui.Widget;
+
+/**
+ * A widget to facilitate selection of a group of hosts for running a job. The
+ * widget displays two side-by-side tables; the left table is a normal
+ * {@link HostTable} displaying available, unselected hosts, and the right table
+ * displays selected hosts. Click on a host in either table moves it to the
+ * other (i.e. selects or deselects a host). The widget provides several
+ * convenience controls (such as one to remove all selected hosts) and a special
+ * section for adding meta-host entries.
+ */
+public class HostSelector {
+ public static final int TABLE_SIZE = 10;
+ public static final String META_PREFIX = "Any ";
+
+ class HostSelection {
+ public List hosts = new ArrayList();
+ public List metaHosts = new ArrayList();
+ }
+
+ protected HostTable availableTable = new HostTable(TABLE_SIZE);
+ protected DynamicTable selectedTable =
+ new DynamicTable(HostTable.HOST_COLUMNS);
+
+ public HostSelector() {
+ selectedTable.setClickable(true);
+ selectedTable.addPaginator(TABLE_SIZE);
+ selectedTable.sortOnColumn(0);
+
+ availableTable.setClickable(true);
+ availableTable.getHosts();
+
+ availableTable.setListener(new DynamicTable.DynamicTableListener() {
+ public void onRowClicked(int dataRow, int column) {
+ selectRow(dataRow);
+ }
+ });
+ selectedTable.setListener(new DynamicTable.DynamicTableListener() {
+ public void onRowClicked(int dataRow, int column) {
+ deselectRow(dataRow);
+ }
+ });
+
+ RootPanel.get("create_available_table").add(availableTable);
+ RootPanel.get("create_selected_table").add(selectedTable);
+
+ Button addVisibleButton = new Button("Add currently displayed");
+ addVisibleButton.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ addVisible();
+ }
+ });
+ Button addFilteredButton = new Button("Add all filtered");
+ addFilteredButton.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ moveAll(availableTable, selectedTable);
+ }
+ });
+ Button removeAllButton = new Button("Remove all");
+ removeAllButton.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ moveAll(selectedTable, availableTable);
+ }
+ });
+
+ Panel availableControls = new HorizontalPanel();
+ availableControls.add(addVisibleButton);
+ availableControls.add(addFilteredButton);
+ RootPanel.get("create_available_controls").add(availableControls);
+ RootPanel.get("create_selected_controls").add(removeAllButton);
+
+ final ListBox metaLabelSelect = new ListBox();
+ populateLabels(metaLabelSelect);
+ final TextBox metaNumber = new TextBox();
+ metaNumber.setVisibleLength(4);
+ final Button metaButton = new Button("Add");
+ metaButton.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ int selected = metaLabelSelect.getSelectedIndex();
+ String label = metaLabelSelect.getItemText(selected);
+ String number = metaNumber.getText();
+ try {
+ Integer.parseInt(number);
+ }
+ catch (NumberFormatException exc) {
+ String error = "Invalid number " + number;
+ NotifyManager.getInstance().showError(error);
+ return;
+ }
+ String[] rowData = new String[4];
+ rowData[0] = META_PREFIX + number;
+ rowData[1] = label;
+ rowData[2] = rowData[3] = "";
+ selectedTable.addRowFromData(rowData);
+ selectedTable.updateData();
+ }
+ });
+ RootPanel.get("create_meta_select").add(metaLabelSelect);
+ RootPanel.get("create_meta_number").add(metaNumber);
+ RootPanel.get("create_meta_button").add(metaButton);
+ }
+
+ protected void moveRow(int row, DynamicTable from, DataTable to) {
+ String[] rowData = from.removeDataRow(row);
+ if(isMetaEntry(rowData))
+ return;
+ to.addRowFromData(rowData);
+ }
+
+ protected void updateAfterMove(DynamicTable from, DynamicTable to) {
+ from.updateData();
+ to.updateData();
+ }
+
+ protected void selectRow(int row) {
+ moveRow(row, availableTable, selectedTable);
+ updateAfterMove(availableTable, selectedTable);
+ }
+
+ protected void deselectRow(int row) {
+ moveRow(row, selectedTable, availableTable);
+ updateAfterMove(selectedTable, availableTable);
+ }
+
+ protected void addVisible() {
+ int start = availableTable.getVisibleStart();
+ int count = availableTable.getVisibleCount();
+ for (int i = 0; i < count; i++) {
+ moveRow(start, availableTable, selectedTable);
+ }
+ updateAfterMove(availableTable, selectedTable);
+ }
+
+ protected void moveAll(DynamicTable from, DynamicTable to) {
+ int total = from.getFilteredRowCount();
+ for (int i = 0; i < total; i++) {
+ moveRow(0, from, to);
+ }
+ updateAfterMove(from, to);
+ }
+
+ protected void populateLabels(ListBox list) {
+ StaticDataRepository staticData = StaticDataRepository.getRepository();
+ JSONArray labels = staticData.getData("labels").isArray();
+ for(int i = 0; i < labels.size(); i++) {
+ list.addItem(labels.get(i).isString().stringValue());
+ }
+ }
+
+ protected boolean isMetaEntry(String[] row) {
+ return row[0].startsWith(META_PREFIX);
+ }
+
+ protected int getMetaNumber(String[] row) {
+ return Integer.parseInt(row[0].substring(META_PREFIX.length()));
+ }
+
+ /**
+ * Retrieve the set of selected hosts.
+ */
+ public HostSelection getSelectedHosts() {
+ HostSelection selection = new HostSelection();
+ for(int i = 0; i < selectedTable.getFilteredRowCount(); i++) {
+ String[] row = selectedTable.getDataRow(i);
+ if (isMetaEntry(row)) {
+ int count = getMetaNumber(row);
+ String platform = row[1];
+ for(int counter = 0; counter < count; counter++) {
+ selection.metaHosts.add(platform);
+ }
+ }
+ else {
+ String hostname = row[0];
+ selection.hosts.add(hostname);
+ }
+ }
+
+ return selection;
+ }
+
+ /**
+ * Reset the widget (deselect all hosts).
+ */
+ public void reset() {
+ moveAll(selectedTable, availableTable);
+ }
+}
diff --git a/frontend/client/src/afeclient/client/HostTable.java b/frontend/client/src/afeclient/client/HostTable.java
new file mode 100644
index 0000000..ac1776b
--- /dev/null
+++ b/frontend/client/src/afeclient/client/HostTable.java
@@ -0,0 +1,58 @@
+package afeclient.client;
+
+import com.google.gwt.json.client.JSONArray;
+import com.google.gwt.json.client.JSONObject;
+import com.google.gwt.json.client.JSONString;
+import com.google.gwt.json.client.JSONValue;
+
+/**
+ * A table to display hosts.
+ */
+public class HostTable extends DynamicTable {
+ protected static final String LOCKED_TEXT = "locked_text";
+ public static final String[][] HOST_COLUMNS = {
+ {"hostname", "Hostname"}, {"platform", "Platform"},
+ {"status", "Status"}, {LOCKED_TEXT, "Locked"}
+ };
+
+ protected static JSONArray hosts = null;
+
+ protected static final JsonRpcProxy rpcProxy = JsonRpcProxy.getProxy();
+
+ public HostTable(int hostsPerPage) {
+ super(HOST_COLUMNS);
+ makeClientSortable();
+
+ String[] searchColumns = {"Hostname"};
+ addSearchBox(searchColumns, "Hostname:");
+ addColumnFilter("Platform");
+ addColumnFilter("Status");
+ addColumnFilter("Locked");
+
+ addPaginator(hostsPerPage);
+ }
+
+ protected void preprocessRow(JSONObject row) {
+ super.preprocessRow(row);
+ boolean locked = row.get("locked").isNumber().getValue() > 0;
+ String lockedText = locked ? "Yes" : "No";
+ row.put(LOCKED_TEXT, new JSONString(lockedText));
+ }
+
+ public void getHosts() {
+ clear();
+ JsonRpcCallback handleHosts = new JsonRpcCallback() {
+ public void onSuccess(JSONValue result) {
+ hosts = result.isArray();
+ addRows(hosts);
+ sortOnColumn(0);
+ updateData();
+ }
+ };
+
+ if(hosts == null)
+ rpcProxy.rpcCall("get_hosts", null, handleHosts);
+ else
+ handleHosts.onSuccess(hosts);
+ }
+}
diff --git a/frontend/client/src/afeclient/client/JobDetailView.java b/frontend/client/src/afeclient/client/JobDetailView.java
new file mode 100644
index 0000000..35fdda1
--- /dev/null
+++ b/frontend/client/src/afeclient/client/JobDetailView.java
@@ -0,0 +1,230 @@
+package afeclient.client;
+
+import com.google.gwt.json.client.JSONArray;
+import com.google.gwt.json.client.JSONNumber;
+import com.google.gwt.json.client.JSONObject;
+import com.google.gwt.json.client.JSONString;
+import com.google.gwt.json.client.JSONValue;
+import com.google.gwt.user.client.DOM;
+import com.google.gwt.user.client.ui.Button;
+import com.google.gwt.user.client.ui.ClickListener;
+import com.google.gwt.user.client.ui.KeyboardListener;
+import com.google.gwt.user.client.ui.RootPanel;
+import com.google.gwt.user.client.ui.TextBox;
+import com.google.gwt.user.client.ui.Widget;
+
+import java.util.Iterator;
+import java.util.Set;
+
+public class JobDetailView extends TabView {
+ public static final String NO_URL = "about:blank";
+ public static final String NO_JOB = "No job selected";
+ public static final String GO_TEXT = "Go";
+ public static final String REFRESH_TEXT = "Refresh";
+ public static final int NO_JOB_ID = -1;
+
+ public String getElementId() {
+ return "view_job";
+ }
+
+ protected JsonRpcProxy rpcProxy = JsonRpcProxy.getProxy();
+
+ protected int jobId = NO_JOB_ID;
+
+ protected RootPanel allJobData;
+
+ protected JobHostsTable hostsTable;
+ protected TextBox idInput = new TextBox();
+ protected Button idFetchButton = new Button(GO_TEXT);
+ protected Button abortButton = new Button("Abort job");
+
+ protected void showText(String text, String elementId) {
+ DOM.setInnerText(RootPanel.get(elementId).getElement(), text);
+ }
+
+ protected void showField(JSONObject job, String field, String elementId) {
+ JSONString jsonString = job.get(field).isString();
+ String value = "";
+ if (jsonString != null)
+ value = jsonString.stringValue();
+ showText(value, elementId);
+ }
+
+ public void setJobID(int id) {
+ this.jobId = id;
+ idInput.setText(Integer.toString(id));
+ idFetchButton.setText(REFRESH_TEXT);
+ refresh();
+ }
+
+ public void resetPage() {
+ showText(NO_JOB, "view_title");
+ allJobData.setVisible(false);
+ }
+
+ public void refresh() {
+ pointToResults(NO_URL, NO_URL);
+ JSONObject params = new JSONObject();
+ params.put("id", new JSONNumber(jobId));
+ rpcProxy.rpcCall("get_jobs_summary", params, new JsonRpcCallback() {
+ public void onSuccess(JSONValue result) {
+ JSONArray resultArray = result.isArray();
+ if(resultArray.size() == 0) {
+ NotifyManager.getInstance().showError("No such job found");
+ resetPage();
+ return;
+ }
+ JSONObject jobObject = resultArray.get(0).isObject();
+ String name = jobObject.get("name").isString().stringValue();
+ String owner = jobObject.get("owner").isString().stringValue();
+ String jobLogsId = jobId + "-" + owner;
+ String title = "Job: " + name + " (" + jobLogsId + ")";
+ showText(title, "view_title");
+
+ showText(name, "view_label");
+ showText(owner, "view_owner");
+ showField(jobObject, "priority", "view_priority");
+ showField(jobObject, "created_on", "view_created");
+ showField(jobObject, "control_type", "view_control_type");
+ showField(jobObject, "control_file", "view_control_file");
+
+ String synchType = jobObject.get("synch_type").isString().stringValue();
+ showText(synchType.toLowerCase(), "view_synch_type");
+
+ JSONObject counts = jobObject.get("status_counts").isObject();
+ String countString = Utils.formatStatusCounts(counts, ", ");
+ showText(countString, "view_status");
+ abortButton.setVisible(!allFinishedCounts(counts));
+
+ pointToResults(getResultsURL(jobId), getLogsURL(jobLogsId));
+
+ allJobData.setVisible(true);
+
+ hostsTable.getHosts(jobId);
+ }
+
+ public void onError(JSONObject errorObject) {
+ super.onError(errorObject);
+ resetPage();
+ }
+ });
+ }
+
+ protected boolean allFinishedCounts(JSONObject statusCounts) {
+ Set keys = statusCounts.keySet();
+ for (Iterator i = keys.iterator(); i.hasNext(); ) {
+ String key = (String) i.next();
+ if (!(key.equals("Completed") ||
+ key.equals("Failed") ||
+ key.equals("Aborted") ||
+ key.equals("Stopped")))
+ return false;
+ }
+ return true;
+ }
+
+ public void fetchById() {
+ String id = idInput.getText();
+ try {
+ setJobID(Integer.parseInt(id));
+ updateHistory();
+ }
+ catch (NumberFormatException exc) {
+ String error = "Invalid job ID " + id;
+ NotifyManager.getInstance().showError(error);
+ }
+ }
+
+ public void initialize() {
+ allJobData = RootPanel.get("view_data");
+
+ resetPage();
+
+ RootPanel.get("job_id_fetch_controls").add(idInput);
+ RootPanel.get("job_id_fetch_controls").add(idFetchButton);
+ idInput.setVisibleLength(5);
+ idInput.addKeyboardListener(new KeyboardListener() {
+ public void onKeyPress(Widget sender, char keyCode, int modifiers) {
+ if (keyCode == (char) KEY_ENTER)
+ fetchById();
+ }
+
+ public void onKeyDown(Widget sender, char keyCode, int modifiers) {}
+ public void onKeyUp(Widget sender, char keyCode, int modifiers) {}
+ });
+ idFetchButton.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ fetchById();
+ }
+ });
+ idInput.addKeyboardListener(new KeyboardListener() {
+ public void onKeyPress(Widget sender, char keyCode, int modifiers) {
+ idFetchButton.setText(GO_TEXT);
+ }
+ public void onKeyDown(Widget sender, char keyCode, int modifiers) {}
+ public void onKeyUp(Widget sender, char keyCode, int modifiers) {}
+ });
+
+ hostsTable = new JobHostsTable(rpcProxy);
+ RootPanel.get("job_hosts_table").add(hostsTable);
+
+ abortButton.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ abortJob();
+ }
+ });
+ RootPanel.get("view_abort").add(abortButton);
+ }
+
+ protected void abortJob() {
+ JSONObject params = new JSONObject();
+ params.put("id", new JSONNumber(jobId));
+ rpcProxy.rpcCall("abort_job", params, new JsonRpcCallback() {
+ public void onSuccess(JSONValue result) {
+ refresh();
+ }
+ });
+ }
+
+ protected String getResultsURL(int jobId) {
+ return "/tko/compose_query.cgi?" +
+ "columns=test&rows=hostname&condition=tag%7E%27" +
+ Integer.toString(jobId) + "-%25%27&title=Report";
+ }
+
+ /**
+ * Get the path for a job's raw result files.
+ * @param jobLogsId id-owner, e.g. "172-showard"
+ */
+ protected String getLogsURL(String jobLogsId) {
+ return "/results/" + jobLogsId;
+ }
+
+ protected void pointToResults(String resultsUrl, String logsUrl) {
+ DOM.setElementProperty(DOM.getElementById("results_link"),
+ "href", resultsUrl);
+ DOM.setElementProperty(DOM.getElementById("results_iframe"),
+ "src", resultsUrl);
+ DOM.setElementProperty(DOM.getElementById("raw_results_link"),
+ "href", logsUrl);
+ }
+
+ public String getHistoryToken() {
+ String token = super.getHistoryToken();
+ if (jobId != NO_JOB_ID)
+ token += "_" + jobId;
+ return token;
+ }
+
+ public void handleHistoryToken(String token) {
+ int newJobId;
+ try {
+ newJobId = Integer.parseInt(token);
+ }
+ catch (NumberFormatException exc) {
+ return;
+ }
+ if (newJobId != jobId)
+ setJobID(newJobId);
+ }
+}
diff --git a/frontend/client/src/afeclient/client/JobHostsTable.java b/frontend/client/src/afeclient/client/JobHostsTable.java
new file mode 100644
index 0000000..08e7944
--- /dev/null
+++ b/frontend/client/src/afeclient/client/JobHostsTable.java
@@ -0,0 +1,65 @@
+package afeclient.client;
+
+import java.util.Iterator;
+import java.util.Set;
+
+import com.google.gwt.json.client.JSONNumber;
+import com.google.gwt.json.client.JSONObject;
+import com.google.gwt.json.client.JSONString;
+import com.google.gwt.json.client.JSONValue;
+
+/**
+ * A table to display host queue entries associated with a job.
+ */
+public class JobHostsTable extends DynamicTable {
+ public static final int HOSTS_PER_PAGE = 30;
+
+ protected JsonRpcProxy rpcProxy;
+
+ public static final String[][] JOB_HOSTS_COLUMNS = {
+ {"hostname", "Host"}, {"status", "Status"}
+ };
+
+ public JobHostsTable(JsonRpcProxy proxy) {
+ super(JOB_HOSTS_COLUMNS);
+ this.rpcProxy = proxy;
+
+ makeClientSortable();
+
+ String[] searchColumns = {"Host"};
+ addSearchBox(searchColumns, "Hostname:");
+ addColumnFilter("Status");
+ addPaginator(HOSTS_PER_PAGE);
+ }
+
+ public void getHosts(int jobId) {
+ clear();
+ JSONObject params = new JSONObject();
+ params.put("id", new JSONNumber(jobId));
+ rpcProxy.rpcCall("job_status", params, new JsonRpcCallback() {
+ public void onSuccess(JSONValue result) {
+ JSONObject resultObj = result.isObject();
+ Set hostnames = resultObj.keySet();
+ for(Iterator i = hostnames.iterator(); i.hasNext(); ) {
+ String host = (String) i.next();
+ JSONObject hostData = resultObj.get(host).isObject();
+ String status = hostData.get("status").isString().stringValue();
+ JSONValue metaCountValue = hostData.get("meta_count");
+ if (metaCountValue.isNull() == null) {
+ int metaCount = (int) metaCountValue.isNumber().getValue();
+ host += " (label)";
+ status = Integer.toString(metaCount) + " unassigned";
+ }
+
+ JSONObject row = new JSONObject();
+ row.put("hostname", new JSONString(host));
+ row.put("status", new JSONString(status));
+ addRow(row);
+ }
+
+ sortOnColumn(0);
+ updateData();
+ }
+ });
+ }
+}
diff --git a/frontend/client/src/afeclient/client/JobListView.java b/frontend/client/src/afeclient/client/JobListView.java
new file mode 100644
index 0000000..870fb64
--- /dev/null
+++ b/frontend/client/src/afeclient/client/JobListView.java
@@ -0,0 +1,236 @@
+package afeclient.client;
+
+import com.google.gwt.json.client.JSONArray;
+import com.google.gwt.json.client.JSONBoolean;
+import com.google.gwt.json.client.JSONNull;
+import com.google.gwt.json.client.JSONNumber;
+import com.google.gwt.json.client.JSONObject;
+import com.google.gwt.json.client.JSONString;
+import com.google.gwt.json.client.JSONValue;
+import com.google.gwt.user.client.ui.Button;
+import com.google.gwt.user.client.ui.ChangeListener;
+import com.google.gwt.user.client.ui.ClickListener;
+import com.google.gwt.user.client.ui.HorizontalPanel;
+import com.google.gwt.user.client.ui.Hyperlink;
+import com.google.gwt.user.client.ui.ListBox;
+import com.google.gwt.user.client.ui.RootPanel;
+import com.google.gwt.user.client.ui.Widget;
+
+public class JobListView extends TabView {
+ protected static final String ALL_USERS = "All Users";
+ protected static final String SELECTED_LINK_STYLE = "selected-link";
+ protected static final int JOBS_PER_PAGE = 30;
+ protected static final String QUEUED_TOKEN = "queued",
+ RUNNING_TOKEN = "running", FINISHED_TOKEN = "finished",
+ ALL_TOKEN = "all";
+
+ public String getElementId() {
+ return "job_list";
+ }
+
+ protected JsonRpcProxy rpcProxy = JsonRpcProxy.getProxy();
+ protected JSONObject jobFilterArgs = new JSONObject();
+ protected JobTable.JobTableListener selectListener;
+
+ protected JobTable jobTable;
+ protected Hyperlink jobQueueLink, jobRunningLink, jobHistoryLink,
+ allJobsLink, selectedLink = null;
+ protected Button refreshButton;
+ protected ListBox userList;
+ protected Paginator paginator;
+
+ protected Hyperlink nextLink, prevLink;
+
+ protected void addCommonParams() {
+ jobFilterArgs.put("query_limit", new JSONNumber(JOBS_PER_PAGE));
+ jobFilterArgs.put("sort_by", new JSONString("-id"));
+ }
+
+ protected void addUserParam() {
+ String user = userList.getValue(userList.getSelectedIndex());
+ JSONValue value = JSONNull.getInstance();
+ if (!user.equals(ALL_USERS))
+ value = new JSONString(user);
+ jobFilterArgs.put("owner", value);
+ }
+
+ protected void onFiltersChanged() {
+ addUserParam();
+ addCommonParams();
+ resetPagination();
+ }
+
+ protected void selectQueued() {
+ selectLink(jobQueueLink);
+ jobFilterArgs = new JSONObject();
+ jobFilterArgs.put("not_yet_run", JSONBoolean.getInstance(true));
+ onFiltersChanged();
+ }
+
+ protected void selectRunning() {
+ selectLink(jobRunningLink);
+ jobFilterArgs = new JSONObject();
+ jobFilterArgs.put("running", JSONBoolean.getInstance(true));
+ onFiltersChanged();
+ }
+
+ protected void selectHistory() {
+ selectLink(jobHistoryLink);
+ jobFilterArgs = new JSONObject();
+ jobFilterArgs.put("finished", JSONBoolean.getInstance(true));
+ onFiltersChanged();
+ }
+
+ protected void selectAll() {
+ selectLink(allJobsLink);
+ jobFilterArgs = new JSONObject();
+ onFiltersChanged();
+ }
+
+ protected void refresh() {
+ updateNumJobs();
+ jobTable.getJobs(jobFilterArgs);
+ }
+
+ protected void updateNumJobs() {
+ rpcProxy.rpcCall("get_num_jobs", jobFilterArgs, new JsonRpcCallback() {
+ public void onSuccess(JSONValue result) {
+ int numJobs = (int) result.isNumber().getValue();
+ paginator.setNumTotalResults(numJobs);
+ }
+ });
+ }
+
+ protected void populateUsers() {
+ userList.addItem(ALL_USERS);
+
+ StaticDataRepository staticData = StaticDataRepository.getRepository();
+ JSONArray userArray = staticData.getData("users").isArray();
+ String currentUser = staticData.getData("user_login").isString().stringValue();
+ int numUsers = userArray.size();
+ for (int i = 0; i < numUsers; i++) {
+ String name = userArray.get(i).isString().stringValue();
+ userList.addItem(name);
+ if (name.equals(currentUser))
+ userList.setSelectedIndex(i + 1); // +1 for "All users" at top
+ }
+ }
+
+ protected void selectLink(Hyperlink link) {
+ jobQueueLink.removeStyleName(SELECTED_LINK_STYLE);
+ jobRunningLink.removeStyleName(SELECTED_LINK_STYLE);
+ jobHistoryLink.removeStyleName(SELECTED_LINK_STYLE);
+ allJobsLink.removeStyleName(SELECTED_LINK_STYLE);
+ link.addStyleName(SELECTED_LINK_STYLE);
+ selectedLink = link;
+ }
+
+ protected void resetPagination() {
+ jobFilterArgs.put("query_start", new JSONNumber(0));
+ paginator.setStart(0);
+ }
+
+ public JobListView(JobTable.JobTableListener listener) {
+ selectListener = listener;
+ }
+
+ public void initialize() {
+ jobTable = new JobTable(selectListener);
+ jobTable.setClickable(true);
+ RootPanel.get("job_table").add(jobTable);
+
+ paginator = new Paginator(JOBS_PER_PAGE, new Paginator.PaginatorCallback() {
+ public void doRequest(int start) {
+ jobFilterArgs.put("query_start", new JSONNumber(start));
+ refresh();
+ }
+ });
+ RootPanel.get("job_pagination").add(paginator);
+
+ ClickListener linkListener = new ClickListener() {
+ public void onClick(Widget sender) {
+ Hyperlink senderLink = (Hyperlink) sender;
+ String fullToken = senderLink.getTargetHistoryToken();
+ int prefixLength = JobListView.super.getHistoryToken().length();
+ String linkToken = fullToken.substring(prefixLength + 1); // +1 for underscore
+ handleHistoryToken(linkToken);
+ }
+ };
+
+ jobQueueLink = new Hyperlink("Queued Jobs",
+ super.getHistoryToken() + "_" + QUEUED_TOKEN);
+ jobQueueLink.addClickListener(linkListener);
+ jobQueueLink.setStyleName("job-filter-link");
+
+ jobRunningLink = new Hyperlink("Running Jobs",
+ super.getHistoryToken() + "_" + RUNNING_TOKEN);
+ jobRunningLink.addClickListener(linkListener);
+ jobRunningLink.setStyleName("job-filter-link");
+
+ jobHistoryLink = new Hyperlink("Finished Jobs",
+ super.getHistoryToken() + "_" + FINISHED_TOKEN);
+ jobHistoryLink.addClickListener(linkListener);
+ jobHistoryLink.setStyleName("job-filter-link");
+
+ allJobsLink = new Hyperlink("All Jobs",
+ super.getHistoryToken() + "_" + ALL_TOKEN);
+ allJobsLink.addClickListener(linkListener);
+ allJobsLink.setStyleName("job-filter-link");
+
+ HorizontalPanel jobLinks = new HorizontalPanel();
+ RootPanel.get("job_control_links").add(jobLinks);
+ jobLinks.add(jobQueueLink);
+ jobLinks.add(jobRunningLink);
+ jobLinks.add(jobHistoryLink);
+ jobLinks.add(allJobsLink);
+
+ refreshButton = new Button("Refresh");
+ refreshButton.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ refresh();
+ }
+ });
+ jobLinks.add(refreshButton);
+
+ userList = new ListBox();
+ userList.addChangeListener(new ChangeListener() {
+ public void onChange(Widget sender) {
+ addUserParam();
+ resetPagination();
+ refresh();
+ }
+ });
+ populateUsers();
+ RootPanel.get("user_list").add(userList);
+
+ // all jobs is selected by default
+ selectAll();
+ }
+
+ public String getHistoryToken() {
+ return selectedLink.getTargetHistoryToken();
+ }
+
+ public void handleHistoryToken(String token) {
+ if (token.equals(QUEUED_TOKEN)) {
+ selectQueued();
+ }
+ else if (token.equals(RUNNING_TOKEN)) {
+ selectRunning();
+ }
+ else if (token.equals(FINISHED_TOKEN)) {
+ selectHistory();
+ }
+ else if (token.equals(ALL_TOKEN)) {
+ selectAll();
+ }
+ }
+
+ /**
+ * Override to refresh the list every time the tab is displayed.
+ */
+ public void display() {
+ super.display();
+ refresh();
+ }
+}
diff --git a/frontend/client/src/afeclient/client/JobTable.java b/frontend/client/src/afeclient/client/JobTable.java
new file mode 100644
index 0000000..675b9e5
--- /dev/null
+++ b/frontend/client/src/afeclient/client/JobTable.java
@@ -0,0 +1,65 @@
+package afeclient.client;
+
+import com.google.gwt.json.client.JSONArray;
+import com.google.gwt.json.client.JSONObject;
+import com.google.gwt.json.client.JSONString;
+import com.google.gwt.json.client.JSONValue;
+import com.google.gwt.user.client.ui.SourcesTableEvents;
+import com.google.gwt.user.client.ui.TableListener;
+
+/**
+ * A table to display jobs, including a summary of host queue entries.
+ */
+public class JobTable extends DataTable {
+ public static final String HOSTS_SUMMARY = "hosts_summary";
+ public static final String CREATED_TEXT = "created_text";
+
+ interface JobTableListener {
+ public void onJobClicked(int jobId);
+ }
+
+ protected JsonRpcProxy rpcProxy = JsonRpcProxy.getProxy();
+
+ public static final String[][] JOB_COLUMNS = { { "id", "ID" },
+ { "owner", "Owner" }, { "name", "Name" },
+ { "priority", "Priority" }, { "control_type", "Client/Server" },
+ { CREATED_TEXT, "Created" }, { HOSTS_SUMMARY, "Status" } };
+
+ public JobTable(final JobTableListener listener) {
+ super(JOB_COLUMNS);
+
+ if (listener != null) {
+ table.addTableListener(new TableListener() {
+ public void onCellClicked(SourcesTableEvents sender, int row, int cell) {
+ int jobId = Integer.parseInt(table.getHTML(row, 0));
+ listener.onJobClicked(jobId);
+ }
+ });
+ }
+ }
+
+ protected void preprocessRow(JSONObject row) {
+ JSONObject counts = row.get("status_counts").isObject();
+ String countString = Utils.formatStatusCounts(counts, "<br>");
+ row.put(HOSTS_SUMMARY, new JSONString(countString));
+
+ // remove seconds from created time
+ JSONValue createdValue = row.get("created_on");
+ String created = "";
+ if (createdValue.isNull() == null) {
+ created = createdValue.isString().stringValue();
+ created = created.substring(0, created.length() - 3);
+ }
+ row.put(CREATED_TEXT, new JSONString(created));
+ }
+
+ public void getJobs(JSONObject filterInfo) {
+ rpcProxy.rpcCall("get_jobs_summary", filterInfo, new JsonRpcCallback() {
+ public void onSuccess(JSONValue result) {
+ clear();
+ JSONArray jobs = result.isArray();
+ addRows(jobs);
+ }
+ });
+ }
+}
diff --git a/frontend/client/src/afeclient/client/JsonRpcCallback.java b/frontend/client/src/afeclient/client/JsonRpcCallback.java
new file mode 100644
index 0000000..f0c0c92
--- /dev/null
+++ b/frontend/client/src/afeclient/client/JsonRpcCallback.java
@@ -0,0 +1,14 @@
+package afeclient.client;
+
+import com.google.gwt.json.client.JSONObject;
+import com.google.gwt.json.client.JSONValue;
+
+abstract class JsonRpcCallback {
+ public abstract void onSuccess(JSONValue result);
+ public void onError(JSONObject errorObject) {
+ String name = errorObject.get("name").isString().stringValue();
+ String message = errorObject.get("message").isString().stringValue();
+ String errorString = name + ": " + message;
+ NotifyManager.getInstance().showError(errorString);
+ }
+}
diff --git a/frontend/client/src/afeclient/client/JsonRpcProxy.java b/frontend/client/src/afeclient/client/JsonRpcProxy.java
new file mode 100644
index 0000000..93825a8
--- /dev/null
+++ b/frontend/client/src/afeclient/client/JsonRpcProxy.java
@@ -0,0 +1,109 @@
+package afeclient.client;
+
+import com.google.gwt.json.client.JSONArray;
+import com.google.gwt.json.client.JSONException;
+import com.google.gwt.json.client.JSONNull;
+import com.google.gwt.json.client.JSONNumber;
+import com.google.gwt.json.client.JSONObject;
+import com.google.gwt.json.client.JSONParser;
+import com.google.gwt.json.client.JSONString;
+import com.google.gwt.json.client.JSONValue;
+import com.google.gwt.user.client.HTTPRequest;
+import com.google.gwt.user.client.ResponseTextHandler;
+
+import java.util.Iterator;
+import java.util.Set;
+
+/**
+ * A singleton class to facilitate RPC calls to the server.
+ */
+public class JsonRpcProxy {
+ public static final JsonRpcProxy theInstance = new JsonRpcProxy();
+
+ protected NotifyManager notify = NotifyManager.getInstance();
+
+ protected String url;
+
+ // singleton
+ private JsonRpcProxy() {}
+
+ public static JsonRpcProxy getProxy() {
+ return theInstance;
+ }
+
+ /**
+ * Set the URL to which requests are sent.
+ */
+ public void setUrl(String url) {
+ this.url = url;
+ }
+
+ protected JSONArray processParams(JSONObject params) {
+ JSONArray result = new JSONArray();
+ JSONObject newParams = new JSONObject();
+ if (params != null) {
+ Set keys = params.keySet();
+ for (Iterator i = keys.iterator(); i.hasNext(); ) {
+ String key = (String) i.next();
+ if (params.get(key) != JSONNull.getInstance())
+ newParams.put(key, params.get(key));
+ }
+ }
+ result.set(0, newParams);
+ return result;
+ }
+
+ /**
+ * Make an RPC call.
+ * @param method name of the method to call
+ * @param params dictionary of parameters to pass
+ * @param callback callback to be notified of RPC call results
+ * @return true if the call was successfully initiated
+ */
+ public boolean rpcCall(String method, JSONObject params,
+ final JsonRpcCallback callback) {
+ //GWT.log("RPC " + method, null);
+ //GWT.log("args: " + params, null);
+ JSONObject request = new JSONObject();
+ request.put("method", new JSONString(method));
+ request.put("params", processParams(params));
+ request.put("id", new JSONNumber(0));
+
+ notify.setLoading(true);
+
+ boolean success = HTTPRequest.asyncPost(url,
+ request.toString(),
+ new ResponseTextHandler() {
+ public void onCompletion(String responseText) {
+ //GWT.log("Response: " + responseText, null);
+
+ notify.setLoading(false);
+
+ JSONValue responseValue = null;
+ try {
+ responseValue = JSONParser.parse(responseText);
+ }
+ catch (JSONException exc) {
+ notify.showError(exc.toString());
+ return;
+ }
+
+ JSONObject responseObject = responseValue.isObject();
+ JSONObject error = responseObject.get("error").isObject();
+ if (error != null) {
+ callback.onError(error);
+ return;
+ }
+
+ JSONValue result = responseObject.get("result");
+ callback.onSuccess(result);
+
+ // Element error_iframe =
+ // DOM.getElementById("error_iframe");
+ // DOM.setInnerHTML(error_iframe,
+ // responseText);
+ }
+ });
+ return success;
+ }
+}
diff --git a/frontend/client/src/afeclient/client/NotifyManager.java b/frontend/client/src/afeclient/client/NotifyManager.java
new file mode 100644
index 0000000..3663101
--- /dev/null
+++ b/frontend/client/src/afeclient/client/NotifyManager.java
@@ -0,0 +1,86 @@
+package afeclient.client;
+
+import com.google.gwt.user.client.ui.Label;
+import com.google.gwt.user.client.ui.PopupPanel;
+
+/**
+ * A singleton class to manage popup notifications, including error messages and
+ * the "loading..." box.
+ */
+public class NotifyManager {
+ // singleton
+ public static final NotifyManager theInstance = new NotifyManager();
+
+ class NotifyBox {
+ protected PopupPanel panel;
+ protected Label message = new Label();
+
+ public NotifyBox(boolean autoHide) {
+ message.setStyleName("notify");
+ panel = new PopupPanel(autoHide);
+ panel.add(message);
+ }
+
+ public void addStyle(String style) {
+ message.addStyleName(style);
+ }
+
+ public void hide() {
+ panel.hide();
+ }
+
+ public void show() {
+ panel.setPopupPosition(0, 0);
+ panel.show();
+ }
+
+ public void showMessage(String messageString) {
+ message.setText(messageString);
+ show();
+ }
+ }
+
+ protected NotifyBox errorNotify = new NotifyBox(true);
+ protected NotifyBox messageNotify = new NotifyBox(true);
+ protected NotifyBox loadingNotify = new NotifyBox(false);
+
+ private NotifyManager() {
+ errorNotify.addStyle("error");
+ }
+
+ /**
+ * Should be called a page loading time.
+ */
+ public void initialize() {
+ errorNotify.hide();
+ messageNotify.hide();
+ }
+
+ public static NotifyManager getInstance() {
+ return theInstance;
+ }
+
+ /**
+ * Show an error message.
+ */
+ public void showError(String error) {
+ errorNotify.showMessage(error);
+ }
+
+ /**
+ * Show a notification message.
+ */
+ public void showMessage(String message) {
+ messageNotify.showMessage(message);
+ }
+
+ /**
+ * Set whether the loading box is displayed or not.
+ */
+ public void setLoading(boolean visible) {
+ if (visible)
+ loadingNotify.showMessage("Loading...");
+ else
+ loadingNotify.hide();
+ }
+}
diff --git a/frontend/client/src/afeclient/client/Paginator.java b/frontend/client/src/afeclient/client/Paginator.java
new file mode 100644
index 0000000..6b40c1f
--- /dev/null
+++ b/frontend/client/src/afeclient/client/Paginator.java
@@ -0,0 +1,171 @@
+package afeclient.client;
+
+import com.google.gwt.user.client.ui.ClickListener;
+import com.google.gwt.user.client.ui.Composite;
+import com.google.gwt.user.client.ui.FlowPanel;
+import com.google.gwt.user.client.ui.HorizontalPanel;
+import com.google.gwt.user.client.ui.Hyperlink;
+import com.google.gwt.user.client.ui.Label;
+import com.google.gwt.user.client.ui.Panel;
+import com.google.gwt.user.client.ui.Widget;
+
+/**
+ * A widget to faciliate pagination of tables. Shows currently displayed rows,
+ * total row count, and buttons for moving among pages.
+ */
+public class Paginator extends Composite {
+
+ public interface PaginatorCallback {
+ public void doRequest(int start);
+ }
+
+ class LinkWithDisable extends Composite {
+ protected Panel panel = new FlowPanel();
+ protected Label label;
+ protected Hyperlink link;
+
+ public LinkWithDisable(String text) {
+ label = new Label(text);
+ link = new SimpleHyperlink(text);
+ panel.add(link);
+ panel.add(label);
+ link.setStyleName("paginator-link");
+ label.setStyleName("paginator-link");
+ initWidget(panel);
+ }
+
+ public void setEnabled(boolean enabled) {
+ link.setVisible(enabled);
+ label.setVisible(!enabled);
+ }
+
+ public void addClickListener(ClickListener listener) {
+ link.addClickListener(listener);
+ }
+
+ public void removeClickListener(ClickListener listener) {
+ link.removeClickListener(listener);
+ }
+ }
+
+ protected int resultsPerPage, numTotalResults;
+ protected PaginatorCallback callback;
+ protected int currentStart = 0;
+
+ protected HorizontalPanel mainPanel = new HorizontalPanel();
+ protected LinkWithDisable nextControl, prevControl,
+ firstControl, lastControl;
+ protected Label statusLabel = new Label();
+
+ public Paginator(int resultsPerPage, PaginatorCallback callback) {
+ this.resultsPerPage = resultsPerPage;
+ this.callback = callback;
+
+ prevControl = new LinkWithDisable("< Previous");
+ prevControl.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ currentStart -= Paginator.this.resultsPerPage;
+ Paginator.this.callback.doRequest(currentStart);
+ update();
+ }
+ });
+ nextControl = new LinkWithDisable("Next >");
+ nextControl.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ currentStart += Paginator.this.resultsPerPage;
+ Paginator.this.callback.doRequest(currentStart);
+ update();
+ }
+ });
+ firstControl = new LinkWithDisable("<< First");
+ firstControl.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ currentStart = 0;
+ Paginator.this.callback.doRequest(currentStart);
+ update();
+ }
+ });
+ lastControl = new LinkWithDisable("Last >>");
+ lastControl.addClickListener(new ClickListener() {
+ public void onClick(Widget sender) {
+ currentStart = getLastPageStart();
+ Paginator.this.callback.doRequest(currentStart);
+ update();
+ }
+ });
+
+ statusLabel.setWidth("10em");
+ statusLabel.setHorizontalAlignment(Label.ALIGN_CENTER);
+
+ mainPanel.setVerticalAlignment(HorizontalPanel.ALIGN_MIDDLE);
+ mainPanel.add(firstControl);
+ mainPanel.add(prevControl);
+ mainPanel.add(statusLabel);
+ mainPanel.add(nextControl);
+ mainPanel.add(lastControl);
+
+ initWidget(mainPanel);
+ }
+
+ /**
+ * Get the current starting row index.
+ */
+ public int getStart() {
+ return currentStart;
+ }
+
+ /**
+ * Get the current ending row index (one past the last currently displayed
+ * row).
+ */
+ public int getEnd() {
+ int end = currentStart + resultsPerPage;
+ if (end < numTotalResults)
+ return end;
+ return numTotalResults;
+ }
+
+ /**
+ * Get the size of each page.
+ */
+ public int getResultsPerPage() {
+ return resultsPerPage;
+ }
+
+ /**
+ * Set the total number of results in the current working set.
+ */
+ public void setNumTotalResults(int numResults) {
+ this.numTotalResults = numResults;
+ if (currentStart >= numResults)
+ currentStart = getLastPageStart();
+ update();
+ }
+
+ /**
+ * Set the current starting index.
+ */
+ public void setStart(int start) {
+ this.currentStart = start;
+ update();
+ }
+
+ protected int getLastPageStart() {
+ // compute start of last page using truncation
+ return ((numTotalResults - 1) / resultsPerPage) * resultsPerPage;
+ }
+
+ protected void update() {
+ boolean prevEnabled = !(currentStart == 0);
+ boolean nextEnabled = currentStart + resultsPerPage < numTotalResults;
+ firstControl.setEnabled(prevEnabled);
+ prevControl.setEnabled(prevEnabled);
+ nextControl.setEnabled(nextEnabled);
+ lastControl.setEnabled(nextEnabled);
+ int displayStart = getStart() + 1;
+ if(numTotalResults == 0)
+ displayStart = 0;
+ statusLabel.setText(displayStart + "-" + getEnd() +
+ " of " + numTotalResults);
+ }
+}
diff --git a/frontend/client/src/afeclient/client/SimpleCallback.java b/frontend/client/src/afeclient/client/SimpleCallback.java
new file mode 100644
index 0000000..0f51bab
--- /dev/null
+++ b/frontend/client/src/afeclient/client/SimpleCallback.java
@@ -0,0 +1,5 @@
+package afeclient.client;
+
+public interface SimpleCallback {
+ public void doCallback();
+}
diff --git a/frontend/client/src/afeclient/client/SimpleHyperlink.java b/frontend/client/src/afeclient/client/SimpleHyperlink.java
new file mode 100644
index 0000000..8b9c74b
--- /dev/null
+++ b/frontend/client/src/afeclient/client/SimpleHyperlink.java
@@ -0,0 +1,43 @@
+package afeclient.client;
+
+import com.google.gwt.user.client.DOM;
+import com.google.gwt.user.client.Event;
+import com.google.gwt.user.client.ui.ClickListener;
+import com.google.gwt.user.client.ui.ClickListenerCollection;
+import com.google.gwt.user.client.ui.Hyperlink;
+
+/**
+ * Hyperlink widget that doesn't mess with browser history. Most of this code
+ * is copied from gwt.user.client.ui.Hyperlink; unfortunately, due to the way
+ * that class is built, we can't get rid of it.
+ *
+ */
+public class SimpleHyperlink extends Hyperlink {
+ private ClickListenerCollection clickListeners;
+
+ public SimpleHyperlink(String text) {
+ super(text, "");
+ }
+
+ public void onBrowserEvent(Event event) {
+ if (DOM.eventGetType(event) == Event.ONCLICK) {
+ if (clickListeners != null) {
+ clickListeners.fireClick(this);
+ }
+ DOM.eventPreventDefault(event);
+ }
+ }
+
+ public void addClickListener(ClickListener listener) {
+ if (clickListeners == null) {
+ clickListeners = new ClickListenerCollection();
+ }
+ clickListeners.add(listener);
+ }
+
+ public void removeClickListener(ClickListener listener) {
+ if (clickListeners != null) {
+ clickListeners.remove(listener);
+ }
+ }
+}
diff --git a/frontend/client/src/afeclient/client/SiteClassFactory.java b/frontend/client/src/afeclient/client/SiteClassFactory.java
new file mode 100644
index 0000000..a63ba5a
--- /dev/null
+++ b/frontend/client/src/afeclient/client/SiteClassFactory.java
@@ -0,0 +1,6 @@
+package afeclient.client;
+
+import afeclient.client.CreateJobView.JobCreateListener;
+
+public class SiteClassFactory extends ClassFactory {
+}
diff --git a/frontend/client/src/afeclient/client/StaticDataRepository.java b/frontend/client/src/afeclient/client/StaticDataRepository.java
new file mode 100644
index 0000000..380bec5
--- /dev/null
+++ b/frontend/client/src/afeclient/client/StaticDataRepository.java
@@ -0,0 +1,47 @@
+package afeclient.client;
+
+import com.google.gwt.json.client.JSONObject;
+import com.google.gwt.json.client.JSONValue;
+
+/**
+ * A singleton class to manage a set of static data, such as the list of users.
+ * The data will most likely be retrieved once at the beginning of program
+ * execution. Other classes can then retrieve the data from this shared
+ * storage.
+ */
+public class StaticDataRepository {
+ interface FinishedCallback {
+ public void onFinished();
+ }
+ // singleton
+ public static final StaticDataRepository theInstance = new StaticDataRepository();
+
+ protected JSONObject dataObject = null;
+
+ private StaticDataRepository() {}
+
+ public static StaticDataRepository getRepository() {
+ return theInstance;
+ }
+
+ /**
+ * Update the local copy of the static data from the server.
+ * @param finished callback to be notified once data has been retrieved
+ */
+ public void refresh(final FinishedCallback finished) {
+ JsonRpcProxy.getProxy().rpcCall("get_static_data", null,
+ new JsonRpcCallback() {
+ public void onSuccess(JSONValue result) {
+ dataObject = result.isObject();
+ finished.onFinished();
+ }
+ });
+ }
+
+ /**
+ * Get a value from the static data object.
+ */
+ public JSONValue getData(String key) {
+ return dataObject.get(key);
+ }
+}
diff --git a/frontend/client/src/afeclient/client/TabView.java b/frontend/client/src/afeclient/client/TabView.java
new file mode 100644
index 0000000..1e25410
--- /dev/null
+++ b/frontend/client/src/afeclient/client/TabView.java
@@ -0,0 +1,57 @@
+package afeclient.client;
+
+import com.google.gwt.user.client.DOM;
+import com.google.gwt.user.client.Element;
+import com.google.gwt.user.client.History;
+import com.google.gwt.user.client.ui.Composite;
+
+/**
+ * A widget to facilitate building a tab panel from elements present in the
+ * static HTML document. Each <code>TabView</code> grabs a certain HTML
+ * element, removes it from the document, and wraps it. It can then be added
+ * to a TabPanel. The <code>getTitle()</code> method retrieves a title for the
+ * tab from the "title" attribute of the HTML element. This class also supports
+ * lazy initialization of the tab by waiting until the tab is first displayed.
+ */
+public abstract class TabView extends Composite {
+ public static final String HISTORY_PREFIX = "h_";
+ protected boolean initialized = false;
+ protected String title;
+
+ public TabView() {
+ Element thisTabElement = DOM.getElementById(getElementId());
+ ElementWidget thisTab = new ElementWidget(thisTabElement);
+ thisTab.removeFromDocument();
+ initWidget(thisTab);
+ title = DOM.getElementAttribute(thisTabElement, "title");
+ }
+
+ public void ensureInitialized() {
+ if (!initialized) {
+ initialize();
+ initialized = true;
+ }
+ }
+
+ // primarily for subclasses to override
+ public void display() {
+ ensureInitialized();
+ }
+
+ public String getTitle() {
+ return title;
+ }
+
+ public void updateHistory() {
+ History.newItem(getHistoryToken());
+ }
+
+ public String getHistoryToken() {
+ return HISTORY_PREFIX + getElementId();
+ }
+
+ public void handleHistoryToken(String token) {}
+
+ public abstract void initialize();
+ public abstract String getElementId();
+}
diff --git a/frontend/client/src/afeclient/client/Utils.java b/frontend/client/src/afeclient/client/Utils.java
new file mode 100644
index 0000000..c1a0ddf
--- /dev/null
+++ b/frontend/client/src/afeclient/client/Utils.java
@@ -0,0 +1,42 @@
+package afeclient.client;
+
+import com.google.gwt.json.client.JSONArray;
+import com.google.gwt.json.client.JSONObject;
+import com.google.gwt.json.client.JSONString;
+
+import java.util.Collection;
+import java.util.Iterator;
+import java.util.Set;
+
+/**
+ * Utility methods.
+ */
+public class Utils {
+ public static final ClassFactory factory = new SiteClassFactory();
+
+ /**
+ * Converts a collection of Java <code>String</code>s into a <code>JSONArray
+ * </code> of <code>JSONString</code>s.
+ */
+ public static JSONArray stringsToJSON(Collection strings) {
+ JSONArray result = new JSONArray();
+ for(Iterator i = strings.iterator(); i.hasNext(); ) {
+ String s = (String) i.next();
+ result.set(result.size(), new JSONString(s));
+ }
+ return result;
+ }
+
+ public static String formatStatusCounts(JSONObject counts, String joinWith) {
+ String result = "";
+ Set statusSet = counts.keySet();
+ for (Iterator i = statusSet.iterator(); i.hasNext();) {
+ String status = (String) i.next();
+ int count = (int) counts.get(status).isNumber().getValue();
+ result += Integer.toString(count) + " " + status;
+ if (i.hasNext())
+ result += joinWith;
+ }
+ return result;
+ }
+}
diff --git a/frontend/client/src/afeclient/public/ClientMain.html b/frontend/client/src/afeclient/public/ClientMain.html
new file mode 100644
index 0000000..d81ef4c
--- /dev/null
+++ b/frontend/client/src/afeclient/public/ClientMain.html
@@ -0,0 +1,118 @@
+<html>
+ <head>
+ <title>Autotest Frontend</title>
+ <script language='javascript' src='afeclient.ClientMain.nocache.js'>
+ </script>
+ </head>
+
+ <body>
+ <!-- gwt history support -->
+ <iframe src="javascript:''" id="__gwt_historyFrame"
+ style="width:0;height:0;border:0"></iframe>
+
+
+ <span class="links-box" style="float: right;">
+ <b>Links</b><br>
+ <a href="server/admin">Admin interface</a><br>
+ <a href="/tko">Results database</a><br>
+ <a href="http://test.kernel.org/autotest">Documentation</a><br>
+ </span>
+ <img src="header.png" />
+ <br /><br />
+
+ <div id="tabs" class="hidden">
+ <div id="job_list" title="Job List">
+ <div id="job_control_links" class="job-control"></div>
+ <div class="job-control">
+ View jobs for user: <span id="user_list"></span>
+ </div>
+ <div id="job_pagination"></div>
+ <div id="job_table"></div>
+ </div>
+
+ <div id="view_job" title="View Job">
+ <span id="job_id_fetch" class="box-full">Fetch job by ID:
+ <span id="job_id_fetch_controls"></span>
+ </span><br><br>
+ <div id="view_title" class="title"></div><br>
+ <div id="view_data">
+ <span class="field-name">Label:</span>
+ <span id="view_label"></span><br>
+ <span class="field-name">Owner:</span>
+ <span id="view_owner"></span><br>
+ <span class="field-name">Priority:</span>
+ <span id="view_priority"></span><br>
+ <span class="field-name">Created:</span>
+ <span id="view_created"></span><br>
+ <span class="field-name">Status:</span>
+ <span id="view_status"></span>
+ <span id="view_abort"></span><br>
+
+ <span class="field-name">
+ <span id="view_control_type"></span> control file
+ (<span id="view_synch_type"></span>):
+ </span>
+ <pre><div id="view_control_file" class="box"></div></pre>
+ <span class="field-name">
+ Full results
+ <a id="results_link" target="_blank">(open in new window)</a>
+ <a id="raw_results_link" target="_blank">(raw results logs)</a><br>
+ </span>
+
+ <iframe src="javascript:''" id="results_iframe" class="results-frame">
+ </iframe><br>
+
+ <span class="field-name">Hosts</span>
+ <div id="job_hosts_table"></div>
+ </div>
+ </div>
+
+ <div id="create_job" title="Create Job">
+ <table class="form-table">
+ <tr><td class="field-name">Job name:</td>
+ <td id="create_job_name"></td><td></td></tr>
+ <tr><td class="field-name">Priority:</td>
+ <td id="create_priority"></td><td></td></tr>
+ <tr><td class="field-name">Kernel:</td>
+ <td id="create_kernel"></td><td></td></tr>
+ <tr><td class="field-name">Client tests:</td>
+ <td id="create_client_tests" colspan="2"></td></tr>
+ <tr><td class="field-name">Server tests:</td>
+ <td id="create_server_tests" colspan="2"></td></tr>
+ <tr><td colspan="3" id="create_edit_control"></td></tr>
+ <tr>
+ <td colspan="3" class="box">
+ <table><tr>
+ <td>
+ <div class="box-full">
+ Run on any <span id="create_meta_select"></span><br>
+ Number: <span id="create_meta_number"></span>
+ <span id="create_meta_button"></span>
+ </div>
+ <span class="field-name">Available hosts: </span>
+ <span class="help">(Click on a host to add it)</span>
+ <div id="create_available_table"></div>
+ <div id="create_available_controls"></div>
+ </td>
+ <td>
+ <span class="field-name">Selected hosts:</span>
+ <span class="help">(Click on a host to remove it)</span>
+ <div id="create_selected_table"></div>
+ <div id="create_selected_controls"></div>
+ </td>
+ </tr></table>
+ </td>
+ </tr>
+ <tr><td colspan="3" id="create_submit"></td></tr>
+ </table>
+ </div>
+
+ <div id="hosts" title="Hosts">
+ <div id="hosts_list"></div>
+ </div>
+ </div>
+
+ <!-- for debugging only -->
+ <div id="error_display"></div>
+ </body>
+</html>
diff --git a/frontend/client/src/afeclient/public/afeclient.css b/frontend/client/src/afeclient/public/afeclient.css
new file mode 100644
index 0000000..7534f26
--- /dev/null
+++ b/frontend/client/src/afeclient/public/afeclient.css
@@ -0,0 +1,204 @@
+body {
+ margin-top: 0px;
+}
+
+.data-table {
+ border: 2px solid #A0A0A0;
+}
+
+.data-table td {
+ margin: 0;
+ padding: 0.2em 0.4em;
+ white-space: nowrap;
+}
+
+.data-row-header {
+ background: #AACCFF;
+ font-weight: bold;
+}
+
+.data-row-header-sortable {
+ cursor: pointer;
+}
+
+.data-row-header-sortable td:hover {
+ background: #6699FF;
+}
+
+.data-row {
+ background: white;
+}
+
+.data-row-alternate {
+ background: #CCDDFF;
+}
+
+tr.data-row-clickable {
+ cursor: pointer;
+}
+
+tr.data-row-clickable:hover {
+ background: #FFFFBB;
+}
+
+.gwt-TabBar .gwt-TabBarItem {
+ background: #CCDDFF;
+ margin-right: 2px;
+ padding: 0.2em;
+ padding-left: 0.4em;
+ padding-right: 0.4em;
+ font-weight: bold;
+ cursor: pointer;
+}
+
+.gwt-TabBar .gwt-TabBarItem-selected {
+ background: #003399;
+ color: white;
+ cursor: default;
+}
+
+.gwt-TabPanelBottom {
+ border: 2px solid #6699FF;
+ padding: 5px;
+}
+
+.job-control {
+ margin-bottom: 0.5em;
+}
+
+.job-filter-link {
+ padding: 0.3em;
+ margin-right: 1em;
+}
+
+.selected-link {
+ background-color: #AACCFF;
+}
+
+.box {
+ border: 1px solid #CCDDFF;
+ padding: 0.2em;
+ margin: 0.2em;
+}
+
+.box-full {
+ background: #CCDDFF;
+ padding: 0.2em;
+ margin: 0.2em;
+}
+
+.title {
+ font-weight: bold;
+ font-size: larger;
+}
+
+#job_id_fetch {
+ padding: 0.2em 0.2em 0.4em 0.2em;
+ margin: 0.1em;
+}
+
+.form-table {
+}
+
+.form-table td {
+ vertical-align: top;
+}
+
+.gwt-CheckBox {
+ margin-right: 1em;
+}
+
+.field-name {
+ font-weight: bold;
+ white-space: nowrap;
+}
+
+.gwt-DisclosurePanel {
+ border: 1px solid gray;
+ padding: 0.2em;
+}
+
+.gwt-DisclosurePanel .header {
+ cursor: pointer;
+}
+
+.gwt-DisclosurePanel a.header {
+ text-decoration: none;
+}
+
+.gwt-DisclosurePanel .gwt-Label {
+ text-decoration: underline;
+}
+
+.filter-box {
+ margin-left: 1em;
+}
+
+.gwt-Button {
+ margin: 0.2em;
+}
+
+.results-frame {
+ width: 700px;
+ height: 500px;
+ margin-bottom: 0.2em;
+ border: 2px solid gray;
+}
+
+.gwt-DialogBox {
+ background: white;
+ border: 2px solid #AACCFF;
+ width: 30em;
+ text-align: center;
+ cursor: default;
+}
+
+.gwt-DialogBox .Caption {
+ background: #AACCFF;
+ font-weight: bold;
+ text-align: center;
+}
+
+.notify {
+ background: #CCDDFF;
+ padding: 0.2em;
+ border: 1px solid black;
+ position: fixed;
+}
+
+.error {
+ color: red;
+}
+
+.gwt-TextArea {
+ border: 1px solid black;
+}
+
+.gwt-TextArea-readonly {
+ background: #F0F0F0;
+}
+
+.help {
+ font-style: italic;
+ text-align: left;
+ width: 100%;
+}
+
+.hidden {
+ display: none;
+}
+
+.paginator-link {
+ padding: 0 0.2em;
+}
+
+.links-box {
+ border: 2px solid #AACCFF;
+ background: #CCDDFF;
+ font-size: larger;
+ padding: 0.2em;
+}
+
+.extra-space-left {
+ margin-left: 1em;
+}
diff --git a/frontend/client/src/afeclient/public/arrow_down.png b/frontend/client/src/afeclient/public/arrow_down.png
new file mode 100644
index 0000000..0b108ae
--- /dev/null
+++ b/frontend/client/src/afeclient/public/arrow_down.png
Binary files differ
diff --git a/frontend/client/src/afeclient/public/arrow_up.png b/frontend/client/src/afeclient/public/arrow_up.png
new file mode 100644
index 0000000..8a3e8b5
--- /dev/null
+++ b/frontend/client/src/afeclient/public/arrow_up.png
Binary files differ
diff --git a/frontend/client/src/afeclient/public/header.png b/frontend/client/src/afeclient/public/header.png
new file mode 100644
index 0000000..3611e3d
--- /dev/null
+++ b/frontend/client/src/afeclient/public/header.png
Binary files differ
diff --git a/frontend/manage.py b/frontend/manage.py
new file mode 100755
index 0000000..e70d6d2
--- /dev/null
+++ b/frontend/manage.py
@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+from django.core.management import execute_manager
+try:
+ import settings # Assumed to be in the same directory.
+except ImportError:
+ import sys
+ sys.stderr.write("Error: Can't find the file 'settings.py' in the directory containing %r. It appears you've customized things.\nYou'll have to run django-admin.py, passing it your settings module.\n(If the file settings.py does indeed exist, it's causing an ImportError somehow.)\n" % __file__)
+ sys.exit(1)
+
+if __name__ == "__main__":
+ execute_manager(settings)
diff --git a/frontend/migrations/001_initial_db.py b/frontend/migrations/001_initial_db.py
new file mode 100644
index 0000000..0773ea0
--- /dev/null
+++ b/frontend/migrations/001_initial_db.py
@@ -0,0 +1,175 @@
+import os
+
+required_tables = ('acl_groups', 'acl_groups_hosts', 'acl_groups_users',
+ 'autotests', 'host_queue_entries', 'hosts', 'hosts_labels',
+ 'ineligible_host_queues', 'jobs', 'labels', 'users')
+
+
+def migrate_up(manager):
+ assert not manager.check_migrate_table_exists()
+ manager.execute("SHOW TABLES")
+ tables = [row[0] for row in manager.cursor.fetchall()]
+ db_initialized = True
+ for table in required_tables:
+ if table not in tables:
+ db_initialized = False
+ break
+ if not db_initialized:
+ response = raw_input(
+ 'Your autotest_web database does not appear to be '
+ 'initialized. Do you want to recreate it (this will '
+ 'result in loss of any existing data) (yes/No)? ')
+ if response != 'yes':
+ raise Exception('User has chosen to abort migration')
+
+ manager.execute_script(CREATE_DB_SQL)
+
+ manager.create_migrate_table()
+
+
+CREATE_DB_SQL = """\
+--
+-- Table structure for table `acl_groups`
+--
+
+DROP TABLE IF EXISTS `acl_groups`;
+CREATE TABLE `acl_groups` (
+ `id` int(11) NOT NULL auto_increment,
+ `name` varchar(255) default NULL,
+ `description` varchar(255) default NULL,
+ PRIMARY KEY (`id`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+--
+-- Table structure for table `acl_groups_hosts`
+--
+
+DROP TABLE IF EXISTS `acl_groups_hosts`;
+CREATE TABLE `acl_groups_hosts` (
+ `acl_group_id` int(11) default NULL,
+ `host_id` int(11) default NULL
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+--
+-- Table structure for table `acl_groups_users`
+--
+
+DROP TABLE IF EXISTS `acl_groups_users`;
+CREATE TABLE `acl_groups_users` (
+ `acl_group_id` int(11) default NULL,
+ `user_id` int(11) default NULL
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+--
+-- Table structure for table `autotests`
+--
+
+DROP TABLE IF EXISTS `autotests`;
+CREATE TABLE `autotests` (
+ `id` int(11) NOT NULL auto_increment,
+ `name` varchar(255) default NULL,
+ `test_class` varchar(255) default NULL,
+ `params` varchar(255) default NULL,
+ `description` text,
+ `test_type` int(11) default NULL,
+ `path` varchar(255) default NULL,
+ PRIMARY KEY (`id`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+DROP TABLE IF EXISTS `host_queue_entries`;
+CREATE TABLE `host_queue_entries` (
+ `id` int(11) NOT NULL auto_increment,
+ `job_id` int(11) default NULL,
+ `host_id` int(11) default NULL,
+ `priority` int(11) default NULL,
+ `status` varchar(255) default NULL,
+ `created_on` datetime default NULL,
+ `meta_host` int(11) default NULL,
+ `active` tinyint(1) default '0',
+ `complete` tinyint(1) default '0',
+ PRIMARY KEY (`id`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+--
+-- Table structure for table `hosts`
+--
+
+DROP TABLE IF EXISTS `hosts`;
+CREATE TABLE `hosts` (
+ `id` int(11) NOT NULL auto_increment,
+ `hostname` varchar(255) default NULL,
+ `locked` tinyint(1) default '0',
+ `synch_id` int(11) default NULL,
+ `status` varchar(255) default NULL,
+ PRIMARY KEY (`id`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+--
+-- Table structure for table `hosts_labels`
+--
+
+DROP TABLE IF EXISTS `hosts_labels`;
+CREATE TABLE `hosts_labels` (
+ `host_id` int(11) default NULL,
+ `label_id` int(11) default NULL
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+--
+-- Table structure for table `ineligible_host_queues`
+--
+
+DROP TABLE IF EXISTS `ineligible_host_queues`;
+CREATE TABLE `ineligible_host_queues` (
+ `id` int(11) NOT NULL auto_increment,
+ `job_id` int(11) default NULL,
+ `host_id` int(11) default NULL,
+ PRIMARY KEY (`id`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+--
+-- Table structure for table `jobs`
+--
+
+DROP TABLE IF EXISTS `jobs`;
+CREATE TABLE `jobs` (
+ `id` int(11) NOT NULL auto_increment,
+ `owner` varchar(255) default NULL,
+ `name` varchar(255) default NULL,
+ `priority` int(11) default NULL,
+ `control_file` text,
+ `control_type` int(11) default NULL,
+ `kernel_url` varchar(255) default NULL,
+ `status` varchar(255) default NULL,
+ `created_on` datetime default NULL,
+ `submitted_on` datetime default NULL,
+ `synch_type` int(11) default NULL,
+ `synch_count` int(11) default NULL,
+ `synchronizing` tinyint(1) default NULL,
+ PRIMARY KEY (`id`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+--
+-- Table structure for table `labels`
+--
+
+DROP TABLE IF EXISTS `labels`;
+CREATE TABLE `labels` (
+ `id` int(11) NOT NULL auto_increment,
+ `name` varchar(255) default NULL,
+ `kernel_config` varchar(255) default NULL,
+ `platform` tinyint(1) default '0',
+ PRIMARY KEY (`id`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+--
+-- Table structure for table `users`
+--
+
+DROP TABLE IF EXISTS `users`;
+CREATE TABLE `users` (
+ `id` int(11) NOT NULL auto_increment,
+ `login` varchar(255) default NULL,
+ `access_level` int(11) default '0',
+ PRIMARY KEY (`id`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+"""
diff --git a/frontend/migrations/002_cleanup_fields.py b/frontend/migrations/002_cleanup_fields.py
new file mode 100644
index 0000000..ccd8c1c
--- /dev/null
+++ b/frontend/migrations/002_cleanup_fields.py
@@ -0,0 +1,5 @@
+def migrate_up(manager):
+ manager.execute('ALTER TABLE autotests DROP params')
+ manager.execute('ALTER TABLE jobs DROP kernel_url, DROP status, '
+ 'DROP submitted_on')
+ manager.execute('ALTER TABLE host_queue_entries DROP created_on')
diff --git a/frontend/settings.py b/frontend/settings.py
new file mode 100644
index 0000000..f4adfa5
--- /dev/null
+++ b/frontend/settings.py
@@ -0,0 +1,97 @@
+# Django settings for frontend project.
+
+DEBUG = True
+TEMPLATE_DEBUG = DEBUG
+
+FULL_ADMIN = False
+
+ADMINS = (
+ # ('Your Name', 'your_email@domain.com'),
+)
+
+MANAGERS = ADMINS
+
+DATABASE_ENGINE = 'mysql_old' # 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'ado_mssql'.
+DATABASE_NAME = 'autotest_web' # Or path to database file if using sqlite3.
+DATABASE_USER = 'autotest' # Not used with sqlite3.
+DATABASE_PASSWORD = 'password' # Not used with sqlite3.
+DATABASE_HOST = '' # Set to empty string for localhost. Not used with sqlite3.
+DATABASE_PORT = '' # Set to empty string for default. Not used with sqlite3.
+
+# prefix applied to all URLs - useful if requests are coming through apache,
+# and you need this app to coexist with others
+URL_PREFIX = 'afe/server/'
+
+# Local time zone for this installation. Choices can be found here:
+# http://www.postgresql.org/docs/8.1/static/datetime-keywords.html#DATETIME-TIMEZONE-SET-TABLE
+# although not all variations may be possible on all operating systems.
+# If running in a Windows environment this must be set to the same as your
+# system time zone.
+TIME_ZONE = 'America/Los_Angeles'
+
+# Language code for this installation. All choices can be found here:
+# http://www.w3.org/TR/REC-html40/struct/dirlang.html#langcodes
+# http://blogs.law.harvard.edu/tech/stories/storyReader$15
+LANGUAGE_CODE = 'en-us'
+
+SITE_ID = 1
+
+# If you set this to False, Django will make some optimizations so as not
+# to load the internationalization machinery.
+USE_I18N = True
+
+# Absolute path to the directory that holds media.
+# Example: "/home/media/media.lawrence.com/"
+MEDIA_ROOT = ''
+
+# URL that handles the media served from MEDIA_ROOT.
+# Example: "http://media.lawrence.com"
+MEDIA_URL = ''
+
+# URL prefix for admin media -- CSS, JavaScript and images. Make sure to use a
+# trailing slash.
+# Examples: "http://foo.com/media/", "/media/".
+ADMIN_MEDIA_PREFIX = '/media/'
+
+# Make this unique, and don't share it with anybody.
+SECRET_KEY = 'pn-t15u(epetamdflb%dqaaxw+5u&2#0u-jah70w1l*_9*)=n7'
+
+# List of callables that know how to import templates from various sources.
+TEMPLATE_LOADERS = (
+ 'django.template.loaders.filesystem.load_template_source',
+ 'django.template.loaders.app_directories.load_template_source',
+# 'django.template.loaders.eggs.load_template_source',
+)
+
+MIDDLEWARE_CLASSES = (
+ 'django.middleware.common.CommonMiddleware',
+ 'django.contrib.sessions.middleware.SessionMiddleware',
+ 'frontend.apache_auth.ApacheAuthMiddleware',
+ 'django.contrib.auth.middleware.AuthenticationMiddleware',
+ 'django.middleware.doc.XViewMiddleware',
+)
+
+ROOT_URLCONF = 'frontend.urls'
+
+TEMPLATE_DIRS = (
+ # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates".
+ # Always use forward slashes, even on Windows.
+ # Don't forget to use absolute paths, not relative paths.
+)
+
+INSTALLED_APPS = (
+ 'frontend.afe',
+ 'django.contrib.admin',
+ 'django.contrib.auth',
+ 'django.contrib.contenttypes',
+ 'django.contrib.sessions',
+ 'django.contrib.sites',
+)
+
+# you'll need to grant your database user permissions on database
+# test_<normal database name> to run tests
+TEST_RUNNER = 'frontend.afe.test.run_tests'
+
+AUTHENTICATION_BACKENDS = (
+ 'frontend.apache_auth.SimpleAuthBackend',
+)
diff --git a/frontend/urls.py b/frontend/urls.py
new file mode 100644
index 0000000..b66c8d4
--- /dev/null
+++ b/frontend/urls.py
@@ -0,0 +1,9 @@
+from django.conf.urls.defaults import *
+from django.conf import settings
+
+RE_PREFIX = '^' + settings.URL_PREFIX
+
+urlpatterns = patterns('',
+ (RE_PREFIX + r'admin/', include('django.contrib.admin.urls')),
+ (RE_PREFIX, include('frontend.afe.urls')),
+)
