add support for showing only the latest test run per cell in spreadsheet mode. this involved some extensive refactorings on both the client and server which then made the actual change quite simple.
Refactorings:
-refactored group querying code in TempManager to be much more general (it's no longer oriented around fetching COUNT(*)). it also now returns dicts instead of raw rows.
-refactored much of the logic from the group-related RPCs into GroupDataProcessor, which now handles almost all the work, again in a more general fashion
-made group RPCs always return a sample test index with each group; this allows the client to drill down to single-test groups without having to make an extra query to get the test index (it also makes the latest test feature easier to implement)
-refactor TestSet to support a new getTestIndex method for single test sets
-added new SingleTestSet class for single test sets, and made TestSet creation code use this instead of ConditionTestSet
-made ConditionTestSet always be for multiple tests
-changed drilldown code to use TestSet.getTestIndex() to avoid making an extra RPC call (in both spreadsheet and table views)
-got rid fo TkoUtils.getTestId, which is no longer needed since test IDs are passed down with the groups
New features:
-added get_latest_tests RPC to get the latest test per group, but still return information in the same format as get_status_counts
-added "show only latest test per cell" to spreadsheet view, made it control the RPC that gets called, and added history support for it
About the get_latest_tests RPC - it uses two rather simple SQL queries and some processing in Python. I tried six different ways of computing this information, some using a single SQL query to do everything and some doing everything in Python, and this approach was by far the fastest.
Signed-off-by: Steve Howard <showard@google.com>
git-svn-id: http://test.kernel.org/svn/autotest/trunk@2216 592f7852-d20e-0410-864c-8624ca9c26a4
diff --git a/new_tko/tko/models.py b/new_tko/tko/models.py
index 066e965..51d40a7 100644
--- a/new_tko/tko/models.py
+++ b/new_tko/tko/models.py
@@ -15,20 +15,23 @@
return [self._get_key_unless_is_function(field) for field in fields]
+ @staticmethod
+ def _get_field_alias(field_sql):
+ field_sql = field_sql.lower()
+ if ' as ' in field_sql:
+ return field_sql.rsplit(' as ', 1)[1]
+ return field_sql
+
+
def _get_group_query_sql(self, query, group_by, extra_select_fields):
group_fields = self._get_field_names(group_by)
- if query._distinct:
- pk_field = self._get_key_on_this_table(self.model._meta.pk.name)
- count_sql = 'COUNT(DISTINCT %s)' % pk_field
- else:
- count_sql = 'COUNT(1)'
- select_fields = (group_fields +
- [count_sql + ' AS ' + self._GROUP_COUNT_NAME] +
- extra_select_fields)
+ select_fields = group_fields + extra_select_fields
- # add the count field to the query selects, so they'll be sortable and
+ # add the extra fields to the query selects, so they'll be sortable and
# Django won't mess with any of them
- query._select[self._GROUP_COUNT_NAME] = count_sql
+ for field_sql in extra_select_fields:
+ field_name = self._get_field_alias(field_sql)
+ query._select[field_name] = field_sql
_, where, params = query._get_sql_clause()
@@ -44,22 +47,39 @@
return ('SELECT ' + ', '.join(select_fields) + where), params
- def get_group_counts(self, query, group_by, extra_select_fields=[]):
+ def execute_group_query(self, query, group_by, extra_select_fields=[]):
"""
- Performs the given query grouped by the fields in group_by. Returns a
- list of rows, where each row is a list containing the value of each
- field in group_by, followed by the group count.
+ Performs the given query grouped by the fields in group_by with the
+ given extra select fields added. Usually, the extra fields will use
+ group aggregation functions. Returns a list of dicts, where each dict
+ corresponds to single row and contains a key for each grouped field as
+ well as all of the extra select fields.
"""
sql, params = self._get_group_query_sql(query, group_by,
extra_select_fields)
cursor = readonly_connection.connection.cursor()
- num_rows = cursor.execute(sql, params)
- return cursor.fetchall()
+ cursor.execute(sql, params)
+ field_names = [column_info[0] for column_info in cursor.description]
+ row_dicts = [dict(zip(field_names, row)) for row in cursor.fetchall()]
+ return row_dicts
+
+
+ def get_count_sql(self, query):
+ """
+ Get the SQL to properly select a per-group count of unique matches for
+ a grouped query.
+ """
+ if query._distinct:
+ pk_field = self._get_key_on_this_table(self.model._meta.pk.name)
+ count_sql = 'COUNT(DISTINCT %s)' % pk_field
+ else:
+ count_sql = 'COUNT(1)'
+ return count_sql + ' AS ' + self._GROUP_COUNT_NAME
def _get_num_groups_sql(self, query, group_by):
group_fields = self._get_field_names(group_by)
- query._order_by = None # this can mess up the query is isn't needed
+ query._order_by = None # this can mess up the query and isn't needed
_, where, params = query._get_sql_clause()
return ('SELECT COUNT(DISTINCT %s) %s' % (','.join(group_fields),
where),