Control characters in FilteredRelation column aliases could be used for
SQL injection attacks. This affected QuerySet.annotate(), aggregate(),
extra(), values(), values_list(), and alias() when using dictionary
expansion with **kwargs.
Thanks Solomon Kebede for the report, and Simon Charette, Jacob Walls,
and Natalia Bidart for reviews.
Regression in b8e5a8a9a2.
Refs #36404.
The replace_expressions method was innapropriately dealing with falsey
but not None source expressions causing them to also be potentially
evaluated when __bool__ was invoked (e.g. QuerySet.__bool__ evaluates
the queryset).
The changes introduced in b8e5a8a9a2, which were to deal with a similar
issue, surfaced the problem as aggregation over an annotated queryset
requires an inlining (or pushdown) of aggregate references which is
achieved through replace_expressions.
In cases where an empty Q object was provided as an aggregate filter,
such as when the admin facetting feature was used as reported, it would
wrongly be turned into None, instead of an empty WhereNode, causing a
crash at aggregate filter compilation.
Note that the crash signature differed depending on whether or not the
backend natively supports aggregate filtering
(supports_aggregate_filter_clause) as the fallback, which makes use
Case / When expressions, would result in a TypeError instead of a
NoneType AttributeError.
Thanks Rafael Urben for the report, Antoliny and Youngkwang Yang for
the triage.
Thanks Simon Charette for the guidance and review. Thanks Tim Schilling for the
documentation review. Thanks David Wobrock for investigation and solution proposals.
This moves the behaviors of `order_by` used in Postgres aggregates into
the `Aggregate` class. This allows for creating aggregate functions that
support this behavior across all database engines. This is shown by
moving the `StringAgg` class into the shared `aggregates` module and
adding support for all databases. The Postgres `StringAgg` class is now
a thin wrapper on the new shared `StringAgg` class.
Thank you Simon Charette for the review.
Regression in b0ad41198b.
Refs #34013. The initial logic did not consider that annotation aliases
can include lookup or transform separators.
Thanks Gert Van Gool for the report and Mariusz Felisiak for the review.
Refactored the filter and order_by expressions in the Aggregate class to
return a list of Expression (or None) values, ensuring that the list
item is always available and represents the filter expression.
For the PostgreSQL OrderableAggMixin, the returned list will always
include the filter and the order_by value as the last two elements.
Lastly, emtpy Q objects passed directly into aggregate objects using
Aggregate.filter in admin facets are filtered out when resolving the
expression to avoid errors in get_refs().
Thanks Simon Charette for the review.
Adjustments made to solve_lookup_type to defer the resolving of
references for summarized aggregates failed to account for similar
requirements for lookup values which can also reference annotations
through Aggregate.filter.
Regression in b181cae2e3.
Refs #25307.
Thanks Sergey Nesterenko for the report.
On SQLite < 3.39, this forces a GROUP BY clause with a HAVING clause
when no grouping is specified.
Co-authored-by: Simon Charette <charette.s@gmail.com>
Regression in b7b28c7c18.
Refs #31377.
Thanks Shai Berger for the report and reviews.
test_aggregation_subquery_annotation_values_collision() has been
updated as queries that are explicitly grouped by a subquery should
always be grouped by it and not its outer columns even if its alias
collides with referenced table columns. This was not possible to
accomplish at the time 10866a10 landed because we didn't have compiler
level handling of colliding aliases.
By avoiding to annotate aggregations meant to be possibly pushed to an
outer query until their references are resolved it is possible to
aggregate over a query with the same alias.
Even if #34176 is a convoluted case to support, this refactor seems
worth it given the reduction in complexity it brings with regards to
annotation removal when performing a subquery pushdown.
Just like when using .annotate(), the .alias() method will generate the
necessary JOINs to resolve the alias even if not selected.
Since these JOINs could be multi-valued non-selected aggregates must be
considered to require subquery wrapping as a GROUP BY is required to
combine duplicated tuples from the base table.
Regression in 59bea9efd2.
This ensures implicit grouping from aggregate function annotations
groups by uncollapsed selected aliases if supported.
The feature is disabled on Oracle because it doesn't support it.
This required moving the combined queries slicing logic to the compiler
in order to allow Query.exists() to be called at expression resolving
time.
It allowed for Query.exists() to be called at Exists() initialization
time and thus ensured that get_group_by_cols() was operating on the
terminal representation of the query that only has a single column
selected.
A more in-depth solution is likely to make sure that we always GROUP BY
selected annotations or revisit how we use Query.exists() in the Exists
expression but that requires extra work that isn't suitable for a
backport.
Regression in e5a92d400a.
Thanks Fernando Flores Villaça for the report.
Thanks Splunk team: Preston Elder, Jacob Davis, Jacob Moore,
Matt Hanson, David Briggs, and a security researcher: Danylo Dmytriiev
(DDV_UA) for the report.
This includes refactoring of CombinedExpression._resolve_output_field()
so it no longer uses the behavior inherited from Expression of guessing
same output type if argument types match, and instead we explicitly
define the output type of all supported operations.
This also makes nonsensical operations involving dates
(e.g. date + date) raise a FieldError, and adds support for
automatically inferring output_field for cases such as:
* date - date
* date + duration
* date - duration
* time + duration
* time - time
