批量选择多对多和一对多字段(有助于避免N+1查询问题)
django-batch-select的Python项目详细描述
django批处理选择的思想是提供 django的select_related功能。因此,这是另一个方便 避免“n+1查询问题”的工具。
select_related对于最小化所需查询的数量很方便 在某些情况下制造。然而,这只是通常的 预先选择ForeignKey关系。
批处理选择对于预先选择ManyToManyField关系和 反向ForeignKey关系。
它的工作原理是在QuerySet之后执行一个额外的sql查询 被评估为在要求的额外字段中缝合。这需要 添加自定义Manager,然后返回自定义QuerySet 附加额外的方法。
示例用法
假设我们有如下定义的模型:
from batch_select.models import BatchManager class Tag(models.Model): name = models.CharField(max_length=32) class Section(models.Model): name = models.CharField(max_length=32) objects = BatchManager() class Entry(models.Model): title = models.CharField(max_length=255) section = models.ForeignKey(Section, blank=True, null=True) tags = models.ManyToManyField(Tag) objects = BatchManager()
我还将定义一个helper函数来显示生成的sql查询:
from django import db def show_queries(): for query in db.connection.queries: print query["sql"] db.reset_queries()
下面是几个示例(使用生成的sql查询):
>>> Entry.objects.batch_select('tags').all() [] >>> show_queries() # no results, so no 2nd query SELECT "batch_select_entry"."id", "batch_select_entry"."title", "batch_select_entry"."section_id" FROM "batch_select_entry" >>> Entry.objects.create() >>> Entry.objects.create() >>> tag1 = Tag.objects.create(name='tag1') >>> tag2 = Tag.objects.create(name='tag2') >>> db.reset_queries() >>> entries = Entry.objects.batch_select('tags').all() >>> entry = entries[0] >>> print entry.tags_all [] >>> show_queries() SELECT "batch_select_entry"."id", "batch_select_entry"."title", "batch_select_entry"."section_id" FROM "batch_select_entry" LIMIT 1 SELECT (`batch_select_entry_tags`.`entry_id`) AS "entry_id", "batch_select_tag"."id", "batch_select_tag"."name" FROM "batch_select_tag" INNER JOIN "batch_select_entry_tags" ON ("batch_select_tag"."id" = "batch_select_entry_tags"."tag_id") WHERE "batch_select_entry_tags".entry_id IN (1) >>> entry.tags.add(tag1) >>> db.reset_queries() >>> entries = Entry.objects.batch_select('tags').all() >>> entry = entries[0] >>> print entry.tags_all [<Tag: Tag object>] >>> show_queries() SELECT "batch_select_entry"."id", "batch_select_entry"."title", "batch_select_entry"."section_id" FROM "batch_select_entry" LIMIT 1 SELECT (`batch_select_entry_tags`.`entry_id`) AS "entry_id", "batch_select_tag"."id", "batch_select_tag"."name" FROM "batch_select_tag" INNER JOIN "batch_select_entry_tags" ON ("batch_select_tag"."id" = "batch_select_entry_tags"."tag_id") WHERE "batch_select_entry_tags".entry_id IN (1) >>> entries = Entry.objects.batch_select('tags').all() >>> for entry in entries: .... print entry.tags_all .... [<Tag: Tag object>] [] >>> show_queries() SELECT "batch_select_entry"."id", "batch_select_entry"."title", "batch_select_entry"."section_id" FROM "batch_select_entry" SELECT (`batch_select_entry_tags`.`entry_id`) AS "entry_id", "batch_select_tag"."id", "batch_select_tag"."name" FROM "batch_select_tag" INNER JOIN "batch_select_entry_tags" ON ("batch_select_tag"."id" = "batch_select_entry_tags"."tag_id") WHERE "batch_select_entry_tags".entry_id IN (1, 2)
在不使用批处理选择的情况下重新运行相同的last for循环 生成三个查询而不是两个(n+1查询):
>>> entries = Entry.objects.all() >>> for entry in entries: .... print entry.tags.all() .... [<Tag: Tag object>] [] >>> show_queries() SELECT "batch_select_entry"."id", "batch_select_entry"."title", "batch_select_entry"."section_id" FROM "batch_select_entry" SELECT "batch_select_tag"."id", "batch_select_tag"."name" FROM "batch_select_tag" INNER JOIN "batch_select_entry_tags" ON ("batch_select_tag"."id" = "batch_select_entry_tags"."tag_id") WHERE "batch_select_entry_tags"."entry_id" = 1 SELECT "batch_select_tag"."id", "batch_select_tag"."name" FROM "batch_select_tag" INNER JOIN "batch_select_entry_tags" ON ("batch_select_tag"."id" = "batch_select_entry_tags"."tag_id") WHERE "batch_select_entry_tags"."entry_id" = 2
这也适用于反向外键。例如,我们可以 属于每个部分的条目:
>>> section1 = Section.objects.create(name='section1') >>> section2 = Section.objects.create(name='section2') >>> Entry.objects.create(section=section1) >>> Entry.objects.create(section=section1) >>> Entry.objects.create(section=section2) >>> db.reset_queries() >>> Section.objects.batch_select('entry_set') [<Section: Section object>, <Section: Section object>] >>> show_queries() SELECT "batch_select_section"."id", "batch_select_section"."name" FROM "batch_select_section" LIMIT 21 SELECT ("batch_select_entry"."section_id") AS "__section_id", "batch_select_entry"."id", "batch_select_entry"."title", "batch_select_entry"."section_id", "batch_select_entry"."location_id" FROM "batch_select_entry" WHERE "batch_select_entry"."section_id" IN (1, 2)
查询中的每个section对象都有一个entry\u set\u all字段 包含相关条目的。
您需要通过批选择外键的“相关名称”, 在本例中为“条目集”。nb默认情况下,外文的相关名称 键实际上不包含_set后缀,因此您可以仅使用“entry” 在这种情况下。我已经确定了后缀集版本也适用于 尽量让api更简单。
更高级的用法
默认情况下,批处理字段插入到名为<name>_all的字段中, 在每个物体上。所以:
Entry.objects.batch_select('tags').all()
结果在条目实例中包含名为^{tt2}的字段$ 包含与该条目关联的标记对象。
如果您想给字段起一个不同的名称,只需使用关键字 参数-与使用Aggregationapi相同:
Entry.objects.batch_select(selected_tags='tags').all()
将意味着标记对象将被分配到名为 'selected_tags'。
如果要对相关对象执行筛选,则需要 使用批处理对象。通过这样做,您可以传递额外的关键字参数 与使用queryset的filter方法相同:
from batch_select.models import Batch Entry.objects.batch_select(tags_containing_blue=Batch('tags', name__contains='blue'))
将返回名为“tags\u containing\u name”的字段的条目对象 只有那些名字中含有“蓝色”的标签。
除了使用关键字参数进行筛选之外,还可以调用 对批处理对象执行以下方法,并传递其效果 到底层QuerySet对象:
(注意,distinct()、values()等不包括在内 附加查询如何与原始查询关联的副作用) 例如,为了达到与上面的过滤器相同的效果 执行以下操作:
from batch_select.models import Batch Entry.objects.batch_select(tags_containing_blue=Batch('tags').filter(name__contains='blue'))
但是,以下内容将排除包含“蓝色”和按名称排序的标记:
from batch_select.models import Batch batch = Batch('tags').exclude(name__contains='blue').order_by('name') Entry.objects.batch_select(tags_not_containing_blue=batch)
兼容性
Django批量选择至少应与Django 1.1-1.3一起使用。