使用Itertools优化Django查询

Question

我有一个教育网站，提供基于年级的课程。在

1. GradeLevel表存储所有可能的等级。然后我有课程类别和课程表。

2. GradeLevel表与课程表和类别表建立了反向关系。

3. 我在GradeLevel表中循环查看每个年级（8个年级），然后一路抓住各自的课程和类别。

4. 完成后，我将收集到的所有课程和类别放入一个列表中，并将其传递给我的模板。

现在，问题是Django至少对每个查询求值两次。第一次是第一次请求，第二次是我把它放进名单。（我正在使用itertools来链接结果。Itertools正在导致查询再次运行。）这有一个不利的影响，使我的服务器减速到爬行状态。在

我的问题是，是否有人可以看看我的模型和查询，并就如何更好地进行查询提出建议，以避免和/或减轻这一主要的性能瓶颈。在

等级模型：

class GradeLevel(models.Model):
    title = models.CharField('Grade',max_length=10, null=True, blank=True, db_index=True)
    fullname = models.CharField('Description',max_length=100, null=True, blank=True, db_index=True)

课程类别模型：

^{pr2}$

课程设置：

class LessonCurriculum(models.Model):         
    title = models.CharField(max_length=255, null=True, blank=True, db_index=True)
    ...
    gradelevel = models.ManyToManyField(GradeLevel, related_name='grade_curriculum', null=True, blank=True)

我的观点：

from itertools import chain
from operator import attrgetter

def my_view(request):
    grade_pk =  GradeLevel.objects.prefetch_related().get(title='pre-k')
    grade_pk_categories = grade_pk.grade_cats.filter(active=True,featured=True)
    grade_pk_galleries = grade_pk.grade_curriculum.filter(active=True,featured=True)

    grade_k =  GradeLevel.objects.prefetch_related().get(title='k')
    grade_k_categories = grade_k.grade_cats.filter(active=True,featured=True)
    grade_k_galleries = grade_k.grade_curriculum.filter(active=True,featured=True)

    grade_1 =  GradeLevel.objects.prefetch_related().get(title='1')
    grade_1_categories = grade_1.grade_cats.filter(active=True,featured=True)
    grade_1_galleries = grade_1.grade_curriculum.filter(active=True,featured=True)

    grade_2 =  GradeLevel.objects.prefetch_related().get(title='2')
    grade_2_categories = grade_2.grade_cats.filter(active=True,featured=True)
    grade_2_galleries = grade_2.grade_curriculum.filter(active=True,featured=True)

    grade_3 =  GradeLevel.objects.prefetch_related().get(title='3')
    grade_3_categories = grade_3.grade_cats.filter(active=True,featured=True)
    grade_3_galleries = grade_3.grade_curriculum.filter(active=True,featured=True)

    grade_4 =  GradeLevel.objects.prefetch_related().get(title='4')
    grade_4_categories = grade_4.grade_cats.filter(active=True,featured=True)
    grade_4_galleries = grade_4.grade_curriculum.filter(active=True,featured=True)

    grade_5 =  GradeLevel.objects.prefetch_related().get(title='5')
    grade_5_categories = grade_5.grade_cats.filter(active=True,featured=True)
    grade_5_galleries = grade_5.grade_curriculum.filter(active=True,featured=True)

    grade_6 =  GradeLevel.objects.prefetch_related().get(title='6')
    grade_6_categories = grade_6.grade_cats.filter(active=True,featured=True)
    grade_6_galleries = grade_6.grade_curriculum.filter(active=True,featured=True)

    grade_7 =  GradeLevel.objects.prefetch_related().get(title='7')
    grade_7_categories = grade_7.grade_cats.filter(active=True,featured=True)
    grade_7_galleries = grade_7.grade_curriculum.filter(active=True,featured=True)

    grade_8 =  GradeLevel.objects.prefetch_related().get(title='8')
    grade_8_categories = grade_8.grade_curriculum.filter(active=True,featured=True)
    grade_8_galleries = grade_8.grade_curriculum.filter(active=True,featured=True)

    gallery_list = list(set(sorted(chain(grade_pk_categories,grade_pk_galleries,grade_k_categories,grade_k_galleries,grade_1_categories,grade_1_galleries,grade_2_categories,grade_2_galleries,grade_3_categories,grade_3_galleries,grade_4_categories,grade_4_galleries,grade_5_categories,grade_5_galleries,grade_6_categories,grade_6_galleries,grade_7_categories,grade_7_galleries,grade_8_categories,grade_8_galleries), key=attrgetter('display_order'))))

Answer 1

您要使用的是chain.from_iterable。这将防止查询多次运行。在

gallery_list = tuple(chain.from_iterable(grade_pk_categories,grade_pk_galleries,grade_k_categories,grade_k_galleries,grade_1_categories,grade_1_galleries,grade_2_categories,grade_2_galleries,grade_3_categories,grade_3_galleries,grade_4_categories,grade_4_galleries,grade_5_categories,grade_5_galleries,grade_6_categories,grade_6_galleries,grade_7_categories,grade_7_galleries,grade_8_categories,grade_8_galleries))

您还希望在通过set运行项目之后进行排序，而不是之前。在

^{pr2}$

我会重新考虑你如何获取所有这些数据。您正在为每个职等执行两个查询，而您可能会将其替换为针对所有数据的一个或两个查询。在

至少可以使用for loop来减少代码量：

grade_titles = ['pre-k', 'k', '1', '2', '3', '4', '5', '6', '7', '8']
gallery_list = []
for grade_title in grade_titles:
    grade = GradeLevel.objects.prefetch_related().get(title=grade_title)
    grade_categories = grade_pk.grade_cats.filter(active=True,featured=True)
    grade_galleries = grade_pk.grade_curriculum.filter(active=True,featured=True)
    gallery_list.extend(grade_categories)
    gallery_list.extend(grade_galleries)
gallery_list = list(sorted(frozenset(gallery_list), key=attrgetter('display_order')))

Answer 2

我会更进一步优化@nathanvilaescusa的答案

grade_titles = ['pre-k', 'k', '1', '2', '3', '4', '5', '6', '7', '8']

# Turn list into list of Q items
queries = [models.Q(title=grade_title) for grade_title in grade_titles]

# Take one item of the list
query = queries.pop()

# OR the Q objects with the ones remaining in the list
for item in queries:
    query |= item

grades = GradeLevel.objects.prefetch_related().filter(query)
# grades should have everything you need and operation is more CPU
# and less DB bound now