Django AJAX上传的AWS S3后端

Question

我正在尝试在我的DJANGO应用中实现一个ajax文件上传功能，使用的是django-ajax-uploader这个库。看起来一切都正常，但当我上传文件时，出现了上传失败的情况，错误代码是500，这个错误是因为AWS S3返回了一个不好的响应：

S3ResponseError at /ajax-upload↵S3ResponseError: 400 Bad Request↵<Error><Code>MalformedXML</Code><Message>你提供的XML格式不正确或者没有通过验证...

这是我的后端类：

from ajaxuploader.backends.base import AbstractUploadBackend
class S3Upload(AbstractUploadBackend):
    NUM_PARALLEL_PROCESSES = 4

    def upload_chunk(self, chunk, *args, **kwargs):
        self._counter += 1
        buffer = StringIO()
        buffer.write(chunk)
        self._pool.apply_async(
            self._mp.upload_part_from_file(buffer, self._counter))
        buffer.close()

    def setup(self, filename, *args, **kwargs):
        self._bucket = boto.connect_s3(
            settings.AWS_ACCESS_KEY_ID,
            settings.AWS_SECRET_ACCESS_KEY
        ).lookup(settings.AWS_BUCKET_NAME)

        self._mp = self._bucket.initiate_multipart_upload(filename)
        self._pool = Pool(processes=self.NUM_PARALLEL_PROCESSES)
        self._counter = 0

    def upload_complete(self, request, filename, *args, **kwargs):
        # Tie up loose ends, and finish the upload
        self._pool.close()
        self._pool.join()
        self._mp.complete_upload()

模板（javascript）：

<script src="{% static "ajaxuploader/js/fileuploader.js" %}"></script>

<script>
    $(function(){
    var uploader = new qq.FileUploader({
        action: "{% url "my_ajax_upload" %}",
        element: $('#file-uploader')[0],
        multiple: true,
        onComplete: function(id, fileName, responseJSON) {
            if(responseJSON.success) {
                alert("success!");
            } else {
                alert("upload failed!");
            }
        },
        onAllComplete: function(uploads) {
            // uploads is an array of maps
            // the maps look like this: {file: FileObject, response: JSONServerResponse}
            alert("All complete!");
        },
        params: {
            'csrf_token': '{{ csrf_token }}',
            'csrf_name': 'csrfmiddlewaretoken',
            'csrf_xname': 'X-CSRFToken',
        },
    });
    });
</script>

Answer 1

这个对我有效：

def upload_chunk(self, chunk, *args, **kwargs):
    self._counter += 1
    buffer = BytesIO(chunk)
    self._pool.apply_async(
        self._mp.upload_part_from_file(buffer, self._counter))
    buffer.close()

Answer 2

我用一个自定义的s3后端解决了这个问题，这个后端重写了上传功能，并且使用django-storages来保存文件，而不是用boto。你可以试试这个：

from ajaxuploader.backends.base import AbstractUploadBackend
from django.core.files.storage import default_storage

class S3CustomUpload(AbstractUploadBackend):
    NUM_PARALLEL_PROCESSES = 4

    def upload_chunk(self, chunk):
        #save file to s3
        self._fd.write(chunk)
        self._fd.close()

    def setup(self, filename):
        self._fd = default_storage.open('%s/%s' % ('uploads/materials/', str(filename)), 'wb')

    def upload(self, uploaded, filename, raw_data, *args, **kwargs):
        try:
            if raw_data:
                # File was uploaded via ajax, and is streaming in.
                chunk = uploaded.read(self.BUFFER_SIZE)
                while len(chunk) > 0:
                    self.upload_chunk(chunk, *args, **kwargs)
                    chunk = uploaded.read(self.BUFFER_SIZE)
            else:
                # File was uploaded via a POST, and is here.
                for chunk in uploaded.chunks():
                    self.upload_chunk(chunk, *args, **kwargs)
            return True
        except:
            # things went badly.
            return False

    def upload_complete(self, request, filename, *args, **kwargs):
        upload = Upload()
        upload.upload = settings.S3_URL + "uploads/materials/"+ filename
        upload.name = filename
        upload.save()

        return {'pk': upload.pk}