读取appengine backup_info文件可以获得EOFE

Question

我正在检查我的appengine备份文件，以确定数据损坏的时间。我使用gsutil定位并下载文件：

gsutil ls -l gs://my_backup/ > my_backup.txt
gsutil cp gs://my_backup/LongAlphaString.Mymodel.backup_info file://1.backup_info

然后我创建了一个小python程序，试图读取文件并使用appengine库解析它。在

^{pr2}$

parse_backup_info_file的代码是从复制的 backup_handler.py

运行程序时，我得到以下输出：

./view_record.py 
Traceback (most recent call last):
  File "./view_record.py", line 30, in <module>
    records = parse_backup_info_file(content)
  File "./view_record.py", line 19, in parse_backup_info_file
    version = reader.read()
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/files/records.py", line 335, in read
    (chunk, record_type) = self.__try_read_record()
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/files/records.py", line 307, in __try_read_record
    (length, len(data)))
EOFError: Not enough data read. Expected: 24898 but got 2112

我尝试过六个不同的备份信息文件，它们都显示相同的错误（数字不同） 我注意到它们都有相同的预期长度：当我观察到同一型号的不同版本时，当我查看其他模块的备份文件时，这是不正确的。

EOFError: Not enough data read. Expected: 24932 but got 911
EOFError: Not enough data read. Expected: 25409 but got 2220

我的方法有什么明显的错误吗？在

我想另一个选择是appengine backup utility没有创建有效的备份文件。 任何你能提出的建议都是非常欢迎的。 提前谢谢

Answer 1

运行AppEngine数据存储备份时会创建多个元数据文件：

LongAlphaString.backup_信息只创建一次。它包含有关在数据存储备份中创建的所有实体类型和备份文件的元数据。在

LongAlphaString.[EntityType].backup\u info为每个实体类型创建一次。它包含为[EntityType]创建的特定备份文件的元数据以及[EntityType]的架构信息。在

您的代码用于查询LongAlphaString.backup_信息，但似乎您正在尝试询问LongAlphaString.[EntityType].backup\u info的文件内容。下面是一个脚本，它将以人类可读的格式为每种文件类型打印内容：

import cStringIO
import os
import sys

sys.path.append('/usr/local/google_appengine')
from google.appengine.api import datastore
from google.appengine.api.files import records
from google.appengine.ext.datastore_admin import backup_pb2

ALL_BACKUP_INFO = 'long_string.backup_info'
ENTITY_KINDS = ['long_string.entity_kind.backup_info']


def parse_backup_info_file(content):
    """Returns entities iterator from a backup_info file content."""
    reader = records.RecordsReader(cStringIO.StringIO(content))
    version = reader.read()
    if version != '1':
        raise IOError('Unsupported version')
    return (datastore.Entity.FromPb(record) for record in reader)


print "*****" + ALL_BACKUP_INFO + "*****"
with open(ALL_BACKUP_INFO, 'r') as myfile:
    parsed = parse_backup_info_file(myfile.read())
    for record in parsed:
        print record

for entity_kind in ENTITY_KINDS:
    print os.linesep + "*****" + entity_kind + "*****"
    with open(entity_kind, 'r') as myfile:
        backup = backup_pb2.Backup()
        backup.ParseFromString(myfile.read())
        print backup