mmarch项目的存档实用程序，mmap友好存档
mmarch-pack的Python项目详细描述
映射内存友好的存档格式-mmarch
目标
- 提供可扩展的独立于平台的文件归档格式（多棒的发明！）
- 有效的内存访问，o（1）完整路径文件访问，o（n）readdir（文件列表立即可用）
- 内存映射-依赖操作系统缓存而不是用户空间中的缓存
- 没有动态内存分配
- 所有函数均以本机和外部endianess模式编译，加速本机访问
结构
- C/C-库
- python/python archiver
- 解包/简单解包器，C库测试工具
文件格式
关于格式的一些设计注意事项：
- 仅使用32/64位类型进行有效的对齐访问
- 没有模糊的算法，头中的所有偏移都是绝对的
- 在第一个4g处对所有报头进行分组，节省内部偏移量
- 可能是乐也可能是
1. File Header
u32 "MARC" magic, "CRAM"
u32 version, current == 1 (still experimental)
u32 page size, normally 4096, but you can use any arbitrary number, alignment for file data
u32 total header size. If you want to mmap header, use this number (aligned to page boundary)
u64 total file size. Aligned to page size. Reader shall not read anything past this point.
//all the following table offsets in header should be aligned (at least 4 bytes)
u32 object record table offset
u32 filename table offset
u32 readdir table offset
2. Object Record Table Header
u32 record fields counter, R (could be viewed as version, every record here has R*4 bytes size)
u32 record count, N. Total record counter. Size of this table is N * R + this header
u32 directory count, DN, all IDs below DN is directories
N *
3. Object Record, each of size R * u32
u64 file data offset (0 for directories)
u64 file size (0 for directories)
u32 full path offset (usually string pool below, must've been within header size)
u32 full path length (strings ARE NOT zero terminated, you shall read exact name size bytes from name offset)
[string pool]
4. Filename table
u32 hash function id. (0 - FNV1, 1 - FNV1A, 2 - PYTHON, 3 - R5A)
u32 bucket count, B
Filename table index:
(B + 1) * u32
u32 offset to bucket 0
u32 offset to bucket 1
…
u32 offset to the end of bucket B-1
Filename table entry:
u32 object id (see object record table, p.2)
5. Readdir table
(D + 1) * readdir table index follows
u32 offset to object list for directory 0
u32 offset to object list for directory 1
…
u32 offset to the end of for directory D - 1
Readdir table entry
u32 object id (see object record table, p.2)
u32 local name offset (no path)
u32 local name length
[gap to align to page size]
[file data referenced by offset/size in object table]
[gap to align to page size]
[file data referenced by offset/size in object table]
[gap to align to page size]
[file data referenced by offset/size in object table]
欢迎加入QQ群-->： 979659372
