llvm.org GIT mirror llvm / 0b5677c
Update MSF File Documentation. This adds some more detail about the PDB container format, specifically surrounding the layout of the Free Page Map. Patch by Colden Cullen Differential Revision: https://reviews.llvm.org/D41825 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@322404 91177308-0d34-0410-b5e6-96231b3b80d8 Zachary Turner 2 years ago
1 changed file(s) with 70 addition(s) and 12 deletion(s). Raw diff Collapse all Expand all
33
44 .. contents::
55 :local:
6
7 .. _msf_layout:
8
9 File Layout
10 ===========
11
12 The MSF file format consists of the following components:
13
14 1. :ref:`msf_superblock`
15 2. :ref:`msf_freeblockmap` (also know as Free Page Map, or FPM)
16 3. Data
17
18 Each component is stored as an indexed block, the length of which is specified
19 in ``SuperBlock::BlockSize``. The file consists of 1 or more iterations of the
20 following pattern (sometimes referred to as an "interval"):
21
22 1. 1 block of data
23 2. Free Block Map 1 (corresponds to ``SuperBlock::FreeBlockMapBlock`` 1)
24 3. Free Block Map 2 (corresponds to ``SuperBlock::FreeBlockMapBlock`` 2)
25 4. ``SuperBlock::BlockSize - 3`` blocks of data
26
27 In the first interval, the first data block is used to store
28 :ref:`msf_superblock`.
29
30 The following diagram demonstrates the general layout of the file (\| denotes
31 the end of an interval, and is for visualization purposes only):
32
33 +-------------+-----------------------+------------------+------------------+----------+----+------+------+------+-------------+----+-----+
34 | Block Index | 0 | 1 | 2 | 3 - 4095 | \| | 4096 | 4097 | 4098 | 4099 - 8191 | \| | ... |
35 +=============+=======================+==================+==================+==========+====+======+======+======+=============+====+=====+
36 | Meaning | :ref:`msf_superblock` | Free Block Map 1 | Free Block Map 2 | Data | \| | Data | FPM1 | FPM2 | Data | \| | ... |
37 +-------------+-----------------------+------------------+------------------+----------+----+------+------+------+-------------+----+-----+
38
39 The file may end after any block, including immediately after a FPM1.
40
41 .. note::
42 LLVM only supports 4096 byte blocks (sometimes referred to as the "BigMsf"
43 variant), so the rest of this document will assume a block size of 4096.
644
745 .. _msf_superblock:
846
3169 sizes of 4KiB, and all further discussion assumes a block size of 4KiB.
3270 - **FreeBlockMapBlock** - The index of a block within the file, at which begins
3371 a bitfield representing the set of all blocks within the file which are "free"
34 (i.e. the data within that block is not used). This bitfield is spread across
35 the MSF file at ``BlockSize`` intervals.
36 **Important**: ``FreeBlockMapBlock`` can only be ``1`` or ``2``! This field
37 is designed to support incremental and atomic updates of the underlying MSF
38 file. While writing to an MSF file, if the value of this field is `1`, you
39 can write your new modified bitfield to page 2, and vice versa. Only when
40 you commit the file to disk do you need to swap the value in the SuperBlock
41 to point to the new ``FreeBlockMapBlock``.
72 (i.e. the data within that block is not used). See :ref:`msf_freeblockmap` for
73 more information.
74 **Important**: ``FreeBlockMapBlock`` can only be ``1`` or ``2``!
4275 - **NumBlocks** - The total number of blocks in the file. ``NumBlocks * BlockSize``
4376 should equal the size of the file on disk.
4477 - **NumDirectoryBytes** - The size of the stream directory, in bytes. The stream
5285 contains the list of blocks that the stream directory occupies, and the stream
5386 directory itself can be stitched together accordingly. The number of
5487 ``ulittle32_t``'s in this array is given by ``ceil(NumDirectoryBytes / BlockSize)``.
55
88
89 .. _msf_freeblockmap:
90
91 The Free Block Map
92 ==================
93
94 The Free Block Map (sometimes referred to as the Free Page Map, or FPM) is a
95 series of blocks which contains a bit flag for every block in the file. The
96 flag will be set to 0 if the block is in use, and 1 if the block is unused.
97
98 Each file contains two FPMs, one of which is active at any given time. This
99 feature is designed to support incremental and atomic updates of the underlying
100 MSF file. While writing to an MSF file, if the active FPM is FPM1, you can
101 write your new modified bitfield to FPM2, and vice versa. Only when you commit
102 the file to disk do you need to swap the value in the SuperBlock to point to
103 the new ``FreeBlockMapBlock``.
104
105 The Free Block Maps are stored as a series of single blocks thoughout the file
106 at intervals of BlockSize. Because each FPM block is of size ``BlockSize``
107 bytes, it contains 8 times as many bits as an interval has blocks. This means
108 that the first block of each FPM refers to the first 8 intervals of the file
109 (the first 32768 blocks), the second block of each FPM refers to the next 8
110 blocks, and so on. This results in far more FPM blocks being present than are
111 required, but in order to maintain backwards compatibility the format must stay
112 this way.
113
56114 The Stream Directory
57115 ====================
58116 The Stream Directory is the root of all access to the other streams in an MSF
65123 ulittle32_t StreamSizes[NumStreams];
66124 ulittle32_t StreamBlocks[NumStreams][];
67125 };
68
126
69127 And this structure occupies exactly ``SuperBlock->NumDirectoryBytes`` bytes.
70128 Note that each of the last two arrays is of variable length, and in particular
71 that the second array is jagged.
129 that the second array is jagged.
72130
73131 **Example:** Suppose a hypothetical PDB file with a 4KiB block size, and 4
74132 streams of lengths {1000 bytes, 8000 bytes, 16000 bytes, 9000 bytes}.
96154 {10, 15, 12}
97155 };
98156 };
99
157
100158 In total, this occupies ``15 * 4 = 60`` bytes, so ``SuperBlock->NumDirectoryBytes``
101159 would equal ``60``, and ``SuperBlock->BlockMapAddr`` would be an array of one
102160 ``ulittle32_t``, since ``60 <= SuperBlock->BlockSize``.