Home | Files | Attributes | Concepts | Glossary

# File - $LogFile (2)

Previous Next

## Overview

## Attributes

Type | Description | Name
--- | --- | ---
0x10 | $STANDARD_INFORMATION |
0x30 | $FILE_NAME | $LogFile
0x80 | $DATA | [Unnamed]

## Layout of the File

### Unnamed Data Stream

Little is known about the LogFile's structure.

## Notes

### Other Information

The logging area consists of a sequence of 4KB log records. Each logrecord is structured as follows:

        offset(length)   contents
        0(4)             Magic number 'RCRD'
        1E(12)           Fixup

The logrecord supposedly contains a sequence of variable sized records. The structuring of those is not clear. File 2 is $LogFile, which contains transaction records to guarantee data integrity in case of a system failure. As pp. 37 describe, it consists of 2 copies of the restart area, and the 'infinite' logging area.

When you want to write a file on a storage unit, you have to update the file itself plus some tables of the filesystem (say as an example the date of the file). At this point, you need a transaction made of 2 operations (update the file itself, update the date of the file).

If the transaction is realized, you are sure that your file is written on the storage unit, and that the filesystem has been left in a defined state.

If the transaction is not realized (in case of e.g. power failure, or system failure in general), the filesystem is in an undefined state. The only way for you to put it back in a defined (and sane) state (this operation is called a roll-back) is to log in a special file, the log file, which operations of the transaction have been successfully completed.

At the first access to the disk after a system failure, the system read the log file and rolls back all the operations to the beginning of the last transaction.

- When the system writes to the log file, the operation must be atomic and immediate.
- You can put back your volume in sane state in a short time which is not related to the size of your disk but only to the complexity of the transaction that failed.

  Note: This operation is not performed by the Windows NT chkdsk utility, but by the system: this normal and reliable operation is a feature of NTFS.

- If your hardware is reliable, you are sure that you always have access to all the files of your volume, because it is consistent. But you can't restore eventual data losses.

        Log file organization:
        Two restart areas present in the first two pages (restart pages). When
        the volume is unmounted they should be identical.
        These are followed by log records organized in pages headed by a record
        header going up to log file size. Not all pages contain log records when a
        volume is first formatted, but as the volume ages, all records will be used.
        When the log file fills up, the records at the beginning are purged (by
        modifying the oldest_lsn to a higher value presumably) and writing begins
        at the beginning of the file. Effectively, the log file is viewed as a
        circular entity.

        Log file restart page header (begins the restart area).

        struct {
                NTFS_RECORD;                The magic is "RSTR".
                __u64 chkdsk_lsn;        The check disk log file sequence number for
                                           this restart page. Only used when the
                                           magic is changed to "CHKD". = 0
                __u32 system_page_size;        Byte size of system pages, has to be >= 512
                                           and a power of 2. Use this to calculate the
                                           required size of the usa and add this to the
                                           ntfs.usa_offset value. Then verify that the
                                           result is less than the value of the
                                           restart_offset. = 0x1000
                __u32 log_page_size;        Byte size of log file records, has to be
                                           >= 512 and a power of 2. = 0x1000
                __u16 restart_offset;        Byte offset from the start of the record to
                                           the restart record. Value has to be aligned
                                           to 8-byte boundary. = 0x30
                __s16 minor_ver;        Log file minor version. Only check if major
                                           version is 1. (=1 but >=1 is treated the
                                           same and <=0 is also ok)
                __u16 major_ver;        Log file major version (=1 but =0 is ok)
        }  RESTART_PAGE_HEADER;

        Log file restart area record. The offset of this record is found by adding
        the offset of the RESTART_PAGE_HEADER to the restart_offset value found in
        it.

        struct {
                __u64 current_lsn;        Log file record. = 0x700000, 0x700808
                __u16 log_clients;        Number of log client records following
                                           the restart_area. = 1
                __u16 client_free_list;        How many clients are free(?). If != 0xffff,
                                           check that log_clients > client_free_list.
                                           = 0xffff
                __u16 client_in_use_list;How many clients are in use(?). If != 0xffff
                                           check that log_clients > client_in_use_list.
                                           = 0
                __u16 flags;                ??? = 0
                __u32 seq_number_bits;        ??? = 0x2c or 0x2d
                __u16 restart_area_length;Length of the restart area. Following
                                           checks required if version matches.
                                           Otherwise, skip them. restart_offset +
                                           restart_area_length has to be <lt;=
                                           system_page_size. Also, restart_area_length
                                           has to be >= client_array_offset +
                                           (log_clients * 0xa0). = 0xd0
                __u16 client_array_offset;Offset from the start of this record to
                                           the first client record if versions are
                                           matched. The offset is otherwise assumed to
                                           be (sizeof(RESTART_AREA) + 7) & ~7, i.e.
                                           rounded up to first 8-byte boundary. Either
                                           way, the offset to the client array has to be
                                           aligned to an 8-byte boundary. Also,
                                           restart_offset + offset to the client array
                                           have to be <lt;= 510. Also, the offset to the
                                           client array + (log_clients * 0xa0) have to
                                           be <lt;= SystemPageSize. = 0x30
                __u64 file_size;        Byte size of the log file. If the
                                           restart_offset + the offset of the file_size
                                           are > 510 then corruption has occured. This
                                           is the very first check when starting with
                                           the restart_area as if it fails it means
                                           that some of the above values will be
                                           corrupted by the multi sector transfer
                                           protection! If the structure is deprotected
                                           then these checks are futile of course.
                                           Calculate the file_size bits and check that
                                           seq_number_bits == 0x43 - file_size bits.
                                           = 0x400000
                __u32 last_lsn_data_length;??? = 0, 0x40
                __u16 record_length;        Byte size of this record. If the version
                                           matches then check that the value of
                                           record_length is a multiple of 8, i.e.
                                           (record_length + 7) & ~7 == record_length.
                                           = 0x30
                __u16 log_page_data_offset;??? = 0x40
        }  RESTART_AREA;

        Log file client record. Starts at 0x58 even though AFAIU the above it should
        start at 0x60. Something fishy is going on. /-:

        struct {
                __u64 oldest_lsn;        Oldest log file sequence number for this
                                           client record. = 0xbd16951d
                __u64 client_restart_lsn;??? = 0x700000, 0x700827, 0x700d07
                __u16 prev_client;        ??? = 0x808, 0xd07, 0xd5d
                __u16 next_client;        ??? = 0x70
                __u16 seq_number;        ??? = 0, 4 size uncertain, Regis calls this
                                           "volume clear flag" and gives a size of one
                                           byte.
                __u16 client_name;       ??? = empty string??? size uncertain
        }  RESTART_CLIENT;

        NOTE: Above client record is followed by 0xffffffff probably to indicate
        the end of the restart area.
        Then there are 8 bytes = 0, then one __u32 = 8, followed by the Unicode
        string "NTFS" and then zeroes till the end of the page.
        Is this important at all?

        Log page record page header. Each log page begins with this header and is
        followed by several LOG_RECORD structures.

        struct {
                NTFS_RECORD;                        The magic is "RCRD".
                union {
                        __u64 last_lsn;
                        __u32 file_offset;
                }  copy;
                __u32 flags;
                __u16 page_count;
                __u16 page_position;
                union {
                        struct {
                                __u64 next_record_offset;
                                __u64 last_end_lsn;
                        }  packed;
                }  header;
        }  RECORD_PAGE_HEADER;

        Possible flags for log records.

        enum {
                LOG_RECORD_MULTI_PAGE = 1,        ???
                LOG_RECORD_SIZE_PLACE_HOLDER = 0xffff,
                        This has nothing to do with the log record. It is only so
                           gcc knows to make the flags 16-bit.
        }  LOG_RECORD_FLAGS;

        Log record header.

        struct {
                __u64 this_lsn;
                __u64 client_previous_lsn;
                __u64 client_undo_next_lsn;
                __u32 client_data_length;
                struct {
                        __u16 seq_number;
                        __u16 client_index;
                }  client_id;
                __u32 record_type;
                __u32 transaction_id;
                LOG_RECORD_FLAGS flags;
                __u16 reserved_or_alignment[3];
        Now are at ofs 0x30 into struct.
                __u16 redo_operation;
                __u16 undo_operation;
                __u16 redo_offset;
                __u16 redo_length;
                __u16 undo_offset;
                __u16 undo_length;
                __u16 target_attribute;
                __u16 lcns_to_follow;                   Number of lcn_list entries following this entry.
                __u16 record_offset;
                __u16 attribute_offset;
                __u32 alignment_or_reserved;
                __u32 target_vcn;
                __u32 alignment_or_reserved1;
                struct {                           Only present if lcns_to_follow is not 0.
                        __u32 lcn;
                        __u32 alignment_or_reserved;
                }  lcn_list[0];
        }  LOG_RECORD;

        The restart area (supposedly) has a pointer into the log area, such as
        the first and last log records written and the last checkpoint record
        written. If the restart area is screwed, recovery will be very hard -
        therefore you have two copies of the restart areas.
        Individual log records are identified by logical sequence numbers
        (LSNs). The log area wraps around, but the LSNs don't (at least not
        anytime soon), so they are used for identifying log records instead of
        the offset in the log file.
        Any modification of meta data (such as updating the time stamp that
        the file system was opened) will result in log file actions, which in
        turn result in restart area changes. It might well be that the dirty
        bit is implicit rather than explicit: The file system is clean if the
        last log record says that there are no pending transactions.

Copyright (C) Validate HTML Validate CSS