J
Jarret W. Buse
Guest
JOURNALING
If a system crashes, sometimes the loss of data occurs. Files written (created or modified), during a system crash, can be corrupted if not closed when the system shuts down. Using a journal allows data recovery of files and the data within it.
What occurs during the journaling process is that a user submits a change to a file. The first thing the file system does is to mark the changes in a 'journal', or a separate file used specifically for journaling. The size of the journal file is a set size which when full, older entries are overwritten (often called a circular file).
Three events can cause the journal entries to be written to the specified files:
1. File system buffer is low
2. Timed setting has expired
3. Journal is getting full and could start overwriting itself
If a crash occurs, the journal entries and files are compared. Data is written to the files that are in the journal, but not yet on the disk. The process recovers the data to its wanted state.
There are three types of Journaling: writeback, ordered and data.
1. writeback
Here, only the metadata is journaled and data is written to the file on the disk. In a crash, the file system is recoverable, but the physical data can be corrupted. Corruption can occur if a crash happens after the journal is made of the metadata, but before the writing of the physical data. File system recovery is the worst, but the performance is the best.
2. ordered (default)
This mode is the reverse of writeback. The physical data is written first before the metadata is journaled. The ordered mode allows the data and file system to be uncorrupted if a system crashes before the journal is written. File system recovery is medial.
3. data
In the data mode, the metadata and file contents are journaled. System performance can be poorer than the other two modes, but the fault tolerance is much better.
NOTE: Be aware that fault tolerance is not 100% guaranteed if a hard disk failure occurs that is irrecoverable. Data backups need to be performed persistently for data recovery by a hardware failure (or destruction of a computer - fire, and the like).
Let's back up a bit. The Journal itself may or may not be on the same physical hard disk as the data it journals. Some file systems may place the log in the same partition as the data on the same physical disk, but its own partition, or on a separate physical drive. Be aware that the Journal file sometimes is not readily visible to a user. Occasionally, the journal is part of the file system and therefore not viewable at all. For some file systems (ext 3 and 4) you can use the following code to read the journal showing block pointers:
You should get something similar to the following (from an ext4 file system with writeback):
The "type 1" entries are entries in the journal at the beginning of a transaction (in this case the writing of 3 data blocks). The "type 2" are the blocks of the data written to the journal, either metadata and/or file information depending on the Journaling type selected. Shown in the previous example, the journal starts at block 1. It has 2 transactions and ends at block 94.
Overall, you need to decide if a journal is needed to help with recovery of data. If so, the Journal mode needs to be determined, but usually the default of the ordered type gives a median of performance and recovery. Always keep in mind not to rely on the journal for data redundancy. The journal only assists in data recovery for a file system error, not a hardware failure. Nothing ever compares to data backups for full recovery of data.
If a system crashes, sometimes the loss of data occurs. Files written (created or modified), during a system crash, can be corrupted if not closed when the system shuts down. Using a journal allows data recovery of files and the data within it.
What occurs during the journaling process is that a user submits a change to a file. The first thing the file system does is to mark the changes in a 'journal', or a separate file used specifically for journaling. The size of the journal file is a set size which when full, older entries are overwritten (often called a circular file).
Three events can cause the journal entries to be written to the specified files:
1. File system buffer is low
2. Timed setting has expired
3. Journal is getting full and could start overwriting itself
If a crash occurs, the journal entries and files are compared. Data is written to the files that are in the journal, but not yet on the disk. The process recovers the data to its wanted state.
There are three types of Journaling: writeback, ordered and data.
1. writeback
Here, only the metadata is journaled and data is written to the file on the disk. In a crash, the file system is recoverable, but the physical data can be corrupted. Corruption can occur if a crash happens after the journal is made of the metadata, but before the writing of the physical data. File system recovery is the worst, but the performance is the best.
2. ordered (default)
This mode is the reverse of writeback. The physical data is written first before the metadata is journaled. The ordered mode allows the data and file system to be uncorrupted if a system crashes before the journal is written. File system recovery is medial.
3. data
In the data mode, the metadata and file contents are journaled. System performance can be poorer than the other two modes, but the fault tolerance is much better.
NOTE: Be aware that fault tolerance is not 100% guaranteed if a hard disk failure occurs that is irrecoverable. Data backups need to be performed persistently for data recovery by a hardware failure (or destruction of a computer - fire, and the like).
Let's back up a bit. The Journal itself may or may not be on the same physical hard disk as the data it journals. Some file systems may place the log in the same partition as the data on the same physical disk, but its own partition, or on a separate physical drive. Be aware that the Journal file sometimes is not readily visible to a user. Occasionally, the journal is part of the file system and therefore not viewable at all. For some file systems (ext 3 and 4) you can use the following code to read the journal showing block pointers:
Code:
sudo debugfs /dev/sda# (where # is the partition number - the default is 1 for the first partition)
debugfs: logdumpYou should get something similar to the following (from an ext4 file system with writeback):
Code:
Journal starts at block 1, transaction 2
Found expected sequence 2, type 1 (descriptor block) at block 1
Found expected sequence 2, type 2 (commit block) at block 4
Found expected sequence 3, type 1 (descriptor block) at block 5
Found expected sequence 3, type 2 (commit block) at block 7
Found expected sequence 4, type 1 (descriptor block) at block 8
Found expected sequence 4, type 2 (commit block) at block 10
Found expected sequence 5, type 1 (descriptor block) at block 11
Found expected sequence 5, type 2 (commit block) at block 13
Found expected sequence 6, type 1 (descriptor block) at block 14
Found expected sequence 6, type 2 (commit block) at block 16
Found expected sequence 7, type 1 (descriptor block) at block 17
Found expected sequence 7, type 2 (commit block) at block 20
Found expected sequence 8, type 1 (descriptor block) at block 21
Found expected sequence 8, type 2 (commit block) at block 23
Found expected sequence 9, type 1 (descriptor block) at block 24
Found expected sequence 9, type 2 (commit block) at block 30
Found expected sequence 10, type 1 (descriptor block) at block 31
Found expected sequence 10, type 2 (commit block) at block 33
Found expected sequence 11, type 1 (descriptor block) at block 34
Found expected sequence 11, type 2 (commit block) at block 36
Found expected sequence 12, type 1 (descriptor block) at block 37
Found expected sequence 12, type 2 (commit block) at block 39
Found expected sequence 13, type 1 (descriptor block) at block 40
Found expected sequence 13, type 2 (commit block) at block 42
Found expected sequence 14, type 1 (descriptor block) at block 43
Found expected sequence 14, type 2 (commit block) at block 45
Found expected sequence 15, type 1 (descriptor block) at block 46
Found expected sequence 15, type 2 (commit block) at block 48
Found expected sequence 16, type 1 (descriptor block) at block 49
Found expected sequence 16, type 2 (commit block) at block 51
Found expected sequence 17, type 1 (descriptor block) at block 52
Found expected sequence 17, type 2 (commit block) at block 54
Found expected sequence 18, type 1 (descriptor block) at block 55
Found expected sequence 18, type 2 (commit block) at block 57
Found expected sequence 19, type 1 (descriptor block) at block 58
Found expected sequence 19, type 2 (commit block) at block 60
Found expected sequence 20, type 1 (descriptor block) at block 61
Found expected sequence 20, type 2 (commit block) at block 63
Found expected sequence 21, type 1 (descriptor block) at block 64
Found expected sequence 21, type 2 (commit block) at block 66
Found expected sequence 22, type 1 (descriptor block) at block 67
Found expected sequence 22, type 2 (commit block) at block 69
Found expected sequence 23, type 1 (descriptor block) at block 70
Found expected sequence 23, type 2 (commit block) at block 72
Found expected sequence 24, type 1 (descriptor block) at block 73
Found expected sequence 24, type 2 (commit block) at block 75
Found expected sequence 25, type 1 (descriptor block) at block 76
Found expected sequence 25, type 2 (commit block) at block 78
Found expected sequence 26, type 1 (descriptor block) at block 79
Found expected sequence 26, type 2 (commit block) at block 81
Found expected sequence 27, type 1 (descriptor block) at block 82
Found expected sequence 27, type 2 (commit block) at block 84
Found expected sequence 28, type 1 (descriptor block) at block 85
Found expected sequence 28, type 2 (commit block) at block 87
Found expected sequence 29, type 1 (descriptor block) at block 88
Found expected sequence 29, type 2 (commit block) at block 90
Found expected sequence 30, type 1 (descriptor block) at block 91
Found expected sequence 30, type 2 (commit block) at block 93
No magic number at block 94: end of journal.The "type 1" entries are entries in the journal at the beginning of a transaction (in this case the writing of 3 data blocks). The "type 2" are the blocks of the data written to the journal, either metadata and/or file information depending on the Journaling type selected. Shown in the previous example, the journal starts at block 1. It has 2 transactions and ends at block 94.
Overall, you need to decide if a journal is needed to help with recovery of data. If so, the Journal mode needs to be determined, but usually the default of the ordered type gives a median of performance and recovery. Always keep in mind not to rely on the journal for data redundancy. The journal only assists in data recovery for a file system error, not a hardware failure. Nothing ever compares to data backups for full recovery of data.
