du -sh showing more space used than the size of the disk - how come?

FlavioB

New Member
Joined
Jan 16, 2022
Messages
8
Reaction score
1
Credits
58
Hi all.
I got a Synology NAS with 2x6TB HDDs in RAID-1 installed.
I'm also running "Active Backup for Business" on it to backup 2 PCs.
The output of "du -sh" on the root of the file structure for this software shows 7.9 TB being used, whereas this is not possible!
Can somebody help me understand why it is shown like this and how to find out how much "effective" disk space is being used?

Thanks!
F.
 


N

NorthWest

Guest
What does: df -h, have to say? What does: fdisk /dev/<disk> (and then: p), say? Is there a disagreement?
 
OP
F

FlavioB

New Member
Joined
Jan 16, 2022
Messages
8
Reaction score
1
Credits
58
So I deleted some of my backups, nevertheless it looks like this:

[email protected]:~$ df -h Filesystem Size Used Avail Use% Mounted on /dev/md0 2.3G 1.1G 1.2G 50% / none 911M 0 911M 0% /dev /tmp 915M 1.9M 913M 1% /tmp /run 915M 58M 857M 7% /run /dev/shm 915M 20K 915M 1% /dev/shm none 4.0K 0 4.0K 0% /sys/fs/cgroup cgmfs 100K 0 100K 0% /run/cgmanager/fs /dev/vg1000/lv 5.3T 4.0T 1.3T 77% /volume1 [email protected]:~$ cd / [email protected]:/$ cd volume1/ [email protected]:/volume1$ du -sh ActiveBackupforBusiness/ 5.9T ActiveBackupforBusiness/ [email protected]:/volume1$

I'm not eager to do "fdisk" on that device, I'm not sure it's a good thing as it's BTRFS.
F.
 

JasKinasis

Well-Known Member
Joined
Apr 25, 2017
Messages
1,451
Reaction score
2,057
Credits
10,088
Running du -sh / on root will show you the total disk usage from the root of the entire file-system. So that will be the disk usage for ALL devices/drives/partitions connected to the file-system root.
I've never had a RAID setup myself, so I don't know exactly how they interact with du. But still - you have two 6Tb HDD's attached to the system.
And however they've been configured - both of these drives will be located somewhere off the root of the file-system. So despite the fact that they are in a RAID configuration, du is probably traversing through both drives and adding the disk usage on both to the total.
So I'd imagine that's why the total usage of 7.9Tb is being reported.
It's the amount of storage used in root / plus the usage used on both of the 6Tb HDD's, plus any other storage devices that are connected.

So I'm guessing that if you half the 7.9Tb - you'll have a figure that is closer to the actual disk usage.

And from looking at your final post, which popped up as I was writing this post - In the output from df - the 5.3Tb listing for /dev/vg1000/lv, or /volume1 will obviously be your RAID array.

So your 6Tb drives only actually have 5.3Tb of storage each (because it's RAID1, data is mirrored on both drives, so both drives size and content should be identical). The reason it's only 5.3Tb is due to some reserved space for overheads for the file-system that is in use in that drive/partition.
e.g. used for the file-system journal, or other file-system metadata.
Of that 5.3Tb, 4Tb is used (or approximately 4Tb) so you have approximately 1.3Tb of space left on your RAID array.
So that is your actual answer.

So I think your original problem was, you were using du in the wrong place. And despite the RAID configuration, the amount of data on both drives in the RAID array was somehow being counted twice - once per drive, yielding a usage that was much higher than you expected.

Perhaps running du -sh /volume1 might yield a more correct value? But don't quote me on it. I've never set up, or used RAID before. In this case, it may even be that du still somehow counts the space used on both of the drives in the array. In which case, you'd have to half the value reported by du.

But from using the df command - you can already see how much space has actually been used on the RAID array, so there's no need to use the du command anyway!
 
Last edited:
OP
F

FlavioB

New Member
Joined
Jan 16, 2022
Messages
8
Reaction score
1
Credits
58
Running du -sh / on root will show you the total disk usage from the root of the entire file-system. So that will be the disk usage for ALL devices/drives/partitions connected to the file-system root.
I've never had a RAID setup myself, so I don't know exactly how they interact with du. But still - you have two 6Tb HDD's attached to the system.
And however they've been configured - both of these drives will be located somewhere off the root of the file-system. So despite the fact that they are in a RAID configuration, du is probably traversing through both drives and adding the disk usage on both to the total.
So I'd imagine that's why the total usage of 7.9Tb is being reported.
It's the amount of storage used in root / plus the usage used on both of the 6Tb HDD's, plus any other storage devices that are connected.

So I'm guessing that if you half the 7.9Tb - you'll have a figure that is closer to the actual disk usage.

And from looking at your final post, which popped up as I was writing this post - In the output from df - the 5.3Tb listing for /dev/vg1000/lv will obviously be your RAID array.

So your 6Tb drives only actually have 5.3Tb of storage each (because it's RAID, data is mirrored on both drives, so both drives size and content should be identical). The reason it's only 5.3Tb is due to some reserved space for overheads for the file-system that is in use in that drive/partition.
e.g. used for the file-system journal, or other file-system metadata.
Of that 5.3Tb, 4Tb is used (or approximately 4Gb) so you have approximately 1.3Tb of space left on your RAID array.
So that is your actual answer.

So I think your original problem was, you were using du in the wrong place. And despite the RAID configuration, the amount of data on both drives in the RAID array was somehow being counted twice - once per drive, yielding a usage that was much higher than you expected.

Perhaps running du -sh /dev/vg1000/lv might yield a more correct value? But don't quote me on it. I've never set up, or used RAID before. In this case, it may even be that du still somehow counts the space used on both of the drives in the array. In which case, you'd have to half the value reported by du.

But from using the df command - you can already see how much space has actually been used on the RAID array, so there's no need to use the du command anyway!
Hi and thanks for your thorough reply. I worked with RAID (setting it up manually) many years ago, but I tend to disagree with your view of it. It doesn't make sense (to me at least) that the system would go through *both* RAID members. I know that the physical HDDs are 2x6TB, but still the system is only seeing one LV (logical volume) of 5.3 TB...

Let's have maybe someone more experienced with RAID chime in on this :)
Thanks!
F.
 

f33dm3bits

Gold Member
Gold Supporter
Joined
Dec 11, 2019
Messages
4,463
Reaction score
3,169
Credits
32,360
If it's raid1 than you would you should normally only see the the one raid device if it's software raid and if it's hardware raid you would only see the one disk which you created in your raid manager. It would be be interesting to see the output of the following command.
Code:
lsblk
If you run the following you will get a better overview of how much space each directory in the tree, if you could share the output of that as well.
Code:
du -h --max-depth=1 / 2> /dev/null | egrep -v "proc|sys|lost|run"
 
Last edited:
OP
F

FlavioB

New Member
Joined
Jan 16, 2022
Messages
8
Reaction score
1
Credits
58
Hi.
Synology NAS does not have the "lsblk" command, but the output of the other one looks like this:

[email protected]:/volume1$ du -h --max-depth=1 / 2> /dev/null | egrep -v "proc|sys|lost|run" 20K /dev 36K /.old_patch_info 4.0K /root 4.0K /tmpRoot 4.0K /initrd 8.0K /boot 4.0K /mnt 4.0K /opt 464K /tmp 9.7M /etc 132M /var 884M /usr 0 /config 9.8T /volume1 6.4M /var.defaults 24M /.syno 8.9M /etc.defaults 9.8T /

I still don't understand those 9.8T on /volume1...
F.
 

f33dm3bits

Gold Member
Gold Supporter
Joined
Dec 11, 2019
Messages
4,463
Reaction score
3,169
Credits
32,360
DSM is a very customized GNU/Linux setup so it's doesn't have everything that a normal Linux distribution has so that must be the reason why lsblk cannot be found. Can you share the output of the following?
Code:
mount | grep volume1
grep  volume1 /etc/fstab
 

f33dm3bits

Gold Member
Gold Supporter
Joined
Dec 11, 2019
Messages
4,463
Reaction score
3,169
Credits
32,360
Code:
mount | grep volume1
grep  volume1 /etc/fstab
Nevermind these last two, I think I saw you mentioning btrfs in this topic somewhere and I helped a friend recently with a Synology which had btrfs as filesystem as well. So I think it has to do with how btrfs work.
Try running these.
Code:
btrfs filesystem df -h /
btrfs filesystem df -h /volume1
 
OP
F

FlavioB

New Member
Joined
Jan 16, 2022
Messages
8
Reaction score
1
Credits
58
here we go!

Code:
[email protected]:/volume1$ btrfs filesystem df -h /
ERROR: not a btrfs filesystem: /
[email protected]:/volume1$ btrfs
btrfs              btrfs-debug-tree   btrfs-find-root    btrfs-map-logical  btrfs-super-hack   btrfsck
btrfs-block-csum   btrfs-debugfs      btrfs-image        btrfs-show-super   btrfs-zero-log
[email protected]:/volume1$ btrfs filesystem df -h /volume1
Data, single: total=4.75TiB, used=3.98TiB
System, DUP: total=8.00MiB, used=96.00KiB
Metadata, DUP: total=97.50GiB, used=2.88GiB
GlobalReserve, single: total=1.51GiB, used=0.00B
[email protected]:/volume1$
 

f33dm3bits

Gold Member
Gold Supporter
Joined
Dec 11, 2019
Messages
4,463
Reaction score
3,169
Credits
32,360
I have no experience with btrfs but it looks like from that output your btrfs filesystem(/volume1) is using 3.98T and has the total size of 4.75T. That seems to come closer to the actual disk size you have, so from my understanding of that it seems to normal commands used to find information about the filesystem used on ext4 and xfs don't seem to play well with btrfs.
 
OP
F

FlavioB

New Member
Joined
Jan 16, 2022
Messages
8
Reaction score
1
Credits
58
I have no experience with btrfs but it looks like from that output your btrfs filesystem(/volume1) is using 3.98T and has the total size of 4.75T. That seems to come closer to the actual disk size you have, so from my understanding of that it seems to normal commands used to find information about the filesystem used on ext4 and xfs don't seem to play well with btrfs.
mmmhhh... OK, so what do you suggest next?
I mean: my original goal was to find out where and how much space is used on my NAS, so that I can decide where to start cleaning up and when eventually to buy bigger capacity HDDs. But you might understand that if I see 9.8T it is not helping me - it's rather confusing me! :-(
 

f33dm3bits

Gold Member
Gold Supporter
Joined
Dec 11, 2019
Messages
4,463
Reaction score
3,169
Credits
32,360
mmmhhh... OK, so what do you suggest next?
Read the page I linked in my last post, it's got some useful information about btrfs. Also With a Synology you have access to DSM, you should be able view from the file explorer how much each directory is using. So if you have a subdir in /volume1, right click and then I think go to properties but I can't remember exactly because I don't have a Synology myself.
 

f33dm3bits

Gold Member
Gold Supporter
Joined
Dec 11, 2019
Messages
4,463
Reaction score
3,169
Credits
32,360

f33dm3bits

Gold Member
Gold Supporter
Joined
Dec 11, 2019
Messages
4,463
Reaction score
3,169
Credits
32,360
You will probably get better answers if you were to ask this on the official/unofficial Synology forums or subreddit.
 

wizardfromoz

Administrator
Staff member
Gold Supporter
Joined
Apr 30, 2017
Messages
7,115
Reaction score
5,993
Credits
23,554

JasKinasis

Well-Known Member
Joined
Apr 25, 2017
Messages
1,451
Reaction score
2,057
Credits
10,088
Maybe also mention wherever you go, in the first Post, that you are using BTRFS? ;)

From a friend's signature

Learn how to ask a good support question.

Cheers

Chris Turner
wizardfromoz
To be fair, the OP DID mention it in their second post.
I noticed that they'd mentioned BTRFS, but I've never really used BTRFS, or RAID. So I wasn't aware that standard Linux/Unix disk utilities are problematic with BTRFS. So perhaps I was somewhat mistaken in my interpretation!
 

wizardfromoz

Administrator
Staff member
Gold Supporter
Joined
Apr 30, 2017
Messages
7,115
Reaction score
5,993
Credits
23,554
For the rest of The Viewers - lsblk does actually work on BTRFS under Linux (I am in Garuda and it works), but it is not on Synology by default, whose repository is virtually its own appstore.

You can install Linux commands such as lsblk and others using Optware, then update your packages and run

ipkg install name-of-the-tools

but you might risk your setup.

That being said, the OP is better advised as was offered

You will probably get better answers if you were to ask this on the official/unofficial Synology forums or subreddit.

Wiz
 
$100 Digital Ocean Credit
Get a free VM to test out Linux!

Members online


Top