4GB USB flash drive reports size as 8MB - parted fdisk df dd - can't re-create linux installer / burn image with dd

[spaz]

Hi all.

Any help on the following greatly appreciated.

I have a 4GB USB 2.0 drive which is reporting the wrong size (8MB) in lsblk, df, fdisk.
I had previously been using it as an installer for my distro (which was outdated by 2 versions).

Yesterday, I tried to burn using dd (like I always do) a newer version of the distro to this USB.

Several times it finished too quickly, and booting into the USB, the old distro installer was still there.

Tried a few times things such as

sudo dd if=/home/<path-to-image>  of=/dev/<target-disk> status=progress

and

    sudo dd if=/home/<path-to-image>  of=/dev/<target-disk> bs=1024 status=progress

and

    sudo dd if=/home/<path-to-image>  of=/dev/<target-disk> bs=2048 status=progress

I tried parted:
mklabel (msdos), mkpart (p), followed by mkfs.ext4.

Then dd again.


Somewhere through this process,
this flash drive has lost its drive information, and
all command line disk reporting programs show the drive as being only 8MB.

Doing mkpart primary seemed to create 4MB partitions.
Running again seemed to create a second partition same size.

Running mklabel and repeating gives same results.


With mkpart, I was trying to get it to align properly (usually with the minimal alignment option specified when starting parted).
I tried
mkpart
start 512b end 100%
start 1024b end 100%
start 1 end 100%

Repeated a few times with minor variations for sanity checks.

I also tried gparted "attempt data rescue",
and several other suggestions found in various forums.

Reading the parted manual section on flash drives again,
I realize I probably made a mistake and bungled the block size allocations and a couple of other parameters,
and I should have followed the guide for cheap USB Flash drives in the manual
(I haven't done this in a long time, and was in a bit of a rush)

Namely I should have followed these sections in the parted manual

https://www.gnu.org/software/parted/manual/parted.html

2.4.6 mkpart

...
Now, we will show how to partition a low-end flash device (“low-end”, as of 2011/2012). For such devices, you should use 4MiB-aligned partitions. This command creates a tiny place-holder partition at the beginning, and then uses all remaining space to create the partition you’ll actually use:

$ parted -s /dev/sdX -- mklabel msdos \
mkpart primary fat32 64s 4MiB \
mkpart primary fat32 4MiB -1s

Note the use of ‘--’, to prevent the following ‘-1s’ last-sector indicator from being interpreted as an invalid command-line option. The above creates two empty partitions. The first is unaligned and tiny, with length less than 4MiB. The second partition starts precisely at the 4MiB mark and extends to the end of the device.

The next step is typically to create a file system in the second partition:


$ mkfs.vfat /dev/sdX2


Footnotes

(2)
Cheap flash drives will be with us for a long time to come, and, for them, 1MiB alignment is not enough. Use at least 4MiB-aligned partitions. For details, see Arnd Bergman’s article, http://lwn.net/Articles/428584/ and its many comments.

I tried the above hoping it would restore the USB to reporting correctly.

I feel this USB must be fine,
it has just lost some kind of essential information.

Any suggestions for how I can fix this?
I probably have other old USBs floating around here which report similar things,
which I feel may be salvageable with the correct info.

 

[Egzoset]

Hummm... That maybe too much caution but i personally wouldn't want to risk inserting that thing in any flash reader i may still value.

 

[Brickwizard]

Sounds like time to ditch it, USB's do not have an indefinite life, depending on the quality and the amount of read/write use it's had, we have many examples of members having problems with either old or cheap pen-drives, [it may be ok for storage]

 

[osprey]

@spaz wrote:

I feel this USB must be fine

If only feelings could have such power as to make things work and be fine

If you followed the instructions from the parted manual validly and were unsuccessful, and you were also unsuccessful with gparted and dd, then with three strikes like that suspicion falls upon the state of the usb.

To test the current state of the usb, you could use other tools to format the usb like fdisk, or a GUI formatter like the gnome disk utility.

The last thing to try is to use a tool like shred to write over the whole usb, or fill the usb with zeros using the dd command to clear it of any code that might be obstructing it's proper use. Then try to format it again. If those approaches fail, the message of its broken state should become relatively unmistakable.

 

[bob466]

When this happens...the only thing to do is...

 

[spaz]

Hi Egzoset.
Surely a simple passive flash drive does not pose any risk to a flash reader?
They don't generate volts or amps, they are bus powered by the reader - which itself would probably have some small amount of overvoltage protection.
What risk do you think a possibly dead USB flash drive could pose?

################

Hi Brickwizard!!
Ah, the unwanted facts.
I do have an unfortunate bin-aversion.
Dislike chucking anything electronic if I have a hunch there is life in it.

In fact, I would go out of my way to find some kind of ewaste for such things,
tree-hugging dirty hippie that I am.

Yeah, I know flash drives don't have an indefinite life,
but the fact that it was working just fine before attempting to re-burn a new installer image,
and even when reporting 8MB, I could STILL boot into the previous live Linux distro/installer makes me think there is just something simple that needs to be done to make it play nice,
and that the drive is just a bit corrupted, but perhaps can be uncorrupted.

Yes, I also suppose
"it may be ok for storage"

But unfortunately I can't seem to create proper partition any larger than 8MB that uses the whole drive.
I could try mounting that 8MB, and see if I can fit larger files on it.
Maybe I'll try that tomorrow, but all signs seem to point to "No".

USB 2.0 flash drives are becoming harder to come across,
and not all old computers like to see USB3.0 USB 3.0 drives as linux live installer disks.
Often USB 3.0 flash drives don't play nice for this purpose.

#########
@ osprey

I feel this USB must be fine

If only feelings could have such power as to make things work and be fine

Yeah, yeah, wise-guy

The "magical feeling" is based partly on some of the reasoning in my reply to Brickwizard above.

Yes, I acknowledge the drive may just be stuffed / a bin-job,
but there are just a few factors that make me think something is just a bit corrupted on the drive and could potentially be uncorrupted.

If you followed the instructions from the parted manual validly and were unsuccessful, and you were also unsuccessful with gparted and dd, then with three strikes like that suspicion falls upon the state of the usb.

To test the current state of the usb, you could use other tools to format the usb like fdisk, or a GUI formatter like the gnome disk utility.

The last thing to try is to use a tool like shred to write over the whole usb, or fill the usb with zeros using the dd command to clear it of any code that might be obstructing it's proper use. Then try to format it again. If those approaches fail, the message of its broken state should become relatively unmistakable.

That's all excellent constructive information / sanity checks!
Thanks very much.
I'll try those things.

I actually don't use fdisk often, although I did today.
I should try again with that, cfdisk, fsdisk, gnome disk utility, and maybe just do a general search around for various disk / flash disk utilities.

...and I will definitely try shred!
(Why didn't I think of that? 0__o )

I'll try and find time to have a play-around and if anything comes of it,
I'll report back.

 

[CaffeineAddict]

@spaz

What probably happens is that the new partition table is not immediately recognized by the kernel.

When using fdisk what you need is to delete partition table and write changes to disk, after this step the fdisk will tell you that kernel needs to reload fstab, at this point simply reboot system.

Then run again fdisk and create a new partition table and write changes to disk and again reboot system again.
After second reboot create a new partition and this should work, if not your USB is dead.

 

[Egzoset]

Hummm... That maybe too much caution but i personally wouldn't want to risk inserting that thing in any flash reader i may still value.

What risk do you think a possibly dead USB flash drive could pose?

Does data corruption ring a bell? Additionally, i vaguely remember reading about "fake" (fraudulent/counterfit) units misrepresenting their total capacity, causing read & write speed issues. That was a decade ago, i think...

 

[Trenix25]

Hi Egzoset.
Surely a simple passive flash drive does not pose any risk to a flash reader?
They don't generate volts or amps, they are bus powered by the reader - which itself would probably have some small amount of overvoltage protection.
What risk do you think a possibly dead USB flash drive could pose?

This is not entirely accurate. There is a device that looks like a USB flash drive that has been weaponized to destroy USB ports. It saves up the current from the port itself and delivers spikes of voltage to the port, over and over again, until it destroys the port and often the whole bus. This can be very expensive to fix. Why people make such things is beyond me. Some people are just sinister.

Amazon sells cheap 32 GB USB flash drives, for maybe US$5 a piece if you buy a pack of four or five. It looks to me like it's time to replace your "problem."

Signed,

Matthew Campbell

 

[MikeWalsh]

@spaz :-

I'm assuming this is quite an elderly flash drive? I would guess it is, since I can't remember the last time I saw, or indeed purchased one that small. Must be nearly 10 years ago, at least; the smallest available these days are at least 8GB, if not 16GB.

SanDisk had an entire class-action lawsuit levelled at them around a decade back. Millions of their 8GB Cruzer 'Blades' began going into 'read-only' mode. Their forums were awash with complaints; there was a long-running thread dedicated to this one issue, and the last time I looked, it had been running for something like 4 1/2 yrs & was up to almost 6000 posts....

The trouble was eventually tracked back to a shipment of dodgy controller chips from Hynix. Last I heard, the two of 'em were locked in a long-lasting court case, Hynix denying all responsibility and SanDisk refusing to back down! I don't know what the outcome was, except that anyone who subsequently made a complaint was automatically offered a free replacement drive....

I agree with you about older 'puters and USB 3.0 drives. Old boxes won't boot from USB 3.0 drives, even though they'll physically plug-in and work fine for storage.....because the USB 2.0 controller hubs are NOT set-up to recognise the boot flag on anything other than USB 2.0 drives; indeed, they don't even know what the USB 3.0 standard is, or does. The problems don't stop there, either, because the firmware on later USB 2.0 drives reports them as USB-HDDs.......and the BIOS on many older machines doesn't understand that.


Mike.

 

[Egzoset]

Hummm... Actually i wish i could provide a reference about some horror story i once read relatively some milder version of a "weaponized" USB flash drive, involving a malfunction loop after reaching the last real record, e.g. before reaching the bogus target which didn't exist. In case of a doubt i might find appropriate to check such device using H2testw, CapacityTester, ValiDrive or else, except i'd still prefer not to risk my internal SD card reader (no longer present on many new products anyway)... Nonetheless data storage reliability remains a major concern as this defeats the drive's purpose to begin with.

In the case of vintage USB booting there happens to be a utility called 'Plop Boot Manager' and version 5.0.15 dates back to March 15, 2013 (ref.: www . plop . at), as i recall it was even incorporated into the Slax and/or Porteus boot menu(s) for that exact added convenience; i myself used to keep a bootable CD equipped with it so i could play with USB bootable drives via USB2, not USB3 as the 3 extension cards i tried kept making smoke - i mean literally! Because my PC's BiOS was quite old back then...

 

[spaz]

When using fdisk, how can I delete the delete partition table of /dev/sdc
and write changes to disk?

To delete the partition table of /dev/sdc and write changes to the disk, you can use the following commands:

sudo fdisk /dev/sdc: Open the fdisk utility for the specified device.

d: Delete the partition table.
w: Write the changes to the disk.

Note: Be cautious when using these commands, as they can permanently alter the disk. Make sure to backup your data before proceeding.

#################


Hi All.
Taking LOTS of amateurish shots in the dark at this just now:
Been having a bit of a novice play with e2fsprogs suite (e2fsck etc) and badblocks.

If I ever learn to bash script, maybe I'll write one that executes various programs and sanity checks for this sort of thing. I might search around to see if something like that exists already.

Trying variations on the instructions from the parted manual,
which recommends 2 partitions,

I keep getting

    sdc            8:32   1     8M  0 disk
    ├─sdc1         8:33   1     4M  0 part
    └─sdc2         8:34   1     4M

Even when I try to set the end of partition 2 as 100%

(This is a 4GB or 2GB stick, I forget now ,it's not marked on it.
I was using it as my MX installer just previous to reformatting it the other day, which has caused the problem. This is NOT some counterfeit stick as Egzoset suggests it might possibly be).


I realizd e2fsck may be the wrong tool for this job as I think it is designed for ext filesystems,
and I initially formatted the drive as fat using the parted manual.
Later, I tried reformatting disk as ext.

Conflicting information, as badblocks doesn't report any bad blocks,
but badblocks doesn't report how much memory space it has scanned.
It reports how many blocks are scanned though.
Assuming blocksize is 512B (I forget)
512B * 8191 Blocks
= 4,193,792 Bytes

Which I guess means it is only scanning 4MB worth, so not the whole disk.


When running

    $ sudo e2fsck -pcv /dev/sdc
    e2fsck: Bad magic number in super-block while trying to open /dev/sdc
    /dev/sdc:
    The superblock could not be read or does not describe a valid ext2/ext3/ext4
    filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
    filesystem (and not swap or ufs or something else), then the superblock
    is corrupt, and you might try running e2fsck with an alternate superblock:
        e2fsck -b 8193 <device>
        e2fsck -b 32768 <device>

or

    $ sudo badblocks -v /dev/sdc
    Checking blocks 0 to 8191
    Checking for bad blocks (read-only test): done                                              
    Pass completed, 0 bad blocks found. (0/0/0 errors)

    $ sudo badblocks -v /dev/sdc1
    Checking blocks 0 to 4063
    Checking for bad blocks (read-only test): done                                              
    Pass completed, 0 bad blocks found. (0/0/0 errors)

    $ sudo badblocks -v /dev/sdc2
    Checking blocks 0 to 4095
    Checking for bad blocks (read-only test): done                                              
    Pass completed, 0 bad blocks found. (0/0/0 errors)

Mmmm.....

Formatting /dev/sdc2 to ext4 and re-running e2fsck, I get some kind of magic

    $ sudo e2fsck /dev/sdc2
    e2fsck 1.46.2 (28-Feb-2021)
    Superblock has an invalid journal (inode 8).
    Clear<y>? yes
    *** journal has been deleted ***

    Resize inode not valid.  Recreate<y>? yes
    e2fsck: Inode checksum does not match inode while reading bad blocks inode
    This doesn't bode well, but we'll try to go on...
    Pass 1: Checking inodes, blocks, and sizes
    The bad block inode looks invalid.  Clear<y>? yes
    Root inode is not a directory.  Clear<y>? yes
    Pass 2: Checking directory structure
    Pass 3: Checking directory connectivity
    Root inode not allocated.  Allocate<y>? yes
    /lost+found not found.  Create<y>? yes
    Pass 4: Checking reference counts
    Pass 5: Checking group summary information
    Block bitmap differences:  +1 +(3--5) +(7--33) -(37--38) +50 +(67--70) +(72--97) +(101--102) +104 +107 +(109--112) +114 +116 +(123--129) +(131--192) -(195--197) -(200--201) -(205--208) -210 -213 -215 -(217--218) -(220--221) -223 -225 -227 -(229--234) -(241--244) -(247--248) -(252--253) -255 -449 -(451--453) -(455--456) -(459--461) -463 -465 -(467--469) -476 -478
    Fix<y>? yes
    Free blocks count wrong for group #0 (2892, counted=3929).
    Fix<y>? yes
    Free blocks count wrong (2892, counted=3929).
    Fix<y>? yes
    Inode bitmap differences:  +1 +(3--5) +(7--10) -34 -(37--38) -66 -71 -(98--100) -103 -(105--106) -108 -113 -115 -(117--122) -130 -(193--197) -(200--201) -(205--208) -210 -213 -215 -(217--218) -(220--221) -223 -225 -227 -(229--234) -(241--244) -(247--248) -(252--253) -255 -449 -(451--453) -(455--456) -(459--461) -463 -465 -(467--469) -476 -478
    Fix ('a' enables 'yes' to all) <y>? yes
    Free inodes count wrong for group #0 (1012, counted=1013).
    Fix ('a' enables 'yes' to all) <y>? yes
    Directories count wrong for group #0 (3, counted=2).
    Fix ('a' enables 'yes' to all) <y>? yes
    Free inodes count wrong (1012, counted=1013).
    Fix ('a' enables 'yes' to all) <y>? yes
    Padding at end of inode bitmap is not set. Fix<y>? yes
    Recreate journal<y>? yes
    Creating journal (1024 blocks):  Done.

    *** journal has been regenerated ***

    /dev/sdc2: ***** FILE SYSTEM WAS MODIFIED *****
    /dev/sdc2: 11/1024 files (0.0% non-contiguous), 1191/4096 blocks

But lsblk still reports

    sdc            8:32   1     8M  0 disk
    ├─sdc1         8:33   1     4M  0 part
    └─sdc2         8:34   1     4M

    $fdisk
    Command (m for help): o      

    Created a new DOS disklabel with disk identifier 0xa534fac7.

    Command (m for help): w
    The partition table has been altered.
    Syncing disks.

lsblk still reports same.

Back to parted,
tried simply created a primary partition starting at 4MiB ending 100%
lsblk reports same.

Tried creating primary part starting 4MiB ending -1s.

    sdc            8:32   1     8M  0 disk
    ├─sdc1         8:33   1     4M  0 part
    └─sdc2         8:34   1   512B  0 part

##################

Hi @CaffeineAddict

Thanks for your help!
I'm not very familiar with fdisk (normally used parted) so excuse any fudging.

    $ sudo fdisk /dev/sdc

    Device does not contain a recognized partition table.
    Created a new DOS disklabel with disk identifier 0x9496ff02.

    Command (m for help): d
    No partition is defined yet!

Tried both delete and wipe on a partition

    $ sudo fdisk /dev/sdc2

    Welcome to fdisk (util-linux 2.36.1).
    Changes will remain in memory only, until you decide to write them.
    Be careful before using the write command.

    The device contains 'ext4' signature and it will be removed by a write command. See fdisk(8) man page and --wipe option for more details.

    Device does not contain a recognized partition table.
    Created a new DOS disklabel with disk identifier 0x70187767.

    Command (m for help): d
    No partition is defined yet!

    Command (m for help): w   

    The partition table has been altered.
    Syncing disks.


    $sudo fdisk /dev/sdc
 
        Command (m for help): o      

    Created a new DOS disklabel with disk identifier 0xa534fac7.

    Command (m for help): w
    The partition table has been altered.
    Syncing disks.

Reboot computer, reload fstab.

tried

    $ sudo fdisk /dev/sdc2

and

    $ sudo fdisk /dev/sdc
    Command (m for help): n
    Partition type
       p   primary (0 primary, 0 extended, 4 free)
       e   extended (container for logical partitions)
    Select (default p): p
    Partition number (1-4, default 1): 1
    First sector (2048-16383, default 2048):
    Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-16383, default 16383):

    Created a new partition 1 of type 'Linux' and of size 7 MiB.

lsblk still reports same.

The exact sequence of what I did here with fstab is not perfect, I tried several ways.

Not looking good,
but thank you for the information!




##################
Hi @Egzoset

The risk of data corruption is really only to the USB flash drive itself, which is already corrupted, and at this point, is only a cheap experiment on something of low value.

I don't think a simple USB drive has any risk of damage to any normal USB reader or computer, which I think was your initial concern.

*unless it is a "BadUSB" or one that goes "BANG" such as this
https://arstechnica.com/gadgets/202...-drive-mailed-to-him-it-exploded-in-his-face/

Also, this is not a "fake" or counterfeit USB that misrepresents its size as you suggest it might be, as I was previously using it at full size capacity, per my original post.

Still, your concerns are worth thinking about (made me think a bit!) so thanks!


########
Hi @Trenix25

Hi Matthew

Please see my above reply to Egzoset, which discusses much the same thing.

I think that such weaponized devices as what you are talking about are usually purpose-made for a specific job, target, or purpose, to cause harm and cost to a specific target.

Probably more expensive to make and / or modify such a device than it is to make a standard mass-produced USB flash drive, hence I would guess such things are for rare applications.

I don't think this is something that one is at all likely to find by purchasing USBs from any normal vendor, especially name-brand USB's.

I will look into what you say though, in case somehow it is more common than I think

Perhaps occasionaly PRC manufacturers of no-brand devices roll out such things to cause distress to end users abroad? I could imagine something like that.


"Amazon sells cheap 32 GB USB flash drives, for maybe US$5 a piece if you buy a pack of four or five. It looks to me like it's time to replace your "problem."

I avoid scAmazon like the plague, too evil for me
However, as mentioned, I have found that older laptops often don't SEE USB 3.0 devices on boot, (especially laptops that use USB1.0, but some that use 2.0) and I often cannot use them to install a distro with.

"It looks to me like it's time to replace your "problem."

Yes, maybe

###########

Hi @MikeWalsh

Yeah, it's fairly old

Hasn't seen much use though, but sh*t happens and things do die I suppose.

Your post is extremely interesting and informative overall!!!

Kind of makes me want to research a little history on bodgy USB flash drives and product recalls over things like the class-action you mentioned.

"Last I heard, the two of 'em were locked in a long-lasting court case, Hynix denying all responsibility and SanDisk refusing to back down! I don't know what the outcome was, except that anyone who subsequently made a complaint was automatically offered a free replacement drive...."

Mmmm.... this sounds like a fantastic way to get a few free USB drives

Your last problem is golden information for me.
(suggestions of "just use USB 3.0" are missing the point a bit - and most of my computers are USB 2.0).
I was curious what the reasoning is as to why new USB drives clearly aren't backward compatible to boot from.

"The problems don't stop there, either, because the firmware on later USB 2.0 drives reports them as USB-HDDs.......and the BIOS on many older machines doesn't understand that."

Extremely interesting and useful info also.
Something I will DEFINITELY need to watch out for in future.
You probably just saved me a LOT of wasted time and future frustration with this post.

A sanity-check I didn't know I needed

Thank you, Mike

#########

@Egzoset

Thanks so much again!
That is a lot of really useful tools and information I have never heard of.

In this case, it's not an SD Card, it's a USB flash drive, but what you are saying is still true.

"except i'd still prefer not to risk my internal SD card reader
(no longer present on many new products anyway)... Nonetheless data storage reliability remains a major concern as this defeats the drive's purpose to begin with."

Indeed - that is kind of my feeling.
I cannot be so paranoid as to be afraid of the extremely improbable and miniscule chance somebody has put a weaponized USB drive into a shipment of brand-name USB drives,
as that defeats the purpose of using USB devices to begin with
Strongly agree.

I will research the tools you mention and see if Porteus / Slax etc still include Plop (that's a terrible name for a program! Haha

Thanks!

 

[bob466]

Dear spaz,

By the time it took you write all of the above and preformed all your "tests"...you could have received from Amazon some nice new Flash Drives that actually work...just saying.