cd and chown suddenly stopped working when in a bash script

[rado84]

Until a few days ago both commands were working just fine from within a bash script. But suddenly cd gets completely ignored and skipped by the terminal and chown says the directory doesn't exist. But it does, cuz I created it myself + ls lists it as a directory. The weird part is that if I write the commands manually or even use an alias for chown, they work just fine. Only from within a bash script they won't. I even rewrote the script from scratch and they still wouldn't work. I checked all the filesystems, one was broken, so I reformatted the SSD and now gnome-disk-utility says this FS is OK. I also wrote a test script in my home dir containing just the cd command and it still won't work. I changed the command to change dir on one of the hard disks and it still won't work.
I also made a full system upgrade, hoping that the problem was simply old packages but the version doesn't seem to matter - full system upgrade or older kernel with older packages, these commands still won't work from within a bash script. And yes, I do know that the script must be executable! How else would I be able to use it for months without a single problem and suddenly it won't work?

I made a video, you can see for yourselves. I'm completely lost and have no idea WTH is going on here.

 

[osprey]

Until a few days ago both commands were working just fine from within a bash script. But suddenly cd gets completely ignored and skipped by the terminal and chown says the directory doesn't exist. But it does, cuz I created it myself + ls lists it as a directory. The weird part is that if I write the commands manually or even use an alias for chown, they work just fine. Only from within a bash script they won't. I even rewrote the script from scratch and they still wouldn't work. I checked all the filesystems, one was broken, so I reformatted the SSD and now gnome-disk-utility says this FS is OK. I also wrote a test script in my home dir containing just the cd command and it still won't work. I changed the command to change dir on one of the hard disks and it still won't work.
I also made a full system upgrade, hoping that the problem was simply old packages but the version doesn't seem to matter - full system upgrade or older kernel with older packages, these commands still won't work from within a bash script. And yes, I do know that the script must be executable! How else would I be able to use it for months without a single problem and suddenly it won't work?

I made a video, you can see for yourselves. I'm completely lost and have no idea WTH is going on here.

After watching the video, I wasn't able to see anything I could comment upon. In any case the following is a proof of concept that cd works in a bash script here. The script will cd into a directory, then copy all the files from that directory to another directory. I'm not sure whether your script has these particular characteristics, but here goes:

Here are the content of the directory of interest all run from the home directory:

$ ls
dir1 dir2 dir3 testsh
<snip>

Here is the bash script:

$ cat testsh
#!/bin/bash
cd /home/ben/dir1
cp * ../dir3

The directory which the script will cd into is the following, with its contents listed:

$ ls dir1
file1  file2  file3  file4  file5

The target directory into which the script will move the files is empty:

$ ls dir3

The script is run:

$ ./testsh

There are no errors in the running of the script:

$ echo $?
0

The result of running the script is to place the files into dir3, which has occurred according to the listing:

$ ls dir3
file1  file2  file3  file4  file5

The permissions are as follows:

$ ls -alh
drwxrwxr-x  2 ben  ben  4.0K Feb 18 07:32  dir1
drwxrwxr-x  2 ben  ben  4.0K Feb 18 07:32  dir2
drwxrwxr-x  2 ben  ben  4.0K Feb 18 07:52  dir3
drwxr-xr-x  4 ben  ben  4.0K Sep 24 14:05  docs
-rwxr-xr-x  1 ben  ben    43 Feb 18 07:42  testsh
<snip>

Is that useful? It's hard to know the full nature of the issues without seeing all the details.

 

[rado84]

I didn't give you the whole script bc it's... let's call it "interactive" - it expects user input from which input it creates an archive which then it copies to the external SSD named /BACKUP. But before the archive creation, ownership by my user must be taken, otherwise p7zip refuses to create the archive. Or I'd have to use sudo which complicates everything.

cd /E/BACKUP_IMAGES && sudo chown -Rv rado:rado "$1" && 7z a "$1".7z "$1"/ -mx=0 -mmt=20 && rsync --info=progress2 "$1".7z /BACKUP/LINUX && rm -v /E/BACKUP_IMAGES/*.7z

If you start executing the first two commands from the bash script, well, at least on my computer, cd is completely ignored (as you saw on the video) and it doesn't change to thew new dir. Hence chown not working - cd isn't done properly, so it can't find the dir I've entered. I suppose I could change it to have to receive only the last part of the path, thus cd won't be needed but I'd prefer to find out why cd doesn't work in the first place.

Edit: as I suspected, chown works quite well on its own from the script. So the problem is cd being completely ignored. That I can't figure it out and neither does an AI - I already tried asking, it was only guessing, nothing certain.

 

[osprey]

Thanks for the command. I would be inclined to deconstruct the single line command by writing it into a script with conditionals so that each step of a command taken is dependent on the previous step being accomplished. The inclusion of echo commands for every other command, letting the user know on screen whether or not a command has been successful could be helpful with identifying issues.

 

[rado84]

The normal script does have echoes but I removed them bc I thought they might confuse you. Plus they're in my language and you wouldn't understand them anyway.
AFAIK the double "&" means that the next command will be executed only if the previous one was successful.

 

[MikeRocor]

You are correct about the double '&' - although "successful" only means whether or not it returned a non-zero exit code.

 

[rado84]

You are correct about the double '&' - although "successful" only means whether or not it returned a non-zero exit code.

My observation is that the exit code doesn't matter anymore. If a command returns any errors, there's a break and it doesn't continue to the next. At least it's like that in Arch, IDK about the other distros.

 

[JasKinasis]

I didn't give you the whole script bc it's... let's call it "interactive" - it expects user input from which input it creates an archive which then it copies to the external SSD named /BACKUP. But before the archive creation, ownership by my user must be taken, otherwise p7zip refuses to create the archive. Or I'd have to use sudo which complicates everything.

cd /E/BACKUP_IMAGES && sudo chown -Rv rado:rado "$1" && 7z a "$1".7z "$1"/ -mx=0 -mmt=20 && rsync --info=progress2 "$1".7z /BACKUP/LINUX && rm -v /E/BACKUP_IMAGES/*.7z

If you start executing the first two commands from the bash script, well, at least on my computer, cd is completely ignored (as you saw on the video) and it doesn't change to thew new dir. Hence chown not working - cd isn't done properly, so it can't find the dir I've entered. I suppose I could change it to have to receive only the last part of the path, thus cd won't be needed but I'd prefer to find out why cd doesn't work in the first place.

Edit: as I suspected, chown works quite well on its own from the script. So the problem is cd being completely ignored. That I can't figure it out and neither does an AI - I already tried asking, it was only guessing, nothing certain.

I notice that you ran your script via a right-click context menu in your terminal.
Does that shortcut run your one-liner and pass any parameters to it?
I ask because your bash one-liner uses the first parameter to the script as a parameter to both chown and 7z.
If you aren't passing any parameters to the script/one-liner, then that is almost certainly where it is failing, becaue the chmod command will not have anything to act upon if $1 is empty.

How is this set up? Is this actually in a standalone script? A bash function in your .bashrc? Or an alias in your .bashrc, or a separate .bash_aliases file? I ask because bash aliases do not accept positional parameters like "$1".

 

[rado84]

You saw wrong, watch the video again. I wrote the script's name and then pasted the cd command which I had copied before running simplescreenrecorder. Copy-pasted works, manually written works but it doesn't work when ran from a bash script. I'm beginning to think that the SSD is dying because cd works from scripts when the target directory is on another storage.

 

[JasKinasis]

OK, sorry. I guess I was mistaken.
I admit I can't actually read any Cyrillic based languages. I studied Russian for about 3 months at primary school. But that was ~40 years ago.
I thought the menu item you selected on the right-click was when you ran your script.

I'm not sure what would cause this kind of problem. I've never had problems with things like cd and chmod in scripts. A failing drive could possibly explain the problem.

Have you tried using the +x flag to debug in bash?
That should allow you to see what is failing. But because the code is essentially a massive one liner, I'm not sure offhand whether the bash debugger will echo the individual parts of the line that were ran, or whether it will just display the whole line. I haven't got time to test that ATM.

But for ease of debugging/troubleshooting, it might be worth putting each operation in the script onto separate lines. It will involve writing a little bit more code, but it may make it easier to understand what is failing in the script.

Maybe add some extra "bookkeeping" code to ensure that everything is in the state that you're expecting.
So for example:
Check any paths are valid and that they exist before attempting to use them.
Checking that the previous operation succeeded, etc etc.
e..g

#!/usr/bin/env bash

die()
{
    echo -e "Error: $1\n"
    exit 1;
}

BackupPath="/path/to/backup"
if [ ! -d "$BackupPath" ]; then
    die "Backup path does not exist: ${BackupPath}!"
fi

cd "$BackupPath"
if [ "$(pwd)" -ne "$BackupPath"; then
    die "Unable to cd into ${BackupPath}"
fi

if [ ! -d "$1" ]; then
    die "$1 is NOT a valid directory"
fi

# etc etc....

It may be a bit over the top, but if your script quits and displays an error at every possible point of failure, then when you run your backup script at the backup location that is failing, the script should show you an error message that will clearly indicate where the problem is occurring.

Once you know the exact point of failure, then you can start trying to diagnose why it's failing.

 

[WinterOwl]

Does it work if you cd into any other directory?

Edit: Oh, wait, sorry, I missed the part where you already mentioned that, saying that "cd works from scripts when the target directory is on another storage."

 

[rado84]

Does it work if you cd into any other directory?

If it's via an sh script - no, it doesn't. Only if I type it manually in terminal - only then it works with another directory.

@JasKinasis , your script had a code error which I fixed based on the output:

$ ./jas-test.sh
./jas-test.sh: line 15: [: missing „]“

So I fixed that by adding the missing "]" where it was required.
But after that the script returns it's expecting an integer which is weird cuz it's supposed to only check whether the backup directory exists, it's not supposed to expect integer. If anything, it should expect another dir name (the dir name of the clonezilla's image).

[rado@arch]: ~>$ ./jas-test.sh
./jas-test.sh: line 15: [: /E/BACKUP-IMAGES: expecting an integer
Error:  is NOT a valid directory

I know that the dir name in the video is /E/BACKUP_IMAGES but I decided to change the name with a dash in it, bc I thought that maybe the underscore was causing the problems.

Note that there's still cyrillic text in these outputs, I simply translated it for you. My locale is bg_BG, so naturally, part of the terminal text is in Bulgarian.

 

[WinterOwl]

If it's via an sh script - no, it doesn't. Only if I type it manually in terminal - only then it works with another directory.

Wow, that's weird ah. Going kinda on a whim here, does changing the shebang line do anything?

To add to it: Have you tried testing it in a guest session? Or in any other OS (if you have them installed), or in live USB mode?

 

[rado84]

Wow, that's weird ah. Going kinda on a whim here, does changing the shebang line do anything?

WTH is a "shebang"? TBH to my dirty mind that sounds like something I'm not comfortable to share in a public forum.

 

[WinterOwl]

lol, yeah if you put it like that, haha. Shebang is a hash sign and an exclamation mark (#!) in the first line of the script which is used to define the interpreter that will run your script.

Additionally, I don't know if it matters that much, I just noticed that you use #!/usr/bin/bash. I usually use #!/usr/bin/env bash. I'm going blindly here, lol, so if I had to check, I'd check everything I can.

 

[JasKinasis]

@JasKinasis , your script had a code error which I fixed based on the output:

Sorry about that. It was coded completely off the top of my head on my phone whilst between meetings at work. I had no way to run or test it.
It was more of a snippet to give you a general idea to work from.
Normally I'd provide an entire script example that was tested and guaranteed to work, but I was short on time and in a rush.

Edit:
Looking at my previous post again, I spotted the typo immediately. I'm surprised I missed that!
Ooops! Ha ha!
/Edit

But what I was demonstrating in that snippet is what I'd usually do if I have a script that is failing and the cause is not immediately obvious/apparent. I will code as defensively as possible and will try to isolate every single possible point of failure and output an error message for each, so it is at least obvious where the script is failing. Then it's just a case of working out why it fails at that point.

And once any problems have been ironed out, I can remove some of the checks/error messages and simplify the code.
But checks on things like user-supplied paths is something you should never skip out on. Even if you're the only user of that script! Ha ha!

 

[rado84]

I'm not as adept at scripting as you are. The most of the time I write them myself, test them to see if they work and if they do, I start using them. If they don't, I usually ask an AI for help (bc it answers immediately).
The weird part is that I used this entire script for more than a year and suddenly the first command stopped working. At first I thought it was because I made a full system upgrade with all the latest packages - kernel, drivers and all, but after restoring a backup to the same version I had performed a full system upgrade upon and cd still wouldn't work, that meant the problem wasn't the upgrade.

Unfortunately I couldn't test whether the reason was that the SSD which is mounted as /E is failing because when I removed it, suddenly BIOS/UEFI stopped detecting the SSD where linux is. Or, in the rare cases it did detect the linux SSD, it kept changing the sd identification - once it's sda1, then gets unmounted and later mounted as sde1 or sg2. I had to attach the /E SSD back to the motherboard, so that the linux SSD can get the proper sdid and I can boot into the OS. I've ordered a new SSD which will replace two of the current ones (including /E) and when it arrives, I'll take the computer to the only firm around here that uses linux only, so its owner should figure it out what's going on.

 

[rado84]

Here's an example for a script that works PERFECTLY, even though cd is at the beginning of the script. So at this point I'm lost why some scripted changedir commands work and others don't.

cd /B/PROGRAMS/MULTIMEDIA && 7z a MULTIMEDIA-PLAYERS.7z AUDACIOUS/ HANDBRAKE/ SMPLAYER/ -mx=0 -mmt=20 && mv MULTIMEDIA-PLAYERS.7z /B/123 && cd /B/123 && rsync --info=progress2 MULTIMEDIA-PLAYERS.7z /BACKUP/PROGRAMS

 

[JasKinasis]

If the disk is corrupted or failing, that could explain why cd cannot open the specified location. Most other commands that access the disk or the filesystem could be problematic whilst trying to access that drive too.