Command to screen files containing specific line

satimis

Member
Joined
Jul 9, 2021
Messages
53
Reaction score
8
Credits
557
What will be the command line?

If running "find ./ ".htaccess" finding a list of .htaccess files, I need to select those .htaccess files which contain the line "Deny from all"?

Thanks

Regards
 


JasKinasis

Well-Known Member
Joined
Apr 25, 2017
Messages
1,321
Reaction score
1,870
Credits
8,343
Use this:
Bash:
find ./ -type f -name ".htaccess" -exec \grep -Ril 'deny from all' {} \;
The above command will give you a list of .htaccess files that contain the string "deny from all", using find with it's -exec option using grep to perform a case insensitive search via grep.

In the options for grep, -R means recursive, -i means use a case-insensitive search and -l means "only list files that contain the search pattern".
{} is a placeholder used by find which will be replaced with the paths/filenames of any files called ".htaccess" that find finds. And the \; represents the end of the parameters to the -exec command.

And I've deliberately used \grep instead of grep. The backslash version escapes any aliases that might have already been set up for grep - so grep will only use the parameters that we specify.


And if you want to see the exact lines that match in the files, you can do this:
Bash:
find ./ -type f -name ".htaccess" -exec \grep -RiHn --color 'deny from all' {} \;
That will show you where all of the matches occurred in the files that were found.

In the above, the -R and -i in the options to grep are the same.
The -H option will show the path/filename in the result.
The -n option shows the line number in the file where the match occurred.
And the --color option will highlight the search-pattern in the results that are returned.
 
OP
S

satimis

Member
Joined
Jul 9, 2021
Messages
53
Reaction score
8
Credits
557
Hi JasKinasis,

Both your command lines generate a long list of files. I have to stop the command pressing [Ctrl]+c keys

I think I have to save the printout to a file and read the content afterwards

What command lines I have to execute finding files containing following pattern;
<FilesMatch '.(php|php5|suspected|py|phtml)$'>
Order allow,deny
Deny from all
</FilesMatch>


Thanks

Regards
 

JasKinasis

Well-Known Member
Joined
Apr 25, 2017
Messages
1,321
Reaction score
1,870
Credits
8,343
Both your command lines generate a long list of files. I have to stop the command pressing [Ctrl]+c keys

I think I have to save the printout to a file and read the content afterwards

Wow, how many websites are you dealing with?

If you wanted, you could redirect the output from my commands to a file and then view the file afterwards?
And to free up the terminal whilst it's running, you can run the command in the background.
e.g.
Add the following to the end of both of the commands I posted previously:
Bash:
> /path/to/outputfile &
But replace /path/to/outputfile with the path to save the file to. And I'd suggest using a different file-name for the output of each command, because > will overwrite the output file if it already exists!

With the above modification, you can continue to use the terminal whilst the job runs in the background. And when the job has finished you'll have a text file that you can view with the results in it.

What command lines I have to execute finding files containing following pattern;
<FilesMatch '.(php|php5|suspected|py|phtml)$'>
Order allow,deny
Deny from all
</FilesMatch>

Offhand, you could write a script which uses find and grep to create a list of files containing a basic pattern, similar to my previous examples. the list could be written to a temporary file. Then use sed on each file in the list to find instances of the multi-line pattern you have specified.

Which would look something like this:
Bash:
#!/usr/bin/env bash
tmpDir="$(mktemp -d)"
tmpFile="$tmpDir/tempFile"

find ./ -name ".htaccess" -type f -exec grep -Rl "php\|php5\|suspected\|py\|phtml" {} \; > "$tmpFile"

while read -r filename; do
    echo "In $filename:"
    sed -n "/<FilesMatch .*php\|php5\|suspected\|py\|phtml.*>\nOrder allow.*deny\nDeny from all\n/,/<\/FilesMatch>/p" "$filename"
    echo
done < "$tmpFile"

rm -r "$tmpDir"
Save that as whatever you'd like to call it. Make it executable using chmod +x /path/to/script, where /path/to/script is whatever you called the script.
Then run it.

The above script uses mktemp to create a temporary directory in /tmp/. Then we set up a filename for a temporary file to store in our temporary directory.

Then we use find to find all .htaccess files, and uses grep in find's -exec section, to determine whether any of them contain the basic file filter you're interested in. Any .htaccess files that match are written to our temporary file, in the temporary directory in /tmp/.

Then we use a while read loop to read each filename in the temporary file and use sed to find the multiline pattern you're interested in and show them on the screen using sed's p (print) command.
Finally we use the rm command to recursively remove the temporary directory and temporary file that we added to /tmp/.

Running the above script will yield output that looks like this:
In /path/to/website1/.htaccess:
<FilesMatch '.(php|php5|suspected|py|phtml)$'>
Order allow,deny
Deny from all
</FilesMatch>

In /path/to/website2/.htaccess:
<FilesMatch '.(php|php5|suspected|py|phtml)$'>
Order allow,deny
Deny from all
</FilesMatch>

And if you have a lot of websites, you may have a lot of lines that look like that. Also, if there are multiple occurrences in a file - you'll get multiple copies output too.

So when you run the script, you may want to consider redirecting the output to a file so you can take a look at it.
e.g.
Code:
./script > /path/to/outputFile

Optionally, you might want to redirect and run it in the background, as a background job, to free up your terminal:
Code:
./script > /path/to/outputFile &
Or perhaps pipe the scripts output through a pager like less, or more, or a terminal based browser like w3m.
e.g.
Code:
./script | w3m

However you decide to run the script, if you're happy that the script has picked out the correct lines in the files, the next thing you could do is modify the sed command in the script to delete the offending lines from the files:
Bash:
sed -i -n "/<FilesMatch .*php\|php5\|suspected\|py\|phtml.*>\nOrder allow.*deny\nDeny from all\n/,/<\/FilesMatch>/d" "$filename"
In the above snippet, we've added sed's -i option (in-place editing) - which will directly edit the files.
And instead of using sed's p (print) command to print the lines, we're using the d (delete) command to delete the matching lines.

So if you edit the sed command in the above script to do that, the output will look like this:
In /path/to/website1/.htaccess:

In /path/to/website2/.htaccess:

etc etc.....
But those are just from the echo commands in the script.
It might be useful to leave the echo commands in there, so you can see which files were edited, but optionally, you could comment them out, or delete them.

Also, it's worth noting that because the modified sed attempts to modify the .htaccess files to remove the matching lines, you will need to run the script as root. Either directly logged in as root, or via another mechanism like sudo, or wheel (depending on your distro).
So if you get any errors about permissions after editing the sed command to delete the matching lines from the files - you will need to run the script as root.
 
Last edited:

wizardfromoz

Administrator
Staff member
Gold Supporter
Joined
Apr 30, 2017
Messages
6,629
Reaction score
5,394
Credits
19,658
OP
S

satimis

Member
Joined
Jul 9, 2021
Messages
53
Reaction score
8
Credits
557
:) Hi, Jas

From another thread...

https://www.linux.org/threads/comma...ing-and-save-output-to-file.35493/post-128259

More than 40 sites, it appears.

Cheers

Wiz
Hi Wiz

The complication here are;
1) All my live websites are running on the server of the hosting company

2) I run commands on the cPanel Terminal of the hosting company. Only limited Linux commands and software are installed there and I'm not allowed running all of them

rpm -qa
showing all of the them

I'll store the scripts, created by myself, on local PC. Although I can upload them to the server of my hosting company, but it makes the thing even more complicate and opens a loophole easily to be attacked.

3)
I don't know whether I'm allowed adding Linux software on the server. I'm subscribling shared hosting

Regards
 

wizardfromoz

Administrator
Staff member
Gold Supporter
Joined
Apr 30, 2017
Messages
6,629
Reaction score
5,394
Credits
19,658
Right over my head, mate, but it may be useful to Jas.

Good luck :)

Wiz
 
OP
S

satimis

Member
Joined
Jul 9, 2021
Messages
53
Reaction score
8
Credits
557
Wow, how many websites are you dealing with?

If you wanted, you could redirect the output from my commands to a file and then view the file afterwards?
And to free up the terminal whilst it's running, you can run the command in the background.
......
Hi JasKinasis,

Lot of thanks for your continue support.

If I understand your advice correctly I need to create a script file with following content on it.

script;
<FilesMatch '.(php|php5|suspected|py|phtml)$'>
Order allow,deny
Deny from all
</FilesMatch>


Can I save the script on local PC? I'll run the script on the cPanel Terminal of my hosting company although I can upload it to their server.

Do I need to run chmod as;
$ sudo chmod 777 /path/to/script
?

Wow, how many websites are you dealing with?
I would screen one website (a suspected attacked website) one time, not all 40 websites collectively.

Or perhaps pipe the scripts output through a pager like less, or more, or a terminal based browser like w3m.
I prefer to run the command on background and check the output on outputFile file after finish.

However you decide to run the script, if you're happy that the script has picked out the correct lines in the files, .....
Pls explain "script has picked out the correct lines in the files" ?
Whether you meant picking out the exact pattern as on the script file?

sed -i -n "/<FilesMatch .*php\|php5\|suspected\|py\|phtml.*>\nOrder allow.*deny\nDeny from all\n/,/<\/FilesMatch>/d" "$filename"
Pls help me to understand their functions. Thanks

Regards
 
$100 Digital Ocean Credit
Get a free VM to test out Linux!

Members online


Top