Website for non-profit Americares doesn't seem to work using Linux

KGIII

Super Moderator
Staff member
Gold Supporter
Credits
13,144
Yeah, if you want to access it with ESR then you'll want to just use one of the many user-agent switching plugins so that you can spoof a different agent and bypass their security. When you can bypass it this easily, that tells you how much value it really has.

I did some looking and a little over 20% of all web traffic is malicious. That's about right for what I see at my end. The rest is just computers talking to computers and a tiny amount of human traffic. For every human that visits, I must have at least 35 to 50 bots scraping the site or just making sure the site exists or indexing the site or whatever it is they're trying to do.

Some sites are worse than others. My hosting site probably has close to 1% legitimate traffic. There's nothing automated in there, so breaking in wouldn't do them any good. Still, they're like foxes and think there are chickens inside... Or something like that...
 


wizardfromoz

Super Moderator
Staff member
Gold Supporter
Credits
13,656
It doesn't work with Firefox ESR.
That would appear to be the case, David. And top marks to David G @KGIII for going the extra yards.

I have just come in from checking out the LMDE's (Linux Mint Debian Edition) in my stable, namely v3 'Cindy' and v4 'Debbie', same situation with them using Firefox ESR. In both those cases I ran updates to get them up to 78.8.0 but in both cases the previous version also didn't run that site.

So it is not likely to change anytime soon.

I'll be interested to here what Jas might have to say (he is from England, usually in pretty much daily).

Wiz
 

KGIII

Super Moderator
Staff member
Gold Supporter
Credits
13,144
And top marks to David G @KGIII for going the extra yards.
I cheated.

I opened the web developer console and saw something about cookies.

This made me aware that they were intentionally pushing the traffic to the 404 error page. So, the first tool to avoid that is to pretend to be something else. I installed the first user-agent spoofing tool I saw in the add-ons and tested my theory.

It loaded immediately.

That was my troubleshooting steps. It helps that I understand how IPS, anti-bot, anti-DDOS tools work.

(I explain so that folks can see the troubleshooting steps. In this case, not many nor anything terribly difficult.)
 

f33dm3bits

Gold Member
Gold Supporter
Credits
15,054
Alright, I've got it.

It blocks Firefox ESR at the server level.

Grab one of the many user-agent spoofing extensions, set it to a different browser (I picked Chrome on Windows 10) and it works like a champ.

There's no other real choice, except to use a different browser. It's done on the server side.
Good catch @KGIII!! Didn't even think of that, the Firefox on my main machine uses the Firefox/78.0 user-agent instead of the original Firefox 86 user-agent(Firefox/86.0) as the one downloaded from the Firefox website.
 

wizardfromoz

Super Moderator
Staff member
Gold Supporter
Credits
13,656
@david.kit.friedman - so in summary then
  1. It is a Firefox ESR issue
  2. You could employ user-agent switching plugins so that you can spoof a different agent and bypass their security, and if you pursue that course, I know @KGIII would assist if you need it
  3. You could install a different Browser, which might include Google Chrome since you are familiar with it, or its free and open source equivalent is Chromium
  4. You could completely remove Firefox ESR and install in its place "regular" Firefox
  5. Other - as yet undetermined
If you choose Option 4., you would likely want to consider using Option 3 as well, so as to not have any downtime from the Internet access.

And I would install Timeshift and take a snapshot you can use to roll back your system if need be.

Let us know what you think and keep us posted.

Wizard
 

JasKinasis

Well-Known Member
Credits
5,977
Late to the party, but the website doesn't work for me using Firefox-ESR on Debian either.
But it works using chrome on Debian. So it seems to be some kind of user-agent related thing. Their server, or some other service protecting the server seems to be checking the user-agent before allowing/disallowing connections. It may be that Firefox-ESR isn't in their list of allowed agents. Or perhaps they are explicitly blocking it. IDK?!
 

wizardfromoz

Super Moderator
Staff member
Gold Supporter
Credits
13,656
I've checked the site through Firefox, Waterfox, Chrome, Chromium, Vivaldi, Slimjet and Brave across half a dozen distros (I run 67 ... today), and it works fine.
I find it curious that Waterfox, Vivaldi, Slimjet and Brave can access it but Firefox ESR can't.

The 4 I mention are not so well-known, unless something to do with the version number for ESR throws a spanner in the works.

Curiouser and curiouser, said Alice.
 

KGIII

Super Moderator
Staff member
Gold Supporter
Credits
13,144
In the ESR case, it could be that it appears to be an older version of Firefox, a not-current version, which makes it unusual or look like a bot because bots will still have the hard-coded user-agent from when they were written. Those browsers may send their respective parent user agent strings, or at least may be current versions.
 

wizardfromoz

Super Moderator
Staff member
Gold Supporter
Credits
13,656
Yes that is what I was opining, but you talk dirty better. :)
 

david.kit.friedman

New Member
Credits
59
@wizardfromoz

I just occasionally donate to Americares, but don't access the website on a regular basis to get information. So I'm just planning to continue on without making any additions or changes (also when I did a Google search the ad link to donate still worked in Firefox ESR).

But I think moving forward I can be more aware that some sites will not work with Firefox ESR. I would still think that most sites would be okay, but maybe if it doesn't work that's because the server doesn't recognize the browser.

And actually this new found knowledge already seems to have helped.

On a number of occasions I've tried with Firefox ESR to use a publicly available proxy found say via a Google search: "web proxy". But they didn't seem to work right.

What I was trying to do was to see what the result would be for a Google search for other people -- to try to get a sense of how Google is sorting information in response to queries for people in general.

And so I now just tried ProxySite.com using my Android phone, and it worked okay (I also used Google Chrome Incognito mode). What I wanted to see is what would come up as a result when doing a search for say: Americares site doesn't work in Firefox.

And so this very thread came up for that search. :D

So I couldn't know exactly what would come up for every single different user, but perhaps it'd be the same result.

That way if somebody is having trouble getting to the site for that reason if they do a Google search it could help them to figure out what's going on, and to either use a different device, a different browser, or install something which changes the User-agent information.

I just found this one which looks to be reasonable:

User-Agent Switcher and Manager by Ray
 

wizardfromoz

Super Moderator
Staff member
Gold Supporter
Credits
13,656
That looks interesting, David, and ta for sharing (in Australia and the UK we say "ta" for thank you, pronounced "tar") :)

Although the difficulty with access may have been a PITA for you, I found it extraordinarily interesting.

I have been using Firefox ESR, not as my go to browser, but across a number of Distros, for several years now, and I had not seen this phenomenon before. Nor had Jas, by the sound of it, and he is a Debian man who uses it all the time.

So it's good that we can learn a new thing every day, and that is only one of the many things I enjoy about Linux.

Enjoy your Linux, Friday here in Oz, so

Avagudweegend

Chris
 
$100 Digital Ocean Credit
Get a free VM to test out Linux!

Members online


Top