Discussion: Anonymous traffic limit trial

Anonymous traffic limit trial

We are trialing anonymous use limits to help with server load. We believe that the performance issue is primarily caused by excessive anonymous traffic which we think a lot is not genuine.

Some initial tests showed a significant reduction on server load.

Please feel free to provide your feedback as we trial this potential improvement.

Good move. Does seem better, it's been way slow logging routes via a challenged mobile connection lately.

Hmmm, I have quite a few 500's that disappear with a refresh now.

For what it's worth, this forum thread didn't load the the first time I clicked the link on my dashboard. I measured 2.5 mins from I when appreciated the irony enough to start timing, it was probably ~3 minutes total. Firefox said there was no network traffic during the last minute.

It worked immediately when tried a second time with developer tools open to try and collect some more useful data for you.

Firefox 109.0 with Ublock Origin on Ubuntu.

I did a bulk edit on this node node: https://www.thecrag.com/en/climbing/germany/area/5491058940 and waited minutes until the form was saved and the browser returned to the index.

I've been getting sporadic "Internal Server Error"s since this change. I reported one on github, but just now I also got it on the home page.

Same here, many pages only load correctly after reloading a couple of times.

Christoph Rauch, Danny van Bruggen: I have created a github issue for the 500s: https://github.com/theCrag/website/issues/4184

It also seems that activity feeds for anyone who doesn't follow me are broken. https://github.com/theCrag/website/issues/4183

Hmmm it is not obvious what is going on from our side of things. If you are logged in you should not be effected by the limiting logic. We have got one theory we are testing.

Seems to be killing shared links from Facebook too...

Ahh, this is a problem. We will have to fix this.

At least on my phone I don‘t see any 500s anymore. But instead, the graphs in profiles take quite long to load. Also, the issue with requests taking much too long still persists. I often need a few attempts/clicks to even get to a page (doesn‘t matter whether it‘s index or profile or anything else), some requests just load forever.

When I get through, I noticed that the history stream is already preloaded for a long stretch of time into the past.

Links from FB shares are working again. My dashboard took an age to load but I guess you will have flushed some caches.

After clicking around randomly for a bit the performance seems to be hugely improved.

And I haven't seen any 500 errors since Simon Dale's post.

Good move, also looking forward to a hopefully much more response site now! Haven't used it enough recently to say whether the previously common occurrences of several minute-long loads have now been reduced. Haven't seen any HTTP500.

Sergey Komarov FYI, previous discussions here https://github.com/theCrag/website/issues/3976

Software engineer here. Gonna be blunt: this smells like bad software design. There's no need for this kind of rate limit in a properly engineered website. Cache all the content as seen by logged-out users and serve it to these anonymous visitors from Cloudflare (or any other CDN network) . Your server will breathe freely.

Seems like the changes have improved the site performance A LOT. Thanks and congrats for the actions taken 👍

No detailed measurements, but I would say that it is not better than it was and maybe even slightly worse. Europe, with very fast and stable internet connection.

Actually in the last few days the site working for me so smooth

Haven't done any editing lately, but I haven't had a wait for a week or so now.

Is "TheCrag" still looking into this? For me it feels i sometimes get a rediculous delay, while other times the site works as excpected.

There is still a lot of load, but the 32 server processes are no longer at 100% all the time.

Our load is still peaky and there is no way of us managing the peaks, so maybe this is what is going on for the occasional delay. Or it could be something else.

Whatever, the situation is greatly improved, but I still think we have we need to keep investigating and looking into ways of further improving.

I confirm Andreas Aachen, still sometimes unusable long delays (browser timeout) in germany. It's so annoying that i stop contributing in these times at all.

Yeah somewhat slow still.

Same for me.

During the last week its been very spotty / laggy again with lots of timed out requests.

Yeah, here too.

Yesterday it was also a nightmare on my side.

The mobile internet is not the best at my current location, but I didn't have issues with other pages.

Is there a way to download the entire theCrag content of a region to the phone?

More and more often i am at the wall and want to look something up and i get minute long lags, which makes it a bit unusable for me :-/

You can use the download/offline feature of the browser. But this will only download the overview without the individual route pages 🤷

Having the most important parts open as a tab in the browser is what I'm doing as well. Shouldn't the Topo Guru app have an offline function as well, or was that only if you pay for it?

PDF guide download may be an option

There is the html guide as well.