Discussion: Two main problems of TheCrag

Two main problems of TheCrag

I really love this site and I am a happy contributer but I see still two main problem:

a) performance: this is still a big issue (here in Germany I often get no answer for over 30 seconds).

b) mobile app: this is in my opinion a real game changer problem. Sorry but Topo Guru is no real alternative. It is just bad. Compare this to the 27 crags app. You mainly need a good Topo view and ability to tick with some caching mechanism. All other would be nice to have. And I think if there would be a good app it is possible to monetize it like it cost a fixed price or only works if you a supporter...

Hey Martin,

Good post. Yeah, the slowness of this site has been a point of ongoing discussion for some years. There are/have been at least a couple of discussion threads about the website speed vs slowness, perhaps someone can link them in here?

☹😑

Maybe we all should be payed members to support the site and compute resources should be easier to pay for the site owner.

I agree, Topo guru isn't a good enough app yet, it does a barebones job compared to others. It has the potential to be amazing if updated and expanded upon, we will see if that happens.

As a paid member, I would like to say I highly frown on the concept of paying in advance for a feature that doesn't yet exist. A product needs to prove that it's worth paying for. With that said, I love The Crag, and I use it all the time, so I still feel like I get my money's worth.

I do agree though that the site does have some hang ups with performance, and it would be nice for this to be improved.

The crag is the best

I don’t get why thecrag doesn’t open source and let other software engineers contribute. It’s a community project after all. There are quite a few bugs that have been around for years that could easily be fixed by motivated software developers.

As for the performance of the site, I’d go for a premium model where paying members get their content delivered via a CDN and backed with cloud native backends that scale on demand. I’d be more than happy to pay a premium for that.

I agree on both points. I've learned to live without the mobile app, but performance is an absolute PITA.

Dmt Bolaños +1

The Crag is a volunteer initiative , not a resource to be plundered.

I have no idea where the 'plundering' is coming from. What are you trying to say? Many of the people 'complaining' here have the Karma to show that they are contributing back not just by donating but also curating.

Of course it's a volunteer initiative. No one has to do anything if there isn't time. But why not point out the most pressing pain points? Especially since the Github issue treats problem a) as more or less solved.

Performance has been mostly OK for me, but yeah I have also seen slow days.

A mobile app could really be a gamechanger. If you compare theCrag to Mountain Proj, which is its main "competitor" esp. in North America, they have a pretty good app that even allows you to store entire crags on your phone for when you go somewhere without reception. That would be great for theCrag.

For reference, I am a paying supporter and have also contributed quite a lot to add and update climbing areas.

Eww nobody wants to plunder a big ball of Perl files ;-)

Im speaking of the crag. The place where climbing happens.

I totally agree with Tommy Sunshine, open source is the way if the burden of maintaining an application is to high for the budget of a small company.

Just an app would be amazing. Cache your favourites. Would be an absolute game changer for users.

If you use a Chromium based browser on Android (like Brave or Chrome) you should be able to download the Topo for offline:

- image-1711581995991.jpg

Make sure you expand everything before you do though, because non of the interactions work when you're offline. Agree it'd be great if thecrag was open source though!

I agree that (lack of) responsiveness is a major issue with thecrag, it was much better when I started using it. Cost of success?

Also, yeah, the per-route modal route ticker adds to the pain. I was happy to bulk-tick my routes before, and now it is much slower, even without data-loading issues. Wish we could go back, or, at least, have a "bulk tick" type interface.

I would say that the bad performance for people around Europe is a big problem. This personally drives me away from logging my activity on thecrag and makes bigger edits a bit painful. I still use it to make guides and find information but day to day use is a no go.

I can see why that is since the company is based in Australia and majority of the community is based there. My guess is that is where the servers are located as well.

We would greatly benefit from Europe mirror of the servers guys, please!

For me the webpage resolves to a server in Germany. It's the main page which loads 10-30s. Some static content is hosted on CDN, they may move more be hosted from CDN but the first thing to optimize is the dynamically rendered content. Admin: I'm happy to help looking into it if you want

You are right, I'm getting various addresses from US/Germany. But still that does not tell us whether the data is being pulled from regional server or some instance far far away. I bet is the latter.

Amusingly, this page took upwards of 30s to load just now...

In regional nsw, australia, it takes about 0.5 seconds. However i do not use an app.

Hi, Crag support team still there or not? Bsc I give an email to them regarding topo adjustment. Not reply till today since last year.

> Hi, Crag support team still there or not? Bsc I give an email to them regarding topo adjustment

Nabilaa Mohamed

Have you tried reaching out to the community through the forum?

If it's about configuring topos and areas - as opposed to a feature request - then there are probably a lot of people that can help out immediately. And they don't have to be theCrag admin.

Germany, Greece, Spain....all painfully slow these days to the point that I revert back to other sides. Which is such a shame as I love thecrag

Here in austria I never had speed/performance issues. Everything works fast and responsive.

This just shows where are the data centralized and pulled from. It's Australia only. As I said, I hope we can get an European mirror or at least a cache server.

please read my comment again

My last information is that the servers are located in Europe. Even if the servers would be located in Australia, it would not explain or excuse the lag of responsiveness.

Obviously resources and money are the show stoppers. I assume the source code is as old as the website itself. Maybe it's just too much work to get it fit for the current user base.

In the past I proposed to open the source code to gain from the community, but it didn't happen and I don't know why. I think theCrag is about data, not the software that delivers it. So I don't know why it should be a problem to open the sources.

+1 for painfully slow response. Right now it seems normal from italy but in the past year or so I almost stopped using theCrag because of it. (Germany)

Florian I live in Austria right now and I can confirm that your statement is utter, total, complete bullshit. It’s as slow as in the States or everywhere else for that matter in Austria as well.

+1 from Germany. Although it's a nice flashback to the 90', waiting 1-2 mins for a crag to load is a real downer.

+1 Tommy Sunshine Florian has to be trolling…

Hey guys, come on, no need to rail against one another, just because local or national networks vary.

I changed telcos recently, and that has vastly improved my access speed to thecrag.com. 😑

I agree with Kai - distances to servers are very unlikely to cause the ongoing performance issues. Does server distance even cause minute long delays in 2024?

I encourage those blaming geography to measure their connection's latency to some of AWS's datacenters around the globe. What range of times do you get? Is the difference between Frankfurt and Sydney enough to explain the delays you're seeing on theCrag? My fastest and slowest were US East (N. Virgina) at 0.088s and China (Ningxia) at 0.284s. Yes, farther regions were slower but the difference is ~0.1 seconds, not multiple minutes.

Furthermore, the performance is not consistent across pages. I often have one page repeatedly timeout while others successfully load in another tab. This leaves me unsurprised that Florian has had a different experience from others.

It seems to me that time of day is a more plausible explanation for any apparent geographic dependence. Or weirdness of specific ISPs, as suggested by Mark Gamble. Or the database struggling with popular crags.

Florian, could you share a little about how and when you're accessing the website? Maybe I can submit edits more efficiently by emulating you.

The delay is real. No other website I use has any sort of delay - thecrag is entirely unique in this aspect and honestly does feel like a step back in time 20 years. I can sit at a wildwrness cliff in Sydney, Australia and watch real time 4K videos on youtube using the cell network - but thecrag can often take 20+ seconds to load a single page of text. It's not the worldwide internet to blame - something is seriously wrong with the site itself.

What is going on with the performance of the site tonight. Everything is running like 1000 times faster. AWESOME !!!

I don't know if my specifics can help shed some more light on the access issues with this website, but I'll expand my post above:

Last December I changed telco's. Prior to that, I had used one for about 15yrs. (I don't see a need to name then here.) Using that previous telco (ISP), trying to access thecrag.com on my mobile phone was often a nightmare. The phone would literally time out after about 45-60seconds & I'd get a message saying: "server did not respond."

Persistence would eventually pay-off. Since I changed telco's last December, I haven't had an issue accessing thecrag.com on my mobile. The longest I've waited since, has been about 5-6 seconds.

On other occasions, I will access thecrag.com from a library computer, and on these occasions, I've never waited longer than about 3 seconds max, to access the website, or any of its pages.

😶

Agreed on both points

Ehm, I'm sitting on a fiber here with unrivaled performance within my region. The ISP problems are completely out of the question yet I'm still experiencing bad performance from thecrag. And requests sometimes taking up to a minute+ to complete.

People that argue that 200+ms latency is not a problem probably never worked with any networked solution in their life and don't even know that packet loss exist. Do the 200+ms call there and back multiple times and you'll eventually hit 1s, then loose the packed couple of times (which is very likely with big payloads) and you can even produce a call that never gets delivered. It all comes down to how many hoops the data have to jump between the user and the backend.

I'm not here to leash out on others but if something works for you do not directly assume that something works for others the same way. In Australia I'm certain everything works very well. In central Europe... not so much.

I would love to see thecrag utilized by more people in my community and this is one of the reason I'm having hard time recommending it and spending more of my time producing content on the platform.

Once the performance issues are gone thecrag is going to be a viable option for a lot more people. Till then I'll continue supporting and crossing my fingers.

Performance issues still exist Australia wide - we are not immune.

Looks like the performance is an issue for a lot of us. Support theCrag plz provide feedback when this will be fixed. Are there any efforts on improving performance? Is the root cause know? Do you need more information from us?

For those of you having issues and wanting to report something, you'll provide the most bang for buck by getting Google PageSpeed Insights, and/or Cloudflare Radar to do some analysis on the site at the time you're seeing slow response times.

Drop the link to your specific point in time analysis here into this thread and/or report it to Support theCrag

e.g. here's some from earlier today, no major delays at the time, just the usual minor lag, though as you can see, some room for improvement.

https://pagespeed.web.dev/analysis/https-thecrag-com/ejdc4xcjg7

https://radar.cloudflare.com/scan/c7d06ab4-07c1-44da-88ba-7f2c675b7927/summary

The results from pagespeed for crags and routes (instead of the main page) are quite bad 😅

https://pagespeed.web.dev/analysis/https-www-thecrag-com-en-climbing-spain-lleida-area-route-260107860/jx693ryonv?form_factor=mobile

https://pagespeed.web.dev/analysis/https-www-thecrag-com-en-climbing-spain-lleida-area-area-260078058/ke1exfptae?form_factor=mobile

Colin

In my subjective experience, the worst loading times are when editing content (crags, photos, topos, ...).

However, all of these interactions are for logged in users only. This pagespeed tool has no means of measuring those interactions, because it cannot use my logged in session, can it?

e.g. I was trying with this URL, and it looks like the analysis is blocked by the login screen

https://www.thecrag.com/processmap/bulkedit/102868617?select=0&expand=children

- image-1712165765603.png

https://pagespeed.web.dev/analysis/https-www-thecrag-com-processmap-bulkedit-102868617/ecj9akmkr1?form_factor=mobile

We know we have some performance issues. We have resolved some of these issues and are continuing to work through them. One of the underlying issues is increased load, which is partly caused by bots, but in the case of Easter a big increase in genuine traffic. There are some specific additional issues that we are trying to track down though. Since last year we have made many fixes to improve the situation, including something during Easter.

If you find the site slow on the first page view for an area then this could be a cold cache.

If the performance is fine then suddenly gets bad for a small time then gets good again then this is something we still have to track down.

Ultimately we recognise that we need to make a major infrastructure upgrade. This is planned for the second half of the year.

One observation I've had: load time for areas with many routes is anecdotally far slower. And in particular, areas with many topos/rich topo data.

For example, this area takes 10-20 seconds every time for me (over many days/weeks, so I doubt it's a cache miss every time): https://www.thecrag.com/en/climbing/australia/mt-coolum

As opposed to other areas, which are <10s.

If I had to guess, it could be a combination of:

• no pagination on the routes query; all routes in the area are returned.
• Topo data for all routes blocks the initial response (all the topo data is loaded server-side, not async on the client)

I wonder if fetching topo data client-side (similar to the area bounding boxes) would help 🤔

Re: b) I noticed people mentioned Topo Guru, but not the Chalk App. Also informed by The Crag, much better than Topo Guru (at least I think so.)

Still not great, still not a substitute for a true TheCrag app but it's good for getting info on your day out.

Also admins: we love you, keep up the good work!

Make screenshots before going outside, works for me

I’d love to see thecrag open sourced.

The website is slow for me in Australia, perhaps the backend needs to be performance profiled?

This is something the community could do, there are SO many software engineers who climb.

Simon Eschbach https://github.com/theCrag

Jonathan Bright As far as I can see, thats just an issue tracker and a few other little tools for using the api - not a repo for the open sourced code for the website.

Open sourcing the solution is not going to solve the issues. It might even introduce new problems and challenges.

I would say that I'm happy to hear that there are plans and enthusiasm for the future of thecrag. I understand that it is not an easy task to scale the project up.

With that I would like to offer help if the development team would be interested in getting a volunteer on board.

I'm a developer based in central Europe with about 9 years of professional game development experience. I optimize and rebuild existing networked solutions for living. I would be very happy learn what you guys are dealing with and help to build it better.

Yep, once there was code there, there's now nothing but an issue tracker that may as well have a stale bot just mark everything as done. While it's not planned, while they chase bad UI 'features'

OpenBeta.io may get traction. For the moment at least my effort will be duplicated there.

Of course they'll likely sell out to Outside, like all the other similar projects in MTB and trails of the last decade

Luky How does open-sourcing not solve any issues?

Because it won't immediately resolve the performance problems, as they are primarily an architectural issue stemming from a lack of paying users contributing enough money to pay for the necessary compute and development time in order to improve the architecture and as deployed architectural reality.

There is also the potential to create more problems in terms of bugs/vulnerabilities.

"Just open source it because it will fix all the things" is a dumb approach.

Exactly as colin says. Letting people in means you have to implement a curation structure to prevent people from contributing malicious or bad code. It comes with a set of challenges that are not simple to deal with and it does not mean that anyone is going to go and fix the issues in the first place. Or will be able to.

Saying that open sourcing will solve the issue is like calling the current maintainers incompetent which they are obviously not.

I'm of the opinion that well commercialized software with closed sources with in-house maintained infrastructure will outperform in quality and availability most of the open source. Especially when it comes to Saas services.

The thing just has a clearly defined owner who usually cares for the product. Either to make money or else. I've seen way to many great open source projects fail due to lack of ownership, motivation and funding issues.

I'm not saying there are not great open source communities producing great software (looking your direction blender foundation) but I can for sure tell you that open source is hard and even more complicated than closed source in many ways.

Luky if you are interested in contributing I think the best is to send a mail to the team to support@thecrag.com, you may need to sign a contract with them. Btw are you anyhow related to Factorio development?

You are right, that is probably a better way of doing it.

I'm not involved in Factorio but I know a couple of people that are. Its a really nice project and great people behind it. During a period when I worked on Space Engineers we had offices in the same part of Prague and regularly met during lunch times and conferences.

> There is also the potential to create more problems in terms of bugs/vulnerabilities.

Security by obscurity.

Would there be a chance to build an app from the community? Maybe we have some designers, app devs, it guys here in the community and we can do this together?

> Would there be a chance to build an app from the community?

We are open to a community-driven open source mobile app! But it needs a proper discussion and some key people.

It is very likely that there will be many months of planning, discussions and scoping before the first line of code is written.

Nicky than better start now than later. I would like to be part of it. (20 years experience in frontend and backend dev). Every large adventure starts with the first step...

I think TheCrag already had at least three unsuccessful app attempts, the requirements are pretty high because of all the data here. But I don't know enough about this problem, maybe the framework options nowadays are much better. Personally I find the site pretty good on mobile so I don't need an app.

I recommend giving the whole podcast with TheCrag team a listen, Simon talks about apps at 26 minutes in

> Would there be a chance to build an app from the community?

IMHO there is no point in building an app as long as the backend performance isn’t right. And if I’m not mistaken, the one thing this community agrees upon is that performance could be better.

I am in no position of judging the development of theCrag and I respect you guys for your work. But if those performance problems have been around for a while and efforts from the team haven’t gotten a breakthrough yet, maybe some new pair of eyes could help?