You have a good point. You also need to understand that UI/UX design has very low barrier to entry and is not so difficult.

Please note that I do not support the ageism but that is the reality we live in. Now how can you navigate past that? Imagine getting into a leadership role and managing/mentoring 10 UI/UX designers. That would be a solution to the problem rather than denying it exists and suffering the consequences.