Innovation versus קֶבַע in prayer

It's well-known how the rabbis of the Talmud said one's prayer should have כַּוָנָה, intent, behind it, and not be a fixed routine (קֶבַע). It's also well-known how over the following generations, one rabbi's personal freeform wording ended up becoming the next generation's fixed prayer. (A well-known example is the paragraph after the Amidah, אֶלֹהָי נְצוֹר, which started out as the personal meditation of Mar Rabina.) In general in early times you could use your own wording for many of the prayers, provided you were knowledgeable enough about what to put in it, so long as you finished with the correct חֲתִימָה (conclusion) to each בְּרָכָה.

What bugs me is: how did we come so far from this that the law codes (I forget which) now say that the prayer of anyone who changes the wording they have inherited from their forefathers will not be received, to the extent that even if one is eating alone, one cannot say וְעַל שׁוּלְחָן זֶה שֶׁאָכַלְתִּי עָלָיו "[May the Compassionate One send many blessings...] upon this table I have eaten at", but must instead say שֶׁאָכַלְנוּ עָלָיו, "...that we have eaten at"?

How in any way is this sensible? It strikes me as just plain wrong. Yet I learned this not from a meshugge-machmir chareidi rabbi, but from a senior Masorti rabbi, on account of which I feel uncomfortable disregarding it in a way I would not had I learned it from the former. (True, the Masorti movement in the UK is divided between those who adhere strictly to the traditional wording of the Singer's Prayerbook, and those who embrace the more innovative Sim Shalom; but the minyan which both I and the abovementioned senior rabbi attend falls decidedly into the former camp.)

There are prayerbooks nowadays which offer alternative wordings for prayers, such as the Sim Shalom, but those feel wrong to me too. They're substituting one form of קֶבַע for another. What it seems to me they should be doing is saying, "This בְּרָכָה is on the themes of such-and-such. If you like, you may use your own wording on these themes, provided you close with the following words." But then I feel that people reading that should say "But I can't possibly put in Biblical allusions and numerologically-significant references, etc, like the original, and I don't know enough about the levels at which the text of this בְּרָכָה is operating to be able to devise a wording that is worthy of substituting for the original; so I shall have to stick with the original wording instead."

Which brings me full circle, to a stance where I am uncomfortable with (in some cases) retaining the traditional wording of a בְּרָכָה, but also uncomfortable with changing it. Any thoughts as to how to break out of this situation without going against the ideology of the denomination I belong to, or losing the authority of the traditional wording?

[ETA: See also (if you have permission) the discussion on the Facebook import of this post.]

[ET further A: It appears I misremembered what I heard in the drasha on Shabbos. The rabbi quoted (who turned out to be a Talmudic one) did not say that one who changed the wording of the prayers would not have their prayer received, but that they had not fulfilled their obligation; further that the Talmud juxtaposes this with another prominent rabbi (Rabbi Meir, if I skim-interpret Berachos 40b correctly) who does favour spontaneous prayer. Finally, a reason is also given for retaining the traditional wording where it does not make sense.]