Rosh Hashanah is a very serious day, a day in which we're being judged and all of the events of the upcoming year are being determined. The gravity of the day is very heavy upon us, the blowing of the shofar (ram's horn) and such. Nevertheless, when a person prays on Rosh Hashanah, they should pray with happiness and with joy. They should take upon themselves the joy of being Hashem's subjects, of crowning Hashem as King, and they should rely on the fact that Hashem is merciful and He's going to give us a good judgment.
So we wear our best suit, shave, groom, and prepare for the fact that we're going to have a good judgment. We go home and eat a festive meal, and so we enjoy ourselves and are happy, as on any Yom Tov (festival day), on the day of Rosh Hashanah.
There's a story in the Navi (Prophets) where the people started crying in the times of Ezra and Nehemiah because they heard the gravity of Hashem's judgment, and they started to cry and weep. Nehemiah told them, “Stop crying. Today's a happy day. Go home and enjoy your food.” It's not a day for weeping and crying, and therefore one should not try to arouse thoughts of remorse and crying on the day of Rosh Hashanah. That should be left for Yom Kippur.
If, however, in the course of the prayer, the gravity is so, so overwhelming that the person spontaneously bursts into tears, there's no issur (prohibition), there's no violation on the matter, but the person should try to convince themselves out of it. When praying on Rosh Hashanah, one should try to hunch over a little bit more to show humility, but when saying the Berachot (blessings) in Hashem's name, they should make sure to stand up straight. Have a great day.