You need to show the "Order" field before you can move players around.
There was an alert message that warned about this but it also gave an option of not showing the message again (which I am assuming was
So, to fix:
- Select the "Modify Batting/Fielding Statistics" menu item from the Games menu
- Select the "Stats.BatOrder" show field
- Select save.
The order field will now be shown and moving players around will now stick.