I mean, trains, even diesel trains, are shockingly fuel efficient. (Also, as far as I can recall, no one has used fuel costs as the justification for cutting service.) Likewise, IIRC, the weight of a train derives primarily from its passenger load, so, insofar as fuel costs do contribute, they’ll be lower in off-peak.
But anyway, to @Java King’s point, the lack of flexibility itself depresses ridership even further. You’re always going to have some low ridership runs at the shoulder periods of any service — that’s part of what enables higher ridership during other periods.