Casting a character column to integer on sort in postgres

One of my clients had a field that they entered session numbers into, but sometimes they had session 101a and 101b so it was a varchar field. However when sorting by it, 1001 would show up before 101 as postgres was using a character sorting algorithm.

This was the original SORT BY session_number ASC and I googled a bit and found that you can cast within postgres so now it works with SORT BY (session_number::text)::integer ASC.

It works well until it finds an entry which isn't an integer, then it complains. Will investigate further options.