I'm not the resident expert on this stuff, there are some VERY fast racer and track guys on this site... but I will throw my $.02 in...
Sounds like you're carrying
decent speed into and through the corners if you're knee is down (assuming your body position is correct). And you mentioned you have times that are floating up there w/the expert class on a track day. If your goal is to get better times and go faster, what is making you think getting your body off more will achieve this? Sounds like your body position isn't too bad already. The reality is, w/out pictures or video - it's hard to say or recommend something like that. Now as far as times/speeds... I was just at the track this past Monday and paid (for his track time) to have a buddy (expert racer) go there and coach me for the day in prep for my first race... He helped shave almost 6 seconds off my time

The cause... less brake, more gas (and some lines)! Sounds simple, but basically there were a couple/few key turns where some critical time could be made up... where I (like plenty others) get on the brakes too hard and cut their entry speed down too much. The other piece that shaved at least a second off, was getting on the gas sooner. I was being very cautious getting on the gas at the apex... finally I started getting on sooner and sooner until he and his wife (another expert racer) complimented the application (sooner) of throttle in that turn.
So my answer to you quite honestly is that I don't believe rearsets will make you faster... are aftermarket rearsets a good idea - yes... for plenty of reasons. But for your ability level that you described I think your lap times and speed have more to do with entry speed and getting on the gas sooner.